forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
90 lines
15 KiB
HTML
90 lines
15 KiB
HTML
<a name="ALM-16008"></a><a name="ALM-16008"></a>
|
|
|
|
<h1 class="topictitle1">ALM-16008 Non-Heap Memory Usage of the Hive Process Exceeds the Threshold</h1>
|
|
<div id="body50730563"><div class="section" id="ALM-16008__sd19cf02cf2e54d77a18850a9f9f2a92c"><h4 class="sectiontitle">Description</h4><p id="ALM-16008__en-us_topic_0070543665_p9742220">The system checks the Hive service status every 30 seconds. The alarm is generated when the non-heap memory usage of an Hive service exceeds the threshold (95% of the maximum memory).</p>
|
|
<p id="ALM-16008__en-us_topic_0070543665_p20571123">Users can choose <strong id="ALM-16008__b3860175017451"><strong id="ALM-16008__b10860125016452">O&M > Alarm > Thresholds ></strong></strong> <em id="ALM-16008__i198631750124518">Name of the desired cluster</em> ><strong id="ALM-16008__b158618508457"> <strong id="ALM-16008__b1586111508450">Hive</strong></strong> to change the threshold.</p>
|
|
<p id="ALM-16008__en-us_topic_0070543665_p33806510">The alarm is cleared when the non-heap memory usage is less than or equal to the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16008__s0ac9ce190a5a42d98e1636e4aacc04e3"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-16008__en-us_topic_0070543665_table53972770" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-16008__en-us_topic_0070543665_row7971945"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-16008__en-us_topic_0070543665_p41747840">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-16008__en-us_topic_0070543665_p26131842">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-16008__en-us_topic_0070543665_p36304473">Automatically Cleared</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-16008__en-us_topic_0070543665_row54981229"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-16008__en-us_topic_0070543665_p24294565">16008</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-16008__en-us_topic_0070543665_p21702762">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-16008__en-us_topic_0070543665_p13093283">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-16008__s91c9a72ef469400bb3bdb915c3116280"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-16008__en-us_topic_0070543665_table53923001" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-16008__en-us_topic_0070543665_row52132201"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-16008__en-us_topic_0070543665_p61958749">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-16008__en-us_topic_0070543665_p52602802">Meaning</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-16008__row1324733112275"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16008__p192431315431">Source</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16008__p692551319435">Specifies the cluster for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16008__en-us_topic_0070543665_row32968592"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16008__en-us_topic_0070543665_p53210270">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16008__en-us_topic_0070543665_p15064580">Specifies the service name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16008__en-us_topic_0070543665_row1363496"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16008__en-us_topic_0070543665_p43334342">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16008__en-us_topic_0070543665_p20420817">Specifies the role name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16008__en-us_topic_0070543665_row49569626"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16008__en-us_topic_0070543665_p55716787">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16008__en-us_topic_0070543665_p16765868">Specifies the object (host ID) for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-16008__s91cefcf8865b41bca1229984dc18f2ce"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-16008__en-us_topic_0070543665_p15858029">When the non-heap memory usage of Hive is overhigh, the performance of Hive task operation is affected. In addition, a memory overflow may occur so that the Hive service is unavailable.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16008__sf43ef8b7c0134bb1a5f677020e2ca6cc"><h4 class="sectiontitle">Possible Causes</h4><p id="ALM-16008__en-us_topic_0070543665_p9431967">The non-heap memory of the Hive instance on the node is overused or the non-heap memory is inappropriately allocated. As a result, the usage exceeds the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16008__scbdc083fc9d84e368b2dd5a9311d5452"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-16008__en-us_topic_0070543665_p25791849"><strong id="ALM-16008__b9408304144231">Check non-heap memory usage.</strong></p>
|
|
<ol id="ALM-16008__ol48868481144238"><li id="ALM-16008__li40371232144225"><span>On the FusionInsight Manager portal, click <strong id="ALM-16008__b28662750155624">O&M > Alarm > Alarms</strong> and select the alarm whose <strong id="ALM-16008__b18052580144225">Alarm ID</strong> is <strong id="ALM-16008__b28255492144225">16008</strong>. Then check the role name in <strong id="ALM-16008__b14790172183618">Location </strong>and confirm the IP adress of the instance.</span><p><ul class="subitemlist" id="ALM-16008__ul49224935144225"><li id="ALM-16008__li62941809144225">If the role for which the alarm is generated is HiveServer, go to <a href="#ALM-16008__li54453327144225">2</a>.</li><li id="ALM-16008__li65121760144225">If the role for which the alarm is generated is MetaStore, go to <a href="#ALM-16008__li31617556144225">3</a>.</li></ul>
|
|
</p></li><li id="ALM-16008__li54453327144225"><a name="ALM-16008__li54453327144225"></a><a name="li54453327144225"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16008__b6111239135016">Cluster </strong>><em id="ALM-16008__i1174271119515"> Name of the desired <em id="ALM-16008__i1557110281490">cluster</em></em> ><strong id="ALM-16008__b207406111513"> Services</strong> > <strong id="ALM-16008__b13023504143018">Hive</strong> > <strong id="ALM-16008__b50102677143018">Instance</strong> and click the HiveServer for which the alarm is generated to go to the<strong id="ALM-16008__b14303164441516"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-16008__b9966756057">Chart </strong>area and choose <strong id="ALM-16008__b996617562515">Customize </strong>> <strong id="ALM-16008__b15702441192211">CPU and Memory</strong>, and select <strong id="ALM-16008__b23132627144225">HiveServer Memory Usage Statistics</strong> and click <strong id="ALM-16008__b6867054144225">OK</strong>, check whether the used non-heap memory of the HiveServer service reaches the threshold(default value: 95%) of the maximum non-heap memory specified for HiveServer.</span><p><ul class="subitemlist" id="ALM-16008__ul20963450144225"><li id="ALM-16008__li19360527144225">If yes, go to <a href="#ALM-16008__li24754013144225">4</a>.</li><li id="ALM-16008__li24698893144225">If no, go to <a href="#ALM-16008__li3071924144225">7</a>.</li></ul>
|
|
</p></li><li id="ALM-16008__li31617556144225"><a name="ALM-16008__li31617556144225"></a><a name="li31617556144225"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16008__b759842155210">Cluster </strong>><em id="ALM-16008__i186334275213"> Name of the desired </em><em id="ALM-16008__i102301871901">cluster</em> ><strong id="ALM-16008__b1360114219527"> Services</strong> > <strong id="ALM-16008__b20216101535513">Hive</strong> > <strong id="ALM-16008__b1217715135510">Instance</strong> and click the MetaStore for which the alarm is generated to go to the<strong id="ALM-16008__b123917399189"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-16008__b426537693">Chart </strong>area and choose <strong id="ALM-16008__b42651713917">Customize </strong>> <strong id="ALM-16008__b101144984915">CPU and Memory</strong>, and select <strong id="ALM-16008__b46636353144225">MetaStore Memory Usage Statistics</strong> and click <strong id="ALM-16008__b17073995144225">OK</strong>, check whether the used non-heap memory of the MetaStore service reaches the threshold(default value: 95%) of the maximum non-heap memory specified for MetaStore.</span><p><ul class="subitemlist" id="ALM-16008__ul25882683144225"><li id="ALM-16008__li40816336144225">If yes, go to <a href="#ALM-16008__li24754013144225">4</a>.</li><li id="ALM-16008__li17788934144225">If no, go to <a href="#ALM-16008__li3071924144225">7</a>.</li></ul>
|
|
</p></li><li id="ALM-16008__li24754013144225"><a name="ALM-16008__li24754013144225"></a><a name="li24754013144225"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16008__b144872611532">Cluster</strong><strong id="ALM-16008__b1848712612530"> ></strong><em id="ALM-16008__i17153095311"> Name of the desired </em><em id="ALM-16008__i01857245012">cluster</em> ><strong id="ALM-16008__b181212045318"> Services</strong> > <strong id="ALM-16008__b40561099143018">Hive</strong> > <strong id="ALM-16008__b29505572143018">Configurations > All Configurations</strong>. Choose <strong id="ALM-16008__b14516323144225">HiveServer/MetaStore</strong> > <strong id="ALM-16008__b63538044144225">JVM</strong>. Adjust the value of <strong id="ALM-16008__b34971491144225">-XX:MaxMetaspaceSize</strong> in <strong id="ALM-16008__b46307968144225">HIVE_GC_OPTS/METASTORE_GC_OPTS</strong> as the following rules. Click <strong id="ALM-16008__b14118532144225">Save</strong>.</span><p><div class="note" id="ALM-16008__note19231155073715"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><div class="p" id="ALM-16008__p863882712519">Suggestions for GC parameter settings for the HiveServer:<ul id="ALM-16008__ul15827113432817"><li id="ALM-16008__li1364191319387">It is recommended that you set the value of <strong id="ALM-16008__b1164219130384">-XX:</strong><strong id="ALM-16008__b1264281310382">MaxMetaspaceSize</strong> to 1/8 of the value of <strong id="ALM-16008__b1264218137382">-Xmx</strong>. For example, if <strong id="ALM-16008__b19642141312384">-Xmx</strong> is set to 2 GB, <strong id="ALM-16008__b164213133387">-XX:</strong><p id="ALM-16008__p089212226386"><strong id="ALM-16008__b9892172217381">MaxMetaspaceSize </strong>is set to 256 MB. If <strong id="ALM-16008__b1289202203816">-Xmx</strong> is set to 4 GB, <strong id="ALM-16008__b1989217222389">-XX:</strong><strong id="ALM-16008__b42019612384">MaxMetaspaceSize </strong>is set to 512 MB.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="p" id="ALM-16008__p141314122620">Suggestions for GC parameter settings for the MetaServer:<ul id="ALM-16008__ul97921420153913"><li id="ALM-16008__li1279272011391">It is recommended that you set the value of <strong id="ALM-16008__b279211200399">-XX:</strong><strong id="ALM-16008__b1679292018395">MaxMetaspaceSize</strong> to 1/8 of the value of <strong id="ALM-16008__b679213204398">-Xmx</strong>. For example, if <strong id="ALM-16008__b579242017395">-Xmx</strong> is set to 2 GB, <strong id="ALM-16008__b17792172014392">-XX:</strong><p id="ALM-16008__p15792920133916"><strong id="ALM-16008__b13792620113911">MaxMetaspaceSize </strong>is set to 256 MB. If <strong id="ALM-16008__b479272033910">-Xmx</strong> is set to 4 GB, <strong id="ALM-16008__b679212023910">-XX:</strong><strong id="ALM-16008__b157927208397">MaxMetaspaceSize </strong>is set to 512 MB</p>
|
|
</li></ul>
|
|
</div>
|
|
</div></div>
|
|
</p></li><li id="ALM-16008__li1138110412316"><span>Click <strong id="ALM-16008__b193941191220">More > Restart Service </strong>to restart the service.</span></li><li id="ALM-16008__li14088790144225"><span>Check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-16008__ul7630476144225"><li id="ALM-16008__li21459527144225">If yes, no further action is required.</li><li id="ALM-16008__li60500154144225">If no, go to <a href="#ALM-16008__li3071924144225">7</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-16008__p1565421144225"><strong id="ALM-16008__b491248144253">Collect fault information.</strong></p>
|
|
<ol start="7" id="ALM-16008__ol62965554144257"><li id="ALM-16008__li3071924144225"><a name="ALM-16008__li3071924144225"></a><a name="li3071924144225"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16008__b39977366113627">O&M</strong> > <strong id="ALM-16008__b24251979113627">Log > Download</strong>.</span></li><li id="ALM-16008__li65916619144225"><span>Select <strong id="ALM-16008__b194951125851">Hive</strong> in the required cluster from the <strong id="ALM-16008__b27647320144225">Service</strong>.</span></li><li id="ALM-16008__li1145664103113"><span>Click <span><img id="ALM-16008__image1945644173117" src="en-us_image_0269417384.png"></span> in the upper right corner, and set <strong id="ALM-16008__b6456941173117">Start Date</strong> and <strong id="ALM-16008__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-16008__b13456164113319">Download</strong>.</span></li><li id="ALM-16008__li63485823144225"><span>Contact the <span id="ALM-16008__text4614151421417">O&M personnel</span> and send the collected logs.</span></li></ol>
|
|
</div>
|
|
<div class="section" id="ALM-16008__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-16008__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16008__s28fab4793385455082bb51190961c77e"><h4 class="sectiontitle">Related Information</h4><p id="ALM-16008__en-us_topic_0070543665_p48980088">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|