forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
89 lines
16 KiB
HTML
89 lines
16 KiB
HTML
<a name="ALM-16005"></a><a name="ALM-16005"></a>
|
|
|
|
<h1 class="topictitle1">ALM-16005 The Heap Memory Usage of the Hive Process Exceeds the Threshold</h1>
|
|
<div id="body50043141"><div class="section" id="ALM-16005__sb7895bc66bf443a1a4a83751136113df"><h4 class="sectiontitle">Description</h4><p id="ALM-16005__en-us_topic_0070543662_p13017478">The system checks the Hive service status every 30 seconds. The alarm is generated when the heap memory usage of an Hive service exceeds the threshold (95% of the maximum memory).</p>
|
|
<p id="ALM-16005__en-us_topic_0070543662_p50048441">Users can choose <strong id="ALM-16005__b199761321154512"><strong id="ALM-16005__b0976182114455">O&M > Alarm > Thresholds ></strong></strong> <em id="ALM-16005__i119791021154517">Name of the desired cluster</em> ><strong id="ALM-16005__b159771521194518"> <strong id="ALM-16005__b1697792184519">Hive</strong></strong> to change the threshold.</p>
|
|
<p id="ALM-16005__en-us_topic_0070543662_p37388429">The alarm is cleared when the heap memory usage is less than or equal to the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16005__sd712e9f74f264f04813993bcd973c244"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-16005__en-us_topic_0070543662_table8563940" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-16005__en-us_topic_0070543662_row39916433"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-16005__en-us_topic_0070543662_p12005641">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-16005__en-us_topic_0070543662_p32932878">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-16005__en-us_topic_0070543662_p50317447">Automatically Cleared</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-16005__en-us_topic_0070543662_row49181405"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-16005__en-us_topic_0070543662_p24270839">16005</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-16005__en-us_topic_0070543662_p19780925">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-16005__en-us_topic_0070543662_p58751129">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-16005__s7f3cbf7785b84bb98ce93b9c28d8a160"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-16005__en-us_topic_0070543662_table61220976" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-16005__en-us_topic_0070543662_row17170274"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-16005__en-us_topic_0070543662_p48614920">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-16005__en-us_topic_0070543662_p45494447">Meaning</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-16005__row06274459278"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16005__p192431315431">Source</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16005__p692551319435">Specifies the cluster for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16005__en-us_topic_0070543662_row61171582"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16005__en-us_topic_0070543662_p55951134">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16005__en-us_topic_0070543662_p35748023">Specifies the service name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16005__en-us_topic_0070543662_row53296752"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16005__en-us_topic_0070543662_p22069616">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16005__en-us_topic_0070543662_p42808444">Specifies the role name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-16005__en-us_topic_0070543662_row49731683"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-16005__en-us_topic_0070543662_p1734509">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-16005__en-us_topic_0070543662_p6277560">Specifies the object (host ID) for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-16005__s7a9ce79058de48528bfe98fade5981f8"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-16005__en-us_topic_0070543662_p38720364">When the heap memory usage of Hive is overhigh, the performance of Hive task operation is affected. In addition, a memory overflow may occur so that the Hive service is unavailable.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16005__s080a16c4bdf048edbf1a36c9a6912069"><h4 class="sectiontitle">Possible Causes</h4><p id="ALM-16005__en-us_topic_0070543662_p49341743">The heap memory of the Hive instance on the node is overused or the heap memory is inappropriately allocated. As a result, the usage exceeds the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16005__s7cd387c209044647b9e56ae743525fa5"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-16005__en-us_topic_0070543662_p37258284"><strong id="ALM-16005__b2314202143026">Check heap memory usage.</strong></p>
|
|
<ol id="ALM-16005__ol63242796143034"><li id="ALM-16005__li9274333143018"><span>On the FusionInsight Manager portal, click <strong id="ALM-16005__b28662750155624">O&M > Alarm > Alarms</strong> and select the alarm whose <strong id="ALM-16005__b55594573143018">Alarm ID</strong> is <strong id="ALM-16005__b30589116143018">16005</strong>. Then check the role name in <strong id="ALM-16005__b14790172183618">Location </strong>and confirm the IP adress of the instance.</span><p><ul class="subitemlist" id="ALM-16005__ul60682805143018"><li id="ALM-16005__li19322956143018">If the role for which the alarm is generated is HiveServer, go to <a href="#ALM-16005__li2900058143018">2</a>.</li><li id="ALM-16005__li21655614143018">If the role for which the alarm is generated is MetaStore, go to <a href="#ALM-16005__li46068501143018">3</a>.</li></ul>
|
|
</p></li><li id="ALM-16005__li2900058143018"><a name="ALM-16005__li2900058143018"></a><a name="li2900058143018"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16005__b1838728195111">Cluster </strong>><strong id="ALM-16005__b138381228115118"> </strong><em id="ALM-16005__i557610274502">Name of the desired cluster</em><strong id="ALM-16005__b155741227145016"> > Services</strong> > <strong id="ALM-16005__b13023504143018">Hive</strong> > <strong id="ALM-16005__b50102677143018">Instance</strong> and click the HiveServer for which the alarm is generated to go to the<strong id="ALM-16005__b14303164441516"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-16005__b1823435435813">Chart </strong>area and choose <strong id="ALM-16005__b184931609596">Customize </strong>> <strong id="ALM-16005__b15702441192211">CPU and Memory</strong>, and select <strong id="ALM-16005__b17629780143018">HiveServer Memory Usage Statistics</strong> and click <strong id="ALM-16005__b24450295143018">OK</strong>, check whether the used heap memory of the HiveServer service reaches the threshold(default value: 95%) of the maximum heap memory specified for HiveServer.</span><p><ul class="subitemlist" id="ALM-16005__ul52518011143018"><li id="ALM-16005__li34316893143018">If yes, go to <a href="#ALM-16005__li39802450143018">4</a>.</li><li id="ALM-16005__li28204955143018">If no, go to <a href="#ALM-16005__li7710755143018">7</a>.</li></ul>
|
|
</p></li><li id="ALM-16005__li46068501143018"><a name="ALM-16005__li46068501143018"></a><a name="li46068501143018"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16005__b71933016441">Cluster </strong>> <em id="ALM-16005__i31966004417">Name of the desired cluster</em> ><strong id="ALM-16005__b5195703449"> Services</strong> > <strong id="ALM-16005__b1467784285620">Hive</strong> > <strong id="ALM-16005__b16795426564">Instance</strong> and click the MetaStore for which the alarm is generated to go to the<strong id="ALM-16005__b128611815151618"> Dashboard </strong>page. Click the drop-down menu in the <strong id="ALM-16005__b664415911014">Chart </strong>area and choose <strong id="ALM-16005__b891911511112">Customize </strong>> <strong id="ALM-16005__b14230173114814">CPU and Memory</strong>, and select <strong id="ALM-16005__b55136170143018">MetaStore Memory Usage Statistics</strong> and click <strong id="ALM-16005__b26463487143018">OK</strong>, check whether the used heap memory of the MetaStore service reaches the threshold(default value: 95%) of the maximum heap memory specified for MetaStore.</span><p><ul class="subitemlist" id="ALM-16005__ul12575262143018"><li id="ALM-16005__li63167703143018">If yes, go to <a href="#ALM-16005__li39802450143018">4</a>.</li><li id="ALM-16005__li16310332143018">If no, go to <a href="#ALM-16005__li7710755143018">7</a>.</li></ul>
|
|
</p></li><li id="ALM-16005__li39802450143018"><a name="ALM-16005__li39802450143018"></a><a name="li39802450143018"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16005__b1830093614444">Cluster</strong> > <em id="ALM-16005__i103041036154412">Name of the desired cluster</em> > <strong id="ALM-16005__b83021536194413">Services</strong> > <strong id="ALM-16005__b40561099143018">Hive</strong> > <strong id="ALM-16005__b29505572143018">Configurations > All Configurations</strong>. Choose <strong id="ALM-16005__b34725611143018">HiveServer/MetaStore</strong> > <strong id="ALM-16005__b44095051143018">JVM</strong>. Adjust the value of <strong id="ALM-16005__b61311143143018">-Xmx</strong> in <strong id="ALM-16005__b14929375143018">HIVE_GC_OPTS/METASTORE_GC_OPTS</strong> as the following rules. Click <strong id="ALM-16005__b146654143018">Save</strong>.</span><p><div class="note" id="ALM-16005__note638551412512"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><div class="p" id="ALM-16005__p863882712519">Suggestions for GC parameter settings for the HiveServer:<ul id="ALM-16005__ul15827113432817"><li id="ALM-16005__li188271534122816">When the heap memory used by the HiveServer process reaches the threshold (default value: 95%) of the maximum heap memory set by the HiveServer process, change the value of <strong id="ALM-16005__b1682193112812">-Xmx</strong> to twice the default value. For example, if <strong id="ALM-16005__b126821431192818">-Xmx</strong> is set to 2GB by default, change the value of <strong id="ALM-16005__b6682193152813">-Xmx</strong> to 4GB. You are advised to change the value of <strong id="ALM-16005__b1968218317289">-Xms</strong> to set the ratio of <strong id="ALM-16005__b1168233118286">-Xms</strong> and <strong id="ALM-16005__b66821031142810">-Xmx</strong> to 1:2 to avoid performance problems when JVM dynamically. On the FusionInsight Manager home page, choose <strong id="ALM-16005__b218061112461">O&M</strong>> <strong id="ALM-16005__b318071144616">Alarm</strong>> <strong id="ALM-16005__b1418071144614">Thresholds </strong>> <em id="ALM-16005__i6180171144619">Name of the desired cluster</em> <strong id="ALM-16005__b21801611124617">> Hive </strong>> <strong id="ALM-16005__b4947174718469">CPU and Memory </strong>> <strong id="ALM-16005__b151716111465">HiveServer Heap Memory Usage Statistics (HiveServer)</strong> to view<strong id="ALM-16005__b952271223218"> Threshold</strong>.</li></ul>
|
|
</div>
|
|
<div class="p" id="ALM-16005__p141314122620">Suggestions for GC parameter settings for the MetaServer:<ul id="ALM-16005__ul181611956102812"><li id="ALM-16005__li177461313114818">When the heap memory used by the MetaStore process reaches the threshold (default value: 95%) of the maximum heap memory set by the MetaStore process, change the value of <strong id="ALM-16005__b12763121319489">-Xmx</strong> to twice the default value. For example, if <strong id="ALM-16005__b476312138488">-Xmx</strong> is set to 2GB by default, change the value of <strong id="ALM-16005__b676391314816">-Xmx</strong> to 4GB. On the FusionInsight Manager home page, choose <strong id="ALM-16005__b17636132481">O&M</strong>> <strong id="ALM-16005__b1976361314819">Alarm</strong>> <strong id="ALM-16005__b7763141318482">Thresholds </strong>> <em id="ALM-16005__i176311334810">Name of the desired cluster</em> <strong id="ALM-16005__b15763161313487">> Hive </strong>> <strong id="ALM-16005__b11763131311486">CPU and Memory </strong>> <strong id="ALM-16005__b87457138489">MetaStore Heap Memory Usage Statistics (MetaStore)</strong> to view<strong id="ALM-16005__b167451849114711"> Threshold</strong>.</li></ul>
|
|
<ul id="ALM-16005__ul13161155662810"><li id="ALM-16005__li19161165672820">You are advised to change the value of <strong id="ALM-16005__b1716105652813">-Xms</strong> to set the ratio of <strong id="ALM-16005__b816117562284">-Xms</strong> and <strong id="ALM-16005__b1816185619281">-Xmx</strong> to 1:2 to avoid performance problems when JVM dynamically.</li></ul>
|
|
</div>
|
|
</div></div>
|
|
</p></li><li id="ALM-16005__li86575575118"><span>Click <strong id="ALM-16005__b193941191220">More > Restart Service </strong>to restart the service.</span></li><li id="ALM-16005__li7098893143018"><span>Check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-16005__ul23285766143018"><li id="ALM-16005__li22677731143018">If yes, no further action is required.</li><li id="ALM-16005__li24956928143018">If no, go to <a href="#ALM-16005__li7710755143018">7</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-16005__p8245306143018"><strong id="ALM-16005__b20117236143050">Collect fault information.</strong></p>
|
|
<ol start="7" id="ALM-16005__ol20140076143053"><li id="ALM-16005__li7710755143018"><a name="ALM-16005__li7710755143018"></a><a name="li7710755143018"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-16005__b39977366113627">O&M</strong> > <strong id="ALM-16005__b24251979113627">Log > Download</strong>.</span></li><li id="ALM-16005__li45849806143018"><span>Select <strong id="ALM-16005__b187584411594">Hive</strong> in the required cluster from the <strong id="ALM-16005__b2287932143018">Service</strong>.</span></li><li id="ALM-16005__li1145664103113"><span>Click <span><img id="ALM-16005__image1945644173117" src="en-us_image_0269417381.png"></span> in the upper right corner, and set <strong id="ALM-16005__b6456941173117">Start Date</strong> and <strong id="ALM-16005__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-16005__b13456164113319">Download</strong>.</span></li><li id="ALM-16005__li12295179143018"><span>Contact the <span id="ALM-16005__text4614151421417">O&M personnel</span> and send the collected logs.</span></li></ol>
|
|
</div>
|
|
<div class="section" id="ALM-16005__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-16005__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
|
|
</div>
|
|
<div class="section" id="ALM-16005__s5175a2c34baa4b89adf7ed8c49c3824a"><h4 class="sectiontitle">Related Information</h4><p id="ALM-16005__en-us_topic_0070543662_p4141974">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|