forked from docs/doc-exports
Yang, Tong
3b1f73dece
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
87 lines
12 KiB
HTML
87 lines
12 KiB
HTML
<a name="ALM-13002"></a><a name="ALM-13002"></a>
|
|
|
|
<h1 class="topictitle1">ALM-13002 ZooKeeper Direct Memory Usage Exceeds the Threshold</h1>
|
|
<div id="body52592944"><div class="section" id="ALM-13002__s0b47d477dd064787a3d676fd6e4726c0"><h4 class="sectiontitle">Description</h4><p id="ALM-13002__en-us_topic_0070543634_p53727196">The system checks the direct memory usage of the ZooKeeper service every 30 seconds. The alarm is generated when the direct memory usage of a ZooKeeper instance exceeds the threshold (80% of the maximum memory).</p>
|
|
<p id="ALM-13002__p11080609104723">When the <strong id="ALM-13002__b48421890111935">Trigger Count</strong> is 1, this alarm is cleared when the ZooKeeper Direct memory usage is less than the threshold. When the <strong id="ALM-13002__b2523856173714">Trigger Count</strong> is greater than 1, this alarm is cleared when the ZooKeeper Direct memory usage is less than 80% of the threshold.</p>
|
|
</div>
|
|
<div class="section" id="ALM-13002__s8b66b5b6de0549c6a79e8e70e1b502d8"><h4 class="sectiontitle">Attribute</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-13002__en-us_topic_0070543634_table42658515" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-13002__en-us_topic_0070543634_row35944862"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-13002__en-us_topic_0070543634_p25852739">Alarm ID</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-13002__en-us_topic_0070543634_p13697137">Alarm Severity</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-13002__en-us_topic_0070543634_p35726287">Auto Clear</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-13002__en-us_topic_0070543634_row8148101"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-13002__en-us_topic_0070543634_p56016444">13002</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-13002__en-us_topic_0070543634_p41038107">Major</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-13002__en-us_topic_0070543634_p35752395">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-13002__sca1390c43b3a455eb45e4d273c0ced55"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-13002__en-us_topic_0070543634_table10262900" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-13002__en-us_topic_0070543634_row20946753"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-13002__en-us_topic_0070543634_p18965458">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-13002__en-us_topic_0070543634_p59807168">Meaning</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="ALM-13002__row1748205863711"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-13002__p192431315431">Source</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-13002__p692551319435">Specifies the cluster for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-13002__en-us_topic_0070543634_row12542450"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-13002__en-us_topic_0070543634_p9305553">ServiceName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-13002__en-us_topic_0070543634_p15552324">Specifies the service name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-13002__en-us_topic_0070543634_row5753193"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-13002__en-us_topic_0070543634_p63355478">RoleName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-13002__en-us_topic_0070543634_p31520075">Specifies the role name for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-13002__en-us_topic_0070543634_row15245225"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-13002__en-us_topic_0070543634_p26903678">HostName</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-13002__en-us_topic_0070543634_p31714287">Specifies the object (host ID) for which the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="ALM-13002__en-us_topic_0070543634_row16993134"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-13002__en-us_topic_0070543634_p34266646">Trigger Condition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-13002__en-us_topic_0070543634_p24134926">Specifies the threshold triggering the alarm. If the current indicator value exceeds this threshold, the alarm is generated.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="ALM-13002__s048d5e2bb874491a923413408ad28fdc"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-13002__en-us_topic_0070543634_p8771966">If the available direct memory of the ZooKeeper service is insufficient, a memory overflow occurs and the service breaks down.</p>
|
|
</div>
|
|
<div class="section" id="ALM-13002__sa6c3e8fcef2741a693f14118f4cc17c1"><h4 class="sectiontitle">Possible Causes</h4><p id="ALM-13002__en-us_topic_0070543634_p39440655">The direct memory of the ZooKeeper instance is overused or the direct memory is inappropriately allocated.</p>
|
|
</div>
|
|
<div class="section" id="ALM-13002__s0aabef9072fe4f32bda5f7ba27ae7dbc"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-13002__en-us_topic_0070543634_p40576488"><strong id="ALM-13002__b28090164161211">Check the direct memory usage.</strong></p>
|
|
<ol id="ALM-13002__ol64537219161223"><li id="ALM-13002__li38263492161213"><span>On the FusionInsight Manager portal, choose <strong id="ALM-13002__b11820144812162">O&M</strong> > <strong id="ALM-13002__b10820184871617">Alarm </strong>> <strong id="ALM-13002__b18729102141710">Alarms</strong>. On the displayed interface, click the drop-down button of <strong id="ALM-13002__b616420396303">ZooKeeper Direct Memory Usage Exceeds the Threshold</strong>. Check the IP address of the instance that reports the alarm.</span></li><li id="ALM-13002__li43973246161213"><span>On the FusionInsight Manager portal, choose <strong id="ALM-13002__b6212173752610">Cluster > </strong><em id="ALM-13002__i421463722619">Name of the desired cluster</em><strong id="ALM-13002__b1421320375260"> > Services</strong> > <strong id="ALM-13002__b12335119161213">ZooKeeper</strong> > <strong id="ALM-13002__b43907211161213">Instance</strong> > <strong id="ALM-13002__b59620587161213">quorumpeer(the IP address checked)</strong>. Click the drop-down menu in the upper right corner of <strong id="ALM-13002__b3273144141318">Chart</strong>, choose <strong id="ALM-13002__b7246166191312">Customize</strong> > <strong id="ALM-13002__b24815175518">CPU and Memory</strong>, and select<strong id="ALM-13002__b540245213164"> ZooKeeper Heap And Direct Buffer Resource Percentage</strong>, click <strong id="ALM-13002__b11181773617">OK</strong>.</span></li><li id="ALM-13002__li734112161213"><span>Check whether the used direct buffer memory of ZooKeeper reaches 80% of the maximum direct buffer memory specified for ZooKeeper.</span><p><ul class="subitemlist" id="ALM-13002__ul81568161213"><li id="ALM-13002__li5063194161213">If yes, go to <a href="#ALM-13002__li57922773161213">4</a>.</li><li id="ALM-13002__li7465603161213">If no, go to <a href="#ALM-13002__li43327670161213">8</a>.</li></ul>
|
|
</p></li><li id="ALM-13002__li57922773161213"><a name="ALM-13002__li57922773161213"></a><a name="li57922773161213"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-13002__b20558172772612">Cluster > </strong><em id="ALM-13002__i195605279266">Name of the desired cluster</em><strong id="ALM-13002__b7559227192615"> > Service</strong><strong id="ALM-13002__b121070207117">s</strong> > <strong id="ALM-13002__b59463130161213">ZooKeeper</strong> > <strong id="ALM-13002__b65406125161213">Configurations</strong> > <strong id="ALM-13002__b51784216161213">All</strong> <strong id="ALM-13002__b1186732083116">Configuration</strong><strong id="ALM-13002__b178951428143111">s</strong> > <strong id="ALM-13002__b63404764161213">quorumpeer</strong> > <strong id="ALM-13002__b2471049112813">System</strong> to check whether "-XX:MaxDirectMemorySize" exists in the <strong id="ALM-13002__b10657132211322">GC_OPTS</strong> parameter.</span><p><ul id="ALM-13002__ul139842016192914"><li id="ALM-13002__li3995171717297">If yes, in the <strong id="ALM-13002__b35332405299">GC_OPTS</strong> parameter, delete "-XX:MaxDirectMemorySize" and go to <a href="#ALM-13002__li51542910161213">5</a>.</li><li id="ALM-13002__li2995617122918">If no, go to <a href="#ALM-13002__li16393123713315">6</a>.</li></ul>
|
|
</p></li><li id="ALM-13002__li51542910161213"><a name="ALM-13002__li51542910161213"></a><a name="li51542910161213"></a><span>Save the configuration and restart the ZooKeeper service.</span></li><li id="ALM-13002__li16393123713315"><a name="ALM-13002__li16393123713315"></a><a name="li16393123713315"></a><span>Check whether the <strong id="ALM-13002__b10209103115301">ALM-13004 ZooKeeper Heap Memory Usage Exceeds the Threshold</strong> exists.</span><p><ul id="ALM-13002__ul11331814326"><li id="ALM-13002__li933171163212">If yes, handle the alarm by referring to <strong id="ALM-13002__b153433993014">ALM-13004 ZooKeeper Heap Memory Usage Exceeds the Threshold</strong>.</li><li id="ALM-13002__li13338117321">If no, go to <a href="#ALM-13002__li56397739161213">7</a>.</li></ul>
|
|
</p></li><li id="ALM-13002__li56397739161213"><a name="ALM-13002__li56397739161213"></a><a name="li56397739161213"></a><span>Check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-13002__ul11466826161213"><li id="ALM-13002__li61233006161213">If yes, no further action is required.</li><li id="ALM-13002__li60926415161213">If no, go to <a href="#ALM-13002__li43327670161213">8</a>.</li></ul>
|
|
</p></li></ol>
|
|
<p class="tableheading" id="ALM-13002__p36092577161213"><strong id="ALM-13002__b28608517161228">Collect fault information.</strong></p>
|
|
<ol start="8" id="ALM-13002__ol48231698161231"><li id="ALM-13002__li43327670161213"><a name="ALM-13002__li43327670161213"></a><a name="li43327670161213"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-13002__b39977366113627">O&M</strong> > <strong id="ALM-13002__b24251979113627">Log > Download</strong>.</span></li><li id="ALM-13002__li66808045161213"><span>Select <strong id="ALM-13002__b54404715161213">ZooKeeper</strong> in the required cluster from the <strong id="ALM-13002__b19880394161213">Service</strong>.</span></li><li id="ALM-13002__li1145664103113"><span>Click <span><img id="ALM-13002__image1945644173117" src="en-us_image_0269383943.png"></span> in the upper right corner, and set <strong id="ALM-13002__b6456941173117">Start Date</strong> and <strong id="ALM-13002__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-13002__b13456164113319">Download</strong>.</span></li><li id="ALM-13002__li495644512588"><span>Contact the <span id="ALM-13002__text4614151421417">O&M personnel</span> and send the collected logs.</span></li></ol>
|
|
</div>
|
|
<div class="section" id="ALM-13002__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-13002__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
|
|
</div>
|
|
<div class="section" id="ALM-13002__s298a98437e924ca9a98073626bfd3fce"><h4 class="sectiontitle">Related Information</h4><p id="ALM-13002__en-us_topic_0070543634_p48419974">None</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
|
|
</div>
|
|
</div>
|
|
|