vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/umn/ALM-27004.html
Yang, Tong 3b1f73dece MRS UMN 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2022-12-13 12:03:34 +00:00

108 lines
15 KiB
HTML
Raw Permalink Blame History

<a name="ALM-27004"></a><a name="ALM-27004"></a>
<h1 class="topictitle1">ALM-27004 Data Inconsistency Between Active and Standby DBServices</h1>
<div id="body15367042"><div class="section" id="ALM-27004__s3678be2b7aee4084ab56f98baddb1f16"><h4 class="sectiontitle">Description</h4><p id="ALM-27004__en-us_topic_0070543556_p41093009">The system checks the data synchronization status between the active and standby DBService every 10 seconds. This alarm is generated when the synchronization status cannot be queried for six consecutive times or when the synchronization status is abnormal.</p>
<p id="ALM-27004__en-us_topic_0070543556_p34292765">This alarm is cleared when the synchronization status becomes normal.</p>
</div>
<div class="section" id="ALM-27004__s6a075db9ed1f4a0e8ef947a07d96570c"><h4 class="sectiontitle">Attribute</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-27004__en-us_topic_0070543556_table26250571" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-27004__en-us_topic_0070543556_row39337857"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.1"><p id="ALM-27004__en-us_topic_0070543556_p32249809">Alarm ID</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.2"><p id="ALM-27004__en-us_topic_0070543556_p62097709">Alarm Severity</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.2.1.4.1.3"><p id="ALM-27004__en-us_topic_0070543556_p63858558">Automatically Cleared</p>
</th>
</tr>
</thead>
<tbody><tr id="ALM-27004__en-us_topic_0070543556_row5160748"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p15367454">27004</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p36804285">Critical</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.2.1.4.1.3 "><p id="ALM-27004__en-us_topic_0070543556_p28357104">Yes</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="ALM-27004__se1aaee53feaa49d7b36201c6b08a9d01"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="ALM-27004__en-us_topic_0070543556_table15224096" frame="border" border="1" rules="all"><thead align="left"><tr id="ALM-27004__en-us_topic_0070543556_row44426301"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.1"><p id="ALM-27004__en-us_topic_0070543556_p41760635">Name</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.1.3.1.2"><p id="ALM-27004__en-us_topic_0070543556_p27168314">Meaning</p>
</th>
</tr>
</thead>
<tbody><tr id="ALM-27004__row585010218215"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__p192431315431">Source</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__p692551319435">Specifies the cluster for which the alarm is generated.</p>
</td>
</tr>
<tr id="ALM-27004__en-us_topic_0070543556_row53149858"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p10171226">ServiceName</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p18562954">Specifies the service for which the alarm is generated.</p>
</td>
</tr>
<tr id="ALM-27004__en-us_topic_0070543556_row32848863"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p43512247">RoleName</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p34831132">Specifies the role for which the alarm is generated.</p>
</td>
</tr>
<tr id="ALM-27004__en-us_topic_0070543556_row45044734"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p24744865">HostName</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p58177028">Specifies the host for which the alarm is generated.</p>
</td>
</tr>
<tr id="ALM-27004__en-us_topic_0070543556_row53831205"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p65360331">Local DBService HA Name</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p59695495">Specifies the HA name of the local DBService.</p>
</td>
</tr>
<tr id="ALM-27004__en-us_topic_0070543556_row388543"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p31472052">Peer DBService HA Name</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p66208306">Specifies the HA name of the peer DBService.</p>
</td>
</tr>
<tr id="ALM-27004__en-us_topic_0070543556_row59003843"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.1 "><p id="ALM-27004__en-us_topic_0070543556_p14581984">SYNC_PERCENT</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.1.3.1.2 "><p id="ALM-27004__en-us_topic_0070543556_p40290074">Specifies the synchronization percentage.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="ALM-27004__s488af6a7b5bb4de7aa442990e37ab0a4"><h4 class="sectiontitle">Impact on the System</h4><p id="ALM-27004__en-us_topic_0070543556_p42270552">When data is not synchronized between the active and standby DBServices, data may be lost or abnormal if the active instance becomes abnormal.</p>
</div>
<div class="section" id="ALM-27004__secf54b6474d5433894639ec235c7ba5a"><h4 class="sectiontitle">Possible Causes</h4><ul id="ALM-27004__en-us_topic_0070543556_ul1362691"><li id="ALM-27004__en-us_topic_0070543556_li12264220">The network between the active and standby nodes is unstable.</li><li id="ALM-27004__en-us_topic_0070543556_li43269122">The standby DBService is abnormal.</li><li id="ALM-27004__en-us_topic_0070543556_li53877781">The standby node disk space is full.</li><li id="ALM-27004__li859235917312">The CPU usage of the GaussDB process on the active DBService node is high. You need to locate the failure cause based on logs.</li></ul>
</div>
<div class="section" id="ALM-27004__scce81324029343cea2b760105d74f10c"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="ALM-27004__en-us_topic_0070543556_p2024110"><strong id="ALM-27004__b55334415195038">Check whether the network between the active and standby nodes is normal.</strong></p>
<ol id="ALM-27004__ol19209640195053"><li id="ALM-27004__li10550292195034"><span>On FusionInsight Manager, choose <strong id="ALM-27004__b1172254195034">Cluster &gt; Services &gt; DBService &gt; Instance</strong>, check the service IP address of the standby DBServer instance.</span></li><li id="ALM-27004__li49267300195034"><span>Log in to the active DBService node as user <strong id="ALM-27004__b27843765195034">root</strong>. <span id="ALM-27004__text151633211120"></span></span></li><li id="ALM-27004__li15983355195034"><span>Run the <strong id="ALM-27004__b40752521195034">ping </strong><em id="ALM-27004__i31228373195034">Standby DBService heartbeat IP address</em> command to check whether the standby DBService node is reachable.</span><p><ul class="subitemlist" id="ALM-27004__ul53971711195034"><li id="ALM-27004__li46470286195034">If yes, go to <a href="#ALM-27004__li53069850195034">6</a>.</li><li id="ALM-27004__li5996856195034">If no, go to <a href="#ALM-27004__li40047077195034">4</a>.</li></ul>
</p></li><li id="ALM-27004__li40047077195034"><a name="ALM-27004__li40047077195034"></a><a name="li40047077195034"></a><span>Contact the network administrator to check whether the network is faulty.</span><p><ul class="subitemlist" id="ALM-27004__ul49188917195034"><li id="ALM-27004__li19583358195034">If yes, go to <a href="#ALM-27004__li22196300195034">5</a>.</li><li id="ALM-27004__li42748137195034">If no, go to <a href="#ALM-27004__li53069850195034">6</a>.</li></ul>
</p></li><li id="ALM-27004__li22196300195034"><a name="ALM-27004__li22196300195034"></a><a name="li22196300195034"></a><span>Rectify the network fault and check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-27004__ul17672622195034"><li id="ALM-27004__li24879377195034">If yes, no further action is required.</li><li id="ALM-27004__li1963624195034">If no, go to <a href="#ALM-27004__li53069850195034">6</a>.</li></ul>
</p></li></ol>
<p class="tableheading" id="ALM-27004__p47944044195055"><strong id="ALM-27004__b28843215195055">Check whether the standby DBService is normal.</strong></p>
<ol start="6" id="ALM-27004__ol46282316195113"><li id="ALM-27004__li53069850195034"><a name="ALM-27004__li53069850195034"></a><a name="li53069850195034"></a><span>Log in to the standby DBService node as user <strong id="ALM-27004__b65548973195034">root</strong>.</span></li><li id="ALM-27004__li33215044195034"><span>Run the <strong id="ALM-27004__b7866602195034">su - omm</strong> command to switch to user <strong id="ALM-27004__b3690560195034">omm</strong>.</span></li><li id="ALM-27004__li53185819195034"><span>Go to the <strong id="ALM-27004__b30499943195034">${DBSERVER_HOME}/sbin</strong> directory and run the <strong id="ALM-27004__b6064034195034">./status-dbserver.sh </strong>command to check whether the GaussDB resource status of the standby DBService is normal. In the command output, check whether the following information is displayed in the row where <strong id="ALM-27004__b54576310195034">ResName</strong> is <strong id="ALM-27004__b21424749195034">gaussDB</strong>:</span><p><p class="litext" id="ALM-27004__p58605014195034">For example:</p>
<pre class="screen" id="ALM-27004__screen57683084195034">10_10_10_231 gaussDB Standby_normal Normal Active_standby</pre>
<ul class="subitemlist" id="ALM-27004__ul18055209195034"><li id="ALM-27004__li41818216195034">If yes, go to <a href="#ALM-27004__li13084069195034">9</a>.</li><li id="ALM-27004__li31832296195034">If no, go to <a href="#ALM-27004__li45928829195034">16</a>.</li></ul>
</p></li></ol>
<p class="tableheading" id="ALM-27004__p28279156195034"><strong id="ALM-27004__b4164191619518">Check whether the standby node disk space is full.</strong></p>
<ol start="9" id="ALM-27004__ol45091172195134"><li id="ALM-27004__li13084069195034"><a name="ALM-27004__li13084069195034"></a><a name="li13084069195034"></a><span>Log in to the standby DBService node as user <strong id="ALM-27004__b8910325195034">root</strong>.</span></li><li id="ALM-27004__li8828149195034"><span>Run the <strong id="ALM-27004__b50647763195034">su - omm</strong> command to switch to user <strong id="ALM-27004__b53176688195034">omm</strong>.</span></li><li id="ALM-27004__li33564107195034"><span>Go to the <strong id="ALM-27004__b12344484195034">${DBSERVER_HOME}</strong> directory, and run the following commands to obtain the DBService data directory:</span><p><p class="litext" id="ALM-27004__p60379111195034"><strong id="ALM-27004__b43991492195034">cd ${DBSERVER_HOME}</strong></p>
<p class="litext" id="ALM-27004__p58869860195034"><strong id="ALM-27004__b6541095195034">source .dbservice_profile</strong></p>
<p class="litext" id="ALM-27004__p3729345195034"><strong id="ALM-27004__b60066695195034">echo ${DBSERVICE_DATA_DIR}</strong></p>
</p></li><li id="ALM-27004__li34338140195034"><span>Run the <strong id="ALM-27004__b33641510195034">df -h</strong> command to view the system disk partition usage information.</span></li><li id="ALM-27004__li50541929195034"><span>Check whether the DBService data directory space is full.</span><p><ul class="subitemlist" id="ALM-27004__ul5615769195034"><li id="ALM-27004__li29925921195034">If yes, go to <a href="#ALM-27004__li255547195034">14</a>.</li><li id="ALM-27004__li8080514195034">If no, go to <a href="#ALM-27004__li45928829195034">16</a>.</li></ul>
</p></li><li id="ALM-27004__li255547195034"><a name="ALM-27004__li255547195034"></a><a name="li255547195034"></a><span>Expand the disk capacity.</span></li><li id="ALM-27004__li47287758195034"><span>After the disk capacity is expanded, wait 2 minutes and check whether the alarm is cleared.</span><p><ul class="subitemlist" id="ALM-27004__ul66035654195034"><li id="ALM-27004__li2299929195034">If yes, no further action is required.</li><li id="ALM-27004__li52076537195034">If no, go to <a href="#ALM-27004__li45928829195034">16</a>.</li></ul>
</p></li></ol>
<p class="tableheading" id="ALM-27004__p57449978195034"><strong id="ALM-27004__b43968539195126">Collect fault information.</strong></p>
<ol start="16" id="ALM-27004__ol28010640195139"><li id="ALM-27004__li45928829195034"><a name="ALM-27004__li45928829195034"></a><a name="li45928829195034"></a><span>On the FusionInsight Manager portal, choose <strong id="ALM-27004__b186765355233">O&amp;M</strong> &gt; <strong id="ALM-27004__b5103203195034">Log &gt; Download</strong>.</span></li><li id="ALM-27004__li1380597414"><span>In the<strong id="ALM-27004__b1762114361643"> Service </strong>area, select <strong id="ALM-27004__b1561415401145">DBService</strong> of the target cluster and <strong id="ALM-27004__b985935110415">OS</strong>, <strong id="ALM-27004__b17631258843">OS Statistics</strong>, and <strong id="ALM-27004__b199391821454">OS Performance</strong> under<strong id="ALM-27004__b12745361514"> OMS</strong>, and click<strong id="ALM-27004__b145371019514"> OK</strong>.</span></li><li id="ALM-27004__li1145664103113"><span>Click <span><img id="ALM-27004__image1945644173117" src="en-us_image_0269417468.png"></span> in the upper right corner, and set <strong id="ALM-27004__b6456941173117">Start Date</strong> and <strong id="ALM-27004__b11456154113318">End Date</strong> for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click <strong id="ALM-27004__b13456164113319">Download</strong>.</span></li><li id="ALM-27004__li34760622195034"><span>Contact the <span id="ALM-27004__text4614151421417">O&amp;M personnel</span> and send the collected logs.</span></li></ol>
</div>
<div class="section" id="ALM-27004__section1529716184534"><h4 class="sectiontitle">Alarm Clearing</h4><p id="ALM-27004__p4677152685316">After the fault is rectified, the system automatically clears this alarm.</p>
</div>
<div class="section" id="ALM-27004__s7bc06ad6317b4705a3aeefc423a2adfc"><h4 class="sectiontitle">Related Information</h4><p id="ALM-27004__en-us_topic_0070543556_p37231957">None</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1298.html">Alarm Reference (Applicable to MRS 3.x)</a></div>
</div>
</div>
View Git Blame Copy Permalink