forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
18 lines
2.2 KiB
HTML
18 lines
2.2 KiB
HTML
<a name="mrs_01_1643"></a><a name="mrs_01_1643"></a>
|
|
|
|
<h1 class="topictitle1">Why Does the HBase BulkLoad Task (One Table Has 26 TB Data) Consisting of 210,000 Map Tasks and 10,000 Reduce Tasks Fail?</h1>
|
|
<div id="body1596003894113"><div class="section" id="mrs_01_1643__sdaadca59fe5641c28123e8a7fb075c20"><h4 class="sectiontitle">Question</h4><p id="mrs_01_1643__a9b80c5a831424fa7a045abb1afc1d6d3">The HBase bulkLoad task (a single table contains 26 TB data) has 210,000 maps and 10,000 reduce tasks (in MRS 3.x or later), and the task fails.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1643__s1f1d579b37ac42ed92a4e9c587124b6d"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_1643__a51bf8a6e5145419b9ecd44568db344bb"><strong id="mrs_01_1643__b7171111112208">ZooKeeper I/O bottleneck observation methods:</strong></p>
|
|
<ol id="mrs_01_1643__o4254576295f54b459b83b756e461f514"><li id="mrs_01_1643__ld817ffbf536e4f92a48104caea0d7ca4">On the monitoring page of Manager, check whether the number of ZooKeeper requests on a single node exceeds the upper limit.</li><li id="mrs_01_1643__le5aa66e2fcf246ccb967efdbe30c64ba">View ZooKeeper and HBase logs to check whether a large number of I/O Exception Timeout or SocketTimeout Exception exceptions occur.</li></ol>
|
|
<p id="mrs_01_1643__aa17c2d571cac416f9a4a7b4b49c899c4"><strong id="mrs_01_1643__b563711610202">Optimization suggestions:</strong></p>
|
|
<ol id="mrs_01_1643__o3f28c2d1e09b4c11ba1181ec37579c80"><li id="mrs_01_1643__lebb7a09423724bdc9799029f65cafd7d">Change the number of ZooKeeper instances to 5 or more. You are advised to set <strong id="mrs_01_1643__b2333134132112">peerType</strong> to <strong id="mrs_01_1643__b203382417219">observer</strong> to increase the number of observers.</li><li id="mrs_01_1643__lccb79ae84fa8442fb81fd6ddb053112b">Control the number of concurrent maps of a single task or reduce the memory for running tasks on each node to lighten the node load.</li><li id="mrs_01_1643__l69474c4649db4403967cbf4580eff2df">Upgrade ZooKeeper data disks, such as SSDs.</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1638.html">Common Issues About HBase</a></div>
|
|
</div>
|
|
</div>
|
|
|