forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
64 lines
6.7 KiB
HTML
64 lines
6.7 KiB
HTML
<a name="mrs_01_1646"></a><a name="mrs_01_1646"></a>
|
|
|
|
<h1 class="topictitle1">Why Does SocketTimeoutException Occur When a Client Queries HBase?</h1>
|
|
<div id="body1596003894351"><div class="section" id="mrs_01_1646__s65bb8080562a4185b148f9fd8f535c44"><h4 class="sectiontitle">Question</h4><p id="mrs_01_1646__a7bcfd5e99448411182924b411fb1bbeb">Why does the following exception occur on the client when I use the HBase client to operate table data?</p>
|
|
<pre class="screen" id="mrs_01_1646__s322fcf3dddfb4853b253fdb92454ee88">2015-12-15 02:41:14,054 | WARN | [task-result-getter-2] | Lost task 2.0 in stage 58.0 (TID 3288, linux-175):
|
|
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
|
|
Tue Dec 15 02:41:14 CST 2015, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=60303:
|
|
row 'xxxxxx' on table 'xxxxxx' at region=xxxxxx,\x05\x1E\x80\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x000\x00\x80\x00\x00\x00\x80\x00\x00\x00\x80\x00\x00,
|
|
1449912620868.6a6b7d0c272803d8186930a3bfdb10a9., hostname=xxxxxx,16020,1449941841479, seqNum=5
|
|
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:275)
|
|
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:223)
|
|
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
|
|
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
|
|
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:323)</pre>
|
|
</div>
|
|
<p id="mrs_01_1646__ac433a1490a8746a2a6288d5550e256e5">At the same time, the following log is displayed on RegionServer:</p>
|
|
<pre class="screen" id="mrs_01_1646__s3f99036ba4904a7e872a657896f619d4">2015-12-15 02:45:44,551 | WARN | PriorityRpcServer.handler=7,queue=1,port=16020 | (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)
|
|
","starttimems":1450118730780,"responsesize":416,"method":"Scan","processingtimems":13770,"client":"10.91.8.175:41182","queuetimems":0,"class":"HRegionServer"} |
|
|
org.apache.hadoop.hbase.ipc.RpcServer.logResponse(RpcServer.java:2221)
|
|
2015-12-15 02:45:57,722 | WARN | PriorityRpcServer.handler=3,queue=1,port=16020 | (responseTooSlow):
|
|
{"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1450118746297,"responsesize":416,
|
|
"method":"Scan","processingtimems":11425,"client":"10.91.8.175:41182","queuetimems":1746,"class":"HRegionServer"} | org.apache.hadoop.hbase.ipc.RpcServer.logResponse(RpcServer.java:2221)
|
|
2015-12-15 02:47:21,668 | INFO | LruBlockCacheStatsExecutor | totalSize=7.54 GB, freeSize=369.52 MB, max=7.90 GB, blockCount=406107,
|
|
accesses=35400006, hits=16803205, hitRatio=47.47%, , cachingAccesses=31864266, cachingHits=14806045, cachingHitsRatio=46.47%,
|
|
evictions=17654, evicted=16642283, evictedPerRun=942.69189453125 | org.apache.hadoop.hbase.io.hfile.LruBlockCache.logStats(LruBlockCache.java:858)
|
|
2015-12-15 02:52:21,668 | INFO | LruBlockCacheStatsExecutor | totalSize=7.51 GB, freeSize=395.34 MB, max=7.90 GB, blockCount=403080,
|
|
accesses=35685793, hits=16933684, hitRatio=47.45%, , cachingAccesses=32150053, cachingHits=14936524, cachingHitsRatio=46.46%,
|
|
evictions=17684, evicted=16800617, evictedPerRun=950.046142578125 | org.apache.hadoop.hbase.io.hfile.LruBlockCache.logStats(LruBlockCache.java:858)</pre>
|
|
<div class="section" id="mrs_01_1646__s16d4e44099f243b3882c1d080e6ecce6"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_1646__aecfffed988a647979ed7799cd16d456f">The memory allocated to RegionServer is too small and the number of Regions is too large. As a result, the memory is insufficient during the running, and the server responds slowly to the client. Modify the following memory allocation parameters in the <span class="filepath" id="mrs_01_1646__fc6b00092dbe0479f94e2d88ce91d4fed"><b>hbase-site.xml</b></span> configuration file of RegionServer:</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1646__t55a5bb70755a4368b0f178b740ba0480" frame="border" border="1" rules="all"><caption><b>Table 1 </b>RegionServer memory allocation parameters</caption><thead align="left"><tr id="mrs_01_1646__r43fe334d68bf41148e5872663da4985d"><th align="left" class="cellrowborder" valign="top" width="18.86%" id="mcps1.3.4.3.2.4.1.1"><p id="mrs_01_1646__a8392c8c4a8f4454f995e4f2ae25a2260"><strong id="mrs_01_1646__b12653131016567">Parameter</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="57.21000000000001%" id="mcps1.3.4.3.2.4.1.2"><p id="mrs_01_1646__a019204afb78c40c5a9c8f66b92f7a7ca"><strong id="mrs_01_1646__b1231501445612">Description</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="23.93%" id="mcps1.3.4.3.2.4.1.3"><p id="mrs_01_1646__af0b40bb325a04b9cb3fe622a8f7c27ed"><strong id="mrs_01_1646__b1923701520564">Default Value</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1646__r41365c7e0a2945a7988631c672003c53"><td class="cellrowborder" valign="top" width="18.86%" headers="mcps1.3.4.3.2.4.1.1 "><p id="mrs_01_1646__a858a4b4015664ea38787829c0e6b2807">GC_OPTS</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="57.21000000000001%" headers="mcps1.3.4.3.2.4.1.2 "><p id="mrs_01_1646__a77b7258316eb4abf89074f8c8f6094c4">Initial memory and maximum memory allocated to RegionServer in startup parameters.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.93%" headers="mcps1.3.4.3.2.4.1.3 "><p id="mrs_01_1646__a699abcbedd8b4a299c21482893cc852f">-Xms8G -Xmx8G</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1646__r37c47a7b92724b9a9297b364e796d8b6"><td class="cellrowborder" valign="top" width="18.86%" headers="mcps1.3.4.3.2.4.1.1 "><p id="mrs_01_1646__a3cdc1927f51341f2ae41fabf31821efa">hfile.block.cache.size</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="57.21000000000001%" headers="mcps1.3.4.3.2.4.1.2 "><p id="mrs_01_1646__a336d5f963cd3431daf20d79f81056f45">Percentage of the maximum heap (-Xmx setting) allocated to the block cache of HFiles or StoreFiles.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.93%" headers="mcps1.3.4.3.2.4.1.3 "><p id="mrs_01_1646__ae915a88990744c048a79161228108024">When <strong id="mrs_01_1646__b12378173515915">offheap</strong> is disabled, the default value is <strong id="mrs_01_1646__b738053585916">0.25</strong>. When <strong id="mrs_01_1646__b163561138185914">offheap</strong> is enabled, the default value is <strong id="mrs_01_1646__b4360338125913">0.1</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1638.html">Common Issues About HBase</a></div>
|
|
</div>
|
|
</div>
|
|
|