forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
23 lines
3.0 KiB
HTML
23 lines
3.0 KiB
HTML
<a name="mrs_01_1691"></a><a name="mrs_01_1691"></a>
|
|
|
|
<h1 class="topictitle1">NameNode Startup Is Slow</h1>
|
|
<div id="body1597735018402"><div class="section" id="mrs_01_1691__sf3a6b34408cf42edb05de2b274b66cbb"><h4 class="sectiontitle">Question</h4><p id="mrs_01_1691__afea7d29fc6a54aeeab0fd991d7e44b65">The NameNode startup is slow when it is restarted immediately after a large number of files (for example, 1 million files) are deleted.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1691__sae6b9482a55143aea893161108bd2547"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_1691__a9c1718311a2841478050e8342b1fee4e">It takes time for the DataNode to delete the corresponding blocks after files are deleted. When the NameNode is restarted immediately, it checks the block information reported by all DataNodes. If a deleted block is found, the NameNode generates the corresponding INFO log information, as shown below:</p>
|
|
<pre class="screen" id="mrs_01_1691__s68193abc02454dccb9017a809ea7555a">2015-06-10 19:25:50,215 | INFO | IPC Server handler 36 on 25000 | BLOCK* processReport:
|
|
blk_1075861877_2121067 on node 10.91.8.218:9866 size 10249 does not belong to any file |
|
|
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.processReport(BlockManager.java:1854)</pre>
|
|
<p id="mrs_01_1691__a8d1db27752d643eeb3692288b99de792">A log is generated for each deleted block. A file may contain one or more blocks. Therefore, after startup, the NameNode spends a large amount of time printing logs when a large number of files are deleted. As a result, the NameNode startup becomes slow.</p>
|
|
<p id="mrs_01_1691__aed332e0e5b714cceaa8334918bbc91ed">To address this issue, the following operations can be performed to speed up the startup:</p>
|
|
<ol id="mrs_01_1691__o091392cf930d486f86144e336da1c189"><li id="mrs_01_1691__l4e61d1a1c5134653b94a7a065e1fc3d1">After a large number of files are deleted, wait until the DataNode deletes the corresponding blocks and then restart the NameNode.<p id="mrs_01_1691__a89d3a977360f49d482f6a9dab8f93c11"><a name="mrs_01_1691__l4e61d1a1c5134653b94a7a065e1fc3d1"></a><a name="l4e61d1a1c5134653b94a7a065e1fc3d1"></a>You can run the <i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1691__cmdname3653122582119">hdfs dfsadmin -report</span></b></i> command to check the disk space and check whether the files have been deleted.</p>
|
|
</li><li id="mrs_01_1691__leb4c3de4e05c41b4ad45690418066be7">If a large number of the preceding logs are generated, you can change the NameNode log level to <strong id="mrs_01_1691__b150993753122021">ERROR</strong> so that the NameNode stops printing such logs.<p id="mrs_01_1691__a2d9a11b0ac074f57aa84794d90bd5005">After the NameNode is restarted, change the log level back to <strong id="mrs_01_1691__b181849470822021">INFO</strong>. You do not need to restart the service after changing the log level.</p>
|
|
</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1690.html">FAQ</a></div>
|
|
</div>
|
|
</div>
|
|
|