forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
34 lines
4.6 KiB
HTML
34 lines
4.6 KiB
HTML
<a name="mrs_01_24485"></a><a name="mrs_01_24485"></a>
|
|
|
|
<h1 class="topictitle1">Closing HDFS Files</h1>
|
|
<div id="body0000001535782562"><div class="section" id="mrs_01_24485__se9608011680e423ca403d5207c374daa"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24485__p1123815249257">By default, an HDFS file can be closed only if all blocks are reported (in the <strong id="mrs_01_24485__b1883911260207">COMPLETED</strong> state). Therefore, the write performance of HDFS is affected by waiting for DataNode blocks and NameNode processing blocks to be reported. For a cluster with heavy load, the waiting consumption has a great impact on the cluster. You can configure the <strong id="mrs_01_24485__b1349784320226">dfs.namenode.file.close.num-committed-allowed</strong> parameter of HDFS to close files in advance to improve data write performance. However, data may fail to be read because the block cannot be found or the data block information recorded in the NameNode metadata is inconsistent with that stored in the DataNode. Therefore, this feature does not apply to the scenario where data is read immediately after being written. Exercise caution when using this feature.</p>
|
|
<div class="note" id="mrs_01_24485__note1089385819285"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_24485__p58931658152810">This section applies to MRS 3.<span id="mrs_01_24485__ph09721017131516">2.0</span> or later.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_24485__section9887727153615"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24485__ol184915551278"><li id="mrs_01_24485__li17755113014329"><span>Log in to FusionInsight Manager.</span></li><li id="mrs_01_24485__li17801143153219"><span>Choose <strong id="mrs_01_24485__b213536079341810">Cluster</strong> > <strong id="mrs_01_24485__b214285239841810">Services</strong> > <strong id="mrs_01_24485__b174888889141810">HDFS</strong> and click the <strong id="mrs_01_24485__b171547269841810">Configurations</strong> tab and then <strong id="mrs_01_24485__b56889936641810">All Configurations</strong>.</span></li><li id="mrs_01_24485__li15984184613212"><span>Search for and modify the <strong id="mrs_01_24485__b192482464241810">dfs.namenode.file.close.num-committed-allowed</strong> parameter. For more information, see the following table.</span><p>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24485__table58492554276" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24485__row12849185520278"><th align="left" class="cellrowborder" valign="top" width="44.16%" id="mcps1.3.2.2.3.2.1.1.3.1.1"><p id="mrs_01_24485__p78491955112717">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="55.84%" id="mcps1.3.2.2.3.2.1.1.3.1.2"><p id="mrs_01_24485__p88491055132712">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24485__row1784914559276"><td class="cellrowborder" valign="top" width="44.16%" headers="mcps1.3.2.2.3.2.1.1.3.1.1 "><p id="mrs_01_24485__p126070434328">dfs.namenode.file.close.num-committed-allowed</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="55.84%" headers="mcps1.3.2.2.3.2.1.1.3.1.2 "><p id="mrs_01_24485__p9646152418303">Maximum number of blocks in the <strong id="mrs_01_24485__b8804104912181">COMMITTED</strong> state in the file to be closed.</p>
|
|
<p id="mrs_01_24485__p4271310388">The default value is 0, indicating that this feature is disabled. If this feature is enabled, the recommended value is <strong id="mrs_01_24485__b16844144010171">1</strong> or <strong id="mrs_01_24485__b1874824311717">2</strong>.</p>
|
|
<p id="mrs_01_24485__p89708718314">For example, if this parameter is set to <strong id="mrs_01_24485__b1216055661717">1</strong>, it indicates that a file can be closed without waiting for status of the last block status to change to <strong id="mrs_01_24485__b1811117553188">COMPLETED</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</p></li><li id="mrs_01_24485__li919503517492"><span>Save the configuration.</span></li><li id="mrs_01_24485__li5580141944914"><span>On the <strong id="mrs_01_24485__b03861611184017">Instance</strong> page of HDFS, select the active and standby NameNode instances, choose <strong id="mrs_01_24485__b33861111124015">More</strong> > <strong id="mrs_01_24485__b1386191124014">Instance Rolling Restart</strong>, and wait until the rolling restart is complete.</span></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0790.html">Using HDFS</a></div>
|
|
</div>
|
|
</div>
|
|
|