forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
49 lines
5.1 KiB
HTML
49 lines
5.1 KiB
HTML
<a name="mrs_01_1687"></a><a name="mrs_01_1687"></a>
|
|
|
|
<h1 class="topictitle1">Improving Write Performance</h1>
|
|
<div id="body1595904097002"><div class="section" id="mrs_01_1687__s888991eaee3541dab114038504907c33"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1687__a838e1a26f18747d4bc6822fa93f9c023">Improve the HDFS write performance by modifying the HDFS attributes.</p>
|
|
<div class="note" id="mrs_01_1687__note17408121619136"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1687__p788202018132">This section applies to MRS 3.<em id="mrs_01_1687__i1520317115116">x</em> or later.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1687__s7653e7791b1346bb8050de05d8a69938"><h4 class="sectiontitle">Procedure</h4><p id="mrs_01_1687__ab2e157acbcaf4afaae47fcb0bb351433">Navigation path for setting parameters:</p>
|
|
<p id="mrs_01_1687__a5ec540c7b51643fe879f73b61ac6b255">On FusionInsight Manager, choose <span id="mrs_01_1687__text193118495113"><strong id="mrs_01_1687__b1293074185119">Cluster</strong> > </span><em id="mrs_01_1687__i18931134175116">Name of the desired cluster</em> > <strong id="mrs_01_1687__b9932174195111">Services</strong> > <strong id="mrs_01_1687__b1193215415513">HDFS</strong> > <strong id="mrs_01_1687__b199321548511">Configurations</strong> and select <strong id="mrs_01_1687__b393318412510">All Configurations</strong>. Enter a parameter name in the search box.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1687__td00e5f3489fe44289a2464a9369aff34" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters for improving HDFS write performance</caption><thead align="left"><tr id="mrs_01_1687__rff041b07a8b340a1926304b1f6ab1ea1"><th align="left" class="cellrowborder" valign="top" width="26.57%" id="mcps1.3.2.4.2.4.1.1"><p id="mrs_01_1687__a92784d1c93974d69ba94e0e101ac08c6">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="52.5%" id="mcps1.3.2.4.2.4.1.2"><p id="mrs_01_1687__ae56e2b0fba104ff6a445ab7f6d1ca1a3">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20.93%" id="mcps1.3.2.4.2.4.1.3"><p id="mrs_01_1687__a44eb6738bb594cea9aef6fbc10f1bc21">Default Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1687__r352c6dad6ecf4b0dbc9010f8f7c2248b"><td class="cellrowborder" valign="top" width="26.57%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1687__a188f9c9023604b36a8c652e3adf97030">dfs.datanode.drop.cache.behind.reads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="52.5%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1687__p2041064841615">Specifies whether to enable a DataNode to automatically clear all data in the cache after the data in the cache is transferred to the client.</p>
|
|
<ul id="mrs_01_1687__ul138431841142112"><li id="mrs_01_1687__li20843104110215"><strong id="mrs_01_1687__b31301930505">true</strong>: The cached data is discarded. This parameter needs to be configured on the DataNode.<p id="mrs_01_1687__p1945617816227">You are advised to set it to <strong id="mrs_01_1687__b465065417504">true</strong> if data is repeatedly read only a few times, so that the cache can be used by other operations.</p>
|
|
</li><li id="mrs_01_1687__li8843124110219"><strong id="mrs_01_1687__b739117206519">false</strong>: You are advised to set it to <strong id="mrs_01_1687__b661411425218">false</strong> if data is read repeatedly for many times to improve the read speed.</li></ul>
|
|
<div class="note" id="mrs_01_1687__note17182138121219"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1687__p171847381122">This parameter is optional for improving write performance. You can configure it as needed.</p>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.93%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1687__a2b1406210a2e4777933e8fb14b2b0116">false</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1687__r4fd857e1bc634a5d8e0e5e5a5dfb0139"><td class="cellrowborder" valign="top" width="26.57%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1687__aa60447591b0d418fbe1545052b1b9667">dfs.client-write-packet-size</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="52.5%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1687__afe715bbc8fc648a4bbd6988ed39e5c67">Specifies the size of the client write packet. When the HDFS client writes data to the DataNode, the data will be accumulated until a packet is generated. Then, the packet is transmitted over the network. This parameter specifies the size (unit: byte) of the data packet to be transmitted, which can be specified by each job.</p>
|
|
<p id="mrs_01_1687__a4d8a5adafac2427eb940f27a2e494a40">In the 10-Gigabit network, you can increase the value of this parameter to enhance the transmission throughput.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.93%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1687__ac16fd39b79724596b4662e3a05f77392">262144</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0829.html">HDFS Performance Tuning</a></div>
|
|
</div>
|
|
</div>
|
|
|