forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
62 lines
7.3 KiB
HTML
62 lines
7.3 KiB
HTML
<a name="mrs_01_1689"></a><a name="mrs_01_1689"></a>
|
|
|
|
<h1 class="topictitle1">Improving the Connection Between the Client and NameNode Using Current Active Cache</h1>
|
|
<div id="body1595904097006"><div class="section" id="mrs_01_1689__s60a41fab02a745dc8c086b48cb2b85dd"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1689__af142ecb1738345c2bbeb29bbccfcdcee">When HDFS is deployed in high availability (HA) mode with multiple NameNode instances, the HDFS client needs to connect to each NameNode in sequence to determine which is the active NameNode and use it for client operations.</p>
|
|
<p id="mrs_01_1689__aa72120dc82ba40b9a0918d0eacc9d472">Once the active NameNode is identified, its details can be cached and shared to all clients running on the client host. In this way, each new client first tries to load the details of the active Name Node from the cache and save the RPC call to the standby NameNode, which can help a lot in abnormal scenarios, for example, when the standby NameNode cannot be connected for a long time.</p>
|
|
<p id="mrs_01_1689__ac0aadfb3c2a140e3bae2c1340f93baab">When a fault occurs and the other NameNode is switched to the active state, the cached details are updated to the information about the current active NameNode.</p>
|
|
<div class="note" id="mrs_01_1689__note17408121619136"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1689__p788202018132">This section applies to MRS 3.<em id="mrs_01_1689__i20492142914549">x</em> or later.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1689__sac1b4679e677447aac77640111dd4421"><h4 class="sectiontitle">Procedure</h4><p id="mrs_01_1689__a5d73b837da074480a1bc0235abd12a48">Navigation path for setting parameters:</p>
|
|
<p id="mrs_01_1689__acaffa0053ca84bbb856c62cd5abb4c3f">On FusionInsight Manager, choose <span id="mrs_01_1689__text79112367811"><strong id="mrs_01_1689__b1364918321547">Cluster</strong> > </span><em id="mrs_01_1689__i186505322541">Name of the desired cluster</em> > <strong id="mrs_01_1689__b14650932105412">Services</strong> > <strong id="mrs_01_1689__b13651113219547">HDFS</strong> > <strong id="mrs_01_1689__b56517328548">Configurations</strong>, select <strong id="mrs_01_1689__b1965218321542">All Configurations</strong>, and enter the parameter name in the search box.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1689__tf253de762d4741ac8f169e1a13f91cf2" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Configuration parameters</caption><thead align="left"><tr id="mrs_01_1689__r2d2ff346fdfb4034a32663928f15d53b"><th align="left" class="cellrowborder" valign="top" width="26.57%" id="mcps1.3.2.4.2.4.1.1"><p id="mrs_01_1689__a065ffb00c249483db9abfa465b8c87dc">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="52.31%" id="mcps1.3.2.4.2.4.1.2"><p id="mrs_01_1689__a9a818fd65ef04a8a87eb4c7945e94b4c">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="21.12%" id="mcps1.3.2.4.2.4.1.3"><p id="mrs_01_1689__a74b3c743dc4d4aa6b4120e0302589aee">Default Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1689__r5bd5378cbcfc4320b00917b682466e25"><td class="cellrowborder" valign="top" width="26.57%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1689__a0c456bb74ffb4c7d8209fbc8af69f08c">dfs.client.failover.proxy.provider.[nameservice ID]</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="52.31%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1689__a0a40956f57cd4aacb47551243af34e1e">Client Failover proxy provider class which creates the NameNode proxy using the authenticated protocol. If this parameter is set to <strong id="mrs_01_1689__b19434195013118">org.apache.hadoop.hdfs.server.namenode.ha.BlackListingFailoverProxyProvider</strong>, you can use the NameNode blacklist feature on the HDFS client. If this parameter is set to <strong id="mrs_01_1689__b1247944317329">org.apache.hadoop.hdfs.server.namenode.ha.ObserverReadProxyProvider</strong>, you can configure the observer NameNode to process read requests.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.12%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1689__abf9ec7f6d2db4e1499c02340fdb08697">org.apache.hadoop.hdfs.server.namenode.ha.AdaptiveFailoverProxyProvider</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1689__r914f57aec3bd4f4baa6b5ff14b9d34d4"><td class="cellrowborder" valign="top" width="26.57%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1689__ac8103c41d78e4ceea9664e91bcdc70ab">dfs.client.failover.activeinfo.share.flag</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="52.31%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1689__a8d7b512c09364679a0d704afff25645c">Specifies whether to enable the cache function and share the detailed information about the current active NameNode with other clients. Set it to <span class="parmvalue" id="mrs_01_1689__parmvalue181867855613"><b>true</b></span> to enable the cache function.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.12%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1689__a6ada73e30a95401ab39215f38b8617a0">false</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1689__rdb26491ead654a9d870485df417938f9"><td class="cellrowborder" valign="top" width="26.57%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1689__a159b9c70afc5401f890e7560544fe06d">dfs.client.failover.activeinfo.share.path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="52.31%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1689__a744c476371e241ca8a3d2763ab840edb">Specifies the local directory for storing the shared files created by all clients in the host. If a cache area is to be shared by different users, the directory must have required permissions (for example, creating, reading, and writing cache files in the specified directory).</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.12%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1689__acafb4626845a4b8a9a145e0abeb8bcfa">/tmp</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1689__rf1d42cb2e95e4acea710d1cb6dda84a2"><td class="cellrowborder" valign="top" width="26.57%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1689__ac1304fcfca3d451db4ebc4da82334eeb">dfs.client.failover.activeinfo.share.io.timeout.sec</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="52.31%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1689__afeb8f45d107646f9ae0bd2b2b85f865c">(Optional) Used to control timeout. The cache file is locked when it is being read or written, and if the file cannot be locked within the specified time, the attempt to read or update the caches will be abandoned. The unit is second.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.12%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1689__ae087a5e12f974deab5229527f0afd03d">5</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div class="note" id="mrs_01_1689__n22a1ff884b284234b1dc44107ac15637"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1689__a45bdce72430a4f459b7c86b1b0b71019">The cache files created by the HDFS client are reused by other clients, and thus these files will not be deleted from the local system. If this function is disabled, you may need to manually clear the data.</p>
|
|
</div></div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0829.html">HDFS Performance Tuning</a></div>
|
|
</div>
|
|
</div>
|
|
|