forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
30 lines
7.6 KiB
HTML
30 lines
7.6 KiB
HTML
<a name="mrs_01_0502"></a><a name="mrs_01_0502"></a>
|
|
|
|
<h1 class="topictitle1">Enabling Cross-Cluster Copy</h1>
|
|
<div id="body8662426"><div class="section" id="mrs_01_0502__en-us_topic_0000001219230537_section4982615016933"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0502__en-us_topic_0000001219230537_p938634316933">DistCp is used to copy the data stored on HDFS from a cluster to another cluster. DistCp depends on the cross-cluster copy function, which is disabled by default. This function needs to be enabled in both clusters.</p>
|
|
<p id="mrs_01_0502__en-us_topic_0000001219230537_p6464882916933">This section describes how to enable cross-cluster copy.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0502__en-us_topic_0000001219230537_section4496855716933"><h4 class="sectiontitle">Impact on the System</h4><p id="mrs_01_0502__en-us_topic_0000001219230537_p1857453216933">Yarn needs to be restarted to enable the cross-cluster copy function and cannot be accessed during the restart.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0502__en-us_topic_0000001219230537_section3295306016933"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0502__en-us_topic_0000001219230537_p5195218816933">The <strong id="mrs_01_0502__en-us_topic_0000001219230537_b1521625426">hadoop.rpc.protection</strong> parameter of the two HDFS clusters must be set to the same data transmission mode, which can be <strong id="mrs_01_0502__en-us_topic_0000001219230537_b1677512291125">privacy</strong> (encryption enabled) or <strong id="mrs_01_0502__en-us_topic_0000001219230537_b0623153411211">authentication</strong> (encryption disabled).</p>
|
|
<div class="note" id="mrs_01_0502__en-us_topic_0000001219230537_note36461715185452"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__en-us_topic_0000001219230537_p191517115388">Go to the <strong id="mrs_01_0502__en-us_topic_0000001219230537_b549132718174">All Configurations</strong> page by referring to <a href="mrs_01_1293.html">Modifying Cluster Service Configuration Parameters</a> and search for <strong id="mrs_01_0502__en-us_topic_0000001219230537_b134399372176">hadoop.rpc.protection</strong>.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_0502__en-us_topic_0000001219230537_section6491650916933"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0502__en-us_topic_0000001219230537_ol2374590516933"><li id="mrs_01_0502__en-us_topic_0000001219230537_li1966213718714"><span>Go to the <strong id="mrs_01_0502__en-us_topic_0000001219230537_b2751163393918">All Configurations</strong> page of the Yarn service. For details, see <a href="mrs_01_1293.html">Modifying Cluster Service Configuration Parameters</a>.</span><p><div class="note" id="mrs_01_0502__en-us_topic_0000001219230537_note3349205551313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__en-us_topic_0000001219230537_p10595164105119">If the <strong id="mrs_01_0502__en-us_topic_0000001219230537_b6579171584310">Components</strong> tab is not displayed on the cluster details page, complete IAM user synchronization first. (In the <strong id="mrs_01_0502__en-us_topic_0000001219230537_b6580111534311">Dashboard</strong> area of the cluster details page, click <strong id="mrs_01_0502__en-us_topic_0000001219230537_b258071534319">Click to synchronize</strong> on the right of <span class="parmname" id="mrs_01_0502__en-us_topic_0000001219230537_parmname95810158433"><b>IAM User Sync</b></span> to synchronize IAM users.)</p>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_0502__en-us_topic_0000001219230537_li6378706216933"><span>In the navigation pane, choose <strong id="mrs_01_0502__en-us_topic_0000001219230537_b1640775534813">Yarn</strong> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b1794423334719">Distcp</strong>.</span></li><li id="mrs_01_0502__en-us_topic_0000001219230537_li3721265416933"><span>Set <strong id="mrs_01_0502__en-us_topic_0000001219230537_b1569813219450">haclusterX.remotenn1</strong> of <strong id="mrs_01_0502__en-us_topic_0000001219230537_b068010164452">dfs.namenode.rpc-address</strong> to the service IP address and RPC port number of one NameNode instance of the peer cluster, and set <strong id="mrs_01_0502__en-us_topic_0000001219230537_b124923454911">haclusterX.remotenn2</strong> to the service IP address and RPC port number of the other NameNode instance of the peer cluster. Enter a value in the <em id="mrs_01_0502__en-us_topic_0000001219230537_i17809946303">IP address:port</em> format.</span><p><div class="note" id="mrs_01_0502__en-us_topic_0000001219230537_note60675655181130"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__en-us_topic_0000001219230537_p690124713013">For MRS 1.9.2 or later, log in to the MRS console, click the cluster name, and choose <strong id="mrs_01_0502__en-us_topic_0000001219230537_b6816222010">Components</strong> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b5973547205">HDFS</strong> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b239568172014">Instances</strong> to obtain the service IP address of the NameNode instance.</p>
|
|
<p id="mrs_01_0502__en-us_topic_0000001219230537_p1831135910351">You can also log in to FusionInsight Manager, and choose <strong id="mrs_01_0502__en-us_topic_0000001219230537_b16571115211">Cluster</strong> > <em id="mrs_01_0502__en-us_topic_0000001219230537_i964575115238">Name of the desired cluster</em> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b1646915518210">Services</strong> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b26321353152115">HDFS</strong> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b2805951162112">Instance</strong> to obtain the service IP address of the NameNode instance.</p>
|
|
</div></div>
|
|
<p id="mrs_01_0502__en-us_topic_0000001219230537_p6647843216933"><strong id="mrs_01_0502__en-us_topic_0000001219230537_b625311530509">dfs.namenode.rpc-address.haclusterX.remotenn1</strong> and <strong id="mrs_01_0502__en-us_topic_0000001219230537_b158577145115">dfs.namenode.rpc-address.haclusterX.remotenn2</strong> do not distinguish active and standby NameNode instances. The default NameNode RPC port is 9820 and cannot be modified on MRS Manager.</p>
|
|
<p id="mrs_01_0502__en-us_topic_0000001219230537_p6143498316933">For example, <strong id="mrs_01_0502__en-us_topic_0000001219230537_b258505053117">10.1.1.1:9820</strong> and <strong id="mrs_01_0502__en-us_topic_0000001219230537_b16903545315">10.1.1.2:9820</strong>.</p>
|
|
</p></li><li id="mrs_01_0502__en-us_topic_0000001219230537_li2449066216933"><span>Save the configuration. On the <strong id="mrs_01_0502__en-us_topic_0000001219230537_b169576303174">Dashboard</strong> tab page, and choose <strong id="mrs_01_0502__en-us_topic_0000001219230537_b169571530131719">More</strong> > <strong id="mrs_01_0502__en-us_topic_0000001219230537_b16957123012175">Restart Service</strong> to restart the Yarn service.</span><p><p id="mrs_01_0502__en-us_topic_0000001219230537_p1908936716933"><strong id="mrs_01_0502__en-us_topic_0000001219230537_b181312576571">Operation succeeded</strong> is displayed. Click <strong id="mrs_01_0502__en-us_topic_0000001219230537_b177011150586">Finish</strong>. The Yarn service is started successfully.</p>
|
|
</p></li><li id="mrs_01_0502__en-us_topic_0000001219230537_li3758657716933"><span>Log in to the other cluster and repeat the preceding operations.</span></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0500.html">Using HBase</a></div>
|
|
</div>
|
|
</div>
|
|
|