forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
24 lines
6.6 KiB
HTML
24 lines
6.6 KiB
HTML
<a name="mrs_01_1829"></a><a name="mrs_01_1829"></a>
|
|
|
|
<h1 class="topictitle1">Submitting a DistCp Job</h1>
|
|
<div id="body1591347175449"><div class="section" id="mrs_01_1829__s26fe7077456a424ea246b3aa6d24fc8e"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1829__a1492a0bbdca64153860263304849bebc">This section describes how to submit an Oozie job of the DistCp type on the Hue web UI.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1829__s32fd4b7ddc844fb0a40b374d0fab9b81"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1829__oc119de0018c9458dba1ccb9c13c09b81"><li id="mrs_01_1829__l1b4c10709a3240d6ac39edf0eb51174d"><span>Create a workflow. For details, see <a href="mrs_01_1818.html">Creating a Workflow</a>.</span></li><li id="mrs_01_1829__lac0765f30ee443d3bf3ca65938df3c24"><span>On the workflow editing page, select <span><img id="mrs_01_1829__i7eb48927033d46099667e89070e4d7ac" src="en-us_image_0000001349089981.jpg"></span> next to <strong id="mrs_01_1829__b117141794712">Distcp</strong> and drag it to the operation area.</span></li><li id="mrs_01_1829__li16760834133919"><span>Determine whether the current DistCp operation is performed across clusters.</span><p><ul id="mrs_01_1829__ul789195253913"><li id="mrs_01_1829__li12683185917392">If yes, go to <a href="#mrs_01_1829__li15967531204920">4</a>.</li><li id="mrs_01_1829__li689219524398">If no, go to <a href="#mrs_01_1829__le4a671e8c2a94b948e244fe188cfe08f">7</a>.</li></ul>
|
|
</p></li><li id="mrs_01_1829__li15967531204920"><a name="mrs_01_1829__li15967531204920"></a><a name="li15967531204920"></a><span>Establish cross-Manager mutual trust between two clusters.</span></li><li id="mrs_01_1829__li6622173114373"><span>In the <strong id="mrs_01_1829__b646613210492">Distcp</strong> window that is displayed, set the value of <span class="parmname" id="mrs_01_1829__parmname54674214495"><b>Source</b></span>, for example, to <strong id="mrs_01_1829__b104676212491">hdfs://hacluster/user/admin/examples/input-data/text/data.txt</strong>. Set <span class="parmname" id="mrs_01_1829__parmname9160207195015"><b>Destination</b></span>, for example, to <strong id="mrs_01_1829__b1816519765019">hdfs://target_ip:target_port/user/admin/examples/output-data/distcp-workflow/data.txt</strong>. Click <strong id="mrs_01_1829__b28201921195013">Add</strong>.</span></li><li id="mrs_01_1829__li13807104654112"><span>Click the configuration button <span><img id="mrs_01_1829__image18101717423" src="en-us_image_0000001296090140.jpg"></span> in the upper right corner. On the <strong id="mrs_01_1829__b1543221211519">Properties</strong> tab page, click <strong id="mrs_01_1829__b693202045116">PROPERTIES+</strong>, enter the attribute name <strong id="mrs_01_1829__b1543814281519">oozie.launcher.mapreduce.job.hdfs-servers</strong> in the text box on the left, enter the attribute value <strong id="mrs_01_1829__b649705575214">hdfs://</strong><em id="mrs_01_1829__i145381641193412">source_ip</em><em id="mrs_01_1829__i1330365916161">:source_port</em><strong id="mrs_01_1829__b197891939132011">,</strong><strong id="mrs_01_1829__b1413018172013">hdfs://</strong><em id="mrs_01_1829__i327271417178">target_ip:target_port</em> in the text box on the right, and go to <a href="#mrs_01_1829__lfc85380934e241ed8783edbf9b9a4e0f">8</a>.</span><p><div class="note" id="mrs_01_1829__note1826103901612"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1829__p175601324102317"><em id="mrs_01_1829__i123911869532">source_ip</em>: service address of the HDFS NameNode in the source cluster</p>
|
|
<p id="mrs_01_1829__p128277393163"><em id="mrs_01_1829__i1443511915310">source_port</em>: port number of the HDFS NameNode in the source cluster.</p>
|
|
<p id="mrs_01_1829__p1399011118244"><em id="mrs_01_1829__i1912394045319">target_ip</em>: service address of the HDFS NameNode in the target cluster</p>
|
|
<p id="mrs_01_1829__p1137434116185"><em id="mrs_01_1829__i196081659205314">target_port</em>: port number of the HDFS NameNode in the target cluster.</p>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_1829__le4a671e8c2a94b948e244fe188cfe08f"><a name="mrs_01_1829__le4a671e8c2a94b948e244fe188cfe08f"></a><a name="le4a671e8c2a94b948e244fe188cfe08f"></a><span>In the <strong id="mrs_01_1829__b255832512476">Distcp</strong> window that is displayed, set the value of <span class="parmname" id="mrs_01_1829__parmname47921330162917"><b>Source</b></span>, for example, to <strong id="mrs_01_1829__b7559162534712">/user/admin/examples/input-data/text/data.txt</strong>. Set <span class="parmname" id="mrs_01_1829__parmname5391039122912"><b>Destination</b></span>, for example, to <strong id="mrs_01_1829__b63141327164714">/user/admin/examples/output-data/distcp-workflow/data.txt</strong>. Click <strong id="mrs_01_1829__b129499285476">Add</strong>.</span></li><li id="mrs_01_1829__lfc85380934e241ed8783edbf9b9a4e0f"><a name="mrs_01_1829__lfc85380934e241ed8783edbf9b9a4e0f"></a><a name="lfc85380934e241ed8783edbf9b9a4e0f"></a><span>Click <span><img id="mrs_01_1829__if18ee05828cb4f45b5cd244181c9af4b" src="en-us_image_0000001349289449.jpg"></span> in the upper right corner. On the configuration page that is displayed, click <span class="uicontrol" id="mrs_01_1829__uicontrol1223083054716"><b>Delete+</b></span> and add the directory to be deleted, for example, <strong id="mrs_01_1829__b1723115308470">/user/admin/examples/output-data/distcp-workflow</strong>.</span><p><p id="mrs_01_1829__p5262163113017"><span><img id="mrs_01_1829__image1526214311203" src="en-us_image_0000001295770356.png"></span></p>
|
|
</p></li><li id="mrs_01_1829__la0464a04972143f898582ba514dfb983"><span>Click <span><img id="mrs_01_1829__image1236335111194" src="en-us_image_0000001349289453.png"></span> in the upper right corner of the Oozie editor.</span><p><p id="mrs_01_1829__a22f905d43ed7482788c19638c96e5615">If you need to modify the job name before saving the job (default value: <span class="parmname" id="mrs_01_1829__parmname178541856144713"><b>My Workflow</b></span>), click the name directly for modification, for example, <strong id="mrs_01_1829__b1785975644718">Distcp-Workflow</strong>.</p>
|
|
</p></li><li id="mrs_01_1829__lcc30e61755fd4c3bbc246067b1827117"><span>After the configuration is saved, click <span><img id="mrs_01_1829__ibf7141bce5fc4e8c98612031747e06a3" src="en-us_image_0000001349169877.jpg"></span>, and submit the job.</span><p><p id="mrs_01_1829__a70fa221968564db2bd8a0f4e5df52793">After the job is submitted, you can view the related contents of the job, such as the detailed information, logs, and processes, on Hue.</p>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1819.html">Submitting a Workflow Job</a></div>
|
|
</div>
|
|
</div>
|
|
|