forked from docs/doc-exports
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
509 lines
69 KiB
HTML
509 lines
69 KiB
HTML
<a name="mrs_01_1157"></a><a name="mrs_01_1157"></a>
|
|
|
|
<h1 class="topictitle1">loader-tool Usage Guide</h1>
|
|
<div id="body8662426"><div class="section" id="mrs_01_1157__en-us_topic_0000001219148949_s7f6e7d12184e42879e3fdabf52f6f058"><h4 class="sectiontitle">Overview</h4><p id="mrs_01_1157__en-us_topic_0000001219148949_ad31c5ebd94ab408fb26f45048a203de8">loader-tool is a Loader client tool. It consists of three tools: <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_f09ab752eeb8f4c079f565d5ceaea86b3"><b>lt-ucc</b></span>, <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_f8fcd96b25a8d46e69bb7c6e5f4834319"><b>lt-ucj</b></span>, <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_f8863af29ad1b491ba48f10db7ecacdc2"><b>lt-ctl</b></span>.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a105ecf89319b49629b5fa7c51d6f9d12">Loader supports two modes, parameter mode and job template mode. Either mode can be used to create, update, query, and delete connectors, and to create, update, query, delete, start, and stop Loader jobs.</p>
|
|
<div class="note" id="mrs_01_1157__en-us_topic_0000001219148949_note175617521295"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1157__en-us_topic_0000001219148949_p147561652393">loader-tool implements an asynchronous interface. After a command is submitted, the command output is not returned to the console in real time. Therefore, the results of the creation, update, query, and deletion operations on a connector and the creation, update, query, deletion, start, and stop operations on a Loader job must be confirmed on the Loader WebUI or by querying server logs.</p>
|
|
</div></div>
|
|
<ul id="mrs_01_1157__en-us_topic_0000001219148949_ub9a4446306564ce2965856c5633aa7ea"><li id="mrs_01_1157__en-us_topic_0000001219148949_l52077bd3bc774c57b2a7f9f27ad1bd45">Parameter mode:<p id="mrs_01_1157__en-us_topic_0000001219148949_a0596ec67ffb742e19459c629c0a6867c"><a name="mrs_01_1157__en-us_topic_0000001219148949_l52077bd3bc774c57b2a7f9f27ad1bd45"></a><a name="en-us_topic_0000001219148949_l52077bd3bc774c57b2a7f9f27ad1bd45"></a>Add a parameter invoking script with specific parameters.</p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l296529f28c2b4c6e9418ae670a8d149c">Job template mode:<p id="mrs_01_1157__en-us_topic_0000001219148949_a6309bbd5be39449991d293a8d43b4602"><a name="mrs_01_1157__en-us_topic_0000001219148949_l296529f28c2b4c6e9418ae670a8d149c"></a><a name="en-us_topic_0000001219148949_l296529f28c2b4c6e9418ae670a8d149c"></a>Change the values of all parameters in a job template and reference the job template when invoking a script.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a0fedeb329db54e6e9207e6ef8e7418d0">After a Loader client is installed, the system automatically generates job templates for various scenarios in the <em id="mrs_01_1157__en-us_topic_0000001219148949_i0360442111415">Loader client installation directory</em><strong id="mrs_01_1157__en-us_topic_0000001219148949_b340723115423">/loader-tools-1.99.3/loader-tool/job-config/</strong> directory. The parameters vary according to job templates. Job templates contain information about jobs and associated connectors.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a352a7dbc45b447ce8948f1d77a2c74b8">Job templates are XML files. The file name format is <em id="mrs_01_1157__en-us_topic_0000001219148949_i13395718172019">original data location</em><strong id="mrs_01_1157__en-us_topic_0000001219148949_b1924415919517">-to-</strong><em id="mrs_01_1157__en-us_topic_0000001219148949_i1525562014">new data location</em><strong id="mrs_01_1157__en-us_topic_0000001219148949_b17599445218">.xml</strong>, for example, <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_f5c039f8c63814f7bbae63bc1abb63cb7"><b>sftp-to-hdfs.xml</b></span>. If a job supports conversion step, a json conversion step configuration file with the same name exists, for example, <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_f4503170b5142435ea0562f109fd4b8a8"><b>sftp-to-hdfs.json</b></span>.</p>
|
|
<div class="note" id="mrs_01_1157__en-us_topic_0000001219148949_n150efc8e33da40d8bc87fa807d652130"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1157__en-us_topic_0000001219148949_a93e4e9a1d746445d8787f94f47453b3e">Job templates contain the configuration information of connectors. During the connector creation and updating, only the connector information in job templates is invoked.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1157__en-us_topic_0000001219148949_s8e030f32f945412b8ec4b6c84492648f"><h4 class="sectiontitle">Scenarios</h4><p id="mrs_01_1157__en-us_topic_0000001219148949_a75f2adc559aa402697e4026420d5ce67">The parameters vary according to connectors or jobs.</p>
|
|
<ul id="mrs_01_1157__en-us_topic_0000001219148949_u05df6775128a4894ac9ef8eaa7584b5d"><li id="mrs_01_1157__en-us_topic_0000001219148949_l417049c28e824640b009c141b452dfc8">To modify some parameters, use the parameter mode.</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l1f4fe82cdf054c4cadb8a30fc708bf6e">To create a connector or job, use the job template mode.<div class="note" id="mrs_01_1157__en-us_topic_0000001219148949_n88cf620d1c134c52a0d5bcea0b074054"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1157__en-us_topic_0000001219148949_af7ccd2a067264b099ba87ad0ac669fe6">This tool currently supports the FTP, HDFS, JDBC, MySQL, Oracle, and Oracle dedicated connectors. If other types of connectors are used, you are advised to use the open-source sqoop-shell tool.</p>
|
|
</div></div>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1157__en-us_topic_0000001219148949_s79ecfa8408cc4652ba043dc767fb3664"><h4 class="sectiontitle">Parameters</h4><p id="mrs_01_1157__en-us_topic_0000001219148949_ae7affcdb52d4448da54d5cca677f4c3e">Assume that the Loader client installation directory is <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_filepath57903902591843"><b>/opt/hadoopclient/Loader/</b></span>.</p>
|
|
<ul id="mrs_01_1157__en-us_topic_0000001219148949_u6114bbeec7394ffc8401b1f1287ca2fc"><li id="mrs_01_1157__en-us_topic_0000001219148949_lb879066400684ee2a8848c60c7f92659"><strong id="mrs_01_1157__en-us_topic_0000001219148949_a0e160086157146fa822a03960eaf4c6a">lt-ucc usage description</strong><p id="mrs_01_1157__en-us_topic_0000001219148949_acc455846d6e9486fac909955f4d87dc0">lt-ucc is a connector configuration tool of loader-tool user-configuration-connection and is used to create, update, and delete connectors.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1157__en-us_topic_0000001219148949_td2259c1e51c14dc4b28dfac4025d6709" frame="border" border="1" rules="all"><caption><b>Table 1 </b>lt-ucc script parameter description</caption><thead align="left"><tr id="mrs_01_1157__en-us_topic_0000001219148949_r98c164292c334e808623adf70e7f147e"><th align="left" class="cellrowborder" valign="top" width="15.790000000000001%" id="mcps1.3.3.3.1.3.2.4.1.1"><p id="mrs_01_1157__en-us_topic_0000001219148949_a1d19ce237c5c4bb09d2c037c699f1cee">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="54.89000000000001%" id="mcps1.3.3.3.1.3.2.4.1.2"><p id="mrs_01_1157__en-us_topic_0000001219148949_a98b7673922384ad590e70e7b7d40ea68">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="29.32%" id="mcps1.3.3.3.1.3.2.4.1.3"><p id="mrs_01_1157__en-us_topic_0000001219148949_a68612dbe84244a5bb4952b7b128ad645">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1157__en-us_topic_0000001219148949_r52eb51481b3c40cea08c3e96d7d7e0ac"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afebd2246c77b4b618c0dd3d2853e4c3e">-help</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a29d28f8d1fac4c7b8f0e139a22408b9c">Help information.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a13e859d068634402974b06dff3de2c01">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r32aca508930144cc92a32b696c3e928b"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0ffc98f1534a44f6afefcf46bf2ffd6a">-a <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a46e7fa10b5da47f5919419a8acda8b42">Connector action. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b248221971">create</strong>, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b1645229717">update</strong> and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b8511221575">delete</strong> for creating, updating, and deleting connectors respectively.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afe005cad87cf44d29ec40caa9161085e">create</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r81926ea483de4c198de44a4839e6c800"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a33171f319172450983a6f88981a2c3c6">-at <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a39faa98379554449bae7f50144e5a523">Login authentication type. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b48701026691843">kerberos</strong> and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b28339102191843">simple</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ad74b97cd662644b7a4a8587849f280cf">kerberos</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rcb9d1d8b27944b83a8c4e3ba1dc23f8b"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a63f76cdfadfd4a02832a1e20a07959aa">-uk <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a603ac2ca81574aba8aa0393ea69f3b78">Whether to use the keytab file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a2846e38d3cba4ba4be9913e17a2bf1fa">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rda4d874b4daa41a58a5311e9a2cd9feb"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_af365f762ac834706ab3643c71eb2e9cb">-au <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a049702a1f34544eb8b13ef6834bfde96">Login authentication username.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1fde91cb78014265804bcd20fceecaee">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r85d70ee059f8417bb5e068182f4715b1"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a9907dd5c0bfb4b7f93b0407b687bb49b">-ap <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aa86ecc9059b842fd91d694802aba8df4">Login authentication password. The value must be an encrypted password.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_ad4ec7cf17731480e991394a758d06f09">The password encryption method is described as follows:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a82b0e6c51b2b472ebcdf18655b70c4cd"><strong id="mrs_01_1157__en-us_topic_0000001219148949_b925038165214">sh </strong><i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_varname10251638105210">Loader client installation directory</span></i><strong id="mrs_01_1157__en-us_topic_0000001219148949_b102512387526">/Loader/loader-tools-1.99.3/encrypt_tool </strong><i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_varname126163815526">non-encrypted user password</span></i></p>
|
|
<div class="note" id="mrs_01_1157__en-us_topic_0000001219148949_note8349540161234"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1157__en-us_topic_0000001219148949_p8036997161234">If a non-encrypted password contains special characters, the special characters must be escaped. For example, the dollar sign ($) is a special character and can be escaped using single quotation marks ('), for example, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b1941594055516">'1q2w#e$r'</strong>. If a non-encrypted password contains single quotation marks, use double quotation marks to escape the single quotation marks. If a non-encrypted password contains double quotation marks, use backslashes (\) to escape the double quotation marks. For details, see the shell escape character rules.</p>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a106d6ffa733d4f4eb10abcb362692c9b">d2NjX2NyeXB0ATQxNDU1MzVGNDM0Mj</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r35f5c4ded97b4233adf78866588da987"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_af68864c6d7e646348750820ef0a8e6c5">-c <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a47a2e02b41c848d29269b5cf910ea1d9">Login authentication principal.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a87c3a459045a484ea2e37c379fef6df5">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r6e3b83bb9cfc4766800e2b9905c48b1d"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a7f18b43b32174dc6bc37bdbd1c689d1a">-k <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ab76769200c4a4964a9dea72ca325197e">Login authentication keytab file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a73ef739f54754bd59fb5ecc75a0fa3d1">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/hadoop-config/user.keytab</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r48693f0175204700aa114a2c0742cc2e"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a08150691e96d4422858723146845dc79">-h <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ab63ce0b173824d5d9b3599f2bd1eb54a">Specifies the configuration file path of the MRS cluster.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0b6aa701a5444d20a6e42ad8ab4e4709">-h /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/hadoop-config</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rd30df39369984e33a15531b2ddf5387f"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a7a9fe793898f4c2daef555b7a6823f5d">-l <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8b00ecf1299b4dcf924856f3f2a5bd96">Login template file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0fb007bd0d2f4642b7c0454fb20df1dd">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r7987eb44399e40ea8d7720df1db74068"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aebd923eb106944a3b12bde6565fd2592">-s <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a9ab5b8b0dc79408189285718bc59b47b">Floating IP address and port for Loader.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_aed2833039ed74564aa581996b1bea9c5">Format: <i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_v43f0764f23b64a5981922f4548747fed">floating IP address: port</span></i></p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_abd13f32af8b04b4a95dae74bce506478">The default port is <strong id="mrs_01_1157__en-us_topic_0000001219148949_b1985992913717">21351</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a027f8e78a8b84486a283fd1026a144e4">127.0.0.1:21351</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rae7ce82fa1bd4dfc84ae4cd9dd0b1d39"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ae30a75402b244b6087b029999a299e3d">-w <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a11d21d3647ca4d9183cf11729abb40b7">Job template file path for obtaining job details.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8a5264e1ed484e93af1e027b25aba89e">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r23c4b815a4864b2c877bb22f912fd68c"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a4005560d931840718dc03837d4d51c55">-z <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ae8feac474bcd453181bce11927431438">IP address and port number of ZooKeeper quorum instances. The format is <i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_vf300242e58f44c578c29c7ffa267e942">IP address: port</span></i>. Use <span class="parmvalue" id="mrs_01_1157__en-us_topic_0000001219148949_p3abe28e97ab448f4b821a6fb9a7b98ab"><b>commas (,)</b></span> to separate multiple addresses and port numbers.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a2901f6168480458bada0374b25200e80">127.0.0.0:2181, 127.0.0.1:2181</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_ref338dcb9de043ecb54b8980081ccf9e"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a5c4561742d444d6fa07c7f3dcad3d63f">-n <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_acd3298a1a0364fd08da6bf1937c199ff">Connector name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a015e178d4b6c4a00ab2af1931ba4539d">vt_sftp_test</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r98f4feb25d9d4f41aea70453a1c13f75"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a4fd729f4aea64f94ad2017e1360b8659">-t <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac00e14398d9945aa8865ac7d3fbe34b0">Connector type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aadfbc468aea34ec58ecf8648bd494948">sftp-connector</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rc1dd7d14953d43ac9c65499a81a7c31e"><td class="cellrowborder" valign="top" width="15.790000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a3afe95befec1401ab672569535742d4c">-P <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="54.89000000000001%" headers="mcps1.3.3.3.1.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a10903583571a463d92d435aa7bd467c1">Used to update the value of an attribute. The format is -Pparam1=value1. param1 indicates the attribute name of the connector in the job template. Password parameters are required for updating SFTP and FTP connector information.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a87f12617968f41aab1f7d0289225f2c4">-Pconnection.sftpPassword=<i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_v438c7203cc9e48cc809098d855b526cc">Encrypted password</span></i></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="29.32%" headers="mcps1.3.3.3.1.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_acd1e8119b5754964852e8d425cb4ebb4">-Pconnection.sftpServerIp=10.6.26.11</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_ada830f64f64d46cfbfeaeab0d756c191">A complete example is as follows:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a8b0c921efd2a4ccea1269a0edd7c676f"><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname65534289289">./bin/lt-ucc -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -n vt_sftp_test -t sftp-connector -Pconnection.sftpPassword=d2NjX2NyeXB0ATQxNDU1MzVGNDM0MjQzOzQ2MzEzMzQ0MzYzMjM1MzkzMjM2MzkzMTQxMzMzNjQ0NDE0MjMwMzgzMzQ1NDMzMzM3MzU0MTQ0MzUzMDM2Mzk7OzMyMzUzMDMwOzU3MUFDNTA4MEJENTUwNDNEQTc2Mjg3QkJERTc4NDI3OzQyODc0NTMwN0FEQzkzNjA7NTc0MzQzNUY0MzUyNTk1MDU0NUY0NDQ1NDY0MTU1NEM1NDVGNDQ0RjRENDE0OTRFOzMwOzMxMzQzNTM2MzMzMTMyMzgzMzMzMzIzNzMwOw -Pconnection.sftpServerIp=10.6.26.111 -a update</span></b></i></p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a88ce3acee5d54e2a8252a69cb50148a0">Configuration description of a lt-ucc script job template:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_ae69ca12dbc764140945866a693bb46be">Use the operation of saving SFTP data to HDFS as an example. Edit the <span class="filepath" id="mrs_01_1157__en-us_topic_0000001219148949_f3f87ff7a19b84c2186089b1ddc98f570"><b>sftp-to-hdfs.xml</b></span> file in <em id="mrs_01_1157__en-us_topic_0000001219148949_i1853410185715">Loader client installation directory</em><strong id="mrs_01_1157__en-us_topic_0000001219148949_b49412232572">/loader-tools-1.99.3/loader-tool/job-config/</strong> directory. The connector configuration is as follows:</p>
|
|
<pre class="screen" id="mrs_01_1157__en-us_topic_0000001219148949_s225faf94da1a47d5a5eface1ab040d45"><!-- Database connection information -->
|
|
<sqoop.connection name="<strong id="mrs_01_1157__en-us_topic_0000001219148949_a9f75a538c0be473baeee682afb0d38a3">vt_sftp_test</strong>" type="sftp-connector">
|
|
<connection.sftpServerIp>10.96.26.111</connection.sftpServerIp>
|
|
<connection.sftpServerPort>22</connection.sftpServerPort>
|
|
<connection.sftpUser>root</connection.sftpUser>
|
|
<connection.sftpPassword>d2NjX2NyeXB0ATQxNDU1MzVGNDM0MjQzOzMzMzkzOTMwMzI0NTM5MzQzOTM1Mzk0NTMwMzIzNTM4NDEzNzQ2MzIzNjQyMzMzMDM4MzMzNzQ1MzYzODQxMzQ7OzMyMzUzMDMwO0EzMTUzM0ExNTAyNDhENzE3QTRBRTlCQkRBQzlFRkFEOzYyOEE4NTlDODc2MkMyNzU7NTc0MzQzNUY0MzUyNTk1MDU0NUY0NDQ1NDY0MTU1NEM1NDVGNDQ0RjRENDE0OTRFOzMwOzMxMzQzNTM2MzMzMTMyMzgzMzMzMzIzNzMwOw</connection.sftpPassword>
|
|
</sqoop.connection></pre>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a6c32e744e1c54d3a8728dc495049cf81"></p>
|
|
<ul id="mrs_01_1157__en-us_topic_0000001219148949_ue48673902b674ec399c310f533820790"><li id="mrs_01_1157__en-us_topic_0000001219148949_lb9c8d112fafd49d7846d09ad9b5bf421">Creation command:<p id="mrs_01_1157__en-us_topic_0000001219148949_a8fb335106a704452afd127f8a496000b"><a name="mrs_01_1157__en-us_topic_0000001219148949_lb9c8d112fafd49d7846d09ad9b5bf421"></a><a name="en-us_topic_0000001219148949_lb9c8d112fafd49d7846d09ad9b5bf421"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname15183153611282">./lt-ucc -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -w /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/ftp-to-hdfs.xml -a create</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l16b51135d3a14082aab850dd2836f091">Update command:<p id="mrs_01_1157__en-us_topic_0000001219148949_ab653d2b3391d4ddda64059c47135a768"><a name="mrs_01_1157__en-us_topic_0000001219148949_l16b51135d3a14082aab850dd2836f091"></a><a name="en-us_topic_0000001219148949_l16b51135d3a14082aab850dd2836f091"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname1561749182817">./lt-ucc -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -w /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/ftp-to-hdfs.xml -a update</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_ld847520c8be34096a312c7406d0072df">Deletion command:<p id="mrs_01_1157__en-us_topic_0000001219148949_a757738678867428c817d20a5b2956394"><a name="mrs_01_1157__en-us_topic_0000001219148949_ld847520c8be34096a312c7406d0072df"></a><a name="en-us_topic_0000001219148949_ld847520c8be34096a312c7406d0072df"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname447895572812">./lt-ucc -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -w /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/ftp-to-hdfs.xml -a delete</span></b></i></p>
|
|
</li></ul>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_lba32d7fcd6284c5fa8546550400bd550"><strong id="mrs_01_1157__en-us_topic_0000001219148949_a29208ada0c3d4089a12703ed0566647f">lt-ucj usage description</strong><p id="mrs_01_1157__en-us_topic_0000001219148949_a074430fa47b24d308a186895cad40adf">lt-ucj is a job configuration tool of loader-tool user-configuration-job and is used to create, update, and delete jobs.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1157__en-us_topic_0000001219148949_tb9e1208879f44137b8705c0f7f389477" frame="border" border="1" rules="all"><caption><b>Table 2 </b>lt-ucj script parameter description</caption><thead align="left"><tr id="mrs_01_1157__en-us_topic_0000001219148949_re6a3bd45bbb94672a4bc3194ab431707"><th align="left" class="cellrowborder" valign="top" width="16.6%" id="mcps1.3.3.3.2.3.2.4.1.1"><p id="mrs_01_1157__en-us_topic_0000001219148949_a4d14b5b930924429b8565402a13479f4">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50.07%" id="mcps1.3.3.3.2.3.2.4.1.2"><p id="mrs_01_1157__en-us_topic_0000001219148949_a49b2eacdb0fc4091a5c1139967ac14ac">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33%" id="mcps1.3.3.3.2.3.2.4.1.3"><p id="mrs_01_1157__en-us_topic_0000001219148949_a32955735817741e392c4d2855085e899">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1157__en-us_topic_0000001219148949_r15642e40b9404623a4e5385ddd7371cc"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac4f74b506a2f4214ba379947bd6b8f67">-help</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_acf948f2c6b7d4342a78dd97c90b169a3">Help information.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afc0a6daabceb4221865f18c599d8b69d">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rf5561860abe3492e82bbcd42308bb5b4"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a5056f342bd6a49c4a14b3812ae5cb4a1">-a <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_af097834fef8740e9aa0c6112a1f85412">Job action. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b106762421012">create</strong>, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b15677646101">update</strong>, and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b8677204111015">delete</strong> for creating, updating and deleting jobs respectively.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac44917515a5646f2bc7810ccd43d0db1">create</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r8181939716ad4fe3bf513f303ff8d4f3"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a5fee23fa483d409a9bb5a2f5b62d519c">-at <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_abe511429177a4f7e8d7a6c26e89505f7">Login authentication type. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b1031297906">kerberos</strong> and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b1419770453">simple</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1810212a328947c496bfa857133c40e7">kerberos</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r3f79cbb1d1a94237b7ee80365f087588"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ae4a0bc9958774145a6fe50ab5fa4142c">-uk <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ab71267b62c8c48e1a9abe2a845a0c6dd">Whether to use the keytab file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1ed397b1dd1945d7b68f75f0702ddb81">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r7e3d4e325788447faf021813328330a8"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ab403538a46ef4f26b5abeb3a20cbcaab">-au <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a88b9e98c3d7e434487aa8b5d3f79aff7">Login authentication username.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0accf674b4a6442da1349ab03100ca5c">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r290b005346fa4c60bf2174a1f1974f2e"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a7a7d1a4abdaa4400a4af153804ec36f5">-ap <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ad006595c5fdc4176902a2b6a2a713160">Login authentication password. The value must be an encrypted password.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a03311f14aac54fe992360105fc43d6c3">The password encryption method is described as follows:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a9fdd00d17b8a4bab891dc15c31d879e7"><strong id="mrs_01_1157__en-us_topic_0000001219148949_b981031711113">sh </strong><i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_varname1181791710113">Loader client installation directory</span></i><strong id="mrs_01_1157__en-us_topic_0000001219148949_b381761713115">/Loader/loader-tools-1.99.3/encrypt_tool </strong><i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_varname1681713171713">non-encrypted user password</span></i></p>
|
|
<div class="note" id="mrs_01_1157__en-us_topic_0000001219148949_note1890410416511"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1157__en-us_topic_0000001219148949_p3591921416511">If a non-encrypted password contains special characters, the special characters must be escaped. For example, the dollar sign ($) is a special character and can be escaped using single quotation marks ('), for example, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b109871447621">'1q2w#e$r'</strong>. If a non-encrypted password contains single quotation marks, use double quotation marks to escape the single quotation marks. If a non-encrypted password contains double quotation marks, use backslashes (\) to escape the double quotation marks. For details, see the shell escape character rules.</p>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a427478ff99f44257a1adf2730dcfe91e">d2NjX2NyeXB0ATQxNDU1MzVGNDM0Mj</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rac2bd101d6664004a827ee67cf727c2c"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a42fa0e00c95540d88dfa7bdc9b0e24a5">-c <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a675316b380084171944ec72a5c0f35cd">Login authentication principal.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a50931de0ea924a819b458a059ff0a1e3">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rf4cf88835da346009c127abbc376a9b3"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_adb1017ea10e7457dad417d59ce79ca34">-k <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a139a8c022b314134ae4a634aecd056a7">Login authentication keytab file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a34828216edcb416f90101d9715af7687">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/hadoop-config/user.keytab</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_re67ed0d99e584fb89522f657652a24cb"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a67f233778f394b82b415db5ebeaaefc8">-h <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0ec10735ea644dcda1a7fee1c3acf448">Specifies the configuration file path of the MRS cluster.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ab52d4cb1ad314245b721cafde3020058">-h /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/hadoop-config</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rc65e10b85a0f4f009c31283f4857f877"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a2df25d3dc1744ca59f0cd74667df7ef2">-l <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_abcc999c37edc4dcd9707807f3432f0e5">Login template file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a98f1a061015748739f3b2e701c64c863">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rce0bbcf8c1e24a0f9ff5ac34bf030dc2"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a17506eb71bbc46f8ac6e216d59f57425">-s <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a46ef3a24c6e54ed5b89ba484fe35cee8">Floating IP address and port for Loader.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_af44262a23de44970a9ae8ec097c8b182">Format: <i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_vc692093d645549a0859839a9df60f506">floating IP address: port</span></i></p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a5a282b796ddb4f27a88fb3c3faa0d635">The default port is 21351.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a5745a8d52db54a1bbfebf940fd8f1231">127.0.0.1:21351</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r10d43da286274c7bb9663a0813f6a4d0"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a907bd6ddc1c64d6aa7ddbc0ed27688db">-w <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aed71a8b8227647eaaf5e6b13aed03f52">Job template file for obtaining job details.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afef79e28aaca471fa2620ef416326819">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rb6748b638e4f4ce4b32c5539de390500"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a00ecd8c70d7e49ed9eba3f43742737ca">-z <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8512722fe9b247e18519614c3e170ae5">IP address and port number of ZooKeeper quorum instances. The format is <i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_v4553a7d8e3c34cdbbb1861ebe9ab1c5f">IP address: port</span></i>. Use <span class="parmvalue" id="mrs_01_1157__en-us_topic_0000001219148949_pe742dc42d57846a7b81690b88234e386"><b>commas (,)</b></span> to separate multiple addresses and port numbers.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a7785562a66fe4da59103992e9e6629b4">127.0.0.0:2181, 127.0.0.1:2181</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r0780752d38764a4098be28083dca2b7e"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a2c80f0e6a3ac4e06a4b45283f184abc7">-n <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a252e5c83599745fba89347f67a72830a">Name of the job.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a6a056897511742ff92000f1cf4f73f50">Sftp.to.Hdfs</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r110a435c80ea4baf9a9f13dd48e7f41a"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac54365d2d78f4e9f91b5a88071ce3d7f">-cn <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1b9c19e9796e488cb71269971fe3505c">Connector name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a526f4b26c4c246129844ece43e0db00b">vt_sftp_test</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r82e663c945354d96bc5cc2b4e83603c2"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ae8d938618e3a4a3f8ee1285441f4f0d8">-ct <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a66840e6092d54dddb65fdd49c8e76ac2">Connector type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a402da7cab6e441019f56c48455530d54">sftp-connector</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_re509634de7d0439f81fd8a50585ca4e4"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a28a5557f1bb64843b6cfa3912e4440f4">-t <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aa1568479b4e645f8b3f641f3951cdb97">Job type. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b9324153941016">IMPORT</strong> and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b1432910395109">EXPORT</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aa519521b78774111afde2960ba5c93cb">IMPORT</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r98017566299c46c899d732e94566e2d9"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a742685feefb942b3b5c2225ef043b041">-trans <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ae82efdece9664dd2954556933b444e62">Job associated conversion step file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0560b7cd4eef4dbf931bef96cde8da3d">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.json</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rf5af237cf45345a7abfea817f19408b7"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afeeab8b0671c4d3c8d31e2abd1c57f96">-priority <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aab483546604249a08cce6b4b9dc3989d">Job priority. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b107576588891843">LOW</strong>, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b211188318891843">NORMAL</strong>, and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b140497372091843">HIGH</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1a848088cf504c66882238294fd192a2">NORMAL</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r2778ddb65fb34abaa29e855d15c19021"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a67a946c6e55c4636a5e0ea3a91ab14b3">-queue <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aeca19d04884c436c8b78219c334ac823">Queues</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8c437ef0a85342b1b9f10f6a18f83629">default</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_row1297266916236"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_p4964516216236">-storageType <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_p6183522016236">Storage type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_p4259695116236">HDFS</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r8e69c46004b34033a77423bf40b83301"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.3.3.2.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aef7ee03cbc804a8996b5d471f596b34d">-P <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50.07%" headers="mcps1.3.3.3.2.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a191f0aaabacf4b8897ec632896fee453">Used to update the value of an attribute. The format is -Pparam1=value1. param1 indicates the attribute name of the connector in the job template. Password parameters are required for updating SFTP and FTP connector information.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_acb51b88d389941bf9349173624e03492">-Pconnection.sftpPassword=<i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_v974987adca9a4b0bbacdd28417a60c8d">Encrypted password</span></i></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.2.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a5ef8e6a4ce95414f8431fa1f4ff36ba4">-Pconnection.sftpServerIp=10.6.26.11</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a92bd5fec8356462f97d69cb1f413739e">A complete example is as follows:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a1d5ff95444bb4ba4b2ccacd5532422c4"><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname1354021812516">./bin/lt-ucj -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -n Sftp.to.Hdfs -t IMPORT -ct sftp-connector -Poutput.outputDirectory=/user/loader/sftp-to-hdfs-test8888 -a update</span></b></i></p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_aec56e102901540ea85f4d28c467a4ddf">Configuration description of a lt-ucj script job template:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a0cb2998ecce44c498e0504a3ba88898b">Use the operation of saving SFTP data to HDFS as an example. Edit the file <em id="mrs_01_1157__en-us_topic_0000001219148949_i19317182117223">loader client installation directory</em><strong id="mrs_01_1157__en-us_topic_0000001219148949_b1014931210418">/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml</strong>. The job configuration is as follows:</p>
|
|
<pre class="screen" id="mrs_01_1157__en-us_topic_0000001219148949_sf7ad11b2aab0499ebd45d7a4bf93018d"><!--Job name, globally unique.-->
|
|
<sqoop.job name="Sftp.to.Hdfs" type="IMPORT" queue="default" priority=" Priority NORMAL ">
|
|
|
|
<!-- External data source parameter configuration -->
|
|
<data.source connectionName="<strong id="mrs_01_1157__en-us_topic_0000001219148949_a5cc43443bff943199c738d7e3947c976">vt_sftp_test</strong>" connectionType="sftp-connector">
|
|
<file.inputPath>/opt/houjt/hive/all</file.inputPath>
|
|
<file.splitType>FILE</file.splitType>
|
|
<file.filterType>WILDCARD</file.filterType>
|
|
<file.pathFilter>*</file.pathFilter>
|
|
<file.fileFilter>*</file.fileFilter>
|
|
<file.encodeType>GBK</file.encodeType>
|
|
<file.suffixName></file.suffixName>
|
|
<file.isCompressive>FALSE</file.isCompressive>
|
|
</data.source>
|
|
|
|
<!-- MRS cluster, parameter configuration -->
|
|
<hadoop.source storageType="HDFS" >
|
|
<output.outputDirectory>/user/loader/sftp-to-hdfs</output.outputDirectory>
|
|
<output.fileOprType>OVERRIDE</output.fileOprType>
|
|
<throttling.extractors>3</throttling.extractors>
|
|
<output.fileType>TEXT_FILE</output.fileType>
|
|
</hadoop.source>
|
|
|
|
<!-- Job associated conversion step file -->
|
|
<sqoop.job.trans.file>/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.json</sqoop.job.trans.file>
|
|
</sqoop.job></pre>
|
|
<ul id="mrs_01_1157__en-us_topic_0000001219148949_u3827153dec8c4b2d971d6d73189663f8"><li id="mrs_01_1157__en-us_topic_0000001219148949_lb7b8cd3baa5643dea0039d3383fe9959">Creation command:<p id="mrs_01_1157__en-us_topic_0000001219148949_a7e2aaf4d6d5546ec85ef5b6a4570b674"><a name="mrs_01_1157__en-us_topic_0000001219148949_lb7b8cd3baa5643dea0039d3383fe9959"></a><a name="en-us_topic_0000001219148949_lb7b8cd3baa5643dea0039d3383fe9959"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_c035a98f9fd424b6a8100ce8e6f863242_1">./bin/lt-ucj -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -w /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml -a create</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l6f45ca405e824394a1c3a03ca3fce943">Update command:<p id="mrs_01_1157__en-us_topic_0000001219148949_a7e95ad2c840244628b4d789530e1d0e4"><a name="mrs_01_1157__en-us_topic_0000001219148949_l6f45ca405e824394a1c3a03ca3fce943"></a><a name="en-us_topic_0000001219148949_l6f45ca405e824394a1c3a03ca3fce943"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname43036518290">./bin/lt-ucj -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -w /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml -a update</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l45700e2163f7479fbc4c126438e2fc0c">Deletion command:<p id="mrs_01_1157__en-us_topic_0000001219148949_ac7194831bb6043328d13d0767b56ba06"><a name="mrs_01_1157__en-us_topic_0000001219148949_l45700e2163f7479fbc4c126438e2fc0c"></a><a name="en-us_topic_0000001219148949_l45700e2163f7479fbc4c126438e2fc0c"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname4503161412294">./bin/lt-ucj -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -w /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml -a delete</span></b></i></p>
|
|
</li></ul>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_lf1d4be91406948d6a373e39a9edcc10b"><strong id="mrs_01_1157__en-us_topic_0000001219148949_a05387e11915f4400a8e950e5ee19864f">lt-ctl usage description</strong><p id="mrs_01_1157__en-us_topic_0000001219148949_a95460332dcae400d83deafe2879769f2">lt-ctl is a job management tool of loader-tool controller and is used to start or stop jobs, query job status and progress, and check whether jobs are running.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1157__en-us_topic_0000001219148949_t2ecc0fa55b2147e696e5609ca5594f12" frame="border" border="1" rules="all"><caption><b>Table 3 </b>lt-ctl script parameter description</caption><thead align="left"><tr id="mrs_01_1157__en-us_topic_0000001219148949_r3aff6e21f914414091afadd151551a5f"><th align="left" class="cellrowborder" valign="top" width="15.290000000000001%" id="mcps1.3.3.3.3.3.2.4.1.1"><p id="mrs_01_1157__en-us_topic_0000001219148949_a7fdf8e55ee284c248747e565457c9069">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="51.38%" id="mcps1.3.3.3.3.3.2.4.1.2"><p id="mrs_01_1157__en-us_topic_0000001219148949_afb27c6de61754791a2ba950f24afa21a">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33%" id="mcps1.3.3.3.3.3.2.4.1.3"><p id="mrs_01_1157__en-us_topic_0000001219148949_afce7755fb2c14abfb47d9e11ed33c3c2">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1157__en-us_topic_0000001219148949_r568d8f46ea024ac586005ba74932a6d0"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afd9d0e6ba09d46d59af47c93bd50b9f0">-help</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ade59618f3c54463a80db47bc596ea86f">Help information.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a6824d8cdca6745ebb9ddb513fe680eae">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rac8a66d6a7c347269e749ad4322d7a09"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a720faddb324c4e919edc96af49b58424">-a <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a427d8b8cc18a4f3bbcc39e22204b6d8b">Job action. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b128549099391843">status</strong>, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b158089780691843">start</strong>, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b37039155991843">stop</strong>, and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b174172569591843">is running</strong> for querying job status, starting or stopping jobs, and checking whether jobs are running.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a36dca316c44f46ea935ed1c88207d29f">create</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r5691b42261e7457b88a60f55facf1a4f"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ae857fe1c81fa42609f468124a2c6e8b7">-at <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aed93199629224eebac7fc39459f71b40">Login authentication type. The values include <strong id="mrs_01_1157__en-us_topic_0000001219148949_b616667518">kerberos</strong> and <strong id="mrs_01_1157__en-us_topic_0000001219148949_b351966027">simple</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a6bb38653b85a4f15b713d0eb544bd422">kerberos</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_ra56896748053481680f6b0f64e8b8863"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac93c8ce609ac4b85b6155143b04b5f21">-uk <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a461715fe13c441c2a98c71aca6008607">Whether to use the keytab file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_af36fd384b7a94239994ec39657d1d9f2">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r1671fb94d92e45e3b2b88f9a2d8ae536"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1a345fb996fb4b67a583925600596df5">-au <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8b6d1f37e7994b5b9499aad2e7ee66fa">Login authentication username.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afaf6823c8af444c0bcea8aa5a372a7b5">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rca75ecd5147e486cafce59ae8c66c857"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a40debe9dd7eb46d1b5829fd9811626a9">-ap <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_af1c011b639cb47eb84829db0eaab8cf8">Login authentication password. The value must be an encrypted password.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a7c9e67f8ffe44114a6aee47010bad8b4">The password encryption method is described as follows:</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_adba7ae8c5bc24a41a090ae9d3e512f60"><strong id="mrs_01_1157__en-us_topic_0000001219148949_b1089132619418">sh </strong><i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_varname08919269413">Loader client installation directory</span></i><strong id="mrs_01_1157__en-us_topic_0000001219148949_b9909261940">/Loader/loader-tools-1.99.3/encrypt_tool </strong><i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_varname2905265419">non-encrypted user password</span></i></p>
|
|
<div class="note" id="mrs_01_1157__en-us_topic_0000001219148949_note9008118165114"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1157__en-us_topic_0000001219148949_p13964205165114">If a non-encrypted password contains special characters, the special characters must be escaped. For example, the dollar sign ($) is a special character and can be escaped using single quotation marks ('), for example, <strong id="mrs_01_1157__en-us_topic_0000001219148949_b093819283416">'1q2w#e$r'</strong>. If a non-encrypted password contains single quotation marks, use double quotation marks to escape the single quotation marks. If a non-encrypted password contains double quotation marks, use backslashes (\) to escape the double quotation marks. For details, see the shell escape character rules.</p>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a0e38515330c942deaf9b66d9f5102923">d2NjX2NyeXB0ATQxNDU1MzVGNDM0Mj</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_ra9be32f5f8dc45b4bebe250d37e2c166"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a65f4daab22e44c42b545d9f8b8519ba0">-c <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aabed52a9d54647d79fc2525c411ebfea">Login authentication principal.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_acb7631a76af046bba35f399aa06457bc">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rcd321b721caa483a89ba05e93b2227d2"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ad9b5c90fba2843088b27c48021cabc84">-k <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a899a628d6cee4eabbfb34da07b974938">Login authentication keytab file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac75bd316b9a24851912d2f0280031f33">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/hadoop-config/user.keytab</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rd5a8b02c98a1495892949dda617b03f0"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a52a401b0095f4c7ca2e9e774a9b36eb6">-h <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_af8f8e41ad12a4722adb863f4ee07814b">Specifies the configuration file path of the MRS cluster.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a15d480438e9d4709b4355ecbf3564720">-h /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/hadoop-config</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_rf2d3314c1b9042a6bcab1065876f9078"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_aef2ff2a5d3e540a4b1125e529268bcd9">-l <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a1d55e5ca4ea247f9a944396b6b37f624">Login template file.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a9f5dcb286a6e4827ae1260a67c3c1691">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_refc2fd85c8aa4a7ea42a714a5df83e09"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_afcd8cd872a66429ea714de90da61ea25">-n <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8cbd529704074342bcc084ac734161fd">Name of the job.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ad30545c400a947f1872618482aa3aafc">Sftp.to.Hdfs</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r8b4caf0da4324d52945a63f76a2b29e9"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ab2ca98a051ae41c582f185efe3d3b268">-s <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_ac54875d2fda444c094c4016fb50a031e">Floating IP address and port for Loader.</p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_aaeeac5ca6e604023a6301b86a42f67ad">Format: <i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_vf7565acd8d58496691c36ac1e3e6e06c">floating IP address: port</span></i></p>
|
|
<p id="mrs_01_1157__en-us_topic_0000001219148949_a434c367898de4e88a09369c9066d27f8">The default port is 21351.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a8897d4cdbbe54fd0a4cf5b902d1b1917">127.0.0.1:21351</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r0711f1d283a64c56a216ee33a9839e6e"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_abd83ef440f2049f9b664d0784250c48e">-w <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a3fa2b9f81d1b456fb789d698592e3d6b">Job template file for obtaining job details.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a799187d2f58a4a51a78e498f19d2231a">/opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/sftp-to-hdfs.xml</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1157__en-us_topic_0000001219148949_r7de76d1044b74540b1e743855400224b"><td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.3.3.3.3.2.4.1.1 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a709739366d744ea5ad00b21c79ff8c0d">-z <arg></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="51.38%" headers="mcps1.3.3.3.3.3.2.4.1.2 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a2a47084dfc114fdfb4a062631e332e56">IP address and port number of ZooKeeper quorum instances. The format is <i><span class="varname" id="mrs_01_1157__en-us_topic_0000001219148949_v85e02b12c1f04f7a87ee27540d4dafb8">IP address: port</span></i>. Use <span class="parmvalue" id="mrs_01_1157__en-us_topic_0000001219148949_pd6af321ec2a1479bac24064c748fa604"><b>commas (,)</b></span> to separate multiple addresses and port numbers.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33%" headers="mcps1.3.3.3.3.3.2.4.1.3 "><p id="mrs_01_1157__en-us_topic_0000001219148949_a22918b0a62bf4478869b9ef36669f519">127.0.0.0:2181, 127.0.0.1:2181</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<ul id="mrs_01_1157__en-us_topic_0000001219148949_ucc5b5f4637024c1d9958aac85cc64506"><li id="mrs_01_1157__en-us_topic_0000001219148949_l9d2d8fad04724fd29c6e59f409600131">Command for starting jobs:<p id="mrs_01_1157__en-us_topic_0000001219148949_a239f9fa985c24d7b8bf0f1f5c5cf1f4e"><a name="mrs_01_1157__en-us_topic_0000001219148949_l9d2d8fad04724fd29c6e59f409600131"></a><a name="en-us_topic_0000001219148949_l9d2d8fad04724fd29c6e59f409600131"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname6782192511292">./bin/lt-ctl -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -n Sftp.to.Hdfs -a start</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_lc4f9d4ebc4d045f3af8aae46d8ce495f">Command for viewing job status:<p id="mrs_01_1157__en-us_topic_0000001219148949_a422007e911584f24a27517dc56e54bbb"><a name="mrs_01_1157__en-us_topic_0000001219148949_lc4f9d4ebc4d045f3af8aae46d8ce495f"></a><a name="en-us_topic_0000001219148949_lc4f9d4ebc4d045f3af8aae46d8ce495f"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname151511532152914">./bin/lt-ctl -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -n Sftp.to.Hdfs -a status</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l65bc53599cdb44089435260a6c5cf0ca">Command for checking whether jobs are running:<p id="mrs_01_1157__en-us_topic_0000001219148949_a0738233a385347d4a5949eb940ee1505"><a name="mrs_01_1157__en-us_topic_0000001219148949_l65bc53599cdb44089435260a6c5cf0ca"></a><a name="en-us_topic_0000001219148949_l65bc53599cdb44089435260a6c5cf0ca"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname172471836162913">./bin/lt-ctl -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -n Sftp.to.Hdfs -a isrunning</span></b></i></p>
|
|
</li><li id="mrs_01_1157__en-us_topic_0000001219148949_l17e27ffdeb164c7292995bd5d9b65222">Command for stopping jobs:<p id="mrs_01_1157__en-us_topic_0000001219148949_abf0fba7d8d8b43d4b656e41e28ae645e"><a name="mrs_01_1157__en-us_topic_0000001219148949_l17e27ffdeb164c7292995bd5d9b65222"></a><a name="en-us_topic_0000001219148949_l17e27ffdeb164c7292995bd5d9b65222"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1157__en-us_topic_0000001219148949_cmdname5454164711295">./bin/lt-ctl -l /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/login-info.xml -n Sftp.to.Hdfs -a stop</span></b></i></p>
|
|
</li></ul>
|
|
</li></ul>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1155.html">Client Tool Description</a></div>
|
|
</div>
|
|
</div>
|
|
|