vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_1105.html
Yang, Tong 3f5759eed2 MRS comp-lts 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2023-01-19 17:08:45 +00:00

169 lines
30 KiB
HTML
Raw Blame History

<a name="mrs_01_1105"></a><a name="mrs_01_1105"></a>
<h1 class="topictitle1">Typical Scenario: Exporting Data from Hive to an SFTP Server</h1>
<div id="body8662426"><div class="section" id="mrs_01_1105__en-us_topic_0000001219230969_s16d318ba998148cab6c5ed4b0091fa15"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1105__en-us_topic_0000001219230969_adc09c5bb1f5b469092c8702a82176404">Use Loader to export data from Hive to an SFTP server.</p>
</div>
<div class="section" id="mrs_01_1105__en-us_topic_0000001219230969_sc8b54ec480b048fabde9633a108696c2"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_1105__en-us_topic_0000001219230969_u0a5e603443cd41308bfcbd91d5c5ec7d"><li id="mrs_01_1105__en-us_topic_0000001219230969_l9e85f15735a24cb9bbce5acea1496c0e">You have obtained the service username and password for creating a Loader job.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_ldfe81b14d21b4899aac93fbf9e0c4bfd">You have had the permission to access the Hive table specified in the job.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_lacd303ac531c42da94a316d718ad9968">You have obtained the username and password of the SFTP server and the user has the write permission of the data export directory on the SFTP server.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_ldfeab582546e4c6cae6530bc60e19fdf">No disk space alarm is reported, and the available disk space is sufficient for importing and exporting data.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l220e4fb703f44cdbb641a7e85bd86c7b">If a configured task requires the Yarn queue function, the user must be authorized with related Yarn queue permission.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l2184446e85b348eb8dc29fcea2f95816">The user who configures a task must obtain execution permission on the task and obtain usage permission on the related connection of the task.</li></ul>
</div>
<div class="section" id="mrs_01_1105__en-us_topic_0000001219230969_s6485886206084356bc2fb55c0beb2c2b"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="mrs_01_1105__en-us_topic_0000001219230969_a335c5eca2f454e8b91759525cd55d01d"><strong id="mrs_01_1105__en-us_topic_0000001219230969_a359828fb050545158738ef2d34a10f97">Setting Basic Job Information</strong></p>
<ol id="mrs_01_1105__en-us_topic_0000001219230969_ofc32af95aa07404d9250d334f032f3b0"><li id="mrs_01_1105__en-us_topic_0000001219230969_lfcdaf9f05e4149858f55d6b4dc2965da"><span>Access the Loader web UI.</span><p><ol type="a" id="mrs_01_1105__en-us_topic_0000001219230551_obbc8c37dc53040efb21dc541b1dfc22c"><li id="mrs_01_1105__en-us_topic_0000001219230551_l6f0ffda40ca543d5a5660461b5e311dd">Log in to FusionInsight Manager. For details, see <a href="mrs_01_2124.html">Accessing FusionInsight Manager</a>.</li><li id="mrs_01_1105__en-us_topic_0000001219230551_la11b479a24ad4659a55365c2ede06015">Choose <strong id="mrs_01_1105__en-us_topic_0000001219230551_b39357591297">Cluster</strong> &gt; <em id="mrs_01_1105__en-us_topic_0000001219230551_i18941259112918">Name of the desired cluster</em> &gt; <strong id="mrs_01_1105__en-us_topic_0000001219230551_b19941115916297">Services</strong> &gt; <strong id="mrs_01_1105__en-us_topic_0000001219230551_b1894175916294">Loader</strong>.</li><li id="mrs_01_1105__en-us_topic_0000001219230551_l6601e8a2fd1b4780bf69238d6f5cc7f2">Click <strong id="mrs_01_1105__en-us_topic_0000001219230551_b13499742135213">LoaderServer(</strong><em id="mrs_01_1105__en-us_topic_0000001219230551_i187499244301">Node name</em><strong id="mrs_01_1105__en-us_topic_0000001219230551_b18777163515522">, Active)</strong>. The Loader web UI is displayed.<div class="fignone" id="mrs_01_1105__fig10607922203218"><span class="figcap"><b>Figure 1 </b>Loader web UI</span><br><span><img id="mrs_01_1105__mrs_01_1089_image155554364292" src="en-us_image_0000001438241209.png"></span></div>
</li></ol>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_lf1bbce467863405f99abbc8a5daa4f4f"><span>Click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_ufba563fbedb84911b7c3d281bd9b2c6e"><b>New Job</b></span> to go to the <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_p20e4e6182d2544639d45d4974f5843aa"><b>Basic Information</b></span> page and set basic job information.</span><p><div class="fignone" id="mrs_01_1105__en-us_topic_0000001219230969_f1575d09f7dfb4f2c864bfe604d183333"><span class="figcap"><b>Figure 2 </b><strong id="mrs_01_1105__en-us_topic_0000001219230969_b9918283719199">Basic Information</strong></span><br><span><img id="mrs_01_1105__en-us_topic_0000001219230969_i7e745cb75ea14c0eb09bd7442e6eaf7d" src="en-us_image_0000001349139405.png"></span></div>
<ol class="subitemlist" type="a" id="mrs_01_1105__en-us_topic_0000001219230969_o6bcf34fb3a184153b2b728683b889838"><li id="mrs_01_1105__en-us_topic_0000001219230969_l2c20daf4cb964e77a293c06e46ea06c3">Set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_p86b4eefabcf744c89cc4a9baa2a160c5"><b>Name</b></span> to the name of the job.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_lbce896fbb3ae42b893a743756f9e0ae6">Set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_p84bf35ed23cd4cab8d31afe4daf73cf8"><b>Type</b></span> to <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_pfce35e425edc47f59d93e1f4279fccbb"><b>Export</b></span>.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_laa97c4f541c94873ace651c474b8f2bd">Set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname10945996839199"><b>Group</b></span> to the group to which the job belongs. No group is created by default. You need to click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol12122314439199"><b>Add</b></span> to create a group and click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol10753219639199"><b>OK</b></span> to save the created group.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l2413b105414640dbbb71147cd705be4b">Set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_p1c532ed067234fc7b3be8666509cefe0"><b>Queue</b></span> to the Yarn queue that executes the job. The default value is <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue411751143910"><b>root.default</b></span>.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_lfbfacbf5dd7344c882c6ad9ebd23137f">Set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_pddeded75c85043a8bcf03f468483a696"><b>Priority</b></span> to the priority of the Yarn queue that executes the job. The default value is <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_p99dbf01c013d45ebaab8dabc34f7b695"><b>NORMAL</b></span>. The options are <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue1948669799199"><b>VERY_LOW</b></span>, <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue2168583169199"><b>LOW</b></span>, <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue21353777979199"><b>NORMAL</b></span>, <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue12415413439199"><b>HIGH</b></span>, and <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue7133554679199"><b>VERY_HIGH</b></span>.</li></ol>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_lbcf1e8880e9d4b88b79c0762bb4b7459"><span>In the <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname2957882479199"><b>Connection</b></span> area, click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol15668390659199"><b>Add</b></span> to create a connection, set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname16711552829199"><b>Connector</b></span> to <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue1067231409199"><b>sftp-connector</b></span>, click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol4960273269199"><b>Add</b></span>, set connection parameters, and click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol16107358209199"><b>Test</b></span> to verify whether the connection is available. When "<span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue15658710409199"><b>Test Success</b></span>" is displayed, click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol9452220099199"><b>OK</b></span>. Loader allows multiple SFTP servers to be configured. Click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_u76f9774b17834c71b1cde432a04e4043"><b>Add</b></span> to add the configuration information of multiple SFTP servers.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1105__en-us_topic_0000001219230969_t196b9c788abc4bde892c5a8a1d49c4e5" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Connection parameters</caption><thead align="left"><tr id="mrs_01_1105__en-us_topic_0000001219230969_r15ea1323d3fd485d9047304df476901c"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.3.3.2.1.2.4.1.1"><p id="mrs_01_1105__en-us_topic_0000001219230969_ae6f1ba81a9a04a34a9a148eaa2280e49">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="53.949999999999996%" id="mcps1.3.3.3.3.2.1.2.4.1.2"><p id="mrs_01_1105__en-us_topic_0000001219230969_a1fe7bd8c74ac4673b9b018d4e3657241">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="21.05%" id="mcps1.3.3.3.3.2.1.2.4.1.3"><p id="mrs_01_1105__en-us_topic_0000001219230969_acb4e318e251341b4af9776ca3d83d00f">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1105__en-us_topic_0000001219230969_r56416b6c613a4a6db836d8eadd232278"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a5a62d0d5238e43e7904a9ff861992cf8"><strong id="mrs_01_1105__en-us_topic_0000001219230969_b6000651009199">Name</strong></p>
</td>
<td class="cellrowborder" valign="top" width="53.949999999999996%" headers="mcps1.3.3.3.3.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a295a08b723bc46798ce16b221111d6e8">Specifies the name of the SFTP server connection.</p>
</td>
<td class="cellrowborder" valign="top" width="21.05%" headers="mcps1.3.3.3.3.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a3387b670ec254316bb5dc45059003370">sftpName</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_r20e3290d1de048e2a4917b4837e7ba25"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a01713d9e2a484001bc74173fd59725ce">SFTP server IP</p>
</td>
<td class="cellrowborder" valign="top" width="53.949999999999996%" headers="mcps1.3.3.3.3.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ae75ab679273d4c9ab4e6d5c8f82e8f32">Specifies the IP address of the SFTP server.</p>
</td>
<td class="cellrowborder" valign="top" width="21.05%" headers="mcps1.3.3.3.3.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a0355c43a0da04064b7c1b78e779b9a41">10.16.0.1</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_rf0294988d871447eb960f08c6c477118"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ac666d955295245eeb639af6c17e8543b">SFTP server port</p>
</td>
<td class="cellrowborder" valign="top" width="53.949999999999996%" headers="mcps1.3.3.3.3.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ac30cb65c5e9e41f69cca7fddc56b8d93">Specifies the port number of the SFTP server.</p>
</td>
<td class="cellrowborder" valign="top" width="21.05%" headers="mcps1.3.3.3.3.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a73ab7512193945b889f587609c06bce4">22</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_r397ad29b56084592a7ce06900909d5c7"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_af30fa63144664e37aa794d3c7ab623df">SFTP username</p>
</td>
<td class="cellrowborder" valign="top" width="53.949999999999996%" headers="mcps1.3.3.3.3.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a34b36d800b444a37ba55d91f916e34f8">Specifies the user name for accessing the SFTP server.</p>
</td>
<td class="cellrowborder" valign="top" width="21.05%" headers="mcps1.3.3.3.3.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a46c2b05ddf834424a60dfcef64663aaf">root</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_r394a0e95b7f64234b8ccac025e0be00b"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_aa8c81a15e84647f79f736f5b1d2739ff">SFTP password</p>
</td>
<td class="cellrowborder" valign="top" width="53.949999999999996%" headers="mcps1.3.3.3.3.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a23c448bc92fe495a86dad957a968235c">Specifies the password for accessing the SFTP server.</p>
</td>
<td class="cellrowborder" valign="top" width="21.05%" headers="mcps1.3.3.3.3.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a1947035b19d74d08a584f915225caaab">xxxx</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_r111f5039c2d84cb2a83622969bd89640"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_acfa7e5b872954a209c7de976f87efc8e">SFTP public key</p>
</td>
<td class="cellrowborder" valign="top" width="53.949999999999996%" headers="mcps1.3.3.3.3.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_aaa572535c98649f4bb221c5511336f2b">Specifies public key of the SFTP server.</p>
</td>
<td class="cellrowborder" valign="top" width="21.05%" headers="mcps1.3.3.3.3.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a6ce9925c19f84b779535395a219faa48">OdDt/yn...etM</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="note" id="mrs_01_1105__en-us_topic_0000001219230969_nb660cd91aa1d4b7799c5904dd068489b"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p class="text" id="mrs_01_1105__en-us_topic_0000001219230969_a85d5695f594e4f93b9ddf168e7b27407">When multiple SFTP servers are configured, the data of Hive tables will be divided into multiple parts and saved to the SFTP servers randomly.</p>
</div></div>
<p class="tableheading" id="mrs_01_1105__en-us_topic_0000001219230969_a17379c5fd46a4e59983527b8bad677cc"><strong id="mrs_01_1105__en-us_topic_0000001219230969_a691426c351a44b3d98f14ad25f74e70b">Setting Data Source Information</strong></p>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_l3de8a828a6db4063888567bc4e8689ab"><span>Click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_uicontrol12187373689199"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname4392923129199"><b>From</b></span> page, set <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname9733591129199"><b>Source type</b></span> to <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue102705719199"><b>HIVE</b></span>.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1105__en-us_topic_0000001219230969_td5b8619a75074733a2651f78d5b9e467" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Data source parameters</caption><thead align="left"><tr id="mrs_01_1105__en-us_topic_0000001219230969_r56711de050154379a490c77696719767"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.3.4.2.1.2.4.1.1"><p id="mrs_01_1105__en-us_topic_0000001219230969_a3bf030580fa6480499a7cc66e9085cb1">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="67.47999999999999%" id="mcps1.3.3.3.4.2.1.2.4.1.2"><p id="mrs_01_1105__en-us_topic_0000001219230969_a5f6b59e7ce4e436286251731ddf82e21">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="7.5200000000000005%" id="mcps1.3.3.3.4.2.1.2.4.1.3"><p id="mrs_01_1105__en-us_topic_0000001219230969_af5eb0a67c55b4d04bb895113b850b984">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1105__en-us_topic_0000001219230969_r6b620b471ef64b35ba0ca0ac0b1f1d33"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.4.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a96f0d65976614318beca7efd89ab4bf8">Hive instance</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.3.4.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a007f032062d14314aba75a98701535e6">Specifies the Hive service instance that Loader selects from all available Hive service instances in the cluster. If the selected Hive service instance is not added to the cluster, the Hive job cannot run properly.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.3.4.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_aa3660ceb3c994a23b74650582d54c617">hive</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_r7f89d81407554fe7b9de3d395426ad4f"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.4.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a6ba12d6b99484271b12d38d942c11a70">Quantity</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.3.4.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a0d0a81fce3294c2391571bb975af0fc3">Specifies the number of maps that are started at the same time in a MapReduce job of a data configuration operation. The value must be less than or equal to 3000. You are advised to set the parameter to the maximum number of connections on the SFTP server.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.3.4.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a3c542b1a77b94fdc836c77c10718b639">20</p>
</td>
</tr>
</tbody>
</table>
</div>
<p class="tableheading" id="mrs_01_1105__en-us_topic_0000001219230969_a74364c9bc47141129f18b34a1b953c9e"><strong id="mrs_01_1105__en-us_topic_0000001219230969_aa32d491a2050468ebbd8c9f408e5e983">Setting Data Transformation</strong></p>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_l955635917683496e8735ee27cca2a21e"><span>Click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_u6a6beca106ea4611ac65c2ef38a50518"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_pec0b8f638f6f4b9d877788a4e28a7e29"><b>Transform</b></span> page, set the transformation operations in the data transformation process. For details about how to select operators and set parameters, see <a href="mrs_01_1119.html">Operator Help</a> and <a href="#mrs_01_1105__en-us_topic_0000001219230969_table895989011525">Table 3</a>.</span><p>
<div class="tablenoborder"><a name="mrs_01_1105__en-us_topic_0000001219230969_table895989011525"></a><a name="en-us_topic_0000001219230969_table895989011525"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1105__en-us_topic_0000001219230969_table895989011525" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Setting the input and output parameters of the operator</caption><thead align="left"><tr id="mrs_01_1105__en-us_topic_0000001219230969_row1060779011525"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.3.5.2.1.2.3.1.1"><p id="mrs_01_1105__en-us_topic_0000001219230969_p1556013211525">Input Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.3.5.2.1.2.3.1.2"><p id="mrs_01_1105__en-us_topic_0000001219230969_p5241116811525">Export Type</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1105__en-us_topic_0000001219230969_row671911525"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.3.5.2.1.2.3.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_p54425911525">Hive input</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.3.5.2.1.2.3.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_p25487286115718">File output</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="fignone" id="mrs_01_1105__en-us_topic_0000001219230969_f82e5b49f6c014364854d210a45c21a98"><span class="figcap"><b>Figure 3 </b>Operator operation procedure</span><br><span><img id="mrs_01_1105__en-us_topic_0000001219230969_i307428511ebd453aaca6e5afd6b8dafd" src="en-us_image_0000001349258993.png"></span></div>
<p class="tableheading" id="mrs_01_1105__en-us_topic_0000001219230969_a572ff3171e354c76a58d0000c42ebcc5"><strong id="mrs_01_1105__en-us_topic_0000001219230969_aad5eaee976974726bf4ccb8e39386b56">Setting Data Storage Information and Executing the Job</strong></p>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_l9b20ae966b244ee18de9d555b7f738a1"><span>Click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_u7417796014944bcca12d1b25566ec3ea"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_p68cef2122b2547b18324ac575f014f4a"><b>To</b></span> page, set the data storage mode.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1105__en-us_topic_0000001219230969_tb8105d16e5b94b5d991838c9b62b9c54" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_1105__en-us_topic_0000001219230969_re191ae1aae2548378c984841792eb6f3"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.3.6.2.1.2.4.1.1"><p id="mrs_01_1105__en-us_topic_0000001219230969_aed8c81c71b8149feb8e704ff722d6449">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="67.47999999999999%" id="mcps1.3.3.3.6.2.1.2.4.1.2"><p id="mrs_01_1105__en-us_topic_0000001219230969_a0e42dd6b02184c84b3a2fd0b497955b9">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="7.5200000000000005%" id="mcps1.3.3.3.6.2.1.2.4.1.3"><p id="mrs_01_1105__en-us_topic_0000001219230969_aaf5f287ab4df475089ef08895de4dfeb">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1105__en-us_topic_0000001219230969_r7dd36eea53ae4d20b431f63a90a91f54"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.6.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a6ae33497b58a4c8f8e61d0993d35208f">Output path</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.3.6.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ad60e9df5becb42e38ac4fede170202f8">Specifies the path or file name of the exported file on an SFTP server. If multiple SFTP server IP addresses are configured for the connector, you can set this parameter to multiple paths or file names separated with semicolons (<span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_p293e6f679dca4a86b8363467a1bf67e4"><b>;</b></span>). Ensure that the number of paths or file names is the same as the number of SFTP servers configured for the connector.</p>
<div class="note" id="mrs_01_1105__en-us_topic_0000001219230969_n4e83aa19bb5e45f7a61b954e8ff35c61"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1105__en-us_topic_0000001219230969_a628569bc469146bea7060a5a9162224a">You can use macros to define path parameters. For details, see <a href="mrs_01_1153.html">Using Macro Definitions in Configuration Items</a>.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.3.6.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a86b3d71b0f2b47459bdb6dbe87e6d3e9">/opt/tempfile</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_r6bdf8ea980e246ab9f8c9b5c8625f7a4"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.6.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_aae160621361d454aaf666151b239fa33">Operation</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.3.6.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a36c01188ab5849e0aec9852f9783eb76">Specifies the action during data import. When all data is to be imported from the input path to the destination path, the data is stored in a temporary directory and then copied from the temporary directory to the destination path. After the data is imported successfully, the data is deleted from the temporary directory. One of the following actions can be taken when duplicate file names exist during data transfer:</p>
<ul id="mrs_01_1105__en-us_topic_0000001219230969_u874b92c6ddee42bf86b682fbda1ceb0f"><li id="mrs_01_1105__en-us_topic_0000001219230969_lc626e71b75cf43a1af463663d193ef16"><span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_p0ac4164b52fb4d439affc0df681d6560"><b>OVERRIDE</b></span>: overrides the old file.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l04a386fa67474a01b95ce20c6fc05c43"><span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_pbc1b25b0108a481aae893f13dda31f59"><b>RENAME</b></span>: renames as new file. For a file without an extension, a string is added to the file name as the extension; for a file with an extension, a string is added to the extension. The string is unique.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l1b02c6563c5a45588892456c699b8fe0"><span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_p82f586ab53a64b4e9a73dbf7b7afcbb6"><b>APPEND</b></span>: adds the content of the new file to the end of the old file. This action only adds content regardless of whether the file can be used. For example, a text file can be used after this operation, while a compressed file cannot.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l5e11899393e942058ac05e6e3e869b47"><span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_p46cdf74f46474159a5207db32dd72679"><b>IGNORE</b></span>: reserves the old file and does not copy the new file.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l52afee1a8312486891d273011fab6798"><span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_p85c8a5bce9384702b4485c698e1566b8"><b>ERROR</b></span>: stops the task and reports an error if duplicate file names exist. Transferred files are imported successfully, while files that have duplicate names and files that are not transferred fail to be imported.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.3.6.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ab832945fe8654ea1b235cb247db2a6e3">OVERRIDE</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_ra076935394dd42a0b5dc336b9a54be46"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.6.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a07abe539752c4882b0245f4560975a9f">Encode type</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.3.6.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_aab416f2f99f04492891f126f38a2095a">Specifies the exported file encoding format, for example, UTF-8. This parameter can be set only in text file export.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.3.6.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ac843f885ef1c49338811b08aa23c12e4">UTF-8</p>
</td>
</tr>
<tr id="mrs_01_1105__en-us_topic_0000001219230969_rf6716b1e4ca24b63bf59875663fa5dfd"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.6.2.1.2.4.1.1 "><p id="mrs_01_1105__en-us_topic_0000001219230969_ad228e66069e14d7e9b54c342809e096e">Compression</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.3.6.2.1.2.4.1.2 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a6c97384be87b4335af30670473b31f33">Indicates whether to enable the compressed transmission function when SFTP is used to export data.</p>
<ul id="mrs_01_1105__en-us_topic_0000001219230969_ub2b22fe4b7a74851a1310073821110f3"><li id="mrs_01_1105__en-us_topic_0000001219230969_l9e630a2d617e400ea9e1d1362a7ae65e">The value <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_pb116739b44b4498e839348c83ab4112c"><b>true</b></span> indicates that compression is enabled.</li><li id="mrs_01_1105__en-us_topic_0000001219230969_l261df65dd4da463ea9422d3a93af6d55">The value <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_pb30220438bae4623ac940b551b22da14"><b>false</b></span> indicates that compression is disabled.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.3.6.2.1.2.4.1.3 "><p id="mrs_01_1105__en-us_topic_0000001219230969_a55673b4f3f9142e0b76efe951598635e">true</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_l35ceed23740341e0858c42fabdea8f7a"><span>Click <span class="uicontrol" id="mrs_01_1105__en-us_topic_0000001219230969_ubc19e73c9e7e486bad58d9a79c3d7d56"><b>Save and run</b></span> to save and run the job.</span><p><p id="mrs_01_1105__en-us_topic_0000001219230969_acaea06faaae149fca400764ccfde4cd5"><strong id="mrs_01_1105__en-us_topic_0000001219230969_a350f547107cb4cf0a7591b2e30937a5c">Checking the Job Execution Result</strong></p>
</p></li><li id="mrs_01_1105__en-us_topic_0000001219230969_l6f3906ef7aaf4715ab8b8b5dfa7fd42d"><span>Go to the <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname7023014899199"><b>Loader WebUI</b></span>. When <span class="parmname" id="mrs_01_1105__en-us_topic_0000001219230969_parmname1487120449199"><b>Status</b></span> is <span class="parmvalue" id="mrs_01_1105__en-us_topic_0000001219230969_parmvalue7905672749199"><b>Succeeded</b></span>, the job is complete.</span><p><div class="fignone" id="mrs_01_1105__en-us_topic_0000001173630728_f6c7d800e4d2a45da9f9672d8d2b33ad1"><span class="figcap"><b>Figure 4 </b>Viewing job </span><br><span><img id="mrs_01_1105__en-us_topic_0000001173630728_ia5e94abc31a3428aaaca819fd1f345ba" src="en-us_image_0000001439347017.png"></span></div>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1100.html">Exporting Data</a></div>
</div>
</div>
View Git Blame Copy Permalink