vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24173.html
Yang, Tong 3f5759eed2 MRS comp-lts 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2023-01-19 17:08:45 +00:00

150 lines
25 KiB
HTML
Raw Blame History

<a name="mrs_01_24173"></a><a name="mrs_01_24173"></a>
<h1 class="topictitle1">Typical Scenario: Importing Data from HDFS to ClickHouse</h1>
<div id="body32001227"><div class="section" id="mrs_01_24173__en-us_topic_0000001173471260_seeb6c6a3b7224b1ca00450981abf947a"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24173__en-us_topic_0000001173471260_ae7d5f654d94042b2bc73315ef3751614">Use Loader to import data from HDFS to ClickHouse.</p>
</div>
<div class="section" id="mrs_01_24173__en-us_topic_0000001173471260_s08a38f542df748faa85d14556dc5b6b1"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_24173__en-us_topic_0000001173471260_u7d08b15f5f9c44e392e04a1d2ba0e218"><li id="mrs_01_24173__en-us_topic_0000001173471260_l11338be6668b49a2a88ffbdff78ca129">You have obtained the service username and password for creating a Loader job.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l52c1c2a809f74a0583ab2cf5da561596">You have had the permission to access the HDFS directories and data involved in job execution.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_leef7df31c13541afa80beb692f5e0741">You have created a ClickHouse table, and you have operation permissions on the table during job execution.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l5823d7b24d484a92a92abe610fddc816">No disk space alarm is reported, and the available disk space is sufficient for importing and exporting data.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_lc9a71f9211f0426bb26e6dc85ce4f045">When using Loader to import data from HDFS, the input paths and input path subdirectories of HDFS and the name of the files in these directories do not contain any of the special characters <strong id="mrs_01_24173__en-us_topic_0000001173471260_b1822853121318">/"':;</strong>.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l539c4ad39be244e7894da5d385a4d44b">If a configured task requires the Yarn queue function, the user must be authorized with related Yarn queue permission.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l8e9fa6a7333246d0b53e380c5fffdc7a">The user who configures a task must obtain execution permission on the task and obtain usage permission on the related connection of the task.</li></ul>
</div>
<div class="section" id="mrs_01_24173__en-us_topic_0000001173471260_section115819105386"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="mrs_01_24173__en-us_topic_0000001173471260_aa68c381323e54258b0024e1217b231a5"><strong id="mrs_01_24173__en-us_topic_0000001173471260_ac3e555f0a4fa4a5084f49e8f271aae4e">Setting Basic Job Information</strong></p>
<ol id="mrs_01_24173__en-us_topic_0000001173471260_of7d104621add43fca96569fc36ba3213"><li id="mrs_01_24173__en-us_topic_0000001173471260_l450ee36108fd4598b573a344f12821d3"><span>Access the Loader web UI.</span><p><ol type="a" id="mrs_01_24173__en-us_topic_0000001219230551_obbc8c37dc53040efb21dc541b1dfc22c"><li id="mrs_01_24173__en-us_topic_0000001219230551_l6f0ffda40ca543d5a5660461b5e311dd">Log in to FusionInsight Manager. For details, see <a href="mrs_01_2124.html">Accessing FusionInsight Manager</a>.</li><li id="mrs_01_24173__en-us_topic_0000001219230551_la11b479a24ad4659a55365c2ede06015">Choose <strong id="mrs_01_24173__en-us_topic_0000001219230551_b39357591297">Cluster</strong> &gt; <em id="mrs_01_24173__en-us_topic_0000001219230551_i18941259112918">Name of the desired cluster</em> &gt; <strong id="mrs_01_24173__en-us_topic_0000001219230551_b19941115916297">Services</strong> &gt; <strong id="mrs_01_24173__en-us_topic_0000001219230551_b1894175916294">Loader</strong>.</li><li id="mrs_01_24173__en-us_topic_0000001219230551_l6601e8a2fd1b4780bf69238d6f5cc7f2">Click <strong id="mrs_01_24173__en-us_topic_0000001219230551_b13499742135213">LoaderServer(</strong><em id="mrs_01_24173__en-us_topic_0000001219230551_i187499244301">Node name</em><strong id="mrs_01_24173__en-us_topic_0000001219230551_b18777163515522">, Active)</strong>. The Loader web UI is displayed.<div class="fignone" id="mrs_01_24173__fig10607922203218"><span class="figcap"><b>Figure 1 </b>Loader web UI</span><br><span><img id="mrs_01_24173__mrs_01_1089_image155554364292" src="en-us_image_0000001438241209.png"></span></div>
</li></ol>
</p></li><li id="mrs_01_24173__en-us_topic_0000001173471260_l7a81094f6ff844a2be8b9292514ffe9e"><span>Click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_u9df961ce633b4c1e94bf10958a04f654"><b>New Job</b></span> to go to the <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_p93eac6bca2994113b744025668d62577"><b>Basic Information</b></span> page and set basic job information.</span><p><div class="fignone" id="mrs_01_24173__en-us_topic_0000001173471260_f9a48d25a6db1444780a2660d087f0af5"><span class="figcap"><b>Figure 2 </b>Basic Information</span><br><span><img id="mrs_01_24173__en-us_topic_0000001173471260_i4f78400285a247019dfdb140c4b11ef2" src="en-us_image_0000001349139741.png"></span></div>
<ol class="subitemlist" type="a" id="mrs_01_24173__en-us_topic_0000001173471260_ocf70368396f94a5aa51a5bd657da083f"><li id="mrs_01_24173__en-us_topic_0000001173471260_l95bc1de446cd4e10b0561aa0f91c182c">Set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_p2b33ae8f36c149c59cff066d82623269"><b>Name</b></span> to the name of the job.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l0e50a16623154d9382a8720905e1ad20">Set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname68109355854345"><b>Type</b></span> to <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname78862284554345"><b>Import</b></span>.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l6d47ded5a4a4443ea795f0613c2893bb">Set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname7820804054345"><b>Group</b></span> to the group to which the job belongs. No group is created by default. You need to click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol33661534754345"><b>Add</b></span> to create a group and click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol23856276054345"><b>OK</b></span> to save the created group.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l65875585deac4d0db9361c8e66dfc6cc">Set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_p57a5555696e144e2866cdf530658b373"><b>Queue</b></span> to the Yarn queue that executes the job. The default value is <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_pd53e248d3387466cb5e9e9e58eab0970"><b>root.default</b></span>.</li><li id="mrs_01_24173__en-us_topic_0000001173471260_l292cd5fb0aea4641ae975a4cd586ff8f">Set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_p869bc389de784414b33cadef994ad32b"><b>Priority</b></span> to the priority of the Yarn queue that executes the job. The default value is <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_pb17526e439f84964af8899a8fa562e2d"><b>NORMAL</b></span>. The options are <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue206824595354345"><b>VERY_LOW</b></span>, <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue85999416754345"><b>LOW</b></span>, <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue63095174054345"><b>NORMAL</b></span>, <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue79692517954345"><b>HIGH</b></span>, and <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue167594091154345"><b>VERY_HIGH</b></span>.</li></ol>
</p></li><li id="mrs_01_24173__en-us_topic_0000001173471260_l44aa1b24d5e444c8a558af45a992b667"><span>In the <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname59735652554345"><b>Connection</b></span> area, click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol212247789454345"><b>Add</b></span> to create a connection, set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname162028133254345"><b>Connector</b></span> to <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue138969679954345"><b>hdfs-connector</b></span>, set connection parameters, and click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol64202321554345"><b>Test</b></span> to verify whether the connection is available. When "Test Success" is displayed, click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol126885552054345"><b>OK</b></span>.</span></li></ol>
<p class="tableheading" id="mrs_01_24173__en-us_topic_0000001173471260_p14348135282610"><strong id="mrs_01_24173__en-us_topic_0000001173471260_a2d57cbd922684e7489dd942ee4571443">Setting Data Source Information</strong></p>
<ol start="4" id="mrs_01_24173__en-us_topic_0000001173471260_ol955265217264"><li id="mrs_01_24173__en-us_topic_0000001173471260_li13550552202613"><span>Click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol1454985210266"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname854965232617"><b>From</b></span> page, set the data source information.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24173__en-us_topic_0000001173471260_table1355035252616" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_24173__en-us_topic_0000001173471260_row3549175219264"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.5.1.2.1.2.4.1.1"><p id="mrs_01_24173__en-us_topic_0000001173471260_p154935218265">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="67.47999999999999%" id="mcps1.3.3.5.1.2.1.2.4.1.2"><p id="mrs_01_24173__en-us_topic_0000001173471260_p17549145211265">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="7.5200000000000005%" id="mcps1.3.3.5.1.2.1.2.4.1.3"><p id="mrs_01_24173__en-us_topic_0000001173471260_p55491752122611">Example</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24173__en-us_topic_0000001173471260_row1055095211268"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p45495528262">Input Path</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p1055015218265">Input path of source files in HDFS</p>
<div class="note" id="mrs_01_24173__en-us_topic_0000001173471260_note11550252162615"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24173__en-us_topic_0000001173471260_p1855011522260">You can use macros to define path parameters. For details, see <a href="mrs_01_1153.html">Using Macro Definitions in Configuration Items</a>.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p14550145211265">/user/test</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row355018522261"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p455015212269">Path Filter</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p355016526261">Wildcard for filtering the directories in the input paths of the source files. <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname8550135262617"><b>Input Path</b></span> is not used for filtering. If there are multiple filter conditions, use commas (<span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue055075219266"><b>,</b></span>) to separate them. If the parameter is empty, the directories are not filtered. The regular expression filtering is not supported.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p14550125215265">*</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row1355014523267"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p185501552172616">File Filter</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p1355011521265">Wildcard for filtering the file names of the source files. If there are multiple filter conditions, use commas (<span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue555095215269"><b>,</b></span>) to separate them. The value cannot be left blank. The regular expression filtering is not supported.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p155025211269">*</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row1055017524269"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p35501852132619">Encoding Type</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p185502526267">Source file encoding format, for example, UTF-8. This parameter can be set only in text file import.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p1755035211266">UTF-8</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row455065214269"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p2550185220269">Suffix</p>
</td>
<td class="cellrowborder" valign="top" width="67.47999999999999%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p9550852122614">File name extension added to a source file after the source file is imported. If this parameter is empty, no file name extension is added to the source file.</p>
</td>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p1755018529261">.log</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li></ol>
<p class="tableheading" id="mrs_01_24173__en-us_topic_0000001173471260_p1652364132716"><strong id="mrs_01_24173__en-us_topic_0000001173471260_b95506522264">Setting Data Transformation</strong></p>
<ol start="5" id="mrs_01_24173__en-us_topic_0000001173471260_ol06098482712"><li id="mrs_01_24173__en-us_topic_0000001173471260_li26071422720"><span>Click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol460704102718"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname3607114132718"><b>Transform</b></span> page, set the transformation operations in the data transformation process. For details about how to select operators and set parameters, see <a href="mrs_01_1119.html">Operator Help</a> and <a href="#mrs_01_24173__en-us_topic_0000001173471260_table4607134162717">Table 2</a>.</span><p>
<div class="tablenoborder"><a name="mrs_01_24173__en-us_topic_0000001173471260_table4607134162717"></a><a name="en-us_topic_0000001173471260_table4607134162717"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24173__en-us_topic_0000001173471260_table4607134162717" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Input and output parameters of the operator</caption><thead align="left"><tr id="mrs_01_24173__en-us_topic_0000001173471260_row16607149276"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.7.1.2.1.2.3.1.1"><p id="mrs_01_24173__en-us_topic_0000001173471260_p126072420270">Input Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.7.1.2.1.2.3.1.2"><p id="mrs_01_24173__en-us_topic_0000001173471260_p106071440279">Output Type</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24173__en-us_topic_0000001173471260_row14607154162718"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.7.1.2.1.2.3.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p860719419278">CSV File Input</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.7.1.2.1.2.3.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p116072415270">ClickHouse Output</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="fignone" id="mrs_01_24173__en-us_topic_0000001173471260_fig360717462712"><span class="figcap"><b>Figure 3 </b>Operator operation procedure</span><br><span><img id="mrs_01_24173__image1158932416594" src="en-us_image_0000001388071772.png"></span></div>
</p></li></ol>
<p class="tableheading" id="mrs_01_24173__en-us_topic_0000001173471260_p116171513112719"><strong id="mrs_01_24173__en-us_topic_0000001173471260_b196078414272">Setting Data Storage Information and Executing the Job</strong></p>
<ol start="6" id="mrs_01_24173__en-us_topic_0000001173471260_ol136621013132710"><li id="mrs_01_24173__en-us_topic_0000001173471260_li19662151313273"><span>Click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol12977163915168"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname16984163915161"><b>To</b></span> page, set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname3985439191616"><b>Storage type</b></span> to <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue139852399164"><b>CLICKHOUSE</b></span> based on the actual situation.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24173__en-us_topic_0000001173471260_table9662101310274" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_24173__en-us_topic_0000001173471260_row1366217130275"><th align="left" class="cellrowborder" valign="top" width="14.41%" id="mcps1.3.3.9.1.2.1.2.5.1.1"><p id="mrs_01_24173__en-us_topic_0000001173471260_p1166221315279">Storage Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="22.79%" id="mcps1.3.3.9.1.2.1.2.5.1.2"><p id="mrs_01_24173__en-us_topic_0000001173471260_p9662913182710">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50.63999999999999%" id="mcps1.3.3.9.1.2.1.2.5.1.3"><p id="mrs_01_24173__en-us_topic_0000001173471260_p196621513192712">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.16%" id="mcps1.3.3.9.1.2.1.2.5.1.4"><p id="mrs_01_24173__en-us_topic_0000001173471260_p0662171312274">Example</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24173__en-us_topic_0000001173471260_row866291372717"><td class="cellrowborder" rowspan="5" valign="top" width="14.41%" headers="mcps1.3.3.9.1.2.1.2.5.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p12662161312715">CLICKHOUSE</p>
<p id="mrs_01_24173__en-us_topic_0000001173471260_p566241312720"></p>
</td>
<td class="cellrowborder" valign="top" width="22.79%" headers="mcps1.3.3.9.1.2.1.2.5.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p4662111362714">ClickHouse instance</p>
</td>
<td class="cellrowborder" valign="top" width="50.63999999999999%" headers="mcps1.3.3.9.1.2.1.2.5.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p76629130278">ClickHouse service instance that Loader selects from all available ClickHouse service instances in the cluster. If the selected ClickHouse service instance is not added to the cluster, the ClickHouse job cannot be run properly.</p>
</td>
<td class="cellrowborder" valign="top" width="12.16%" headers="mcps1.3.3.9.1.2.1.2.5.1.4 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p136621913102713">ClickHouse</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row1566231314277"><td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p1466211311271">Clear data before import</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p666281332718">Whether to clear data in the original table before importing data <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue30089717754345"><b>true</b></span> indicates clearing data and <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue114037544654345"><b>false</b></span> indicates not to clear data. If you do not set this parameter, the original table is not cleared by default.</p>
<div class="note" id="mrs_01_24173__en-us_topic_0000001173471260_note17299142213408"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24173__en-us_topic_0000001173471260_p202994228408">ClickHouse distributed tables do not support clearing data before import. You need to manually clear the table.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p2662813172719">false</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row96622138273"><td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p15662181392712">Extractors</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p4662181319273">Number of Maps that are started at the same time in a MapReduce task of a data configuration operation. The value must be less than or equal to 3000.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p9662713102717">20</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row26625132272"><td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p186621713122710">Extractor size</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p186621813182720">ClickHouse does not support this parameter. Please set <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname7898194518219"><b>Extractors</b></span>.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p126627139271">-</p>
</td>
</tr>
<tr id="mrs_01_24173__en-us_topic_0000001173471260_row146625131274"><td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.1 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p8662181372711">Number</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.2 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p6662191332712">Number of Map tasks.</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.3.9.1.2.1.2.5.1.3 "><p id="mrs_01_24173__en-us_topic_0000001173471260_p15662171314275">-</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="mrs_01_24173__en-us_topic_0000001173471260_li866251318275"><span>Click <span class="uicontrol" id="mrs_01_24173__en-us_topic_0000001173471260_uicontrol1223215165548"><b>Save and Run</b></span> to save and run the job.</span></li></ol>
<p id="mrs_01_24173__en-us_topic_0000001173471260_p4811202362713"><strong id="mrs_01_24173__en-us_topic_0000001173471260_b1766281316272">Checking the Job Execution Result</strong></p>
<ol start="8" id="mrs_01_24173__en-us_topic_0000001173471260_ol14827123142720"><li id="mrs_01_24173__en-us_topic_0000001173471260_li1582702311271"><span>Go to the Loader web UI. When <span class="parmname" id="mrs_01_24173__en-us_topic_0000001173471260_parmname26969852254345"><b>Status</b></span> is <span class="parmvalue" id="mrs_01_24173__en-us_topic_0000001173471260_parmvalue206919793254345"><b>Succeeded</b></span>, the job is complete.</span><p><div class="fignone" id="mrs_01_24173__en-us_topic_0000001173471260_fig138271623192712"><span class="figcap"><b>Figure 4 </b>Viewing job details</span><br><span><img id="mrs_01_24173__en-us_topic_0000001173471260_image138271623102714" src="en-us_image_0000001296219656.png"></span></div>
</p></li><li id="mrs_01_24173__en-us_topic_0000001173471260_li94867196227"><span>On the ClickHouse client, check whether the data in the ClickHouse table is the same as that imported from HDFS.</span></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1086.html">Importing Data</a></div>
</div>
</div>
View Git Blame Copy Permalink