vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24172.html
Yang, Tong 3f5759eed2 MRS comp-lts 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2023-01-19 17:08:45 +00:00

179 lines
32 KiB
HTML
Raw Blame History

<a name="mrs_01_24172"></a><a name="mrs_01_24172"></a>
<h1 class="topictitle1">Typical Scenario: Importing Data from a Relational Database to ClickHouse</h1>
<div id="body32001227"><div class="section" id="mrs_01_24172__en-us_topic_0000001173471080_sc127ab9b81ba479a9cad923854cb8c5d"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24172__en-us_topic_0000001173471080_a16f7abbc8c90428783c6956d67d7539e">This section describes how to use Loader to import data from a relational database to ClickHouse using MySQL as an example.</p>
</div>
<div class="section" id="mrs_01_24172__en-us_topic_0000001173471080_s7cbb81d52aa54fa893f7ecc6b282ccbd"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_24172__en-us_topic_0000001173471080_ubfb0076c7946405cb23e3a8486c6d8db"><li id="mrs_01_24172__en-us_topic_0000001173471080_lc099b5e7fe1a4411b917fdfb4971aa08">You have obtained the service username and password for creating a Loader job.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_la9a1b99ed54449e0bdb83996414aeda0">You have created a ClickHouse table, and you have operation permissions on the table during job execution.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l8b497f24254f4f4b9421d6e853de73ce">You have obtained the user name and password of the MySQL database.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l8bc5ad36ccd840a79ce2e4bd71711516">No disk space alarm is reported, and the available disk space is sufficient for importing and exporting data.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l8ba46a5e84b94077a4d59af06e2d83eb">If a configured task requires the Yarn queue function, the user must be authorized with related Yarn queue permission.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l943785ceca344825a7de3438d9c3e71a">The user who configures a task must obtain execution permission on the task and obtain usage permission on the related connection of the task.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l1ea6a03d3f564ea081f14ed543af6d57">Before the operation, perform the following steps:<ol id="mrs_01_24172__en-us_topic_0000001173471080_o4f990a0c190143de9dbc4cec3b443df2"><li id="mrs_01_24172__en-us_topic_0000001173471080_led61689a591d4ea39bb5ce14b2bdbd89">Obtain the MySQL client JAR file (for example, <strong id="mrs_01_24172__en-us_topic_0000001173471080_b9764305672113">mysqlclient-5.8.1.jar</strong>) from the MySQL database installation path and save it to the following directory on the active and standby Loader nodes: <strong id="mrs_01_24172__en-us_topic_0000001173471080_b18701592282113">${BIGDATA_HOME}/FusionInsight_Porter_<span id="mrs_01_24172__en-us_topic_0000001173471080_text1827215402113">8.1.2.2</span>/install/FusionInsight-Sqoop-1.99.3/FusionInsight-Sqoop-1.99.3/server/webapps/loader/WEB-INF/ext-lib</strong>.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_li16408082412">Obtain the <strong id="mrs_01_24172__en-us_topic_0000001173471080_b16182143364811">clickhouse-jdbc-*.jar</strong> file from the ClickHouse installation directory and save it in the <strong id="mrs_01_24172__en-us_topic_0000001173471080_b15321448174910">${BIGDATA_HOME}/FusionInsight_Porter_<span id="mrs_01_24172__en-us_topic_0000001173471080_text1756741511432">8.1.2.2</span>/install/FusionInsight-Sqoop-1.99.3/FusionInsight-Sqoop-1.99.3/server/webapps/loader/WEB-INF/ext-lib</strong> directory on the active and standby Loader nodes.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_lf8ef04c43049400fadcf6e4cfadec682">Run the following command on the active and standby nodes as user <strong id="mrs_01_24172__en-us_topic_0000001173471080_b10194507022113">root</strong> to modify the permission:<p id="mrs_01_24172__en-us_topic_0000001173471080_p131744244568"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b75232536535">cd </strong><strong id="mrs_01_24172__en-us_topic_0000001173471080_b8525153175310">${BIGDATA_HOME}/FusionInsight_Porter_<span id="mrs_01_24172__en-us_topic_0000001173471080_text04931257152514">8.1.2.2</span></strong><strong id="mrs_01_24172__en-us_topic_0000001173471080_b14526175315316">/install/FusionInsight-Sqoop-1.99.3/FusionInsight-Sqoop-1.99.3/server/webapps/loader/WEB-INF/ext-lib</strong></p>
<p id="mrs_01_24172__en-us_topic_0000001173471080_p5532031084"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b19504961952113">chown omm:wheel</strong> <em id="mrs_01_24172__en-us_topic_0000001173471080_i9055486602113">JAR</em><em id="mrs_01_24172__en-us_topic_0000001173471080_i5789053652113"> file name</em></p>
<p id="mrs_01_24172__en-us_topic_0000001173471080_p148971528287"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b18853388572113">chmod 600</strong> <em id="mrs_01_24172__en-us_topic_0000001173471080_i21045385842113">JAR</em><em id="mrs_01_24172__en-us_topic_0000001173471080_i9863095032113"> file name</em></p>
</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l2e9dbae6ce644e2eb2e9e8c967494fac">Log in to FusionInsight Manager, choose <span id="mrs_01_24172__en-us_topic_0000001173471080_text97979461241"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b12107517526">Cluster</strong> &gt; </span><strong id="mrs_01_24172__en-us_topic_0000001173471080_b12510132385214">Services</strong> &gt; <strong id="mrs_01_24172__en-us_topic_0000001173471080_b384462865213">Loader</strong> &gt; <strong id="mrs_01_24172__en-us_topic_0000001173471080_b1110141055310">More</strong> &gt; <strong id="mrs_01_24172__en-us_topic_0000001173471080_b1364131605316">Restart Service</strong> and enter the system administrator password to restart the Loader service.</li></ol>
</li></ul>
</div>
<div class="section" id="mrs_01_24172__en-us_topic_0000001173471080_section636894614548"><h4 class="sectiontitle">Procedure</h4><p class="tableheading" id="mrs_01_24172__en-us_topic_0000001173471080_adfb5fc20cfb74a4484386ceae79ca548"><strong id="mrs_01_24172__en-us_topic_0000001173471080_a9f804e1ec1f8474787277df06522d198">Setting Basic Job Information</strong></p>
<ol id="mrs_01_24172__en-us_topic_0000001173471080_o46ae2f61d272420692191f2d9ca72ef1"><li id="mrs_01_24172__en-us_topic_0000001173471080_la0f8015ba9de4203bf5f978108135cbb"><span>Access the Loader web UI.</span><p><ol type="a" id="mrs_01_24172__en-us_topic_0000001219230551_obbc8c37dc53040efb21dc541b1dfc22c"><li id="mrs_01_24172__en-us_topic_0000001219230551_l6f0ffda40ca543d5a5660461b5e311dd">Log in to FusionInsight Manager. For details, see <a href="mrs_01_2124.html">Accessing FusionInsight Manager</a>.</li><li id="mrs_01_24172__en-us_topic_0000001219230551_la11b479a24ad4659a55365c2ede06015">Choose <strong id="mrs_01_24172__en-us_topic_0000001219230551_b39357591297">Cluster</strong> &gt; <em id="mrs_01_24172__en-us_topic_0000001219230551_i18941259112918">Name of the desired cluster</em> &gt; <strong id="mrs_01_24172__en-us_topic_0000001219230551_b19941115916297">Services</strong> &gt; <strong id="mrs_01_24172__en-us_topic_0000001219230551_b1894175916294">Loader</strong>.</li><li id="mrs_01_24172__en-us_topic_0000001219230551_l6601e8a2fd1b4780bf69238d6f5cc7f2">Click <strong id="mrs_01_24172__en-us_topic_0000001219230551_b13499742135213">LoaderServer(</strong><em id="mrs_01_24172__en-us_topic_0000001219230551_i187499244301">Node name</em><strong id="mrs_01_24172__en-us_topic_0000001219230551_b18777163515522">, Active)</strong>. The Loader web UI is displayed.<div class="fignone" id="mrs_01_24172__fig10607922203218"><span class="figcap"><b>Figure 1 </b>Loader web UI</span><br><span><img id="mrs_01_24172__mrs_01_1089_image155554364292" src="en-us_image_0000001438241209.png"></span></div>
</li></ol>
</p></li><li id="mrs_01_24172__en-us_topic_0000001173471080_l8346efaf9af34a2ab08420069eeaa75e"><span>Click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_u42a8a6577af7491f9c8b9244a9c19c72"><b>New Job</b></span> to go to the <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_p5fcef536cf0e4d38b24ff0291e09713e"><b>Basic Information</b></span> page and set basic job information.</span><p><div class="fignone" id="mrs_01_24172__en-us_topic_0000001173471080_f5aaea1bc10444832b9ff7ea748a1b890"><span class="figcap"><b>Figure 2 </b>Basic Information</span><br><span><img id="mrs_01_24172__en-us_topic_0000001173471080_id51dda5818f649d580a026bd9f0d010c" src="en-us_image_0000001296219364.png"></span></div>
<ol class="subitemlist" type="a" id="mrs_01_24172__en-us_topic_0000001173471080_o09c9dfd64a6a432dab4c58101a69dbed"><li id="mrs_01_24172__en-us_topic_0000001173471080_ld5f7677c2dd74e3f9126fe29e873cafe">Set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_pbf04ef523a3941b78d30d9c43da74b64"><b>Name</b></span> to the name of the job.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_ld1d263cfebd94083b1490926a9504495">Set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname20332993222113"><b>Type</b></span> to <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname11091950342113"><b>Import</b></span>.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l809ee3119cd34c47ba539dcec19d3d20">Set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname13007875282113"><b>Group</b></span> to the group to which the job belongs. No group is created by default. You need to click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol12723156422113"><b>Add</b></span> to create a group and click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol565640362113"><b>OK</b></span> to save the created group.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_l4139d6dc4a5e4040a7538e71878ff838">Set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_pc77bd730731d41c2866600848f9a1e0f"><b>Queue</b></span> to the Yarn queue that executes the job. The default value is <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_pf1ddf7e7e57e4ca984b4d87e01093332"><b>root.default</b></span>.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_la31e0186bdf1411cbb8785b0562d6743">Set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_pec1c4b77f7774b9980ab7a3fff1c2669"><b>Priority</b></span> to the priority of the Yarn queue that executes the job. The default value is <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_p9536cd4b5b7746f3b9fc8957e4bb9f39"><b>NORMAL</b></span>. The options are <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue9087017212113"><b>VERY_LOW</b></span>, <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue7423014232113"><b>LOW</b></span>, <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue19324741622113"><b>NORMAL</b></span>, <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue9273693562113"><b>HIGH</b></span>, and <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue12764034612113"><b>VERY_HIGH</b></span>.</li></ol>
</p></li><li id="mrs_01_24172__en-us_topic_0000001173471080_lafa0a56f6b994ef18580fb90cae6e488"><span>In the <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname788214619418"><b>Connection</b></span> area, click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol38821368414"><b>Add</b></span> to create a connection, set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname38831361643"><b>Connector</b></span> to <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue11883567418"><b>generic-jdbc-connector</b></span> or a dedicated database connector (oracle-connector, oracle-partition-connector, or mysql-fastpath-connector), set connection parameters, and click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol188326142"><b>Test</b></span> to verify whether the connection is available. When "Test Success" is displayed, click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol198841862045"><b>OK</b></span>.</span><p><div class="note" id="mrs_01_24172__en-us_topic_0000001173471080_n9eb286cca3864ab1a9511038598ec14d"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_24172__en-us_topic_0000001173471080_u3650cf4545454e51ba9acf314126037f"><li id="mrs_01_24172__en-us_topic_0000001173471080_l81e3372e79054652ae76e224876b5d6a">For connection to relational databases, general database connectors (generic-jdbc-connector) or dedicated database connectors (oracle-connector, oracle-partition-connector, and mysql-fastpath-connector) are available. However, compared with general database connectors, dedicated database connectors perform better in data import and export because they are optimized for specific database types.</li><li id="mrs_01_24172__en-us_topic_0000001173471080_lf03fd4f537cf4d338547260bf4e9b166">When mysql-fastpath-connector is used, the <strong id="mrs_01_24172__en-us_topic_0000001173471080_b15417127962113">mysqldump </strong>and <strong id="mrs_01_24172__en-us_topic_0000001173471080_b20423154012113">mysqlimport</strong> commands of MySQL must be available on NodeManager nodes, and the MySQL client version to which the two commands belong must be compatible with the MySQL server version. If the two commands are unavailable or the versions are incompatible, install the MySQL client applications and tools following the instructions at <a href="http://dev.mysql.com/doc/refman/5.7/en/linux-installation-rpm.html" target="_blank" rel="noopener noreferrer">http://dev.mysql.com/doc/refman/5.7/en/linux-installation-rpm.html</a>.</li></ul>
</div></div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24172__en-us_topic_0000001173471080_t28700deed8cb4e3eac36db33c6b0b1b7" frame="border" border="1" rules="all"><caption><b>Table 1 </b>generic-jdbc-connector connection parameters</caption><thead align="left"><tr id="mrs_01_24172__en-us_topic_0000001173471080_r6705d0bec14f46a495d77bc87c501cbf"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.3.3.2.2.2.4.1.1"><p id="mrs_01_24172__en-us_topic_0000001173471080_ac719381587e942c88376130ff38d87eb">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="35.620000000000005%" id="mcps1.3.3.3.3.2.2.2.4.1.2"><p id="mrs_01_24172__en-us_topic_0000001173471080_a3c57a233691140d8be33efef067e9134">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="39.379999999999995%" id="mcps1.3.3.3.3.2.2.2.4.1.3"><p id="mrs_01_24172__en-us_topic_0000001173471080_aa6a4543369f64ef5884064c6f60755d5">Example</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24172__en-us_topic_0000001173471080_rf0f9d97d1b1a4b6a8cc5e6766274df73"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.2.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a9d3b50a0ade74529b2e9ef87803d2ef5">Name</p>
</td>
<td class="cellrowborder" valign="top" width="35.620000000000005%" headers="mcps1.3.3.3.3.2.2.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a956cad24ccca4511b4ff4a0ba33fc67f">Name of a relational database connection</p>
</td>
<td class="cellrowborder" valign="top" width="39.379999999999995%" headers="mcps1.3.3.3.3.2.2.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_af4ff5c3cf32547e89e8edf9d62b2e102">mysql_test</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_reff2b6a6503045afb9d71e6065a5738e"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.2.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a598363318b734605b757252de4d2379f">JDBC Driver Class</p>
</td>
<td class="cellrowborder" valign="top" width="35.620000000000005%" headers="mcps1.3.3.3.3.2.2.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a5fd25cf100c64a78987009eeb066688c">Name of a JDBC driver class</p>
</td>
<td class="cellrowborder" valign="top" width="39.379999999999995%" headers="mcps1.3.3.3.3.2.2.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p169311331454">com.mysql.jdbc.Driver</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_rc48128237652440fb751bd9fa0f38e84"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.2.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a25327c38906d46659a4ab272752a60ca">JDBC Connection String</p>
</td>
<td class="cellrowborder" valign="top" width="35.620000000000005%" headers="mcps1.3.3.3.3.2.2.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a9938ddad4a7b424d9d58ab7eef04fe48">JDBC connection string</p>
</td>
<td class="cellrowborder" valign="top" width="39.379999999999995%" headers="mcps1.3.3.3.3.2.2.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1265874415514">jdbc:mysql://10.254.144.102:3306/test?useUnicode=true&amp;characterEncoding=UTF-8</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_rd2b9fa08135743c686d9a6330cf00354"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.2.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a512f153469a3475a90518978f99a2968">Username</p>
</td>
<td class="cellrowborder" valign="top" width="35.620000000000005%" headers="mcps1.3.3.3.3.2.2.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a7a24f0872fc546a6875e869fe84688e8">Username for connecting to the database</p>
</td>
<td class="cellrowborder" valign="top" width="39.379999999999995%" headers="mcps1.3.3.3.3.2.2.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a9682070f467e4680a7a640faf6d29bb4">root</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_r2805a523516c412c9120de1b86fb2379"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.3.3.2.2.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a139176436241470abc082b5e93e156fa">Password</p>
</td>
<td class="cellrowborder" valign="top" width="35.620000000000005%" headers="mcps1.3.3.3.3.2.2.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_ab0cefa7570f544cab5faa35d47c1949f">Password for connecting to the database</p>
</td>
<td class="cellrowborder" valign="top" width="39.379999999999995%" headers="mcps1.3.3.3.3.2.2.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_a0abf485732004b1a8bfa807efb27b058">xxxx</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li></ol>
<p class="tableheading" id="mrs_01_24172__en-us_topic_0000001173471080_p357611251254"><strong id="mrs_01_24172__en-us_topic_0000001173471080_a9881a2d2bfe14026a6620fdf8fb3ea19">Setting Data Source Information</strong></p>
<ol start="4" id="mrs_01_24172__en-us_topic_0000001173471080_ol197564252513"><li id="mrs_01_24172__en-us_topic_0000001173471080_li107541525758"><span>Click <strong id="mrs_01_24172__en-us_topic_0000001173471080_b59431232570">Next</strong>. On the displayed <strong id="mrs_01_24172__en-us_topic_0000001173471080_b8253514175710">From</strong> page, configure the data source information. Currently, only <strong id="mrs_01_24172__en-us_topic_0000001173471080_b1720212563457">Table name</strong> is supported.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24172__en-us_topic_0000001173471080_table175412251555" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_24172__en-us_topic_0000001173471080_row1754102514510"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.5.1.2.1.2.4.1.1"><p id="mrs_01_24172__en-us_topic_0000001173471080_p197547259518">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="58.269999999999996%" id="mcps1.3.3.5.1.2.1.2.4.1.2"><p id="mrs_01_24172__en-us_topic_0000001173471080_p19754172517511">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="16.73%" id="mcps1.3.3.5.1.2.1.2.4.1.3"><p id="mrs_01_24172__en-us_topic_0000001173471080_p3754125956">Example</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24172__en-us_topic_0000001173471080_row87541525659"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p2075413251858">Schema name</p>
</td>
<td class="cellrowborder" valign="top" width="58.269999999999996%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1375411252519">Schema name of the specified database</p>
</td>
<td class="cellrowborder" valign="top" width="16.73%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p27544253514">public</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_row1075419257511"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1975412251855">Table name</p>
</td>
<td class="cellrowborder" valign="top" width="58.269999999999996%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1975416258520">Table name</p>
</td>
<td class="cellrowborder" valign="top" width="16.73%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p12754725152">test</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_row177543257519"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1575417251517">Table column names</p>
</td>
<td class="cellrowborder" valign="top" width="58.269999999999996%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1275414251553">Names of the columns to be imported</p>
</td>
<td class="cellrowborder" valign="top" width="16.73%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p6754102516511">id,name</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_row27545252514"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.5.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1775414257517">Need partition column</p>
</td>
<td class="cellrowborder" valign="top" width="58.269999999999996%" headers="mcps1.3.3.5.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p16754625951">Currently, only the unspecified partition mode is supported.</p>
</td>
<td class="cellrowborder" valign="top" width="16.73%" headers="mcps1.3.3.5.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1875417251513">false</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li></ol>
<p class="tableheading" id="mrs_01_24172__en-us_topic_0000001173471080_p1315294513512"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b137541251359">Setting Data Transformation</strong></p>
<ol start="5" id="mrs_01_24172__en-us_topic_0000001173471080_ol122200451056"><li id="mrs_01_24172__en-us_topic_0000001173471080_li1521919451152"><span>Click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol82182458512"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname921864518511"><b>Transform</b></span> page, set the transformation operations in the data transformation process. For details about how to select operators and set parameters, see <a href="mrs_01_1119.html">Operator Help</a> and <a href="#mrs_01_24172__en-us_topic_0000001173471080_table92181845553">Table 3</a>.</span><p>
<div class="tablenoborder"><a name="mrs_01_24172__en-us_topic_0000001173471080_table92181845553"></a><a name="en-us_topic_0000001173471080_table92181845553"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24172__en-us_topic_0000001173471080_table92181845553" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Input and output parameters of the operator</caption><thead align="left"><tr id="mrs_01_24172__en-us_topic_0000001173471080_row8218184518516"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.7.1.2.1.2.3.1.1"><p id="mrs_01_24172__en-us_topic_0000001173471080_p112181445557">Input Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.7.1.2.1.2.3.1.2"><p id="mrs_01_24172__en-us_topic_0000001173471080_p1221894516516">Output Type</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24172__en-us_topic_0000001173471080_row621814451651"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.7.1.2.1.2.3.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p821812455513">MySQL Input</p>
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.7.1.2.1.2.3.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p721810451852">ClickHouse Output</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="mrs_01_24172__en-us_topic_0000001173471080_p5218124514520">Drag <strong id="mrs_01_24172__en-us_topic_0000001173471080_b142368204481">Table Input</strong> to the grid, double-click <strong id="mrs_01_24172__en-us_topic_0000001173471080_b894052812482">Table Input</strong>, and select <strong id="mrs_01_24172__en-us_topic_0000001173471080_b1542217401110">autoRecognition</strong>, as shown in <a href="#mrs_01_24172__en-us_topic_0000001173471080_fig1621911456515">Figure 3</a>.</p>
<div class="fignone" id="mrs_01_24172__en-us_topic_0000001173471080_fig1621911456515"><a name="mrs_01_24172__en-us_topic_0000001173471080_fig1621911456515"></a><a name="en-us_topic_0000001173471080_fig1621911456515"></a><span class="figcap"><b>Figure 3 </b>Operator input</span><br><span><img id="mrs_01_24172__image156413311196" src="en-us_image_0000001391556190.png"></span></div>
<p id="mrs_01_24172__p09141717135818"></p>
<p id="mrs_01_24172__en-us_topic_0000001173471080_p192197451352">Drag <strong id="mrs_01_24172__en-us_topic_0000001173471080_b37820610373">ClickHouse Output</strong> to the grid, double-click <strong id="mrs_01_24172__en-us_topic_0000001173471080_b1364523711370">Table Output</strong>, and select <strong id="mrs_01_24172__en-us_topic_0000001173471080_b184471526389">associate</strong> or manually edit the table to correspond to the input table, as shown in <a href="#mrs_01_24172__en-us_topic_0000001173471080_fig721910459510">Figure 4</a>.</p>
<div class="fignone" id="mrs_01_24172__en-us_topic_0000001173471080_fig721910459510"><a name="mrs_01_24172__en-us_topic_0000001173471080_fig721910459510"></a><a name="en-us_topic_0000001173471080_fig721910459510"></a><span class="figcap"><b>Figure 4 </b>Operator output</span><br><span><img id="mrs_01_24172__en-us_topic_0000001173471080_image62191645958" src="en-us_image_0000001349259037.png"></span></div>
</p></li></ol>
<p class="tableheading" id="mrs_01_24172__en-us_topic_0000001173471080_p1476125318516"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b62191345355">Setting Data Storage Information and Executing the Job</strong></p>
<ol start="6" id="mrs_01_24172__en-us_topic_0000001173471080_ol211110531457"><li id="mrs_01_24172__en-us_topic_0000001173471080_li1311110531557"><span>Click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol13800923622113"><b>Next</b></span>. On the displayed <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname3981199722113"><b>To</b></span> page, set <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname16941204762113"><b>Storage type</b></span> to <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue7309656712113"><b>CLICKHOUSE</b></span>.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24172__en-us_topic_0000001173471080_table41114531158" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_24172__en-us_topic_0000001173471080_row1511095319517"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.9.1.2.1.2.4.1.1"><p id="mrs_01_24172__en-us_topic_0000001173471080_p011055315518">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.940000000000005%" id="mcps1.3.3.9.1.2.1.2.4.1.2"><p id="mrs_01_24172__en-us_topic_0000001173471080_p1110853956">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.06%" id="mcps1.3.3.9.1.2.1.2.4.1.3"><p id="mrs_01_24172__en-us_topic_0000001173471080_p61104533512">Example</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24172__en-us_topic_0000001173471080_row111106531754"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.9.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p61101253757">Storage type</p>
</td>
<td class="cellrowborder" valign="top" width="59.940000000000005%" headers="mcps1.3.3.9.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1711018531955">Select <strong id="mrs_01_24172__en-us_topic_0000001173471080_b393210102614">CLICKHOUSE</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="15.06%" headers="mcps1.3.3.9.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p121101853952">-</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_row911019533520"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.9.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p151108539515">ClickHouse instance</p>
</td>
<td class="cellrowborder" valign="top" width="59.940000000000005%" headers="mcps1.3.3.9.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1411011532513">Select <strong id="mrs_01_24172__en-us_topic_0000001173471080_b7764423864">ClickHouse</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="15.06%" headers="mcps1.3.3.9.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1111010537517">-</p>
</td>
</tr>
<tr id="mrs_01_24172__en-us_topic_0000001173471080_row4111145317515"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.9.1.2.1.2.4.1.1 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1211112531354">Clear data before import</p>
</td>
<td class="cellrowborder" valign="top" width="59.940000000000005%" headers="mcps1.3.3.9.1.2.1.2.4.1.2 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p71110531857">Select <strong id="mrs_01_24172__en-us_topic_0000001173471080_b1533323817612">true</strong> or <strong id="mrs_01_24172__en-us_topic_0000001173471080_b491714433614">false</strong>.</p>
<div class="note" id="mrs_01_24172__en-us_topic_0000001173471080_note17299142213408"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24172__en-us_topic_0000001173471080_p202994228408">ClickHouse distributed tables do not support clearing data before import. You need to manually clear the table.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="15.06%" headers="mcps1.3.3.9.1.2.1.2.4.1.3 "><p id="mrs_01_24172__en-us_topic_0000001173471080_p1411113535513">true</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="mrs_01_24172__en-us_topic_0000001173471080_li311120531550"><span>Click <span class="uicontrol" id="mrs_01_24172__en-us_topic_0000001173471080_uicontrol411125311519"><b>Save and Run</b></span> to save and run the job.</span></li></ol>
<p id="mrs_01_24172__en-us_topic_0000001173471080_p522720114618"><strong id="mrs_01_24172__en-us_topic_0000001173471080_b2011145319519">Checking the Job Execution Result</strong></p>
<ol start="8" id="mrs_01_24172__en-us_topic_0000001173471080_ol12243811960"><li id="mrs_01_24172__en-us_topic_0000001173471080_li10243313613"><span>Go to the Loader web UI. When <span class="parmname" id="mrs_01_24172__en-us_topic_0000001173471080_parmname10693314122113"><b>Status</b></span> is <span class="parmvalue" id="mrs_01_24172__en-us_topic_0000001173471080_parmvalue1923779182113"><b>Succeeded</b></span>, the job is complete.</span><p><div class="fignone" id="mrs_01_24172__en-us_topic_0000001173471080_fig16242614610"><span class="figcap"><b>Figure 5 </b>Viewing job details</span><br><span><img id="mrs_01_24172__en-us_topic_0000001173471080_image1824211116619" src="en-us_image_0000001349139449.png"></span></div>
</p></li><li id="mrs_01_24172__en-us_topic_0000001173471080_li94867196227"><span>On the ClickHouse client, check whether the data in the ClickHouse table is the same as that in the MySql table.</span></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1086.html">Importing Data</a></div>
</div>
</div>
View Git Blame Copy Permalink