forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
226 lines
21 KiB
HTML
226 lines
21 KiB
HTML
<a name="mrs_01_24754"></a><a name="mrs_01_24754"></a>
|
|
|
|
<h1 class="topictitle1">Synchronizing Data from Hudi to ClickHouse</h1>
|
|
<div id="body0000001583390741"><div class="section" id="mrs_01_24754__section7722164042318"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24754__p664594413238">This section describes how to import data from Hudi to ClickHouse by using the CDLService web UI of a cluster with Kerberos authentication enabled.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24754__section1438820494231"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_24754__ul2041871362311"><li id="mrs_01_24754__li12418213122313">The CDL, Hudi, and ClickHouse services have been installed in a cluster and are running properly.</li><li id="mrs_01_24754__li16897181718011">You have operation permissions on ClickHouse. For details, see <a href="mrs_01_24057.html">ClickHouse User and Permission Management</a>.</li><li id="mrs_01_24754__li1441819135232">You have created a human-machine user with the ClickHouse administrator permissions (for details, see <a href="mrs_01_24057.html">ClickHouse User and Permission Management</a>), for example, <strong id="mrs_01_24754__b946310492615">cdluser</strong>, added the user to user groups <strong id="mrs_01_24754__b846310414266">cdladmin</strong> (primary group), <strong id="mrs_01_24754__b12463749266">hadoop</strong>, <strong id="mrs_01_24754__b16463174142616">kafka</strong>, and <strong id="mrs_01_24754__b54633492616">supergroup</strong>, and associated the user with the <strong id="mrs_01_24754__b154637411265">System_administrator</strong> role on FusionInsight Manager.</li><li id="mrs_01_24754__li1732935195416">You have manually created a local table and distributed table on ClickHouse. The local table uses the ReplicatedReplacingMergeTree engine. For details, see <a href="mrs_01_2398.html">Creating a ClickHouse Table</a>.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_24754__section19362472245"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24754__ol18871133225317"><li id="mrs_01_24754__li1365211683614"><span>Log in to FusionInsight Manager as user <strong id="mrs_01_24754__b1819341153110">cdluser</strong> (change the password upon your first login), choose <strong id="mrs_01_24754__b61941411318">Cluster</strong> > <strong id="mrs_01_24754__b519510114316">Services</strong> > <strong id="mrs_01_24754__b1019516163118">CDL</strong>, and click the link next to <strong id="mrs_01_24754__b141961113111">CDLService UI</strong> to go to the CDLService web UI.</span></li><li id="mrs_01_24754__li3618121145616"><span>Choose <strong id="mrs_01_24754__b1663681114317">Link Management</strong> and click <strong id="mrs_01_24754__b56371311183115">Add Link</strong>. In the displayed dialog box, set parameters for adding the <strong id="mrs_01_24754__b15638181120315">clickhouse</strong> and <strong id="mrs_01_24754__b2639111143114">hudi</strong> links by referring to the following tables.</span><p>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24754__table134120364291" frame="border" border="1" rules="all"><caption><b>Table 1 </b>ClickHouse data link parameters</caption><thead align="left"><tr id="mrs_01_24754__row2041836162918"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.2.2.1.2.3.1.1"><p id="mrs_01_24754__p4411136172915">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.2.2.1.2.3.1.2"><p id="mrs_01_24754__p19411736142919">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24754__row10411236152920"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="mrs_01_24754__p154119368298">Link Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="mrs_01_24754__p154153632911">clickhouse</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row16426365294"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="mrs_01_24754__p44220365290">Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="mrs_01_24754__p104253620293">cklink</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row442636152915"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="mrs_01_24754__p9427368295">Host</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="mrs_01_24754__p17303172811553">10.10.10.10:21428</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row164263622915"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="mrs_01_24754__p1442636162910">User</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="mrs_01_24754__p4916845152114"><em id="mrs_01_24754__i146321513295">cdluser</em></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row104317366296"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="mrs_01_24754__p194314367297">Password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="mrs_01_24754__p391712458215"><em id="mrs_01_24754__i62283183115">Password of the <strong id="mrs_01_24754__b3141631173114">cdluser</strong> user</em></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row124313614299"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="mrs_01_24754__p74353613299">Description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="mrs_01_24754__p184383615296">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24754__table1437691715220" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Hudi data link parameters</caption><thead align="left"><tr id="mrs_01_24754__row1437618171029"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.2.2.2.2.3.1.1"><p id="mrs_01_24754__p137651713218">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.2.2.2.2.3.1.2"><p id="mrs_01_24754__p1237720171128">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24754__row33815178212"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.1 "><p id="mrs_01_24754__p153814179219">Link Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.2 "><p id="mrs_01_24754__p938112176210">hudi</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row193811117822"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.1 "><p id="mrs_01_24754__p138161715211">Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.2 "><p id="mrs_01_24754__p1738113173219">hudilink</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row83811417425"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.1 "><p id="mrs_01_24754__p125527208312">Storage Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.2 "><p id="mrs_01_24754__p35331203318">hdfs</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row736874119319"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.1 "><p id="mrs_01_24754__p1814611463913">Auth KeytabFile</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.2 "><p id="mrs_01_24754__p137021903412">/opt/Bigdata/third_lib/CDL/user_libs/cdluser.keytab</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row936915411130"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.1 "><p id="mrs_01_24754__p14146546699">Principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.2 "><p id="mrs_01_24754__p1414619465918">cdluser</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row938313177210"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.1 "><p id="mrs_01_24754__p1938310171727">Description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.2.2.2.3.1.2 "><p id="mrs_01_24754__p5383161716218">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</p></li><li id="mrs_01_24754__li1829512373488"><span>After the parameters are configured, click <strong id="mrs_01_24754__b1223375013110">Test</strong> to check whether the data link is normal.</span><p><p id="mrs_01_24754__p13530140134817">After the test is successful, click <strong id="mrs_01_24754__b166513532312">OK</strong>.</p>
|
|
</p></li><li id="mrs_01_24754__li976974910717"><span>(Optional) Choose <strong id="mrs_01_24754__en-us_topic_0000001446993949_b45961936164618">ENV Management</strong> and click <strong id="mrs_01_24754__en-us_topic_0000001446993949_b03551454164613">Add Env</strong>. In the displayed dialog box, configure the parameters based on the following table.</span><p>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24754__en-us_topic_0000001446993949_table54201432181218" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Parameters for adding an ENV</caption><thead align="left"><tr id="mrs_01_24754__en-us_topic_0000001446993949_row10418153291216"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.4.2.1.2.3.1.1"><p id="mrs_01_24754__en-us_topic_0000001446993949_p18418032161213">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.4.2.1.2.3.1.2"><p id="mrs_01_24754__en-us_topic_0000001446993949_p441810320123">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24754__en-us_topic_0000001446993949_row7419193251216"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p1141815324123">Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p14419183241213">test-env</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row04191332161212"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p18419133214122">Driver Memory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p341923215124">1 GB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row1468718181894"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p156889181495">Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p106881618096">spark</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row2419632151212"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p941918323125">Executor Memory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p1419832121219">1 GB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row20419143211122"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p041933213124">Executor Cores</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p1341983217125">1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row341915321123"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p194191032181217">Number Executors</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p841933251210">1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row77610527328"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p1525831203317">Queue</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p17675211325">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001446993949_row34191032171216"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p94191032181210">Description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.4.2.1.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001446993949_p5419193231211">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_24754__en-us_topic_0000001446993949_p145511354104314">Click <strong id="mrs_01_24754__en-us_topic_0000001446993949_b1436468274744">OK</strong>.</p>
|
|
</p></li><li id="mrs_01_24754__li8419191320242"><span>Choose <strong id="mrs_01_24754__b630712953220">Job Management</strong> > <strong id="mrs_01_24754__b93141795323">Data synchronization task</strong> and click <strong id="mrs_01_24754__b4314692324">Add Job</strong>. In the displayed dialog box, set parameters and click <strong id="mrs_01_24754__b03151913323">Next</strong>.</span><p><div class="p" id="mrs_01_24754__p1442915568513">Parameters are as follows.
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24754__table1379713112014" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24754__row13797011905"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.5.2.1.1.1.3.1.1"><p id="mrs_01_24754__p17797411309">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.5.2.1.1.1.3.1.2"><p id="mrs_01_24754__p137971011309">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24754__row1679841118018"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.5.2.1.1.1.3.1.1 "><p id="mrs_01_24754__p391613171005">Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.5.2.1.1.1.3.1.2 "><p id="mrs_01_24754__p149141171404">job_huditock</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row779916111602"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.5.2.1.1.1.3.1.1 "><p id="mrs_01_24754__p83069155152">Desc</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.5.2.1.1.1.3.1.2 "><p id="mrs_01_24754__p14306191511512">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</p></li><li id="mrs_01_24754__li2737114320203"><span>Configure Hudi job parameters.</span><p><ol type="a" id="mrs_01_24754__en-us_topic_0000001381930701_ol14971759152111"><li id="mrs_01_24754__en-us_topic_0000001381930701_li637518545212">On the <strong id="mrs_01_24754__b14390416115520">Job Management</strong> page, drag the <strong id="mrs_01_24754__b1339051695516">hudi</strong> icon in the Source area on the left to the editing area on the right and double-click the icon to go to the Hudi job configuration page. Configure parameters based on the following table.
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24754__en-us_topic_0000001381930701_table1415410186546" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Source Hudi job parameters</caption><thead align="left"><tr id="mrs_01_24754__en-us_topic_0000001381930701_row2153121875414"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.6.2.1.1.3.2.3.1.1"><p id="mrs_01_24754__en-us_topic_0000001381930701_p191530188543">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.6.2.1.1.3.2.3.1.2"><p id="mrs_01_24754__en-us_topic_0000001381930701_p2153818135411">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24754__en-us_topic_0000001381930701_row81531218125420"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.6.2.1.1.3.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001381930701_p21538187547">Link</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.6.2.1.1.3.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001381930701_p1615361825417">hudilink</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001381930701_row01531318175417"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.6.2.1.1.3.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001381930701_p3153121835411">Interval</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.6.2.1.1.3.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001381930701_p71531018205410">10</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__en-us_topic_0000001381930701_row115410183541"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.6.2.1.1.3.2.3.1.1 "><p id="mrs_01_24754__en-us_topic_0000001381930701_p7154121815414">Table Info</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.6.2.1.1.3.2.3.1.2 "><p id="mrs_01_24754__en-us_topic_0000001381930701_p315451805416">{"table1":[{"source.database":"db","source.tablename":"tabletest","target.tablename":"default.tabletest"}]}</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_24754__p315544843016"><span><img id="mrs_01_24754__image5165174183111" src="en-us_image_0000001532951888.png"></span></p>
|
|
</li><li id="mrs_01_24754__en-us_topic_0000001381930701_li5602185288">Click <strong id="mrs_01_24754__b7626162910557">OK</strong>. The Hudi job parameters are configured.</li></ol>
|
|
</p></li><li id="mrs_01_24754__li1736682513364"><span>Configure ClickHouse job parameters.</span><p><ol type="a" id="mrs_01_24754__ol5126919103712"><li id="mrs_01_24754__li1838281915919">On the <strong id="mrs_01_24754__b107051331133214">Job Management</strong> page, drag the <strong id="mrs_01_24754__b470613313322">clickhouse</strong> icon on the left to the editing area on the right and double-click the icon to go to the ClickHouse job configuration page. Configure parameters based on the following table.
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24754__table12483144010172" frame="border" border="1" rules="all"><caption><b>Table 5 </b>ClickHouse job parameters</caption><thead align="left"><tr id="mrs_01_24754__row124841940201711"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.7.2.1.1.3.2.3.1.1"><p id="mrs_01_24754__p348424041717">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.7.2.1.1.3.2.3.1.2"><p id="mrs_01_24754__p1948413401179">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24754__row1048434016174"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.7.2.1.1.3.2.3.1.1 "><p id="mrs_01_24754__p1484144016178">Link</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.7.2.1.1.3.2.3.1.2 "><p id="mrs_01_24754__p194845404178">cklink</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row119811045965"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.7.2.1.1.3.2.3.1.1 "><p id="mrs_01_24754__p298254518611">Query Timeout</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.7.2.1.1.3.2.3.1.2 "><p id="mrs_01_24754__p149821145463">60000</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24754__row3521451161"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.7.2.1.1.3.2.3.1.1 "><p id="mrs_01_24754__p175310515612">Batch Size</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.7.2.1.1.3.2.3.1.2 "><p id="mrs_01_24754__p185310519611">100</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_24754__p1424411385814"><span><img id="mrs_01_24754__image72532381483" src="en-us_image_0000001583151869.png"></span></p>
|
|
</li><li id="mrs_01_24754__li16689025123917">Click <strong id="mrs_01_24754__b1535917349339">OK</strong>.</li></ol>
|
|
</p></li><li id="mrs_01_24754__li032595183911"><span>Drag the two icons to associate the job parameters and click <strong id="mrs_01_24754__b799211103415">Save</strong>. The job configuration is complete.</span><p><p id="mrs_01_24754__p51802192443"><span><img id="mrs_01_24754__image1941514168918" src="en-us_image_0000001583391865.png"></span></p>
|
|
</p></li><li id="mrs_01_24754__li7968152124413"><span>In the job list on the <strong id="mrs_01_24754__b10882121312349">Job Management</strong> page, locate the created job, click <strong id="mrs_01_24754__b1388271316347">Start</strong> in the <strong id="mrs_01_24754__b15882141311348">Operation</strong> column, and wait until the job is started.</span><p><p id="mrs_01_24754__p29726217442">Check whether the data transmission takes effect, for example, insert data into the Hudi table and view the content of the file imported to ClickHouse.</p>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24240.html">Common CDL Jobs</a></div>
|
|
</div>
|
|
</div>
|
|
|