vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24239.html
Yang, Tong 48706b7552 MRS COMP-LTS 320-lts.1 version 
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2024-04-12 12:51:10 +00:00

714 lines
62 KiB
HTML
Raw Blame History

<a name="mrs_01_24239"></a><a name="mrs_01_24239"></a>
<h1 class="topictitle1">Creating a CDL Data Synchronization Job</h1>
<div id="body0000001583150753"><div class="section" id="mrs_01_24239__section768174512333"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24239__p126201948193313">The CDLService web UI provides a visualized page for users to quickly create CDL jobs and import real-time data into the data lake.</p>
</div>
<div class="section" id="mrs_01_24239__section296352011367"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_24239__p1525051552">A user with the CDL management permission has been created for the cluster with Kerberos authentication enabled.</p>
</div>
<div class="section" id="mrs_01_24239__section09718589438"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24239__ol4252101010395"><li id="mrs_01_24239__li2138162063820"><span>Access the CDLService web UI as a user with the CDL management permissions or the <strong id="mrs_01_24239__b510619504578">admin</strong> user (for the cluster where Kerberos authentication is not enabled). For details, see <a href="mrs_01_24236.html">Logging In to the CDLService WebUI</a>.</span></li><li id="mrs_01_24239__li1713802043818"><span>Choose <strong id="mrs_01_24239__b11766053191210">Job Management</strong> &gt; <strong id="mrs_01_24239__b2068191471416">Data synchronization task</strong> and click <strong id="mrs_01_24239__b2766185318123">Add Job</strong>. In the displayed dialog box, set related job parameters and click <strong id="mrs_01_24239__b1276785341214">Next</strong>.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table1379713112014" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24239__row13797011905"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.2.1.1.4.1.1"><p id="mrs_01_24239__p17797411309">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.2.1.1.4.1.2"><p id="mrs_01_24239__p379716113014">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.2.1.1.4.1.3"><p id="mrs_01_24239__p137971011309">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row1679841118018"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.1.4.1.1 "><p id="mrs_01_24239__p391613171005">Name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.1.4.1.2 "><p id="mrs_01_24239__p159159171708">Job name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.1.4.1.3 "><p id="mrs_01_24239__p149141171404">job_pgsqltokafka</p>
</td>
</tr>
<tr id="mrs_01_24239__row779916111602"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.1.4.1.1 "><p id="mrs_01_24239__p2917945132119">Desc</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.1.4.1.2 "><p id="mrs_01_24239__p17917124517216">Job description</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.1.4.1.3 "><p id="mrs_01_24239__p5917174518212">xxx</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="mrs_01_24239__li5419142012411"><span>On the <strong id="mrs_01_24239__b114301337184516">Job Management</strong> page, select and drag the target element from <strong id="mrs_01_24239__b85761510512">Source</strong> and <strong id="mrs_01_24239__b340758125114">Sink</strong> to the GUI on the right.</span><p><p id="mrs_01_24239__p1057318323564"><span><img id="mrs_01_24239__image18645183335614" src="en-us_image_0000001582952073.png"></span></p>
<p id="mrs_01_24239__p3967012286">Double-click the two elements to connect them and set related parameters as required.</p>
<p id="mrs_01_24239__p79972317104">To delete an element, select the element to be deleted and click <strong id="mrs_01_24239__b233881615315">Delete</strong> in the lower right corner of the page.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table844617269491" frame="border" border="1" rules="all"><caption><b>Table 1 </b>MySQL job parameters</caption><thead align="left"><tr id="mrs_01_24239__row1444718265496"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.4.2.4.1.1"><p id="mrs_01_24239__p544722634916">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.4.2.4.1.2"><p id="mrs_01_24239__p17447126184916">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.4.2.4.1.3"><p id="mrs_01_24239__p134471265496">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row94475260496"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p1344742616493">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p1944712617499">Created MySQL link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p844717264495">mysqllink</p>
</td>
</tr>
<tr id="mrs_01_24239__row1644713268495"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p774114465112">Tasks Max</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p59911153327">Maximum number of tasks that can be created by a connector. For a connector of the database type, this parameter must be set to <strong id="mrs_01_24239__b1330110515194">1</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p54471826174912">1</p>
</td>
</tr>
<tr id="mrs_01_24239__row674113585014"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p101909509512">Mode</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p874117515019">Type of the CDC event to be captured by the job. Value options are as follows:</p>
<ul id="mrs_01_24239__ul520532854115"><li id="mrs_01_24239__li1973353334116"><strong id="mrs_01_24239__b1883756522737">insert</strong></li><li id="mrs_01_24239__li1067420356412"><strong id="mrs_01_24239__b5323118175716">update</strong></li><li id="mrs_01_24239__li420562818415"><strong id="mrs_01_24239__b419782357737">delete</strong></li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p87411595011">insert, update, and delete</p>
</td>
</tr>
<tr id="mrs_01_24239__row4342183211517"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p143421432165116">DB Name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p1234223219519">MySQL database name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p4342153214514">cdl-test</p>
</td>
</tr>
<tr id="mrs_01_24239__row197411505017"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p19742455503">Schema Auto Create</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p187421565015">Whether to create table schemas after the job is started</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p874265165015">No</p>
</td>
</tr>
<tr id="mrs_01_24239__row41484181507"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p6148101815504">Connect With Hudi</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p1580474914421">Whether to connect to Hudi</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p1414813188503">Yes</p>
</td>
</tr>
<tr id="mrs_01_24239__row165081137112018"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p65091337102012">DBZ Snapshot Locking Mode</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p2737335172115">Lock mode used when a task starts to execute a snapshot. Value options are as follows:</p>
<ul id="mrs_01_24239__ul5868140172111"><li id="mrs_01_24239__li1170418412225"><strong id="mrs_01_24239__b9549644416">minimal</strong>: A global read lock is held only when the database schema and other metadata are obtained.</li><li id="mrs_01_24239__li1616231722212"><strong id="mrs_01_24239__b45841246194118">extend</strong>: A global read lock is held during the entire snapshot execution process, blocking all write operations.</li><li id="mrs_01_24239__li9868134015210"><strong id="mrs_01_24239__b19180840102713">none</strong>: No lock mode. The schema cannot be changed when a CDL task is started.<p id="mrs_01_24239__p1511094082219">Optional. Click <span><img id="mrs_01_24239__image31101400220" src="en-us_image_0000001583151845.png"></span> to display this parameter.</p>
</li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p125090379204">none</p>
</td>
</tr>
<tr id="mrs_01_24239__row537220271122"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p920212613333">WhiteList</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p6142315145419">Whitelisted tables to be captured.</p>
<p id="mrs_01_24239__p0622153115213">Separate multiple tables using commas (,). Wildcards are supported.</p>
<p id="mrs_01_24239__p12022026143313">(Optional) This parameter is displayed when you click <span><img id="mrs_01_24239__image72021266332" src="en-us_image_0000001583391841.png"></span>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p1737320270216">testtable</p>
</td>
</tr>
<tr id="mrs_01_24239__row1037320271522"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p14202126123318">BlackList</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p17873161815419">Whitelisted tables not to be captured.</p>
<p id="mrs_01_24239__p11498171020466">Separate multiple tables using commas (,). Wildcards are supported.</p>
<p id="mrs_01_24239__p1620219263331">(Optional) This parameter is displayed when you click <span><img id="mrs_01_24239__image142028263330" src="en-us_image_0000001583272145.png"></span>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p1537314271427">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row1981743185614"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p2289520105717">Multi Partition</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p12289112055710">Whether to enable multi-partition mode for topics.</p>
<p id="mrs_01_24239__p523519561465">If enabled, you need to set <strong id="mrs_01_24239__b187600145118">Topic Table Mapping</strong> and specify the number of topic partitions, and the data of a single table will be scattered in multiple partitions.</p>
<p id="mrs_01_24239__p5910142520575">(Optional) This parameter is displayed when you click <span><img id="mrs_01_24239__image6943851195720" src="en-us_image_0000001532632184.png"></span>.</p>
<div class="note" id="mrs_01_24239__note14289520125719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24239__p1028912204578">The data receiving sequence cannot be ensured. Exercise caution when setting this parameter.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p3289820155711">No</p>
</td>
</tr>
<tr id="mrs_01_24239__row11373162718220"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_24239__p620332623319">Topic Table Mapping</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_24239__p4909158122120">Mapping between topics and tables.</p>
<p id="mrs_01_24239__p169961421155416">If configured, table data can be sent to the specified topic. If multi-partitioning is enabled, you need to set the number of partitions, which must be greater than 1.</p>
<p id="mrs_01_24239__p62031826143311">This parameter is displayed when you click <span><img id="mrs_01_24239__image520392619339" src="en-us_image_0000001532472712.png"></span>. This parameter is mandatory if <strong id="mrs_01_24239__b46021826193617">Connect With Hudi</strong> is set to <strong id="mrs_01_24239__b1789722983611">Yes</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_24239__p121279954817">testtable</p>
<p id="mrs_01_24239__p2037392716217">testtable_topic</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table1814971517167" frame="border" border="1" rules="all"><caption><b>Table 2 </b>PgSQL job parameters</caption><thead align="left"><tr id="mrs_01_24239__row1415041510166"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.5.2.4.1.1"><p id="mrs_01_24239__p4150915161614">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.5.2.4.1.2"><p id="mrs_01_24239__p5150101511616">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.5.2.4.1.3"><p id="mrs_01_24239__p215051514168">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row3150915191617"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p2015091561617">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p20150121519164">Created PgSQL link.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p171507153167">pgsqllink</p>
</td>
</tr>
<tr id="mrs_01_24239__row5150151516169"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p915061521613">Tasks Max</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p16150191510161">Maximum number of tasks that can be created by a connector. For a connector of the database type, this parameter must be set to <strong id="mrs_01_24239__b18719027202019">1</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p815017159166">1</p>
</td>
</tr>
<tr id="mrs_01_24239__row15150815201614"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p23892122226">Mode</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p7389111210223">Type of the CDC event to be captured by the job. The options are as follows:</p>
<ul id="mrs_01_24239__ul1038931211221"><li id="mrs_01_24239__li6389101252219"><strong id="mrs_01_24239__b78418490547">insert</strong></li><li id="mrs_01_24239__li6389121252217"><strong id="mrs_01_24239__b8280135219545">update</strong></li><li id="mrs_01_24239__li163891212102216"><strong id="mrs_01_24239__b173753545549">delete</strong></li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p112792033162019">insert, update, and delete</p>
</td>
</tr>
<tr id="mrs_01_24239__row17151191512169"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p123284921814">dbName Alias</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p432629121812">Database name.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p10323109111811">test</p>
</td>
</tr>
<tr id="mrs_01_24239__row1826931185019"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p14132114025015">Schema</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p1313215405501">Schema of the database to be connected to.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p1813216407502">public</p>
</td>
</tr>
<tr id="mrs_01_24239__row17151315131617"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p13169919184">Slot Name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p105621538124811">Name of the PostgreSQL logical replication slot.</p>
<p id="mrs_01_24239__p69417243476">The value can contain lowercase letters, digits, and underscores (_), and cannot be the same in any other job.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p1231269121813">test_solt_1</p>
</td>
</tr>
<tr id="mrs_01_24239__row11199132315251"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p2200152316257">Enable FailOver Slot</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p2020012232251">Whether to enable the failover slot function. After it is enabled, the information about the logical replication slot specified as the failover slot is synchronized from the active instance to the standby instance. In this manner, logical subscription can continue even upon an active/standby switchover, implementing the failover of the logical replication slot.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p320012312510">No</p>
</td>
</tr>
<tr id="mrs_01_24239__row1117615572287"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p43649942919">Slot Drop</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p2176857152811">Whether to delete the slot when a task is stopped</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p1417614573284">No</p>
</td>
</tr>
<tr id="mrs_01_24239__row1199065145613"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p1115038115617">Connect With Hudi</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p159751491958">Whether to connect to Hudi.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p91501588564">Yes</p>
</td>
</tr>
<tr id="mrs_01_24239__row3628145053015"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p76291850163010">Use Exist Publication</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p962912509301">Use a created publication</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p06297503308">Yes</p>
</td>
</tr>
<tr id="mrs_01_24239__row69601258143015"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p1096011589306">Publication Name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p1196011585302">Name of a created publication</p>
<p id="mrs_01_24239__p593225961111">This parameter is available when <strong id="mrs_01_24239__b75292469259">Use Exist Publication</strong> is set to <strong id="mrs_01_24239__b3590848202515">Yes</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p39601158193014">test</p>
<p id="mrs_01_24239__p128014124212"></p>
</td>
</tr>
<tr id="mrs_01_24239__row51477585323"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p1214775893214">Start Time</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p17147175812324">Start time for synchronizing tables</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p10147195803218">2022/03/16 11:33:37</p>
</td>
</tr>
<tr id="mrs_01_24239__row173216365241"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p5940152519">WhiteList</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p179416119253">Whitelisted tables to be captured.</p>
<p id="mrs_01_24239__p16549204494914">Separate multiple tables using commas (,). Wildcards are supported.</p>
<p id="mrs_01_24239__p49410119255">(Optional) This parameter is displayed when you click <span><img id="mrs_01_24239__image109411192519" src="en-us_image_0000001532791932.png"></span>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p1873383622414">testtable</p>
</td>
</tr>
<tr id="mrs_01_24239__row873333622412"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p8948122511">BlackList</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p139411115250">Whitelisted tables not to be captured.</p>
<p id="mrs_01_24239__p2166195464918">Separate multiple tables using commas (,). Wildcards are supported.</p>
<p id="mrs_01_24239__p129421102511">(Optional) This parameter is displayed when you click <span><img id="mrs_01_24239__image159413172519" src="en-us_image_0000001532951868.png"></span>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p1873316362242">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row1369173213410"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p66963215344">Start Position</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p2693325343">Start LSN of the data captured by a task</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p56903218348">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row623883793410"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p11238193712343">Start Txid</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p10238163723411">Start TXID of the data captured by a task</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p023893712344">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row11727753155814"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p55006095911">Multi Partition</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p205011608592">Whether to enable multi-partition mode for topics.</p>
<p id="mrs_01_24239__p1440710318506">If enabled, you need to set <strong id="mrs_01_24239__b1464032319580">Topic Table Mapping</strong> and specify the number of topic partitions, and the data of a single table will be scattered in multiple partitions.</p>
<p id="mrs_01_24239__p35747425918">(Optional) This parameter is displayed when you click <span><img id="mrs_01_24239__image99319415593" src="en-us_image_0000001583151849.png"></span>.</p>
<div class="note" id="mrs_01_24239__note105010019599"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24239__p17501402597">The data receiving sequence cannot be ensured. Exercise caution when setting this parameter.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p1450140185917">No</p>
</td>
</tr>
<tr id="mrs_01_24239__row1473313602419"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.1 "><p id="mrs_01_24239__p5131169142515">Topic Table Mapping</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.2 "><p id="mrs_01_24239__p713119972515">Mapping between topics and tables.</p>
<p id="mrs_01_24239__p1283112513516">If configured, table data can be sent to the specified topic. If multi-partitioning is enabled, you need to set the number of partitions, which must be greater than 1.</p>
<p id="mrs_01_24239__p1113114992519">This parameter is displayed when you click <span><img id="mrs_01_24239__image101314917258" src="en-us_image_0000001583391845.png"></span>. This parameter is mandatory if <strong id="mrs_01_24239__b153271421778">Connect With Hudi</strong> is set to <strong id="mrs_01_24239__b03283212714">Yes</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.5.2.4.1.3 "><p id="mrs_01_24239__p16268113018544">testtable</p>
<p id="mrs_01_24239__p11733113612248">testtable_topic</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table1075717168395" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Source Hudi job parameters</caption><thead align="left"><tr id="mrs_01_24239__row8757416133915"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.6.2.4.1.1"><p id="mrs_01_24239__p1275861683918">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.6.2.4.1.2"><p id="mrs_01_24239__p4758181615393">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.6.2.4.1.3"><p id="mrs_01_24239__p375861613391">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row177581516173915"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p12758316113917">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p175841663916">Link used by the Hudi app</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p775811616394">hudilink</p>
</td>
</tr>
<tr id="mrs_01_24239__row5787653116"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p3785619310">Interval</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p11781260318">Interval for synchronizing the Hudi table, in seconds</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p167811643112">10</p>
</td>
</tr>
<tr id="mrs_01_24239__row240422344019"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p14041823174010">Start Time</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p64041323134019">Start time for synchronizing tables</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p140416235404">2022/03/16 11:40:52</p>
</td>
</tr>
<tr id="mrs_01_24239__row1332121207"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p43323112014">Max Commit Number</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p173331917018">Maximum number of commits that can be pulled from an incremental view at a time.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p1133310120010">10</p>
</td>
</tr>
<tr id="mrs_01_24239__row261773224015"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p1161783254019">Hudi Custom Config</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p7617163210400">Customized configuration related to Hudi.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p11617532154018">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row18973142774012"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p9974627164014">Table Info</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p169741827184013">Detailed configuration information about the synchronization table. Hudi and DWS must have the same table names and field types.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p3285183316339">{"table1":[{"source.database":"base1","source.tablename":"table1"}],"table2":[{"source.database":"base2","source.tablename":"table2"}],"table3":[{"source.database":"base3","source.tablename":"table3"}]}</p>
</td>
</tr>
<tr id="mrs_01_24239__row126712099412"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.1 "><p id="mrs_01_24239__p11880101144114">Execution Env</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.2 "><p id="mrs_01_24239__p108809119415">Environment variable required for running the Hudi App. If no ENV is available, manually create one.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.6.2.4.1.3 "><p id="mrs_01_24239__p188116119411">defaultEnv</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table8128935153416" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Source Kafka job parameters</caption><thead align="left"><tr id="mrs_01_24239__row5128173510348"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.7.2.4.1.1"><p id="mrs_01_24239__p912817350347">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.7.2.4.1.2"><p id="mrs_01_24239__p312823517345">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.7.2.4.1.3"><p id="mrs_01_24239__p11128635103416">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row1012803517346"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.7.2.4.1.1 "><p id="mrs_01_24239__p31281035143413">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.7.2.4.1.2 "><p id="mrs_01_24239__p19129123583411">Created Kafka link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.7.2.4.1.3 "><p id="mrs_01_24239__p712933513348">kafkalink</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table192710519543" frame="border" border="1" rules="all"><caption><b>Table 5 </b>thirdparty-kafka job parameters</caption><thead align="left"><tr id="mrs_01_24239__row12815511544"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.8.2.4.1.1"><p id="mrs_01_24239__p1328105145411">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.8.2.4.1.2"><p id="mrs_01_24239__p14284555420">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.8.2.4.1.3"><p id="mrs_01_24239__p172814525410">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row1928155165415"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p12825195418">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p32815525412">Created thirdparty-kafka link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p1928750545">thirdparty-kafkalink</p>
</td>
</tr>
<tr id="mrs_01_24239__row182814518541"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p172812518540">DB Name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p62865165417">Name of the database to be connected to.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p3288517546">opengaussdb</p>
</td>
</tr>
<tr id="mrs_01_24239__row9282505415"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p928195175410">Schema</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p1872531435718">Schema of the database to be checked</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p1472211145572">oprngaussschema</p>
</td>
</tr>
<tr id="mrs_01_24239__row1283517540"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p429159541">Datastore Type</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p197195145572">Type of the upper-layer source. Value options are as follows:</p>
<ul id="mrs_01_24239__ul441104154819"><li id="mrs_01_24239__li44115414816">opengauss</li><li id="mrs_01_24239__li4259710164814">ogg</li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p870710144571">opengauss</p>
</td>
</tr>
<tr id="mrs_01_24239__row15462111053616"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p5463171013363">Avro Schema Topic</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p1646301083612">Schema topic used by OGG Kafka to store table schemas in JSON format.</p>
<div class="note" id="mrs_01_24239__note558611434148"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24239__p1758674318142">This parameter is available when <strong id="mrs_01_24239__b111822855943742">Datastore Type</strong> is set to <strong id="mrs_01_24239__b46432691043742">ogg</strong>.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p15463181093617">ogg_topic</p>
</td>
</tr>
<tr id="mrs_01_24239__row93081751174810"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p3308115111482">Source Topics</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p1630816514485">Source topics can contain letters, digits, and special characters (-,_). Topics must be separated by commas (,).</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p19308125120487">topic1</p>
</td>
</tr>
<tr id="mrs_01_24239__row429175195419"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p132918545411">Tasks Max</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p177055144578">Maximum number of tasks that can be created by a connector. For a connector of the database type, this parameter must be set to <strong id="mrs_01_24239__b109219017346">1</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p2070471445712">10</p>
</td>
</tr>
<tr id="mrs_01_24239__row14296535410"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p132918517547">Tolerance</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p1652763151414">Fault tolerance policy.</p>
<ul id="mrs_01_24239__ul137076121415"><li id="mrs_01_24239__li5901013161417"><strong id="mrs_01_24239__b93011428162">none</strong>: indicates low tolerance and the Connector task will fail if an error occurs.</li><li id="mrs_01_24239__li1737036201416"><strong id="mrs_01_24239__b12748279163">all</strong>: indicates high tolerance and all failed records will be ignored if an error occurs.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p3698614125719">all</p>
</td>
</tr>
<tr id="mrs_01_24239__row1130959545"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p43075195413">Start Time</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p736314814586">Start time for synchronizing tables</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p1736114814584">2022/03/16 14:14:50</p>
</td>
</tr>
<tr id="mrs_01_24239__row176271732183913"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p162743210395">Multi Partition</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p562743253914">Whether to enable multi-partitioning for topics. If it is enabled, you need to set <strong id="mrs_01_24239__b149441294348">Topic Table Mapping</strong> and specify the number of topic partitions, and the data of a single table will be scattered in multiple partitions.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p18627173216399">No</p>
</td>
</tr>
<tr id="mrs_01_24239__row11309511543"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.1 "><p id="mrs_01_24239__p153095195412">Topic Table Mapping</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.2 "><p id="mrs_01_24239__p3637112812162">Mapping between topics and tables.</p>
<p id="mrs_01_24239__p1163882816167">If configured, table data can be sent to the specified topic. If multi-partitioning is enabled, you need to set the number of partitions, which must be greater than 1.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.8.2.4.1.3 "><p id="mrs_01_24239__p4962174713263">testtable</p>
<p id="mrs_01_24239__p10637142195811">testtable_topic</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><a name="mrs_01_24239__table12483144010172"></a><a name="table12483144010172"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table12483144010172" frame="border" border="1" rules="all"><caption><b>Table 6 </b>Sink Hudi job parameters</caption><thead align="left"><tr id="mrs_01_24239__row124841940201711"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.9.2.4.1.1"><p id="mrs_01_24239__p348424041717">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.9.2.4.1.2"><p id="mrs_01_24239__p19484540121710">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.9.2.4.1.3"><p id="mrs_01_24239__p1948413401179">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row1048434016174"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p1484144016178">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p1048474016170">Created Hudi link.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p194845404178">hudilink</p>
</td>
</tr>
<tr id="mrs_01_24239__row180882194"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p15013813190">Path</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p20178201912">Path for storing data.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p77992305319">/cdldata</p>
</td>
</tr>
<tr id="mrs_01_24239__row5681835171915"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p11871110202515">Interval</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p12682193591914">Spark RDD execution interval, in seconds.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p1468223521916">1</p>
</td>
</tr>
<tr id="mrs_01_24239__row1846319595191"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p20360540152420">Max Rate Per Partition</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p446315598194">Maximum rate for reading data from each Kafka partition using the Kafka direct stream API. It is the number of records per second. <strong id="mrs_01_24239__b167401712103317">0</strong> indicates that the rate is not limited.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p194631759111911">0</p>
</td>
</tr>
<tr id="mrs_01_24239__row16719954172016"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p787862912417">Parallelism</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p17201954122017">Parallelism for writing data to Hudi.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p1472055412016">100</p>
</td>
</tr>
<tr id="mrs_01_24239__row10118191192319"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p611813110230">Target Hive Database</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p111181142314">Database of the target Hive</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p101184114233">default</p>
</td>
</tr>
<tr id="mrs_01_24239__row10126211566"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p712710117619">Configuring Hudi Table Attributes</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p512713116615">View for configuring attributes of the Hudi table. The value can be:</p>
<ul id="mrs_01_24239__ul1015910238483"><li id="mrs_01_24239__li1215992310487">Visual View</li><li id="mrs_01_24239__li1159223104811">JSON View</li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p10127011463">Visual View</p>
</td>
</tr>
<tr id="mrs_01_24239__row1855215444614"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p1055215442614">Global Configuration of Hudi Table Attributes</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p355313441265">Global parameters on Hudi.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p55538447612">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row1544112524617"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p2441125215616">Configuring the Attributes of the Hudi Table</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p1441052966">Configuration of the Hudi table attributes.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p184413524614">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row10921956199"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p2921135619910">Configuring the Attributes of the Hudi Table: Table Name</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p13921125611911">Hudi table name, which must be the same as the source table name.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p89211056690">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row1547102641111"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p17982055191113">Configuring the Attributes of the Hudi Table: Table Type Opt Key</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p798115581110">Hudi table type. The options are as follows:</p>
<ul id="mrs_01_24239__ul199855515119"><li id="mrs_01_24239__li5986557113">COPY_ON_WRITE</li><li id="mrs_01_24239__li7983556115">MERGE_ON_READ</li></ul>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p1198145531118">MERGE_ON_READ</p>
</td>
</tr>
<tr id="mrs_01_24239__row03690388322"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p4370038123217">Configuring the Attributes of the Hudi Table: Hudi TableName Mapping</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p1337073817327">Hudi table name. If this parameter is not set, the name of the Hudi table is the same as that of the source table by default.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p14370103818321">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row357894482214"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p35781944172219">Configuring the Attributes of the Hudi Table: Hive TableName Mapping</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p1259610471613">Mapping between Hudi tables and Hive tables.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p2057804416228">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row1140912671518"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p11410106131519">Configuring the Attributes of the Hudi Table: Table Primarykey Mapping</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p9410176191519">Primary key mapping of the Hudi table</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p04105614156">id</p>
</td>
</tr>
<tr id="mrs_01_24239__row9805194191212"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p58051541131215">Configuring the Attributes of the Hudi Table: Table Hudi Partition Type</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p68055414120">Mapping between the Hudi table and partition fields. If the Hudi table uses partitioned tables, you need to configure the mapping between the table name and partition fields. The value can be <strong id="mrs_01_24239__b01233525465">time</strong> or <strong id="mrs_01_24239__b1131015554469">customized</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p1180517418125">time</p>
</td>
</tr>
<tr id="mrs_01_24239__row1292002219163"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p1892052219165">Configuring the Attributes of the Hudi Table: Custom Config</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p18920122181610">Custom configuration</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p129202221165">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row1844135012146"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.1 "><p id="mrs_01_24239__p084595051413">Execution Env</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.2 "><p id="mrs_01_24239__p2845105016141">Environment variable required for running the Hudi App. If no ENV is available, create one by referring to <a href="mrs_01_24255.html">Managing ENV</a>.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.9.2.4.1.3 "><p id="mrs_01_24239__p19845650151414">defaultEnv</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table13809103244916" frame="border" border="1" rules="all"><caption><b>Table 7 </b>Sink Kafka job parameters</caption><thead align="left"><tr id="mrs_01_24239__row2811632204917"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.10.2.4.1.1"><p id="mrs_01_24239__p128111932134919">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.10.2.4.1.2"><p id="mrs_01_24239__p781183264913">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.10.2.4.1.3"><p id="mrs_01_24239__p1581163220491">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row13811832154912"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.10.2.4.1.1 "><p id="mrs_01_24239__p20812732114913">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.10.2.4.1.2 "><p id="mrs_01_24239__p12812183244911">Created Kafka link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.10.2.4.1.3 "><p id="mrs_01_24239__p881223254914">kafkalink</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table07383379266" frame="border" border="1" rules="all"><caption><b>Table 8 </b>DWS job parameters</caption><thead align="left"><tr id="mrs_01_24239__row1739037142612"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.11.2.4.1.1"><p id="mrs_01_24239__p973903742618">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.11.2.4.1.2"><p id="mrs_01_24239__p173933762610">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.11.2.4.1.3"><p id="mrs_01_24239__p14739137132611">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row18740153716261"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.1 "><p id="mrs_01_24239__p774063718262">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.2 "><p id="mrs_01_24239__p1740183712261">Link used by Connector</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.3 "><p id="mrs_01_24239__p1774043732611">dwslink</p>
</td>
</tr>
<tr id="mrs_01_24239__row197406370261"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.1 "><p id="mrs_01_24239__p87401637122616">Query Timeout</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.2 "><p id="mrs_01_24239__p10941853192717">Timeout interval for connecting to DWS, in milliseconds</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.3 "><p id="mrs_01_24239__p42711737132810">180000</p>
</td>
</tr>
<tr id="mrs_01_24239__row1474583718267"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.1 "><p id="mrs_01_24239__p1274515377261">Batch Size</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.2 "><p id="mrs_01_24239__p1274523714266">Amount of data batch written to DWS</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.3 "><p id="mrs_01_24239__p7745153717263">50</p>
</td>
</tr>
<tr id="mrs_01_24239__row635454017511"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.1 "><p id="mrs_01_24239__p1235454013510">Sink Task Number</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.2 "><p id="mrs_01_24239__p835413404513">Maximum number of concurrent jobs when a table is written to DWS.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.3 "><p id="mrs_01_24239__p8354340115115">-</p>
</td>
</tr>
<tr id="mrs_01_24239__row15746637132614"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.1 "><p id="mrs_01_24239__p4746183710266">DWS Custom Config</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.2 "><p id="mrs_01_24239__p4746143762617">Custom configuration</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.11.2.4.1.3 "><p id="mrs_01_24239__p5746123772611">-</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="mrs_01_24239__p10846172041616">Table 11 ClickHouse job parameters</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24239__table2164201415178" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24239__row1516419141177"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.13.1.4.1.1"><p id="mrs_01_24239__p01645144179">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.13.1.4.1.2"><p id="mrs_01_24239__p161641114101710">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.3.2.13.1.4.1.3"><p id="mrs_01_24239__p1316517142178">Example Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24239__row91651814151714"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.1 "><p id="mrs_01_24239__p1916521481715">Link</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.2 "><p id="mrs_01_24239__p18165121419173">Link used by Connector</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.3 "><p id="mrs_01_24239__p16165714141713">dwslink</p>
</td>
</tr>
<tr id="mrs_01_24239__row227555214385"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.1 "><p id="mrs_01_24239__p12276165215386">Query Timeout</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.2 "><p id="mrs_01_24239__p82762524383">Timeout interval for connecting to ClickHouse, in milliseconds</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.3 "><p id="mrs_01_24239__p9276185215382">60000</p>
</td>
</tr>
<tr id="mrs_01_24239__row9525175915389"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.1 "><p id="mrs_01_24239__p1525105983815">Batch Size</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.2 "><p id="mrs_01_24239__p552514599387">Amount of data batch written to ClickHouse</p>
<div class="note" id="mrs_01_24239__note10187163933917"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_24239__p1718743910394">It is best practice to set this parameter to a large value. The recommended value range is 10000-100000.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.3.2.13.1.4.1.3 "><p id="mrs_01_24239__p12525105953811">100000</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="mrs_01_24239__li984825817403"><span>After the job parameters are configured, drag the two icons to associate the job parameters and click <strong id="mrs_01_24239__b991733720194">Save</strong>. The job configuration is complete.</span><p><p id="mrs_01_24239__p15185241533"><span><img id="mrs_01_24239__image138401924135313" src="en-us_image_0000001532951876.png"></span></p>
</p></li><li id="mrs_01_24239__li1997814344117"><span>In the job list on the <strong id="mrs_01_24239__b1689257072737">Job Management</strong> page, locate the created jobs, click <strong id="mrs_01_24239__b1911754246737">Start</strong> in the <strong id="mrs_01_24239__b1878629080737">Operation</strong> column, and wait until the jobs are started.</span><p><p id="mrs_01_24239__p171241147191215">Check whether the data transmission takes effect, for example, insert data into the table in the MySQL database and view the content of the file imported to Hudi.</p>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24774.html">Creating a CDL Job</a></div>
</div>
</div>
View Git Blame Copy Permalink