Yang, Tong 6182f91ba8 MRS component operation guide_normal 2.0.38.SP20 version
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2022-12-09 14:55:21 +00:00

53 lines
9.4 KiB
HTML

<a name="mrs_01_24198"></a><a name="mrs_01_24198"></a>
<h1 class="topictitle1">Using the ClickHouse Data Migration Tool</h1>
<div id="body0000001136546900"><p id="mrs_01_24198__p176811527111117">The ClickHouse data migration tool can migrate some partitions of one or more partitioned MergeTree tables on several ClickHouseServer nodes to the same tables on other ClickHouseServer nodes. In the capacity expansion scenario, you can use this tool to migrate data from an original node to a new node to balance data after capacity expansion.</p>
<div class="section" id="mrs_01_24198__section135233106402"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_24198__ul1366411569378"><li id="mrs_01_24198__li16664115614371">The ClickHouse and Zookeeper services are running properly. The ClickHouseServer instances on the source and destination nodes are normal.</li><li id="mrs_01_24198__li13180716392">The destination node has the data table to be migrated and the table is a partitioned MergeTree table.</li><li id="mrs_01_24198__li68491131104015">Before creating a migration task, ensure that all tasks for writing data to a table to be migrated have been stopped. After the task is started, you can only query the table to be migrated and cannot write data to or delete data from the table. Otherwise, data may be inconsistent before and after the migration.</li><li id="mrs_01_24198__li1631232264013">The ClickHouse data directory on the destination node has sufficient space.</li></ul>
</div>
<div class="section" id="mrs_01_24198__section1041216526381"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24198__ol466517515449"><li id="mrs_01_24198__li17664185174415"><span>Log in to Manager and choose <strong id="mrs_01_24198__b36417385543741">Cluster</strong> &gt; <strong id="mrs_01_24198__b16015138843741">Services</strong> &gt; <strong id="mrs_01_24198__b25756644243741">ClickHouse</strong>. On the ClickHouse service page, click the <strong id="mrs_01_24198__b954828643741">Data Migration</strong> tab.</span><p><p id="mrs_01_24198__p2664135164419"></p>
</p></li><li id="mrs_01_24198__li106653518442"><span>Click <strong id="mrs_01_24198__b185335751243741">Add Task</strong>.</span><p><p id="mrs_01_24198__p4665251184418"></p>
</p></li></ol><ol start="3" id="mrs_01_24198__ol266495154417"><li id="mrs_01_24198__li18211183015203"><span>On the page for creating a migration task, set the migration task parameters. For details, see <a href="#mrs_01_24198__table1724256152117">Table 1</a>.</span><p><p id="mrs_01_24198__p131077571306"></p>
<div class="tablenoborder"><a name="mrs_01_24198__table1724256152117"></a><a name="table1724256152117"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24198__table1724256152117" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Migration task parameters</caption><thead align="left"><tr id="mrs_01_24198__row137242561217"><th align="left" class="cellrowborder" valign="top" width="16.08%" id="mcps1.3.3.3.1.2.2.2.3.1.1"><p id="mrs_01_24198__p9724956122115">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="83.91999999999999%" id="mcps1.3.3.3.1.2.2.2.3.1.2"><p id="mrs_01_24198__p472413569211">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24198__row1272412569214"><td class="cellrowborder" valign="top" width="16.08%" headers="mcps1.3.3.3.1.2.2.2.3.1.1 "><p id="mrs_01_24198__p127241256132114">Task Name</p>
</td>
<td class="cellrowborder" valign="top" width="83.91999999999999%" headers="mcps1.3.3.3.1.2.2.2.3.1.2 "><p id="mrs_01_24198__p16903162412233">Enter a specific task name. The value can contain 1 to 50 characters, including letters, arrays, and underscores (_), and cannot be the same as that of an existing migration task.</p>
</td>
</tr>
<tr id="mrs_01_24198__row201022590231"><td class="cellrowborder" valign="top" width="16.08%" headers="mcps1.3.3.3.1.2.2.2.3.1.1 "><p id="mrs_01_24198__p9103159122314">Task Type</p>
</td>
<td class="cellrowborder" valign="top" width="83.91999999999999%" headers="mcps1.3.3.3.1.2.2.2.3.1.2 "><ul id="mrs_01_24198__ul197102410243"><li id="mrs_01_24198__li3719245244"><strong id="mrs_01_24198__b14541296243741">Scheduled Task</strong>: When the scheduled task is selected, you can set <strong id="mrs_01_24198__b120791045143741">Started</strong> to specify a time point later than the current time to execute the task.</li><li id="mrs_01_24198__li1540830182410"><strong id="mrs_01_24198__b190696367643741">Immediate task</strong>: The task is executed immediately after it is started.</li></ul>
</td>
</tr>
<tr id="mrs_01_24198__row139217266276"><td class="cellrowborder" valign="top" width="16.08%" headers="mcps1.3.3.3.1.2.2.2.3.1.1 "><p id="mrs_01_24198__p129352622714">Started</p>
</td>
<td class="cellrowborder" valign="top" width="83.91999999999999%" headers="mcps1.3.3.3.1.2.2.2.3.1.2 "><p id="mrs_01_24198__p89521736192719">Set this parameter when <strong id="mrs_01_24198__b100129550643741">Task Type</strong> is set to <strong id="mrs_01_24198__b39770578943741">Scheduled Task</strong>. The valid value is a time point within 90 days from now.</p>
</td>
</tr>
</tbody>
</table>
</div>
</p></li><li id="mrs_01_24198__li962162204719"><span>On the <strong id="mrs_01_24198__b106879775743741">Select Node</strong> page, specify <strong id="mrs_01_24198__b113538508743741">Source Node Host Name</strong> and <strong id="mrs_01_24198__b91523162143741">Destination Node Host Name</strong>, and click <strong id="mrs_01_24198__b30322705943741">Next</strong>.</span><p><div class="p" id="mrs_01_24198__p18752794472"><div class="note" id="mrs_01_24198__note131789372488"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_24198__ul2427203173712"><li id="mrs_01_24198__li1795297164114">Only one host name can be entered in <strong id="mrs_01_24198__b159031699043741">Source Node Host Name</strong> and <strong id="mrs_01_24198__b79033275243741">Destination Node Host Name</strong>, respectively. Multi-node migration is not supported.<p id="mrs_01_24198__p75131118418">To obtain the parameter values, click the <strong id="mrs_01_24198__b192966930043741">Instance</strong> tab on the ClickHouse service page and view the <strong id="mrs_01_24198__b24953829443741">Host Name</strong> column of the current ClickHouseServer instance.</p>
</li><li id="mrs_01_24198__li339953343711"><strong id="mrs_01_24198__b120408332243741">Maximum Bandwidth</strong> is optional. If it is not specified, there is no upper limit. The maximum bandwidth can be set to <strong id="mrs_01_24198__b88217714843741">10000</strong> MB/s.</li></ul>
</div></div>
</div>
</p></li><li id="mrs_01_24198__li14328732105418"><span>On the <strong id="mrs_01_24198__b146696648743741">Select Data Table</strong> page, click <span><img id="mrs_01_24198__image17328113212549" src="en-us_image_0000001349170269.png"></span> next to <strong id="mrs_01_24198__b214356550443741">Database</strong>, select the database to be migrated on the source node, and select the data table to be migrated for <strong id="mrs_01_24198__b7708180143741">Data Table</strong>. The data table drop-down list displays the partitioned MergeTree tables in the selected database. In the <strong id="mrs_01_24198__b98833755943741">Node Information</strong> area, the space usage of the ClickHouse service data directory on the current source and destination nodes is displayed. Click <strong id="mrs_01_24198__b205875581643741">Next</strong>.</span><p><p id="mrs_01_24198__p685212359549"></p>
</p></li><li id="mrs_01_24198__li16513141410554"><span>Confirm the task information and click <strong id="mrs_01_24198__b24258809143741">Submit</strong>.</span><p><p id="mrs_01_24198__p41571131164719">The data migration tool automatically calculates the partitions to be migrated based on the size of the data table. The amount of data to be migrated is the total size of the partitions to be migrated.</p>
<p id="mrs_01_24198__p2011461625510"></p>
</p></li><li id="mrs_01_24198__li13031048164711"><span>After the migration task is submitted, click <strong id="mrs_01_24198__b141637131043741">Start</strong> in the <strong id="mrs_01_24198__b115324939843741">Operation </strong>column. If the task is an immediate task, the task starts to be executed. If the task is a scheduled task, the countdown starts.</span><p><p id="mrs_01_24198__p99041851164710"></p>
</p></li><li id="mrs_01_24198__li8264388117"><span>During the migration task execution, you can click <strong id="mrs_01_24198__b17204283943741">Cancel</strong> to cancel the migration task that is being executed. If you cancel the task, the migrated data on the destination node will be rolled back.</span><p><p id="mrs_01_24198__p102551999111">You can choose <strong id="mrs_01_24198__b115661169043741">More </strong>&gt; <strong id="mrs_01_24198__b53684025843741">Details</strong> to view the log information during the migration.</p>
</p></li><li id="mrs_01_24198__li25232034442"><span>After the migration is complete, choose <strong id="mrs_01_24198__b19375385743741">More</strong> &gt; <strong id="mrs_01_24198__b89893508643741">Results</strong> to view the migration result and choose <strong id="mrs_01_24198__b95997519543741">More</strong> &gt; <strong id="mrs_01_24198__b26216866643741">Delete</strong> to delete the directories related to the migration task on ZooKeeper and the source node.</span></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24250.html">Migrating ClickHouse Data</a></div>
</div>
</div>