forked from docs/doc-exports
Chen, Junjie
2b90aa9fff
Reviewed-by: Mützel, Andrea <andrea.muetzel@t-systems.com> Co-authored-by: Chen, Junjie <chenjunjie@huawei.com> Co-committed-by: Chen, Junjie <chenjunjie@huawei.com>
89 lines
15 KiB
HTML
89 lines
15 KiB
HTML
<a name="dcs-migrate-demo02"></a><a name="dcs-migrate-demo02"></a>
|
|
|
|
<h1 class="topictitle1">Self-Hosted Redis Cluster Migration with redis-shake</h1>
|
|
<div id="body1562070011766"><p id="dcs-migrate-demo02__p8060118">redis-shake is an open-source tool for migrating data online or offline (by importing backup files) between Redis Clusters. Data can be migrated to DCS Redis Cluster instances seamlessly because DCS Redis Cluster inherits the native Redis Cluster design.</p>
|
|
<p id="dcs-migrate-demo02__p1814154103513">The following describes how to use redis-shake to migrate data to a DCS Redis Cluster instance.</p>
|
|
<div class="section" id="dcs-migrate-demo02__section1127137154413"><h4 class="sectiontitle">Migrating Data Online</h4><p id="dcs-migrate-demo02__p33841304419">You can migrate data online from a self-hosted Redis Cluster to a DCS Redis Cluster instance as long as the two clusters are directly connected or connected through a transit server.</p>
|
|
<p id="dcs-migrate-demo02__p10133393441">Data in Redis Clusters of another cloud cannot be migrated online because the <strong id="dcs-migrate-demo02__b18590191983014">SYNC</strong> and <strong id="dcs-migrate-demo02__b16322112117309">PSYNC</strong> commands are disabled by some vendors.</p>
|
|
<ol id="dcs-migrate-demo02__ol19848121112139"><li id="dcs-migrate-demo02__li138131117443">Create a Redis Cluster instance on the DCS console.<p id="dcs-migrate-demo02__p16382142134417"><a name="dcs-migrate-demo02__li138131117443"></a><a name="li138131117443"></a>The memory of this instance cannot be smaller than that of the source Redis.</p>
|
|
</li><li id="dcs-migrate-demo02__li8848101114133">Prepare a cloud server and install redis-shake.<p id="dcs-migrate-demo02__p1236334153517"><a name="dcs-migrate-demo02__li8848101114133"></a><a name="li8848101114133"></a>redis-shake must be able to access both the source and target Redis. Bound an EIP to the cloud server.</p>
|
|
<p id="dcs-migrate-demo02__p7453161552818">You can use ECS and configure the same VPC, subnet, and security group for the ECS and the DCS instance. If the source Redis is deployed on cloud servers of another cloud, allow public access to the servers.</p>
|
|
<p id="dcs-migrate-demo02__dcs-migrate-demo02_p88217579422"><a href="https://github.com/tair-opensource/RedisShake/releases/download/release-v2.1.2-20220329/release-v2.1.2-20220329.tar.gz" target="_blank" rel="noopener noreferrer">Download</a> and decompress the release version of redis-shake. (The following uses v2.1.2 as an example. You can also use <a href="https://github.com/alibaba/RedisShake/releases" target="_blank" rel="noopener noreferrer">other redis-shake versions</a>.)</p>
|
|
<p id="dcs-migrate-demo02__dcs-migrate-demo02_p15821257194213"><span><img id="dcs-migrate-demo02__dcs-migrate-demo02_image1782145704210" src="en-us_image_0000001420267897.png"></span></p>
|
|
</li><li id="dcs-migrate-demo02__li284891111130">Locate the masters of the source and target Redis Clusters and obtain the IP addresses of the masters.<p id="dcs-migrate-demo02__p918255413395"><a name="dcs-migrate-demo02__li284891111130"></a><a name="li284891111130"></a>Online data migration must be performed node by node. Run the following command to query the IP addresses and port numbers of all nodes in both the source and target Redis Clusters.</p>
|
|
<p id="dcs-migrate-demo02__p64034338434"><strong id="dcs-migrate-demo02__b1740393319431">redis-cli -h <em id="dcs-migrate-demo02__i174031933184315">{redis_address}</em> -p <em id="dcs-migrate-demo02__i44036333433">{redis_port}</em> -a <em id="dcs-migrate-demo02__i184031833204314">{redis_password}</em> cluster nodes</strong></p>
|
|
<p id="dcs-migrate-demo02__p12403163316433">In the command output similar to the following, obtain the IP addresses and ports of all masters.</p>
|
|
<p id="dcs-migrate-demo02__p1612719464562"><span><img id="dcs-migrate-demo02__image19820346211" src="en-us_image_0177600650.png"></span></p>
|
|
<div class="note" id="dcs-migrate-demo02__note7693550185411"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dcs-migrate-demo02__p136936501544">After Redis is installed, it runs with redis-cli. To install Redis on CentOS, run the <strong id="dcs-migrate-demo02__b44791937105511">yum install redis</strong> command.</p>
|
|
</div></div>
|
|
</li><li id="dcs-migrate-demo02__li984815114135">Edit the redis-shake configuration file.<p id="dcs-migrate-demo02__p165301315101"><a name="dcs-migrate-demo02__li984815114135"></a><a name="li984815114135"></a>Edit the <strong id="dcs-migrate-demo02__b15129291552">redis-shake.conf</strong> file by providing the following information about all the masters of both the source and the target:</p>
|
|
<pre class="screen" id="dcs-migrate-demo02__screen24734391550">source.type = cluster
|
|
# If there is no password, skip the following parameter.
|
|
source.password_raw = {source_redis_password}
|
|
# IP addresses and port numbers of all masters of the source Redis Cluster, which are separated by semicolons (;).
|
|
source.address = {master1_ip}:{master1_port};{master2_ip}:{master2_port}…{masterN_ip}:{masterN_port}
|
|
target.type = cluster
|
|
# If there is no password, skip the following parameter.
|
|
target.password_raw = {target_redis_password}
|
|
# IP addresses and port numbers of all masters of the target instance, which are separated by semicolons (;).
|
|
target.address = {master1_ip}:{master1_port};{master2_ip}:{master2_port}…{masterN_ip}:{masterN_port}</pre>
|
|
<p id="dcs-migrate-demo02__p253913201019">Save and exit.</p>
|
|
</li><li id="dcs-migrate-demo02__li13477121417134">Migrate data online.<p id="dcs-migrate-demo02__p133087271005"><a name="dcs-migrate-demo02__li13477121417134"></a><a name="li13477121417134"></a>Run the following command to synchronize data between the source and the target Redis:</p>
|
|
<p id="dcs-migrate-demo02__p830810272002"><strong id="dcs-migrate-demo02__b03085277020">./redis-shake -type sync -conf redis-shake.conf</strong></p>
|
|
<p id="dcs-migrate-demo02__p53081327906">If the following information is displayed, the full synchronization has been completed and incremental synchronization begins.</p>
|
|
<pre class="screen" id="dcs-migrate-demo02__screen8323174812">sync rdb done.</pre>
|
|
<p id="dcs-migrate-demo02__p11308152711011">If the following information is displayed, no new data is incremented. You can stop the incremental synchronization by pressing <strong id="dcs-migrate-demo02__b852141612475">Ctrl</strong>+<strong id="dcs-migrate-demo02__b1351220201478">C</strong>.</p>
|
|
<pre class="screen" id="dcs-migrate-demo02__screen14186891677">sync: +forwardCommands=0 +filterCommands=0 +writeBytes=0</pre>
|
|
<div class="fignone" id="dcs-migrate-demo02__fig1758112170323"><span class="figcap"><b>Figure 1 </b>Online migration using redis-shake</span><br><span><img id="dcs-migrate-demo02__image18502120133014" src="en-us_image_0177653842.png"></span></div>
|
|
</li><li id="dcs-migrate-demo02__li1362193441315">Verify the migration.<p id="dcs-migrate-demo02__p16921143609"><a name="dcs-migrate-demo02__li1362193441315"></a><a name="li1362193441315"></a>After data synchronization, access the DCS Redis Cluster instance using redis-cli. Run the <strong id="dcs-migrate-demo02__b4901173855318">info</strong> command to query the number of keys in the <strong id="dcs-migrate-demo02__b1424873735415">Keyspace</strong> section to confirm that data has been fully imported.</p>
|
|
<p id="dcs-migrate-demo02__p492154311014">If the data has not been fully imported, run the <strong id="dcs-migrate-demo02__b12143507542">flushall</strong> or <strong id="dcs-migrate-demo02__b221515012545">flushdb</strong> command to clear the cached data in the instance, and synchronize data again.</p>
|
|
</li><li id="dcs-migrate-demo02__li18136153116519">Clear the redis-shake configuration file.</li></ol>
|
|
</div>
|
|
<div class="section" id="dcs-migrate-demo02__section2230197183316"><h4 class="sectiontitle">Importing Backup Files</h4><p id="dcs-migrate-demo02__p1897312853312">If the source Redis and the destitution Redis cannot be connected, or the source Redis is deployed on other clouds, you can migrate data by importing backup files.</p>
|
|
<p id="dcs-migrate-demo02__p1937501419433"></p>
|
|
<ol id="dcs-migrate-demo02__ol082075784215"><li id="dcs-migrate-demo02__li18820145718421">Create a Redis Cluster instance on the DCS console.<p id="dcs-migrate-demo02__p17820457194216"><a name="dcs-migrate-demo02__li18820145718421"></a><a name="li18820145718421"></a>The memory of this instance cannot be smaller than that of the source Redis.</p>
|
|
</li><li id="dcs-migrate-demo02__li104985015119">Run the following command to obtain the IP addresses and port numbers of all masters of the source Redis and target Redis:<p id="dcs-migrate-demo02__p522517529213"><a name="dcs-migrate-demo02__li104985015119"></a><a name="li104985015119"></a><strong id="dcs-migrate-demo02__b15225175214219">redis-cli -h <em id="dcs-migrate-demo02__i1422585211213">{redis_address}</em> -p <em id="dcs-migrate-demo02__i1722525219212">{redis_port}</em> -a <em id="dcs-migrate-demo02__i1022516521523">{redis_password}</em> cluster nodes</strong></p>
|
|
<p id="dcs-migrate-demo02__p6225352528">In the command output similar to the following, obtain the IP addresses and ports of all masters.</p>
|
|
<p id="dcs-migrate-demo02__p1222510522212"><span><img id="dcs-migrate-demo02__image122251352628" src="en-us_image_0177654980.png"></span></p>
|
|
<div class="note" id="dcs-migrate-demo02__note222585210212"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dcs-migrate-demo02__p5225252728">After Redis is installed, it runs with redis-cli. To install Redis on CentOS, run the <strong id="dcs-migrate-demo02__b44671243078">yum install redis</strong> command.</p>
|
|
</div></div>
|
|
</li><li id="dcs-migrate-demo02__li18820857174214">Prepare a cloud server and install redis-shake.<p id="dcs-migrate-demo02__p1682145784219"><a name="dcs-migrate-demo02__li18820857174214"></a><a name="li18820857174214"></a>redis-shake must be able to access the target Redis and bound to an EIP.</p>
|
|
<p id="dcs-migrate-demo02__p1082175714429">You can use ECS and configure the same VPC, subnet, and security group for the ECS and the DCS instance.</p>
|
|
<p id="dcs-migrate-demo02__p1073511343447"><a href="https://github.com/tair-opensource/RedisShake/releases/download/release-v2.1.2-20220329/release-v2.1.2-20220329.tar.gz" target="_blank" rel="noopener noreferrer">Download</a> and decompress the release version of redis-shake. (The following uses v2.1.2 as an example.)</p>
|
|
<p id="dcs-migrate-demo02__p1073517349442"><span><img id="dcs-migrate-demo02__image1735234114418" src="en-us_image_0000001420151641.png"></span></p>
|
|
<div class="note" id="dcs-migrate-demo02__note1279444614610"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dcs-migrate-demo02__p979434611610">If the source Redis is deployed in the data center intranet, install redis-shake on the intranet server. Export data and then upload the data to the cloud server as instructed by the following steps</p>
|
|
</div></div>
|
|
</li><li id="dcs-migrate-demo02__li31771323165319">Export the RDB file.<ul id="dcs-migrate-demo02__ul181391212136"><li id="dcs-migrate-demo02__li5813141221318">Edit the <strong id="dcs-migrate-demo02__b165661730896">redis-shake.conf</strong> file by providing the following information about all the masters of both the source and the target:<pre class="screen" id="dcs-migrate-demo02__screen1248155110107">source.type = cluster
|
|
# If there is no password, skip the following parameter.
|
|
source.password_raw = {source_redis_password}
|
|
# IP addresses and port numbers of all masters of the source Redis Cluster, which are separated by semicolons (;).
|
|
source.address = {master1_ip}:{master1_port};{master2_ip}:{master2_port}…{masterN_ip}:{masterN_port}</pre>
|
|
</li><li id="dcs-migrate-demo02__li8813201217136">Run the following command to export the RDB file:<p id="dcs-migrate-demo02__p331212346137"><a name="dcs-migrate-demo02__li8813201217136"></a><a name="li8813201217136"></a><strong id="dcs-migrate-demo02__b1931273412131">./redis-shake -type dump -conf redis-shake.conf</strong></p>
|
|
<p id="dcs-migrate-demo02__p0312193411319">If the following information is displayed in the execution log, the backup file is exported successfully:</p>
|
|
<pre class="screen" id="dcs-migrate-demo02__screen5845144815132">execute runner[*run.CmdDump] finished!</pre>
|
|
</li></ul>
|
|
</li><li id="dcs-migrate-demo02__li2088910245149">Import the RDB file.<ol type="a" id="dcs-migrate-demo02__ol2327112018216"><li id="dcs-migrate-demo02__li6327192011215">Import the RDB file (or files) to the cloud server. The cloud server must be connected to the target DCS instance.</li><li id="dcs-migrate-demo02__li832782018219">Edit the redis-shake configuration file.<div class="p" id="dcs-migrate-demo02__p1825892672116"><a name="dcs-migrate-demo02__li832782018219"></a><a name="li832782018219"></a>Edit the <strong id="dcs-migrate-demo02__b5238125171018">redis-shake.conf</strong> file by providing the following information about all the masters of both the source and the target:<pre class="screen" id="dcs-migrate-demo02__screen1882155716429">target.type = cluster
|
|
# If there is no password, skip the following parameter.
|
|
target.password_raw = {target_redis_password}
|
|
# IP addresses and port numbers of all masters of the target instance, which are separated by semicolons (;).
|
|
target.address = {master1_ip}:{master1_port};{master2_ip}:{master2_port}…{masterN_ip}:{masterN_port}
|
|
# List the RDB files to be imported, separated by semicolons (;).
|
|
rdb.input = {local_dump.0};{local_dump.1};{local_dump.2};{local_dump.3}</pre>
|
|
</div>
|
|
<p id="dcs-migrate-demo02__p2840103122112">Save and exit.</p>
|
|
</li><li id="dcs-migrate-demo02__li1132710201214">Run the following command to import the RDB file to the target instance:<p id="dcs-migrate-demo02__p592783512112"><a name="dcs-migrate-demo02__li1132710201214"></a><a name="li1132710201214"></a><strong id="dcs-migrate-demo02__b89111406213">./redis-shake -type restore -conf redis-shake.conf</strong></p>
|
|
<p id="dcs-migrate-demo02__p123611640122113">If the following information is displayed in the execution log, the backup file is imported successfully:</p>
|
|
<pre class="screen" id="dcs-migrate-demo02__screen152647172316">Enabled http stats, set status (incr), and wait forever.</pre>
|
|
</li></ol>
|
|
</li><li id="dcs-migrate-demo02__li1382275724216">Verify the migration.<p id="dcs-migrate-demo02__p1682214570429"><a name="dcs-migrate-demo02__li1382275724216"></a><a name="li1382275724216"></a>After data synchronization, access the DCS Redis Cluster instance using redis-cli. Run the <strong id="dcs-migrate-demo02__b05331233101217">info</strong> command to query the number of keys in the <strong id="dcs-migrate-demo02__b0427123710120">Keyspace</strong> section to confirm that data has been fully imported.</p>
|
|
<p id="dcs-migrate-demo02__p0822357184215">If the data has not been fully imported, run the <strong id="dcs-migrate-demo02__b1185520418128">flushall</strong> or <strong id="dcs-migrate-demo02__b1585610471215">flushdb</strong> command to clear the cached data in the instance, and synchronize data again.</p>
|
|
</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dcs-migration-0312002.html">Migrating Data from Self-Hosted Redis to DCS</a></div>
|
|
</div>
|
|
</div>
|
|
|