vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0949.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

150 lines
24 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001188642162"></a><a name="EN-US_TOPIC_0000001188642162"></a>
<h1 class="topictitle1">Importing Data from One GaussDB(DWS) Cluster to Another</h1>
<div id="body0000001125261886"><div class="section" id="EN-US_TOPIC_0000001188642162__en-us_topic_0059778169_s0867185fef0f4a228532d432b598cb26"><h4 class="sectiontitle">Function</h4><p id="EN-US_TOPIC_0000001188642162__p1479251195413">You can create foreign tables to perform associated queries and import data between clusters.</p>
</div>
<div class="section" id="EN-US_TOPIC_0000001188642162__section3206325195115"><h4 class="sectiontitle">Scenarios</h4><ul id="EN-US_TOPIC_0000001188642162__ul14161184010576"><li id="EN-US_TOPIC_0000001188642162__li3161040175715">Import data from one GaussDB(DWS) cluster to another.</li><li id="EN-US_TOPIC_0000001188642162__li53372414574">Perform associated queries between clusters.</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001188642162__section116519262516"><h4 class="sectiontitle">Precautions</h4><ul id="EN-US_TOPIC_0000001188642162__en-us_topic_0059777857_u61501e0cbe1d4aa7a840d26b0548d7a7"><li id="EN-US_TOPIC_0000001188642162__li682853162814">The two clusters must be in the same region and AZ, and can communicate with each other through the VPC network.</li><li id="EN-US_TOPIC_0000001188642162__li207801346131312">The created foreign table must be of the same type and have the same columns as its corresponding remote table, which can only be a row-store, column-store, hash, or replication table.</li><li id="EN-US_TOPIC_0000001188642162__en-us_topic_0059777857_l830d3ebb05f242ebb8db501c7fa4efc2">If the associated table in another cluster is a replication table or has data skew, the query performance may be poor.</li><li id="EN-US_TOPIC_0000001188642162__li12619390491">The status of the two clusters is <strong id="EN-US_TOPIC_0000001188642162__b1288144814559">Normal</strong>.</li><li id="EN-US_TOPIC_0000001188642162__li7397174420312">Do not modify, add, or delete the DDL of the source data table in the remote cluster. Otherwise, the query results may be inconsistent.</li><li id="EN-US_TOPIC_0000001188642162__li133223818486">The two clusters can process SQL on other GaussDB databases based on a foreign table.</li><li id="EN-US_TOPIC_0000001188642162__li1467325511587">Ensure that the two databases have the same encoding. Otherwise, an error may occur or the received data may be garbled characters.</li><li id="EN-US_TOPIC_0000001188642162__li652471371415">If statistics have been collected on the remote table, run <strong id="EN-US_TOPIC_0000001188642162__b116471735818">ANALYZE</strong> on the foreign table to obtain a better execution plan.</li><li id="EN-US_TOPIC_0000001188642162__li6364036202212">Only 8.0.0 and later versions are supported.</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001188642162__section99331110112918"><h4 class="sectiontitle">Procedure</h4><ol id="EN-US_TOPIC_0000001188642162__ol3775143614911"><li id="EN-US_TOPIC_0000001188642162__li10775113611499"><span>Create a server.</span><p><div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__screen11719387224"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="n">SERVER</span><span class="w"> </span><span class="n">server_remote</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">DATA</span><span class="w"> </span><span class="n">WRAPPER</span><span class="w"> </span><span class="n">GC_FDW</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="n">address</span><span class="w"> </span><span class="s1">'10.180.157.231:8000,10.180.157.130:8000'</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">dbname</span><span class="w"> </span><span class="s1">'gaussdb'</span><span class="p">,</span><span class="w"> </span>
<span class="w"> </span><span class="n">username</span><span class="w"> </span><span class="s1">'xyz'</span><span class="p">,</span><span class="w"> </span>
<span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="s1">'xxxxxx'</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
<div class="p" id="EN-US_TOPIC_0000001188642162__p119421420143818"><div class="note" id="EN-US_TOPIC_0000001188642162__en-us_topic_0059778673_ne7521c1949524f60b62f5e81622ba30e"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001188642162__ul5713143363911"><li id="EN-US_TOPIC_0000001188642162__li771420335398"><strong id="EN-US_TOPIC_0000001188642162__b196351091618">server_remote</strong> is the server name used for the foreign table.</li><li id="EN-US_TOPIC_0000001188642162__li998963924018"><strong id="EN-US_TOPIC_0000001188642162__b167131125141620">address</strong> indicates the IP addresses and port numbers of CNs in the remote cluster. If LVS is configured, you are advised to enter only one LVS address. Otherwise, you are advised to set multiple CNs as server addresses.</li><li id="EN-US_TOPIC_0000001188642162__li15760207424"><strong id="EN-US_TOPIC_0000001188642162__b3164540131712">dbname</strong> is the database name of the remote cluster.</li><li id="EN-US_TOPIC_0000001188642162__li20517192215428"><strong id="EN-US_TOPIC_0000001188642162__b157434918173">username</strong> is the username used for connecting to the remote cluster. This user cannot be a system administrator.</li><li id="EN-US_TOPIC_0000001188642162__li73381318154315"><strong id="EN-US_TOPIC_0000001188642162__b132412155184">password</strong> is the password used for logging in to the remote cluster.</li></ul>
</div></div>
</div>
</p></li><li id="EN-US_TOPIC_0000001188642162__li1877517363494"><span>Create a foreign table.</span><p><div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__en-us_topic_0059779123_screen5199172173727"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">region</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">R_REGIONKEY</span><span class="w"> </span><span class="n">INT4</span><span class="p">,</span>
<span class="w"> </span><span class="n">R_NAME</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span>
<span class="w"> </span><span class="n">R_COMMENT</span><span class="w"> </span><span class="nb">TEXT</span>
<span class="p">)</span>
<span class="n">SERVER</span>
<span class="w"> </span><span class="n">server_remote</span>
<span class="k">OPTIONS</span>
<span class="p">(</span>
<span class="w"> </span><span class="k">schema_name</span><span class="w"> </span><span class="s1">'test'</span><span class="p">,</span>
<span class="w"> </span><span class="k">table_name</span><span class="w"> </span><span class="s1">'region'</span><span class="p">,</span>
<span class="w"> </span><span class="k">encoding</span><span class="w"> </span><span class="s1">'gbk'</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
<div class="p" id="EN-US_TOPIC_0000001188642162__p1320619544463"><div class="note" id="EN-US_TOPIC_0000001188642162__note24609282452"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001188642162__ul1212634124819"><li id="EN-US_TOPIC_0000001188642162__li2601124655712">Foreign table columns cannot contain any constraints.</li><li id="EN-US_TOPIC_0000001188642162__li141318342485">The column names types of the foreign table must be the same as those of its corresponding remote table.</li><li id="EN-US_TOPIC_0000001188642162__li191315349482"><strong id="EN-US_TOPIC_0000001188642162__b383661013233">schema_name</strong> specifies the schema of the foreign table corresponding to the remote cluster. If this parameter is not specified, the default schema is used.</li><li id="EN-US_TOPIC_0000001188642162__li19576610145113"><strong id="EN-US_TOPIC_0000001188642162__b6853103674810">table_name</strong> specifies the name of the foreign table corresponding to the remote cluster. If this parameter is not specified, the default foreign table name is used.</li><li id="EN-US_TOPIC_0000001188642162__li628631514812"><strong id="EN-US_TOPIC_0000001188642162__b1026223719511">encoding</strong> specifies the encoding format of the remote cluster. If this parameter is not specified, the default encoding format is used.</li></ul>
</div></div>
</div>
</p></li><li id="EN-US_TOPIC_0000001188642162__li19775143610490"><span>View the foreign table.</span><p><div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__en-us_topic_0059779123_screen6393079417356"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span></pre></div></td><td class="code"><div><pre><span></span><span class="err">\</span><span class="n">d</span><span class="o">+</span><span class="w"> </span><span class="n">region</span>
<span class="w"> </span><span class="k">Foreign</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="ss">&quot;public.region&quot;</span>
<span class="w"> </span><span class="k">Column</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Type</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Modifiers</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">FDW</span><span class="w"> </span><span class="k">Options</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Storage</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Stats</span><span class="w"> </span><span class="n">target</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Description</span>
<span class="c1">-------------+---------+-----------+-------------+----------+--------------+-------------</span>
<span class="w"> </span><span class="n">r_regionkey</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">plain</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="n">r_name</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nb">text</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">extended</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="n">r_comment</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="nb">text</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">extended</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span>
<span class="n">Server</span><span class="p">:</span><span class="w"> </span><span class="n">server_remote</span>
<span class="n">FDW</span><span class="w"> </span><span class="k">Options</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="k">schema_name</span><span class="w"> </span><span class="s1">'test'</span><span class="p">,</span><span class="w"> </span><span class="k">table_name</span><span class="w"> </span><span class="s1">'region'</span><span class="p">,</span><span class="w"> </span><span class="k">encoding</span><span class="w"> </span><span class="s1">'gbk'</span><span class="p">)</span>
<span class="n">FDW</span><span class="w"> </span><span class="n">permition</span><span class="p">:</span><span class="w"> </span><span class="k">read</span><span class="w"> </span><span class="k">only</span>
<span class="n">Has</span><span class="w"> </span><span class="k">OIDs</span><span class="p">:</span><span class="w"> </span><span class="k">no</span>
<span class="n">Distribute</span><span class="w"> </span><span class="k">By</span><span class="p">:</span><span class="w"> </span><span class="n">ROUND</span><span class="w"> </span><span class="n">ROBIN</span>
<span class="k">Location</span><span class="w"> </span><span class="n">Nodes</span><span class="p">:</span><span class="w"> </span><span class="k">ALL</span><span class="w"> </span><span class="n">DATANODES</span>
</pre></div></td></tr></table></div>
</div>
</p></li><li id="EN-US_TOPIC_0000001188642162__li29254339918"><span>Check the created server.</span><p><div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__screen263024131212"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="err">\</span><span class="n">des</span><span class="o">+</span><span class="w"> </span><span class="n">server_remote</span>
<span class="w"> </span><span class="n">List</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="k">foreign</span><span class="w"> </span><span class="n">servers</span>
<span class="w"> </span><span class="n">Name</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Owner</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Foreign</span><span class="o">-</span><span class="k">data</span><span class="w"> </span><span class="n">wrapper</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Access</span><span class="w"> </span><span class="k">privileges</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Type</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">Version</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="n">FDW</span><span class="w"> </span><span class="k">Options</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Description</span>
<span class="c1">---------------+---------+----------------------+-------------------+------+---------+-----------------------------------------------------------------</span>
<span class="c1">-----------------------------------------------------------------------------------------------------------------+-------------</span>
<span class="w"> </span><span class="n">server_remote</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">dbadmin</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">gc_fdw</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="p">(</span><span class="n">address</span><span class="w"> </span><span class="s1">'10.180.157.231:8000,10.180.157.130:8000'</span><span class="p">,</span><span class="w"> </span><span class="n">dbname</span><span class="w"> </span><span class="s1">'gaussdb'</span>
<span class="p">,</span><span class="w"> </span><span class="n">username</span><span class="w"> </span><span class="s1">'xyz'</span><span class="p">,</span><span class="w"> </span><span class="n">password</span><span class="w"> </span><span class="s1">'xxxxxx'</span><span class="p">)</span><span class="w"> </span><span class="o">|</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</p></li><li id="EN-US_TOPIC_0000001188642162__li16775153644910"><span>Use the foreign table to import data or perform associated queries.</span><p><ul id="EN-US_TOPIC_0000001188642162__ul19822112321916"><li id="EN-US_TOPIC_0000001188642162__li144722019181920">Import data.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__screen3472719121914"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">local_region</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">R_REGIONKEY</span><span class="w"> </span><span class="n">INT4</span><span class="p">,</span>
<span class="w"> </span><span class="n">R_NAME</span><span class="w"> </span><span class="nb">TEXT</span><span class="p">,</span>
<span class="w"> </span><span class="n">R_COMMENT</span><span class="w"> </span><span class="nb">TEXT</span>
<span class="p">);</span>
<span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">local_region</span><span class="w"> </span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">region</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<div class="p" id="EN-US_TOPIC_0000001188642162__p152452367539"><div class="note" id="EN-US_TOPIC_0000001188642162__note416884524515"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001188642162__ul10234145616549"><li id="EN-US_TOPIC_0000001188642162__li142356560544">If a connection failure is reported, check the server information and ensure that the specified clusters are connected.</li><li id="EN-US_TOPIC_0000001188642162__li52361756175417">If an error is reported, indicating that the table does not exist, check whether the <strong id="EN-US_TOPIC_0000001188642162__b10620340124813">option</strong> information of the foreign table is correct.</li><li id="EN-US_TOPIC_0000001188642162__li1581513517564">If a column mismatch error is reported, check whether the column information of the foreign table is consistent with that of the corresponding table in the remote cluster.</li><li id="EN-US_TOPIC_0000001188642162__li10837131635714">If a version inconsistency error is reported, upgrade the cluster and try again.</li><li id="EN-US_TOPIC_0000001188642162__li51538501492">If garbled characters are displayed, check the encoding format of the source data, re-create a foreign table, and specify the correct coding format.</li></ul>
</div></div>
</div>
</li><li id="EN-US_TOPIC_0000001188642162__li2696732114612">Perform an associated query.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__screen146981632124619"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">region</span><span class="p">,</span><span class="w"> </span><span class="n">local_region</span><span class="w"> </span><span class="k">WHERE</span><span class="w"> </span><span class="n">local_region</span><span class="p">.</span><span class="n">R_NAME</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">region</span><span class="p">.</span><span class="n">R_NAME</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<div class="p" id="EN-US_TOPIC_0000001188642162__p5334193911582"><div class="note" id="EN-US_TOPIC_0000001188642162__note10910429463"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001188642162__ul11678115655910"><li id="EN-US_TOPIC_0000001188642162__li1760557502">A foreign table can be used as a local table to perform complex jobs.</li><li id="EN-US_TOPIC_0000001188642162__li18681115685920">If statistics have been collected on the remote cluster, run <strong id="EN-US_TOPIC_0000001188642162__b19482172818289">ANALYZE</strong> on the foreign table to obtain a better execution plan.</li><li id="EN-US_TOPIC_0000001188642162__li164751358675">If there are fewer DNs in the local cluster than in the remote cluster, the local cluster needs to use SMP for better performance.</li></ul>
</div></div>
</div>
</li></ul>
</p></li><li id="EN-US_TOPIC_0000001188642162__li877510361491"><span>Delete the foreign table.</span><p><div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642162__en-us_topic_0059779123_screen4163821517356"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">DROP</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">region</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0179.html">Importing Data</a></div>
</div>
</div>
View Git Blame Copy Permalink