forked from laiweijian4/doc-exports
Lu, Huayi
a24ca60074
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Lu, Huayi <luhuayi@huawei.com> Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
84 lines
13 KiB
HTML
84 lines
13 KiB
HTML
<a name="EN-US_TOPIC_0000001125261886"></a><a name="EN-US_TOPIC_0000001125261886"></a>
|
|
|
|
<h1 class="topictitle1">Importing Data from One GaussDB(DWS) Cluster to Another</h1>
|
|
<div id="body0000001125261886"><div class="section" id="EN-US_TOPIC_0000001125261886__en-us_topic_0059778169_s0867185fef0f4a228532d432b598cb26"><h4 class="sectiontitle">Function</h4><p id="EN-US_TOPIC_0000001125261886__p1479251195413">You can create foreign tables to perform associated queries and import data between clusters.</p>
|
|
</div>
|
|
<div class="section" id="EN-US_TOPIC_0000001125261886__section3206325195115"><h4 class="sectiontitle">Scenarios</h4><ul id="EN-US_TOPIC_0000001125261886__ul14161184010576"><li id="EN-US_TOPIC_0000001125261886__li3161040175715">Import data from one GaussDB(DWS) cluster to another.</li><li id="EN-US_TOPIC_0000001125261886__li53372414574">Perform associated queries between clusters.</li></ul>
|
|
</div>
|
|
<div class="section" id="EN-US_TOPIC_0000001125261886__section116519262516"><h4 class="sectiontitle">Precautions</h4><ul id="EN-US_TOPIC_0000001125261886__en-us_topic_0059777857_u61501e0cbe1d4aa7a840d26b0548d7a7"><li id="EN-US_TOPIC_0000001125261886__li682853162814">The two clusters must be in the same region and AZ, and can communicate with each other through the VPC network.</li><li id="EN-US_TOPIC_0000001125261886__li207801346131312">The created foreign table must be of the same type and have the same columns as its corresponding remote table, which can only be a row-store, column-store, hash, or replication table.</li><li id="EN-US_TOPIC_0000001125261886__en-us_topic_0059777857_l830d3ebb05f242ebb8db501c7fa4efc2">If the associated table in another cluster is a replication table or has data skew, the query performance may be poor.</li><li id="EN-US_TOPIC_0000001125261886__li12619390491">The status of the two clusters is <strong id="EN-US_TOPIC_0000001125261886__b1288144814559">Normal</strong>.</li><li id="EN-US_TOPIC_0000001125261886__li7397174420312">Do not modify, add, or delete the DDL of the source data table in the remote cluster. Otherwise, the query results may be inconsistent.</li><li id="EN-US_TOPIC_0000001125261886__li133223818486">The two clusters can process SQL on other GaussDB databases based on a foreign table.</li><li id="EN-US_TOPIC_0000001125261886__li9459195515813">You are advised to configure LVS. If it is not configured, you are advised to use multiple CNs as the server addresses. Do not write the CN addresses of multiple clusters together.</li><li id="EN-US_TOPIC_0000001125261886__li1467325511587">Ensure that the two databases have the same encoding. Otherwise, an error may occur or the received data may be garbled characters.</li><li id="EN-US_TOPIC_0000001125261886__li652471371415">If statistics have been collected on the remote table, run <strong id="EN-US_TOPIC_0000001125261886__b116471735818">ANALYZE</strong> on the foreign table to obtain a better execution plan.</li><li id="EN-US_TOPIC_0000001125261886__li6364036202212">Only 8.0.0 and later versions are supported.</li></ul>
|
|
</div>
|
|
<div class="section" id="EN-US_TOPIC_0000001125261886__section99331110112918"><h4 class="sectiontitle">Procedure</h4><ol id="EN-US_TOPIC_0000001125261886__ol3775143614911"><li id="EN-US_TOPIC_0000001125261886__li10775113611499"><span>Create a server.</span><p><pre class="screen" id="EN-US_TOPIC_0000001125261886__screen11719387224"><span id="EN-US_TOPIC_0000001125261886__text2301927133818"></span>CREATE SERVER server_remote FOREIGN DATA WRAPPER GC_FDW OPTIONS
|
|
(address '<em id="EN-US_TOPIC_0000001125261886__i73394197347">10.180.157.231</em>:8000,<em id="EN-US_TOPIC_0000001125261886__i13959142953415">10.180.157.130</em>:8000' ,
|
|
dbname 'gaussdb',
|
|
username '<em id="EN-US_TOPIC_0000001125261886__i2638635184215">xyz</em>',
|
|
password '<em id="EN-US_TOPIC_0000001125261886__i11411317133213">xxxxxx</em>'
|
|
);</pre>
|
|
<div class="p" id="EN-US_TOPIC_0000001125261886__p119421420143818"><div class="note" id="EN-US_TOPIC_0000001125261886__en-us_topic_0059778673_ne7521c1949524f60b62f5e81622ba30e"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001125261886__ul5713143363911"><li id="EN-US_TOPIC_0000001125261886__li771420335398"><strong id="EN-US_TOPIC_0000001125261886__b196351091618">server_remote</strong> is the server name used for the foreign table.</li><li id="EN-US_TOPIC_0000001125261886__li998963924018"><strong id="EN-US_TOPIC_0000001125261886__b167131125141620">address</strong> indicates the IP addresses and port numbers of CNs in the remote cluster. If LVS is configured, you are advised to enter only one LVS address. Otherwise, you are advised to set multiple CNs as server addresses.</li><li id="EN-US_TOPIC_0000001125261886__li15760207424"><strong id="EN-US_TOPIC_0000001125261886__b3164540131712">dbname</strong> is the database name of the remote cluster.</li><li id="EN-US_TOPIC_0000001125261886__li20517192215428"><strong id="EN-US_TOPIC_0000001125261886__b157434918173">username</strong> is the username used for connecting to the remote cluster. This user cannot be a system administrator.</li><li id="EN-US_TOPIC_0000001125261886__li73381318154315"><strong id="EN-US_TOPIC_0000001125261886__b132412155184">password</strong> is the password used for logging in to the remote cluster.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
</p></li><li id="EN-US_TOPIC_0000001125261886__li1877517363494"><span>Create a foreign table.</span><p><pre class="screen" id="EN-US_TOPIC_0000001125261886__en-us_topic_0059779123_screen5199172173727"><span id="EN-US_TOPIC_0000001125261886__text1932121211388"></span> CREATE FOREIGN TABLE region
|
|
(
|
|
R_REGIONKEY INT4,
|
|
R_NAME TEXT,
|
|
R_COMMENT TEXT
|
|
)
|
|
SERVER
|
|
server_remote
|
|
OPTIONS
|
|
(
|
|
schema_name 'test',
|
|
table_name 'region',
|
|
encoding 'gbk'
|
|
);</pre>
|
|
<div class="p" id="EN-US_TOPIC_0000001125261886__p1320619544463"><div class="note" id="EN-US_TOPIC_0000001125261886__note24609282452"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001125261886__ul1212634124819"><li id="EN-US_TOPIC_0000001125261886__li2601124655712">Foreign table columns cannot contain any constraints.</li><li id="EN-US_TOPIC_0000001125261886__li141318342485">The column names types of the foreign table must be the same as those of its corresponding remote table.</li><li id="EN-US_TOPIC_0000001125261886__li191315349482"><strong id="EN-US_TOPIC_0000001125261886__b383661013233">schema_name</strong> specifies the schema of the foreign table corresponding to the remote cluster. If this parameter is not specified, the default schema is used.</li><li id="EN-US_TOPIC_0000001125261886__li19576610145113"><strong id="EN-US_TOPIC_0000001125261886__b6853103674810">table_name</strong> specifies the name of the foreign table corresponding to the remote cluster. If this parameter is not specified, the default foreign table name is used.</li><li id="EN-US_TOPIC_0000001125261886__li628631514812"><strong id="EN-US_TOPIC_0000001125261886__b1026223719511">encoding</strong> specifies the encoding format of the remote cluster. If this parameter is not specified, the default encoding format is used.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
</p></li><li id="EN-US_TOPIC_0000001125261886__li19775143610490"><span>View the foreign table.</span><p><pre class="screen" id="EN-US_TOPIC_0000001125261886__en-us_topic_0059779123_screen6393079417356"><span id="EN-US_TOPIC_0000001125261886__text26571406380"></span>\d+ region
|
|
|
|
Foreign table "public.region"
|
|
Column | Type | Modifiers | FDW Options | Storage | Stats target | Description
|
|
-------------+---------+-----------+-------------+----------+--------------+-------------
|
|
r_regionkey | integer | | | plain | |
|
|
r_name | text | | | extended | |
|
|
r_comment | text | | | extended | |
|
|
Server: server_remote
|
|
FDW Options: (schema_name 'test', table_name 'region', encoding 'gbk')
|
|
FDW permition: read only
|
|
Has OIDs: no
|
|
Distribute By: ROUND ROBIN
|
|
Location Nodes: ALL DATANODES</pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001125261886__li29254339918"><span>Check the created server.</span><p><pre class="screen" id="EN-US_TOPIC_0000001125261886__screen263024131212"><span id="EN-US_TOPIC_0000001125261886__text18686188126"></span>\des+ server_remote
|
|
List of foreign servers
|
|
Name | Owner | Foreign-data wrapper | Access privileges | Type | Version |
|
|
FDW Options | Description
|
|
---------------+---------+----------------------+-------------------+------+---------+-----------------------------------------------------------------
|
|
-----------------------------------------------------------------------------------------------------------------+-------------
|
|
server_remote | dbadmin | gc_fdw | | | | (address '<em id="EN-US_TOPIC_0000001125261886__i184307351313">10.180.157.231</em>:8000,<em id="EN-US_TOPIC_0000001125261886__i1543093151310">10.180.157.130</em>:8000', dbname 'gaussdb'
|
|
, username '<em id="EN-US_TOPIC_0000001125261886__i19211155161310">xyz</em>', password '<em id="EN-US_TOPIC_0000001125261886__i20249144717132">xxx</em>xxx') |
|
|
(1 row)</pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001125261886__li16775153644910"><span>Use the foreign table to import data or perform associated queries.</span><p><ul id="EN-US_TOPIC_0000001125261886__ul19822112321916"><li id="EN-US_TOPIC_0000001125261886__li144722019181920">Import data.<pre class="screen" id="EN-US_TOPIC_0000001125261886__screen3472719121914"><span id="EN-US_TOPIC_0000001125261886__text547261918196"></span> CREATE TABLE local_region
|
|
(
|
|
R_REGIONKEY INT4,
|
|
R_NAME TEXT,
|
|
R_COMMENT TEXT
|
|
);
|
|
<span id="EN-US_TOPIC_0000001125261886__text44721019191912"></span>INSERT INTO local_region SELECT * FROM region;</pre>
|
|
<div class="p" id="EN-US_TOPIC_0000001125261886__p152452367539"><div class="note" id="EN-US_TOPIC_0000001125261886__note416884524515"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001125261886__ul10234145616549"><li id="EN-US_TOPIC_0000001125261886__li142356560544">If a connection failure is reported, check the server information and ensure that the specified clusters are connected.</li><li id="EN-US_TOPIC_0000001125261886__li52361756175417">If an error is reported, indicating that the table does not exist, check whether the <strong id="EN-US_TOPIC_0000001125261886__b10620340124813">option</strong> information of the foreign table is correct.</li><li id="EN-US_TOPIC_0000001125261886__li1581513517564">If a column mismatch error is reported, check whether the column information of the foreign table is consistent with that of the corresponding table in the remote cluster.</li><li id="EN-US_TOPIC_0000001125261886__li10837131635714">If a version inconsistency error is reported, upgrade the cluster and try again.</li><li id="EN-US_TOPIC_0000001125261886__li51538501492">If garbled characters are displayed, check the encoding format of the source data, re-create a foreign table, and specify the correct coding format.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
</li><li id="EN-US_TOPIC_0000001125261886__li2696732114612">Perform an associated query.<pre class="screen" id="EN-US_TOPIC_0000001125261886__screen146981632124619"><span id="EN-US_TOPIC_0000001125261886__text870013274610"></span>SELECT * FROM region, local_region WHERE local_region.R_NAME = region.R_NAME;</pre>
|
|
<div class="p" id="EN-US_TOPIC_0000001125261886__p5334193911582"><div class="note" id="EN-US_TOPIC_0000001125261886__note10910429463"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="EN-US_TOPIC_0000001125261886__ul11678115655910"><li id="EN-US_TOPIC_0000001125261886__li1760557502">A foreign table can be used as a local table to perform complex jobs.</li><li id="EN-US_TOPIC_0000001125261886__li18681115685920">If statistics have been collected on the remote cluster, run <strong id="EN-US_TOPIC_0000001125261886__b19482172818289">ANALYZE</strong> on the foreign table to obtain a better execution plan.</li><li id="EN-US_TOPIC_0000001125261886__li164751358675">If there are fewer DNs in the local cluster than in the remote cluster, the local cluster needs to use SMP for better performance.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
</li></ul>
|
|
</p></li><li id="EN-US_TOPIC_0000001125261886__li877510361491"><span>Delete the foreign table.</span><p><pre class="screen" id="EN-US_TOPIC_0000001125261886__en-us_topic_0059779123_screen4163821517356"><span id="EN-US_TOPIC_0000001125261886__text191752815399"></span><strong id="EN-US_TOPIC_0000001125261886__b1684492919197">DROP FOREIGN TABLE</strong> region; </pre>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0179.html">Data Import</a></div>
|
|
</div>
|
|
</div>
|
|
|