vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/tool/dws_07_0692.html
Lu, Huayi 346ac31da9 DWS TG 8.1.3.200 VERSION 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Reviewed-by: Jiang, Beibei <beibei.jiang@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2023-08-28 09:20:17 +00:00

166 lines
54 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001188202652"></a><a name="EN-US_TOPIC_0000001188202652"></a>
<h1 class="topictitle1">Example of Importing Data Using GDS</h1>
<div id="body1586251235951"><div class="section" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s75a7ddd12c714419b49dafbd87d88966"><h4 class="sectiontitle">Example: Parallel Import from Multiple Data Servers</h4><p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a465f90ab92e44fd1ac2e5331d4eb6574">The data servers reside on the same intranet as the cluster. Their IP addresses are 192.168.0.90 and 192.168.0.91. Source data files are in CSV format.</p>
<ol id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ode83f3e54d66420c8e57326f89df4268"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l82d6ff49bd854aeea1fb1686c49c0bb2">Create the target table <strong id="EN-US_TOPIC_0000001188202652__b8872532141211"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text1057411329231">tpcds.</span>reasons</strong>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s87834e57fce642ac8c5f936643af0060"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">reasons</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lf1732e1798614b418733df5d8d8f2cfd">Log in to each GDS data server as user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706164548">root</strong> and create the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706164555">/input_data</strong> directory for storing data files on the servers. The following takes the data server whose IP address is 192.168.0.90 as an example. Operations on the other server are the same.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sbb53de52dbaf4f8ab79768fa5e2cc514"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a88c5d81c4d0b46f199912a95959b6ecb">mkdir -p</strong> <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a1d3cdbe53d6b43b68e0e3de719ef2e0a">/input_data</em></pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l1b57a31aefde416980683b316b21816d">(Optional) Create a user and the user group it belongs to. The user is used to start GDS. If the user and user group already exist, skip this step.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sbe89ef3a703a45a1a69f1dad29006ed6"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0117407660_b2851994648">groupadd</strong> gdsgrp
<strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a854bc4c25d4b4ce6800680e1c0f2d72f">useradd -g</strong> gdsgrp gds_user</pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lc8cd42b5114a4f55a4d3ca2349c702cd">Evenly distribute source data files to the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706112150">/input_data</strong> directories on the data servers.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_le7af74753c6643c9935ea885ddee7662">Change the owners of source data files and the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b126088571505">/input_data</strong> directory on each data server to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b3608957708">gds_user</strong>. The data server with the IP address 192.168.0.90 is used as an example.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sd7a1012c682d459ab495e70000eb2af5"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ac6448a40312640ce93d6b25f245769da">chown -R</strong> gds_user:gdsgrp /input_data </pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l7d929faa456a4aa3b0bdbbc3bbf95e03">Log in to each data server as user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b94641440618">gds_user</strong> and start GDS.<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a78e5e9eaca374431b422cb3bd11d5376">The GDS installation path is <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_b195111826121719"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0125634743_ph1192719508125">/opt/bin/dws/gds</span></strong>. Source data files are stored in <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b958131472316">/input_data/</strong>. The IP addresses of the data servers are 192.168.0.90 and 192.168.0.91. The GDS listening port is 5000. GDS runs in daemon mode.</p>
<div class="p" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_p14496202813204">Start GDS on the data server whose IP address is 192.168.0.90.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sd1493e57611b4d0c879917bbd0d9ecb9"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a1370245064af4bf58338347d753449e8"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_ph1516615415316">/opt/bin/dws/gds</span>/gds -d</strong> <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a9927edbdac1641c9a5f0b5df13b311e7">/input_data</em> <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aace37c25c51444eabb0f7694bacb7bd7">-p </strong><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ac49811766ffa482884400c77b0defb29">192.168.0.90:5000 </em><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a3037b9267090413fa8f54e1542ddbf65">-H</strong><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a9078fa03c01c4900bcd755e5bab455ab"> </em>10.10.0.1/24 <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a765301ae3be5451eb3ed81736163c5fb">-D</strong></pre>
</div>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a707492c6f8924e33863f0a12d9c34c0f">Start GDS on the data server whose IP address is 192.168.0.91.</p>
<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s47f18bb849b14c9db2490a2cf611b477"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a688abd3aec5242c8819bbe5c9542a972"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_ph437610419325">/opt/bin/dws/gds</span>/gds -d</strong> <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aeac34deaf2cf474aa02c99ac38fbf127">/input_data</em> <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aa409f7fd2b44481eb305d6c3471d05c8">-p </strong><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aa426205d390a49039e8bb9e0e45b53b6">192.168.0.91:5000 </em><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_af40fd292fa1d4a35b7a92d1df8a8648c">-H</strong><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a5034820818d440c1bc936d9e7a21be73"> </em>10.10.0.1/24 <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a727bc594c8f14b90b44cf05bbf1acb6f"> -D</strong></pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ld643549316f44956bd6e12c41f3c86d5">Create the foreign table <strong id="EN-US_TOPIC_0000001188202652__b13299101819387"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text119331748202912">tpcds.</span>foreign_tpcds_reasons</strong> for the source data.<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ac95945d49530478990f21732615003b4">Set import mode parameters as follows:</p>
<ul id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_u1cfdc0d9abb14624901e4bf38067dec5"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l85b86de2e5744b2992583dda59b68633">Set the import mode to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b9537171216820">Normal</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ld1c6accd1aeb447395235306585c872f">When GDS is started, the source data file directory is <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0085032064_en-us_topic_0058967605_b1452941350101319_1">/input_data</strong> and the GDS listening port is 5000. Therefore, set <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0085032064_en-us_topic_0058967605_b1904485453101319_1">location</strong> to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0085032064_en-us_topic_0058967605_b1935725810101319_1">gsfs://192.168.0.90:5000/* | gsfs://192.168.0.91:5000/*</strong>.</li></ul>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a8d376fe2150b4b31a5045accf32abc68">Information about the data format is set based on data format parameters specified during data export. The parameter settings are as follows:</p>
<ul id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_u69a35a9beca94280ae994af65857fb96"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_la3061eb8a2e2490c91e67e469f16769a"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706165627">format</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706165631">CSV</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l0f5e28adc21940a5af6c8e774194e2cc"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706143434">encoding</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706161148">UTF-8</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l3f47d627abe64bae8f5ea5e32c183bd2"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0058967650_b84235270616570_1">delimiter</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0058967650_b84235270616572_1">E'\x08'</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l542a8fb4c8114e8eb652d575356068ef"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706165714">quote</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706165717">0x1b</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l139d936194404f3f96573276c726d7ab"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b554252501012">null</strong> is set to an empty string without quotation marks.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_li1881144018294"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b09471625123211">escape</strong> is set to the same value as that of <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b595002510325">quote</strong> by default.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l130cbb41925d438fa7f8a24d1c70115d"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706165753">header</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706171243">false</strong>, indicating that the first row is regarded as a data row when a file is imported.</li></ul>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a9feb1ca25bd64e62bca39d239602e12e">Set import error tolerance parameters as follows:</p>
<ul id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ubbc3459a022f4ec892782a72745ae08b"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l1b09e44235484400b70292378d841e0d">Set <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b84235270610154">PER NODE REJECT LIMIT</strong> (number of allowed data format errors) to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b1663913257310">unlimited</strong>. In this case, all the data format errors detected during data import will be tolerated.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l4fb119474b184c218959a414532f8053">Set <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b115081710205320">LOG INTO</strong> to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b850819103537">err_tpcds_reasons</strong>. The data format errors detected during data import will be recorded in the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b18510141075311">err_tpcds_reasons</strong> table.</li></ul>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a6cf3b0991f5e40ba916895db17efdbc5">Based on the above settings, the foreign table is created using the following statement:</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sa82bfa7a31244461a5aab277de2ae7d3"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">foreign_tpcds_reasons</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span>
<span class="n">SERVER</span><span class="w"> </span><span class="n">gsmpp_server</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="k">location</span><span class="w"> </span><span class="s1">'gsfs://192.168.0.90:5000/* | gsfs://192.168.0.91:5000/*'</span><span class="p">,</span><span class="w"> </span><span class="n">format</span><span class="w"> </span><span class="s1">'CSV'</span><span class="p">,</span><span class="k">mode</span><span class="w"> </span><span class="s1">'Normal'</span><span class="p">,</span><span class="w"> </span><span class="k">encoding</span><span class="w"> </span><span class="s1">'utf8'</span><span class="p">,</span><span class="w"> </span><span class="k">delimiter</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x08'</span><span class="p">,</span><span class="w"> </span><span class="n">quote</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x1b'</span><span class="p">,</span><span class="w"> </span><span class="k">null</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span><span class="w"> </span><span class="n">fill_missing_fields</span><span class="w"> </span><span class="s1">'false'</span><span class="p">)</span><span class="w"> </span><span class="n">LOG</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">err_tpcds_reasons</span><span class="w"> </span><span class="n">PER</span><span class="w"> </span><span class="n">NODE</span><span class="w"> </span><span class="n">REJECT</span><span class="w"> </span><span class="k">LIMIT</span><span class="w"> </span><span class="s1">'unlimited'</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l9173a914c5fd452680f1045124efe9dc">Import data through the foreign table <strong id="EN-US_TOPIC_0000001188202652__b947718316513"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text12620246131812">tpcds.</span>fforeign_tpcds_reasons</strong> to the target table <strong id="EN-US_TOPIC_0000001188202652__b104771231125111"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text5924132191913">tpcds.</span>reasons</strong>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s7810b86ffab5428799c88973b0550345"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">reasons</span><span class="w"> </span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">foreign_tpcds_reasons</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lf9d7d1cf88204e24805feb2437e8a937">Query data import errors in the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b84235270615248">err_tpcds_reasons</strong> table and rectify the errors (if any). For details, see <a href="dws_07_0056.html">Handling Import Errors</a>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s7ee3163db2af4d87a87d420ab9524e72"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">err_tpcds_reasons</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l691363ba214242029c90f34866efcc04">After data import is complete, log in to each data server as user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0085032064_b84235270617183">gds_user</strong> and stop GDS.<div class="p" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_p1451242814209">The data server with the IP address 192.168.0.90 is used as an example. The GDS process ID is 128954.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s0ff0feeaf6d74de898624df4d545a944"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a5eb857e102e54350b2bdcc57a2223c37">ps -ef|grep gds</strong>
gds_user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a41051798abeb4cfeba2d3608fb6eab86">128954</strong> 1 0 15:03 ? 00:00:00 gds -d /input_data -p 192.168.0.90:5000 -D
gds_user 129003 118723 0 15:04 pts/0 00:00:00 grep gds
<strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a0db9da3db6af4a76ac290745fc925c3a">kill -9</strong> 128954</pre>
</div>
</li></ol>
</div>
<div class="section" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s44373786b7ea42b89484108630417ca7"><h4 class="sectiontitle">Example: Data Import Using Multiple Threads</h4><p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a41c7b34b58fa484daea23c9b58cd6545">The data server resides on the same intranet as the cluster. The server IP address is 192.168.0.90. Source data files are in CSV format. Data will be imported to two tables using multiple threads in <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b175531531309">Normal</strong> mode.</p>
<ol id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_o8e097d1292174fe99af701ff4ab00a06"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l00e54ec394d94491b8379e65bb35c23e">In the database, create the target tables <strong id="EN-US_TOPIC_0000001188202652__b17911946201412"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text3838123644011">tpcds.</span>reasons1</strong> and <strong id="EN-US_TOPIC_0000001188202652__b56311056151416"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text2095633020402">tpcds.</span>reasons2</strong>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sb83dfa8f318f4d33a2e5c576b647814c"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">reasons1</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span><span class="w"> </span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s410c83fc91af46159048944c10cf6ed9"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">reasons2</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span><span class="w"> </span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lc292aa115449439bba98cb727492d139">Log in to the GDS data server as user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b842352706133453">root</strong>, and then create the data file directory <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b84235270613350">/input_data</strong> and its sub-directories <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b84235270613354">/input_data/import1/</strong> and <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b84235270613358">/input_data/import2/</strong>.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sc08176b3568945599582634a1d77035c">mkdir -p <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aab2547f29d2c4b1fb40c52dbbf79ed9c">/input_data</em></pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_la07b86d240794549a11e719c37373963">Store the source data files of the target table <strong id="EN-US_TOPIC_0000001188202652__b686912499614"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text174381754112">tpcds.</span>reasons1</strong> in <strong id="EN-US_TOPIC_0000001188202652__b16870194920612">/input_data/import1/</strong> and the source data files of the target table <strong id="EN-US_TOPIC_0000001188202652__b15871149761"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text15185162417417">tpcds.</span>reasons2</strong> in <strong id="EN-US_TOPIC_0000001188202652__b28723494620">/input_data/import2/</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lc481980cc5844c6dbbb1e90b34c1b85d">(Optional) Create a user and the user group it belongs to. The user is used to start GDS. If the user and user group already exist, skip this step.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sd54c58be74414495bf4acc64ca6f1425"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ad791eedc77524f009a5a3116b5f8455a">groupadd</strong> gdsgrp
<strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0117407660_b17515341034">useradd -g</strong> gdsgrp gds_user</pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l978525e82045443e8651d1b4107d2a32">Change the owners of source data files and the <strong id="EN-US_TOPIC_0000001188202652__b2133012472">/input_data</strong> directory on the data server to <strong id="EN-US_TOPIC_0000001188202652__b1260425376">gds_user</strong>.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sf18823e70247418daf989e4348fc17f7">chown -R <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0117407660_i91561415449">gds_user:gdsgrp</em><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aa7034cebe99b4bc19defd7e1c60f7828"> /input_data </em></pre>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lb45756cd3c7f4e98b0b3b32b96b01d58">Log in to the data server as user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b3415134158">gds_user</strong> and start GDS.<div class="p" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_p172969435223">The GDS installation path is <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b1787342118510">/gds</strong>. Source data files are stored in <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b1688910214519">/input_data/</strong>. The IP address of the data server is 192.168.0.90. The GDS listening port is 5000. GDS runs in daemon mode. The degree of parallelism is 2. A recursive directory is specified.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s861f5b84b6ec402f88917c77a8725f30"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_adc519e55546c4e77b0e7e715f9fce958">/gds/gds -d</strong> <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a26e4e935f59c4716a52191d3b536d80f">/input_data</em> <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a5af0deea7cbd4f719163914d14a3ff36">-p </strong><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_acbdac7fd977648c1a8fd4b931392a7fd">192.168.0.90:5000 </em><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a58fb1ca5a2574e32aac402d07b6908dc">-H</strong><em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a6bc71b79f0744a838c57d230494377c3"> </em>10.10.0.1/24 <em id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a7982b94152e14ac4ba8c1f8e004334fd"> </em><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a029be89589a7428b87959ec98c65e53f">-D -t </strong>2<strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a2a388f742d7b489b8e23d63c4c559eb8"> -r</strong></pre>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l8bc4d54a954c42ef9640b28f3d654888">In the database, create the foreign tables <strong id="EN-US_TOPIC_0000001188202652__b3894151818162"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text63531327183211">tpcds.</span>foreign_tpcds_reasons1</strong> and <strong id="EN-US_TOPIC_0000001188202652__b696610208162"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text1530205716327">tpcds.</span>foreign_tpcds_reasons2</strong> for the source data.<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a10010a0a667a41d09692c20c34365dc6">The foreign table <strong id="EN-US_TOPIC_0000001188202652__b9559132351611"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text17683135012324">tpcds.</span>foreign_tpcds_reasons1</strong> is used as an example to describe how to configure parameters in a foreign table.</p>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a7528b264232f40b8a8cd98876b561d5c">Set import mode parameters as follows:</p>
<ul id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ua5c54221399b46199e585903380c38a3"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lf2365add8ea347f3a0ec903547916b2a">Set the import mode to <strong id="EN-US_TOPIC_0000001188202652__b1930450724">Normal</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lca350094bff14b85af63ac2c37045f0d">When GDS is started, the configured source data file directory is <strong id="EN-US_TOPIC_0000001188202652__b2140773370">/input_data</strong> and the GDS listening port is 5000. However, source data files are actually stored in <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b669334314817">/input_data/import1/</strong>. Therefore, set <strong id="EN-US_TOPIC_0000001188202652__b1879130288">location</strong> to <strong id="EN-US_TOPIC_0000001188202652__b233548190">gsfs://192.168.0.90:5000/import1/*</strong>.</li></ul>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a18068bdc8b5848bbbac24fbf188577dd">Information about the data format is set based on data format parameters specified during data export. The parameter settings are as follows:</p>
<ul id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ub0e2cadbb5bd4bcda86a635f4be054a9"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ld1bcd94006ee419db465f36b0783fe3d"><strong id="EN-US_TOPIC_0000001188202652__b684407649">format</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__b491388463">CSV</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lbf533f6c727e4057a232a32443f0e094"><strong id="EN-US_TOPIC_0000001188202652__b1376828142">encoding</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__b1522878738">UTF-8</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l0a4ec79818e34f5f8375ed36bf6b10f8"><strong id="EN-US_TOPIC_0000001188202652__b1631540503">delimiter</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__b1582141802">E'\x08'</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_la8cdba8723934b32b7125427fd968089"><strong id="EN-US_TOPIC_0000001188202652__b1398271924">quote</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__b1354483014">0x1b</strong>.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lc2b6f1f31ebf413ab8529cefb4e9afbe"><strong id="EN-US_TOPIC_0000001188202652__b1308792772">null</strong> is set to an empty string without quotation marks.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_li123938174309"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b1322012351766">escape</strong> is set to the same value as that of <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b1222083511620">quote</strong> by default.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l6b3b96db12694ac9a77f794b4b0e041c"><strong id="EN-US_TOPIC_0000001188202652__b539972392">header</strong> is set to <strong id="EN-US_TOPIC_0000001188202652__b2033192473">false</strong>, indicating that the first row is regarded as a data row when a file is imported.</li></ul>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a16fef2f92eb3499989cc0756f1015816">Set import error tolerance parameters as follows:</p>
<ul id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_ub890053d27f547d099b6b7ad71ddeeff"><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l8905589a965642fba6d3543169ae2174">Set <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b253042815912">PER NODE REJECT LIMIT</strong> (number of allowed data format errors) to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b175458281198">unlimited</strong>. In this case, all the data format errors detected during data import will be tolerated.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_laed2f99667f04d5fae92a14288d4452a">Set <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b1677914109254">LOG INTO</strong> to <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b2781111013257">err_tpcds_reasons1</strong>. The data format errors detected during data import will be recorded in the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b14782171012253">err_tpcds_reasons1</strong> table.</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l081fccc5c6204faf87e8d133b8046f22">If the last column (<strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b04311713151817">fill_missing_fields</strong>) in a source data file is missing, the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0085032064_b842352706134758">NULL</strong> column will be automatically added to the target file.</li></ul>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aa6f08899b7074df691268403d8265e98">Based on the preceding settings, the foreign table <strong id="EN-US_TOPIC_0000001188202652__b324183714137"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text187166614614">tpcds.</span>foreign_tpcds_reasons1</strong> is created as follows:</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sb351e6f271414321bf3670fd29769e0c"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">foreign_tpcds_reasons1</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span><span class="w"> </span><span class="n">SERVER</span><span class="w"> </span><span class="n">gsmpp_server</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="k">location</span><span class="w"> </span><span class="s1">'gsfs://192.168.0.90:5000/import1/*'</span><span class="p">,</span><span class="w"> </span><span class="n">format</span><span class="w"> </span><span class="s1">'CSV'</span><span class="p">,</span><span class="k">mode</span><span class="w"> </span><span class="s1">'Normal'</span><span class="p">,</span><span class="w"> </span><span class="k">encoding</span><span class="w"> </span><span class="s1">'utf8'</span><span class="p">,</span><span class="w"> </span><span class="k">delimiter</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x08'</span><span class="p">,</span><span class="w"> </span><span class="n">quote</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x1b'</span><span class="p">,</span><span class="w"> </span><span class="k">null</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span><span class="n">fill_missing_fields</span><span class="w"> </span><span class="s1">'on'</span><span class="p">)</span><span class="n">LOG</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">err_tpcds_reasons1</span><span class="w"> </span><span class="n">PER</span><span class="w"> </span><span class="n">NODE</span><span class="w"> </span><span class="n">REJECT</span><span class="w"> </span><span class="k">LIMIT</span><span class="w"> </span><span class="s1">'unlimited'</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a5ebfd2bfc4334f548eaf02d822a9aec4">Based on the preceding settings, the foreign table <strong id="EN-US_TOPIC_0000001188202652__b1817417525257"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text1093415203315">tpcds.</span>foreign_tpcds_reasons2</strong> is created as follows:</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s6b48e7add5304d58b05ad2bf131e4ae2"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">FOREIGN</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">foreign_tpcds_reasons2</span>
<span class="p">(</span>
<span class="w"> </span><span class="n">r_reason_sk</span><span class="w"> </span><span class="nb">integer</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span><span class="w"> </span><span class="k">not</span><span class="w"> </span><span class="k">null</span><span class="p">,</span>
<span class="w"> </span><span class="n">r_reason_desc</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="p">)</span><span class="w"> </span><span class="n">SERVER</span><span class="w"> </span><span class="n">gsmpp_server</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="k">location</span><span class="w"> </span><span class="s1">'gsfs://192.168.0.90:5000/import2/*'</span><span class="p">,</span><span class="w"> </span><span class="n">format</span><span class="w"> </span><span class="s1">'CSV'</span><span class="p">,</span><span class="k">mode</span><span class="w"> </span><span class="s1">'Normal'</span><span class="p">,</span><span class="w"> </span><span class="k">encoding</span><span class="w"> </span><span class="s1">'utf8'</span><span class="p">,</span><span class="w"> </span><span class="k">delimiter</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x08'</span><span class="p">,</span><span class="w"> </span><span class="n">quote</span><span class="w"> </span><span class="n">E</span><span class="s1">'\x1b'</span><span class="p">,</span><span class="w"> </span><span class="k">null</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span><span class="n">fill_missing_fields</span><span class="w"> </span><span class="s1">'on'</span><span class="p">)</span><span class="n">LOG</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">err_tpcds_reasons2</span><span class="w"> </span><span class="n">PER</span><span class="w"> </span><span class="n">NODE</span><span class="w"> </span><span class="n">REJECT</span><span class="w"> </span><span class="k">LIMIT</span><span class="w"> </span><span class="s1">'unlimited'</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_l9393c1d1179b45058b480628b524cd55">Import data to <strong id="EN-US_TOPIC_0000001188202652__b1074295712264"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text1674419561934">tpcds.</span>reasons1</strong> using the foreign table <strong id="EN-US_TOPIC_0000001188202652__b1986665313171"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text197171042536">tpcds.</span>foreign_tpcds_reasons1</strong>, and to <strong id="EN-US_TOPIC_0000001188202652__b2052331814"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text18720621745">tpcds.</span>reasons2</strong> using the foreign table <strong id="EN-US_TOPIC_0000001188202652__b9147141716186"><span id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_text156114485320">tpcds.</span>foreign_tpcds_reasons2</strong>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s844aa16fc22a471aadf145aedee74868"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">reasons1</span><span class="w"> </span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">foreign_tpcds_reasons1</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_s1108f677243c45efb14cd6a66160ae2b"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">reasons2</span><span class="w"> </span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">tpcds</span><span class="p">.</span><span class="n">foreign_tpcds_reasons2</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_le98d1523c9ce4b959db52648f97bf669">Query data import errors in the <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b87801421375">err_tpcds_reasons1</strong> and <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_b678116211278">err_tpcds_reasons2</strong> tables and rectify the errors (if any). For details, see <a href="dws_07_0056.html">Handling Import Errors</a>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_se2beb6c2f39943b88e94da5108cdcac8"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">err_tpcds_reasons1</span><span class="p">;</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">err_tpcds_reasons2</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_lbcb9503544a24337943a3090dab25df4">After data import is complete, log in to the data server as user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_en-us_topic_0085032744_b372029376175142">gds_user</strong> and stop GDS.<div class="p" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_p1737384372211">The GDS process ID is 128954.<pre class="screen" id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_sd8448e03f3444b95b108c8d00bd888c1"><strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_aa5d3e598bacd4153a9d65bfc531b041f">ps -ef|grep gds</strong>
gds_user <strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a2cc1b01346444017a110839b75cae7ac">128954</strong> 1 0 15:03 ? 00:00:00 gds -d /input_data -p 192.168.0.90:5000 -D -t 2 -r
gds_user 129003 118723 0 15:04 pts/0 00:00:00 grep gds
<strong id="EN-US_TOPIC_0000001188202652__en-us_topic_0000001145491143_en-us_topic_0000001080433273_a01e470f496a54cd3b44061380ca97de8">kill -9</strong> 128954</pre>
</div>
</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_gds_index.html">GDS</a></div>
</div>
</div>
View Git Blame Copy Permalink