forked from docs/doc-exports
Lu, Huayi
e6fa411af0
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Lu, Huayi <luhuayi@huawei.com> Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
176 lines
33 KiB
HTML
176 lines
33 KiB
HTML
<a name="EN-US_TOPIC_0000001188163708"></a><a name="EN-US_TOPIC_0000001188163708"></a>
|
|
|
|
<h1 class="topictitle1">Installing, Configuring, and Starting GDS</h1>
|
|
<div id="body8662426"><div class="section" id="EN-US_TOPIC_0000001188163708__s270612443421457abfb2f038846c8c5f"><h4 class="sectiontitle">Scenario</h4><p id="EN-US_TOPIC_0000001188163708__p090613618131"><span id="EN-US_TOPIC_0000001188163708__text1856068094">GaussDB(DWS)</span> uses GDS to allocate the source data for parallel data import. Deploy GDS on the data server.</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__p11264113017269">If a large volume of data is stored on multiple data servers, install, configure, and start GDS on each server. Then, data on all the servers can be imported in parallel. The procedure for installing, configuring, and starting GDS is the same on each data server. This section describes how to perform this procedure on one data server.</p>
|
|
</div>
|
|
<div class="section" id="EN-US_TOPIC_0000001188163708__s929eec5e14084da99aa7906f53ed4f89"><h4 class="sectiontitle">Context</h4><p id="EN-US_TOPIC_0000001188163708__p592842254119">The GDS version must match the cluster version. For example, GDS V100R008C00 matches DWS 1.3.<em id="EN-US_TOPIC_0000001188163708__i514010371396">X</em>. Otherwise, the import or export may fail, or the import or export process may fail to respond. Therefore, use the latest version of GDS.</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__p2087026144512">After the database is upgraded, download the latest version of <span id="EN-US_TOPIC_0000001188163708__text2107851618">GaussDB(DWS)</span> GDS as instructed in <a href="#EN-US_TOPIC_0000001188163708__se2a774b311864348a516131937610513">Procedure</a>. When the import or export starts, <span id="EN-US_TOPIC_0000001188163708__text1390963698">GaussDB(DWS)</span> checks the GDS versions. If the versions do not match, an error message is displayed and the import or export is terminated.</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__p1187111611453">To obtain the version number of GDS, run the following command in the GDS decompression directory:</p>
|
|
<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen887114611453"><strong id="EN-US_TOPIC_0000001188163708__b131114277435">gds</strong> -V</pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p48721061457">To view the database version, run the following SQL statement after connecting to the database:</p>
|
|
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188163708__screen987315613456"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="k">version</span><span class="p">();</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
<div class="section" id="EN-US_TOPIC_0000001188163708__se2a774b311864348a516131937610513"><a name="EN-US_TOPIC_0000001188163708__se2a774b311864348a516131937610513"></a><a name="se2a774b311864348a516131937610513"></a><h4 class="sectiontitle">Procedure</h4><ol id="EN-US_TOPIC_0000001188163708__o213c59f9072d48fd905ceb73687c4f2b"><li id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_li970591810219"><span>Before using GDS to import or export data, see <strong id="EN-US_TOPIC_0000001188163708__b76501399202">Preparing an ECS as the GDS Server</strong> and <strong id="EN-US_TOPIC_0000001188163708__b17695918132010">Downloading the GDS Package</strong> in "Tutorial: Using GDS to Import Data from a Remote Server" in <em id="EN-US_TOPIC_0000001188163708__i163968531911">Data Warehouse Service (DWS) User Guide</em>.</span></li><li id="EN-US_TOPIC_0000001188163708__l242cf61617ff44ffbcfa868e55b91d88"><a name="EN-US_TOPIC_0000001188163708__l242cf61617ff44ffbcfa868e55b91d88"></a><a name="l242cf61617ff44ffbcfa868e55b91d88"></a><span>Log in as user <strong id="EN-US_TOPIC_0000001188163708__b191397561942">root</strong> to the data server where GDS is to be installed and run the following command to create the directory for storing the GDS package:</span><p><pre class="screen" id="EN-US_TOPIC_0000001188163708__s58ae87fad6d045b1a5f9546c58e6c912"><strong id="EN-US_TOPIC_0000001188163708__aae0c14df172e45dbb802cd19ea14559e">mkdir -p </strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_en-us_topic_0117407708_ad72482db75884492bfc4a099330d9db7">/opt/bin/dws</em></pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__li17170131917913"><span>Upload the GDS package to the created directory.</span><p><p id="EN-US_TOPIC_0000001188163708__p1378019206920">Use the SUSE Linux package as an example. Upload the GDS package <strong id="EN-US_TOPIC_0000001188163708__b2088479151911"></strong><strong id="EN-US_TOPIC_0000001188163708__b15884198193">dws_client_8.</strong><em id="EN-US_TOPIC_0000001188163708__i688412916195">x.x</em><strong id="EN-US_TOPIC_0000001188163708__b78841917194">_suse_x64.zip</strong> to the directory created in the previous step.</p>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__li16883354813"><a name="EN-US_TOPIC_0000001188163708__li16883354813"></a><a name="li16883354813"></a><span>(Optional) If SSL is used, upload the SSL certificates to the directory created in <a href="#EN-US_TOPIC_0000001188163708__l242cf61617ff44ffbcfa868e55b91d88">2</a>.</span></li><li id="EN-US_TOPIC_0000001188163708__ld8bed05b353f406880669fe49e1e7b64"><span>Go to the directory and decompress the package.</span><p><pre class="screen" id="EN-US_TOPIC_0000001188163708__s247e3cb84fe747dab5432e40bad96c2d"><strong id="EN-US_TOPIC_0000001188163708__ae4592ad74b8f458298ba5455631fa193">cd<em id="EN-US_TOPIC_0000001188163708__i19453101245115"> </em></strong><em id="EN-US_TOPIC_0000001188163708__i1941214774616">/opt/bin/dws</em>
|
|
<em id="EN-US_TOPIC_0000001188163708__i1073014140224">unzip dws_client_8.x.x_suse_x64.zip</em></pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__le097b7d9742d44e0bb3e053909abf995"><span>Create a GDS user and the user group to which the user belongs. This user is used to start GDS and read source data.</span><p><pre class="screen" id="EN-US_TOPIC_0000001188163708__s87cd50a7bf9e4c5ea82dbe07cb685e25"><strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0117407708_b2851994648">groupadd</strong> gdsgrp
|
|
<strong id="EN-US_TOPIC_0000001188163708__a299d862a8c414cfab5aac6c22cbc23e1">useradd -g</strong> gdsgrp gds_user</pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__l49ccb26244d446e8991ddb3c32dc0cac"><span>Change the owner of the GDS package directory and source data file directory to the GDS user.</span><p><pre class="screen" id="EN-US_TOPIC_0000001188163708__sd38e3e3e6a5a4e1092580559a1337f0d"><strong id="EN-US_TOPIC_0000001188163708__a6d99c661be8140bfbe0b3eab523e267e">chown -R </strong><em id="EN-US_TOPIC_0000001188163708__a96562d1acfb1463e9196cc9d2f217b9a">gds_user:gdsgrp </em><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_en-us_topic_0117407708_i3077079994810">/opt/bin/dws/gds</em>
|
|
<strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0117407704_b542515095412">chown -R </strong><em id="EN-US_TOPIC_0000001188163708__a826a8efb3bba43eca47dd0aa55379f87">gds_user:gdsgrp </em><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0117407704_i472181201076">/input_data </em></pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__li175611031175618"><span>Switch to user <strong id="EN-US_TOPIC_0000001188163708__b101041119141">gds_user</strong>.</span><p><pre class="screen" id="EN-US_TOPIC_0000001188163708__screen4561193113566"><strong id="EN-US_TOPIC_0000001188163708__b1056217312560">su -</strong> <em id="EN-US_TOPIC_0000001188163708__i8562183185618">gds_user</em></pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p1135991195919">If the current cluster version is 8.0.<em id="EN-US_TOPIC_0000001188163708__i1293615525255">x</em> or earlier, skip <a href="#EN-US_TOPIC_0000001188163708__li14421575388">9</a> and go to <a href="#EN-US_TOPIC_0000001188163708__li22515537437">10</a>.</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__p1439316819">If the current cluster version is 8.1.<em id="EN-US_TOPIC_0000001188163708__i1529217495414">x</em>, go to the next step.</p>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__li14421575388"><a name="EN-US_TOPIC_0000001188163708__li14421575388"></a><a name="li14421575388"></a><span>Execute the script on which the environment depends (applicable only to 8.1.<em id="EN-US_TOPIC_0000001188163708__i9210155122717">x</em>).</span><p><pre class="screen" id="EN-US_TOPIC_0000001188163708__screen166571749113911">cd /opt/bin/dws/gds/bin
|
|
source gds_env</pre>
|
|
</p></li><li id="EN-US_TOPIC_0000001188163708__li22515537437"><a name="EN-US_TOPIC_0000001188163708__li22515537437"></a><a name="li22515537437"></a><span>Start GDS.</span><p><p id="EN-US_TOPIC_0000001188163708__p19261863505">GDS is green software and can be started after being decompressed. There are two ways to start GDS. One is to run the <strong id="EN-US_TOPIC_0000001188163708__b1707121102811">gds</strong> command to configure startup parameters. The other is to write the startup parameters into the <strong id="EN-US_TOPIC_0000001188163708__b277151632815">gds.conf</strong> configuration file and run the <strong id="EN-US_TOPIC_0000001188163708__b677721682817">gds_ctl.py</strong> command to start GDS.</p>
|
|
<div class="p" id="EN-US_TOPIC_0000001188163708__p14392195934313">The first method is recommended when you do not need to import data again. The second method is recommended when you need to import data regularly.<ul id="EN-US_TOPIC_0000001188163708__ul195484314543"><li id="EN-US_TOPIC_0000001188163708__li17545431105416">Method 1: Run the <strong id="EN-US_TOPIC_0000001188163708__b1990812236324">gds</strong> command to start GDS.<ul id="EN-US_TOPIC_0000001188163708__ul185441831165418"><li id="EN-US_TOPIC_0000001188163708__li2543631185418">If data is transmitted in non-SSL mode, run the following command to start GDS:<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen145431831105412"><strong id="EN-US_TOPIC_0000001188163708__b554343118547">gds -d</strong><em id="EN-US_TOPIC_0000001188163708__i4543143116548"> dir </em><strong id="EN-US_TOPIC_0000001188163708__b554383110546">-p</strong><em id="EN-US_TOPIC_0000001188163708__i195431931185413"> ip:port </em><strong id="EN-US_TOPIC_0000001188163708__b954383111542">-H</strong><em id="EN-US_TOPIC_0000001188163708__i17543183115548"> </em><em id="EN-US_TOPIC_0000001188163708__i45439313549">address_string</em> <strong id="EN-US_TOPIC_0000001188163708__b1154313125415">-l</strong><em id="EN-US_TOPIC_0000001188163708__i854318313548"> log_file </em><strong id="EN-US_TOPIC_0000001188163708__b15543143119544">-D </strong><strong id="EN-US_TOPIC_0000001188163708__b157821057269">-t</strong> <em id="EN-US_TOPIC_0000001188163708__i1938416132710">worker_num</em></pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p116451418511">Example:</p>
|
|
<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen156421475119">
|
|
<strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b45436313548">/opt/bin/dws/gds/bin/gds -d</strong> <em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i11543831195410">/input_data/</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b19543131115413">-p </strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i154363110545">192.168.0.90:5000</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b1543153165413">-H</strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i95431331175411"> 10.10.0.1/24</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b954311312540">-l</strong> <em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i105434317544">/opt/bin/dws/gds/gds_log.txt</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b155431031155415">-D</strong> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b198572315269">-t</strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i662692819269"> 2</em></pre>
|
|
</li><li id="EN-US_TOPIC_0000001188163708__li254443135419">If data is transmitted in SSL mode, run the following command to start GDS:<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen854410311542"><strong id="EN-US_TOPIC_0000001188163708__b1044214716296">gds -d</strong><em id="EN-US_TOPIC_0000001188163708__i94429722912"> dir </em><strong id="EN-US_TOPIC_0000001188163708__b134420722913">-p</strong><em id="EN-US_TOPIC_0000001188163708__i11442167112918"> ip:port </em><strong id="EN-US_TOPIC_0000001188163708__b1244215782916">-H</strong><em id="EN-US_TOPIC_0000001188163708__i3442187152919"> </em><em id="EN-US_TOPIC_0000001188163708__i144210752914">address_string</em> <strong id="EN-US_TOPIC_0000001188163708__b1844220710291">-l</strong><em id="EN-US_TOPIC_0000001188163708__i14422792915"> log_file </em><strong id="EN-US_TOPIC_0000001188163708__b12442774296">-D</strong>
|
|
<strong id="EN-US_TOPIC_0000001188163708__b7439124032610">-t</strong><em id="EN-US_TOPIC_0000001188163708__i1645444022615"> worker_num</em><strong id="EN-US_TOPIC_0000001188163708__b142787152919"> </strong><strong id="EN-US_TOPIC_0000001188163708__b155446316544">--enable-ssl --ssl-dir </strong><em id="EN-US_TOPIC_0000001188163708__i1354473120546">Cert_file</em></pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p125441831175415">Example:</p>
|
|
<div class="p" id="EN-US_TOPIC_0000001188163708__p175441631105419">Run the following command to upload the SSL certificate mentioned in <a href="#EN-US_TOPIC_0000001188163708__li16883354813">4</a> to <strong id="EN-US_TOPIC_0000001188163708__b176859449223">/opt/bin</strong>:<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen85441231145411">
|
|
<strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b135442031135415">/opt/bin/dws/gds/bin/gds -d</strong> <em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i6544133105417">/input_data/</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b55441531125419">-p </strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i2544113119544">192.168.0.90:5000</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b754443185416">-H</strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i1544103118542"> 10.10.0.1/24</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b1254453145416">-l</strong> <em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i45442031115413">/opt/bin/dws/gds/gds_log.txt</em> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b12544133115544">-D</strong> <strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_b054418314549">--enable-ssl --ssl-dir </strong><em id="EN-US_TOPIC_0000001188163708__en-us_topic_0125634743_i6544163125419">/opt/bin/</em></pre>
|
|
</div>
|
|
</li></ul>
|
|
<p id="EN-US_TOPIC_0000001188163708__p654493110547">Replace the information in italic as required.</p>
|
|
<ul id="EN-US_TOPIC_0000001188163708__ul1545193111540"><li id="EN-US_TOPIC_0000001188163708__li8544531145418"><strong id="EN-US_TOPIC_0000001188163708__b14531536173619">-d</strong><em id="EN-US_TOPIC_0000001188163708__i175311364365"> dir</em>: directory for storing data files that contain data to be imported. This tutorial uses <strong id="EN-US_TOPIC_0000001188163708__b189366987392354">/input_data/</strong> as an example.</li><li id="EN-US_TOPIC_0000001188163708__li165451631195418"><strong id="EN-US_TOPIC_0000001188163708__b18189154185514">-p </strong><em id="EN-US_TOPIC_0000001188163708__i20500124213557">ip:port</em>: listening IP address and port for GDS. The default value is <strong id="EN-US_TOPIC_0000001188163708__b1565476204418">127.0.0.1</strong>. Replace it with the IP address of a 10GE network that can communicate with <span id="EN-US_TOPIC_0000001188163708__text1051910946">GaussDB(DWS)</span>. The port number ranges from 1024 to 65535. The default port is <strong id="EN-US_TOPIC_0000001188163708__b81731895292354">8098</strong>. This tutorial uses <strong id="EN-US_TOPIC_0000001188163708__b110826689892354">192.168.0.90:5000</strong> as an example.</li><li id="EN-US_TOPIC_0000001188163708__li854533110544"><strong id="EN-US_TOPIC_0000001188163708__b1018815567554">-H </strong><em id="EN-US_TOPIC_0000001188163708__i9988756175516">address_string</em>: specifies the hosts that are allowed to connect to and use GDS. The value must be in CIDR format. Configure this parameter to enable a <span id="EN-US_TOPIC_0000001188163708__text642231359">GaussDB(DWS)</span> cluster to access GDS for data import. Ensure that the network segment covers all hosts in a <span id="EN-US_TOPIC_0000001188163708__text93144348">GaussDB(DWS)</span> cluster.</li><li id="EN-US_TOPIC_0000001188163708__li18545143185415"><strong id="EN-US_TOPIC_0000001188163708__b461319510564">-l </strong><em id="EN-US_TOPIC_0000001188163708__i87640615614">log_file</em>: GDS log directory and log file name. This tutorial uses <strong id="EN-US_TOPIC_0000001188163708__b1747013511792">/opt/bin</strong><strong id="EN-US_TOPIC_0000001188163708__b124701151499">/dws</strong><strong id="EN-US_TOPIC_0000001188163708__b44701518917">/gds/gds_log.txt</strong> as an example.</li><li id="EN-US_TOPIC_0000001188163708__li254533115543"><strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0122329961_b87236317355">-D</strong>: GDS in daemon mode. This parameter is used only in Linux.</li><li id="EN-US_TOPIC_0000001188163708__li20481737101317"><strong id="EN-US_TOPIC_0000001188163708__b985215922518">-t </strong><em id="EN-US_TOPIC_0000001188163708__i1469519617268">worker_num</em>: number of concurrent GDS threads. If the data server and <span id="EN-US_TOPIC_0000001188163708__text1139326585">GaussDB(DWS)</span> have available I/O resources, you can increase the number of concurrent GDS threads.<p id="EN-US_TOPIC_0000001188163708__p76318380132">GDS determines the number of threads based on the number of concurrent import transactions. Even if multi-thread import is configured before GDS startup, the import of a single transaction will not be accelerated. By default, an <strong id="EN-US_TOPIC_0000001188163708__b38873534389">INSERT</strong> statement is an import transaction.</p>
|
|
</li><li id="EN-US_TOPIC_0000001188163708__li19545173155416"><strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0122329961_b731614209186">--enable-ssl</strong>: enables SSL for data transmission.</li><li id="EN-US_TOPIC_0000001188163708__li654573145414"><strong id="EN-US_TOPIC_0000001188163708__b16806233529">--ssl-dir </strong><em id="EN-US_TOPIC_0000001188163708__i08071331224">Cert_file</em>: SSL certificate directory. Set this parameter to the certificate directory in <a href="#EN-US_TOPIC_0000001188163708__li16883354813">4</a>.</li><li id="EN-US_TOPIC_0000001188163708__li1945162221114">For details about GDS parameters, see "GDS - Parallel Data Loader > gds" in the <em id="EN-US_TOPIC_0000001188163708__i31073119546">Data Warehouse Service (DWS) Tool Guide</em>.</li></ul>
|
|
</li><li id="EN-US_TOPIC_0000001188163708__li6548153185417">Method 2: Write the startup parameters into the <strong id="EN-US_TOPIC_0000001188163708__b531715623010">gds.conf</strong> configuration file and run the <strong id="EN-US_TOPIC_0000001188163708__b031718613300">gds_ctl.py</strong> command to start GDS.<ol type="a" id="EN-US_TOPIC_0000001188163708__ol125481431145415"><li id="EN-US_TOPIC_0000001188163708__li854723110549">Run the following command to go to the <strong id="EN-US_TOPIC_0000001188163708__b842352706221154">config</strong> directory of the GDS package and modify the <strong id="EN-US_TOPIC_0000001188163708__b84235270622126">gds.conf</strong> configuration file. For details about the parameters in the <strong id="EN-US_TOPIC_0000001188163708__b84235270615467">gds.conf</strong> configuration file, see <a href="#EN-US_TOPIC_0000001188163708__t051f8c4ef816412c85e082e7fb7297dd">Table 1</a>.<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen1545931185419"><strong id="EN-US_TOPIC_0000001188163708__b85454316543">vim </strong><em id="EN-US_TOPIC_0000001188163708__i20545173195410">/opt/bin<span id="EN-US_TOPIC_0000001188163708__ph361219310815">/dws</span>/gds/config/</em><strong id="EN-US_TOPIC_0000001188163708__b1754512317544">gds.conf</strong></pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p15545163110546">Example:</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__p1154683116547">The <strong id="EN-US_TOPIC_0000001188163708__b842352706113710">gds.conf</strong> configuration file contains the following information:</p>
|
|
<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen205464319546"><?xml version="1.0"?>
|
|
<config>
|
|
<gds name="gds1" ip="192.168.0.90" port="5000" data_dir="/input_data/" err_dir="/err" data_seg="100MB" err_seg="100MB" log_file="/log/gds_log.txt" host="10.10.0.1/24" daemon='true' recursive="true" parallel="32"></gds>
|
|
</config></pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p15463310549">Information in the configuration file is described as follows:</p>
|
|
<ul id="EN-US_TOPIC_0000001188163708__ul7547193114548"><li id="EN-US_TOPIC_0000001188163708__li454615314541">The data server IP address is <strong id="EN-US_TOPIC_0000001188163708__b4768123393511">192.168.0.90</strong> and the GDS listening port is <strong id="EN-US_TOPIC_0000001188163708__b199861838183510">5000</strong>.</li><li id="EN-US_TOPIC_0000001188163708__li45462318545">Data files are stored in the <strong id="EN-US_TOPIC_0000001188163708__b842352706113059">/input_data/</strong> directory.</li><li id="EN-US_TOPIC_0000001188163708__li554613316546">Error log files are stored in the <strong id="EN-US_TOPIC_0000001188163708__b842352706113122">/err</strong> directory. The directory must be created by a user who has the GDS read and write permissions.</li><li id="EN-US_TOPIC_0000001188163708__li20546231205411">The size of a single data file is 100 MB.</li><li id="EN-US_TOPIC_0000001188163708__li20546113155413">The size of a single error log file is 100 MB.</li><li id="EN-US_TOPIC_0000001188163708__li1854611312541">Logs are stored in the <strong id="EN-US_TOPIC_0000001188163708__b842352706113229">/log/gds_log.txt</strong> file. The directory must be created by a user who has the GDS read and write permissions.</li><li id="EN-US_TOPIC_0000001188163708__li125471031115415">Only nodes with the IP address <strong id="EN-US_TOPIC_0000001188163708__b94991025184411">10.10.0.</strong><em id="EN-US_TOPIC_0000001188163708__i1313549164314">*</em> can be connected.</li><li id="EN-US_TOPIC_0000001188163708__li145471331145419">The GDS process is running in daemon mode.</li><li id="EN-US_TOPIC_0000001188163708__li1154733114547">Recursive data file directories are used.</li><li id="EN-US_TOPIC_0000001188163708__li5547153113544">The number of concurrent import threads is 2.</li></ul>
|
|
</li><li id="EN-US_TOPIC_0000001188163708__li55481331145419">Start GDS and check whether it has been started.<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen1454783120545"><strong id="EN-US_TOPIC_0000001188163708__b754711311548">python3 gds_ctl.py start</strong></pre>
|
|
<p id="EN-US_TOPIC_0000001188163708__p45470314547">Example:</p>
|
|
<pre class="screen" id="EN-US_TOPIC_0000001188163708__screen205484317540"><strong id="EN-US_TOPIC_0000001188163708__b10547113175416">cd </strong>/opt/bin<em id="EN-US_TOPIC_0000001188163708__i195471631155417"><span id="EN-US_TOPIC_0000001188163708__ph095215409719">/dws</span>/gds/bin</em>
|
|
<strong id="EN-US_TOPIC_0000001188163708__b65471831145411">python3 gds_ctl.py start</strong>
|
|
Start GDS gds1 [OK]
|
|
gds [options]:
|
|
-d dir Set data directory.
|
|
-p port Set GDS listening port.
|
|
ip:port Set GDS listening ip address and port.
|
|
-l log_file Set log file.
|
|
-H secure_ip_range
|
|
Set secure IP checklist in CIDR notation. Required for GDS to start.
|
|
-e dir Set error log directory.
|
|
-E size Set size of per error log segment.(0 < size < 1TB)
|
|
-S size Set size of data segment.(1MB < size < 100TB)
|
|
-t worker_num Set number of worker thread in multi-thread mode, the upper limit is 200. If without setting, the default value is 8.
|
|
-s status_file Enable GDS status report.
|
|
-D Run the GDS as a daemon process.
|
|
-r Read the working directory recursively.
|
|
-h Display usage.</pre>
|
|
</li></ol>
|
|
</li></ul>
|
|
</div>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="EN-US_TOPIC_0000001188163708__sa01cca4537d1447db058fdf42644bfb2"><h4 class="sectiontitle">gds.conf Parameter Description</h4>
|
|
<div class="tablenoborder"><a name="EN-US_TOPIC_0000001188163708__t051f8c4ef816412c85e082e7fb7297dd"></a><a name="t051f8c4ef816412c85e082e7fb7297dd"></a><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001188163708__t051f8c4ef816412c85e082e7fb7297dd" frame="border" border="1" rules="all"><caption><b>Table 1 </b>gds.conf configuration description</caption><thead align="left"><tr id="EN-US_TOPIC_0000001188163708__r4f5e6597cae84ea19e5d0b3909cd80fa"><th align="left" class="cellrowborder" valign="top" width="17%" id="mcps1.3.4.2.2.4.1.1"><p id="EN-US_TOPIC_0000001188163708__ac70b8a11745b4b61bd83b13e841101ab">Attribute</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="35%" id="mcps1.3.4.2.2.4.1.2"><p id="EN-US_TOPIC_0000001188163708__acc5277e0100f40a7914c2032fdd3d523">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="48%" id="mcps1.3.4.2.2.4.1.3"><p id="EN-US_TOPIC_0000001188163708__a2e9d53280a6f415c928a58958c618e5d">Value Range</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="EN-US_TOPIC_0000001188163708__r21e685368ece487b982ed6ed110f6ea1"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__ab2072345bea04792a4b3cad8f1f41105">name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__a8bb36cd8758743cc9e373af368c944eb">Identifier</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__ad3e5b41f2e9c47e8ad0efba62bdaf8ae">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__rff1bfce1488f4fec9ba3b23808978220"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__a67f7f11fd0a941969e799e48cae8521b">ip</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__ae7e272f8847a4b298f2d8ab5c8bb50c7">Listening IP address</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__a4b054808742147f6a49b1e032093dcb6">The IP address must be valid.</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__ae57e98fb75e14cb0a0ef46666c3d988c">Default value: <strong id="EN-US_TOPIC_0000001188163708__b84235270614356">127.0.0.1</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__r8a148f9d4e1d4ba2831ab8c47a1093fe"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__a70b5c97d96e74bd080ccbdb744e087f7">port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__ab3c58bd77ae24a10baae0181d1205c53">Listening port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__ac6871934ceb2414f9f8b3db9446d374b">Value range: 1024 to 65535 (integer)</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__af15ef19754c643738177d2d6a28288fb">Default value: <strong id="EN-US_TOPIC_0000001188163708__b842352706195527">8098</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__re74190a47f204e8eab17597ebead03de"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__acee4731d8012439c8e49119f4c46dd3c">data_dir</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__a57ddbcafad934cd68f0f86752db7e125">Data file directory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__adfdad746ca324c979fc429555c4b2173">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__r445f37b495be4fe6bc30dac699419085"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__a25e6234239e74c2f9fd8973cd807c053">err_dir</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__a622f9ed418274d1796ececd7ae38334f">Error log file directory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__a3ac8f4320444434f9112fec7febfabec">Default value: data file directory</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__rc779d67c1629476da7eb96fa5a6cea85"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__aa49ab80ff5694889af4e08a333a71947">log_file</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__aa4aacd1ed6194f22957cd5c290193894">Log file Path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__aeb13ac5cf0d542acbc1739d535c5851d">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__r777f09ab9dc14f66b52cc3f4d17297df"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__aa71f7a4be1d54a79a922e3e5e5d3c612">host</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__a59627102e5d44a4b92aff1c0969977f8">Host IP address allowed to be connected to GDS (The value must in CIDR format and this parameter is available for the Linux OS only.)</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__a5f2ec4977f8140a3ad9f327b638fa92b">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__rfb93d38f66c942efa5c8840c24987827"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__a59cebce19d4345a3ac84b3869da63a53">recursive</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__a3bc7125facdb4bcdbbe743798aecdda9">Whether the data file directories are recursive</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__a637e8c45e150440c99ef7722f74de1c7">Value range:</p>
|
|
<ul id="EN-US_TOPIC_0000001188163708__u3b1dc280e9f84362a9d58eb3e8a3d629"><li id="EN-US_TOPIC_0000001188163708__l4c13b0b189e149c189bcaf45a201ab8d"><strong id="EN-US_TOPIC_0000001188163708__b842352706135927">true</strong>: recursive</li><li id="EN-US_TOPIC_0000001188163708__l76c548f54f0341c19cb0af274b209ac2"><strong id="EN-US_TOPIC_0000001188163708__b842352706135924">false</strong>: not recursive</li></ul>
|
|
<p id="EN-US_TOPIC_0000001188163708__a00f847e14b3b41d2938abe042c811ac7">Default value: <strong id="EN-US_TOPIC_0000001188163708__b842352706103049">false</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__rd1905902d63d4e95b92eece093993552"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__a906abb6417e744069ccf88c56ca48a57">daemon</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__ae2454e1ccc774a109a34ab408a99ee3d">Whether the process is running in daemon mode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__a8d5d038743c542de95effc3498c4044f">Value range:</p>
|
|
<ul id="EN-US_TOPIC_0000001188163708__ub88a998d58344e4aba43d68123b9cb1c"><li id="EN-US_TOPIC_0000001188163708__lc4aab905487e4820b87d5b50ccbfadbe"><strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0058968094_b84235270611137">true</strong>: The process is running in daemon mode.</li><li id="EN-US_TOPIC_0000001188163708__l19f17a492136479aa2930549ecf98046"><strong id="EN-US_TOPIC_0000001188163708__en-us_topic_0058968094_b84235270611159">false</strong>: The process is not running in daemon mode.</li></ul>
|
|
<p id="EN-US_TOPIC_0000001188163708__af264d4ea92a641d08b239a14e77dfb99">Default value: <strong id="EN-US_TOPIC_0000001188163708__b18941215174510">false</strong></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000001188163708__r24d1f7fba18b472a89c069ea0f51fe12"><td class="cellrowborder" valign="top" width="17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="EN-US_TOPIC_0000001188163708__a42b037afa95e451cb63567f773a681b5">parallel</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35%" headers="mcps1.3.4.2.2.4.1.2 "><p id="EN-US_TOPIC_0000001188163708__a440acda99f834267b35cd8192d644e68">Number of concurrent data import threads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="48%" headers="mcps1.3.4.2.2.4.1.3 "><p id="EN-US_TOPIC_0000001188163708__a2800fd1e06774aa49d13134b135595a6">Value range: 0 to 200 (integer)</p>
|
|
<p id="EN-US_TOPIC_0000001188163708__a09f96f8c34a04d11a641c6a3af192093">Default value: <strong id="EN-US_TOPIC_0000001188163708__b84235270620950">8</strong></p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0189.html">Using GDS to Import Data from a Remote Server</a></div>
|
|
</div>
|
|
</div>
|
|
|