vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0383.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

287 lines
30 KiB
HTML
Raw Blame History

<a name="dli_08_0383"></a><a name="dli_08_0383"></a>
<h1 class="topictitle1">GaussDB(DWS) Source Table</h1>
<div id="body8662426"><div class="section" id="dli_08_0383__en-us_topic_0000001262495782_dli_08_0277_en-us_topic_0113887276_en-us_topic_0111499973_section13926185035213"><h4 class="sectiontitle">Function</h4><p id="dli_08_0383__en-us_topic_0000001262495782_p161343396413">DLI reads data of Flink jobs from GaussDB(DWS). GaussDB(DWS) database kernel is compliant with PostgreSQL. The PostgreSQL database can store data of more complex types and deliver space information services, multi-version concurrent control (MVCC), and high concurrency. It applies to location applications, financial insurance, and e-Commerce.</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p113419391416">GaussDB(DWS) is an online data processing database based on the cloud infrastructure and platform and helps you mine and analyze massive sets of data. </p>
</div>
<div class="section" id="dli_08_0383__en-us_topic_0000001262495782_section121878591648"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0383__en-us_topic_0000001262495782_ul182711681452"><li id="dli_08_0383__en-us_topic_0000001262495782_li162711780513">You have created a GaussDB(DWS) cluster.<p id="dli_08_0383__en-us_topic_0000001262495782_p2027138057"><a name="dli_08_0383__en-us_topic_0000001262495782_li162711780513"></a><a name="en-us_topic_0000001262495782_li162711780513"></a>For details about how to create a GaussDB(DWS) cluster, see <span class="filepath" id="dli_08_0383__en-us_topic_0000001262495782_filepath12302932045255"><b>Creating a Cluster</b></span> in the <em id="dli_08_0383__en-us_topic_0000001262495782_i140638753845255">Data Warehouse Service Management Guide</em>.</p>
</li><li id="dli_08_0383__en-us_topic_0000001262495782_li17271198757">You have created a GaussDB(DWS) database table.</li><li id="dli_08_0383__en-us_topic_0000001262495782_li1627119813515">An enhanced datasource connection has been created for DLI to connect to GaussDB(DWS) clusters, so that jobs can run on the dedicated queue of DLI and you can set the security group rules as required.
</li></ul>
</div>
<div class="section" id="dli_08_0383__en-us_topic_0000001262495782_section17666546857"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0383__en-us_topic_0000001262495782_p17405125142112">When creating a Flink OpenSource SQL job, you need to set <strong id="dli_08_0383__en-us_topic_0000001262495782_b1280612515910">Flink Version</strong> to <strong id="dli_08_0383__en-us_topic_0000001262495782_b3806151590">1.12</strong> on the <strong id="dli_08_0383__en-us_topic_0000001262495782_b8806135996">Running Parameters</strong> tab of the job editing page, select <strong id="dli_08_0383__en-us_topic_0000001262495782_b28073516915">Save Job Log</strong>, and set the OBS bucket for saving job logs.</p>
</div>
<div class="section" id="dli_08_0383__en-us_topic_0000001262495782_dli_08_0277_en-us_topic_0113887276_en-us_topic_0111499973_section1810912715521"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0383__en-us_topic_0000001262495782_dli_08_0277_screen29519537283"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">create</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">dwsSource</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="p">,</span><span class="w"> </span><span class="p">...)</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="n">ENFORCED</span><span class="p">)</span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">watermark</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">rowtime_column_name</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">watermark</span><span class="o">-</span><span class="n">strategy_expression</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">with</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'connector'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'gaussdb'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'url'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'table-name'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'username'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'password'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0383__en-us_topic_0000001262495782_dli_08_0277_section1594243973712"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0383__en-us_topic_0000001262495782_table22661434122314" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0383__en-us_topic_0000001262495782_row32664346238"><th align="left" class="cellrowborder" valign="top" width="13.59%" id="mcps1.3.5.2.2.6.1.1"><p id="dli_08_0383__en-us_topic_0000001262495782_p126653482318">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="7.85%" id="mcps1.3.5.2.2.6.1.2"><p id="dli_08_0383__en-us_topic_0000001262495782_p14266193413234">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9.85%" id="mcps1.3.5.2.2.6.1.3"><p id="dli_08_0383__en-us_topic_0000001262495782_p106158534431">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9.719999999999999%" id="mcps1.3.5.2.2.6.1.4"><p id="dli_08_0383__en-us_topic_0000001262495782_p19515165614316">Data Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="58.989999999999995%" id="mcps1.3.5.2.2.6.1.5"><p id="dli_08_0383__en-us_topic_0000001262495782_p5266193419235">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0383__en-us_topic_0000001262495782_row142669342232"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p16266123442318">connector</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p142661034102314">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1161514533436">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p85151569435">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1526613419233">Connector to be used. Set this parameter to <strong id="dli_08_0383__en-us_topic_0000001262495782_b170705391145255">gaussdb</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row974927301"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1743276019">url</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1474427901">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1161519532438">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1851525617439">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p113191714111819">JDBC connection address. Set the IP address in this parameter to the internal IP address of GaussDB(DWS).</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p17701919201416">If you use the gsjdbc4 driver, set the value in jdbc:postgresql://${ip}:${port}/${dbName} format.</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p87442714011">If you use the gsjdbc200 driver, set the value in jdbc:gaussdb://${ip}:${port}/${dbName} format.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row12266123402313"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p926610342236">table-name</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p192664348233">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p16151353114311">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1051555615437">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p145255595376">Name of the GaussDB(DWS) table to be operated. If the GaussDB(DWS) table is in a schema, refer to the description of <a href="#dli_08_0383__en-us_topic_0000001262495782_li032114411215">GaussDB(DWS) table in a schema</a>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row12266113492315"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p2266173415231">driver</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p92669342234">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p6548133115816">org.postgresql.Driver</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p351585618432">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p18327103319456">JDBC connection driver. The default value is <strong id="dli_08_0383__en-us_topic_0000001262495782_b24851322161713">org.postgresql.Driver</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row142665341232"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p8266113414235">username</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1326653418238">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p16615115334316">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p4515956144319">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1726613410230">Username for GaussDB(DWS) database authentication. This parameter must be configured in pair with <strong id="dli_08_0383__en-us_topic_0000001262495782_b13826151061917">password</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row62661034182310"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p152661434112310">password</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p122667341236">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p16615155310434">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1551517565436">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p14266134122317">Password for GaussDB(DWS) database authentication. This parameter must be configured in pair with <strong id="dli_08_0383__en-us_topic_0000001262495782_b14461105718195">username</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row526616340231"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p18266153418237">scan.partition.column</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p62665344233">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p361510537432">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p16515956174316">String</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p10221143134012">Name of the column used to partition the input.</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p553619143288">Note: This parameter must be used together with <strong id="dli_08_0383__en-us_topic_0000001262495782_b47194306228">scan.partition.lower-bound</strong>, <strong id="dli_08_0383__en-us_topic_0000001262495782_b3388113312214">scan.partition.upper-bound</strong>, and <strong id="dli_08_0383__en-us_topic_0000001262495782_b17246134112225">scan.partition.num</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row526683432311"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p2266133492313">scan.partition.lower-bound</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p16266133472316">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p3615853194314">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1251555654311">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p288018561266">Lower bound of values to be fetched for the first partition.</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p1959201591213">This parameter must be used together with <strong id="dli_08_0383__en-us_topic_0000001262495782_b655514713147">scan.partition.column</strong>, <strong id="dli_08_0383__en-us_topic_0000001262495782_b196625116145">scan.partition.upper-bound</strong>, and <strong id="dli_08_0383__en-us_topic_0000001262495782_b68431958131413">scan.partition.num</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row7266434162314"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1326633422313">scan.partition.upper-bound</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p192669347237">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p126541540155814">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p651511566430">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p93461809428">Upper bound of values to be fetched for the last partition.</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p121666952715">This parameter must be used together with <strong id="dli_08_0383__en-us_topic_0000001262495782_b141263218159">scan.partition.column</strong>, <strong id="dli_08_0383__en-us_topic_0000001262495782_b3649193411152">scan.partition.lower-bound</strong>, and <strong id="dli_08_0383__en-us_topic_0000001262495782_b1324644091512">scan.partition.num</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row2266434202311"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1126623416231">scan.partition.num</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p13266234142320">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p2615155364316">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1751555616438">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p169219816221">Number of partitions to be created.</p>
<p id="dli_08_0383__en-us_topic_0000001262495782_p91551751111210">This parameter must be used together with <strong id="dli_08_0383__en-us_topic_0000001262495782_b12114458171518">scan.partition.column</strong>, <strong id="dli_08_0383__en-us_topic_0000001262495782_b143944121613">scan.partition.upper-bound</strong>, and <strong id="dli_08_0383__en-us_topic_0000001262495782_b382514713164">scan.partition.upper-bound</strong>.</p>
</td>
</tr>
<tr id="dli_08_0383__en-us_topic_0000001262495782_row5266123415232"><td class="cellrowborder" valign="top" width="13.59%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0383__en-us_topic_0000001262495782_p8266734142316">scan.fetch-size</p>
</td>
<td class="cellrowborder" valign="top" width="7.85%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1326663452312">No</p>
</td>
<td class="cellrowborder" valign="top" width="9.85%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0383__en-us_topic_0000001262495782_p1861525384320">0</p>
</td>
<td class="cellrowborder" valign="top" width="9.719999999999999%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0383__en-us_topic_0000001262495782_p195151656174315">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="58.989999999999995%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0383__en-us_topic_0000001262495782_p726673414232">Number of rows fetched from the database each time. The default value is <strong id="dli_08_0383__en-us_topic_0000001262495782_b1558964816181">0</strong>, indicating that the number of rows is not limited.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0383__en-us_topic_0000001262495782_dli_08_0277_en-us_topic_0113887276_en-us_topic_0111499973_section444391525220"><h4 class="sectiontitle">Example</h4><p id="dli_08_0383__en-us_topic_0000001262495782_p11305132402910">In this example, data is read from the GaussDB(DWS) data source and written to the Print result table. The procedure is as follows:</p>
<ol id="dli_08_0383__en-us_topic_0000001262495782_ol533016229516"><li id="dli_08_0383__en-us_topic_0000001262495782_li1033162215515">Create a table named <strong id="dli_08_0383__en-us_topic_0000001262495782_b1573518354226">dws_order</strong> in GaussDB(DWS).<pre class="screen" id="dli_08_0383__en-us_topic_0000001262495782_screen194207919408">create table public.dws_order(
order_id VARCHAR,
order_channel VARCHAR,
order_time VARCHAR,
pay_amount FLOAT8,
real_pay FLOAT8,
pay_time VARCHAR,
user_id VARCHAR,
user_name VARCHAR,
area_id VARCHAR);</pre>
<div class="p" id="dli_08_0383__en-us_topic_0000001262495782_p723114115378">Insert data into the <strong id="dli_08_0383__en-us_topic_0000001262495782_b6101132572912">dws_order</strong> table.<pre class="screen" id="dli_08_0383__en-us_topic_0000001262495782_screen68811823115016">insert into public.dws_order
(order_id,
order_channel,
order_time,
pay_amount,
real_pay,
pay_time,
user_id,
user_name,
area_id) values
('202103241000000001', 'webShop', '2021-03-24 10:00:00', '100.00', '100.00', '2021-03-24 10:02:03', '0001', 'Alice', '330106'),
('202103251202020001', 'miniAppShop', '2021-03-25 12:02:02', '60.00', '60.00', '2021-03-25 12:03:00', '0002', 'Bob', '330110');</pre>
</div>
</li><li id="dli_08_0383__en-us_topic_0000001262495782_li6127741153217">Create an enhanced datasource connection in the VPC and subnet where GaussDB(DWS) locates, and bind the connection to the required Flink elastic resource pool.</li><li id="dli_08_0383__en-us_topic_0000001262495782_li1233002211514">Set GaussDB(DWS) security groups and add inbound rules to allow access from the Flink queue. Test the connectivity using the GaussDB(DWS) address. If the connection is successful, the datasource is bound to the queue. Otherwise, the binding fails.</li><li id="dli_08_0383__en-us_topic_0000001262495782_li11292155921016">Create a Flink OpenSource SQL job. Enter the following job script and submit the job. The job script uses the GaussDB(DWS) data source and the Print result table.<div class="p" id="dli_08_0383__en-us_topic_0000001262495782_p349610071117"><a name="dli_08_0383__en-us_topic_0000001262495782_li11292155921016"></a><a name="en-us_topic_0000001262495782_li11292155921016"></a>When you create a job, set <strong id="dli_08_0383__en-us_topic_0000001262495782_b7669255210">Flink Version</strong> to <strong id="dli_08_0383__en-us_topic_0000001262495782_b18661827527">1.12</strong> on the <strong id="dli_08_0383__en-us_topic_0000001262495782_b14667215526">Running Parameters</strong> tab. Select <strong id="dli_08_0383__en-us_topic_0000001262495782_b16710235214">Save Job Log</strong>, and specify the OBS bucket for saving job logs. <strong id="dli_08_0383__en-us_topic_0000001262495782_b136910349527">Change the values of the parameters in bold as needed in the following script.</strong><pre class="screen" id="dli_08_0383__en-us_topic_0000001262495782_screen861833362619">CREATE TABLE dwsSource (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'gaussdb',
'url' = 'jdbc:postgresql://<em id="dli_08_0383__en-us_topic_0000001262495782_i539711516437"><strong id="dli_08_0383__en-us_topic_0000001262495782_b356461184710">DWSIP</strong></em><em id="dli_08_0383__en-us_topic_0000001262495782_i5451059114619">:</em><em id="dli_08_0383__en-us_topic_0000001262495782_i711916716448"><strong id="dli_08_0383__en-us_topic_0000001262495782_b793023204719">DWSPort</strong></em><em id="dli_08_0383__en-us_topic_0000001262495782_i545159114619">/</em><em id="dli_08_0383__en-us_topic_0000001262495782_i119922214444"><strong id="dli_08_0383__en-us_topic_0000001262495782_b12452624718">DWS</strong><strong id="dli_08_0383__en-us_topic_0000001262495782_b16451665472">dbName</strong></em>',
'table-name' = 'dws_order',
'driver' = 'org.postgresql.Driver',
'username' = '<em id="dli_08_0383__en-us_topic_0000001262495782_i582314401442"><strong id="dli_08_0383__en-us_topic_0000001262495782_b4884158104711">DWSUserName</strong></em>',
'password' = '<em id="dli_08_0383__en-us_topic_0000001262495782_i993394210449"><strong id="dli_08_0383__en-us_topic_0000001262495782_b15765101017476">DWSPassword</strong></em>'
);
CREATE TABLE printSink (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'print'
);
insert into printSink select * from dwsSource;</pre>
</div>
</li><li id="dli_08_0383__en-us_topic_0000001262495782_li163311122555">Perform the following operations to view the data result in the <strong id="dli_08_0383__en-us_topic_0000001262495782_b12613112812538">taskmanager.out</strong> file:<ol type="a" id="dli_08_0383__en-us_topic_0000001262495782_ol864115198285"><li id="dli_08_0383__en-us_topic_0000001262495782_li10901621122819">Log in to the DLI console. In the navigation pane, choose <strong id="dli_08_0383__en-us_topic_0000001262495782_b181122323534">Job Management</strong> &gt; <strong id="dli_08_0383__en-us_topic_0000001262495782_b611283285319">Flink Jobs</strong>.</li><li id="dli_08_0383__en-us_topic_0000001262495782_li1912163912282">Click the name of the corresponding Flink job, choose <strong id="dli_08_0383__en-us_topic_0000001262495782_b9856640105316">Run Log</strong>, click <strong id="dli_08_0383__en-us_topic_0000001262495782_b178628408534">OBS Bucket</strong>, and locate the folder of the log you want to view according to the date.</li><li id="dli_08_0383__en-us_topic_0000001262495782_li0641191914285">Go to the folder of the date, find the folder whose name contains <strong id="dli_08_0383__en-us_topic_0000001262495782_b119815571533">taskmanager</strong>, download the <strong id="dli_08_0383__en-us_topic_0000001262495782_b18104155717533">taskmanager.out</strong> file, and view result logs.</li></ol>
<p id="dli_08_0383__en-us_topic_0000001262495782_p115301915124812">The data result is as follows:</p>
<pre class="screen" id="dli_08_0383__en-us_topic_0000001262495782_screen1253018150482">+I(202103241000000001,webShop,2021-03-24 10:00:00,100.0,100.0,2021-03-24 10:02:03,0001,Alice,330106)
+I(202103251202020001,miniAppShop,2021-03-25 12:02:02,60.0,60.0,2021-03-25 12:03:00,0002,Bob,330110)</pre>
</li></ol>
</div>
<div class="section" id="dli_08_0383__en-us_topic_0000001262495782_section81622487610"><h4 class="sectiontitle">FAQ</h4><ul id="dli_08_0383__en-us_topic_0000001262495782_ul02692574612"><li id="dli_08_0383__en-us_topic_0000001262495782_li911113565153">Q: What should I do if the job execution fails and the log contains the following error information?<pre class="screen" id="dli_08_0383__en-us_topic_0000001262495782_screen16823191517519">java.io.IOException: unable to open JDBC writer
...
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
...
Caused by: java.net.SocketTimeoutException: connect timed out</pre>
<div class="p" id="dli_08_0383__en-us_topic_0000001262495782_p1329122071212">A: The datasource connection is not bound or the binding fails.
</div>
</li></ul>
</div>
<ul id="dli_08_0383__en-us_topic_0000001262495782_ul7510655181415"><li id="dli_08_0383__en-us_topic_0000001262495782_li032114411215"><a name="dli_08_0383__en-us_topic_0000001262495782_li032114411215"></a><a name="en-us_topic_0000001262495782_li032114411215"></a>Q: How can I configure a GaussDB(DWS) table that is in a schema?<p id="dli_08_0383__en-us_topic_0000001262495782_p460925613243"><a name="dli_08_0383__en-us_topic_0000001262495782_li032114411215"></a><a name="en-us_topic_0000001262495782_li032114411215"></a>A: The following provides an example of configuring the <strong id="dli_08_0383__en-us_topic_0000001262495782_b1212664017313">dws_order</strong> table in the <strong id="dli_08_0383__en-us_topic_0000001262495782_b913216401831">dbuser2</strong> schema:</p>
<pre class="screen" id="dli_08_0383__en-us_topic_0000001262495782_screen33814269157">CREATE TABLE dwsSource (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'gaussdb',
'url' = 'jdbc:postgresql://<em id="dli_08_0383__en-us_topic_0000001262495782_i34111639521">D</em><em id="dli_08_0383__en-us_topic_0000001262495782_i775025165216">WSIP:DWSPort/DWSdbName</em>',
'table-name' = '<strong id="dli_08_0383__en-us_topic_0000001262495782_b15370159501">dbuser2\".\"dws_order</strong>',
'driver' = 'org.postgresql.Driver',
'username' = '<em id="dli_08_0383__en-us_topic_0000001262495782_i3977164617522">DWSUserName</em>',
'password' = '<em id="dli_08_0383__en-us_topic_0000001262495782_i1216095055218">DWSPassword</em>'
);</pre>
</li></ul>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0381.html">Creating Source Tables</a></div>
</div>
</div>
View Git Blame Copy Permalink