vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0403.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

359 lines
36 KiB
HTML
Raw Blame History

<a name="dli_08_0403"></a><a name="dli_08_0403"></a>
<h1 class="topictitle1">GaussDB(DWS) Dimension Table</h1>
<div id="body8662426"><div class="section" id="dli_08_0403__en-us_topic_0000001262815690_section2093155184317"><h4 class="sectiontitle">Function</h4><p id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_p7798335163614">Create a GaussDB(DWS) table to connect to source streams for wide table generation.</p>
</div>
<div class="section" id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_section149656154515"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_en-us_topic_0111499975_ul810551215398"><li id="dli_08_0403__en-us_topic_0000001262815690_li6950202711198">Ensure that you have created a GaussDB(DWS) cluster using your account. </li><li id="dli_08_0403__en-us_topic_0000001262815690_li149509273195">A DWS database table has been created.</li><li id="dli_08_0403__en-us_topic_0000001262815690_li495011273191">An enhanced datasource connection has been created for DLI to connect to DWS clusters, so that jobs can run on the dedicated queue of DLI and you can set the security group rules as required.
</li></ul>
</div>
<div class="section" id="dli_08_0403__en-us_topic_0000001262815690_section18206153473218"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0403__en-us_topic_0000001262815690_p07569116563">When you create a Flink OpenSource SQL job, set <strong id="dli_08_0403__en-us_topic_0000001262815690_b626219134385">Flink Version</strong> to <strong id="dli_08_0403__en-us_topic_0000001262815690_b17790717113818">1.12</strong> in the <strong id="dli_08_0403__en-us_topic_0000001262815690_b5774920173816">Running Parameters</strong> tab. Select <strong id="dli_08_0403__en-us_topic_0000001262815690_b2611194023814">Save Job Log</strong>, and specify the OBS bucket for saving job logs.</p>
</div>
<div class="section" id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_section1691193353316"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0277_screen29519537283"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">create</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">dwsSource</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span>
<span class="p">)</span>
<span class="k">with</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'connector'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'gaussdb'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'url'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'table-name'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'username'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'password'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_section52043281918"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_table115011952171611" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row7534452171619"><th align="left" class="cellrowborder" valign="top" width="20.92209220922092%" id="mcps1.3.5.2.2.6.1.1"><p id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_p145430522169">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9.030903090309032%" id="mcps1.3.5.2.2.6.1.2"><p id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_p955415216164">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.58085808580858%" id="mcps1.3.5.2.2.6.1.3"><p id="dli_08_0403__en-us_topic_0000001262815690_p6395346941">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="9.59095909590959%" id="mcps1.3.5.2.2.6.1.4"><p id="dli_08_0403__en-us_topic_0000001262815690_p141615492048">Data Types</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="51.87518751875187%" id="mcps1.3.5.2.2.6.1.5"><p id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_p1056555281615">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row8575105261616"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p16266123442318">connector</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p142661034102314">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1239515461648">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p101644913414">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1526613419233">Connector type. Set this parameter to <strong id="dli_08_0403__en-us_topic_0000001262815690_b147343629544441">gaussdb</strong>.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row11645852191610"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1743276019">url</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1474427901">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p2395104619413">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p0167491045">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p245185011214">JDBC connection address.</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p17701919201416">If you use the gsjdbc4 driver, set the value in jdbc:postgresql://${ip}:${port}/${dbName} format.</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p14301195217217">If you use the gsjdbc200 driver, set the value in jdbc:gaussdb://${ip}:${port}/${dbName} format.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row166821052141616"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p926610342236">table-name</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p192664348233">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p16589192018227">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p17161498415">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p2536191418289">Name of the table where the data will be read from the database</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row1871675211610"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p2266173415231">driver</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p92669342234">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p173959468418">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p11618491240">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p20826516239">JDBC connection driver. The default value is <strong id="dli_08_0403__en-us_topic_0000001262815690_b1166012154318">org.postgresql.Driver</strong>.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row177441952131614"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p8266113414235">username</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1326653418238">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p15395134619418">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p91674918412">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1726613410230">Database authentication user name. This parameter must be configured in pair with <strong id="dli_08_0403__en-us_topic_0000001262815690_b1193622144319">password</strong>.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row69411319206"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p152661434112310">password</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p122667341236">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1487417243229">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p14161649441">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p14266134122317">Database authentication password. This parameter must be configured in pair with <strong id="dli_08_0403__en-us_topic_0000001262815690_b335094014435">username</strong>.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row189061849441"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p5707211344">scan.partition.column</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p62665344233">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p3973142582218">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p191634913417">String</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p10221143134012">Name of the column used to partition the input</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p553619143288">This parameter must be set when <strong id="dli_08_0403__en-us_topic_0000001262815690_b13575194310448">scan.partition.lower-bound</strong>, <strong id="dli_08_0403__en-us_topic_0000001262815690_b199815468443">scan.partition.upper-bound</strong>, and <strong id="dli_08_0403__en-us_topic_0000001262815690_b136231850144415">scan.partition.num</strong> are all configured, and should not be set when other three parameters are not.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_row4905049143"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p17372144312347">scan.partition.lower-bound</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p16266133472316">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p153952046445">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p181614491047">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p288018561266">Lower bound of values to be fetched for the first partition</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p1959201591213">This parameter must be set when <strong id="dli_08_0403__en-us_topic_0000001262815690_b10512133515612">scan.partition.column</strong>, <strong id="dli_08_0403__en-us_topic_0000001262815690_b10512193512566">scan.partition.upper-bound</strong>, and <strong id="dli_08_0403__en-us_topic_0000001262815690_b65139356560">scan.partition.num</strong> are all configured, and should not be set when other three parameters are not.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row523825412155"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1613335913345">scan.partition.upper-bound</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p192669347237">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p144661029182215">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p616174910410">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p121666952715">Upper bound of values to be fetched for the last partition</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p1817519503128">This parameter must be set when <strong id="dli_08_0403__en-us_topic_0000001262815690_b4116454463">scan.partition.column</strong>, <strong id="dli_08_0403__en-us_topic_0000001262815690_b16116165419616">scan.partition.lower-bound</strong>, and <strong id="dli_08_0403__en-us_topic_0000001262815690_b11161549619">scan.partition.num</strong> are all configured, and should not be set when other three parameters are not.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row164711259201519"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p2061411269345">scan.partition.num</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p13266234142320">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p183957467411">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p17163492049">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p169219816221">Number of partitions to be created</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p91551751111210">This parameter must be set when <strong id="dli_08_0403__en-us_topic_0000001262815690_b145671745276">scan.partition.column</strong>, <strong id="dli_08_0403__en-us_topic_0000001262815690_b15737451870">scan.partition.upper-bound</strong>, and <strong id="dli_08_0403__en-us_topic_0000001262815690_b2057412451275">scan.partition.upper-bound</strong> are all configured, and should not be set when other three parameters are not.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row681110575152"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p2689111263515">scan.fetch-size</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1326663452312">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p239517465419">0</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p121674917416">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p726673414232">Number of rows fetched from the database each time. The default value <strong id="dli_08_0403__en-us_topic_0000001262815690_b132081611892">0</strong> indicates that the number of rows is not limited.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row212837153512"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p193225389352">scan.auto-commit</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p412133793512">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p103955461548">true</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1416144915410">Boolean</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p18121437113514">Automatic commit flag.</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p1137773762812">It determines whether each statement is committed in a transaction automatically.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row1051145671512"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1227155019356">lookup.cache.max-rows</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1026603442311">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p8357173412228">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p41694920417">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p16266183413234">The max number of rows of lookup cache. Caches exceeding the TTL will be expired.</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p17903183613314">Lookup cache is disabled by default.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row716712523158"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p37601093615">lookup.cache.ttl</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p52661534182315">No</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p13266103418232"></p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1239514618411">None</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1616549243">Duration</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p526611342233">Maximum time to live (TTL) of for every rows in lookup cache. Caches exceeding the TTL will be expired. The format is {length value}{time unit label}, for example, <strong id="dli_08_0403__en-us_topic_0000001262815690_b9745840644441">123ms, 321s</strong>. The supported time units include d, h, min, s, and ms (default unit).</p>
<p id="dli_08_0403__en-us_topic_0000001262815690_p9272323163315">Lookup cache is disabled by default.</p>
</td>
</tr>
<tr id="dli_08_0403__en-us_topic_0000001262815690_row18932822173319"><td class="cellrowborder" valign="top" width="20.92209220922092%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0403__en-us_topic_0000001262815690_p114031010103611">lookup.max-retries</p>
</td>
<td class="cellrowborder" valign="top" width="9.030903090309032%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0403__en-us_topic_0000001262815690_p293314224335">No</p>
</td>
<td class="cellrowborder" valign="top" width="8.58085808580858%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0403__en-us_topic_0000001262815690_p1439574617410">3</p>
</td>
<td class="cellrowborder" valign="top" width="9.59095909590959%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0403__en-us_topic_0000001262815690_p161611491416">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="51.87518751875187%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0403__en-us_topic_0000001262815690_p99331322133315">Maximum retry times if lookup database failed.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_section13376151217347"><h4 class="sectiontitle">Example</h4><p id="dli_08_0403__en-us_topic_0000001262815690_dli_08_0261_p88543522162">Read data from a Kafka source table, use a GaussDB(DWS) table as the dimension table. Write wide table information generated by the source and dimension tables to a Kafka result table. The procedure is as follows:</p>
<ol id="dli_08_0403__en-us_topic_0000001262815690_ol498915612390"><li id="dli_08_0403__en-us_topic_0000001262815690_li098966153920">Create an enhanced datasource connection in the VPC and subnet where DWS and Kafka locate, and bind the connection to the required Flink elastic resource pool.</li><li id="dli_08_0403__en-us_topic_0000001262815690_li1598914683919">Set GaussDB(DWS) and Kafka security groups and add inbound rules to allow access from the Flink queue. Test the connectivity using the DWS and Kafka address. If the connection passes the test, it is bound to the queue.</li><li id="dli_08_0403__en-us_topic_0000001262815690_li99891967396">Connect to the GaussDB(DWS) database instance, create a table as a dimension table, and name the table <strong id="dli_08_0403__en-us_topic_0000001262815690_b122501430389">area_info</strong>. Example SQL statements are as follows:<pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen8583154151013">create table public.area_info(
area_id VARCHAR,
area_province_name VARCHAR,
area_city_name VARCHAR,
area_county_name VARCHAR,
area_street_name VARCHAR,
region_name VARCHAR);</pre>
</li><li id="dli_08_0403__en-us_topic_0000001262815690_li169891460397">Connect to the database and run the following statement to insert test data into the dimension table <strong id="dli_08_0403__en-us_topic_0000001262815690_b44717220435">area_info</strong>:<pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen17295554152716"> insert into area_info
(area_id, area_province_name, area_city_name, area_county_name, area_street_name, region_name)
values
('330102', 'a1', 'b1', 'c1', 'd1', 'e1'),
('330106', 'a1', 'b1', 'c2', 'd2', 'e1'),
('330108', 'a1', 'b1', 'c3', 'd3', 'e1'),
('330110', 'a1', 'b1', 'c4', 'd4', 'e1');</pre>
</li><li id="dli_08_0403__en-us_topic_0000001262815690_li17989269397">Create a Flink OpenSource SQL job Enter the following job script and submit the job. The job script uses Kafka as the data source and a GaussDB(DWS) table as the dimension table. Data is output to a Kafka result table.<div class="p" id="dli_08_0403__en-us_topic_0000001262815690_p17823134583219"><a name="dli_08_0403__en-us_topic_0000001262815690_li17989269397"></a><a name="en-us_topic_0000001262815690_li17989269397"></a>When you create a job, set <strong id="dli_08_0403__en-us_topic_0000001262815690_b72895493155">Flink Version</strong> to <strong id="dli_08_0403__en-us_topic_0000001262815690_b2290124941516">1.12</strong> in the <strong id="dli_08_0403__en-us_topic_0000001262815690_b17291249171513">Running Parameters</strong> tab. Select <strong id="dli_08_0403__en-us_topic_0000001262815690_b0292649171519">Save Job Log</strong>, and specify the OBS bucket for saving job logs. <strong id="dli_08_0403__en-us_topic_0000001262815690_b88822461320">Set the values of the parameters in bold in the following script as needed.</strong><pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen2909144183214">CREATE TABLE orders (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string,
proctime as Proctime()
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i4543155265320"><strong id="dli_08_0403__en-us_topic_0000001262815690_b1754316528535">KafkaSource</strong></em><em id="dli_08_0403__en-us_topic_0000001262815690_i15341759195319"><strong id="dli_08_0403__en-us_topic_0000001262815690_b5227105913538">Topic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i9319103215313"><strong id="dli_08_0403__en-us_topic_0000001262815690_b0322115133213">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = 'dws-order',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
-- Create an address dimension table
create table area_info (
area_id string,
area_province_name string,
area_city_name string,
area_county_name string,
area_street_name string,
region_name string
) WITH (
'connector' = 'gaussdb',
'driver' = 'org.postgresql.Driver',
'url' = 'jdbc:gaussdb://<em id="dli_08_0403__en-us_topic_0000001262815690_i394816595331"><strong id="dli_08_0403__en-us_topic_0000001262815690_b1680875873314">DwsAddress:DwsPort</strong></em>/<em id="dli_08_0403__en-us_topic_0000001262815690_i15894664376"><strong id="dli_08_0403__en-us_topic_0000001262815690_b1894176133713">Dws</strong></em><em id="dli_08_0403__en-us_topic_0000001262815690_i92608177377"><strong id="dli_08_0403__en-us_topic_0000001262815690_b095351633716">DbName</strong></em>',
'table-name' = 'area_info',
'username' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i1861341433519"><strong id="dli_08_0403__en-us_topic_0000001262815690_b961411411359">Dws</strong></em><em id="dli_08_0403__en-us_topic_0000001262815690_i10513112314351"><strong id="dli_08_0403__en-us_topic_0000001262815690_b5241162315359">UserName</strong></em>',
'password' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i14664919143514"><strong id="dli_08_0403__en-us_topic_0000001262815690_b1666481910355">Dws</strong></em><em id="dli_08_0403__en-us_topic_0000001262815690_i185861225103514"><strong id="dli_08_0403__en-us_topic_0000001262815690_b13340162523512">Password</strong></em>',
'lookup.cache.max-rows' = '10000',
'lookup.cache.ttl' = '2h'
);
-- Generate a wide table based on the address dimension table containing detailed order information.
create table order_detail(
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string,
area_province_name string,
area_city_name string,
area_county_name string,
area_street_name string,
region_name string
) with (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i130544793419"><strong id="dli_08_0403__en-us_topic_0000001262815690_b9305184715347">KafkaSink</strong></em><em id="dli_08_0403__en-us_topic_0000001262815690_i15305547173418"><strong id="dli_08_0403__en-us_topic_0000001262815690_b930544715344">Topic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i840173103510"><strong id="dli_08_0403__en-us_topic_0000001262815690_b164020303518">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'format' = 'json'
);
insert into order_detail
select orders.order_id, orders.order_channel, orders.order_time, orders.pay_amount, orders.real_pay, orders.pay_time, orders.user_id, orders.user_name,
area.area_id, area.area_province_name, area.area_city_name, area.area_county_name,
area.area_street_name, area.region_name from orders
left join area_info for system_time as of orders.proctime as area on orders.area_id = area.area_id;</pre>
</div>
</li><li id="dli_08_0403__en-us_topic_0000001262815690_li1998926183919">Connect to the Kafka cluster and insert the following test data into the source topic in Kafka:<pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen1877314256281">{"order_id":"202103241606060001", "order_channel":"appShop", "order_time":"2021-03-24 16:06:06", "pay_amount":"200.00", "real_pay":"180.00", "pay_time":"2021-03-24 16:10:06", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}
{"order_id":"202103251202020001", "order_channel":"miniAppShop", "order_time":"2021-03-25 12:02:02", "pay_amount":"60.00", "real_pay":"60.00", "pay_time":"2021-03-25 12:03:00", "user_id":"0002", "user_name":"Bob", "area_id":"330110"}
{"order_id":"202103251505050001", "order_channel":"qqShop", "order_time":"2021-03-25 15:05:05", "pay_amount":"500.00", "real_pay":"400.00", "pay_time":"2021-03-25 15:10:00", "user_id":"0003", "user_name":"Cindy", "area_id":"330108"}</pre>
</li><li id="dli_08_0403__en-us_topic_0000001262815690_li99891614394">Connect to the Kafka cluster and read data from the sink topic of Kafka. The result is as follows:<pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen219253572817">{"order_id":"202103241606060001","order_channel":"appShop","order_time":"2021-03-24 16:06:06","pay_amount":200.0,"real_pay":180.0,"pay_time":"2021-03-24 16:10:06","user_id":"0001","user_name":"Alice","area_id":"330106","area_province_name":"a1","area_city_name":"b1","area_county_name":"c2","area_street_name":"d2","region_name":"e1"}
{"order_id":"202103251202020001","order_channel":"miniAppShop","order_time":"2021-03-25 12:02:02","pay_amount":60.0,"real_pay":60.0,"pay_time":"2021-03-25 12:03:00","user_id":"0002","user_name":"Bob","area_id":"330110","area_province_name":"a1","area_city_name":"b1","area_county_name":"c4","area_street_name":"d4","region_name":"e1"}
{"order_id":"202103251505050001","order_channel":"qqShop","order_time":"2021-03-25 15:05:05","pay_amount":500.0,"real_pay":400.0,"pay_time":"2021-03-25 15:10:00","user_id":"0003","user_name":"Cindy","area_id":"330108","area_province_name":"a1","area_city_name":"b1","area_county_name":"c3","area_street_name":"d3","region_name":"e1"}</pre>
</li></ol>
</div>
<div class="section" id="dli_08_0403__en-us_topic_0000001262815690_section135087497515"><h4 class="sectiontitle">FAQs</h4><ul id="dli_08_0403__en-us_topic_0000001262815690_ul141114560153"><li id="dli_08_0403__en-us_topic_0000001262815690_li121131244203916">Q: What should I do if Flink job logs contain the following error information?<pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen16117181494">java.io.IOException: unable to open JDBC writer
...
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
...
Caused by: java.net.SocketTimeoutException: connect timed out</pre>
<div class="p" id="dli_08_0403__en-us_topic_0000001262815690_p116875264365">A: The datasource connection is not bound or the binding fails.
</div>
</li></ul>
<ul id="dli_08_0403__en-us_topic_0000001262815690_ul3321184451210"><li id="dli_08_0403__en-us_topic_0000001262815690_li032114411215">Q: How can I configure a GaussDB(DWS) table that is in a schema?<div class="p" id="dli_08_0403__en-us_topic_0000001262815690_p460925613243"><a name="dli_08_0403__en-us_topic_0000001262815690_li032114411215"></a><a name="en-us_topic_0000001262815690_li032114411215"></a>A: In the following example configures the <strong id="dli_08_0403__en-us_topic_0000001262815690_b1915018516598">area_info</strong> table in the <strong id="dli_08_0403__en-us_topic_0000001262815690_b5250954115910">dbuser2</strong> schema.<pre class="screen" id="dli_08_0403__en-us_topic_0000001262815690_screen1886617541249">-- Create an address dimension table
create table area_info (
area_id string,
area_province_name string,
area_city_name string,
area_county_name string,
area_street_name string,
region_name string
) WITH (
'connector' = 'gaussdb',
'driver' = 'org.postgresql.Driver',
'url' = 'jdbc:postgresql://<strong id="dli_08_0403__en-us_topic_0000001262815690_b981414404110">DwsAddress:DwsPort/DwsDbname</strong>',
'table-name' = '<strong id="dli_08_0403__en-us_topic_0000001262815690_b156772511258">dbuser2.area_info</strong>',
'username' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i7973151815412"><strong id="dli_08_0403__en-us_topic_0000001262815690_b12316103410">Dws</strong><strong id="dli_08_0403__en-us_topic_0000001262815690_b206591918114115">UserName</strong></em>',
'password' = '<em id="dli_08_0403__en-us_topic_0000001262815690_i194432118413"><strong id="dli_08_0403__en-us_topic_0000001262815690_b1246615134116">Dws</strong><strong id="dli_08_0403__en-us_topic_0000001262815690_b1678220201415">Password</strong></em>',
'lookup.cache.max-rows' = '10000',
'lookup.cache.ttl' = '2h'
);</pre>
</div>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0402.html">Creating Dimension Tables</a></div>
</div>
</div>
View Git Blame Copy Permalink