vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0397.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

380 lines
36 KiB
HTML
Raw Blame History

<a name="dli_08_0397"></a><a name="dli_08_0397"></a>
<h1 class="topictitle1">JDBC Result Table</h1>
<div id="body8662426"><div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section1380110114219"><h4 class="sectiontitle">Function</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p17594174162711">DLI outputs the Flink job output data to RDS through the JDBC result table.</p>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section7221153411"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p4749103917489">An enhanced datasource connection with the instances has been established, so that you can configure security group rules as required.</p>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_section962153220221"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0397__en-us_topic_0000001262335930_ul168810224217"><li id="dli_08_0397__en-us_topic_0000001262335930_li1787611685815">When creating a Flink OpenSource SQL job, you need to set <strong id="dli_08_0397__en-us_topic_0000001262335930_b1703819356">Flink Version</strong> to <strong id="dli_08_0397__en-us_topic_0000001262335930_b2119811350">1.12</strong> on the <strong id="dli_08_0397__en-us_topic_0000001262335930_b8219818352">Running Parameters</strong> tab of the job editing page, select <strong id="dli_08_0397__en-us_topic_0000001262335930_b8218103517">Save Job Log</strong>, and set the OBS bucket for saving job logs.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li123134519351">The connector operates in upsert mode if the primary key was defined; otherwise, the connector operates in append mode.<ul id="dli_08_0397__en-us_topic_0000001262335930_ul165969505431"><li id="dli_08_0397__en-us_topic_0000001262335930_li112157463431">In upsert mode, Flink will insert a new row or update the existing row according to the primary key. Flink can ensure the idempotence in this way. To guarantee the output result is as expected, it is recommended to define a primary key for the table.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li678424715435">In append mode, Flink will interpret all records as INSERT messages. The INSERT operation may fail if a primary key or unique constraint violation happens in the underlying database.</li></ul>
</li></ul>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section24316432437"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0243_screen746013311958"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">create</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">jdbcSink</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="p">,</span><span class="w"> </span><span class="p">...)</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="n">ENFORCED</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">with</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'connector'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'jdbc'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'url'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'table-name'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'driver'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'username'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'password'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_section158861859181416"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0397__en-us_topic_0000001262335930_table3535131492817" frame="border" border="1" rules="all"><thead align="left"><tr id="dli_08_0397__en-us_topic_0000001262335930_row053518147287"><th align="left" class="cellrowborder" valign="top" width="16.72%" id="mcps1.3.5.2.1.6.1.1"><p id="dli_08_0397__en-us_topic_0000001262335930_p153513145285">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.19%" id="mcps1.3.5.2.1.6.1.2"><p id="dli_08_0397__en-us_topic_0000001262335930_p10535014202815">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.08%" id="mcps1.3.5.2.1.6.1.3"><p id="dli_08_0397__en-us_topic_0000001262335930_p519717173594">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.309999999999999%" id="mcps1.3.5.2.1.6.1.4"><p id="dli_08_0397__en-us_topic_0000001262335930_p41981824182510">Data Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="54.7%" id="mcps1.3.5.2.1.6.1.5"><p id="dli_08_0397__en-us_topic_0000001262335930_p19535151482813">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0397__en-us_topic_0000001262335930_row653531413287"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p117141241192315">connector</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p81135102314">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p121977179596">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p91981924142514">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1471424117235">Connector to be used. Set this parameter to <strong id="dli_08_0397__en-us_topic_0000001262335930_b1040311111511">jdbc</strong>.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row65351614172820"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15535141432819">url</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1653517144282">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p91971417175910">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p61987241254">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p12535914122818">Database URL.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row11535151410283"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p17535131482814">table-name</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p653621414283">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p19197141712593">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p171981424142513">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2536191418289">Name of the table where the data will be read from the database.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row753618145283"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p95361214202815">driver</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1353613148281">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1234113515504">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1919816241258">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1453631462817">Driver required for connecting to the database. If you do not set this parameter, it will be automatically derived from the URL.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row195361114142812"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p105364142285">username</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1128519229372">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p159012366509">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p71981424112514">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p175361714162814">Database authentication username. This parameter must be configured in pair with <strong id="dli_08_0397__en-us_topic_0000001262335930_b14117625773">password</strong>.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row553641414283"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6536614122819">password</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6536131442814">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p619719176592">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p7198142472520">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p953641462814">Database authentication password. This parameter must be configured in pair with <strong id="dli_08_0397__en-us_topic_0000001262335930_b624515371276">username</strong>.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row493271615415"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p18436104617610">sink.buffer-flush.max-rows</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15436124616614">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p164363461613">100</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p154361446365">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1193391620415">Maximum size of buffered records before flush. You can set this parameter to <strong id="dli_08_0397__en-us_topic_0000001262335930_b5483258171015">0</strong> to disable it.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row4301150867"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1056041674">sink.buffer-flush.interval</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p256021572">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p205601811974">1s</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15560512715">Duration</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p518283975112">Flush interval, during which asynchronous threads will flush data. You can set this parameter to <strong id="dli_08_0397__en-us_topic_0000001262335930_b11302115211018">0</strong> to disable it.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p43014501560">Note that <strong id="dli_08_0397__en-us_topic_0000001262335930_b1652710311210">sink.buffer-flush.max-rows</strong> can be set to <strong id="dli_08_0397__en-us_topic_0000001262335930_b17353116111218">0</strong> with the flush interval set allowing for complete asynchronous processing of buffered actions.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row139491052269"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p17410197472">sink.max-retries</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1041077475">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p94101370714">3</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p34103710719">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p7949105210617">Maximum number of retries if writing records to the database failed.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_section973715249264"><h4 class="sectiontitle">Data Type Mapping</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0397__en-us_topic_0000001262335930_table1763835011280" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Data type mapping</caption><thead align="left"><tr id="dli_08_0397__en-us_topic_0000001262335930_row1638125032818"><th align="left" class="cellrowborder" valign="top" width="27.339999999999996%" id="mcps1.3.6.2.2.4.1.1"><p id="dli_08_0397__en-us_topic_0000001262335930_p2063895012811">MySQL Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="26.419999999999998%" id="mcps1.3.6.2.2.4.1.2"><p id="dli_08_0397__en-us_topic_0000001262335930_p192231755102915">PostgreSQL Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="46.239999999999995%" id="mcps1.3.6.2.2.4.1.3"><p id="dli_08_0397__en-us_topic_0000001262335930_p963855013286">Flink SQL Type</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0397__en-us_topic_0000001262335930_row156381350142816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p31591335132718">TINYINT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p18223175514292">-</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1563885082814">TINYINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row363865082813"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1562412015278">SMALLINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1562440162719">TINYINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p8362313123110">SMALLINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p3641315173118">INT2</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p113023181317">SMALLSERIAL</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1522395514291">SERIAL2</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6638105011280">SMALLINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row11638125012281"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p953995032711">INT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p428445819276">MEDIUMINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p6638950112817">SMALLINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p184911541319">INTEGER</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p6223155512291">SERIAL</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2063818509282">INT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row13638450112816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1729512711282">BIGINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p206381450112812">INT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1828235614304">BIGINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p11223175532910">BIGSERIAL</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1263865022815">BIGINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row13638250192812"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15638115052818">BIGINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p222315558298">-</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p263825032811">DECIMAL(20, 0)</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row2063845012816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p9638750182819">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6223115519291">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2638135012812">BIGINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row20638195018282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1163835062813">FLOAT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p189716430307">REAL</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p6223195512919">FLOAT4</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p116388508284">FLOAT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row1063805022817"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p123932031102812">DOUBLE</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p2638105062810">DOUBLE PRECISION</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1773253420308">FLOAT8</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1022320555298">DOUBLE PRECISION</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6638185017285">DOUBLE</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row66381502282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p169583762812">NUMERIC(p, s)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p14638165042819">DECIMAL(p, s)</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p378392733020">NUMERIC(p, s)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p722355562911">DECIMAL(p, s)</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2638205072819">DECIMAL(p, s)</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row16638650192819"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2413502283">BOOLEAN</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1663875011285">TINYINT(1)</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p622365520291">BOOLEAN</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1263835016287">BOOLEAN</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row46385509281"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1463845017289">DATE</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p9223055192918">DATE</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p3638145012818">DATE</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row106381950122815"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p12638165018287">TIME [(p)]</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p102232556296">TIME [(p)] [WITHOUT TIMEZONE]</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1863875042813">TIME [(p)] [WITHOUT TIMEZONE]</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row1363825082820"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p196381950182814">DATETIME [(p)]</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p622312554298">TIMESTAMP [(p)] [WITHOUT TIMEZONE]</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p5638350132818">TIMESTAMP [(p)] [WITHOUT TIMEZONE]</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row4638175032820"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p182297293">CHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p16753613202914">VARCHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p16638850182817">TEXT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p69611143172914">CHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p197622463293">CHARACTER(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p782394862919">VARCHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1266195692910">CHARACTER</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p174231553192914">VARYING(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p0223555152916">TEXT</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6638195019280">STRING</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row663845017282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p10502721132918">BINARY</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p15293925132912">VARBINARY</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p12638150192818">BLOB</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p922325542917">BYTEA</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p96382503282">BYTES</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row16638450142813"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1638185042816">-</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1322316556293">ARRAY</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1463805022811">ARRAY</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section1734134944319"><h4 class="sectiontitle">Example</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p11985427122614">In this example, Kafka is used to send data, and Kafka data is written to the MySQL database through the JDBC result table.</p>
<ol id="dli_08_0397__en-us_topic_0000001262335930_ol12415133515531"><li id="dli_08_0397__en-us_topic_0000001262335930_li6127741153217">Create an enhanced datasource connection in the VPC and subnet where MySQL and Kafka locate, and bind the connection to the required Flink elastic resource pool.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li1233002211514">Set MySQL and Kafka security groups and add inbound rules to allow access from the Flink queue. Test the connectivity using the MySQL and Kafka address. If the connection is successful, the datasource is bound to the queue. Otherwise, the binding fails.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li34158359532">Log in to the MySQL database and create table <strong id="dli_08_0397__en-us_topic_0000001262335930_b1840925314154">orders</strong> in database <strong id="dli_08_0397__en-us_topic_0000001262335930_b12774531191610">flink</strong>.<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen1823437219">CREATE TABLE `flink`.`orders` (
`order_id` VARCHAR(32) NOT NULL,
`order_channel` VARCHAR(32) NULL,
`order_time` VARCHAR(32) NULL,
`pay_amount` DOUBLE UNSIGNED NOT NULL,
`real_pay` DOUBLE UNSIGNED NULL,
`pay_time` VARCHAR(32) NULL,
`user_id` VARCHAR(32) NULL,
`user_name` VARCHAR(32) NULL,
`area_id` VARCHAR(32) NULL,
PRIMARY KEY (`order_id`)
) ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8mb4
COLLATE = utf8mb4_general_ci;</pre>
</li><li id="dli_08_0397__en-us_topic_0000001262335930_li12772614175712">Create a Flink OpenSource SQL job. Enter the following job script and submit the job.<div class="p" id="dli_08_0397__en-us_topic_0000001262335930_p135710287576"><a name="dli_08_0397__en-us_topic_0000001262335930_li12772614175712"></a><a name="en-us_topic_0000001262335930_li12772614175712"></a>When you create a job, set <strong id="dli_08_0397__en-us_topic_0000001262335930_b1421784961611">Flink Version</strong> to <strong id="dli_08_0397__en-us_topic_0000001262335930_b521814951610">1.12</strong> on the <strong id="dli_08_0397__en-us_topic_0000001262335930_b821834921618">Running Parameters</strong> tab. Select <strong id="dli_08_0397__en-us_topic_0000001262335930_b62191449181610">Save Job Log</strong>, and specify the OBS bucket for saving job logs. <strong id="dli_08_0397__en-us_topic_0000001262335930_b16531351151611">Change the values of the parameters in bold as needed in the following script.</strong><pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen1280662419578">CREATE TABLE kafkaSource (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i4543155265320"><strong id="dli_08_0397__en-us_topic_0000001262335930_b1754316528535">Kafka</strong></em><em id="dli_08_0397__en-us_topic_0000001262335930_i15341759195319"><strong id="dli_08_0397__en-us_topic_0000001262335930_b5227105913538">Topic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i9319103215313"><strong id="dli_08_0397__en-us_topic_0000001262335930_b0322115133213">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i14535174011311"><strong id="dli_08_0397__en-us_topic_0000001262335930_b1466610178329">GroupId</strong></em>',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
CREATE TABLE jdbcSink (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'jdbc',
'url? = 'jdbc:mysql://<em id="dli_08_0397__en-us_topic_0000001262335930_i11316145920161"><strong id="dli_08_0397__en-us_topic_0000001262335930_b19310159101612">MySQLAddress:MySQLPort</strong></em>/flink',-- flink is the MySQL database where the <strong id="dli_08_0397__en-us_topic_0000001262335930_b173175594169">orders</strong> table locates.
'table-name' = 'orders',
'username' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i914912159593"><strong id="dli_08_0397__en-us_topic_0000001262335930_b3149131585919">MySQL</strong></em><em id="dli_08_0397__en-us_topic_0000001262335930_i19846218135920"><strong id="dli_08_0397__en-us_topic_0000001262335930_b12575101815911">Username</strong></em>',
'password' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i42321325155913"><strong id="dli_08_0397__en-us_topic_0000001262335930_b202321525105915">MySQL</strong></em><em id="dli_08_0397__en-us_topic_0000001262335930_i4741529185918"><strong id="dli_08_0397__en-us_topic_0000001262335930_b874482815599">Password</strong></em>',
'sink.buffer-flush.max-rows' = '1'
);
insert into jdbcSink select * from kafkaSource;</pre>
</div>
</li><li id="dli_08_0397__en-us_topic_0000001262335930_li04161435115319">Connect to the Kafka cluster and send the following test data to the Kafka topics:<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen138261187211">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}
{"order_id":"202103241606060001", "order_channel":"appShop", "order_time":"2021-03-24 16:06:06", "pay_amount":"200.00", "real_pay":"180.00", "pay_time":"2021-03-24 16:10:06", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0397__en-us_topic_0000001262335930_li17537162012111">Run the SQL statement in the MySQL database to view data in the table:<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen3359228654">select * from orders;</pre>
<div class="p" id="dli_08_0397__en-us_topic_0000001262335930_p1533922214118">The following is an example of the result (note that the following data is replicated from the MySQL database but not the data style in the MySQL database):<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen1612710234445">202103241000000001,webShop,2021-03-24 10:00:00,100.0,100.0,2021-03-24 10:02:03,0001,Alice,330106
202103241606060001,appShop,2021-03-24 16:06:06,200.0,180.0,2021-03-24 16:10:06,0001,Alice,330106</pre>
</div>
</li></ol>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_section11771646162711"><h4 class="sectiontitle">FAQ</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p1867213182819">None</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0391.html">Creating Result Tables</a></div>
</div>
</div>
View Git Blame Copy Permalink