vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0397.html
Su, Xiaomeng 76a5b1ee83 dli_sqlreference_20240227 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2024-03-27 22:02:33 +00:00

385 lines
37 KiB
HTML
Raw Blame History

<a name="dli_08_0397"></a><a name="dli_08_0397"></a>
<h1 class="topictitle1">JDBC Result Table</h1>
<div id="body8662426"><div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section1380110114219"><h4 class="sectiontitle">Function</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p17594174162711">DLI outputs the Flink job output data to RDS through the JDBC result table.</p>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section7221153411"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0397__en-us_topic_0000001262335930_ul114983042819"><li id="dli_08_0397__en-us_topic_0000001262335930_li204032517344">An enhanced datasource connection with the instances has been established, so that you can configure security group rules as required.
</li></ul>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_section962153220221"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0397__en-us_topic_0000001262335930_ul168810224217"><li id="dli_08_0397__en-us_topic_0000001262335930_li1787611685815">When creating a Flink OpenSource SQL job, you need to set <strong id="dli_08_0397__en-us_topic_0000001262335930_b1703819356">Flink Version</strong> to <strong id="dli_08_0397__en-us_topic_0000001262335930_b2119811350">1.12</strong> on the <strong id="dli_08_0397__en-us_topic_0000001262335930_b8219818352">Running Parameters</strong> tab of the job editing page, select <strong id="dli_08_0397__en-us_topic_0000001262335930_b8218103517">Save Job Log</strong>, and set the OBS bucket for saving job logs.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li123134519351">The connector operates in upsert mode if the primary key was defined; otherwise, the connector operates in append mode.<ul id="dli_08_0397__en-us_topic_0000001262335930_ul165969505431"><li id="dli_08_0397__en-us_topic_0000001262335930_li112157463431">In upsert mode, Flink will insert a new row or update the existing row according to the primary key. Flink can ensure the idempotence in this way. To guarantee the output result is as expected, it is recommended to define a primary key for the table.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li678424715435">In append mode, Flink will interpret all records as INSERT messages. The INSERT operation may fail if a primary key or unique constraint violation happens in the underlying database.</li></ul>
</li></ul>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section24316432437"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0243_screen746013311958"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">create</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">jdbcSink</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="p">,</span><span class="w"> </span><span class="p">...)</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="n">ENFORCED</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">with</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'connector'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'jdbc'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'url'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'table-name'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'driver'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'username'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'password'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_section158861859181416"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0397__en-us_topic_0000001262335930_table3535131492817" frame="border" border="1" rules="all"><thead align="left"><tr id="dli_08_0397__en-us_topic_0000001262335930_row053518147287"><th align="left" class="cellrowborder" valign="top" width="16.72%" id="mcps1.3.5.2.1.6.1.1"><p id="dli_08_0397__en-us_topic_0000001262335930_p153513145285">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.19%" id="mcps1.3.5.2.1.6.1.2"><p id="dli_08_0397__en-us_topic_0000001262335930_p10535014202815">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.08%" id="mcps1.3.5.2.1.6.1.3"><p id="dli_08_0397__en-us_topic_0000001262335930_p519717173594">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.309999999999999%" id="mcps1.3.5.2.1.6.1.4"><p id="dli_08_0397__en-us_topic_0000001262335930_p41981824182510">Data Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="54.7%" id="mcps1.3.5.2.1.6.1.5"><p id="dli_08_0397__en-us_topic_0000001262335930_p19535151482813">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0397__en-us_topic_0000001262335930_row653531413287"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p117141241192315">connector</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p81135102314">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p121977179596">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p91981924142514">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1471424117235">Connector to be used. Set this parameter to <strong id="dli_08_0397__en-us_topic_0000001262335930_b1040311111511">jdbc</strong>.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row65351614172820"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15535141432819">url</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1653517144282">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p91971417175910">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p61987241254">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p12535914122818">Database URL.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row11535151410283"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p17535131482814">table-name</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p653621414283">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p19197141712593">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p171981424142513">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2536191418289">Name of the table where the data will be read from the database.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row753618145283"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p95361214202815">driver</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1353613148281">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1234113515504">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1919816241258">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1453631462817">Driver required for connecting to the database. If you do not set this parameter, it will be automatically derived from the URL.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row195361114142812"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p105364142285">username</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1128519229372">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p159012366509">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p71981424112514">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p175361714162814">Database authentication username. This parameter must be configured in pair with <strong id="dli_08_0397__en-us_topic_0000001262335930_b14117625773">password</strong>.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row553641414283"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6536614122819">password</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6536131442814">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p619719176592">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p7198142472520">String</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p953641462814">Database authentication password. This parameter must be configured in pair with <strong id="dli_08_0397__en-us_topic_0000001262335930_b624515371276">username</strong>.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row493271615415"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p18436104617610">sink.buffer-flush.max-rows</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15436124616614">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p164363461613">100</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p154361446365">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p133163111363">Maximum number of rows to buffer for each write request.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p183311131113612">It can improve the performance of writing data, but may increase the latency.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1833113313362">You can set this parameter to <strong id="dli_08_0397__en-us_topic_0000001262335930_b8230154211373">0</strong> to disable it.</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row4301150867"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1056041674">sink.buffer-flush.interval</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p256021572">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p205601811974">1s</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15560512715">Duration</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p181901938203711">Interval for refreshing the buffer, during which data is refreshed by asynchronous threads.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p919033813711">It can improve the performance of writing data, but may increase the latency.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p7190133816376">You can set this parameter to <strong id="dli_08_0397__en-us_topic_0000001262335930_b118431552173717">0</strong> to disable it.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p2190193811378">Note: If <strong id="dli_08_0397__en-us_topic_0000001262335930_b1195855143711">sink.buffer-flush.max-rows</strong> is set to <strong id="dli_08_0397__en-us_topic_0000001262335930_b121954556374">0</strong> and the buffer refresh interval is configured, the buffer is asynchronously refreshed.</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p10120110194117">The format is <em id="dli_08_0397__en-us_topic_0000001262335930_i20941139113818">{length value}{time unit label}</em>, for example, <strong id="dli_08_0397__en-us_topic_0000001262335930_b294113917381">123ms, 321s</strong>. The supported time units include <strong id="dli_08_0397__en-us_topic_0000001262335930_b179543923814">d</strong>, <strong id="dli_08_0397__en-us_topic_0000001262335930_b1895153993810">h</strong>, <strong id="dli_08_0397__en-us_topic_0000001262335930_b1296133963813">min</strong>, <strong id="dli_08_0397__en-us_topic_0000001262335930_b1096739193819">s</strong>, and <strong id="dli_08_0397__en-us_topic_0000001262335930_b1996193983817">ms</strong> (default unit).</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row139491052269"><td class="cellrowborder" valign="top" width="16.72%" headers="mcps1.3.5.2.1.6.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p17410197472">sink.max-retries</p>
</td>
<td class="cellrowborder" valign="top" width="8.19%" headers="mcps1.3.5.2.1.6.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1041077475">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.08%" headers="mcps1.3.5.2.1.6.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p94101370714">3</p>
</td>
<td class="cellrowborder" valign="top" width="8.309999999999999%" headers="mcps1.3.5.2.1.6.1.4 "><p id="dli_08_0397__en-us_topic_0000001262335930_p34103710719">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="54.7%" headers="mcps1.3.5.2.1.6.1.5 "><p id="dli_08_0397__en-us_topic_0000001262335930_p7949105210617">Maximum number of retries if writing records to the database failed.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_section973715249264"><h4 class="sectiontitle">Data Type Mapping</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0397__en-us_topic_0000001262335930_table1763835011280" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Data type mapping</caption><thead align="left"><tr id="dli_08_0397__en-us_topic_0000001262335930_row1638125032818"><th align="left" class="cellrowborder" valign="top" width="27.339999999999996%" id="mcps1.3.6.2.2.4.1.1"><p id="dli_08_0397__en-us_topic_0000001262335930_p2063895012811">MySQL Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="26.419999999999998%" id="mcps1.3.6.2.2.4.1.2"><p id="dli_08_0397__en-us_topic_0000001262335930_p192231755102915">PostgreSQL Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="46.239999999999995%" id="mcps1.3.6.2.2.4.1.3"><p id="dli_08_0397__en-us_topic_0000001262335930_p963855013286">Flink SQL Type</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0397__en-us_topic_0000001262335930_row156381350142816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p31591335132718">TINYINT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p18223175514292">-</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1563885082814">TINYINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row363865082813"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1562412015278">SMALLINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1562440162719">TINYINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p8362313123110">SMALLINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p3641315173118">INT2</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p113023181317">SMALLSERIAL</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1522395514291">SERIAL2</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6638105011280">SMALLINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row11638125012281"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p953995032711">INT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p428445819276">MEDIUMINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p6638950112817">SMALLINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p184911541319">INTEGER</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p6223155512291">SERIAL</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2063818509282">INT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row13638450112816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1729512711282">BIGINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p206381450112812">INT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1828235614304">BIGINT</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p11223175532910">BIGSERIAL</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1263865022815">BIGINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row13638250192812"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p15638115052818">BIGINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p222315558298">-</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p263825032811">DECIMAL(20, 0)</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row2063845012816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p9638750182819">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6223115519291">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2638135012812">BIGINT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row20638195018282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1163835062813">FLOAT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p189716430307">REAL</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p6223195512919">FLOAT4</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p116388508284">FLOAT</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row1063805022817"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p123932031102812">DOUBLE</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p2638105062810">DOUBLE PRECISION</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1773253420308">FLOAT8</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1022320555298">DOUBLE PRECISION</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6638185017285">DOUBLE</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row66381502282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p169583762812">NUMERIC(p, s)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p14638165042819">DECIMAL(p, s)</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p378392733020">NUMERIC(p, s)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p722355562911">DECIMAL(p, s)</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2638205072819">DECIMAL(p, s)</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row16638650192819"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p2413502283">BOOLEAN</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1663875011285">TINYINT(1)</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p622365520291">BOOLEAN</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1263835016287">BOOLEAN</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row46385509281"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1463845017289">DATE</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p9223055192918">DATE</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p3638145012818">DATE</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row106381950122815"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p12638165018287">TIME [(p)]</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p102232556296">TIME [(p)] [WITHOUT TIMEZONE]</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1863875042813">TIME [(p)] [WITHOUT TIMEZONE]</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row1363825082820"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p196381950182814">DATETIME [(p)]</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p622312554298">TIMESTAMP [(p)] [WITHOUT TIMEZONE]</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p5638350132818">TIMESTAMP [(p)] [WITHOUT TIMEZONE]</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row4638175032820"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p182297293">CHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p16753613202914">VARCHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p16638850182817">TEXT</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p69611143172914">CHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p197622463293">CHARACTER(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p782394862919">VARCHAR(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p1266195692910">CHARACTER</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p174231553192914">VARYING(n)</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p0223555152916">TEXT</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p6638195019280">STRING</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row663845017282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p10502721132918">BINARY</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p15293925132912">VARBINARY</p>
<p id="dli_08_0397__en-us_topic_0000001262335930_p12638150192818">BLOB</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p922325542917">BYTEA</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p96382503282">BYTES</p>
</td>
</tr>
<tr id="dli_08_0397__en-us_topic_0000001262335930_row16638450142813"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.6.2.2.4.1.1 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1638185042816">-</p>
</td>
<td class="cellrowborder" valign="top" width="26.419999999999998%" headers="mcps1.3.6.2.2.4.1.2 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1322316556293">ARRAY</p>
</td>
<td class="cellrowborder" valign="top" width="46.239999999999995%" headers="mcps1.3.6.2.2.4.1.3 "><p id="dli_08_0397__en-us_topic_0000001262335930_p1463805022811">ARRAY</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_dli_08_0244_en-us_topic_0111563976_section1734134944319"><h4 class="sectiontitle">Example</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p11985427122614">In this example, Kafka is used to send data, and Kafka data is written to the MySQL database through the JDBC result table.</p>
<ol id="dli_08_0397__en-us_topic_0000001262335930_ol12415133515531"><li id="dli_08_0397__en-us_topic_0000001262335930_li6127741153217">Create an enhanced datasource connection in the VPC and subnet where MySQL and Kafka locate, and bind the connection to the required Flink elastic resource pool.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li1233002211514">Set MySQL and Kafka security groups and add inbound rules to allow access from the Flink queue. Test the connectivity using the MySQL and Kafka address. If the connection is successful, the datasource is bound to the queue. Otherwise, the binding fails.</li><li id="dli_08_0397__en-us_topic_0000001262335930_li34158359532">Log in to the MySQL database and create table <strong id="dli_08_0397__en-us_topic_0000001262335930_b1840925314154">orders</strong> in database <strong id="dli_08_0397__en-us_topic_0000001262335930_b12774531191610">flink</strong>.<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen1823437219">CREATE TABLE `flink`.`orders` (
`order_id` VARCHAR(32) NOT NULL,
`order_channel` VARCHAR(32) NULL,
`order_time` VARCHAR(32) NULL,
`pay_amount` DOUBLE UNSIGNED NOT NULL,
`real_pay` DOUBLE UNSIGNED NULL,
`pay_time` VARCHAR(32) NULL,
`user_id` VARCHAR(32) NULL,
`user_name` VARCHAR(32) NULL,
`area_id` VARCHAR(32) NULL,
PRIMARY KEY (`order_id`)
) ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8mb4
COLLATE = utf8mb4_general_ci;</pre>
</li><li id="dli_08_0397__en-us_topic_0000001262335930_li12772614175712">Create a Flink OpenSource SQL job. Enter the following job script and submit the job.<div class="p" id="dli_08_0397__en-us_topic_0000001262335930_p135710287576"><a name="dli_08_0397__en-us_topic_0000001262335930_li12772614175712"></a><a name="en-us_topic_0000001262335930_li12772614175712"></a>When you create a job, set <strong id="dli_08_0397__en-us_topic_0000001262335930_b1421784961611">Flink Version</strong> to <strong id="dli_08_0397__en-us_topic_0000001262335930_b521814951610">1.12</strong> on the <strong id="dli_08_0397__en-us_topic_0000001262335930_b821834921618">Running Parameters</strong> tab. Select <strong id="dli_08_0397__en-us_topic_0000001262335930_b62191449181610">Save Job Log</strong>, and specify the OBS bucket for saving job logs. <strong id="dli_08_0397__en-us_topic_0000001262335930_b16531351151611">Change the values of the parameters in bold as needed in the following script.</strong><pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen1280662419578">CREATE TABLE kafkaSource (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i4543155265320"><strong id="dli_08_0397__en-us_topic_0000001262335930_b1754316528535">Kafka</strong></em><em id="dli_08_0397__en-us_topic_0000001262335930_i15341759195319"><strong id="dli_08_0397__en-us_topic_0000001262335930_b5227105913538">Topic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i9319103215313"><strong id="dli_08_0397__en-us_topic_0000001262335930_b0322115133213">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i14535174011311"><strong id="dli_08_0397__en-us_topic_0000001262335930_b1466610178329">GroupId</strong></em>',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
CREATE TABLE jdbcSink (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'jdbc',
'url? = 'jdbc:mysql://<em id="dli_08_0397__en-us_topic_0000001262335930_i11316145920161"><strong id="dli_08_0397__en-us_topic_0000001262335930_b19310159101612">MySQLAddress:MySQLPort</strong></em>/flink',-- flink is the MySQL database where the <strong id="dli_08_0397__en-us_topic_0000001262335930_b173175594169">orders</strong> table locates.
'table-name' = 'orders',
'username' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i914912159593"><strong id="dli_08_0397__en-us_topic_0000001262335930_b3149131585919">MySQL</strong></em><em id="dli_08_0397__en-us_topic_0000001262335930_i19846218135920"><strong id="dli_08_0397__en-us_topic_0000001262335930_b12575101815911">Username</strong></em>',
'password' = '<em id="dli_08_0397__en-us_topic_0000001262335930_i42321325155913"><strong id="dli_08_0397__en-us_topic_0000001262335930_b202321525105915">MySQL</strong></em><em id="dli_08_0397__en-us_topic_0000001262335930_i4741529185918"><strong id="dli_08_0397__en-us_topic_0000001262335930_b874482815599">Password</strong></em>',
'sink.buffer-flush.max-rows' = '1'
);
insert into jdbcSink select * from kafkaSource;</pre>
</div>
</li><li id="dli_08_0397__en-us_topic_0000001262335930_li04161435115319">Connect to the Kafka cluster and send the following test data to the Kafka topics:<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen138261187211">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}
{"order_id":"202103241606060001", "order_channel":"appShop", "order_time":"2021-03-24 16:06:06", "pay_amount":"200.00", "real_pay":"180.00", "pay_time":"2021-03-24 16:10:06", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0397__en-us_topic_0000001262335930_li17537162012111">Run the SQL statement in the MySQL database to view data in the table:<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen3359228654">select * from orders;</pre>
<div class="p" id="dli_08_0397__en-us_topic_0000001262335930_p1533922214118">The following is an example of the result (note that the following data is replicated from the MySQL database but not the data style in the MySQL database):<pre class="screen" id="dli_08_0397__en-us_topic_0000001262335930_screen1612710234445">202103241000000001,webShop,2021-03-24 10:00:00,100.0,100.0,2021-03-24 10:02:03,0001,Alice,330106
202103241606060001,appShop,2021-03-24 16:06:06,200.0,180.0,2021-03-24 16:10:06,0001,Alice,330106</pre>
</div>
</li></ol>
</div>
<div class="section" id="dli_08_0397__en-us_topic_0000001262335930_section11771646162711"><h4 class="sectiontitle">FAQ</h4><p id="dli_08_0397__en-us_topic_0000001262335930_p1867213182819">None</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0391.html">Creating Result Tables</a></div>
</div>
</div>
View Git Blame Copy Permalink