vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0385.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

404 lines
38 KiB
HTML
Raw Blame History

<a name="dli_08_0385"></a><a name="dli_08_0385"></a>
<h1 class="topictitle1">JDBC Source Table</h1>
<div id="body8662426"><div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section11554812174420"><h4 class="sectiontitle">Function</h4><p id="dli_08_0385__en-us_topic_0000001310015801_dli_08_0270_p7254192618812">The JDBC connector is a Flink's built-in connector to read data from a database.</p>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section0641438122515"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0385__en-us_topic_0000001310015801_ul036513387289"><li id="dli_08_0385__en-us_topic_0000001310015801_li036513816282">An enhanced datasource connection with the instances has been established, so that you can configure security group rules as required.</li></ul>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section15384120152215"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0385__en-us_topic_0000001310015801_p1180664142215">When creating a Flink OpenSource SQL job, you need to set <strong id="dli_08_0385__en-us_topic_0000001310015801_b3321543471">Flink Version</strong> to <strong id="dli_08_0385__en-us_topic_0000001310015801_b8372431279">1.12</strong> on the <strong id="dli_08_0385__en-us_topic_0000001310015801_b2037743377">Running Parameters</strong> tab of the job editing page, select <strong id="dli_08_0385__en-us_topic_0000001310015801_b43864314716">Save Job Log</strong>, and set the OBS bucket for saving job logs.</p>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section055264284416"><h4 class="sectiontitle">Syntax</h4><pre class="screen" id="dli_08_0385__en-us_topic_0000001310015801_screen143570576440">create table jbdcSource (
attr_name attr_type
(',' attr_name attr_type)*
(','PRIMARY KEY (attr_name, ...) NOT ENFORCED)
(',' watermark for rowtime_column_name as watermark-strategy_expression)
) with (
'connector' = 'jdbc',
'url' = '',
'table-name' = '',
'username' = '',
'password' = ''
);</pre>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section1918619215263"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0385__en-us_topic_0000001310015801_table3535131492817" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0385__en-us_topic_0000001310015801_row053518147287"><th align="left" class="cellrowborder" valign="top" width="22.29%" id="mcps1.3.5.2.2.6.1.1"><p id="dli_08_0385__en-us_topic_0000001310015801_p153513145285">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.55%" id="mcps1.3.5.2.2.6.1.2"><p id="dli_08_0385__en-us_topic_0000001310015801_p10535014202815">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="7.02%" id="mcps1.3.5.2.2.6.1.3"><p id="dli_08_0385__en-us_topic_0000001310015801_p519717173594">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.63%" id="mcps1.3.5.2.2.6.1.4"><p id="dli_08_0385__en-us_topic_0000001310015801_p41981824182510">Data Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="53.510000000000005%" id="mcps1.3.5.2.2.6.1.5"><p id="dli_08_0385__en-us_topic_0000001310015801_p19535151482813">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0385__en-us_topic_0000001310015801_row653531413287"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p117141241192315">connector</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p81135102314">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p121977179596">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p91981924142514">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1471424117235">Connector to be used. Set this parameter to <strong id="dli_08_0385__en-us_topic_0000001310015801_b492516403812">jdbc</strong>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row65351614172820"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p15535141432819">url</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1653517144282">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p91971417175910">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p61987241254">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p12535914122818">Database URL.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row11535151410283"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p38121811155819">table-name</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p653621414283">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p19197141712593">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p171981424142513">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2536191418289">Name of the table where the data will be read from the database.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row753618145283"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p95361214202815">driver</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1353613148281">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p56391492212">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1919816241258">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1453631462817">Driver required for connecting to the database. If you do not set this parameter, it will be automatically derived from the URL.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row195361114142812"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p105364142285">username</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1128519229372">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p3197141710593">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p71981424112514">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p175361714162814">Database authentication username. This parameter must be configured in pair with <strong id="dli_08_0385__en-us_topic_0000001310015801_b94531467236">password</strong>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row553641414283"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6536614122819">password</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6536131442814">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p619719176592">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p7198142472520">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p953641462814">Database authentication password. This parameter must be configured in pair with <strong id="dli_08_0385__en-us_topic_0000001310015801_b988163862311">username</strong>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row19302036122619"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p4302163618260">scan.partition.column</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2302936102619">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p154557176223">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1730273617268">String</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6302103662613">Name of the column used to partition the input. For details, see <a href="#dli_08_0385__en-us_topic_0000001310015801_section1974210303182">Partitioned Scan</a>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row134301838192610"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p843013819269">scan.partition.num</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p04300381265">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p3811191832217">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p16430143819260">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p14301382269">Number of partitions to be created. For details, see <a href="#dli_08_0385__en-us_topic_0000001310015801_section1974210303182">Partitioned Scan</a>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row79030405264"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1903154032610">scan.partition.lower-bound</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p9903104016266">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p209032404269">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1790315402265">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p8903174032617">Lower bound of values to be fetched for the first partition. For details, see <a href="#dli_08_0385__en-us_topic_0000001310015801_section1974210303182">Partitioned Scan</a>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row116874711260"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p769184712611">scan.partition.upper-bound</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p3691047182611">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1469154711263">None</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1169174715266">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1769647142616">Upper bound of values to be fetched for the last partition. For details, see <a href="#dli_08_0385__en-us_topic_0000001310015801_section1974210303182">Partitioned Scan</a>.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row8938124482614"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p16938164415261">scan.fetch-size</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p16938844172619">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1493854482612">0</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2938194414263">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p20938164416260">Number of rows fetched from the database each time. If this parameter is set to <strong id="dli_08_0385__en-us_topic_0000001310015801_b539817492261">0</strong>, the SQL hint is ignored.</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row128981142132615"><td class="cellrowborder" valign="top" width="22.29%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1289874282616">scan.auto-commit</p>
</td>
<td class="cellrowborder" valign="top" width="8.55%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p78981542132612">No</p>
</td>
<td class="cellrowborder" valign="top" width="7.02%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p118981442102613">true</p>
</td>
<td class="cellrowborder" valign="top" width="8.63%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0385__en-us_topic_0000001310015801_p12898442132616">Boolean</p>
</td>
<td class="cellrowborder" valign="top" width="53.510000000000005%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0385__en-us_topic_0000001310015801_p118981642122610">Whether each statement is committed in a transaction automatically.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section1974210303182"><a name="dli_08_0385__en-us_topic_0000001310015801_section1974210303182"></a><a name="en-us_topic_0000001310015801_section1974210303182"></a><h4 class="sectiontitle">Partitioned Scan</h4><p id="dli_08_0385__en-us_topic_0000001310015801_p199541741131817">To accelerate reading data in parallel Source task instances, Flink provides the partitioned scan feature for the JDBC table. The following parameters describe how to partition the table when reading in parallel from multiple tasks.</p>
<ul id="dli_08_0385__en-us_topic_0000001310015801_ul1994135133816"><li id="dli_08_0385__en-us_topic_0000001310015801_li159416354383"><strong id="dli_08_0385__en-us_topic_0000001310015801_b1398694863315">scan.partition.column</strong>: name of the column used to partition the input. The data type of the column must be number, date, or timestamp.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li1394135133819"><strong id="dli_08_0385__en-us_topic_0000001310015801_b15911855123420">scan.partition.num</strong>: number of partitions.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li1946353389"><strong id="dli_08_0385__en-us_topic_0000001310015801_b101978413355">scan.partition.lower-bound</strong>: minimum value of the first partition.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li12946354389"><strong id="dli_08_0385__en-us_topic_0000001310015801_b447516147356">scan.partition.upper-bound</strong>: maximum value of the last partition.</li></ul>
<div class="note" id="dli_08_0385__en-us_topic_0000001310015801_note0764113618395"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_08_0385__en-us_topic_0000001310015801_ul12418539392"><li id="dli_08_0385__en-us_topic_0000001310015801_li18249530391"><strong id="dli_08_0385__en-us_topic_0000001310015801_b790811474192">When a table is created, the preceding partitioned scan parameters must all be specified if any of them is specified.</strong></li><li id="dli_08_0385__en-us_topic_0000001310015801_li4365165410396">The <strong id="dli_08_0385__en-us_topic_0000001310015801_b15257192543719">scan.partition.lower-bound</strong> and <strong id="dli_08_0385__en-us_topic_0000001310015801_b086772743718">scan.partition.upper-bound</strong> parameters are used to decide the partition stride instead of filtering rows in the table. All rows in the table are partitioned and returned.</li></ul>
</div></div>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section167782952515"><h4 class="sectiontitle">Data Type Mapping</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0385__en-us_topic_0000001310015801_table1763835011280" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Data type mapping</caption><thead align="left"><tr id="dli_08_0385__en-us_topic_0000001310015801_row1638125032818"><th align="left" class="cellrowborder" valign="top" width="27.339999999999996%" id="mcps1.3.7.2.2.4.1.1"><p id="dli_08_0385__en-us_topic_0000001310015801_p2063895012811">MySQL Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="20.25%" id="mcps1.3.7.2.2.4.1.2"><p id="dli_08_0385__en-us_topic_0000001310015801_p192231755102915">PostgreSQL Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="52.410000000000004%" id="mcps1.3.7.2.2.4.1.3"><p id="dli_08_0385__en-us_topic_0000001310015801_p963855013286">Flink SQL Type</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0385__en-us_topic_0000001310015801_row156381350142816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p31591335132718">TINYINT</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p18223175514292">-</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1563885082814">TINYINT</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row363865082813"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1562412015278">SMALLINT</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p1562440162719">TINYINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p8362313123110">SMALLINT</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p3641315173118">INT2</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p113023181317">SMALLSERIAL</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p1522395514291">SERIAL2</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6638105011280">SMALLINT</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row11638125012281"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p953995032711">INT</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p428445819276">MEDIUMINT</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p6638950112817">SMALLINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p184911541319">INTEGER</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p6223155512291">SERIAL</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2063818509282">INT</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row13638450112816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1729512711282">BIGINT</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p206381450112812">INT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1828235614304">BIGINT</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p11223175532910">BIGSERIAL</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1263865022815">BIGINT</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row13638250192812"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p15638115052818">BIGINT UNSIGNED</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p222315558298">-</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p263825032811">DECIMAL(20, 0)</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row2063845012816"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p9638750182819">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6223115519291">BIGINT</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2638135012812">BIGINT</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row20638195018282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1163835062813">FLOAT</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p189716430307">REAL</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p6223195512919">FLOAT4</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p116388508284">FLOAT</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row1063805022817"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p123932031102812">DOUBLE</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p2638105062810">DOUBLE PRECISION</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1773253420308">FLOAT8</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p1022320555298">DOUBLE PRECISION</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6638185017285">DOUBLE</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row66381502282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p169583762812">NUMERIC(p, s)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p14638165042819">DECIMAL(p, s)</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p378392733020">NUMERIC(p, s)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p722355562911">DECIMAL(p, s)</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2638205072819">DECIMAL(p, s)</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row16638650192819"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p2413502283">BOOLEAN</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p1663875011285">TINYINT(1)</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p622365520291">BOOLEAN</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1263835016287">BOOLEAN</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row46385509281"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1463845017289">DATE</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p9223055192918">DATE</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p3638145012818">DATE</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row106381950122815"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p12638165018287">TIME [(p)]</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p102232556296">TIME [(p)] [WITHOUT TIMEZONE]</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1863875042813">TIME [(p)] [WITHOUT TIMEZONE]</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row1363825082820"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p196381950182814">DATETIME [(p)]</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p622312554298">TIMESTAMP [(p)] [WITHOUT TIMEZONE]</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p5638350132818">TIMESTAMP [(p)] [WITHOUT TIMEZONE]</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row4638175032820"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p182297293">CHAR(n)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p16753613202914">VARCHAR(n)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p16638850182817">TEXT</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p69611143172914">CHAR(n)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p197622463293">CHARACTER(n)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p782394862919">VARCHAR(n)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p1266195692910">CHARACTER</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p174231553192914">VARYING(n)</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p0223555152916">TEXT</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p6638195019280">STRING</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row663845017282"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p10502721132918">BINARY</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p15293925132912">VARBINARY</p>
<p id="dli_08_0385__en-us_topic_0000001310015801_p12638150192818">BLOB</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p922325542917">BYTEA</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p96382503282">BYTES</p>
</td>
</tr>
<tr id="dli_08_0385__en-us_topic_0000001310015801_row16638450142813"><td class="cellrowborder" valign="top" width="27.339999999999996%" headers="mcps1.3.7.2.2.4.1.1 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1638185042816">-</p>
</td>
<td class="cellrowborder" valign="top" width="20.25%" headers="mcps1.3.7.2.2.4.1.2 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1322316556293">ARRAY</p>
</td>
<td class="cellrowborder" valign="top" width="52.410000000000004%" headers="mcps1.3.7.2.2.4.1.3 "><p id="dli_08_0385__en-us_topic_0000001310015801_p1463805022811">ARRAY</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section163901016188"><h4 class="sectiontitle">Example</h4><p id="dli_08_0385__en-us_topic_0000001310015801_p21819112258">This example uses JDBC as the data source and Print as the sink to read data from the RDS MySQL database and write the data to the Print result table.</p>
<ol id="dli_08_0385__en-us_topic_0000001310015801_ol2459191311254"><li id="dli_08_0385__en-us_topic_0000001310015801_li6127741153217">Create an enhanced datasource connection in the VPC and subnet where RDS MySQL locates, and bind the connection to the required Flink elastic resource pool.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li1233002211514">Set RDS MySQL security groups and add inbound rules to allow access from the Flink queue. Test the connectivity using the RDS address. If the connection is successful, the datasource is bound to the queue. Otherwise, the binding fails.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li345911319257">Log in to the RDS MySQL database, create table <strong id="dli_08_0385__en-us_topic_0000001310015801_b128719162422">orders</strong> in the Flink database, and insert data. <p id="dli_08_0385__en-us_topic_0000001310015801_p512209192914">Create table <strong id="dli_08_0385__en-us_topic_0000001310015801_b113916411451">orders</strong> in the Flink database.</p>
<pre class="screen" id="dli_08_0385__en-us_topic_0000001310015801_screen1246316892915">CREATE TABLE `flink`.`orders` (
`order_id` VARCHAR(32) NOT NULL,
`order_channel` VARCHAR(32) NULL,
`order_time` VARCHAR(32) NULL,
`pay_amount` DOUBLE UNSIGNED NOT NULL,
`real_pay` DOUBLE UNSIGNED NULL,
`pay_time` VARCHAR(32) NULL,
`user_id` VARCHAR(32) NULL,
`user_name` VARCHAR(32) NULL,
`area_id` VARCHAR(32) NULL,
PRIMARY KEY (`order_id`)
) ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8mb4
COLLATE = utf8mb4_general_ci;</pre>
<div class="p" id="dli_08_0385__en-us_topic_0000001310015801_p770994752911">Insert data into the table.<pre class="screen" id="dli_08_0385__en-us_topic_0000001310015801_screen8463168192910">insert into orders(
order_id,
order_channel,
order_time,
pay_amount,
real_pay,
pay_time,
user_id,
user_name,
area_id) values
('202103241000000001', 'webShop', '2021-03-24 10:00:00', '100.00', '100.00', '2021-03-24 10:02:03', '0001', 'Alice', '330106'),
('202103251202020001', 'miniAppShop', '2021-03-25 12:02:02', '60.00', '60.00', '2021-03-25 12:03:00', '0002', 'Bob', '330110');</pre>
</div>
</li><li id="dli_08_0385__en-us_topic_0000001310015801_li184861035101314">Create a Flink OpenSource SQL job. Enter the following job script and submit the job.<div class="p" id="dli_08_0385__en-us_topic_0000001310015801_p1336815819121"><a name="dli_08_0385__en-us_topic_0000001310015801_li184861035101314"></a><a name="en-us_topic_0000001310015801_li184861035101314"></a>When you create a job, set <strong id="dli_08_0385__en-us_topic_0000001310015801_b14989235164514">Flink Version</strong> to <strong id="dli_08_0385__en-us_topic_0000001310015801_b69891135134517">1.12</strong> on the <strong id="dli_08_0385__en-us_topic_0000001310015801_b09901357455">Running Parameters</strong> tab. Select <strong id="dli_08_0385__en-us_topic_0000001310015801_b129901735174519">Save Job Log</strong>, and specify the OBS bucket for saving job logs. <strong id="dli_08_0385__en-us_topic_0000001310015801_b1113184064519">Change the values of the parameters in bold as needed in the following script.</strong><pre class="screen" id="dli_08_0385__en-us_topic_0000001310015801_screen1825143871915">CREATE TABLE jdbcSource (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'jdbc',
'url' = 'jdbc:mysql://<em id="dli_08_0385__en-us_topic_0000001310015801_i19874103812349"><strong id="dli_08_0385__en-us_topic_0000001310015801_b0655113712348">MySQLAddress</strong></em>:<em id="dli_08_0385__en-us_topic_0000001310015801_i54229417347"><strong id="dli_08_0385__en-us_topic_0000001310015801_b89425406349">MySQLPort</strong></em>/<strong id="dli_08_0385__en-us_topic_0000001310015801_b2202162418408"><em id="dli_08_0385__en-us_topic_0000001310015801_i8136527104011">flink</em></strong>',--flink is the database name created in RDS MySQL.
'table-name' = 'orders',
'username' = '<em id="dli_08_0385__en-us_topic_0000001310015801_i10425633352"><strong id="dli_08_0385__en-us_topic_0000001310015801_b342518323518">MySQL</strong></em><em id="dli_08_0385__en-us_topic_0000001310015801_i1778819517355"><strong id="dli_08_0385__en-us_topic_0000001310015801_b841125143513">Username</strong></em>',
'password' = '<em id="dli_08_0385__en-us_topic_0000001310015801_i1774913116352"><strong id="dli_08_0385__en-us_topic_0000001310015801_b13749181111353">MySQL</strong></em><em id="dli_08_0385__en-us_topic_0000001310015801_i351120154354"><strong id="dli_08_0385__en-us_topic_0000001310015801_b16177215123514">Password</strong></em>'
);
CREATE TABLE printSink (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'print'
);
insert into printSink select * from jdbcSource;</pre>
</div>
</li><li id="dli_08_0385__en-us_topic_0000001310015801_li2045961322515">Perform the following operations to view the data result in the <strong id="dli_08_0385__en-us_topic_0000001310015801_b1514717280479">taskmanager.out</strong> file:<ol type="a" id="dli_08_0385__en-us_topic_0000001310015801_ol864115198285"><li id="dli_08_0385__en-us_topic_0000001310015801_li10901621122819">Log in to the DLI console. In the navigation pane, choose <strong id="dli_08_0385__en-us_topic_0000001310015801_b1655173964710">Job Management</strong> &gt; <strong id="dli_08_0385__en-us_topic_0000001310015801_b461639134718">Flink Jobs</strong>.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li1912163912282">Click the name of the corresponding Flink job, choose <strong id="dli_08_0385__en-us_topic_0000001310015801_b17493104214719">Run Log</strong>, click <strong id="dli_08_0385__en-us_topic_0000001310015801_b149424211470">OBS Bucket</strong>, and locate the folder of the log you want to view according to the date.</li><li id="dli_08_0385__en-us_topic_0000001310015801_li0641191914285">Go to the folder of the date, find the folder whose name contains <strong id="dli_08_0385__en-us_topic_0000001310015801_b13249204624718">taskmanager</strong>, download the <strong id="dli_08_0385__en-us_topic_0000001310015801_b18249346184718">taskmanager.out</strong> file, and view result logs.</li></ol>
<p id="dli_08_0385__en-us_topic_0000001310015801_p11171749104019">The data result is as follows:</p>
<pre class="screen" id="dli_08_0385__en-us_topic_0000001310015801_screen11171949164011">+I(202103241000000001,webShop,2021-03-24 10:00:00,100.0,100.0,2021-03-24 10:02:03,0001,Alice,330106)
+I(202103251202020001,miniAppShop,2021-03-25 12:02:02,60.0,60.0,2021-03-25 12:03:00,0002,Bob,330110)</pre>
</li></ol>
</div>
<div class="section" id="dli_08_0385__en-us_topic_0000001310015801_section35840248116"><h4 class="sectiontitle">FAQ</h4><p id="dli_08_0385__en-us_topic_0000001310015801_p9265163315399">None</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0381.html">Creating Source Tables</a></div>
</div>
</div>
View Git Blame Copy Permalink