vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0400.html
Su, Xiaomeng 76a5b1ee83 dli_sqlreference_20240227 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2024-03-27 22:02:33 +00:00

565 lines
64 KiB
HTML
Raw Blame History

<a name="dli_08_0400"></a><a name="dli_08_0400"></a>
<h1 class="topictitle1">Redis Result Table</h1>
<div id="body8662426"><div class="section" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0253_en-us_topic_0111499977_section981791218502"><h4 class="sectiontitle">Function</h4><p id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0253_en-us_topic_0111499977_p108830172403">DLI outputs the Flink job output data to Redis. Redis is a key-value storage system that supports multiple types of data structures. It can be used in scenarios such as caching, event publish/subscribe, and high-speed queuing. Redis supports direct read/write of strings, hashes, lists, queues, and sets. Redis works with in-memory datasets and provides persistence. For more information about Redis, visit <a href="https://redis.io/" target="_blank" rel="noopener noreferrer">https://redis.io/</a>.</p>
</div>
<div class="section" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0253_en-us_topic_0111499977_section1773605853811"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0400__en-us_topic_0000001309855877_ul73441720185017"><li id="dli_08_0400__en-us_topic_0000001309855877_li203441920145016">An enhanced datasource connection with Redis has been established, so that you can configure security group rules as required.
</li></ul>
</div>
<div class="section" id="dli_08_0400__en-us_topic_0000001309855877_section2077125612161"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0400__en-us_topic_0000001309855877_ul44457213176"><li id="dli_08_0400__en-us_topic_0000001309855877_li58031112258">When creating a Flink OpenSource SQL job, you need to set <strong id="dli_08_0400__en-us_topic_0000001309855877_b1460673814718">Flink Version</strong> to <strong id="dli_08_0400__en-us_topic_0000001309855877_b56068386712">1.12</strong> on the <strong id="dli_08_0400__en-us_topic_0000001309855877_b160718381874">Running Parameters</strong> tab of the job editing page, select <strong id="dli_08_0400__en-us_topic_0000001309855877_b186081538578">Save Job Log</strong>, and set the OBS bucket for saving job logs.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1244502161719">If the Redis key field is not defined in the statement for creating the Redis result table, the generated UUID is used as the key.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li3342359161712">To specify a key in Redis, you need to define a primary key in the Redis result table of Flink. The value of the primary key is the Redis key.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li20233539172616">If the primary key defined for the Redis result table, it cannot be a composite primary key and only can be one field.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1877444315214"><a name="dli_08_0400__en-us_topic_0000001309855877_li1877444315214"></a><a name="en-us_topic_0000001309855877_li1877444315214"></a>Constraints on <strong id="dli_08_0400__en-us_topic_0000001309855877_b78391147102118">schema-syntax</strong>:<ul id="dli_08_0400__en-us_topic_0000001309855877_ul88618711225"><li id="dli_08_0400__en-us_topic_0000001309855877_li136351521994">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b9945153202120">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b129511453112115">map</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b10952205372117">array</strong>, there can be only one non-primary key and it must be of the same <strong id="dli_08_0400__en-us_topic_0000001309855877_b595355332118">map</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b195315312218">array</strong> type.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li273823533910">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b11697183019225">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b157041530132210">fields-scores</strong>, the number of non-primary keys must be an even number, and the second key of every two keys except the primary key must be of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b117042306226">double</strong> type. The <strong id="dli_08_0400__en-us_topic_0000001309855877_b2705193022217">double</strong> value is the score of the previous key. The following is an example:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen31671225143412">CREATE TABLE redisSink (
order_id string,
order_channel string,
order_time double,
pay_amount STRING,
real_pay double,
pay_time string,
user_id double,
user_name string,
area_id double,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i14964853124313"><strong id="dli_08_0400__en-us_topic_0000001309855877_b118612312311">RedisIP</strong></em>',
'password' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i11381115884314"><strong id="dli_08_0400__en-us_topic_0000001309855877_b269512512236">Redis</strong></em><em id="dli_08_0400__en-us_topic_0000001309855877_i9881358447"><strong id="dli_08_0400__en-us_topic_0000001309855877_b5696135182318">Password</strong></em>',
'data-type' = 'sorted-set',
'deploy-mode' = 'master-replica',
'schema-syntax' = 'fields-scores'
);</pre>
</li></ul>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li817313914378"><a name="dli_08_0400__en-us_topic_0000001309855877_li817313914378"></a><a name="en-us_topic_0000001309855877_li817313914378"></a>Restrictions on <strong id="dli_08_0400__en-us_topic_0000001309855877_b8782125610228">data-type</strong>:<ul id="dli_08_0400__en-us_topic_0000001309855877_ul461819254235"><li id="dli_08_0400__en-us_topic_0000001309855877_li1690199111318">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b9455190142320">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b194558042311">string</strong>, only one non-primary key field is allowed.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li35816281117">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b377123715239">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b1451924117230">sorted-set</strong> and <strong id="dli_08_0400__en-us_topic_0000001309855877_b07161744112318">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b733014492231">fields</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b6115135262313">array</strong>, <strong id="dli_08_0400__en-us_topic_0000001309855877_b1013417311244">default-score</strong> is used as the score.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li20784191104919">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b1324110118246">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b1248101114241">sorted-set</strong> and <strong id="dli_08_0400__en-us_topic_0000001309855877_b16250201132419">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b192519114247">map</strong>, there can be only one non-primary key in addition to the primary key and the non-primary key must be of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b18252411172413">map</strong> type. The <strong id="dli_08_0400__en-us_topic_0000001309855877_b152544111247">map</strong> values of the non-primary key must be of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b1325521114244">double</strong> type, indicating the score. The keys in the map are the values in the Redis set.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li3624115182419">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b132688397243">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b1727083916241">sorted-set</strong> and <strong id="dli_08_0400__en-us_topic_0000001309855877_b14272939202418">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b16274173902412">array-scores</strong>, only two non-primary keys are allowed and must be of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b5276163917248">array</strong> type.<div class="p" id="dli_08_0400__en-us_topic_0000001309855877_p212021262410">The first key indicates values in the Redis set. The second key is of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b2447185712244">array&lt;double&gt;</strong> type, indicating index scores. The following is an example:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen529105693713">CREATE TABLE redisSink (
order_id string,
arrayField Array&lt;String&gt;,
arrayScore array&lt;double&gt;,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i95731223184819">RedisIP</em>',
'password' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i422813064810">Redis</em><em id="dli_08_0400__en-us_topic_0000001309855877_i18228130174815">Password</em>',
'data-type' = 'sorted-set',
"default-score" = '3',
'deploy-mode' = 'master-replica',
'schema-syntax' = 'array-scores'
);</pre>
</div>
</li></ul>
</li></ul>
</div>
<div class="section" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0253_en-us_topic_0111499977_section11770184810457"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0243_screen746013311958"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">create</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">dwsSink</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span>
<span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="k">PRIMARY</span><span class="w"> </span><span class="k">KEY</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="p">)</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="n">ENFORCED</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">with</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'connector'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'redis'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'host'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">''</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0253_section4389114817368"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_table15955155035214" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_row29690509522"><th align="left" class="cellrowborder" valign="top" width="22.08%" id="mcps1.3.5.2.2.6.1.1"><p id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_p697520507529">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.35%" id="mcps1.3.5.2.2.6.1.2"><p id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_p997925010528">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.1%" id="mcps1.3.5.2.2.6.1.3"><p id="dli_08_0400__en-us_topic_0000001309855877_p16760165920121">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="11.360000000000001%" id="mcps1.3.5.2.2.6.1.4"><p id="dli_08_0400__en-us_topic_0000001309855877_p115642046191210">Data Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="46.11%" id="mcps1.3.5.2.2.6.1.5"><p id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_p9983150185215">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_row398835017524"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p86719211514">connector</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p967110210510">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p10803204281212">None</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p056434631215">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p56716255115">Connector to be used. Set this parameter to <strong id="dli_08_0400__en-us_topic_0000001309855877_b163461935132514">redis</strong>.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_row210135125214"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p96717235118">host</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p46710219516">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p9196175482415">None</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p10564846141213">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1967111295116">Redis connector address.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_row124195117528"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p7671328515">port</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p66724275112">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1380319422125">6379</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p456424619121">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p16672823515">Redis connector port.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0238_en-us_topic_0113887276_en-us_topic_0111499973_row15401751115211"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p106726215513">password</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p166728216519">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p2572135532413">None</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p356417468127">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1467219219513">Redis authentication password.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row15192195234419"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p419212521448">namespace</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1219295214413">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1192145214440">None</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p0192195264414">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p219255220440">Redis key namespace.</p>
<p id="dli_08_0400__en-us_topic_0000001309855877_p729413564234">For example, if the value is set to "person" and the key is "jack", the value in the Redis is person:jack.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row1135235419445"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1435275418442">delimiter</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p23521554154418">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p135265404414">:</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1635219546445">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p5352454144413">Delimiter between the Redis key and namespace.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row665722864612"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p5657182854620">data-type</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p06574284465">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p6657192854619">hash</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p565752884615">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1565702864615">Redis data type. Available values are as follows:</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul1995143910473"><li id="dli_08_0400__en-us_topic_0000001309855877_li6995193915475">hash</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1727616434474">list</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1966154411477">set</li><li id="dli_08_0400__en-us_topic_0000001309855877_li291412452478">sorted-set</li><li id="dli_08_0400__en-us_topic_0000001309855877_li107761449134720">string</li></ul>
<p id="dli_08_0400__en-us_topic_0000001309855877_p2054208265">For details about the constraints, see <a href="#dli_08_0400__en-us_topic_0000001309855877_li817313914378">Constraints on data-type</a>.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row991541014813"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1091512109486">schema-syntax</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p491541024812">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p3915131054817">fields</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p9915110124818">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1991571004810">Redis schema semantics. Available values are as follows:</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul13496191015240"><li id="dli_08_0400__en-us_topic_0000001309855877_li431013132299"><strong id="dli_08_0400__en-us_topic_0000001309855877_b122291615123318">fields</strong>: applicable to all data types. This value indicates that multiple fields can be set and the value of each field is read when data is written.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li64966101248"><strong id="dli_08_0400__en-us_topic_0000001309855877_b79642055367">fields-scores</strong>: applicable to <strong id="dli_08_0400__en-us_topic_0000001309855877_b1275952917364">sorted-set</strong> data, indicating that each field is read as an independent score.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li16505143442414"><strong id="dli_08_0400__en-us_topic_0000001309855877_b193214125372">array</strong>: applicable to <strong id="dli_08_0400__en-us_topic_0000001309855877_b1934141283719">list</strong>, <strong id="dli_08_0400__en-us_topic_0000001309855877_b0364125373">set</strong>, and <strong id="dli_08_0400__en-us_topic_0000001309855877_b183721210378">sorted-set</strong> data.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li119731655102413"><strong id="dli_08_0400__en-us_topic_0000001309855877_b13558101873713">array-scores</strong>: applicable to <strong id="dli_08_0400__en-us_topic_0000001309855877_b1256031893714">sorted-set</strong> data.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1520510832516"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1315232411371">map</strong>: applicable to <strong id="dli_08_0400__en-us_topic_0000001309855877_b3152132423712">hash</strong> and <strong id="dli_08_0400__en-us_topic_0000001309855877_b315322473714">sorted-set</strong> data.</li></ul>
<p id="dli_08_0400__en-us_topic_0000001309855877_p135020377268">For details about the constraints, see <a href="#dli_08_0400__en-us_topic_0000001309855877_li1877444315214">Constraints on schema-syntax</a>.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row5416114395"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p104117118390">deploy-mode</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p641201111397">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p680320427129">standalone</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p12564114631212">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p124161110399">Deployment mode of the Redis cluster. The value can be <strong id="dli_08_0400__en-us_topic_0000001309855877_b370815465378">standalone</strong>, <strong id="dli_08_0400__en-us_topic_0000001309855877_b13715194619371">master-replica</strong>, or <strong id="dli_08_0400__en-us_topic_0000001309855877_b15716154610373">cluster</strong>. The default value is <strong id="dli_08_0400__en-us_topic_0000001309855877_b102741637388">standalone</strong>.</p>
<p id="dli_08_0400__en-us_topic_0000001309855877_p68986231377">For details about the setting, see the instance type description of the Redis cluster.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row11421243182419"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p91429439241">retry-count</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1214218431249">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p111422436244">5</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p16142343202411">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1514254312247">Number of attempts to connect to the Redis cluster.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row35139459243"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p5513174592413">connection-timeout-millis</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p12513164512415">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p145134451249">10000</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p15131845162410">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p651384513248">Maximum timeout for connecting to the Redis cluster.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row129182132518"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p10932162519">commands-timeout-millis</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p159122112518">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p59320251">2000</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p139728259">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p181022202517">Maximum time for waiting for a completion response.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row145384872415"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p7541748102418">rebalancing-timeout-millis</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p19541448152415">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p5545483242">15000</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p13548489244">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1854448102418">Sleep time when the Redis cluster fails.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row15644042519"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1864110102520">default-score</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p186414012510">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p106416014258">0</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p106410020257">Double</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1464160172514">Default score when <strong id="dli_08_0400__en-us_topic_0000001309855877_b175901837194017">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b125961837104018">sorted-set</strong>.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row2094592813549"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p394542811545">ignore-retraction</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p29451328175414">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1894572817549">false</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p29452283546">Boolean</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p494562835412">Whether to ignore Retract messages.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row142541352162412"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p2254135215244">skip-null-values</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p225445262410">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p425405214241">true</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p52541552162411">Boolean</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p02543525248">Whether null values will be skipped. If this parameter is <strong id="dli_08_0400__en-us_topic_0000001309855877_b1252992311411">false</strong>, <strong id="dli_08_0400__en-us_topic_0000001309855877_b386435410437">null</strong> will be assigned for null values.</p>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row286172813321"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p657753563215">key-ttl-mode</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1957713351326">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p9578133553220">no-ttl</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p11578163523214">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p135781358326">Whether the Redis sink TTL function will be enabled. The value can be <strong id="dli_08_0400__en-us_topic_0000001309855877_b11273469258">no-ttl</strong>, <strong id="dli_08_0400__en-us_topic_0000001309855877_b52126495259">expire-msec</strong>, <strong id="dli_08_0400__en-us_topic_0000001309855877_b173695152518">expire-at-date</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b19730153142511">expire-at-timestamp</strong>.</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul10578535113212"><li id="dli_08_0400__en-us_topic_0000001309855877_li35781135133219"><strong id="dli_08_0400__en-us_topic_0000001309855877_b6204957172516">no-ttl</strong>: No expiration time is set.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li14578133523219"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1120210302612">expire-msec</strong>: validity period of the key. The parameter is a long string, in milliseconds.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li18578133543210"><strong id="dli_08_0400__en-us_topic_0000001309855877_b3357171602613">expire-at-date</strong>: Date and time when the key expires. The value is in UTC time format.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1557853563213"><strong id="dli_08_0400__en-us_topic_0000001309855877_b650125882618">expire-at-timestamp</strong>: Timestamp when the key expires.</li></ul>
</td>
</tr>
<tr id="dli_08_0400__en-us_topic_0000001309855877_row9984143253215"><td class="cellrowborder" valign="top" width="22.08%" headers="mcps1.3.5.2.2.6.1.1 "><p id="dli_08_0400__en-us_topic_0000001309855877_p457863513214">key-ttl</p>
</td>
<td class="cellrowborder" valign="top" width="8.35%" headers="mcps1.3.5.2.2.6.1.2 "><p id="dli_08_0400__en-us_topic_0000001309855877_p15578135183218">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.1%" headers="mcps1.3.5.2.2.6.1.3 "><p id="dli_08_0400__en-us_topic_0000001309855877_p5578153583216">None</p>
</td>
<td class="cellrowborder" valign="top" width="11.360000000000001%" headers="mcps1.3.5.2.2.6.1.4 "><p id="dli_08_0400__en-us_topic_0000001309855877_p257823517323">String</p>
</td>
<td class="cellrowborder" valign="top" width="46.11%" headers="mcps1.3.5.2.2.6.1.5 "><p id="dli_08_0400__en-us_topic_0000001309855877_p1657843517326">Supplementary parameter of <strong id="dli_08_0400__en-us_topic_0000001309855877_b1093813307279">key-ttl-mode</strong>. Available values are as follows:</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul0578123563216"><li id="dli_08_0400__en-us_topic_0000001309855877_li17579133516323">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b890211548273">key-ttl-mode</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b5331357122718">no-ttl</strong>, this parameter does not need to be configured.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li18506202165813">If<strong id="dli_08_0400__en-us_topic_0000001309855877_b029931022820"> key-ttl-mode</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b1886171220286">expire-msec</strong>, set this parameter to a string that can be parsed into the Long type. For example, <strong id="dli_08_0400__en-us_topic_0000001309855877_b73112214288">5000</strong> indicates that the key will expire in 5000 ms.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1457993510323">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b1534118451285">key-ttl-mode</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b154112470287">expire-at-date</strong>, set this parameter to a date.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li185794354324">If <strong id="dli_08_0400__en-us_topic_0000001309855877_b1644434413298">key-ttl-mode</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b126869511297">expire-at-timestamp</strong>, set this parameter to a timestamp, in milliseconds. For example, <strong id="dli_08_0400__en-us_topic_0000001309855877_b1726311176307">1679385600000</strong> indicates that the expiration time is 2023-03-21 16:00:00.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0400__en-us_topic_0000001309855877_dli_08_0253_en-us_topic_0111499977_section1356813564451"><h4 class="sectiontitle">Example</h4><p id="dli_08_0400__en-us_topic_0000001309855877_p7750124218366">In this example, data is read from the Kafka data source and written to the Redis result table. The procedure is as follows:</p>
<ol id="dli_08_0400__en-us_topic_0000001309855877_ol1815817374391"><li id="dli_08_0400__en-us_topic_0000001309855877_li615817373398">Create an enhanced datasource connection in the VPC and subnet where Redis locates, and bind the connection to the required Flink elastic resource pool.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li915853703913">Set Redis security groups and add inbound rules to allow access from the Flink queue. Test the connectivity using the Redis address. If the connection is successful, the datasource is bound to the queue. Otherwise, the binding fails.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li0158133718390">Create a Flink OpenSource SQL job. Enter the following job script and submit the job.<div class="p" id="dli_08_0400__en-us_topic_0000001309855877_p45531643182917"><a name="dli_08_0400__en-us_topic_0000001309855877_li0158133718390"></a><a name="en-us_topic_0000001309855877_li0158133718390"></a>When you create a job, set <strong id="dli_08_0400__en-us_topic_0000001309855877_b18341135915446">Flink Version</strong> to <strong id="dli_08_0400__en-us_topic_0000001309855877_b17342135944414">1.12</strong> on the <strong id="dli_08_0400__en-us_topic_0000001309855877_b6343155944413">Running Parameters</strong> tab. Select <strong id="dli_08_0400__en-us_topic_0000001309855877_b1834365918449">Save Job Log</strong>, and specify the OBS bucket for saving job logs. <strong id="dli_08_0400__en-us_topic_0000001309855877_b1612612112452">Change the values of the parameters in bold as needed in the following script.</strong><pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen2716373296">CREATE TABLE orders (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' =<strong id="dli_08_0400__en-us_topic_0000001309855877_b167973792713"> '&lt;yourTopic&gt;'</strong>,
'properties.bootstrap.servers' =<strong id="dli_08_0400__en-us_topic_0000001309855877_b1561625114411"> '&lt;yourKafka&gt;:&lt;port&gt;'</strong><strong id="dli_08_0400__en-us_topic_0000001309855877_b1916318275440">,</strong>
'properties.group.id' =<strong id="dli_08_0400__en-us_topic_0000001309855877_b826916514447"> '&lt;yourGroupId&gt;',</strong>
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
--In the following redisSink table, data-type is set to default value hash, schema-syntax is fields, and order_id is defined as the primary key. Therefore, the value of this field is used as the Redis key.
CREATE TABLE redisSink (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = <strong id="dli_08_0400__en-us_topic_0000001309855877_b15341175810445">'&lt;yourRedis&gt;',</strong>
'password' =<strong id="dli_08_0400__en-us_topic_0000001309855877_b994045911446"> '&lt;yourPassword&gt;',</strong>
'deploy-mode' = 'master-replica',
'schema-syntax' = 'fields'
);
insert into redisSink select * from orders;</pre>
</div>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li71581137173917">Connect to the Kafka cluster and insert the following test data into Kafka:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen8612182354415">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}
{"order_id":"202103241606060001", "order_channel":"appShop", "order_time":"2021-03-24 16:06:06", "pay_amount":"200.00", "real_pay":"180.00", "pay_time":"2021-03-24 16:10:06", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1015816379390">Run the following commands in Redis and view the result:<ul id="dli_08_0400__en-us_topic_0000001309855877_ul198855512465"><li id="dli_08_0400__en-us_topic_0000001309855877_li6988555104618">Obtain the result whose key is <strong id="dli_08_0400__en-us_topic_0000001309855877_b461316164494">202103241606060001</strong>.<p id="dli_08_0400__en-us_topic_0000001309855877_p4542743123210">Run following command:</p>
<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen58064314321">HGETALL 202103241606060001</pre>
<div class="p" id="dli_08_0400__en-us_topic_0000001309855877_p1196416451324">Command output:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen980114363218"> 1) "user_id"
2) "0001"
3) "user_name"
4) "Alice"
5) "pay_amount"
6) "200.0"
7) "real_pay"
8) "180.0"
9) "order_time"
10) "2021-03-24 16:06:06"
11) "area_id"
12) "330106"
13) "order_channel"
14) "appShop"
15) "pay_time"
16) "2021-03-24 16:10:06"</pre>
</div>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li63879485479">Obtain the result whose key is <strong id="dli_08_0400__en-us_topic_0000001309855877_b7193645204919">202103241000000001</strong>.<p id="dli_08_0400__en-us_topic_0000001309855877_p103921078333">Run following command:</p>
<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen13901146113311">HGETALL 202103241000000001</pre>
<div class="p" id="dli_08_0400__en-us_topic_0000001309855877_p969012993315">Command output:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen190118663316"> 1) "user_id"
2) "0001"
3) "user_name"
4) "Alice"
5) "pay_amount"
6) "100.0"
7) "real_pay"
8) "100.0"
9) "order_time"
10) "2021-03-24 10:00:00"
11) "area_id"
12) "330106"
13) "order_channel"
14) "webShop"
15) "pay_time"
16) "2021-03-24 10:02:03"</pre>
</div>
</li></ul>
</li></ol>
</div>
<div class="section" id="dli_08_0400__en-us_topic_0000001309855877_section12699021112919"><h4 class="sectiontitle">FAQ</h4><ul id="dli_08_0400__en-us_topic_0000001309855877_ul1884332882915"><li id="dli_08_0400__en-us_topic_0000001309855877_li21901723183415">Q: When data-type is <strong id="dli_08_0400__en-us_topic_0000001309855877_b2622226115013">set</strong>, why is the final result data less than the input data?<p id="dli_08_0400__en-us_topic_0000001309855877_p1119141211351">A: This is because the input data contains duplicate data. Deduplication is performed in the Redis set, and the number of records in the result decreases.</p>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li124491329153510">Q: What should I do if Flink job logs contain the following error information?<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen33862213461">org.apache.flink.table.api.ValidationException: SQL validation failed. From line 1, column 40 to line 1, column 105: Parameters must be of the same type</pre>
<p id="dli_08_0400__en-us_topic_0000001309855877_p559073010358">A: The array type is used. However, the types of fields in the array are different. You need to ensure that the types of fields in the array in Redis are the same.</p>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li839910010386">Q: What should I do if Flink job logs contain the following error information?<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen2851111051718">org.apache.flink.addons.redis.core.exception.RedisConnectorException: Wrong Redis schema for 'map' syntax: There should be a key (possibly) and 1 MAP non-key column.</pre>
<p id="dli_08_0400__en-us_topic_0000001309855877_p10726131163817">A: When <strong id="dli_08_0400__en-us_topic_0000001309855877_b103361142142811">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b1548714465280">map</strong>, the table creation statement in Flink can contain only one non-primary key column, and the column type must be <strong id="dli_08_0400__en-us_topic_0000001309855877_b107315012913">map</strong>.</p>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li77391257183919">Q: What should I do if Flink job logs contain the following error information?<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen567519468420">org.apache.flink.addons.redis.core.exception.RedisConnectorException: Wrong Redis schema for 'array' syntax: There should be a key (possibly) and 1 ARRAY non-key column.</pre>
<p id="dli_08_0400__en-us_topic_0000001309855877_p1091511419404">A: When <strong id="dli_08_0400__en-us_topic_0000001309855877_b16693201118298">schema-syntax</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b27007118298">array</strong>, the table creation statement in Flink can contain only one non-primary key column, and the column type must be <strong id="dli_08_0400__en-us_topic_0000001309855877_b770110114298">array</strong>.</p>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li174734871114">Q: What is the function of <strong id="dli_08_0400__en-us_topic_0000001309855877_b1052610311301">schema-syntax</strong> since <strong id="dli_08_0400__en-us_topic_0000001309855877_b223816119303">data-type</strong> has been set?<p id="dli_08_0400__en-us_topic_0000001309855877_p7624755164013">A: <strong id="dli_08_0400__en-us_topic_0000001309855877_b11455153113303">schema-syntax</strong> is used to process special types, such as <strong id="dli_08_0400__en-us_topic_0000001309855877_b86401936163017">map</strong> and <strong id="dli_08_0400__en-us_topic_0000001309855877_b1470933717309">array</strong>.</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul193705374418"><li id="dli_08_0400__en-us_topic_0000001309855877_li17409103184115">If it is set to <strong id="dli_08_0400__en-us_topic_0000001309855877_b17915104133019">fields</strong>, the value of each field is processed. If it is set to <strong id="dli_08_0400__en-us_topic_0000001309855877_b3499315103120">array</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b92031192312">map</strong>, each element in the field is processed. For <strong id="dli_08_0400__en-us_topic_0000001309855877_b553254953416">fields</strong>, the field value of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b837143423517">map</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b1156253615358">array</strong> type is directly used as a value in Redis.</li><li id="dli_08_0400__en-us_topic_0000001309855877_li4822521144116">For <strong id="dli_08_0400__en-us_topic_0000001309855877_b1763853719378">array</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b9713938193711">map</strong>, each value in the array is used as a Redis value, and the field value of the map is used as the Redis value. <strong id="dli_08_0400__en-us_topic_0000001309855877_b030921216466">array-scores</strong> is used to process the <strong id="dli_08_0400__en-us_topic_0000001309855877_b29073818463">sorted-set</strong> data type. It indicates that two array fields are used, the first one is the value in the set, and the second one is the score. <strong id="dli_08_0400__en-us_topic_0000001309855877_b2080526124813">fields-scores</strong> is used to process the <strong id="dli_08_0400__en-us_topic_0000001309855877_b059619358502">sorted-set</strong> data type, indicating that the score is derived from the defined field. The field of an odd number except the primary key indicates the value in the set, and its next field indicates its score. Therefore, its next field must be of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b4475194145313">double</strong> type.</li></ul>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li17843102812917">Q: If <strong id="dli_08_0400__en-us_topic_0000001309855877_b9347719518">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b14347811956">hash</strong>, what are the differences between <strong id="dli_08_0400__en-us_topic_0000001309855877_b53470118515">schema-syntax</strong> set to <strong id="dli_08_0400__en-us_topic_0000001309855877_b43471611515">fields</strong> and that to <strong id="dli_08_0400__en-us_topic_0000001309855877_b334717115512">map</strong>?<p id="dli_08_0400__en-us_topic_0000001309855877_p16404643143213">A: When <strong id="dli_08_0400__en-us_topic_0000001309855877_b4247371353">fields</strong> is used, the field name in Flink is used as the Redis field of the hash data type, and the value of that field is used as the value of the hash data type in Redis. When <strong id="dli_08_0400__en-us_topic_0000001309855877_b17443182131014">map</strong> is used, the field key in Flink is used as the Redis field of the hash data type, and the value of that field is used as the value of the hash data type in Redis. The following is an example:</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul198266136453"><li id="dli_08_0400__en-us_topic_0000001309855877_li165901115144419">For <strong id="dli_08_0400__en-us_topic_0000001309855877_b854112301113">fields</strong>:<ol id="dli_08_0400__en-us_topic_0000001309855877_ol147348205451"><li id="dli_08_0400__en-us_topic_0000001309855877_li473422054514">The execution script of the Flink job is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen49711391351">CREATE TABLE orders (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i4179144718454"><strong id="dli_08_0400__en-us_topic_0000001309855877_b203861310114613">kafkaTopic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i9319103215313"><strong id="dli_08_0400__en-us_topic_0000001309855877_b0322115133213">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i14535174011311"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1466610178329">GroupId</strong></em>',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
CREATE TABLE redisSink (
order_id string,
maptest Map&lt;string, String&gt;,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i846017301461"><strong id="dli_08_0400__en-us_topic_0000001309855877_b4453552144615">RedisIP</strong></em>',
'password' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i20368164784613"><strong id="dli_08_0400__en-us_topic_0000001309855877_b4161454134619">Redis</strong></em><em id="dli_08_0400__en-us_topic_0000001309855877_i207251450154610"><strong id="dli_08_0400__en-us_topic_0000001309855877_b181642544464">Password</strong></em>',
'deploy-mode' = 'master-replica',
'schema-syntax' = 'fields'
);
insert into redisSink select order_id, Map[user_id, area_id] from orders;</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1073492016455">Connect to the Kafka cluster and insert the following test data into the Kafka topic:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen3849185120529">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li073410201457">In the Redis, the result is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen87218340499">1) "maptest"
2) "{0001=330106}"</pre>
</li></ol>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li159771957184813">For <strong id="dli_08_0400__en-us_topic_0000001309855877_b482001341310">map</strong>:<ol id="dli_08_0400__en-us_topic_0000001309855877_ol6342154104916"><li id="dli_08_0400__en-us_topic_0000001309855877_li1534234184916">The execution script of the Flink job is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen1739054616119">CREATE TABLE orders (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i7869132216492"><strong id="dli_08_0400__en-us_topic_0000001309855877_b13869192211498">kafkaTopic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i1430583113490"><strong id="dli_08_0400__en-us_topic_0000001309855877_b113052031184917">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i16834123812492"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1983416383494">GroupId</strong></em>',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
CREATE TABLE redisSink (
order_id string,
maptest Map&lt;string, String&gt;,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i4889149104913"><strong id="dli_08_0400__en-us_topic_0000001309855877_b48891449164915">RedisIP</strong></em>',
'password' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i76065644913"><strong id="dli_08_0400__en-us_topic_0000001309855877_b760656204910">Redis</strong></em><em id="dli_08_0400__en-us_topic_0000001309855877_i8604565496"><strong id="dli_08_0400__en-us_topic_0000001309855877_b17606560496">Password</strong></em>',
'deploy-mode' = 'master-replica',
'schema-syntax' = 'map'
);
insert into redisSink select order_id, Map[user_id, area_id] from orders;</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li334219454918">Connect to the Kafka cluster and insert the following test data into the Kafka topic:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen918513217211">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li1834217414914">In the Redis, the result is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen23211542921">1) "0001"
2) "330106"</pre>
</li></ol>
</li></ul>
</li></ul>
</div>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul423414011313"><li id="dli_08_0400__en-us_topic_0000001309855877_li1023420123117">Q: If <strong id="dli_08_0400__en-us_topic_0000001309855877_b39531158201317">data-type</strong> is <strong id="dli_08_0400__en-us_topic_0000001309855877_b49548580138">list</strong>, what are the differences between <strong id="dli_08_0400__en-us_topic_0000001309855877_b169555584139">schema-syntax</strong> set to <strong id="dli_08_0400__en-us_topic_0000001309855877_b20955185801315">fields</strong> and that to <strong id="dli_08_0400__en-us_topic_0000001309855877_b149561580133">array</strong>?<p id="dli_08_0400__en-us_topic_0000001309855877_p1654152412311">A: The setting to <strong id="dli_08_0400__en-us_topic_0000001309855877_b1311161131515">fields</strong> or <strong id="dli_08_0400__en-us_topic_0000001309855877_b1278414198159">array</strong> does not result in different results. The only difference is that in the Flink table creation statement. <strong id="dli_08_0400__en-us_topic_0000001309855877_b1140594712159">fields</strong> can be multiple fields. However, <strong id="dli_08_0400__en-us_topic_0000001309855877_b1112185731517">array</strong> requires that the field is of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b101038371618">array</strong> type and the data types in the array must be the same. Therefore, <strong id="dli_08_0400__en-us_topic_0000001309855877_b354371121611">fields</strong> are more flexible.</p>
<ul id="dli_08_0400__en-us_topic_0000001309855877_ul1349642111516"><li id="dli_08_0400__en-us_topic_0000001309855877_li144531427515">For <strong id="dli_08_0400__en-us_topic_0000001309855877_b1403715181616">fields</strong>:<ol id="dli_08_0400__en-us_topic_0000001309855877_ol731814115517"><li id="dli_08_0400__en-us_topic_0000001309855877_li103181711155110">The execution script of the Flink job is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen9728154023315">CREATE TABLE orders (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i8541739205114"><strong id="dli_08_0400__en-us_topic_0000001309855877_b354153905113">kafkaTopic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i043014718515"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1543014785117">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i121551655105111"><strong id="dli_08_0400__en-us_topic_0000001309855877_b615518554514">GroupId</strong></em>',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
CREATE TABLE redisSink (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i6425165135215"><strong id="dli_08_0400__en-us_topic_0000001309855877_b44251859525">RedisIP</strong></em>',
'password' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i1898481112522"><strong id="dli_08_0400__en-us_topic_0000001309855877_b189842112529">Redis</strong></em><em id="dli_08_0400__en-us_topic_0000001309855877_i6984201115520"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1984811115215">Password</strong></em>',
'data-type' = 'list',
'deploy-mode' = 'master-replica',
'schema-syntax' = 'fields'
);
insert into redisSink select * from orders;</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li17318121119518">Connect to the Kafka cluster and insert the following test data into the Kafka topic:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen1772964063316">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li431891135112">View the result.<p id="dli_08_0400__en-us_topic_0000001309855877_p16918153114529"><a name="dli_08_0400__en-us_topic_0000001309855877_li431891135112"></a><a name="en-us_topic_0000001309855877_li431891135112"></a>Run the following command in Redis:</p>
<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen8242143115522">LRANGE 202103241000000001 0 8</pre>
<div class="p" id="dli_08_0400__en-us_topic_0000001309855877_p1021416379527">The command output is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen62421031105216">1) "webShop"
2) "2021-03-24 10:00:00"
3) "100.0"
4) "100.0"
5) "2021-03-24 10:02:03"
6) "0001"
7) "Alice"
8) "330106"</pre>
</div>
</li></ol>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li730991812537">For <strong id="dli_08_0400__en-us_topic_0000001309855877_b15415124211174">array</strong>:<ol id="dli_08_0400__en-us_topic_0000001309855877_ol76431207531"><li id="dli_08_0400__en-us_topic_0000001309855877_li1364352017536">The execution script of the Flink job is as follows:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen12729440153318">CREATE TABLE orders (
order_id string,
order_channel string,
order_time string,
pay_amount double,
real_pay double,
pay_time string,
user_id string,
user_name string,
area_id string
) WITH (
'connector' = 'kafka',
'topic' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i466714355311"><strong id="dli_08_0400__en-us_topic_0000001309855877_b14667134319534">kafkaTopic</strong></em>',
'properties.bootstrap.servers' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i634415235313"><strong id="dli_08_0400__en-us_topic_0000001309855877_b4344052115320">KafkaAddress1:KafkaPort,KafkaAddress2:KafkaPort</strong></em>',
'properties.group.id' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i45014113545"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1750171165410">GroupId</strong></em>',
'scan.startup.mode' = 'latest-offset',
'format' = 'json'
);
CREATE TABLE redisSink (
order_id string,
arraytest Array&lt;String&gt;,
primary key (order_id) not enforced
) WITH (
'connector' = 'redis',
'host' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i131778100543"><strong id="dli_08_0400__en-us_topic_0000001309855877_b18177121025414">RedisIP</strong></em>',
'password' = '<em id="dli_08_0400__en-us_topic_0000001309855877_i524361855414"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1243718115412">Redis</strong></em><em id="dli_08_0400__en-us_topic_0000001309855877_i4243111811546"><strong id="dli_08_0400__en-us_topic_0000001309855877_b1824341818545">Password</strong></em>',
'data-type' = 'list',
'deploy-mode' = 'master-replica',
'schema-syntax' = 'array'
);
insert into redisSink select order_id, array[order_channel,order_time,pay_time,user_id,user_name,area_id] from orders;</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li3643122010531">Connect to the Kafka cluster and insert the following test data into the Kafka topic:<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen2729154014333">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_08_0400__en-us_topic_0000001309855877_li5643132012535">In Redis, view the result. (The result is different from that of <strong id="dli_08_0400__en-us_topic_0000001309855877_b137771042171812">fields</strong> because data of the <strong id="dli_08_0400__en-us_topic_0000001309855877_b4486145361816">double</strong> type is not added to the table creation statement of the sink in Flink. Therefore, two values are missing. This is not caused by the difference between <strong id="dli_08_0400__en-us_topic_0000001309855877_b113971586199">fields</strong> and <strong id="dli_08_0400__en-us_topic_0000001309855877_b37671491197">array</strong>.)<pre class="screen" id="dli_08_0400__en-us_topic_0000001309855877_screen07291740103312">1) "webShop"
2) "2021-03-24 10:00:00"
3) "2021-03-24 10:02:03"
4) "0001"
5) "Alice"
6) "330106"</pre>
</li></ol>
</li></ul>
</li></ul>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0391.html">Creating Result Tables</a></div>
</div>
</div>
View Git Blame Copy Permalink