forked from docs/doc-exports
Su, Xiaomeng
76a5b1ee83
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
149 lines
16 KiB
HTML
149 lines
16 KiB
HTML
<a name="dli_08_0286"></a><a name="dli_08_0286"></a>
|
|
|
|
<h1 class="topictitle1">MRS OpenTSDB Sink Stream</h1>
|
|
<div id="body1607602205184"><div class="section" id="dli_08_0286__en-us_topic_0111563976_section1380110114219"><h4 class="sectiontitle">Function</h4><p id="dli_08_0286__en-us_topic_0111563976_p59449359204">DLI exports the output data of the Flink job to OpenTSDB of MRS.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0286__en-us_topic_0111563976_section7221153411"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0286__ul169351839575"><li id="dli_08_0286__li129356320572">OpenTSDB has been installed in the MRS cluster.</li><li id="dli_08_0286__li14391122610573">In this scenario, jobs must run on the dedicated queue of DLI. Therefore, DLI must interconnect with the enhanced datasource connection that has been connected with MRS clusters. You can also set the security group rules as required.<p id="dli_08_0286__p94091618103916"><a name="dli_08_0286__li14391122610573"></a><a name="li14391122610573"></a>For details about how to create an enhanced datasource connection, see <span class="filepath" id="dli_08_0286__dli_08_0249_en-us_topic_0132966325_filepath13726182511815"><b>Enhanced Datasource Connections</b></span> in the <i><cite id="dli_08_0286__dli_08_0249_cite534516e564224b13bad1de4471da3139181527">Data Lake Insight User Guide</cite></i>.</p>
|
|
<p id="dli_08_0286__p17953337436">For details about how to configure security group rules, see <span class="filepath" id="dli_08_0286__dli_08_0249_filepath78235157113"><b>Security Group</b></span> in the <em id="dli_08_0286__dli_08_0249_i082519153111">Virtual Private Cloud User Guide</em>.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="dli_08_0286__en-us_topic_0111563976_section24316432437"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0286__screen159286201896"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
|
|
<span class="normal"> 2</span>
|
|
<span class="normal"> 3</span>
|
|
<span class="normal"> 4</span>
|
|
<span class="normal"> 5</span>
|
|
<span class="normal"> 6</span>
|
|
<span class="normal"> 7</span>
|
|
<span class="normal"> 8</span>
|
|
<span class="normal"> 9</span>
|
|
<span class="normal">10</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="n">SINK</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">stream_id</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span><span class="p">)</span>
|
|
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"opentsdb"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_metrics</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_timestamps</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_values</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">batch_insert_data_num</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span>
|
|
<span class="w"> </span><span class="p">)</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0286__section158861859181416"><h4 class="sectiontitle">Keywords</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0286__en-us_topic_0111563976_table168611108428" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Keywords</caption><thead align="left"><tr id="dli_08_0286__en-us_topic_0111563976_row188930044212"><th align="left" class="cellrowborder" valign="top" width="18.740000000000002%" id="mcps1.3.4.2.2.4.1.1"><p id="dli_08_0286__en-us_topic_0111563976_p39056054213">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="11.52%" id="mcps1.3.4.2.2.4.1.2"><p id="dli_08_0286__en-us_topic_0111563976_p1991811014215">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="69.74000000000001%" id="mcps1.3.4.2.2.4.1.3"><p id="dli_08_0286__en-us_topic_0111563976_p99261014219">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_08_0286__en-us_topic_0111563976_row169352034217"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p7948180144216">type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p1395820184220">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p296612054212">Sink channel type. <span class="parmvalue" id="dli_08_0286__parmvalue1487717458017"><b>opentsdb</b></span> indicates that data is exported to OpenTSDB of MRS.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__en-us_topic_0111563976_row1797713011427"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p19871702421">region</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p189944010421">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p1414194215">Region where MRS resides.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__row16218164416279"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__p1978635913276">tsdb_link_address</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__p1278665912275">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__p4786185912710">Service address of the OpenTSDB instance in MRS. The format is <strong id="dli_08_0286__b16486511023">http://ip:port</strong> or <strong id="dli_08_0286__b17541661027">https://ip:port</strong>.</p>
|
|
<div class="note" id="dli_08_0286__note17861759162715"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_08_0286__p14222180143419">If <strong id="dli_08_0286__b8735111518216">tsd.https.enabled</strong> is set to <strong id="dli_08_0286__b389111191929">true</strong>, HTTPS must be used. Note that HTTPS does not support certificate authentication.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__en-us_topic_0111563976_row135621144213"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p1969101184212">tsdb_metrics</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p118016144220">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p990818422">Metric of a data point, which can be specified through parameter configurations.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__en-us_topic_0111563976_row39751204220"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p1365154515414">tsdb_timestamps</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p41178115429">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p412510194210">Timestamp of a data point. The data type can be LONG, INT, SHORT, or STRING. Only dynamic columns are supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__en-us_topic_0111563976_row201284194214"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p1614212134220">tsdb_values</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p151571110426">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p1681712419596">Value of a data point. The data type can be SHORT, INT, LONG, FLOAT, DOUBLE, or STRING. Dynamic columns or constant values are supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__en-us_topic_0111563976_row192216355512"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p19221533558">tsdb_tags</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p1221533556">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p222163115516">Tags of a data point. Each of tags contains at least one tag value and up to eight tag values. Tags of the data point can be specified through parameter configurations.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0286__en-us_topic_0111563976_row643812138556"><td class="cellrowborder" valign="top" width="18.740000000000002%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0286__en-us_topic_0111563976_p1243881318551">batch_insert_data_num</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="11.52%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0286__en-us_topic_0111563976_p1043816136552">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.74000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0286__en-us_topic_0111563976_p7438101315551">Number of data records to be written in batches at a time. The value must be a positive integer. The upper limit is <strong id="dli_08_0286__en-us_topic_0111563976_b84235270619346">65536</strong>. The default value is <strong id="dli_08_0286__en-us_topic_0111563976_b842352706193359">8</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0286__section13183202071514"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0286__p776542471516">If a configuration item can be specified through parameter configurations, one or more columns in the record can be used as part of the configuration item. For example, if the configuration item is set to <strong id="dli_08_0286__b179616321336">car_$ {car_brand}</strong> and the value of <strong id="dli_08_0286__b16801153217314">car_brand</strong> in a record is <strong id="dli_08_0286__b17801432130">BMW</strong>, the value of this configuration item is <strong id="dli_08_0286__b15802203215312">car_BMW</strong> in the record.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0286__en-us_topic_0111563976_section1734134944319"><h4 class="sectiontitle">Example</h4><p id="dli_08_0286__en-us_topic_0111563976_p1321261144212">Output data of stream <strong id="dli_08_0286__en-us_topic_0111563976_b84235270693644">weather_out</strong> to OpenTSDB of MRS.</p>
|
|
<div class="codecoloring" codetype="Sql" id="dli_08_0286__en-us_topic_0111563976_screen422210120429"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
|
|
<span class="normal"> 2</span>
|
|
<span class="normal"> 3</span>
|
|
<span class="normal"> 4</span>
|
|
<span class="normal"> 5</span>
|
|
<span class="normal"> 6</span>
|
|
<span class="normal"> 7</span>
|
|
<span class="normal"> 8</span>
|
|
<span class="normal"> 9</span>
|
|
<span class="normal">10</span>
|
|
<span class="normal">11</span>
|
|
<span class="normal">12</span>
|
|
<span class="normal">13</span>
|
|
<span class="normal">14</span>
|
|
<span class="normal">15</span>
|
|
<span class="normal">16</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="n">SINK</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">weather_out</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="n">timestamp_value</span><span class="w"> </span><span class="n">LONG</span><span class="p">,</span><span class="w"> </span><span class="cm">/* Time */</span>
|
|
<span class="w"> </span><span class="n">temperature</span><span class="w"> </span><span class="nb">FLOAT</span><span class="p">,</span><span class="w"> </span><span class="cm">/* Temperature value */</span>
|
|
<span class="w"> </span><span class="n">humidity</span><span class="w"> </span><span class="nb">FLOAT</span><span class="p">,</span><span class="w"> </span><span class="cm">/* Humidity */</span>
|
|
<span class="w"> </span><span class="k">location</span><span class="w"> </span><span class="n">STRING</span><span class="w"> </span><span class="cm">/* Location */</span>
|
|
<span class="p">)</span>
|
|
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"opentsdb"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"xxx"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_link_address</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"https://x.x.x.x:4242"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_metrics</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"weather"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_timestamps</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"${timestamp_value}"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_values</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"${temperature}; ${humidity}"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">tsdb_tags</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"location:${location},signify:temperature; location:${location},signify:humidity"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">batch_insert_data_num</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"10"</span>
|
|
<span class="p">);</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0240.html">Creating a Sink Stream</a></div>
|
|
</div>
|
|
</div>
|
|
|