forked from docs/doc-exports
Su, Xiaomeng
76a5b1ee83
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
120 lines
13 KiB
HTML
120 lines
13 KiB
HTML
<a name="dli_08_0237"></a><a name="dli_08_0237"></a>
|
|
|
|
<h1 class="topictitle1">CloudTable HBase Source Stream</h1>
|
|
<div id="body1574393070639"><div class="section" id="dli_08_0237__en-us_topic_0111555123_section17358104193813"><h4 class="sectiontitle">Function</h4><p id="dli_08_0237__en-us_topic_0111555123_p86201846131111">Create a source stream to obtain data from HBase of CloudTable as input data of the job. HBase is a column-oriented distributed cloud storage system that features enhanced reliability, excellent performance, and elastic scalability. It applies to the storage of massive amounts of data and distributed computing. You can use HBase to build a storage system capable of storing TB- or even PB-level data. With HBase, you can filter and analyze data with ease and get responses in milliseconds, rapidly mining data value. DLI can read data from HBase for filtering, analysis, and data dumping.</p>
|
|
<p id="dli_08_0237__en-us_topic_0111555123_p8060118">CloudTable is a distributed, scalable, and fully-hosted key-value data storage service based on Apache HBase. It provides DLI with high-performance random read and write capabilities, which are helpful when applications need to store and query a massive amount of structured data, semi-structured data, and time series data. CloudTable applies to IoT scenarios and storage and query of massive volumes of key-value data. For more information about CloudTable, see the <em id="dli_08_0237__i1382682841613">CloudTable Service User Guide</em>.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0237__section1144171011544"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_08_0237__p1038754112542">In this scenario, jobs must run on the dedicated queue of DLI. Therefore, DLI must interconnect with the enhanced datasource connection that has been connected with CloudTable HBase. You can also set the security group rules as required.</p>
|
|
<p id="dli_08_0237__p92641249194114">For details about how to create an enhanced datasource connection, see <span class="filepath" id="dli_08_0237__dli_08_0249_en-us_topic_0132966325_filepath13726182511815"><b>Enhanced Datasource Connections</b></span> in the <i><cite id="dli_08_0237__dli_08_0249_cite534516e564224b13bad1de4471da3139181527">Data Lake Insight User Guide</cite></i>.</p>
|
|
<p id="dli_08_0237__p17953337436">For details about how to configure security group rules, see <span class="filepath" id="dli_08_0237__dli_08_0249_filepath78235157113"><b>Security Group</b></span> in the <em id="dli_08_0237__dli_08_0249_i082519153111">Virtual Private Cloud User Guide</em>.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0237__en-us_topic_0111555123_section1832173783817"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0237__screen16634186122414"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
|
|
<span class="normal">2</span>
|
|
<span class="normal">3</span>
|
|
<span class="normal">4</span>
|
|
<span class="normal">5</span>
|
|
<span class="normal">6</span>
|
|
<span class="normal">7</span>
|
|
<span class="normal">8</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">SOURCE</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">stream_id</span><span class="w"> </span><span class="p">(</span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="w"> </span><span class="p">(</span><span class="s1">','</span><span class="w"> </span><span class="n">attr_name</span><span class="w"> </span><span class="n">attr_type</span><span class="p">)</span><span class="o">*</span><span class="w"> </span><span class="p">)</span>
|
|
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"cloudtable"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">cluster_id</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="k">table_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">table_columns</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">""</span>
|
|
<span class="w"> </span><span class="p">);</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0237__section7855114152619"><h4 class="sectiontitle">Keywords</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0237__en-us_topic_0111555123_table72062393411" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Keywords</caption><thead align="left"><tr id="dli_08_0237__en-us_topic_0111555123_row152211139174120"><th align="left" class="cellrowborder" valign="top" width="12.17%" id="mcps1.3.4.2.2.4.1.1"><p id="dli_08_0237__en-us_topic_0111555123_p722443914413">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="9.01%" id="mcps1.3.4.2.2.4.1.2"><p id="dli_08_0237__en-us_topic_0111555123_p14228539134111">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="78.82000000000001%" id="mcps1.3.4.2.2.4.1.3"><p id="dli_08_0237__en-us_topic_0111555123_p142325392417">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_08_0237__en-us_topic_0111555123_row202354393414"><td class="cellrowborder" valign="top" width="12.17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0237__en-us_topic_0111555123_p92383392419">type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.01%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0237__en-us_topic_0111555123_p19241239104110">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78.82000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0237__en-us_topic_0111555123_p22441239144112">Data source type. <span class="parmvalue" id="dli_08_0237__en-us_topic_0111555123_parmvalue7244193916414"><b>CloudTable</b></span> indicates that the data source is CloudTable.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0237__en-us_topic_0111555123_row11246439134116"><td class="cellrowborder" valign="top" width="12.17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0237__en-us_topic_0111555123_p425023924118">region</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.01%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0237__en-us_topic_0111555123_p1525413396416">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78.82000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0237__en-us_topic_0111555123_p162581839104119">Region to which CloudTable belongs.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0237__en-us_topic_0111555123_row226016398415"><td class="cellrowborder" valign="top" width="12.17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0237__en-us_topic_0111555123_p826343918416">cluster_id</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.01%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0237__en-us_topic_0111555123_p1626710395413">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78.82000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0237__en-us_topic_0111555123_p32691239114115">ID of the cluster to which the data table to be read belongs.</p>
|
|
<p id="dli_08_0237__en-us_topic_0111555123_p59953881">For details about how to view the ID of the CloudTable cluster, see section "<span class="filepath" id="dli_08_0237__en-us_topic_0111555123_filepath73101725131011"><b>Viewing Basic Cluster Information</b></span>" in the <em id="dli_08_0237__i21205492193">CloudTable Service User Guide</em>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0237__en-us_topic_0111555123_row6271193974111"><td class="cellrowborder" valign="top" width="12.17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0237__en-us_topic_0111555123_p1127773911412">table_name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.01%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0237__en-us_topic_0111555123_p2281163915413">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78.82000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0237__en-us_topic_0111555123_p927013528303">Name of the table from which data is to be read. If a namespace needs to be specified, set it to <strong id="dli_08_0237__b1832603712566">namespace_name:table_name</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0237__en-us_topic_0111555123_row11285183944117"><td class="cellrowborder" valign="top" width="12.17%" headers="mcps1.3.4.2.2.4.1.1 "><p id="dli_08_0237__en-us_topic_0111555123_p2290193914118">table_columns</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.01%" headers="mcps1.3.4.2.2.4.1.2 "><p id="dli_08_0237__en-us_topic_0111555123_p152931399414">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78.82000000000001%" headers="mcps1.3.4.2.2.4.1.3 "><p id="dli_08_0237__en-us_topic_0111555123_p12296539154111">Column to be read. The format is <strong id="dli_08_0237__en-us_topic_0111555123_b842352706103811">rowKey,f1:c1,f1:c2,f2:c1</strong>. The number of columns must be the same as the number of attributes specified in the source stream.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0237__section1189214810279"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0237__p39756599487">When creating a source stream, you can specify a time model for subsequent calculation. Currently, DLI supports two time models: Processing Time and Event Time. For details about the syntax, see <a href="dli_08_0107.html">Configuring Time Models</a>.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0237__section11421328152716"><h4 class="sectiontitle">Example</h4><p id="dli_08_0237__en-us_topic_0111555123_p16386153984115">Read the <strong id="dli_08_0237__en-us_topic_0111555123_b84235270692121">car_infos</strong> table from HBase of CloudTable.</p>
|
|
<div class="codecoloring" codetype="Sql" id="dli_08_0237__en-us_topic_0111555123_screen33871139204114"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
|
|
<span class="normal"> 2</span>
|
|
<span class="normal"> 3</span>
|
|
<span class="normal"> 4</span>
|
|
<span class="normal"> 5</span>
|
|
<span class="normal"> 6</span>
|
|
<span class="normal"> 7</span>
|
|
<span class="normal"> 8</span>
|
|
<span class="normal"> 9</span>
|
|
<span class="normal">10</span>
|
|
<span class="normal">11</span>
|
|
<span class="normal">12</span>
|
|
<span class="normal">13</span>
|
|
<span class="normal">14</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">SOURCE</span><span class="w"> </span><span class="n">STREAM</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="n">car_id</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">car_owner</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">car_age</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">average_speed</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">total_miles</span><span class="w"> </span><span class="nb">INT</span>
|
|
<span class="p">)</span>
|
|
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="p">(</span>
|
|
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"cloudtable"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">region</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"xxx"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">cluster_id</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"209ab1b6-de25-4c48-8e1e-29e09d02de28"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="k">table_name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"carinfo"</span><span class="p">,</span>
|
|
<span class="w"> </span><span class="n">table_columns</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">"rowKey,info:owner,info:age,car:speed,car:miles"</span>
|
|
<span class="p">);</span><span class="w"> </span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0234.html">Creating a Source Stream</a></div>
|
|
</div>
|
|
</div>
|
|
|