vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0119.html
Su, Xiaomeng 76a5b1ee83 dli_sqlreference_20240227 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2024-03-27 22:02:33 +00:00

123 lines
17 KiB
HTML
Raw Blame History

<a name="dli_08_0119"></a><a name="dli_08_0119"></a>
<h1 class="topictitle1">Creating a DLI Table and Associating It with HBase</h1>
<div id="body1578361312449"><div class="section" id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_section540354616207"><h4 class="sectiontitle">Function</h4><p id="dli_08_0119__p1975691442613">This statement is used to create a DLI table and associate it with an existing HBase table.</p>
<div class="note" id="dli_08_0119__note443995614391"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0119__p964618104214">In Spark cross-source development scenarios, there is a risk of password leakage if datasource authentication information is directly configured. You are advised to use the datasource authentication provided by DLI.</p>
</div></div>
</div>
<div class="section" id="dli_08_0119__section39181250184618"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_08_0119__ul69152143393"><li id="dli_08_0119__li11915161413391">Before creating a DLI table and associating it with HBase, you need to create a datasource connection. For details about operations on the management console, see </li><li id="dli_08_0119__li1933716181390">Ensure that the <span class="filepath" id="dli_08_0119__filepath6456182155610"><b>/etc/hosts</b></span> information of the master node in the MRS cluster is added to the host file of the DLI queue.<p id="dli_08_0119__en-us_topic_0154786456_p12685121631310">For details about how to add an IP-domain mapping, see <span class="filepath" id="dli_08_0119__filepath284011482319"><b>Enhanced Datasource Connection</b></span> in the <i><cite id="dli_08_0119__cite128401648183117">Data Lake Insight User Guide</cite></i>.</p>
</li><li id="dli_08_0119__li1012445583810">The syntax is not supported for security clusters.</li></ul>
</div>
<div class="section" id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_section20884135882017"><h4 class="sectiontitle">Syntax</h4><ul id="dli_08_0119__en-us_topic_0114776213_ul14134191953218"><li id="dli_08_0119__en-us_topic_0114776213_li2134141917327">Single row key<div class="codecoloring" codetype="Sql" id="dli_08_0119__en-us_topic_0114776213_screen19165162014141"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="p">]</span><span class="w"> </span><span class="k">TABLE_NAME</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">ATTR1</span><span class="w"> </span><span class="k">TYPE</span><span class="p">,</span>
<span class="w"> </span><span class="n">ATTR2</span><span class="w"> </span><span class="k">TYPE</span><span class="p">,</span>
<span class="w"> </span><span class="n">ATTR3</span><span class="w"> </span><span class="k">TYPE</span><span class="p">)</span>
<span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="p">[</span><span class="n">CLOUDTABLE</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">HBASE</span><span class="p">]</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'ZKHost'</span><span class="o">=</span><span class="s1">'xx'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'TableName'</span><span class="o">=</span><span class="s1">'TABLE_IN_HBASE'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'RowKey'</span><span class="o">=</span><span class="s1">'ATTR1'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'Cols'</span><span class="o">=</span><span class="s1">'ATTR2:CF1.C1, ATTR3:CF1.C2'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="dli_08_0119__li12781032171312">Combined row key<div class="codecoloring" codetype="Sql" id="dli_08_0119__en-us_topic_0114776213_screen1780618361140"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="p">]</span><span class="w"> </span><span class="k">TABLE_NAME</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">ATTR1</span><span class="w"> </span><span class="n">String</span><span class="p">,</span>
<span class="w"> </span><span class="n">ATTR2</span><span class="w"> </span><span class="n">String</span><span class="p">,</span>
<span class="w"> </span><span class="n">ATTR3</span><span class="w"> </span><span class="k">TYPE</span><span class="p">)</span>
<span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="p">[</span><span class="n">CLOUDTABLE</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">HBASE</span><span class="p">]</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="s1">'ZKHost'</span><span class="o">=</span><span class="s1">'xx'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'TableName'</span><span class="o">=</span><span class="s1">'TABLE_IN_HBASE'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'RowKey'</span><span class="o">=</span><span class="s1">'ATTR1:2, ATTR2:10'</span><span class="p">,</span>
<span class="w"> </span><span class="s1">'Cols'</span><span class="o">=</span><span class="s1">'ATTR2:CF1.C1, ATTR3:CF1.C2'</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div>
<div class="section" id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_section784510261216"><h4 class="sectiontitle">Keywords</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_t2fa8597e0dde4884bd6412ac207ebea8" frame="border" border="1" rules="all"><caption><b>Table 1 </b>CREATE TABLE keywords</caption><thead align="left"><tr id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_r16c75fd8fdbf41c79f9aee82b032351c"><th align="left" class="cellrowborder" valign="top" width="14.2%" id="mcps1.3.4.2.2.3.1.1"><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_a9dd69379a3f54904a9c510fb817659ad"><strong id="dli_08_0119__b1435411214333">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="85.8%" id="mcps1.3.4.2.2.3.1.2"><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_a7ede0d62539542baa7e367cf902bf2b2"><strong id="dli_08_0119__en-us_topic_0093946771_en-us_topic_0053447306_en-us_topic_0039551470_b6335010717346">Description</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0119__en-us_topic_0114776213_row945612181474"><td class="cellrowborder" valign="top" width="14.2%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0119__en-us_topic_0114776213_p1245720181475">USING [CLOUDTABLE | HBASE]</p>
</td>
<td class="cellrowborder" valign="top" width="85.8%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0119__en-us_topic_0114776213_p16457161812478">Specify the HBase datasource to CLOUDTABLE or HBASE. The value is case insensitive.</p>
</td>
</tr>
<tr id="dli_08_0119__en-us_topic_0114776213_row2448851142012"><td class="cellrowborder" valign="top" width="14.2%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0119__en-us_topic_0114776213_p2448105114200">ZKHost</p>
</td>
<td class="cellrowborder" valign="top" width="85.8%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0119__en-us_topic_0114776213_p981318261189">ZooKeeper IP address of the HBase cluster.</p>
<p id="dli_08_0119__en-us_topic_0114776213_p75941420142816">Before obtaining the ZooKeeper IP address, you need to create a datasource connection first..</p>
<ul id="dli_08_0119__en-us_topic_0114776213_ul10649172565316"><li id="dli_08_0119__en-us_topic_0114776213_li206496254531">Access the CloudTable cluster and enter the ZooKeeper IP address (internal network).</li><li id="dli_08_0119__en-us_topic_0114776213_li109404288532">To access the MRS cluster, enter the IP address of the node where the ZooKeeper is located and the external port number of the ZooKeeper. The format is <strong id="dli_08_0119__b257062863313">ZK_IP1:ZK_PORT1,ZK_IP2:ZK_PORT2</strong>.</li></ul>
<div class="note" id="dli_08_0119__en-us_topic_0114776213_note1831342518581"><span class="notetitle"> NOTE: </span><div class="notebody"></div></div>
</td>
</tr>
<tr id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_row166137253465"><td class="cellrowborder" valign="top" width="14.2%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_p584237211576">TableName</p>
</td>
<td class="cellrowborder" valign="top" width="85.8%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_p5902520811576">Specifies the name of a table that has been created in the HBase cluster.</p>
</td>
</tr>
<tr id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_row193609262164"><td class="cellrowborder" valign="top" width="14.2%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_p247311352164">RowKey</p>
</td>
<td class="cellrowborder" valign="top" width="85.8%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_p570649132164">Specifies the row key field of the table connected to DLI. The single and composite row keys are supported. A single row key can be of the numeric or string type. The length does not need to be specified. The composite row key supports only fixed-length data of the string type. The format is <strong id="dli_08_0119__b176382510331"><em id="dli_08_0119__i1363665153313">attribute name 1</em>:<em id="dli_08_0119__i56370517337">Length</em>, <em id="dli_08_0119__i46371051143313">attribute name 2</em>:<em id="dli_08_0119__i196381518335">length</em></strong>.</p>
</td>
</tr>
<tr id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_row469853952167"><td class="cellrowborder" valign="top" width="14.2%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_p477206762167">Cols</p>
</td>
<td class="cellrowborder" valign="top" width="85.8%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_p401695862167">Provides mappings between fields in the DLI table and columns in the HBase table. The mappings are separated by commas (,). In a mapping, the field in the DLI table is located before the colon (:) and information about the HBase table follows the colon (:). In the HBase table information, the column family and column name are separated using a dot (.).</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_section1856617202225"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_ul23381406122620"><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_ldece93075318401a91977668dbc0d8bd">If the to-be-created table exists, an error is reported. To avoid such error, add <strong id="dli_08_0119__en-us_topic_0103157088_en-us_topic_0047959833_b4166539814935">IF NOT EXISTS</strong> in this statement.</li><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_le29363a7cd97427e90619fe10f8ad831">All parameters in OPTIONS are mandatory. Parameter names are case-insensitive, while parameter values are case-sensitive.</li><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_li174001858195616">In OPTIONS, spaces are not allowed before or after the value in the quotation marks because spaces are also considered as a part of the value.</li><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_l3f8a0183662e4f48bfa4dd56f9bf530f">Descriptions of table names and column names support only string constants.</li><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_li725364121919">When creating a table, specify the column name and the corresponding data types. Currently, supported data types include Boolean, short, int, long, float, double, and string.</li><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_li32658394164434">The value of <strong id="dli_08_0119__b12828174673417">row key</strong> (for example, ATTR1) cannot be null, and its length must be greater than 0 and less than or equal to 32767.</li><li id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_li6992212151740">The total number of fields in <strong id="dli_08_0119__b1344617483340">Cols</strong> and <strong id="dli_08_0119__b5446174893411">row key</strong> must be the same as that in the DLI table. Specifically, all fields in the table are mapped to <strong id="dli_08_0119__b94461648113419">Cols</strong> and <strong id="dli_08_0119__b644718486347">row key</strong> without sequence requirements specified.</li><li id="dli_08_0119__en-us_topic_0114776213_li05070236328">The combined row key only supports data of the string type. If the combined row key is used, the length must follow each attribute name. If only one field is specified as the row key, the field type can be any supported data type and you do not need to specify the length.</li><li id="dli_08_0119__en-us_topic_0114776213_li962513443617">If the combined row key is used:<ul id="dli_08_0119__en-us_topic_0114776213_ul4631154643616"><li id="dli_08_0119__en-us_topic_0114776213_li17422440133616">When the row key is inserted, if the actual attribute length is shorter than the specified length when the attribute is used as the row key, add <strong id="dli_08_0119__b185081855153410">\0</strong> after the attribute. If it is longer, the attribute will be truncated when it is inserted into HBase.</li><li id="dli_08_0119__en-us_topic_0114776213_li1857847183712">When reading the <strong id="dli_08_0119__b968514595344">row key</strong> field in HBase, if the actual data length of an attribute is shorter than that specified when the attribute is used as the <strong id="dli_08_0119__b9685145973410">row key</strong>, an error message (<strong id="dli_08_0119__b1068625910349">OutofBoundException</strong>) is reported. If it is longer, the attribute will be truncated during data reading.</li></ul>
</li></ul>
</div>
<div class="section" id="dli_08_0119__en-us_topic_0114776213_en-us_topic_0103157088_section9892337112213"><h4 class="sectiontitle">Example</h4><div class="codecoloring" codetype="Sql" id="dli_08_0119__en-us_topic_0114776213_screen1316415511415"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">test_hbase</span><span class="p">(</span>
<span class="n">ATTR1</span><span class="w"> </span><span class="nb">int</span><span class="p">,</span>
<span class="n">ATTR2</span><span class="w"> </span><span class="nb">int</span><span class="p">,</span>
<span class="n">ATTR3</span><span class="w"> </span><span class="n">string</span><span class="p">)</span>
<span class="k">using</span><span class="w"> </span><span class="n">hbase</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span>
<span class="s1">'ZKHost'</span><span class="o">=</span><span class="s1">'to-hbase-1174405101-CE1bDm5B.datasource.com:2181'</span><span class="p">,</span>
<span class="s1">'TableName'</span><span class="o">=</span><span class="s1">'HBASE_TABLE'</span><span class="p">,</span>
<span class="s1">'RowKey'</span><span class="o">=</span><span class="s1">'ATTR1'</span><span class="p">,</span>
<span class="s1">'Cols'</span><span class="o">=</span><span class="s1">'ATTR2:CF1.C1, ATTR3:CF1.C2'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0118.html">Creating a Datasource Connection with an HBase Table</a></div>
</div>
</div>
View Git Blame Copy Permalink