vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0204.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

139 lines
21 KiB
HTML
Raw Blame History

<a name="dli_08_0204"></a><a name="dli_08_0204"></a>
<h1 class="topictitle1">Creating a DLI Table Using the Hive Syntax</h1>
<div id="body8662426"><div class="section" id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946816_s03a9a8df01184a68831600f336283a25"><h4 class="sectiontitle">Function</h4><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946816_ae11f3982a5344cc59623245f67a56358">This Hive syntax is used to create a DLI table. The main differences between the DataSource and the Hive syntax lie in the supported data formats and the number of supported partitions. For details, see syntax and precautions.</p>
</div>
<div class="section" id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946816_s9c100e961a0c4e9085c14200525b2305"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0204__en-us_topic_0156816283_screen33001332152012"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="p">]</span><span class="w"> </span><span class="p">[</span><span class="n">db_name</span><span class="p">.]</span><span class="k">table_name</span><span class="w"> </span>
<span class="w"> </span><span class="p">[(</span><span class="n">col_name1</span><span class="w"> </span><span class="n">col_type1</span><span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">col_comment1</span><span class="p">],</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">table_comment</span><span class="p">]</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">col_name2</span><span class="w"> </span><span class="n">col_type2</span><span class="p">,</span><span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">col_comment2</span><span class="p">],</span><span class="w"> </span><span class="p">...)]</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="k">ROW</span><span class="w"> </span><span class="n">FORMAT</span><span class="w"> </span><span class="n">row_format</span><span class="p">]</span>
<span class="w"> </span><span class="n">STORED</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="n">file_format</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="n">TBLPROPERTIES</span><span class="w"> </span><span class="p">(</span><span class="n">key1</span><span class="o">=</span><span class="n">val1</span><span class="p">,</span><span class="w"> </span><span class="n">key2</span><span class="o">=</span><span class="n">val2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="p">[</span><span class="k">AS</span><span class="w"> </span><span class="n">select_statement</span><span class="p">];</span>
<span class="n">row_format</span><span class="p">:</span>
<span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="n">SERDE</span><span class="w"> </span><span class="n">serde_cls</span><span class="w"> </span><span class="p">[</span><span class="k">WITH</span><span class="w"> </span><span class="n">SERDEPROPERTIES</span><span class="w"> </span><span class="p">(</span><span class="n">key1</span><span class="o">=</span><span class="n">val1</span><span class="p">,</span><span class="w"> </span><span class="n">key2</span><span class="o">=</span><span class="n">val2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">DELIMITED</span><span class="w"> </span><span class="p">[</span><span class="n">FIELDS</span><span class="w"> </span><span class="n">TERMINATED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="nb">char</span><span class="w"> </span><span class="p">[</span><span class="n">ESCAPED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="nb">char</span><span class="p">]]</span>
<span class="w"> </span><span class="p">[</span><span class="n">COLLECTION</span><span class="w"> </span><span class="n">ITEMS</span><span class="w"> </span><span class="n">TERMINATED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="nb">char</span><span class="p">]</span>
<span class="w"> </span><span class="p">[</span><span class="k">MAP</span><span class="w"> </span><span class="n">KEYS</span><span class="w"> </span><span class="n">TERMINATED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="nb">char</span><span class="p">]</span>
<span class="w"> </span><span class="p">[</span><span class="n">LINES</span><span class="w"> </span><span class="n">TERMINATED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="nb">char</span><span class="p">]</span>
<span class="w"> </span><span class="p">[</span><span class="k">NULL</span><span class="w"> </span><span class="k">DEFINED</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="nb">char</span><span class="p">]</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946816_s9ceb61496680404b879ef5439843c6c7"><h4 class="sectiontitle">Keyword</h4><ul id="dli_08_0204__en-us_topic_0156816283_ul74261946687"><li id="dli_08_0204__en-us_topic_0156816283_li12814539191914">IF NOT EXISTS: Prevents system errors when the created table exists.</li><li id="dli_08_0204__en-us_topic_0156816283_li11143171444414">COMMENT: Field or table description.</li><li id="dli_08_0204__en-us_topic_0156816283_li572161314209">PARTITIONED BY: Partition field.</li><li id="dli_08_0204__li4301114017139">ROW FORMAT: Row data format.</li><li id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946792_l2b4d833ae66e470dae553c300f0783b8">STORED AS: Specifies the format of the file to be stored. Currently, only the TEXTFILE, AVRO, ORC, SEQUENCEFILE, RCFILE, and PARQUET format are supported. This keyword is mandatory when you create DLI tables.</li><li id="dli_08_0204__en-us_topic_0156816283_li1342617461888">TBLPROPERTIES: The TBLPROPERTIES clause allows you to add the <strong id="dli_08_0204__b08348571772">key/value</strong> attribute to a table.<p id="dli_08_0204__p631621212239">For example, if the table storage format is Parquet, you can use <strong id="dli_08_0204__b114842043184919">TBLPROPERTIES(parquet.compression = 'zstd')</strong> to set the table compression format to <strong id="dli_08_0204__b149621947154918">zstd</strong>.</p>
</li><li id="dli_08_0204__en-us_topic_0156816283_li739819389205">AS: Run the CREATE TABLE AS statement to create a table.</li></ul>
</div>
<div class="section" id="dli_08_0204__en-us_topic_0156816283_section1254323371313"><h4 class="sectiontitle">Parameter Description</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0204__en-us_topic_0156816283_table175787333130" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row991835319515"><th align="left" class="cellrowborder" valign="top" width="21.72%" id="mcps1.3.4.2.2.3.1.1"><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p5918185319510"><strong id="dli_08_0204__b3273371882">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="78.28%" id="mcps1.3.4.2.2.3.1.2"><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p1691816531254"><strong id="dli_08_0204__en-us_topic_0093946771_en-us_topic_0053447306_en-us_topic_0039551470_b6335010717346">Description</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row18919165314510"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p1691818532053">db_name</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p29187535520">Database name that contains letters, digits, and underscores (_). The value cannot contain only digits and cannot start with a digit or underscore (_).</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row792017532518"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p792011536519">table_name</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p1292085310518">Table name of a database that contains letters, digits, and underscores (_). The value cannot contain only digits and cannot start with a digit or underscore (_). The matching rule is <strong id="dli_08_0204__b550525134916">^(?!_)(?![0-9]+$)[A-Za-z0-9_$]*$</strong>. If special characters are required, use single quotation marks ('') to enclose them.</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row89201537510"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p69201153552">col_name</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p8920155312511">Column names with data types separated by commas (,). The column name contains letters, digits, and underscores (_). It cannot contain only digits and must contain at least one letter.</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row8920953453"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p1992019538511">col_type</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p12920353254">Field type</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row14921353755"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p892015320519">col_comment</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p149214532516">Field description</p>
</td>
</tr>
<tr id="dli_08_0204__row6193115414135"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__p82281641104318">row_format</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__p72284418433">Line data format</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row19211953658"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p10921125314510">file_format</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p1330615431399">Data storage format: TEXTFILE, AVRO, ORC, SEQUENCEFILE, RCFILE, PARQUET.</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row4661422104814"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p441312774716">table_comment</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p1741310794711">Table description</p>
</td>
</tr>
<tr id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_row16660102274814"><td class="cellrowborder" valign="top" width="21.72%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p55824491274">select_statement</p>
</td>
<td class="cellrowborder" valign="top" width="78.28%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0114776192_p105821649182719">The CREATE TABLE AS statement is used to insert the SELECT query result of the source table or a data record to a newly created DLI table.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0204__en-us_topic_0156816283_section840412012318"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0103084430_ul23381406122620"><li id="dli_08_0204__en-us_topic_0156816283_li10260113951014">When you create a partitioned table, ensure that the specified column in <strong id="dli_08_0204__dli_08_0077_b194512454412">PARTITIONED BY</strong> is not a column in the table and the data type is specified. The partition column supports only the open-source Hive table types including <strong id="dli_08_0204__dli_08_0077_b08051447442">string</strong>, <strong id="dli_08_0204__dli_08_0077_b1180517471947">boolean</strong>, <strong id="dli_08_0204__dli_08_0077_b1180515476411">tinyint</strong>, <strong id="dli_08_0204__dli_08_0077_b2805047046">smallint</strong>, <strong id="dli_08_0204__dli_08_0077_b78063471146">short</strong>, <strong id="dli_08_0204__dli_08_0077_b1580614473415">int</strong>, <strong id="dli_08_0204__dli_08_0077_b118060477415">bigint</strong>, <strong id="dli_08_0204__dli_08_0077_b1680664717411">long</strong>, <strong id="dli_08_0204__dli_08_0077_b178061247341">decimal</strong>, <strong id="dli_08_0204__dli_08_0077_b1880754720416">float</strong>, <strong id="dli_08_0204__dli_08_0077_b88070472413">double</strong>, <strong id="dli_08_0204__dli_08_0077_b68077477413">date</strong>, and <strong id="dli_08_0204__dli_08_0077_b1680715471045">timestamp</strong>.</li><li id="dli_08_0204__li1153995975311">Multiple partition fields can be specified. The partition fields need to be specified after the <strong id="dli_08_0204__dli_08_0077_b498312491414">PARTITIONED BY</strong> keyword, instead of the table name. Otherwise, an error occurs.</li><li id="dli_08_0204__en-us_topic_0156816283_li1849112813523">A maximum of 100,000 partitions can be created in a single table.</li><li id="dli_08_0204__en-us_topic_0156816283_li386022220147">The CREATE TABLE AS statement cannot specify table attributes or create partitioned tables.</li></ul>
</div>
<div class="section" id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946816_s78e8c000b37546edb1c6649aefd793cd"><h4 class="sectiontitle">Example</h4><ul id="dli_08_0204__en-us_topic_0156816283_ul17210511183517"><li id="dli_08_0204__en-us_topic_0156816283_li14210141163512">Create a <strong id="dli_08_0204__b1964818451686">src</strong> table that has <strong id="dli_08_0204__b583404313536">key</strong> and <strong id="dli_08_0204__b362917523532">value</strong> columns in INT and STRING types respectively, and specify a property as required.<div class="codecoloring" codetype="Sql" id="dli_08_0204__en-us_topic_0156816283_screen14310173312396"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">src</span>
<span class="w"> </span><span class="p">(</span><span class="k">key</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span><span class="w"> </span><span class="n">value</span><span class="w"> </span><span class="n">STRING</span><span class="p">)</span>
<span class="w"> </span><span class="n">STORED</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="n">PARQUET</span>
<span class="w"> </span><span class="n">TBLPROPERTIES</span><span class="p">(</span><span class="s1">'key1'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'value1'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="dli_08_0204__en-us_topic_0156816283_li17190329123619">Create a <strong id="dli_08_0204__b6366150181">student</strong> table that has <strong id="dli_08_0204__b136755019810">name</strong>, <strong id="dli_08_0204__b12368155017820">score</strong>, and <strong id="dli_08_0204__b193671150286">classNo</strong> columns, and partition the table by <strong id="dli_08_0204__b555913155117">classNo</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0204__en-us_topic_0156816283_screen491020547381"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">student</span>
<span class="w"> </span><span class="p">(</span><span class="n">name</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span><span class="w"> </span><span class="n">score</span><span class="w"> </span><span class="nb">INT</span><span class="p">)</span>
<span class="w"> </span><span class="n">STORED</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="n">PARQUET</span>
<span class="w"> </span><span class="n">TBLPROPERTIES</span><span class="p">(</span><span class="n">parquet</span><span class="p">.</span><span class="n">compression</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'zstd'</span><span class="p">)</span><span class="w"> </span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="p">(</span><span class="n">classNo</span><span class="w"> </span><span class="nb">INT</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="dli_08_0204__en-us_topic_0156816283_en-us_topic_0093946806_li5175980417415">Create table <strong id="dli_08_0204__b159961135145714">t1</strong> and insert <strong id="dli_08_0204__b17996183516577">t2</strong> data into table <strong id="dli_08_0204__b399743515573">t1</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0204__en-us_topic_0156816283_screen210732211815"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">t1</span>
<span class="w"> </span><span class="n">STORED</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="n">PARQUET</span>
<span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="k">select</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="n">t2</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0224.html">Creating a DLI Table</a></div>
</div>
</div>
View Git Blame Copy Permalink