vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0098.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

128 lines
19 KiB
HTML
Raw Blame History

<a name="dli_08_0098"></a><a name="dli_08_0098"></a>
<h1 class="topictitle1">Creating a DLI Table Using the DataSource Syntax</h1>
<div id="body8662426"><div class="section" id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946816_s03a9a8df01184a68831600f336283a25"><h4 class="sectiontitle">Function</h4><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946816_ae11f3982a5344cc59623245f67a56358">This DataSource syntax can be used to create a DLI table. The main differences between the DataSource and the Hive syntax lie in the supported data formats and the number of supported partitions. For details, see syntax and precautions.</p>
</div>
<div class="section" id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946816_s9c100e961a0c4e9085c14200525b2305"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0098__en-us_topic_0114776192_screen657411815245"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="p">]</span><span class="w"> </span><span class="p">[</span><span class="n">db_name</span><span class="p">.]</span><span class="k">table_name</span><span class="w"> </span>
<span class="w"> </span><span class="p">[(</span><span class="n">col_name1</span><span class="w"> </span><span class="n">col_type1</span><span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">col_comment1</span><span class="p">],</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">file_format</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">key1</span><span class="o">=</span><span class="n">val1</span><span class="p">,</span><span class="w"> </span><span class="n">key2</span><span class="o">=</span><span class="n">val2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">col_name1</span><span class="p">,</span><span class="w"> </span><span class="n">col_name2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">table_comment</span><span class="p">]</span>
<span class="w"> </span><span class="p">[</span><span class="k">AS</span><span class="w"> </span><span class="n">select_statement</span><span class="p">];</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946816_s9ceb61496680404b879ef5439843c6c7"><h4 class="sectiontitle">Keyword</h4><ul id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_ul104627471527"><li id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_li6900512175712">IF NOT EXISTS: Prevents system errors when the created table exists.</li><li id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_li990011212577">USING: Specifies the storage format.</li><li id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_li24623471029">OPTIONS: Specifies the attribute name and attribute value when a table is created.</li><li id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_li37086178437">COMMENT: Field or table description.</li><li id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_li28511440105714">PARTITIONED BY: Partition field.</li><li id="dli_08_0098__dli_08_0076_en-us_topic_0114776170_li15728447102017">AS: Run the CREATE TABLE AS statement to create a table.</li></ul>
</div>
<div class="section" id="dli_08_0098__en-us_topic_0114776192_section1254323371313"><h4 class="sectiontitle">Parameter Description</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0098__en-us_topic_0114776192_table159211531959" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0098__en-us_topic_0114776192_row991835319515"><th align="left" class="cellrowborder" valign="top" width="20.419999999999998%" id="mcps1.3.4.2.2.3.1.1"><p id="dli_08_0098__en-us_topic_0114776192_p5918185319510"><strong id="dli_08_0098__b142488431516">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="79.58%" id="mcps1.3.4.2.2.3.1.2"><p id="dli_08_0098__en-us_topic_0114776192_p1691816531254"><strong id="dli_08_0098__en-us_topic_0093946771_en-us_topic_0053447306_en-us_topic_0039551470_b6335010717346">Description</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0098__en-us_topic_0114776192_row18919165314510"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p1691818532053">db_name</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p29187535520">Database name that contains letters, digits, and underscores (_). The value cannot contain only digits and cannot start with a digit or underscore (_).</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row792017532518"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p792011536519">table_name</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p1292085310518">Table name of a database that contains letters, digits, and underscores (_). The value cannot contain only digits and cannot start with a digit or underscore (_). The matching rule is <strong id="dli_08_0098__b126460511471">^(?!_)(?![0-9]+$)[A-Za-z0-9_$]*$</strong>. If special characters are required, use single quotation marks ('') to enclose them.</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row89201537510"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p69201153552">col_name</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p8920155312511">Column names with data types separated by commas (,). The column name contains letters, digits, and underscores (_). It cannot contain only digits and must contain at least one letter.</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row8920953453"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p1992019538511">col_type</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p12920353254">Field type</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row14921353755"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p892015320519">col_comment</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p149214532516">Field description</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row19211953658"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p10921125314510">file_format</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p1592117532059">Data storage format of DLI tables. The value can be <span class="parmvalue" id="dli_08_0098__parmvalue169319451555"><b>parquet</b></span> only.</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row4661422104814"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p441312774716">table_comment</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p1741310794711">Table description</p>
</td>
</tr>
<tr id="dli_08_0098__en-us_topic_0114776192_row16660102274814"><td class="cellrowborder" valign="top" width="20.419999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_p55824491274">select_statement</p>
</td>
<td class="cellrowborder" valign="top" width="79.58%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_p105821649182719">The CREATE TABLE AS statement is used to insert the SELECT query result of the source table or a data record to a newly created DLI table.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0098__en-us_topic_0114776192_table16713182975016" frame="border" border="1" rules="all"><caption><b>Table 2 </b>OPTIONS parameter description</caption><thead align="left"><tr id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_row127585236212"><th align="left" class="cellrowborder" valign="top" width="20.61%" id="mcps1.3.4.3.2.4.1.1"><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_p1875822313218"><strong id="dli_08_0098__b68661725567">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.489999999999995%" id="mcps1.3.4.3.2.4.1.2"><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_p1175892318219"><strong id="dli_08_0098__b1539670054">Description</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="19.900000000000002%" id="mcps1.3.4.3.2.4.1.3"><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_p1975813231825"><strong id="dli_08_0098__b39971327361">Default Value</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_row14760323425"><td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.4.3.2.4.1.1 "><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_p675919231321">multiLevelDirEnable</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.4.3.2.4.1.2 "><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_p075914233215">Whether to iteratively query data in subdirectories. When this parameter is set to <strong id="dli_08_0098__b25891535666">true</strong>, all files in the table path, including files in subdirectories, are iteratively read when a table is queried.</p>
</td>
<td class="cellrowborder" valign="top" width="19.900000000000002%" headers="mcps1.3.4.3.2.4.1.3 "><p id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0114776170_p1759202312210">false</p>
</td>
</tr>
<tr id="dli_08_0098__row9238173413205"><td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.4.3.2.4.1.1 "><p id="dli_08_0098__p99307557413">compression</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.4.3.2.4.1.2 "><p id="dli_08_0098__p8184133018421">Specified compression format. Generally, you need to set this parameter to <strong id="dli_08_0098__b1990555221914">zstd</strong> for parquet files.</p>
</td>
<td class="cellrowborder" valign="top" width="19.900000000000002%" headers="mcps1.3.4.3.2.4.1.3 "><p id="dli_08_0098__p109301855144112">-</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946816_s1eb23ea343d84a55afe408820775abc5"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0098__en-us_topic_0114776192_ul1422414409"><li id="dli_08_0098__en-us_topic_0114776192_li4225134006">If no delimiter is specified, the comma (,) is used by default.</li></ul>
<ul id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0103084430_ul23381406122620"><li id="dli_08_0098__en-us_topic_0114776192_li10260113951014">When a partitioned table is created, the column specified in PARTITIONED BY must be a column in the table, and the partition type must be specified. The partition column supports only the <strong id="dli_08_0098__dli_08_0076_b1836616402021">string</strong>, <strong id="dli_08_0098__dli_08_0076_b23676401023">boolean</strong>, <strong id="dli_08_0098__dli_08_0076_b93674402023">tinyint</strong>, <strong id="dli_08_0098__dli_08_0076_b1836710408213">smallint</strong>, <strong id="dli_08_0098__dli_08_0076_b4368540123">short</strong>, <strong id="dli_08_0098__dli_08_0076_b73686401225">int</strong>, <strong id="dli_08_0098__dli_08_0076_b9368240227">bigint</strong>, <strong id="dli_08_0098__dli_08_0076_b536819407213">long</strong>, <strong id="dli_08_0098__dli_08_0076_b133699401826">decimal</strong>, <strong id="dli_08_0098__dli_08_0076_b1636984010215">float</strong>, <strong id="dli_08_0098__dli_08_0076_b1136915401028">double</strong>, <strong id="dli_08_0098__dli_08_0076_b73699401828">date</strong>, and <strong id="dli_08_0098__dli_08_0076_b1437010402214">timestamp</strong> type.</li><li id="dli_08_0098__li20349445212">When a partitioned table is created, the partition field must be the last one or several fields of the table field, and the sequence of the partition fields must be the same. Otherwise, an error occurs.</li><li id="dli_08_0098__en-us_topic_0114776192_li1849112813523">A maximum of 7,000 partitions can be created in a single table.</li><li id="dli_08_0098__en-us_topic_0114776192_li386022220147">The CREATE TABLE AS statement cannot specify table attributes or create partitioned tables.</li></ul>
</div>
<div class="section" id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946816_s78e8c000b37546edb1c6649aefd793cd"><h4 class="sectiontitle">Example</h4><ul id="dli_08_0098__en-us_topic_0114776192_ul76161311158"><li id="dli_08_0098__en-us_topic_0114776192_li1561618121512">Create a <strong id="dli_08_0098__b4739444262">src</strong> table that has two columns <strong id="dli_08_0098__b19814447376">key</strong> and <strong id="dli_08_0098__b1427914263711">value</strong> in INT and STRING types respectively, and set the compression format to <strong id="dli_08_0098__b872114119398">zstd</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0098__en-us_topic_0114776192_screen19630122518153"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">src</span><span class="p">(</span><span class="k">key</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span><span class="w"> </span><span class="n">value</span><span class="w"> </span><span class="n">STRING</span><span class="p">)</span><span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">PARQUET</span><span class="w"> </span><span class="k">OPTIONS</span><span class="p">(</span><span class="n">compression</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'zstd'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="dli_08_0098__en-us_topic_0114776192_li10284155111617">Create a <strong id="dli_08_0098__b16414155914612">student</strong> table that has <strong id="dli_08_0098__b6414185917616">name</strong>, <strong id="dli_08_0098__b174151759369">score</strong>, and <strong id="dli_08_0098__b256635964418">classNo</strong> columns and stores data in <strong id="dli_08_0098__b7414125916613">Parquet</strong> format. Partition the table by <strong id="dli_08_0098__b15414145915614">classNo</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0098__en-us_topic_0114776192_screen718701271720"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">student</span><span class="p">(</span><span class="n">name</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span><span class="w"> </span><span class="n">score</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span><span class="w"> </span><span class="n">classNo</span><span class="w"> </span><span class="nb">INT</span><span class="p">)</span><span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">PARQUET</span><span class="w"> </span><span class="k">OPTIONS</span><span class="p">(</span><span class="s1">'key1'</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'value1'</span><span class="p">)</span><span class="w"> </span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="p">(</span><span class="n">classNo</span><span class="p">)</span><span class="w"> </span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<div class="note" id="dli_08_0098__note43241339194916"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0098__p690033371119"><strong id="dli_08_0098__b0725173712508">classNo</strong> is the partition field, which must be placed at the end of the table field, that is, <strong id="dli_08_0098__b535548175019">student(name STRING, score INT, classNo INT)</strong>.</p>
</div></div>
</li><li id="dli_08_0098__en-us_topic_0114776192_en-us_topic_0093946806_li5175980417415">Create table <strong id="dli_08_0098__b177414914717">t1</strong> and insert <strong id="dli_08_0098__b143423198480">t2</strong> data into table <strong id="dli_08_0098__b37589978">t1</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0098__en-us_topic_0114776192_screen210732211815"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">t1</span><span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">parquet</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="k">select</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="n">t2</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0224.html">Creating a DLI Table</a></div>
</div>
</div>
View Git Blame Copy Permalink