vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0076.html
Su, Xiaomeng 76a5b1ee83 dli_sqlreference_20240227 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2024-03-27 22:02:33 +00:00

466 lines
63 KiB
HTML
Raw Blame History

<a name="dli_08_0076"></a><a name="dli_08_0076"></a>
<h1 class="topictitle1">Creating an OBS Table Using the DataSource Syntax</h1>
<div id="body8662426"><div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_sb2a2dad92c524677baa470c6a05dc948"><h4 class="sectiontitle">Function</h4><p id="dli_08_0076__p28131188218">Create an OBS table using the DataSource syntax.</p>
<p id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_p374295012356">The main differences between the DataSource and the Hive syntax lie in the supported data formats and the number of supported partitions. For details, see syntax and precautions.</p>
<div class="note" id="dli_08_0076__note2901719181211"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0076__p1990121912128">You are advised to use the OBS parallel file system for storage. A parallel file system is a high-performance file system that provides latency in milliseconds, TB/s-level bandwidth, and millions of IOPS. It applies to interactive big data analysis scenarios.</p>
</div></div>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_s2af9615060a543218bb4f231acfcd929"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_u0132702d68ac4186be555e682b694899"><li id="dli_08_0076__dli_08_0076_li15318170215">The size of a table is not calculated when the table is created.</li><li id="dli_08_0076__dli_08_0076_li1453121719215">When data is added, the table size will be changed to 0.</li><li id="dli_08_0076__dli_08_0076_li25313179210">You can check the table size on OBS.</li><li id="dli_08_0076__li27231725165616">Table properties cannot be specified using CTAS table creation statements.</li><li id="dli_08_0076__li119261906338"><strong id="dli_08_0076__b8773182162519">An OBS directory containing subdirectories:</strong><p id="dli_08_0076__p19145415173313">If you specify an OBS directory that contains subdirectories when creating a table, all file types and content within those subdirectories will also be included as table content.</p>
<p id="dli_08_0076__p122610181181">Ensure that all file types in the specified directory and its subdirectories are consistent with the storage format specified in the table creation statement. All file content must be consistent with the fields in the table. Otherwise, errors will be reported in the query.</p>
<p id="dli_08_0076__p1316451410185">You can set <strong id="dli_08_0076__b8652100151319">multiLevelDirEnable</strong> to <strong id="dli_08_0076__b136533020131">true</strong> in the <strong id="dli_08_0076__b5653406135">OPTIONS</strong> statement to query the content in the subdirectory. The default value is <strong id="dli_08_0076__b265310161314">false</strong> (Note that this configuration item is a table attribute, exercise caution when performing this operation. Hive tables do not support this configuration item.)</p>
</li><li id="dli_08_0076__li179241944153512"><strong id="dli_08_0076__b14780241252">Instructions on using partitioned tables:</strong><ul id="dli_08_0076__ul1987589193615"><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_li23993349101821">When a partitioned table is created, the column specified in PARTITIONED BY must be a column in the table, and the partition type must be specified. The partition column supports only the <strong id="dli_08_0076__b1836616402021">string</strong>, <strong id="dli_08_0076__b23676401023">boolean</strong>, <strong id="dli_08_0076__b93674402023">tinyint</strong>, <strong id="dli_08_0076__b1836710408213">smallint</strong>, <strong id="dli_08_0076__b4368540123">short</strong>, <strong id="dli_08_0076__b73686401225">int</strong>, <strong id="dli_08_0076__b9368240227">bigint</strong>, <strong id="dli_08_0076__b536819407213">long</strong>, <strong id="dli_08_0076__b133699401826">decimal</strong>, <strong id="dli_08_0076__b1636984010215">float</strong>, <strong id="dli_08_0076__b1136915401028">double</strong>, <strong id="dli_08_0076__b73699401828">date</strong>, and <strong id="dli_08_0076__b1437010402214">timestamp</strong> type.</li><li id="dli_08_0076__li14473217171911">When a partitioned table is created, the partition field must be the last one or several fields of the table field, and the sequence of the partition fields must be the same. Otherwise, an error occurs.</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0114776207_li1849112813523">A maximum of 200,000 partitions can be created in a single table.</li><li id="dli_08_0076__li18680105312310">CTAS table creation statements cannot be used to create partitioned tables.</li></ul>
</li></ul>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_sd38c9dc329eb4bf78ef91dd6fea75cc1"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0076__en-us_topic_0114776170_screen1890355612543"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="p">]</span><span class="w"> </span><span class="p">[</span><span class="n">db_name</span><span class="p">.]</span><span class="k">table_name</span><span class="w"> </span>
<span class="w"> </span><span class="p">[(</span><span class="n">col_name1</span><span class="w"> </span><span class="n">col_type1</span><span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">col_comment1</span><span class="p">],</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">file_format</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs_path'</span><span class="p">,</span><span class="w"> </span><span class="n">key1</span><span class="o">=</span><span class="n">val1</span><span class="p">,</span><span class="w"> </span><span class="n">key2</span><span class="o">=</span><span class="n">val2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">col_name1</span><span class="p">,</span><span class="w"> </span><span class="n">col_name2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">table_comment</span><span class="p">]</span>
<span class="w"> </span><span class="p">[</span><span class="k">AS</span><span class="w"> </span><span class="n">select_statement</span><span class="p">]</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_s399652e89ec6411f9933551a234fbf60"><h4 class="sectiontitle">Keywords</h4><ul id="dli_08_0076__en-us_topic_0114776170_ul104627471527"><li id="dli_08_0076__en-us_topic_0114776170_li6900512175712"><strong id="dli_08_0076__b11367143815255">IF NOT EXISTS</strong>: Prevents system errors when the created table exists.</li><li id="dli_08_0076__en-us_topic_0114776170_li990011212577"><strong id="dli_08_0076__b193481245194313">USING</strong>: Storage format.</li><li id="dli_08_0076__en-us_topic_0114776170_li24623471029"><strong id="dli_08_0076__b454615499417">OPTIONS</strong>: Property name and property value when a table is created.</li><li id="dli_08_0076__en-us_topic_0114776170_li37086178437"><strong id="dli_08_0076__b1765513272079">COMMENT</strong>: Field or table description.</li><li id="dli_08_0076__en-us_topic_0114776170_li28511440105714"><strong id="dli_08_0076__b204856459710">PARTITIONED BY</strong>: Partition field.</li><li id="dli_08_0076__en-us_topic_0114776170_li15728447102017"><strong id="dli_08_0076__b13113811818">AS</strong>: Run the <strong id="dli_08_0076__b102941416788">CREATE TABLE AS</strong> statement to create a table.</li></ul>
</div>
<div class="section" id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_section1130610201331"><h4 class="sectiontitle">Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_table137579239220" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row17564232219"><th align="left" class="cellrowborder" valign="top" width="16.31%" id="mcps1.3.5.2.2.4.1.1"><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p3756152318220"><strong id="dli_08_0076__b1238014269397">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.340000000000002%" id="mcps1.3.5.2.2.4.1.2"><p id="dli_08_0076__p1838381253217"><strong id="dli_08_0076__b1618543183911">Mandatory</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="68.35%" id="mcps1.3.5.2.2.4.1.3"><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p167561023629"><strong id="dli_08_0076__b7629632113920">Description</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1375611231219"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p107564232216">db_name</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p9383612103219">No</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p5100145012220">Database name</p>
<p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1575614231121">The value can contain letters, numbers, and underscores (_), but it cannot contain only numbers or start with a number or underscore (_).</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row14757152312217"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1375610239211">table_name</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p19383131233213">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p45455102310">Name of the table to be created in the database</p>
<p id="dli_08_0076__dli_08_0076_p1311612123233">The value can contain letters, numbers, and underscores (_), but it cannot contain only numbers or start with a number or underscore (_). The matching rule is <strong id="dli_08_0076__b440814816430">^(?!_)(?![0-9]+$)[A-Za-z0-9_$]*$</strong>.</p>
<p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1275619237212">Special characters must be enclosed in single quotation marks ('').</p>
<p id="dli_08_0076__p13682122683318">The table name is case insensitive.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1275719231525"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1875782313210">col_name</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p10383181243218">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p6795658102316">Column names with data types separated by commas (,)</p>
<p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p27571023024">The column name can contain letters, numbers, and underscores (_), but it cannot contain only numbers and must contain at least one letter.</p>
<p id="dli_08_0076__p1236220301333">The column name is case insensitive.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1075772316216"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1875719238211">col_type</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p238351213213">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__p89202488278">Data type of a column field, which is primitive.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row117571623729"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p87577239219">col_comment</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p1038331215320">No</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p675715235211">Column field description, which can only be string constants.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1775713231524"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p2075713231020">file_format</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p113831812123214">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p1430492032918">Format of the table to be created, which can be <strong id="dli_08_0076__b881020333615">orc</strong>, <strong id="dli_08_0076__b125648519367">parquet</strong>, <strong id="dli_08_0076__b14368814367">json</strong>, <strong id="dli_08_0076__b17291106368">csv</strong>, or <strong id="dli_08_0076__b13419161113360">avro</strong>.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_row16987165396"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_p69889613914">path</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p113834127321">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p15181825787">OBS storage path where data files are stored. You are advised to use an OBS parallel file system for storage.</p>
<p id="dli_08_0076__dli_08_0076_p798856103911">Format: <strong id="dli_08_0076__b8986142894110">obs://bucketName/tblPath</strong></p>
<p id="dli_08_0076__dli_08_0076_p1558471313431"><em id="dli_08_0076__i1754183994119">bucketName</em>: bucket name</p>
<p id="dli_08_0076__dli_08_0076_p16877248134212"><em id="dli_08_0076__i13773115411418">tblPath</em>: directory name. You do not need to specify the file name following the directory.</p>
<p id="dli_08_0076__dli_08_0076_p26601939124810">Refer to <a href="#dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1376011233214">Table 2</a> for details about property names and values during table creation.</p>
<p id="dli_08_0076__dli_08_0076_p16223122514491">Refer to <a href="#dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1376011233214">Table 2</a> and <a href="#dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1876517231928">Table 3</a> for details about the table property names and values when <strong id="dli_08_0076__b01913324217">file_format</strong> is set to <strong id="dli_08_0076__b1119833104218">csv</strong>.</p>
<p id="dli_08_0076__p187561410193514">If there is a folder and a file with the same name in the OBS directory, the path pointed to by the OBS table will prioritize the file over the folder.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row551265517465"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p441312774716">table_comment</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p1438341219322">No</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1741310794711">Table description, which can only be string constants.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row258016494275"><td class="cellrowborder" valign="top" width="16.31%" headers="mcps1.3.5.2.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p55824491274">select_statement</p>
</td>
<td class="cellrowborder" valign="top" width="15.340000000000002%" headers="mcps1.3.5.2.2.4.1.2 "><p id="dli_08_0076__p183831812103214">No</p>
</td>
<td class="cellrowborder" valign="top" width="68.35%" headers="mcps1.3.5.2.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p105821649182719">Used in the <strong id="dli_08_0076__b462215591241">CREATE TABLE AS</strong> statement to insert the <strong id="dli_08_0076__b131212014552">SELECT</strong> query results of the source table or a data record to a table newly created in the OBS bucket.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><a name="dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1376011233214"></a><a name="dli_08_0076_en-us_topic_0114776170_table1376011233214"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1376011233214" frame="border" border="1" rules="all"><caption><b>Table 2 </b>OPTIONS parameters</caption><thead align="left"><tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row127585236212"><th align="left" class="cellrowborder" valign="top" width="23.357664233576642%" id="mcps1.3.5.3.2.4.1.1"><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1875822313218"><strong id="dli_08_0076__b6754827153920">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.518448155184483%" id="mcps1.3.5.3.2.4.1.2"><p id="dli_08_0076__p75042029154010"><strong id="dli_08_0076__b1453131171718">Mandatory</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="61.12388761123887%" id="mcps1.3.5.3.2.4.1.3"><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1175892318219"><strong id="dli_08_0076__b1463383412393">Description</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row67591023927"><td class="cellrowborder" valign="top" width="23.357664233576642%" headers="mcps1.3.5.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p475918233217">path</p>
</td>
<td class="cellrowborder" valign="top" width="15.518448155184483%" headers="mcps1.3.5.3.2.4.1.2 "><p id="dli_08_0076__p115048295402">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.12388761123887%" headers="mcps1.3.5.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1759523228">Path where the table is stored, which currently can only be an OBS directory</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row14760323425"><td class="cellrowborder" valign="top" width="23.357664233576642%" headers="mcps1.3.5.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p675919231321">multiLevelDirEnable</p>
</td>
<td class="cellrowborder" valign="top" width="15.518448155184483%" headers="mcps1.3.5.3.2.4.1.2 "><p id="dli_08_0076__p9958311183815">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.12388761123887%" headers="mcps1.3.5.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p075914233215">Whether data in subdirectories is iteratively queried when there are nested subdirectories. When this parameter is set to <strong id="dli_08_0076__b1475419491237">true</strong>, all files in the table path, including files in subdirectories, are iteratively read when a table is queried.</p>
<p id="dli_08_0076__p1036719158387">Default value: <strong id="dli_08_0076__b1863382792517">false</strong></p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row13841192274012"><td class="cellrowborder" valign="top" width="23.357664233576642%" headers="mcps1.3.5.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p5842222204018">dataDelegated</p>
</td>
<td class="cellrowborder" valign="top" width="15.518448155184483%" headers="mcps1.3.5.3.2.4.1.2 "><p id="dli_08_0076__p69581011183813">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.12388761123887%" headers="mcps1.3.5.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1584219227407">Whether data in the path is cleared when deleting a table or partition</p>
<p id="dli_08_0076__p142921323173817">Default value: <strong id="dli_08_0076__b8417153216252">false</strong></p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_row15918142714198"><td class="cellrowborder" valign="top" width="23.357664233576642%" headers="mcps1.3.5.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_p99307557413">compression</p>
</td>
<td class="cellrowborder" valign="top" width="15.518448155184483%" headers="mcps1.3.5.3.2.4.1.2 "><p id="dli_08_0076__p199584110383">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.12388761123887%" headers="mcps1.3.5.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p8184133018421">Compression format. This parameter is typically required for Parquet files and is set to <strong id="dli_08_0076__b208247257417">zstd</strong>.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="p" id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p2076514239211">When <strong id="dli_08_0076__b1994165815175">file_format</strong> is set to <strong id="dli_08_0076__b06877515428">csv</strong>, you can set the following OPTIONS parameters:
<div class="tablenoborder"><a name="dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1876517231928"></a><a name="dli_08_0076_en-us_topic_0114776170_table1876517231928"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1876517231928" frame="border" border="1" rules="all"><caption><b>Table 3 </b>OPTIONS parameters of the CSV data format</caption><thead align="left"><tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row37611323526"><th align="left" class="cellrowborder" valign="top" width="21.5%" id="mcps1.3.5.4.3.2.4.1.1"><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p07606236212">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="16.91%" id="mcps1.3.5.4.3.2.4.1.2"><p id="dli_08_0076__p82274471393">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="61.59%" id="mcps1.3.5.4.3.2.4.1.3"><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1076010231124">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row27626232216"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1376119231129">delimiter</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p822714719390">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p18761323022">Data separator</p>
<p id="dli_08_0076__p38491299386">Default value: comma (,)</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1476312231529"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p176222311218">quote</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p12271247103919">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1276222311210">Quotation character</p>
<p id="dli_08_0076__p168315373380">Default value: double quotation marks ("")</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row276315231224"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p9763172311210">escape</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p22275472390">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p076310238213">Escape character</p>
<p id="dli_08_0076__p5886241163815">Default value: backslash (\)</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row276416231025"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p77631823727">multiLine</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p32273479392">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p076422316218">Whether the column data contains carriage return characters or transfer characters. The value <strong id="dli_08_0076__b1659682791318">true</strong> indicates yes and the value <strong id="dli_08_0076__b1597727181313">false</strong> indicates no.</p>
<p id="dli_08_0076__p354814503818">Default value: <strong id="dli_08_0076__b641912327250">false</strong></p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row576418235217"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p47641123524">dateFormat</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p722784716390">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1176411239212">Date format of the <strong id="dli_08_0076__b46881557131312">date</strong> field in a CSV file</p>
<p id="dli_08_0076__p1731974863815">Default value: yyyy-MM-dd</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row976482319220"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p57641623328">timestampFormat</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p12227114713395">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p57646239215">Date format of the <strong id="dli_08_0076__b128811726111419">timestamp</strong> field in a CSV file</p>
<p id="dli_08_0076__p1914910500382">Default value:</p>
<p id="dli_08_0076__p202362259394">yyyy-MM-dd HH:mm:ss</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row076592316217"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p2076411231822">mode</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p32271947123913">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p476402318215">Mode for parsing CSV files. The options are as follows: Default value: <strong id="dli_08_0076__b1677125720147">PERMISSIVE</strong></p>
<ul id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_ul1097121216518"><li id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_li1697117127512"><strong id="dli_08_0076__b6439125971415">PERMISSIVE</strong>: Permissive mode. If an incorrect field is encountered, set the line to <strong id="dli_08_0076__b20439185921410">Null</strong>.</li><li id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_li6971141211513"><strong id="dli_08_0076__b156028114489">DROPMALFORMED</strong>: When an incorrect field is encountered, the entire line is discarded.</li><li id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_li99721612151"><strong id="dli_08_0076__b10836915131517">FAILFAST</strong>: Error mode. If an error occurs, it is automatically reported.</li></ul>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1176592316217"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p676513231624">header</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p14227547193911">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p1876517231521">Whether the CSV file contains header information. The value <strong id="dli_08_0076__b1665252310159">true</strong> indicates that the table header information is contained, and the value <strong id="dli_08_0076__b106521923161513">false</strong> indicates that the information is not included.</p>
<p id="dli_08_0076__p1580885613386">Default value: <strong id="dli_08_0076__b144211332132512">false</strong></p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row17285185021820"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p4270145031813">nullValue</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p15227184703918">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p16270125011189">Character that represents the null value. For example, <strong id="dli_08_0076__b11622355121815">nullValue="nl"</strong> indicates that <strong id="dli_08_0076__b1262285516186">nl</strong> represents the null value.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_row1328411508189"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p172701750161816">comment</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p42271647163913">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_en-us_topic_0114776170_p4270185051813">Character that indicates the beginning of the comment. For example, <strong id="dli_08_0076__b08061939141913">comment= '#'</strong> indicates that the line starting with <strong id="dli_08_0076__b14806439101918">#</strong> is a comment.</p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_row4439162317221"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_p1943919239227">compression</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p422764720398">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p34394235227">Data compression format. Currently, <strong id="dli_08_0076__b89829503193">gzip</strong>, <strong id="dli_08_0076__b79821850131919">bzip2</strong>, and <strong id="dli_08_0076__b1398225013194">deflate</strong> are supported. If you do not want to compress data, enter <strong id="dli_08_0076__b8982250191911">none</strong>.</p>
<p id="dli_08_0076__p1515419663919">Default value: <strong id="dli_08_0076__b2263104219205">none</strong></p>
</td>
</tr>
<tr id="dli_08_0076__dli_08_0076_row9827236122319"><td class="cellrowborder" valign="top" width="21.5%" headers="mcps1.3.5.4.3.2.4.1.1 "><p id="dli_08_0076__dli_08_0076_p6827163692311">encoding</p>
</td>
<td class="cellrowborder" valign="top" width="16.91%" headers="mcps1.3.5.4.3.2.4.1.2 "><p id="dli_08_0076__p1422774713393">No</p>
</td>
<td class="cellrowborder" valign="top" width="61.59%" headers="mcps1.3.5.4.3.2.4.1.3 "><p id="dli_08_0076__dli_08_0076_p2828163682311">Data encoding format. Available values are <strong id="dli_08_0076__b5109153310215">utf-8</strong>, <strong id="dli_08_0076__b131101933420">gb2312</strong>, and <strong id="dli_08_0076__b1611116339218">gbk</strong>. Value <strong id="dli_08_0076__b10112733625">utf-8</strong> will be used if this parameter is left empty. </p>
<p id="dli_08_0076__p4573599399">Default value: <strong id="dli_08_0076__b18452182612117">utf-8</strong></p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div class="section" id="dli_08_0076__section175482343414"><a name="dli_08_0076__section175482343414"></a><a name="section175482343414"></a><h4 class="sectiontitle">Example 1: Creating an OBS Non-Partitioned Table</h4><p id="dli_08_0076__p6827182974318">Example description: Create an OBS non-partitioned table named <strong id="dli_08_0076__b466094518229">table1</strong> and use the <strong id="dli_08_0076__b7184654172219">USING</strong> keyword to set the storage format of the table to <strong id="dli_08_0076__b182682519238">orc</strong>.</p>
<p id="dli_08_0076__p3386182910426">You can store OBS tables in <strong id="dli_08_0076__b10917133972318">parquet</strong>, <strong id="dli_08_0076__b10805941132316">json</strong>, or <strong id="dli_08_0076__b10692104420237">avro</strong> format.</p>
<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen177838123432"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">table1</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">col_1</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_2</span><span class="w"> </span><span class="nb">INT</span><span class="p">)</span>
<span class="k">USING</span><span class="w"> </span><span class="n">orc</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs://bucketName/filePath'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0076__section288243544316"><h4 class="sectiontitle">Example 2: Creating an OBS Partitioned Table</h4><p id="dli_08_0076__p287518264445">Example description: Create a partitioned table named <strong id="dli_08_0076__b1229816181243">student</strong>. The partitioned table is partitioned using <strong id="dli_08_0076__b10641143412249">facultyNo</strong> and <strong id="dli_08_0076__b252223715244">classNo</strong>. The <strong id="dli_08_0076__b19890643112418">student</strong> table is partitioned by faculty number (<strong id="dli_08_0076__b565311312257">facultyNo</strong>) and class number (<strong id="dli_08_0076__b1416916202512">classNo</strong>).</p>
<p id="dli_08_0076__p387420166445">In practice, you can select a proper partitioning field and add it to the brackets following the <strong id="dli_08_0076__b6884732132518">PARTITIONED BY</strong> keyword.</p>
<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen166364584414"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">student</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">Name</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">facultyNo</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span>
<span class="w"> </span><span class="n">classNo</span><span class="w"> </span><span class="nb">INT</span><span class="p">)</span>
<span class="k">USING</span><span class="w"> </span><span class="n">csv</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs://bucketName/filePath'</span><span class="p">)</span>
<span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">facultyNo</span><span class="p">,</span><span class="w"> </span><span class="n">classNo</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0076__section17654112105220"><h4 class="sectiontitle">Example 3: Using CTAS to Create an OBS Non-Partitioned Table Using All or Part of the Data in the Source Table</h4><p id="dli_08_0076__p1181822615273">Example description: Based on the OBS table <strong id="dli_08_0076__b152713116289">table1</strong> created in <a href="#dli_08_0076__section175482343414">Example 1: Creating an OBS Non-Partitioned Table</a>, use the CTAS syntax to copy data from <strong id="dli_08_0076__b12961132842810">table1</strong> to <strong id="dli_08_0076__b1397912323281">table1_ctas</strong>.</p>
<p id="dli_08_0076__p776612190912">When using CTAS to create a table, you can ignore the syntax used to create the table being copied. This means that regardless of the syntax used to create <strong id="dli_08_0076__b143481547153616">table1</strong>, you can use the DataSource syntax to create <strong id="dli_08_0076__b1369215513366">table1_ctas</strong>.</p>
<p id="dli_08_0076__p11691052145412">In addition, in this example, the storage format of <strong id="dli_08_0076__b1899141493715">table1</strong> is <strong id="dli_08_0076__b1465712374112">orc</strong>, and the storage format of <strong id="dli_08_0076__b6172114723711">table1_ctas</strong> may be <strong id="dli_08_0076__b98410734212">parquet</strong>. This means that the storage format of the table created by CTAS may be different from that of the original table.</p>
<p id="dli_08_0076__p1029322414589">Use the <strong id="dli_08_0076__b3305548174210">SELECT</strong> statement following the <strong id="dli_08_0076__b115742542428">AS</strong> keyword to select required data and insert the data to <strong id="dli_08_0076__b9937151315474">table1_ctas</strong>.</p>
<p id="dli_08_0076__p184204104544">The <strong id="dli_08_0076__b1089313517473">SELECT</strong> syntax is as follows: <strong id="dli_08_0076__b20969943154710">SELECT &lt;</strong><em id="dli_08_0076__i6479102488">Column name</em><strong id="dli_08_0076__b1032034813473"> &gt; FROM &lt;</strong><em id="dli_08_0076__i684914199482">Table name</em><strong id="dli_08_0076__b0771255104714"> &gt; WHERE &lt;</strong><em id="dli_08_0076__i1822342514487">Related filter criteria</em><strong id="dli_08_0076__b441971154818">&gt;</strong>.</p>
<ul id="dli_08_0076__ul143001810811"><li id="dli_08_0076__li19698153264318">In this example, <strong id="dli_08_0076__b138721044124811">SELECT * FROM table1</strong> is used. <strong id="dli_08_0076__b7646252154817">*</strong> indicates that all columns are selected from <strong id="dli_08_0076__b811719394918">table1</strong> and all data in <strong id="dli_08_0076__b146481571493">table1</strong> is inserted into <strong id="dli_08_0076__b119731710204916">table1_ctas</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen389615271111"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">table1_ctas</span>
<span class="k">USING</span><span class="w"> </span><span class="n">parquet</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs:// bucketName/filePath'</span><span class="p">)</span>
<span class="k">AS</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span>
<span class="k">FROM</span><span class="w"> </span><span class="n">table1</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="dli_08_0076__li88207208112">To filter and insert data into <strong id="dli_08_0076__b1646620935316">table1_ctas</strong> in a customized way, you can use the following <strong id="dli_08_0076__b7732131765315">SELECT</strong> statement: <strong id="dli_08_0076__b14723153112531">SELECT col_1 FROM table1 WHERE col_1 = 'Ann'</strong>. This will allow you to select only <strong id="dli_08_0076__b12555545175313">col_1</strong> from <strong id="dli_08_0076__b19631848105312">table1</strong> and insert data into <strong id="dli_08_0076__b250825435314">table1_ctas</strong> where the value equals <strong id="dli_08_0076__b1445669544">'Ann'</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen1825923319119"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">table1_ctas</span>
<span class="k">USING</span><span class="w"> </span><span class="n">parquet</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs:// bucketName/filePath'</span><span class="p">)</span>
<span class="k">AS</span>
<span class="k">SELECT</span><span class="w"> </span><span class="n">col_1</span>
<span class="k">FROM</span><span class="w"> </span><span class="n">table1</span>
<span class="k">WHERE</span><span class="w"> </span><span class="n">col_1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'Ann'</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div>
<div class="section" id="dli_08_0076__section675283719111"><h4 class="sectiontitle">Example 4: Creating an OBS Non-Partitioned Table and Customizing the Data Type of a Column Field</h4><p id="dli_08_0076__p168826548311">Example description: Create an OBS non-partitioned table named <strong id="dli_08_0076__b10765161913564">table2</strong>. You can customize the native data types of column fields based on service requirements.</p>
<ul id="dli_08_0076__ul10265111414415"><li id="dli_08_0076__li926512141646"><strong id="dli_08_0076__b525475665611">STRING</strong>, <strong id="dli_08_0076__b5716584566">CHAR</strong>, or <strong id="dli_08_0076__b56641959205619">VARCHAR</strong> can be used for text characters.</li><li id="dli_08_0076__li1126531412420"><strong id="dli_08_0076__b122743935717">TIMESTAMP</strong> or <strong id="dli_08_0076__b52771014195719">DATE</strong> can be used for time characters.</li><li id="dli_08_0076__li19265514646"><strong id="dli_08_0076__b571555195814">INT</strong>, <strong id="dli_08_0076__b235915613592">SMALLINT/SHORT</strong>, <strong id="dli_08_0076__b1023941010598">BIGINT/LONG</strong>, or <strong id="dli_08_0076__b12904171295915">TINYINT</strong> can be used for integer characters.</li><li id="dli_08_0076__li17265171414419"><strong id="dli_08_0076__b39572026145911">FLOAT</strong>, <strong id="dli_08_0076__b7369152865916">DOUBLE</strong>, or <strong id="dli_08_0076__b4741534135919">DECIMAL</strong> can be used for decimal calculation.</li><li id="dli_08_0076__li126519141646"><strong id="dli_08_0076__b42271812908">BOOLEAN</strong> can be used if only logical switches are involved.</li></ul>
<p id="dli_08_0076__p117310234312">For details, see "Data Types" &gt; "Primitive Data Types".</p>
<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen177818151356"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">table2</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">col_01</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_02</span><span class="w"> </span><span class="nb">CHAR</span><span class="w"> </span><span class="p">(</span><span class="mi">2</span><span class="p">),</span>
<span class="w"> </span><span class="n">col_03</span><span class="w"> </span><span class="nb">VARCHAR</span><span class="w"> </span><span class="p">(</span><span class="mi">32</span><span class="p">),</span>
<span class="w"> </span><span class="n">col_04</span><span class="w"> </span><span class="k">TIMESTAMP</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_05</span><span class="w"> </span><span class="nb">DATE</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_06</span><span class="w"> </span><span class="nb">INT</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_07</span><span class="w"> </span><span class="nb">SMALLINT</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_08</span><span class="w"> </span><span class="nb">BIGINT</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_09</span><span class="w"> </span><span class="n">TINYINT</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_10</span><span class="w"> </span><span class="nb">FLOAT</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_11</span><span class="w"> </span><span class="n">DOUBLE</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_12</span><span class="w"> </span><span class="nb">DECIMAL</span><span class="w"> </span><span class="p">(</span><span class="mi">10</span><span class="p">,</span><span class="w"> </span><span class="mi">3</span><span class="p">),</span>
<span class="w"> </span><span class="n">col_13</span><span class="w"> </span><span class="nb">BOOLEAN</span>
<span class="p">)</span>
<span class="k">USING</span><span class="w"> </span><span class="n">parquet</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs://bucketName/filePath'</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0076__section9184929957"><h4 class="sectiontitle">Example 5: Creating an OBS Partitioned Table and Customizing OPTIONS Parameters</h4><p id="dli_08_0076__p1322114151015">Example description: When creating an OBS table, you can customize property names and values. For details about OPTIONS parameters, see <a href="#dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1376011233214">Table 2</a>.</p>
<p id="dli_08_0076__p18351634151120">In this example, an OBS partitioned table named <strong id="dli_08_0076__b1825119229171">table3</strong> is created and partitioned based on <strong id="dli_08_0076__b0328129161720">col_2</strong>. Configure <strong id="dli_08_0076__b6466740121711">path</strong>, <strong id="dli_08_0076__b225774281718">multiLevelDirEnable</strong>, <strong id="dli_08_0076__b66644411179">dataDelegated</strong>, and <strong id="dli_08_0076__b1317244881719">compression</strong> in <strong id="dli_08_0076__b17847349111712">OPTIONS</strong>.</p>
<ul id="dli_08_0076__ul172625151156"><li id="dli_08_0076__li526261515155"><strong id="dli_08_0076__b724695581718">path</strong>: OBS storage path. In this example, the value is <strong id="dli_08_0076__b1069672741812">obs://</strong><em id="dli_08_0076__i16803133315182">bucketName</em><strong id="dli_08_0076__b18767163981820">/</strong><em id="dli_08_0076__i138031636161812">filePath</em>, where <em id="dli_08_0076__i9624619101812">bucketName</em> indicates the bucket name and <em id="dli_08_0076__i1385616459189">filePath</em> indicates the actual directory name.</li><li id="dli_08_0076__li16262715171518">In big data scenarios, you are advised to use the OBS parallel file system for storage.</li><li id="dli_08_0076__li026291521511"><strong id="dli_08_0076__b17292107171920">multiLevelDirEnable</strong>: In this example, set this parameter to <strong id="dli_08_0076__b313321413194">true</strong>, indicating that all files and subdirectories in the table path are read iteratively when the table is queried. If this parameter is not required, set it to <strong id="dli_08_0076__b194171341171915">false</strong> or leave it blank (the default value is <strong id="dli_08_0076__b163131845111918">false</strong>).</li><li id="dli_08_0076__li0262515141518"><strong id="dli_08_0076__b176791500198">dataDelegated</strong>: In this example, set this parameter to <strong id="dli_08_0076__b129331314122019">true</strong>, indicating that all data in the path is deleted when a table or partition is deleted. If this parameter is not required, set it to <strong id="dli_08_0076__b1741482614206">false</strong> or leave it blank (the default value is <strong id="dli_08_0076__b382582918200">false</strong>).</li><li id="dli_08_0076__li5262715201513"><strong id="dli_08_0076__b204681934162015">compression</strong>: If the created OBS table needs to be compressed, you can use the keyword <strong id="dli_08_0076__b63462459204">compression</strong> to configure the compression format. In this example, the <strong id="dli_08_0076__b181859542205">zstd</strong> compression format is used.</li></ul>
<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen868716251265"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">table3</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">col_1</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_2</span><span class="w"> </span><span class="nb">int</span>
<span class="p">)</span>
<span class="k">USING</span><span class="w"> </span><span class="n">parquet</span>
<span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">col_2</span><span class="p">)</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">path</span><span class="w"> </span><span class="s1">'obs://bucketName/filePath'</span><span class="p">,</span>
<span class="w"> </span><span class="n">multiLeveldirenable</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">true</span><span class="p">,</span>
<span class="w"> </span><span class="n">datadelegated</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">true</span><span class="p">,</span>
<span class="w"> </span><span class="n">compression</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'zstd'</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0076__section18908162241516"><h4 class="sectiontitle">Example 6: Creating an OBS Non-Partitioned Table and Customizing OPTIONS Parameters</h4><p id="dli_08_0076__p89351319281">Example description: A CSV table is a file format that uses commas to separate data values in plain text. It is commonly used for storing and sharing data, but it is not ideal for complex data types due to its lack of structured data concepts. So, when <strong id="dli_08_0076__b209685155014">file_format</strong> is set to <strong id="dli_08_0076__b1331918475018">csv</strong>, more <strong id="dli_08_0076__b176121612501">OPTIONS</strong> parameters can be configured. For details, see <a href="#dli_08_0076__dli_08_0076_en-us_topic_0114776170_table1876517231928">Table 3</a>.</p>
<p id="dli_08_0076__p663517122292">In this example, a non-partitioned table named <strong id="dli_08_0076__b3641825165018">table4</strong> is created with a <strong id="dli_08_0076__b7506443115217">csv</strong> storage format, and additional <strong id="dli_08_0076__b2027025115316">OPTIONS</strong> parameters are used to constrain the data.</p>
<ul id="dli_08_0076__ul177105810346"><li id="dli_08_0076__li1377115833415"><strong id="dli_08_0076__b28116945310">delimiter</strong>: data separator, indicating that commas (,) are used as separators between data</li><li id="dli_08_0076__li977558133415"><strong id="dli_08_0076__b1854943125519">quote</strong>: quotation character, indicating that double quotation marks (") are used to quota the reference information in the data</li><li id="dli_08_0076__li1677165873413"><strong id="dli_08_0076__b02110205560">escape</strong>: escape character, indicating that backslashes (\) are used as the escape character for data storage</li><li id="dli_08_0076__li2077115814340"><strong id="dli_08_0076__b18775171275710">multiLine</strong>: This parameter is used to set the column data to be stored does not include carriage return or newline characters.</li><li id="dli_08_0076__li2771058113411"><strong id="dli_08_0076__b1929513895914">dataFormat</strong>: indicates that the date format of the <strong id="dli_08_0076__b4758452155914">data</strong> field in the CSV file is <strong id="dli_08_0076__b22391156165914">yyyy-MM-dd</strong>.</li><li id="dli_08_0076__li477158203413"><strong id="dli_08_0076__b15271302010">timestamoFormat</strong>: indicates that the timestamp format in the CSV file is <strong id="dli_08_0076__b12617213309">yyyy-MM-dd HH:mm:ss</strong>.</li><li id="dli_08_0076__li147765819340"><strong id="dli_08_0076__b62312201704">header</strong>: indicates that the CSV table contains the table header information.</li><li id="dli_08_0076__li157755813349"><strong id="dli_08_0076__b6992138108">nullValue</strong>: indicates that <strong id="dli_08_0076__b118031447801">null</strong> is set to indicate the null value in the CSV table.</li><li id="dli_08_0076__li077258113413"><strong id="dli_08_0076__b10211354302">comment</strong>: indicates that the CSV table uses a slash (/) to indicate the beginning of a comment.</li><li id="dli_08_0076__li477105816344"><strong id="dli_08_0076__b184397715">compression</strong>: indicates that the CSV table is compressed in the <strong id="dli_08_0076__b11871561213">gzip</strong>, <strong id="dli_08_0076__b934381928">bzip2</strong>, or <strong id="dli_08_0076__b109581415222">deflate</strong> format. If compression is not required, set this parameter to <strong id="dli_08_0076__b13346122616216">none</strong>.</li><li id="dli_08_0076__li1177185813344"><strong id="dli_08_0076__b23208311120">encoding</strong>: indicates that the table uses the <strong id="dli_08_0076__b1440111811318">utf-8</strong> encoding format. You can choose <strong id="dli_08_0076__b16756113014315">utf-8</strong>, <strong id="dli_08_0076__b245416321236">gb2312</strong>, or <strong id="dli_08_0076__b185295281313">gbk</strong> as needed. The default encoding format is <strong id="dli_08_0076__b13844513316">utf-8</strong>.</li></ul>
<div class="codecoloring" codetype="Sql" id="dli_08_0076__screen6591111419353"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span>
<span class="normal">19</span>
<span class="normal">20</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">table4</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">col_1</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span>
<span class="w"> </span><span class="n">col_2</span><span class="w"> </span><span class="nb">INT</span>
<span class="p">)</span>
<span class="k">USING</span><span class="w"> </span><span class="n">csv</span>
<span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="n">path</span><span class="w"> </span><span class="s1">'obs://bucketName/filePath'</span><span class="p">,</span>
<span class="w"> </span><span class="k">delimiter</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">','</span><span class="p">,</span>
<span class="w"> </span><span class="n">quote</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'#'</span><span class="p">,</span>
<span class="w"> </span><span class="k">escape</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'|'</span><span class="p">,</span>
<span class="w"> </span><span class="n">multiline</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">false</span><span class="p">,</span>
<span class="w"> </span><span class="n">dateFormat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'yyyy-MM-dd'</span><span class="p">,</span>
<span class="w"> </span><span class="n">timestampFormat</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'yyyy-MM-dd HH:mm:ss'</span><span class="p">,</span>
<span class="w"> </span><span class="k">mode</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'failfast'</span><span class="p">,</span>
<span class="w"> </span><span class="n">header</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">true</span><span class="p">,</span>
<span class="w"> </span><span class="n">nullValue</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'null'</span><span class="p">,</span>
<span class="w"> </span><span class="k">comment</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'*'</span><span class="p">,</span>
<span class="w"> </span><span class="n">compression</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'deflate'</span><span class="p">,</span>
<span class="w"> </span><span class="k">encoding</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'utf - 8'</span>
<span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0223.html">Creating an OBS Table</a></div>
</div>
</div>
View Git Blame Copy Permalink