vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dli/sqlreference/dli_08_0076.html
Su, Xiaomeng 04d4597cf3 dli_sqlreference_0511_version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
2023-11-02 14:34:08 +00:00

259 lines
35 KiB
HTML
Raw Blame History

<a name="dli_08_0076"></a><a name="dli_08_0076"></a>
<h1 class="topictitle1">Creating an OBS Table Using the DataSource Syntax</h1>
<div id="body8662426"><div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_sb2a2dad92c524677baa470c6a05dc948"><h4 class="sectiontitle">Function</h4><p id="dli_08_0076__p28131188218">Create an OBS table using the DataSource syntax.</p>
<p id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_p374295012356">The main differences between the DataSource and the Hive syntax lie in the supported data formats and the number of supported partitions. For details, see syntax and precautions.</p>
</div>
<div class="section" id="dli_08_0076__section471444971619"><h4 class="sectiontitle">Usage</h4><ul id="dli_08_0076__ul9531201710213"><li id="dli_08_0076__li15318170215">The size of the table will not be calculated during table creation.</li><li id="dli_08_0076__li1453121719215">When data is added, the table size will be changed to 0.</li><li id="dli_08_0076__li25313179210">You can view the table size on OBS.</li></ul>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_s2af9615060a543218bb4f231acfcd929"><h4 class="sectiontitle">Precautions</h4><ul id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_u0132702d68ac4186be555e682b694899"><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_le29363a7cd97427e90619fe10f8ad831">The table and column names are case-insensitive.</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_l3f8a0183662e4f48bfa4dd56f9bf530f">Descriptions of table names and column names support only string constants.</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_lb2b5dfc764244f3795c292ba968a167a">During table creation, you need to specify the column name and corresponding data type. The data type is primitive type.</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_li24449229145816">If a folder and a file have the same name in the OBS directory, the file is preferred as the path when creating an OBS table.</li><li id="dli_08_0076__li8323413181818">During table creation, if the specified path is an OBS directory and it contains subdirectories (or nested subdirectories), all file types and content in the subdirectories are considered table content.<p id="dli_08_0076__p122610181181"><a name="dli_08_0076__li8323413181818"></a><a name="li8323413181818"></a>Ensure that all file types in the specified directory and its subdirectories are consistent with the storage format specified in the table creation statement. All file content must be consistent with the fields in the table. Otherwise, errors will be reported in the query.</p>
<p id="dli_08_0076__p1316451410185">You can set <strong id="dli_08_0076__b78549347572">multiLevelDirEnable</strong> to <strong id="dli_08_0076__b6854173455710">true</strong> in the <strong id="dli_08_0076__b585443411576">OPTIONS</strong> statement to query the content in the subdirectory. The default value is <strong id="dli_08_0076__b188551534115715">false</strong> (Note that this configuration item is a table attribute, exercise caution when performing this operation). Hive tables do not support this configuration item.</p>
</li><li id="dli_08_0076__en-us_topic_0114776170_li118982045154718">The OBS storage path must be a directory on the OBS. The directory must be created in advance and be empty.</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_li23993349101821">When a partitioned table is created, the column specified in PARTITIONED BY must be a column in the table, and the partition type must be specified. The partition column supports only the <strong id="dli_08_0076__b1836616402021">string</strong>, <strong id="dli_08_0076__b23676401023">boolean</strong>, <strong id="dli_08_0076__b93674402023">tinyint</strong>, <strong id="dli_08_0076__b1836710408213">smallint</strong>, <strong id="dli_08_0076__b4368540123">short</strong>, <strong id="dli_08_0076__b73686401225">int</strong>, <strong id="dli_08_0076__b9368240227">bigint</strong>, <strong id="dli_08_0076__b536819407213">long</strong>, <strong id="dli_08_0076__b133699401826">decimal</strong>, <strong id="dli_08_0076__b1636984010215">float</strong>, <strong id="dli_08_0076__b1136915401028">double</strong>, <strong id="dli_08_0076__b73699401828">date</strong>, and <strong id="dli_08_0076__b1437010402214">timestamp</strong> type.</li><li id="dli_08_0076__li14473217171911">When a partitioned table is created, the partition field must be the last one or several fields of the table field, and the sequence of the partition fields must be the same. Otherwise, an error occurs.</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0114776207_li1849112813523">A maximum of 7,000 partitions can be created in a single table.</li><li id="dli_08_0076__en-us_topic_0114776170_li1879214429464">The CREATE TABLE AS statement cannot specify table attributes or create partitioned tables.</li></ul>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_sd38c9dc329eb4bf78ef91dd6fea75cc1"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0076__en-us_topic_0114776170_screen1890355612543"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="p">]</span><span class="w"> </span><span class="p">[</span><span class="n">db_name</span><span class="p">.]</span><span class="k">table_name</span><span class="w"> </span>
<span class="w"> </span><span class="p">[(</span><span class="n">col_name1</span><span class="w"> </span><span class="n">col_type1</span><span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">col_comment1</span><span class="p">],</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">file_format</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs_path'</span><span class="p">,</span><span class="w"> </span><span class="n">key1</span><span class="o">=</span><span class="n">val1</span><span class="p">,</span><span class="w"> </span><span class="n">key2</span><span class="o">=</span><span class="n">val2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span><span class="w"> </span>
<span class="w"> </span><span class="p">[</span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">col_name1</span><span class="p">,</span><span class="w"> </span><span class="n">col_name2</span><span class="p">,</span><span class="w"> </span><span class="p">...)]</span>
<span class="w"> </span><span class="p">[</span><span class="k">COMMENT</span><span class="w"> </span><span class="n">table_comment</span><span class="p">]</span>
<span class="w"> </span><span class="p">[</span><span class="k">AS</span><span class="w"> </span><span class="n">select_statement</span><span class="p">];</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_s399652e89ec6411f9933551a234fbf60"><h4 class="sectiontitle">Keyword</h4><ul id="dli_08_0076__en-us_topic_0114776170_ul104627471527"><li id="dli_08_0076__en-us_topic_0114776170_li6900512175712">IF NOT EXISTS: Prevents system errors when the created table exists.</li><li id="dli_08_0076__en-us_topic_0114776170_li990011212577">USING: Specifies the storage format.</li><li id="dli_08_0076__en-us_topic_0114776170_li24623471029">OPTIONS: Specifies the attribute name and attribute value when a table is created.</li><li id="dli_08_0076__en-us_topic_0114776170_li37086178437">COMMENT: Field or table description.</li><li id="dli_08_0076__en-us_topic_0114776170_li28511440105714">PARTITIONED BY: Partition field.</li><li id="dli_08_0076__en-us_topic_0114776170_li15728447102017">AS: Run the CREATE TABLE AS statement to create a table.</li></ul>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_section1130610201331"><h4 class="sectiontitle">Parameter</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0076__en-us_topic_0114776170_table137579239220" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_08_0076__en-us_topic_0114776170_row17564232219"><th align="left" class="cellrowborder" valign="top" width="21.3%" id="mcps1.3.6.2.2.3.1.1"><p id="dli_08_0076__en-us_topic_0114776170_p3756152318220"><strong id="dli_08_0076__b1093719411907">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="78.7%" id="mcps1.3.6.2.2.3.1.2"><p id="dli_08_0076__en-us_topic_0114776170_p167561023629"><strong id="dli_08_0076__en-us_topic_0093946771_en-us_topic_0053447306_en-us_topic_0039551470_b6335010717346">Description</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0076__en-us_topic_0114776170_row1375611231219"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p107564232216">db_name</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__p5100145012220">Database name</p>
<p id="dli_08_0076__en-us_topic_0114776170_p1575614231121">The value can contain letters, numbers, and underscores (_), but cannot contain only numbers or start with a number or underscore (_).</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row14757152312217"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p1375610239211">table_name</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__p45455102310">Name of the table to be created in the database</p>
<p id="dli_08_0076__p1311612123233">The value can contain letters, numbers, and underscores (_), but cannot contain only numbers or start with a number or underscore (_). The matching rule is <strong id="dli_08_0076__b1495415543615">^(?!_)(?![0-9]+$)[A-Za-z0-9_$]*$</strong>.</p>
<p id="dli_08_0076__en-us_topic_0114776170_p1275619237212">Special characters must be enclosed in single quotation marks ('').</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row1275719231525"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p1875782313210">col_name</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__p6795658102316">Column names with data types separated by commas (,)</p>
<p id="dli_08_0076__en-us_topic_0114776170_p27571023024">The column name contains letters, digits, and underscores (_). It cannot contain only digits and must contain at least one letter.</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row1075772316216"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p1875719238211">col_type</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p197578239211">Data type of a column field</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row117571623729"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p87577239219">col_comment</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p675715235211">Column field description</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row1775713231524"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p2075713231020">file_format</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__p1430492032918">Input format of the table. The value can be <strong id="dli_08_0076__b16289634134410">orc</strong>, <strong id="dli_08_0076__b3734103514443">parquet</strong>, <strong id="dli_08_0076__b1423117376448">json</strong>, <strong id="dli_08_0076__b184213894410">csv</strong>, or <strong id="dli_08_0076__b185801843124420">avro</strong>.</p>
</td>
</tr>
<tr id="dli_08_0076__row16987165396"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__p69889613914">path</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__p1267212538385">OBS storage path where data files are stored</p>
<p id="dli_08_0076__p798856103911">Format: <strong id="dli_08_0076__b8986142894110">obs://bucketName/tblPath</strong></p>
<p id="dli_08_0076__p1558471313431"><em id="dli_08_0076__i1754183994119">bucketName</em>: bucket name</p>
<p id="dli_08_0076__p16877248134212"><em id="dli_08_0076__i13773115411418">tblPath</em>: directory name. You do not need to specify the file name following the directory.</p>
<p id="dli_08_0076__p26601939124810">For details about attribute names and values during table creation, see <a href="#dli_08_0076__en-us_topic_0114776170_table1376011233214">Table 2</a>.</p>
<p id="dli_08_0076__p16223122514491">For details about the table attribute names and values when <strong id="dli_08_0076__b1479225144320">file_format</strong> is set to <strong id="dli_08_0076__b145934269435">csv</strong>, see <a href="#dli_08_0076__en-us_topic_0114776170_table1376011233214">Table 2</a> and <a href="#dli_08_0076__en-us_topic_0114776170_table1876517231928">Table 3</a>.</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row551265517465"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p441312774716">table_comment</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p1741310794711">Description of the table</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row258016494275"><td class="cellrowborder" valign="top" width="21.3%" headers="mcps1.3.6.2.2.3.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p55824491274">select_statement</p>
</td>
<td class="cellrowborder" valign="top" width="78.7%" headers="mcps1.3.6.2.2.3.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p105821649182719">The <strong id="dli_08_0076__b349163774614">CREATE TABLE AS</strong> statement is used to insert the <strong id="dli_08_0076__b1232219415462">SELECT</strong> query result of the source table or a data record to a new table in OBS bucket.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="tablenoborder"><a name="dli_08_0076__en-us_topic_0114776170_table1376011233214"></a><a name="en-us_topic_0114776170_table1376011233214"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0076__en-us_topic_0114776170_table1376011233214" frame="border" border="1" rules="all"><caption><b>Table 2 </b>OPTIONS parameter description</caption><thead align="left"><tr id="dli_08_0076__en-us_topic_0114776170_row127585236212"><th align="left" class="cellrowborder" valign="top" width="20.61%" id="mcps1.3.7.2.4.1.1"><p id="dli_08_0076__en-us_topic_0114776170_p1875822313218"><strong id="dli_08_0076__b76651491819">Parameter</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.489999999999995%" id="mcps1.3.7.2.4.1.2"><p id="dli_08_0076__en-us_topic_0114776170_p1175892318219"><strong id="dli_08_0076__b519098918">Description</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="19.900000000000002%" id="mcps1.3.7.2.4.1.3"><p id="dli_08_0076__en-us_topic_0114776170_p1975813231825">Default Value</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0076__en-us_topic_0114776170_row67591023927"><td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.7.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p475918233217">path</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.7.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p1759523228">Specified table storage location. Currently, only OBS is supported.</p>
</td>
<td class="cellrowborder" valign="top" width="19.900000000000002%" headers="mcps1.3.7.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p1975916231526">-</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row14760323425"><td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.7.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p675919231321">multiLevelDirEnable</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.7.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p075914233215">Whether to iteratively query data in subdirectories when subdirectories are nested. When this parameter is set to <strong id="dli_08_0076__b897914172018">true</strong>, all files in the table path, including files in subdirectories, are iteratively read when a table is queried.</p>
</td>
<td class="cellrowborder" valign="top" width="19.900000000000002%" headers="mcps1.3.7.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p1759202312210">false</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row13841192274012"><td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.7.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p5842222204018">dataDelegated</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.7.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p1584219227407">Whether to clear data in the path when deleting a table or partition</p>
</td>
<td class="cellrowborder" valign="top" width="19.900000000000002%" headers="mcps1.3.7.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p12842152215407">false</p>
</td>
</tr>
<tr id="dli_08_0076__row15918142714198"><td class="cellrowborder" valign="top" width="20.61%" headers="mcps1.3.7.2.4.1.1 "><p id="dli_08_0076__p99307557413">compression</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.7.2.4.1.2 "><p id="dli_08_0076__p8184133018421">Specified compression format. Generally, you need to set this parameter to <strong id="dli_08_0076__b450093155812">zstd</strong> for parquet files.</p>
</td>
<td class="cellrowborder" valign="top" width="19.900000000000002%" headers="mcps1.3.7.2.4.1.3 "><p id="dli_08_0076__p109301855144112">-</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="p" id="dli_08_0076__en-us_topic_0114776170_p2076514239211">When the file format is set to <strong id="dli_08_0076__b66432316110">CSV</strong>, you can set the following OPTIONS parameters:
<div class="tablenoborder"><a name="dli_08_0076__en-us_topic_0114776170_table1876517231928"></a><a name="en-us_topic_0114776170_table1876517231928"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0076__en-us_topic_0114776170_table1876517231928" frame="border" border="1" rules="all"><caption><b>Table 3 </b>OPTIONS parameter description of the CSV data format</caption><thead align="left"><tr id="dli_08_0076__en-us_topic_0114776170_row37611323526"><th align="left" class="cellrowborder" valign="top" width="17.581758175817587%" id="mcps1.3.8.2.2.4.1.1"><p id="dli_08_0076__en-us_topic_0114776170_p07606236212">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="58.095809580958104%" id="mcps1.3.8.2.2.4.1.2"><p id="dli_08_0076__en-us_topic_0114776170_p1076010231124">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="24.322432243224327%" id="mcps1.3.8.2.2.4.1.3"><p id="dli_08_0076__en-us_topic_0114776170_p107613231722">Default Value</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_08_0076__en-us_topic_0114776170_row27626232216"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p1376119231129">delimiter</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p18761323022">Data separator</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p4762152314213">Comma (,)</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row1476312231529"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p176222311218">quote</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p1276222311210">Quotation character</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p1076216231220">Double quotation marks (" ")</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row276315231224"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p9763172311210">escape</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p076310238213">Escape character</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p176319231525">Backslash (\)</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row276416231025"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p77631823727">multiLine</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p076422316218">Whether the column data contains carriage return characters or transfer characters. The value <strong id="dli_08_0076__b1819155110198">true</strong> indicates yes and the value <strong id="dli_08_0076__b1119119516196">false</strong> indicates no.</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p15764823129">false</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row576418235217"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p47641123524">dateFormat</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p1176411239212">Date format of the <strong id="dli_08_0076__b1696314910206">date</strong> field in a CSV file</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p17642231023">yyyy-MM-dd</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row976482319220"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p57641623328">timestampFormat</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p57646239215">Date format of the <strong id="dli_08_0076__b103701222202015">timestamp</strong> field in a CSV file</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p47644231022">yyyy-MM-dd HH:mm:ss</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row076592316217"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p2076411231822">mode</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p476402318215">Mode for parsing CSV files. The options are as follows:</p>
<ul id="dli_08_0076__en-us_topic_0114776170_ul1097121216518"><li id="dli_08_0076__en-us_topic_0114776170_li1697117127512"><strong id="dli_08_0076__b943454615479">PERMISSIVE</strong>: Permissive mode. If an incorrect field is encountered, set the line to <strong id="dli_08_0076__b9260105745614">Null</strong>.</li><li id="dli_08_0076__en-us_topic_0114776170_li6971141211513"><strong id="dli_08_0076__b156028114489">DROPMALFORMED</strong>: When an incorrect field is encountered, the entire line is discarded.</li><li id="dli_08_0076__en-us_topic_0114776170_li99721612151"><strong id="dli_08_0076__b57378354811">FAILFAST</strong>: Error mode. If an error occurs, it is automatically reported.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p1476511232219">PERMISSIVE</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row1176592316217"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p676513231624">header</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p1876517231521">Whether CSV contains header information. The value <strong id="dli_08_0076__b20128235182019">true</strong> indicates that the table header information is contained, and the value <strong id="dli_08_0076__b5129123518205">false</strong> indicates that the information is not included.</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p67651723526">false</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row17285185021820"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p4270145031813">nullValue</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p16270125011189">Character that represents the null value. For example, <strong id="dli_08_0076__b4617353220">nullValue= "\\N"</strong> indicates that <strong id="dli_08_0076__b11618259213">\N</strong> represents the null value.</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p17270175031811">-</p>
</td>
</tr>
<tr id="dli_08_0076__en-us_topic_0114776170_row1328411508189"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__en-us_topic_0114776170_p172701750161816">comment</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__en-us_topic_0114776170_p4270185051813">Character that indicates the beginning of the comment. For example, <strong id="dli_08_0076__b10408128523">comment= '#'</strong> indicates that the line starting with <strong id="dli_08_0076__b2408785219">#</strong> is a comment.</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__en-us_topic_0114776170_p174941850141818">-</p>
</td>
</tr>
<tr id="dli_08_0076__row4439162317221"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__p1943919239227">compression</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__p34394235227">Data compression format. Currently, <strong id="dli_08_0076__b1548344015113">gzip</strong>, <strong id="dli_08_0076__b164831440412">bzip2</strong>, and <strong id="dli_08_0076__b04832409118">deflate</strong> are supported. If you do not want to compress data, enter <strong id="dli_08_0076__b1484840919">none</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__p44401923162218">none</p>
</td>
</tr>
<tr id="dli_08_0076__row9827236122319"><td class="cellrowborder" valign="top" width="17.581758175817587%" headers="mcps1.3.8.2.2.4.1.1 "><p id="dli_08_0076__p6827163692311">encoding</p>
</td>
<td class="cellrowborder" valign="top" width="58.095809580958104%" headers="mcps1.3.8.2.2.4.1.2 "><p id="dli_08_0076__p2828163682311">Data encoding format. Available values are <strong id="dli_08_0076__b5109153310215">utf-8</strong>, <strong id="dli_08_0076__b131101933420">gb2312</strong>, and <strong id="dli_08_0076__b1611116339218">gbk</strong>. Value <strong id="dli_08_0076__b10112733625">utf-8</strong> will be used if this parameter is left empty. </p>
</td>
<td class="cellrowborder" valign="top" width="24.322432243224327%" headers="mcps1.3.8.2.2.4.1.3 "><p id="dli_08_0076__p1082883614231">utf-8</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_sa28eef9723f740488b9ea4748e1d7179"><h4 class="sectiontitle">Example</h4><ul id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_ul23691492171721"><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_li36930129171721">Create a <strong id="dli_08_0076__en-us_topic_0093946797_b842352706111929">parquetTable</strong> OBS table.<div class="codecoloring" codetype="Sql" id="dli_08_0076__en-us_topic_0114776170_screen06771955162710"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">parquetTable</span><span class="w"> </span><span class="p">(</span><span class="n">name</span><span class="w"> </span><span class="n">string</span><span class="p">,</span><span class="w"> </span><span class="n">id</span><span class="w"> </span><span class="nb">int</span><span class="p">)</span><span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">parquet</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="ss">&quot;obs://bucketName/filePath&quot;</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="dli_08_0076__li15909031192019">Create a <strong id="dli_08_0076__b1086314371916">parquetZstdTable</strong> OBS table and set the compression format to <strong id="dli_08_0076__b15200164411916">zstd</strong>.<pre class="screen" id="dli_08_0076__screen1458324902010">CREATE TABLE parquetZstdTable (name string, id string) USING parquet OPTIONS (path "obs://bucketName/filePath",compression='zstd');</pre>
</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946797_li11553619171741">Create a <strong id="dli_08_0076__b12597511622">student</strong> table that has two fields <strong id="dli_08_0076__b525915511828">name</strong> and <strong id="dli_08_0076__b10259105111219">score</strong>and partition the table by <strong id="dli_08_0076__b725915513212">classNo</strong>.<div class="codecoloring" codetype="Sql" id="dli_08_0076__en-us_topic_0114776170_screen47551020102819"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span class="n">student</span><span class="p">(</span><span class="n">name</span><span class="w"> </span><span class="n">STRING</span><span class="p">,</span><span class="w"> </span><span class="n">score</span><span class="w"> </span><span class="n">DOUBLE</span><span class="p">,</span><span class="w"> </span><span class="n">classNo</span><span class="w"> </span><span class="nb">INT</span><span class="p">)</span><span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">csv</span><span class="w"> </span><span class="k">OPTIONS</span><span class="w"> </span><span class="p">(</span><span class="n">PATH</span><span class="w"> </span><span class="s1">'obs://bucketName/filePath'</span><span class="p">)</span><span class="w"> </span><span class="n">PARTITIONED</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="p">(</span><span class="n">classNo</span><span class="p">);</span>
</pre></div></td></tr></table></div>
</div>
<div class="note" id="dli_08_0076__note8898433121114"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_08_0076__p690033371119">The <strong id="dli_08_0076__b6923114271712">classNo</strong> field is a partition field and must be placed at the end of the table field, that is, <strong id="dli_08_0076__b2021565261716">student(name STRING, score DOUBLE, classNo INT)</strong>.</p>
</div></div>
</li><li id="dli_08_0076__en-us_topic_0114776170_en-us_topic_0093946806_li5175980417415">To create table <strong id="dli_08_0076__b4851253029">t1</strong> and insert data of table <strong id="dli_08_0076__b085175311210">t2</strong> into table <strong id="dli_08_0076__b38521253920">t1</strong>, run the following statement:<div class="codecoloring" codetype="Sql" id="dli_08_0076__en-us_topic_0114776170_screen1007537285"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">t1</span><span class="w"> </span><span class="k">USING</span><span class="w"> </span><span class="n">parquet</span><span class="w"> </span><span class="k">OPTIONS</span><span class="p">(</span><span class="n">path</span><span class="w"> </span><span class="s1">'obs://bucketName/tblPath'</span><span class="p">)</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="k">select</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="n">t2</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0223.html">Creating an OBS Table</a></div>
</div>
</div>
View Git Blame Copy Permalink