vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0182.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

98 lines
25 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<a name="EN-US_TOPIC_0000001233883255"></a><a name="EN-US_TOPIC_0000001233883255"></a>
<h1 class="topictitle1">About Parallel Data Import from OBS</h1>
<div id="body8662426"><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_p163116471344">The object storage service (OBS) is an object-based cloud storage service, featuring data storage of high security, proven reliability, and cost-effectiveness. OBS provides large storage capacity for you to store files of any type.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_p731115477347"><span id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_ph20454113013286">GaussDB(DWS)</span>, a data warehouse service, uses OBS as a platform for converting cluster data and external data, satisfying the requirements for secure, reliable, and cost-effective storage.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_p1838219614407">You can import data in TXT, CSV, ORC, CARBONDATA, or JSON format from OBS to <span id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_ph1692512422530">GaussDB(DWS)</span> for query, and can remotely read data from OBS. You are advised to import frequently accessed hot data to <span id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_ph19481014202811">GaussDB(DWS)</span> to facilitate queries and store cold data to OBS for remote read to reduce cost.</p>
<p id="EN-US_TOPIC_0000001233883255__p109692038122818">Currently, data can be imported using either of the following methods:</p>
<ul id="EN-US_TOPIC_0000001233883255__ul10969838142818"><li id="EN-US_TOPIC_0000001233883255__li49691838172818">Method 1: You do not need to create a server. Use the default server to create a foreign table. Data in TXT or CSV format is supported. For details, see <a href="dws_04_0154.html">Importing CSV/TXT Data from the OBS</a>.</li><li id="EN-US_TOPIC_0000001233883255__li39691938142820">Method 2: You need to create a server and use the server to create a foreign table. Data in ORC, CarbonData, TXT, CSV, PARQUET, or JSON format is supported. For details, see <a href="dws_04_0155.html">Importing ORC or CarbonData Data from OBS</a>.</li></ul>
<div class="notice" id="EN-US_TOPIC_0000001233883255__note897083892816"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><ul id="EN-US_TOPIC_0000001233883255__ul16970123832816"><li id="EN-US_TOPIC_0000001233883255__li199705389283">Ensure that no Chinese characters are contained in paths used for importing data to or exporting data from OBS.</li><li id="EN-US_TOPIC_0000001233883255__li89701938162815">Data cannot be imported to or exported from OBS across regions. Ensure that OBS and the DWS cluster are in the same region.</li><li id="EN-US_TOPIC_0000001233883255__li564671113213">To ensure the correctness of data import or export, you need to import or export data from OBS in the same compatibility mode.<p id="EN-US_TOPIC_0000001233883255__p4153128113212"><a name="EN-US_TOPIC_0000001233883255__li564671113213"></a><a name="li564671113213"></a>For example, data imported or exported in MySQL compatibility mode can be exported or imported only in MySQL compatibility mode.</p>
</li></ul>
</div></div>
<div class="section" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_s05f14be7edad4e9fa097a082f7d0fe8d"><h4 class="sectiontitle">Overview</h4><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_a2f692b9a99234bf09ece0de452eb06a2">During data migration and Extract-Transform-Load (ETL), a massive volume of data needs to be imported to <span id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_ph1934510403392">GaussDB(DWS)</span> in parallel. The common import mode is time-consuming. When you import data in parallel using OBS foreign tables, source data files to be imported are identified based on the import URL and data formats specified in the tables. Data is imported in parallel through DNs to <span id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_ph14577134510391">GaussDB(DWS)</span>, which improves the overall import performance.</p>
</div>
<div class="p" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p6588494154829"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_en-us_topic_0117407705_aff1ffd0775a14502a3ae833a0549cfcb">Advantages:</strong><ul id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_ufb2f5ed2a5f84e9f9f0a23ffcd92afaa"><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_l8ebd9b26eb994ba9adc5a4a6dc0bb0bd">The CN only plans and delivers data import tasks, and the DNs execute these tasks. This reduces CN resource usage, enabling the CN to process external requests.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_l6cdb958eb7f04433a02f5791f3329826">In this way, the computing capabilities and bandwidths of all the DNs are fully leveraged to import data.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_lfa0fc7ebad5c4fb68d553726a790c84c">You can preprocess data before the import.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_li38783329154120">Fault tolerance can be configured for data format errors during the data import. You can locate incorrect data based on displayed error information after the data is imported.</li></ul>
</div>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p7460316154832"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_en-us_topic_0117407705_a2330fb58f7c548008b2d3565bacb3b03">Disadvantage:</strong></p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a9f7d3d115f5844529d9d1e5512c9e0ec">You need to create OBS foreign tables and store to-be-imported data on OBS.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p25954835154833"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_en-us_topic_0117407705_a76a54deace1f4304a9ba0ae4123ee050">Application Scenario</strong>:</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a1d99452675fa4f4baef6503ddf90b52d">A large volume of local data is imported concurrently on many DNs.</p>
<div class="section" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_s20cfd5ef263b4693bc92afdc62b55073"><h4 class="sectiontitle">Related Concepts</h4><ul id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_u3b90148024b347319c44296cecdc9f18"><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_l144b275bcfef4133b5e2b0ee84749707"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b2094141619533">Source data file</strong>: a TEXT, CSV, ORC, CARBONDATA, or JSON file that stores data to be imported in parallel.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_leb63e100d58547a28da7301f5f4e0d24"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b90224392174923">OBS</strong>: a cloud storage service used to store unstructured data, such as documents, images, and videos. Data is imported in parallel from the OBS server to <span id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_ph212318549404">GaussDB(DWS)</span>.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_l331e0761386349b9947093320b8a7994"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b84235270617505">Bucket</strong>: a container storing objects on OBS.<ul id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_u36bc73d8d2f44c74a97c80e8b83987bd"><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_l55b1bd39af0f4bbfa400d7b50ed795d8">Object storage is a flat storage mode. Layered file system structures are not needed because all objects in buckets are at the same logical layer.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_le04bb26a4407473fac3047e3506e12c0">In OBS, each bucket name must be unique and cannot be changed. A default access control list (ACL) is created with a bucket. Each item in the ACL contains permissions granted to certain users, such as <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b683413494474">READ</strong>, <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b10834114912474">WRITE</strong>, and <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b7834154919478">FULL_CONTROL</strong>. Only authorized users can perform bucket operations, such as creating, deleting, viewing, and setting ACLs for buckets.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_ledfd08dfba934ae79119138b124c1a41">A user can create a maximum of 100 buckets. The total data size and the number of objects and files in each bucket are not limited.</li></ul>
</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_l8061dcd1e4984bd2b5e246447e965074"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b842352706175240">Object</strong>: a basic data storage unit in OBS. Data uploaded by users is stored in OBS buckets as objects. Object attributes include <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b84235270617535">Key</strong>, <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b84235270617539">Metadata</strong>, and <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b842352706175313">Data</strong>.<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_en-us_topic_0102849790_p918400115144">Generally, objects are managed as files. However, OBS has no file system–related concepts, such as files and folders. To let users easily manage data, OBS allows them to simulate folders. Users can add a slash (/) in the object name, for example, <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b84235270614141">tpcds1000/stock.csv</strong>. In this name, <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b84235270614147">tpcds1000</strong> is regarded as the folder name and <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b842352706141424">stock.csv</strong> the file name. The value of <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b842352706141446">key</strong> (object name) is still <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b10726215229">tpcds1000/stock.csv</strong>, and the content of the object is the content of the <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b571662122213">stock.csv</strong> file.</p>
</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_labd5e1beb0f749ecb462f6a5da04ec90"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b3947538172925">Key</strong>: name of an object. It is a UTF-8 character sequence containing 1 to 1024 characters. A key value must be unique in a bucket. Users can name the objects they stored or obtained as <em id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_i811456131216">Bucket name</em>+<em id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_i316856161214">Object name</em>.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_l49ab78452d15405cb44e1c4bf69416b6"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b7303191641310">Metadata</strong>: object metadata, which contains information about the object. There are system metadata and user metadata. The metadata is uploaded to OBS as key-value pairs together with HTTP headers.<ul id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_u46b9049023414ed58f61bf378c30ab07"><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_l8c1eefbf5d9a4d4b9ab3bcc2f6306b3a">System metadata is generated by OBS and used for processing object data. System metadata includes <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b11604764159">Date</strong>, <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b1160476141518">Content-length</strong>, <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b1060417613155">last-modify</strong>, and <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b106211169151">Content-MD5</strong>.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_lfa7d499b7c1342d6b0d3392aaa2ee0c4">User metadata contains object descriptions specified by users for uploading objects.</li></ul>
</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_lc7e1c47f95dd46e08f24859c44d74fe2"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_en-us_topic_0067789101_a69ee85e08bc34bb6ba07ed4ac9a89afa">Data</strong>: object content. OBS does not sense the content and regards it as stateless binary data.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_la21cf1ce9c6248a8aa725083018aad6f"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_en-us_topic_0117407684_a98fe1bb7ae324eca99ee7cce71c7e4cc">Ordinary table</strong>: A database table that stores data imported to data files in parallel. Ordinary tables are classified into row-store tables and column-store tables.</li><li id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407684_lfb4fbedc76974168853131769719dc1d"><strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b143138423181">Foreign table</strong>: A foreign table is used to identify data in a source data file. The foreign table stores information, such as the location, format, encoding, and inter-data delimiter of a source data file.</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_sefc365e1804e4606aafdeb3398080e73"><a name="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_sefc365e1804e4606aafdeb3398080e73"></a><a name="en-us_topic_0000001082926905_en-us_topic_0117407705_sefc365e1804e4606aafdeb3398080e73"></a><h4 class="sectiontitle">How Data Is Imported</h4><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a9b98bf1e132b4cfca94b2755a4788ee0"><a href="#EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_fb5a2ab52cbc942ce89ac96237380de66">Figure 1</a> shows how data is imported from OBS. The CN plans and delivers data import tasks. It delivers tasks to each DN by file.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a089b7ae8b4454210a8c3663a393faa95">The delivery method is as follows:</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a56034ee268a44473ad33b392b1b6c4ab">In <a href="#EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_fb5a2ab52cbc942ce89ac96237380de66">Figure 1</a>, there are four DNs (DN0 to DN3) and OBS stores six files numbered from t1.data.0 to t1.data.5. The files are delivered as follows:</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a420c1db2d7634133a936db2842ec4ac1">t1.data.0 -&gt; DN0</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a32e40f7b928f42acbdfbae5a97e1f366">t1.data.1 -&gt; DN1</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a254b5ec6ab61440685e4eddb0729d185">t1.data.2 -&gt; DN2</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a61ef3f3eea02484795801f49cd710ee0">t1.data.3 -&gt; DN3</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_afd17720b9f7248239b8e0761daca3757">t1.data.4 -&gt; DN0</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a8d0d969f73d44db2a4c7d0b216fd0218">t1.data.5 -&gt; DN1</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_ac274b23cd8d041fcb9cb65a43b0da25e">Two files are delivered to DN0 and DN1, respectively. One file is delivered to each of the other DNs.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a068f7abd4da342ba9f3a623adc506da9">The import performance is the best when one OBS file is delivered to each DN and all the files have the same size. To improve the performance of loading data from OBS, split the data file into multiple files as evenly as possible before storing it to OBS. The recommended number of split files is an integer multiple of the DN quantity.</p>
<div class="fignone" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_fb5a2ab52cbc942ce89ac96237380de66"><a name="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_fb5a2ab52cbc942ce89ac96237380de66"></a><a name="en-us_topic_0000001082926905_en-us_topic_0117407705_fb5a2ab52cbc942ce89ac96237380de66"></a><span class="figcap"><b>Figure 1 </b>Parallel data import using OBS foreign tables</span><br><span><img id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_image12699145812714" src="figure/en-us_image_0000001233761927.jpg"></span></div>
</div>
<div class="section" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_sefac40b4d2db4ab5a27cbdfc51db2e7e"><h4 class="sectiontitle">Import Flowchart</h4><div class="fignone" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_fig10719036144652"><span class="figcap"><b>Figure 2 </b>Parallel import procedure</span><br><span><img class="vsd" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_image62935603144652" src="figure/en-us_image_0000001233563363.png"></span></div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_t78ad0f0768c14d1da3f813541b886af1" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Procedure description</caption><thead align="left"><tr id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_r705785553fa2471a871247c50b4bd03b"><th align="left" class="cellrowborder" valign="top" width="19.99%" id="mcps1.3.15.3.2.4.1.1"><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a6aa349646b344dbfb57897edfb69e35a">Procedure</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="38.97%" id="mcps1.3.15.3.2.4.1.2"><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a00a12da2678142d797167570349347d6">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="41.04%" id="mcps1.3.15.3.2.4.1.3"><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_ac11d3f9545c2409c8b25496687c299f8">Subtask</p>
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_rf4c60154582c49858ca06ddf579a44ff"><td class="cellrowborder" valign="top" width="19.99%" headers="mcps1.3.15.3.2.4.1.1 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_en-us_topic_0102849791_p95460810530">Upload data to OBS.</p>
</td>
<td class="cellrowborder" valign="top" width="38.97%" headers="mcps1.3.15.3.2.4.1.2 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a29cb3e683e9d4bfa8b5d60d3285fad3e">Plan the storage path on the OBS server and upload data files.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p19738057141549">For details, see <a href="dws_04_0184.html">Uploading Data to OBS</a>.</p>
</td>
<td class="cellrowborder" valign="top" width="41.04%" headers="mcps1.3.15.3.2.4.1.3 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a1bea877d547b4ae2ae0b425eee370bd5">-</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_r3a022a0a568740a8bfbfa1964aa40b63"><td class="cellrowborder" valign="top" width="19.99%" headers="mcps1.3.15.3.2.4.1.1 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a24d5d095cf824691b2b8b754280fcde3">Create an OBS foreign table.</p>
</td>
<td class="cellrowborder" valign="top" width="38.97%" headers="mcps1.3.15.3.2.4.1.2 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a6a5cd1397ce5467ca4ac541ed1fa2728">Create a foreign table to identify source data files on the OBS server. The OBS foreign table stores data source information, such as its bucket name, object name, file format, storage location, encoding format, and delimiter.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p44687930144755">For details, see <a href="dws_04_0185.html">Creating an OBS Foreign Table</a>.</p>
</td>
<td class="cellrowborder" valign="top" width="41.04%" headers="mcps1.3.15.3.2.4.1.3 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p52151607153830">-</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_r03155b3773f142c1a14f6720aad4e917"><td class="cellrowborder" valign="top" width="19.99%" headers="mcps1.3.15.3.2.4.1.1 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_aebd5aa3f24684507961ac936ab3ca185">Import data.</p>
</td>
<td class="cellrowborder" valign="top" width="38.97%" headers="mcps1.3.15.3.2.4.1.2 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a96f31b087f56491196a54e2a6dbda956">After creating the foreign table, run the <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0085031681_b842352706163840">INSERT</strong> statement to efficiently import data to the target tables.</p>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p3726745144813">For details, see <a href="dws_04_0186.html">Importing Data</a>.</p>
</td>
<td class="cellrowborder" valign="top" width="41.04%" headers="mcps1.3.15.3.2.4.1.3 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p36637593104945">-</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_r3388f75b1bbe4b7f995e3bf233c10dee"><td class="cellrowborder" valign="top" width="19.99%" headers="mcps1.3.15.3.2.4.1.1 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_ab238a7afd0ae4ea4a60230b767daae83">Handle the table with import errors.</p>
</td>
<td class="cellrowborder" valign="top" width="38.97%" headers="mcps1.3.15.3.2.4.1.2 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a7904192423774ad690853252f72ecc96">If errors occur during data import, handle them based on the displayed error information described in <a href="dws_04_0187.html">Handling Import Errors</a> to ensure data integrity.</p>
</td>
<td class="cellrowborder" valign="top" width="41.04%" headers="mcps1.3.15.3.2.4.1.3 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a85cf1fff84964d35b14cfa7416ab867a">-</p>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_r2db78628cc2f4ae386def94b4d41b451"><td class="cellrowborder" valign="top" width="19.99%" headers="mcps1.3.15.3.2.4.1.1 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_a612e77c7a0e247bf920f558eec44596a">Improve query efficiency.</p>
</td>
<td class="cellrowborder" valign="top" width="38.97%" headers="mcps1.3.15.3.2.4.1.2 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_aaf7fd6d6dfb040ba804e4b9c8b5812d3">After data is imported, run the <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b84235270618957">ANALYZE</strong> statement to generate table statistics. The <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b842352706195230">ANALYZE</strong> statement stores the statistics in the <strong id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001098971120_b842352706195225">PG_STATISTIC</strong> system catalog. When you run the plan generator, the statistics help you generate an efficient query execution plan.</p>
</td>
<td class="cellrowborder" valign="top" width="41.04%" headers="mcps1.3.15.3.2.4.1.3 "><p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_p66944847144927">-</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<p id="EN-US_TOPIC_0000001233883255__en-us_topic_0000001082926905_en-us_topic_0117407705_ae74e7faff917453382fc0638491747e6"></p>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0181.html">Importing Data from OBS in Parallel</a></div>
</div>
</div>
View Git Blame Copy Permalink