vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0212.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

155 lines
28 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001188642198"></a><a name="EN-US_TOPIC_0000001188642198"></a>
<h1 class="topictitle1">Preparing Data in an MRS Cluster</h1>
<div id="body8662426"><p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p20390204373512">Before importing data from MRS to a <span id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_ph1692512422530">GaussDB(DWS)</span> cluster, you must have:</p>
<ol id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_ol382575493513"><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li482575483516">Created an MRS cluster.</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li6825754183517">Created a Hive/Spark ORC table in the MRS cluster and stored the table data to the HDFS path corresponding to the table.</li></ol>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p48195853145921">If you have completed the preparations, skip this section.</p>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p166216918494">In this tutorial, the Hive ORC table will be created in the MRS cluster as an example to complete the preparation work. The process and the SQL syntax for creating a Spark ORC table in the MRS cluster are similar to those in Hive.</p>
<div class="section" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section55166005141018"><a name="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section55166005141018"></a><a name="en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section55166005141018"></a><h4 class="sectiontitle">Data File</h4><p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p7523163718476">The sample data of the <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b84235270617837">product_info.txt</strong> data file is as follows:</p>
<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen1650215324201">100,XHDK-A-1293-#fJ3,2017-09-01,A,2017 Autumn New Shirt Women,red,M,328,2017-09-04,715,good
205,KDKE-B-9947-#kL5,2017-09-01,A,2017 Autumn New Knitwear Women,pink,L,584,2017-09-05,406,very good!
300,JODL-X-1937-#pV7,2017-09-01,A,2017 autumn new T-shirt men,red,XL,1245,2017-09-03,502,Bad.
310,QQPX-R-3956-#aD8,2017-09-02,B,2017 autumn new jacket women,red,L,411,2017-09-05,436,It's really super nice
150,ABEF-C-1820-#mC6,2017-09-03,B,2017 Autumn New Jeans Women,blue,M,1223,2017-09-06,1200,The seller's packaging is exquisite
200,BCQP-E-2365-#qE4,2017-09-04,B,2017 autumn new casual pants men,black,L,997,2017-09-10,301,The clothes are of good quality.
250,EABE-D-1476-#oB1,2017-09-10,A,2017 autumn new dress women,black,S,841,2017-09-15,299,Follow the store for a long time.
108,CDXK-F-1527-#pL2,2017-09-11,A,2017 autumn new dress women,red,M,85,2017-09-14,22,It's really amazing to buy
450,MMCE-H-4728-#nP9,2017-09-11,A,2017 autumn new jacket women,white,M,114,2017-09-14,22,Open the package and the clothes have no odor
260,OCDA-G-2817-#bD3,2017-09-12,B,2017 autumn new woolen coat women,red,L,2004,2017-09-15,826,Very favorite clothes
980,ZKDS-J-5490-#cW4,2017-09-13,B,2017 Autumn New Women's Cotton Clothing,red,M,112,2017-09-16,219,The clothes are small
98,FKQB-I-2564-#dA5,2017-09-15,B,2017 autumn new shoes men,green,M,4345,2017-09-18,5473,The clothes are thick and it's better this winter.
150,DMQY-K-6579-#eS6,2017-09-21,A,2017 autumn new underwear men,yellow,37,2840,2017-09-25,5831,This price is very cost effective
200,GKLW-l-2897-#wQ7,2017-09-22,A,2017 Autumn New Jeans Men,blue,39,5879,2017-09-25,7200,The clothes are very comfortable to wear
300,HWEC-L-2531-#xP8,2017-09-23,A,2017 autumn new shoes women,brown,M,403,2017-09-26,607,good
100,IQPD-M-3214-#yQ1,2017-09-24,B,2017 Autumn New Wide Leg Pants Women,black,M,3045,2017-09-27,5021,very good.
350,LPEC-N-4572-#zX2,2017-09-25,B,2017 Autumn New Underwear Women,red,M,239,2017-09-28,407,The seller's service is very good
110,NQAB-O-3768-#sM3,2017-09-26,B,2017 autumn new underwear women,red,S,6089,2017-09-29,7021,The color is very good
210,HWNB-P-7879-#tN4,2017-09-27,B,2017 autumn new underwear women,red,L,3201,2017-09-30,4059,I like it very much and the quality is good.
230,JKHU-Q-8865-#uO5,2017-09-29,C,2017 Autumn New Clothes with Chiffon Shirt,black,M,2056,2017-10-02,3842,very good</pre>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p4707194216174"></p>
</div>
<div class="section" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section17710193111494"><h4 class="sectiontitle">Creating a Hive ORC Table in an MRS Cluster</h4><ol id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_ol596585324916"><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li610292175116">Create an MRS cluster.<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_p104057179327"><a name="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li610292175116"></a><a name="en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li610292175116"></a><span id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_ph8407191783217">For details, see "Creating a Cluster &gt; Creating a User-Defined Cluster" in the <em id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_i162795469208">MapReduce Service User Guide</em>.</span></p>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_p18305142719321"><span id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_ph113061627103210"></span></p>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_en-us_topic_0000001082830951_p327965243618"><span id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001146360931_en-us_topic_0000001082830951_ph4697659163611"></span></p>
</li><li id="EN-US_TOPIC_0000001188642198__li1053201072811">Download the client.<ol type="a" id="EN-US_TOPIC_0000001188642198__ol1839312372814"><li id="EN-US_TOPIC_0000001188642198__li12393152319283">Go back to the MRS cluster page. Click the cluster name. On the <strong id="EN-US_TOPIC_0000001188642198__b11625104315505">Dashboard</strong> tab page of the cluster details page, click <strong id="EN-US_TOPIC_0000001188642198__b14632114355020">Access Manager</strong>. If a message is displayed indicating that EIP needs to be bound, bind an EIP first.</li><li id="EN-US_TOPIC_0000001188642198__li144140315297">Enter the username <strong id="EN-US_TOPIC_0000001188642198__b1554812477501">admin</strong> and its password for logging in to MRS Manager. The password is the one you entered when creating the MRS cluster.</li><li id="EN-US_TOPIC_0000001188642198__li156511737192818">Choose <strong id="EN-US_TOPIC_0000001188642198__b196944472595">Cluster</strong> &gt; <em id="EN-US_TOPIC_0000001188642198__i65079501593">Name of the desired cluster</em> &gt; <strong id="EN-US_TOPIC_0000001188642198__b12696352205912">Dashboard</strong>. On the page that is displayed, choose <strong id="EN-US_TOPIC_0000001188642198__b155263564596">More</strong> &gt; <strong id="EN-US_TOPIC_0000001188642198__b7917145935911">Download Client</strong>. The <strong id="EN-US_TOPIC_0000001188642198__b10320655154215">Download Cluster Client</strong> dialog box is displayed.<p id="EN-US_TOPIC_0000001188642198__p444911358014"><span><img id="EN-US_TOPIC_0000001188642198__image1255453414314" src="figure/en-us_image_0000001636122557.png"></span></p>
<div class="note" id="EN-US_TOPIC_0000001188642198__note1384959101013"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="EN-US_TOPIC_0000001188642198__p1533815261551">To obtain the client of an earlier version, choose <strong id="EN-US_TOPIC_0000001188642198__b427504362514">Services</strong> &gt; <strong id="EN-US_TOPIC_0000001188642198__b82752436259">Download Client</strong> and set <strong id="EN-US_TOPIC_0000001188642198__b7275174352512">Select Client Type</strong> to <strong id="EN-US_TOPIC_0000001188642198__b1327612438251">Configuration Files Only</strong>.</p>
</div></div>
</li></ol>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li14725131112614"><a name="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li14725131112614"></a><a name="en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li14725131112614"></a>Log in to the Hive client of the MRS cluster.<ol type="a" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_ol5884554142019"><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li9214205214208">Log in to a Master node.<p id="EN-US_TOPIC_0000001188642198__p1539811431379"><a name="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li9214205214208"></a><a name="en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li9214205214208"></a><span id="EN-US_TOPIC_0000001188642198__ph1339944317712">For details, see "Logging in to a cluster &gt; Logging In to an ECS" in the MapReduce Service User Guide.</span></p>
<p id="EN-US_TOPIC_0000001188642198__p5410561074"><span id="EN-US_TOPIC_0000001188642198__ph243569712"></span></p>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_p1638611202389"><span id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_ph88081243386"></span></p>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li141551359102716">Run the following command to switch the user:<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen1829592632812">sudo su - omm</pre>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li5310293617727">Run the following command to go to the client directory:<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen5555184313282">cd /opt/client</pre>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li6464792217731">Run the following command to configure the environment variables:<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen1755017813295">source bigdata_env</pre>
</li><li id="EN-US_TOPIC_0000001188642198__li999519595101">If Kerberos authentication is enabled for the current cluster, run the following command to authenticate the current user. The current user must have the permission to create Hive tables.<p id="EN-US_TOPIC_0000001188642198__p1092563711115"><a name="EN-US_TOPIC_0000001188642198__li999519595101"></a><a name="li999519595101"></a>For details, see section "Creating a Role"<span id="EN-US_TOPIC_0000001188642198__ph1147611499114"> in the <em id="EN-US_TOPIC_0000001188642198__i8874119131415">MapReduce Service User Guide</em></span>.</p>
<p id="EN-US_TOPIC_0000001188642198__p794217911127"></p>
</li><li id="EN-US_TOPIC_0000001188642198__li16178029171">Configure roles with the corresponding permissions.<p id="EN-US_TOPIC_0000001188642198__p57173816177"><a name="EN-US_TOPIC_0000001188642198__li16178029171"></a><a name="li16178029171"></a>For details, see section "Creating a User"<span id="EN-US_TOPIC_0000001188642198__ph164731150151610"> in the <em id="EN-US_TOPIC_0000001188642198__i34728509161">MapReduce Service User Guide</em></span>.</p>
<p id="EN-US_TOPIC_0000001188642198__p2298152161617"></p>
<p id="EN-US_TOPIC_0000001188642198__p1655494961613"></p>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_li1388912510181">Bind roles to users. If the Kerberos authentication is disabled for the current cluster, skip this step.<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_screen10522753236"><strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b46997547217">kinit</strong> <em id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_i615612311">MRS cluster user</em></pre>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_p188507194203">Example: <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b172871881232">kinit hiveuser</strong></p>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li393313508298">Run the following command to start the Hive client:<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen1033628113010">beeline</pre>
</li></ol>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li2965165364910">Create a database demo on Hive.<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p772915035915"><a name="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li2965165364910"></a><a name="en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li2965165364910"></a>Run the following command to create the database demo:</p>
<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen19556195525916">CREATE DATABASE <em id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_i61291258181811">demo</em>;</pre>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li3918155310532">Create table <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b84235270610625">product_info</strong> of the <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b84235270610636">Hive TEXTFILE</strong> type in the database demo and import the <a href="#EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section55166005141018">Data File</a> (<strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b84235270610711">product_info.txt</strong>) to the HDFS path corresponding to the table.<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p207721924191516">Run the following command to switch to the database demo:</p>
<pre class="screen" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen160482212169">USE <em id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_i1845335410180">demo</em>;</pre>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p15840911125717">Run the following command to create table <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b8423527061091">product_info</strong> and define the table fields based on data in the <a href="#EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section55166005141018">Data File</a>.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen5420145345710"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">DROP</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">product_info</span><span class="p">;</span>
<span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">product_info</span><span class="w"> </span>
<span class="p">(</span><span class="w"> </span>
<span class="w"> </span><span class="n">product_price</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">30</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_time</span><span class="w"> </span><span class="nb">date</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_level</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_name</span><span class="w"> </span><span class="nb">varchar</span><span class="p">(</span><span class="mi">200</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_type1</span><span class="w"> </span><span class="nb">varchar</span><span class="p">(</span><span class="mi">20</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_type2</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_monthly_sales_cnt</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_comment_time</span><span class="w"> </span><span class="nb">date</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_comment_num</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_comment_content</span><span class="w"> </span><span class="nb">varchar</span><span class="p">(</span><span class="mi">200</span><span class="p">)</span><span class="w"> </span>
<span class="p">)</span><span class="w"> </span>
<span class="k">row</span><span class="w"> </span><span class="n">format</span><span class="w"> </span><span class="n">delimited</span><span class="w"> </span><span class="n">fields</span><span class="w"> </span><span class="n">terminated</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="s1">','</span><span class="w"> </span>
<span class="n">stored</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">TEXTFILE</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_p2119181543920"><span id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_ph15936220123916">For details about how to import data to an MRS cluster, see "Cluster Operation Guide &gt; Managing Active Clusters &gt; Managing Data Files" in the <em id="EN-US_TOPIC_0000001188642198__i168716518588">MapReduce Service User Guide</em>.</span></p>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li89651653174915">Create a Hive ORC table named <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b842352706101330">product_info_orc</strong> in the database demo.<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p17914350195812">Run the following command to create the Hive ORC table <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b842352706101611">product_info_orc</strong>. The table fields are the same as those of the <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b842352706101618">product_info</strong> table created in the previous step.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen1312214199597"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">DROP</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">product_info_orc</span><span class="p">;</span>
<span class="k">CREATE</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">product_info_orc</span>
<span class="p">(</span><span class="w"> </span>
<span class="w"> </span><span class="n">product_price</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_id</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">30</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_time</span><span class="w"> </span><span class="nb">date</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_level</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_name</span><span class="w"> </span><span class="nb">varchar</span><span class="p">(</span><span class="mi">200</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_type1</span><span class="w"> </span><span class="nb">varchar</span><span class="p">(</span><span class="mi">20</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_type2</span><span class="w"> </span><span class="nb">char</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_monthly_sales_cnt</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_comment_time</span><span class="w"> </span><span class="nb">date</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_comment_num</span><span class="w"> </span><span class="nb">int</span><span class="w"> </span><span class="p">,</span>
<span class="w"> </span><span class="n">product_comment_content</span><span class="w"> </span><span class="nb">varchar</span><span class="p">(</span><span class="mi">200</span><span class="p">)</span><span class="w"> </span>
<span class="p">)</span><span class="w"> </span>
<span class="k">row</span><span class="w"> </span><span class="n">format</span><span class="w"> </span><span class="n">delimited</span><span class="w"> </span><span class="n">fields</span><span class="w"> </span><span class="n">terminated</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="s1">','</span><span class="w"> </span>
<span class="n">stored</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">orc</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_li139651532491">Insert data in the <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b842352706101724">product_info</strong> table to the Hive ORC table <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b842352706101739">product_info_orc</strong>.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen19890124905512"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">INSERT</span><span class="w"> </span><span class="k">INTO</span><span class="w"> </span><span class="n">product_info_orc</span><span class="w"> </span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">product_info</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p1085414111575">Query table <strong id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_b84235270610182">product_info_orc</strong>.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_screen735414118573"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">product_info_orc</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_p18872064582">If data displayed in the <a href="#EN-US_TOPIC_0000001188642198__en-us_topic_0000001082830951_en-us_topic_0109259515_en-us_topic_0101477888_section55166005141018">Data File</a> can be queried, the data has been successfully inserted to the ORC table.</p>
</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0210.html">Importing Data from MRS to a Cluster</a></div>
</div>
</div>
View Git Blame Copy Permalink