doc-exports/docs/mrs/component-operation-guide/mrs_01_0978.html

<a name="mrs_01_0978"></a><a name="mrs_01_0978"></a>

<h1 class="topictitle1">Creating Table Partitions</h1>
<div id="body1590395285866"><div class="section" id="mrs_01_0978__s6f3b0ae5e79f4012bfdecd42e3888ad3"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0978__en-us_topic_0116526952_p33761356">During the Select query, Hive generally scans the entire table, which is time-consuming. To improve query efficiency, create table partitions based on service requirements and query dimensions.</p>
</div>
<div class="section" id="mrs_01_0978__s5123f1749faf4f1884fd2637a7e79e9d"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0978__ol67755492118"><li id="mrs_01_0978__li4153142615134"><span>For versions earlier than MRS 3.x:</span><p><p id="mrs_01_0978__p19982749101314">Log in to the MRS console. In the left navigation pane, choose <strong id="mrs_01_0978__b70516426953119">Clusters</strong> &gt; <strong id="mrs_01_0978__b150063815153119">Active Clusters</strong>, and click a cluster name. Choose <strong id="mrs_01_0978__b347431405019">Nodes</strong> &gt; <strong id="mrs_01_0978__b1498117389512">Node</strong>. The ECS page is displayed. Click <strong id="mrs_01_0978__b211098822753119">Remote Login</strong> to log in to the Hive node.</p>
<p id="mrs_01_0978__p16511123201412">For MRS 3.<em id="mrs_01_0978__i715645912376">x</em> or later:</p>
<p id="mrs_01_0978__p498213493133">Log in to the node where the Hive client has been installed as user <strong id="mrs_01_0978__b61542023103816">root</strong>.</p>
</p></li><li id="mrs_01_0978__li1877512412216"><span>Run the following command to go to the client installation directory, for example, <strong id="mrs_01_0978__b232414713717">/opt</strong><strong id="mrs_01_0978__b12324373716"></strong><strong id="mrs_01_0978__b153241574712">/client</strong>.</span><p><p class="litext" id="mrs_01_0978__p206251581217"><strong id="mrs_01_0978__en-us_topic_0116526952_b10523083">cd /opt/client</strong></p>
</p></li><li id="mrs_01_0978__li1577584162114"><span>Run the <strong id="mrs_01_0978__b19781449175211">source bigdata_env</strong> command to configure environment variables for the client.</span></li><li id="mrs_01_0978__li9775174142120"><span>Run the following command on the client for login:</span><p><p class="litext" id="mrs_01_0978__p1531231042117"><strong id="mrs_01_0978__a4952e7a738294f70b6f0d16a622aa4d3">kinit</strong> <em id="mrs_01_0978__abf4cdc2ed34c4e39b18842817e571752">Username</em></p>
</p></li><li id="mrs_01_0978__li1477518472118"><span>Run the following command to log in to the client tool:</span><p><p class="litext" id="mrs_01_0978__p5110181211213"><strong id="mrs_01_0978__en-us_topic_0116526952_b48832851">beeline</strong></p>
</p></li><li id="mrs_01_0978__li207768492120"><span>Select the static or dynamic partition.</span><p><ul id="mrs_01_0978__en-us_topic_0116526952_ul31450711"><li id="mrs_01_0978__en-us_topic_0116526952_li14620943">Static partition:<p id="mrs_01_0978__en-us_topic_0116526952_p64479624"><a name="mrs_01_0978__en-us_topic_0116526952_li14620943"></a><a name="en-us_topic_0116526952_li14620943"></a>Manually enter a partition name, and use the keyword <strong id="mrs_01_0978__b51419502533">PARTITIONED BY</strong> to specify partition column name and data type when creating a table. During application development, use the <strong id="mrs_01_0978__b1638164512548">ALTER TABLE ADD PARTITION</strong> statement to add a partition and use the <strong id="mrs_01_0978__b15890115115412">LOAD DATA INTO PARTITION</strong> statement to load data to the partition, which supports only static partitions.</p>
</li><li id="mrs_01_0978__en-us_topic_0116526952_li25747420">Dynamic partition: Use a query command to insert results to a partition of a table. The partition can be a dynamic partition.<p id="mrs_01_0978__en-us_topic_0116526952_p30400195"><a name="mrs_01_0978__en-us_topic_0116526952_li25747420"></a><a name="en-us_topic_0116526952_li25747420"></a>The dynamic partition can be enabled on the client tool by running the following command:</p>
<p id="mrs_01_0978__p11694950171614"><strong id="mrs_01_0978__b753426161711">set hive.exec.dynamic.partition=true</strong><strong id="mrs_01_0978__b77229510115">;</strong></p>
<p id="mrs_01_0978__en-us_topic_0116526952_p46496694">The default mode of the dynamic partition is strict. That is, at least a column must be specified as a static partition, under which dynamic sub-partitions can be created. You can run the following command to enable a completely dynamic partition:</p>
<p id="mrs_01_0978__p14171917171719"><strong id="mrs_01_0978__b16452142371716">set hive.exec.dynamic.partition.mode=nonstrict</strong><strong id="mrs_01_0978__b162313916114">;</strong></p>
</li></ul>
<div class="note" id="mrs_01_0978__en-us_topic_0116526952_note8135907"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_0978__ul14713143051711"><li class="text" id="mrs_01_0978__li271363031719">The dynamic partition may cause a DML statement to create a large number of partitions and new mapping folders, which deteriorates system performance.</li><li id="mrs_01_0978__li1371314302172">If there are a large number of files, it takes a long time to run a SQL statement. You can run the <span class="parmname" id="mrs_01_0978__pd162c1f0ab4545d5b74a7b9f916436b0"><b>set mapreduce.input.fileinputformat.list-status.num-threads = 100;</b></span> statement before running a SQL statement to shorten the time. The parameter <span class="parmname" id="mrs_01_0978__p3050bfb0b39d4afab7d1b21de2fbe1fb"><b>mapreduce.input.fileinputformat.list-status.num-threads</b></span> can be set only after being added to the Hive whitelist.</li></ul>
</div></div>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0977.html">Hive Performance Tuning</a></div>
</div>
</div>