forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
43 lines
4.6 KiB
HTML
43 lines
4.6 KiB
HTML
<a name="mrs_01_24745"></a><a name="mrs_01_24745"></a>
|
|
|
|
<h1 class="topictitle1">Configuring the Drop Partition Command to Support Batch Deletion</h1>
|
|
<div id="body0000001533145206"><div class="note" id="mrs_01_24745__note361691045716"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_24745__p4616191075718">This section applies only to MRS 3.2.0 or later.</p>
|
|
</div></div>
|
|
<div class="section" id="mrs_01_24745__section10411113113617"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24745__p18750184218255">Currently, the <strong id="mrs_01_24745__b138523012318">Drop Partition</strong> command in Spark supports partition deletion using only the equal sign (=). This configuration allows multiple filter criteria to be used to delete partitions in batches, for example, <strong id="mrs_01_24745__b24871218123314"><</strong>, <strong id="mrs_01_24745__b454832083320"><=</strong>, <strong id="mrs_01_24745__b751818227336">></strong>, <strong id="mrs_01_24745__b540112515330">>=</strong>, <strong id="mrs_01_24745__b3852827163311">!></strong>, and <strong id="mrs_01_24745__b12524301338">!<</strong>.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24745__section84932420365"><h4 class="sectiontitle">Configuration</h4><p id="mrs_01_24745__p18420142117236">Log in to FusionInsight Manager and choose <strong id="mrs_01_24745__b1044515776111117">Cluster</strong>. Click the name of the desired cluster and choose <strong id="mrs_01_24745__b1314044280111117">Services</strong> > <strong id="mrs_01_24745__b903444399111117">Spark2x</strong>. On the page that is displayed, click the <strong id="mrs_01_24745__b460023519111117">Configurations</strong> tab then <strong id="mrs_01_24745__b599028096111117">All Configurations</strong>, and search for the following parameters.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24745__t18af5fbd3aeb4bda8d13de81ed31f812" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24745__r74395b638f4442e3aab246ff89b4f414"><th align="left" class="cellrowborder" valign="top" width="32.06%" id="mcps1.3.3.3.1.4.1.1"><p id="mrs_01_24745__a1f516e63d9e642069c06eb1b4f809563"><strong id="mrs_01_24745__a744ab93a07754850a5f6c727a33e62e1">Parameter</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="49.57%" id="mcps1.3.3.3.1.4.1.2"><p id="mrs_01_24745__af9566f38e89e419f83b61605b532f7ad"><strong id="mrs_01_24745__a6645dddca1604ff3a7c65c302142e2da">Description</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="18.37%" id="mcps1.3.3.3.1.4.1.3"><p id="mrs_01_24745__ac817ca09e627493cb99f5b57d2943bd6"><strong id="mrs_01_24745__adff60dc7bdc24a8aaa6c1b5c6630331e">Default Value</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24745__rd52556c4569841b3a5e5a2ebdfcb3b56"><td class="cellrowborder" valign="top" width="32.06%" headers="mcps1.3.3.3.1.4.1.1 "><p id="mrs_01_24745__p19122114565816">spark.sql.dropPartitionsInBatch.enabled</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.57%" headers="mcps1.3.3.3.1.4.1.2 "><p id="mrs_01_24745__p1560611114501">If this parameter is set to <strong id="mrs_01_24745__b20861429374">true</strong>, the <strong id="mrs_01_24745__b17836196133718">Drop Partition</strong> command supports the following filter criteria: <strong id="mrs_01_24745__b125173810375"><</strong>, <strong id="mrs_01_24745__b1951238123711"><=</strong>, <strong id="mrs_01_24745__b5511438133712">></strong>, <strong id="mrs_01_24745__b9511638143712">>=</strong>, <strong id="mrs_01_24745__b05113813371">!></strong>, and <strong id="mrs_01_24745__b651173810377">!<</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="18.37%" headers="mcps1.3.3.3.1.4.1.3 "><p id="mrs_01_24745__p3122194511585">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24745__row0274115415112"><td class="cellrowborder" valign="top" width="32.06%" headers="mcps1.3.3.3.1.4.1.1 "><p id="mrs_01_24745__p1440020175224">spark.sql.dropPartitionsInBatch.limit</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="49.57%" headers="mcps1.3.3.3.1.4.1.2 "><p id="mrs_01_24745__p12452172720222">Indicates the maximum number of partitions that can be batch dropped.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="18.37%" headers="mcps1.3.3.3.1.4.1.3 "><p id="mrs_01_24745__p1227485412511">1000</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1941.html">Scenario-Specific Configuration</a></div>
|
|
</div>
|
|
</div>
|
|
|