forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
126 lines
15 KiB
HTML
126 lines
15 KiB
HTML
<a name="mrs_01_1449"></a><a name="mrs_01_1449"></a>
|
|
|
|
<h1 class="topictitle1">SET/RESET</h1>
|
|
<div id="body1595920215199"><div class="section" id="mrs_01_1449__s50a79a00d0464249907123eb396c627d"><h4 class="sectiontitle">Function</h4><p id="mrs_01_1449__abcdff905c48a4d1ba817c07c37eb0594">This command is used to dynamically add, update, display, or reset the CarbonData properties without restarting the driver.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1449__s9b445ccf2739404e9d1a2c450c384942"><h4 class="sectiontitle">Syntax</h4><ul id="mrs_01_1449__u2ec7ae48b172458bba5e89058447b65f"><li id="mrs_01_1449__l20f3887ef7d54aa5ae9e3a3661035bb2">Add or Update parameter value:<p id="mrs_01_1449__a0d0b28fb0c0c47f2b890f811a14e50df"><a name="mrs_01_1449__l20f3887ef7d54aa5ae9e3a3661035bb2"></a><a name="l20f3887ef7d54aa5ae9e3a3661035bb2"></a><b><span class="cmdname" id="mrs_01_1449__cmdname1591111815208">SET</span></b> <em id="mrs_01_1449__i635693810208">parameter_name</em>=<em id="mrs_01_1449__i18357103842015">parameter_value</em></p>
|
|
<p id="mrs_01_1449__aabda752be0034bb39a17b7881220df39">This command is used to add or update the value of <span class="parmname" id="mrs_01_1449__pce42f0de34c3428a9d4afe1054f2af40"><b>parameter_name</b></span>.</p>
|
|
</li><li id="mrs_01_1449__l865e38fbb6c44488be77115c15e14dd3">Display property value:<p id="mrs_01_1449__a5a26a713773d4516b8459566824f8471"><a name="mrs_01_1449__l865e38fbb6c44488be77115c15e14dd3"></a><a name="l865e38fbb6c44488be77115c15e14dd3"></a><b><span class="cmdname" id="mrs_01_1449__cmdname15406142211206">SET</span></b> <em id="mrs_01_1449__i134418449207">parameter_name</em></p>
|
|
<p id="mrs_01_1449__a4d59a4074c2249539048f9e27b29d09a">This command is used to display the value of <span class="parmname" id="mrs_01_1449__p58306084187846ecb00c1f37bc170d14"><b>parameter_name</b></span>.</p>
|
|
</li><li id="mrs_01_1449__lb4eedc6845604429a935466adb2fa27d">Display session parameter:<p id="mrs_01_1449__a4791c317a73a4fc7ac8ce79d19d3d764"><a name="mrs_01_1449__lb4eedc6845604429a935466adb2fa27d"></a><a name="lb4eedc6845604429a935466adb2fa27d"></a><b><span class="cmdname" id="mrs_01_1449__cmdname132228287208">SET</span></b></p>
|
|
<p id="mrs_01_1449__a073f7cd8dd9d4ff987c342b0418aadb6">This command is used to display all supported session parameters.</p>
|
|
</li><li id="mrs_01_1449__lbfd00437a4194245862d6acec4d66d05">Display session parameters along with usage details:<p id="mrs_01_1449__a26af39c6446b4f08aa394cd67feb3456"><a name="mrs_01_1449__lbfd00437a4194245862d6acec4d66d05"></a><a name="lbfd00437a4194245862d6acec4d66d05"></a><b><span class="cmdname" id="mrs_01_1449__cmdname3359134982014">SET</span></b> -v</p>
|
|
<p id="mrs_01_1449__a629308ec728349e8b3606dd9149e0eb3">This command is used to display all supported session parameters and their usage details.</p>
|
|
</li><li id="mrs_01_1449__l9ff5ee54ac83483c9d20dc3e5057e45a">Reset parameter value:<p id="mrs_01_1449__a308855b1ecd44412b6e0bf9bcb6e1bb1"><a name="mrs_01_1449__l9ff5ee54ac83483c9d20dc3e5057e45a"></a><a name="l9ff5ee54ac83483c9d20dc3e5057e45a"></a><b><span class="cmdname" id="mrs_01_1449__cmdname3577852172020">RESET</span></b></p>
|
|
<p id="mrs_01_1449__aae371db389174e7d80a91bb7a823afc1">This command is used to clear all session parameters.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1449__s874354d5c2a643c8a61ff676853754ef"><h4 class="sectiontitle">Parameter Description</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1449__t24d72ee171974750bc56763d1b4994a5" frame="border" border="1" rules="all"><caption><b>Table 1 </b>SET parameters</caption><thead align="left"><tr id="mrs_01_1449__r150e9b8c930f47638dfd7ab7afed270a"><th align="left" class="cellrowborder" valign="top" width="19.17%" id="mcps1.3.3.2.2.3.1.1"><p id="mrs_01_1449__a48feb400d1724c45a4759914f57d2957">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="80.83%" id="mcps1.3.3.2.2.3.1.2"><p id="mrs_01_1449__ae50041b6ac77460e8b78b145e0dff052">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1449__rfa4122e4f5134925af32bf60b6394a53"><td class="cellrowborder" valign="top" width="19.17%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_1449__a275c70eab8094d72ab6f7bb2a70c1568">parameter_name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="80.83%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_1449__a3dbd111d045d41fb82801b02e1546e1d">Name of the parameter whose value needs to be dynamically added, updated, or displayed</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__r69351225935747b7bdc190f228e8d95c"><td class="cellrowborder" valign="top" width="19.17%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_1449__aee5eaf277eb743bfb0baa0e9b8f3b957">parameter_value</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="80.83%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_1449__a10b8717fda9746ef8f5250e2d0b806fd">New value of <span class="parmname" id="mrs_01_1449__p822304b1443746c09165f377151414dc"><b>parameter_name</b></span> to be set</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1449__s6b7a50f1640f4c589ebb1b63db120f79"><h4 class="sectiontitle">Precautions</h4><p id="mrs_01_1449__a94c9d5eb89fe497192bb75f4022dcaa3">The following table lists the properties which you can set or clear using the SET or RESET command.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1449__t5f527eeacdb14c8b80cb886d6455dbb2" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Properties</caption><thead align="left"><tr id="mrs_01_1449__r0083b193af5b422788ee292d7c8940f3"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.3.2.3.1.1"><p id="mrs_01_1449__a43cd6d0ad30e43fea40bee82c301e047">Property</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.3.2.3.1.2"><p id="mrs_01_1449__ad04c8cc0a4d944e3b32242aad6cbf5e8">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1449__r3caf4a439c25496a921a6f356c03a774"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__a1060bf80618e45439f569794bc2a4fab">carbon.options.bad.records.logger.enable</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a7f98d77ec4164767a342fec2b20c23eb">Whether to enable bad record logger.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__rc182a826cf074195aec37efe577153dd"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__a1e883799c9a14764bd316bdc0d210c39">carbon.options.bad.records.action</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a399597ba124149d3bbb9f0692cc9af00">Operations on bad records, for example, force, redirect, fail, or ignore. For more information, see <a href="mrs_01_1438.html#mrs_01_1438__lcf623574402c443e908646591898c2be">•Bad record handling</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__re82042d924364770b71e4949a06ea6ad"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__a86ebfc82a1084bd8bcc5692b5474023b">carbon.options.is.empty.data.bad.record</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a3f1acfea550644fbbe8ffbaece42b2d7">Whether the empty data is considered as a bad record. For more information, see <a href="mrs_01_1438.html#mrs_01_1438__lcf623574402c443e908646591898c2be">Bad record handling</a>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__r452b376a1a8648098f4dffccf53c82bd"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__ad475b4ad3002477680b38d3f0cba579a">carbon.options.sort.scope</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a2a332e2d3ab2476b9df82211f95cf3f9">Scope of the sort during data loading.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__r34262402716140a99c0bf3d1de66bdd5"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__a1b6bd244db0e409680cfe2ced2797ad0">carbon.options.bad.record.path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a27d8c585d9eb4841955c468a31261c3d">HDFS path where bad records are stored.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__re8f235c2fec04140a74f20b052b3fa07"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__ac8d222d50d9f4aab8b14c5f43c326834">carbon.custom.block.distribution</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a83d2c5f20d6a4469a15d010346257a16">Whether to enable Spark or CarbonData block distribution.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__r830c85b1a9c34b82adc0a379f7f5af72"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__a40944c8718be4227b1b50042cd106784">enable.unsafe.sort</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__aa0cee96d0ddf4b6fa0e3ac95ed29ccf4">Whether to use unsafe sort during data loading. Unsafe sort reduces the garbage collection during data loading, thereby achieving better performance.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__r76ed8dc8e0b2402fb27749be6b3c8f19"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__ab10d82af53164bdba486c1407e7bb52d">carbon.si.lookup.partialstring</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__adfe6d12e427542649469d071e87d4830">If this is set to <strong id="mrs_01_1449__b15952430101112">TRUE</strong>, the secondary index uses the starts-with, ends-with, contains, and LIKE partition condition strings.</p>
|
|
<p id="mrs_01_1449__a04517435ed6d4ad9a37c2e0c9e42582a">If this is set to <strong id="mrs_01_1449__b212319359119">FALSE</strong>, the secondary index uses only the starts-with partition condition string.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1449__rf2bc14ecd89b4da086badbc774a65ac7"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.1 "><p id="mrs_01_1449__a69932ef55364429896d1191eedd1c2f1">carbon.input.segments</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.3.2.3.1.2 "><p id="mrs_01_1449__a2acd14579f784dc2a47757de1d4fef9b">Segment ID to be queried. This property allows you to query a specified segment of a specified table. CarbonScan reads data only from the specified segment ID.</p>
|
|
<p id="mrs_01_1449__ae7416c604f5a4700950cc6886ae500b6">Syntax:</p>
|
|
<p id="mrs_01_1449__a37c5626e17fc4f2c89faa9efb69b369a"><span class="parmname" id="mrs_01_1449__pf5562fe8ec1e47ee88eaeab6f2d927e4"><b>carbon.input.segments. <database_name>. <table_name> = < list of segment ids ></b></span></p>
|
|
<p id="mrs_01_1449__a407744236ed84702990133298c62b5e7">If you want to query a specified segment in multi-thread mode, you can use <strong id="mrs_01_1449__b17501656201113">CarbonSession.threadSet</strong> instead of the <strong id="mrs_01_1449__b10551956191115">SET</strong> statement.</p>
|
|
<p id="mrs_01_1449__a9e2a2329cd744cba9c16a66e411ef226">Syntax:</p>
|
|
<p id="mrs_01_1449__a21b3ac7ddc77496baab4f423cbf0753d"><span class="parmname" id="mrs_01_1449__p3062afcf30994c2aa98faac2308b852a"><b>CarbonSession.threadSet ("carbon.input.segments. <database_name>. <table_name>","< list of segment ids >");</b></span></p>
|
|
<div class="note" id="mrs_01_1449__nc3f3620202784310a4e3cdbe49d73831"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1449__a47ad40ee5dcd4f70ae0a869742c36223">You are advised not to set this property in the <strong id="mrs_01_1449__b147148201216">carbon.properties</strong> file because all sessions contain the segment list unless session-level or thread-level overwriting occurs.</p>
|
|
</div></div>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1449__s0e7b22986c084aebb0a4c962c202121f"><h4 class="sectiontitle">Examples</h4><ul id="mrs_01_1449__u48677368522e476b9e079b83e2b5aea8"><li id="mrs_01_1449__l7ef2458d622e45d09386d0958b42249c">Add or Update:<p id="mrs_01_1449__a0227e05c49cc432b95e6f58c83e331f9"><a name="mrs_01_1449__l7ef2458d622e45d09386d0958b42249c"></a><a name="l7ef2458d622e45d09386d0958b42249c"></a><b><span class="cmdname" id="mrs_01_1449__cmdname188467122115">SET</span></b> <em id="mrs_01_1449__i163935146213">enable.unsafe.sort</em>=<em id="mrs_01_1449__i37031952212">true</em></p>
|
|
</li><li id="mrs_01_1449__lf9248b044cd74484a67e768a3131af97">Display property value:<p id="mrs_01_1449__aeeb2e3501b624fac8b7bbfe450aca2c9"><a name="mrs_01_1449__lf9248b044cd74484a67e768a3131af97"></a><a name="lf9248b044cd74484a67e768a3131af97"></a><b><span class="cmdname" id="mrs_01_1449__cmdname16557191212">SET</span></b> <em id="mrs_01_1449__a8e01200e80c74236877bd85819322758">enable.unsafe.sort</em></p>
|
|
</li><li id="mrs_01_1449__l44c8b60f36e046aab17f9be40f152389">Show the segment ID list, segment status, and other required details, and specify the segment list to be read:<p id="mrs_01_1449__a06333f21575c40dcb9ed159e4f4d1db2"><a name="mrs_01_1449__l44c8b60f36e046aab17f9be40f152389"></a><a name="l44c8b60f36e046aab17f9be40f152389"></a><b><span class="cmdname" id="mrs_01_1449__cmdname14261123215211">SHOW SEGMENTS FOR</span></b> <em id="mrs_01_1449__i16160133612112">TABLE carbontable1;</em></p>
|
|
<p id="mrs_01_1449__a9fc5b66469e84dd4b02eb464840a921e"><b><span class="cmdname" id="mrs_01_1449__cmdname0895741132110">SET</span></b> <em id="mrs_01_1449__a74d223d2dd0548529fd75a343c726af0">carbon.input.segments.db.carbontable1 = 1, 3, 9;</em></p>
|
|
</li><li id="mrs_01_1449__lec09114d4f754e61a5bcef79b04ebcd1">Query a specified segment in multi-thread mode:<p id="mrs_01_1449__a834453eb3fc74bcd9164b705d202e776"><a name="mrs_01_1449__lec09114d4f754e61a5bcef79b04ebcd1"></a><a name="lec09114d4f754e61a5bcef79b04ebcd1"></a><b><span class="cmdname" id="mrs_01_1449__cmdname087019525211">CarbonSession.threadSet</span></b> (<em id="mrs_01_1449__i56501758162115">"carbon.input.segments.default.carbon_table_MulTI_THread", "1,3"</em>);</p>
|
|
</li><li id="mrs_01_1449__l5033f3a0ecc0435e895d0b83091a6d06">Use <strong id="mrs_01_1449__b14510532161314">CarbonSession.threadSet</strong> to query segments in a multi-thread environment (Scala code is used as an example):<pre class="screen" id="mrs_01_1449__sba734853a9a74703843cf12e6b1dbc83">def main(args: Array[String]) {
|
|
Future { CarbonSession.threadSet("carbon.input.segments.default.carbon_table_MulTI_THread", "1")
|
|
spark.sql("select count(empno) from carbon_table_MulTI_THread").show()
|
|
}
|
|
}</pre>
|
|
</li><li id="mrs_01_1449__lad81debf083840bd884c6925144866e2">Reset:<p id="mrs_01_1449__a130ecb4ddb7a43c29bfd1f6f9d928d52"><a name="mrs_01_1449__lad81debf083840bd884c6925144866e2"></a><a name="lad81debf083840bd884c6925144866e2"></a><b><span class="cmdname" id="mrs_01_1449__cmdname1053326162212">RESET</span></b></p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1449__s1bb5666182f149baa0f41ec36ebc85fc"><h4 class="sectiontitle">System Response</h4><ul id="mrs_01_1449__u0f27508b68204df7bf2e912db202d649"><li id="mrs_01_1449__lfabc560ae0614945a1ea107cf852d692">Success will be recorded in the driver log.</li><li id="mrs_01_1449__l9834e6956bf24a05a9a5873fb69fc7f4">Failure will be displayed on the UI.</li></ul>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1437.html">DML</a></div>
|
|
</div>
|
|
</div>
|
|
|