forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
87 lines
8.4 KiB
HTML
87 lines
8.4 KiB
HTML
<a name="mrs_01_2362"></a><a name="mrs_01_2362"></a>
|
|
|
|
<h1 class="topictitle1">Using CarbonData for First Query</h1>
|
|
<div id="body1607673180958"><div class="section" id="mrs_01_2362__section11476128132414"><h4 class="sectiontitle">Tool Overview</h4><p id="mrs_01_2362__p984419577189">The first query of CarbonData is slow, which may cause a delay for nodes that have high requirements on real-time performance.</p>
|
|
<p id="mrs_01_2362__p865493862414">The tool provides the following functions:</p>
|
|
<ul id="mrs_01_2362__ul1234819516198"><li id="mrs_01_2362__li03485511918">Preheat the tables that have high requirements on query delay for the first time.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_2362__section1070795194217"><h4 class="sectiontitle">Tool Usage</h4><p id="mrs_01_2362__p1480192924311">Download and install the client. For example, the installation directory is <strong id="mrs_01_2362__b214715475368">/opt/client</strong>. Go to the <strong id="mrs_01_2362__b1962153844715">/opt/client/Spark2x/spark/bin</strong> directory and run <strong id="mrs_01_2362__b162215386472">start-prequery.sh</strong>.</p>
|
|
<p id="mrs_01_2362__p1869611074817">Configure <strong id="mrs_01_2362__b16878132173714">prequeryParams.properties</strong> by referring to <a href="#mrs_01_2362__table1684114449372">Table 1</a>.</p>
|
|
|
|
<div class="tablenoborder"><a name="mrs_01_2362__table1684114449372"></a><a name="table1684114449372"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2362__table1684114449372" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="mrs_01_2362__row58411844193712"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.4.2.4.1.1"><p id="mrs_01_2362__p188411744173710">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.4.2.4.1.2"><p id="mrs_01_2362__p1384144412371">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.2.4.2.4.1.3"><p id="mrs_01_2362__p184115440374">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2362__row168421844163717"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p1884284463716">spark.prequery.period.max.minute</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p138421544113720">Maximum preheating duration, in minutes.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p0842444133718">60</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2362__row0842144418370"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p884214440379">spark.prequery.tables</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p20450101324113">Table name configuration, <em id="mrs_01_2362__i156177351380">database.table:int</em>. The table name supports the wildcard (*). <strong id="mrs_01_2362__b158113718386">int</strong> indicates the duration (unit: day) within which the table is updated before it is preheated.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p98425445374">default.test*:10</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2362__row7842164411371"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p11842194419379">spark.prequery.maxThreads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p9842944153715">Maximum number of concurrent threads during preheating</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p138424443376">50</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2362__row1484294410371"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p7842844123720">spark.prequery.sslEnable</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p16842174483710">The value is <strong id="mrs_01_2362__b16523104616385">true</strong> in security mode and <strong id="mrs_01_2362__b178821250123817">false</strong> in non-security mode.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p2842164418371">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2362__row128426443371"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p984264483715">spark.prequery.driver</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p6842044193713">IP address and port number of JDBCServer. The format is <em id="mrs_01_2362__i1388118505425">IP address:Port number</em>. If multiple servers need to be preheated, enter multiple <em id="mrs_01_2362__i844831711439">IP address:Port number</em> of the servers and separate them with commas (,).</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p484234403712">192.168.0.2:22550</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2362__row15842104417372"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p1384216441378">spark.prequery.sql</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p108423446373">SQL statement for preheating. Different statements are separated by colons (:).</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p16814594312">SELECT COUNT(*) FROM %s;SELECT * FROM %s LIMIT 1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2362__row18511391434"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_2362__p19852143914313">spark.security.url</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_2362__p1585253918435">URL required by JDBC in security mode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_2362__p1485283918432">;saslQop=auth-conf;auth=KERBEROS;principal=spark2x/hadoop.hadoop.com@HADOOP.COM;</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div class="note" id="mrs_01_2362__note97071914516"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_2362__p177071897457">The statement configured in <strong id="mrs_01_2362__b122731043143914">spark.prequery.sql</strong> is executed in each preheated table. The table name is replaced with <strong id="mrs_01_2362__b1892735412393">%s</strong>.</p>
|
|
</div></div>
|
|
<p id="mrs_01_2362__p16736114616418"><strong id="mrs_01_2362__b8365141294018">Script Usage</strong></p>
|
|
<p id="mrs_01_2362__p73209162017">Command format: <strong id="mrs_01_2362__b12590211546">sh </strong><strong id="mrs_01_2362__b1425916219547">start-prequery.sh</strong></p>
|
|
<p id="mrs_01_2362__p134565111056">To run this command, place <strong id="mrs_01_2362__b12576215403">user.keytab</strong> or <strong id="mrs_01_2362__b5856476405">jaas.conf</strong> (either of them) and <strong id="mrs_01_2362__b17977139405">krb5.conf</strong> (mandatory) in the <strong id="mrs_01_2362__b10561121814403">conf</strong> directory.</p>
|
|
<div class="note" id="mrs_01_2362__note998517371588"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_2362__ul1648184013495"><li id="mrs_01_2362__li10481134012498">Currently, this tool supports only Carbon tables.</li><li id="mrs_01_2362__li44811340124915">This tool initializes the Carbon environment and pre-reads table metadata to JDBCServer. Therefore, this tool is more suitable for multi-active instances and static allocation mode.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
<p id="mrs_01_2362__p8060118"></p>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1926.html">Using Spark2x</a></div>
|
|
</div>
|
|
</div>
|
|
|