forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
315 lines
50 KiB
HTML
315 lines
50 KiB
HTML
<a name="mrs_01_2348"></a><a name="mrs_01_2348"></a>
|
|
|
|
<h1 class="topictitle1">Configuring a Traditional Data Source</h1>
|
|
<div id="body32001227"><div class="section" id="mrs_01_2348__en-us_topic_0000001219351171_section2627184474517"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_2348__en-us_topic_0000001219351171_p2060416220463">This section describes how to add a Hive data source on HSConsole.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p527214713459">Currently, <span id="mrs_01_2348__en-us_topic_0000001219351171_text796213811143">HetuEngine</span> supports data sources of the following traditional data formats: AVRO, TEXT, RCTEXT, Binary, ORC, Parquet, and SequenceFile.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_2348__en-us_topic_0000001219351171_section197211512297"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_2348__en-us_topic_0000001219351171_ul1071215716197"><li id="mrs_01_2348__en-us_topic_0000001219351171_li1712871196">The domain name of the cluster where the data source is located must be different from the <span id="mrs_01_2348__en-us_topic_0000001219351171_text06321706326">HetuEngine</span> cluster domain name.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1371219711915">The cluster where the data source is located and the <span id="mrs_01_2348__en-us_topic_0000001219351171_text15282374329">HetuEngine</span> cluster nodes can communicate with each other.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li117681415182710">A <span id="mrs_01_2348__en-us_topic_0000001219351171_text16432927192716">HetuEngine</span> compute instance has been created.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_2348__en-us_topic_0000001219351171_section9975125193717"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_2348__en-us_topic_0000001219351171_ol19515561382"><li id="mrs_01_2348__en-us_topic_0000001219351171_li4515762384"><a name="mrs_01_2348__en-us_topic_0000001219351171_li4515762384"></a><a name="en-us_topic_0000001219351171_li4515762384"></a><span>Obtain the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b10361826259">hdfs-site.xml</strong> and <strong id="mrs_01_2348__en-us_topic_0000001219351171_b133671214253">core-site.xml</strong> configuration files of the Hive data source cluster.</span><p><ol type="a" id="mrs_01_2348__en-us_topic_0000001219351171_ol1894734085317"><li id="mrs_01_2348__en-us_topic_0000001219351171_li364720417381">Log in to FusionInsight Manager of the cluster where the Hive data source is located.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1594734075312">Choose <strong id="mrs_01_2348__en-us_topic_0000001219351171_b685151015264">Cluster</strong> > <strong id="mrs_01_2348__en-us_topic_0000001219351171_b171181612132610">Dashboard</strong>.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li69471840195319">Choose <strong id="mrs_01_2348__en-us_topic_0000001219351171_b63221113152612">More</strong> > <strong id="mrs_01_2348__en-us_topic_0000001219351171_b732210131268">Download Client</strong> and download the client file to the local computer.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li794744010532">Decompress the downloaded client file package and obtain the <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath18149163382717"><b>core-site.xml</b></span> and <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath16154203320278"><b>hdfs-site.xml</b></span> files in the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b835814392820">FusionInsight_Cluster_1_Services_ClientConfig/HDFS/config</strong> directory.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li199425415100">Check whether the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b114183471619">core-site.xml</strong> file contains the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b19667101192410">fs.trash.interval</strong> configuration item. If not, add the following configuration items:<pre class="screen" id="mrs_01_2348__en-us_topic_0000001219351171_screen12720911969"><property>
|
|
<name>fs.trash.interval</name>
|
|
<value>2880</value>
|
|
</property></pre>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li7947104012532">Change the value of <span class="parmname" id="mrs_01_2348__en-us_topic_0000001219351171_parmname298419441285"><b>dfs.client.failover.proxy.provider.hacluster</b></span> in the <span class="parmname" id="mrs_01_2348__en-us_topic_0000001219351171_parmname4985164462817"><b>hdfs-site.xml</b></span> file to <span class="parmvalue" id="mrs_01_2348__en-us_topic_0000001219351171_parmvalue6986114414287"><b>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</b></span>.<pre class="screen" id="mrs_01_2348__en-us_topic_0000001219351171_screen12915141415212"><property>
|
|
<name>dfs.client.failover.proxy.provider.hacluster</name>
|
|
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
|
|
</property></pre>
|
|
<ul id="mrs_01_2348__en-us_topic_0000001219351171_ul494716404531"><li id="mrs_01_2348__en-us_topic_0000001219351171_li1194744012536">If HDFS has multiple NameServices, change the values of <strong id="mrs_01_2348__en-us_topic_0000001219351171_b13981132202510">dfs.client.failover.proxy.provider.</strong><em id="mrs_01_2348__en-us_topic_0000001219351171_i1198222282517">NameService name</em> for multiple NameServices in the <span class="parmname" id="mrs_01_2348__en-us_topic_0000001219351171_parmname3984162219256"><b>hdfs-site.xml</b></span> file to <span class="parmvalue" id="mrs_01_2348__en-us_topic_0000001219351171_parmvalue4985182210254"><b>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</b></span>.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1994784013530">In addition, if the <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath22551915175218"><b>hdfs-site.xml</b></span> file references the host name of a non-<span id="mrs_01_2348__en-us_topic_0000001219351171_text132561015105213">HetuEngine</span> cluster node, you need to add the mapping between the referenced host name and the corresponding IP address to the <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath725614154523"><b>/etc/hosts</b></span> file of each <span id="mrs_01_2348__en-us_topic_0000001219351171_text925661510529">HetuEngine</span> cluster node. Otherwise, <span id="mrs_01_2348__en-us_topic_0000001219351171_text1825617153527">HetuEngine</span> cannot connect to the node that is not in this cluster based on the host name.</li></ul>
|
|
<div class="notice" id="mrs_01_2348__en-us_topic_0000001219351171_note35907342217"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><p id="mrs_01_2348__en-us_topic_0000001219351171_p859023418218">If the Hive data source to be interconnected is in the same Hadoop cluster with HetuEngine, you can log in to the HDFS client and run the following commands to obtain the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b127632264346">hdfs-site.xml</strong> and <strong id="mrs_01_2348__en-us_topic_0000001219351171_b8411193063413">core-site.xml</strong> configuration files. For details, see <a href="mrs_01_1663.html">Using the HDFS Client</a>.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p939412819510"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b86281215202011">hdfs dfs -get /user/hetuserver/fiber/restcatalog/hive/core-site.xml</strong></p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p17395182813519"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b166291615192019">hdfs dfs -get /user/hetuserver/fiber/restcatalog/hive/hdfs-site.xml</strong></p>
|
|
</div></div>
|
|
</li></ol>
|
|
</p></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li051517619384"><a name="mrs_01_2348__en-us_topic_0000001219351171_li051517619384"></a><a name="en-us_topic_0000001219351171_li051517619384"></a><span>Obtain the <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath13523953302"><b>user.keytab</b></span> and <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath552915517302"><b>krb5.conf</b></span> files of the proxy user of the Hive data source.</span><p><ol type="a" id="mrs_01_2348__en-us_topic_0000001219351171_ol79471840105312"><li id="mrs_01_2348__en-us_topic_0000001219351171_li5961737135520">Log in to FusionInsight Manager of the cluster where the Hive data source is located.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li11141995610">Choose <strong id="mrs_01_2348__en-us_topic_0000001219351171_b2909102812305">System</strong> > <strong id="mrs_01_2348__en-us_topic_0000001219351171_b119152284305">Permission</strong> > <strong id="mrs_01_2348__en-us_topic_0000001219351171_b129151286306">User</strong>.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li17947340185317">Locate the row that contains the target data source user, click <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1441573117302">More</strong> in the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b541612313304">Operation</strong> column, and select <strong id="mrs_01_2348__en-us_topic_0000001219351171_b54162031153012">Download Authentication Credential</strong>.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1694712408531">Decompress the downloaded package to obtain the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1280905233016">user.keytab</strong> and <strong id="mrs_01_2348__en-us_topic_0000001219351171_b11815125218300">krb5.conf</strong> files.<div class="note" id="mrs_01_2348__en-us_topic_0000001219351171_note27045215912"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_2348__en-us_topic_0000001219351171_p1770482894">The proxy user of the Hive data source must be associated with at least the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b398818443269">hive</strong> user group.</p>
|
|
</div></div>
|
|
</li></ol>
|
|
</p></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li05151668382"><a name="mrs_01_2348__en-us_topic_0000001219351171_li05151668382"></a><a name="en-us_topic_0000001219351171_li05151668382"></a><span>Obtain the MetaStore URL and the Principal of the server.</span><p><ol type="a" id="mrs_01_2348__en-us_topic_0000001219351171_ol199471840125311"><li id="mrs_01_2348__en-us_topic_0000001219351171_li1375911411550">Decompress the client package of the cluster where the Hive data source is located and obtain the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b196091702538">hive-site.xml</strong> file from the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b14504053530">FusionInsight_Cluster_1_Services_ClientConfig/Hive/config</strong> directory.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li129471240155310">Open the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b2024635953318">hive-site.xml</strong> file and search for <strong id="mrs_01_2348__en-us_topic_0000001219351171_b0246659163312">hive.metastore.uris</strong>. The value of <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1624645919334">hive.metastore.uris</strong> is the value of MetaStore URL. Search for <strong id="mrs_01_2348__en-us_topic_0000001219351171_b24911015345">hive.server2.authentication.kerberos.principal</strong>. The value of <strong id="mrs_01_2348__en-us_topic_0000001219351171_b64911815345">hive.server2.authentication.kerberos.principal</strong> is the value of Principal on the server.</li></ol>
|
|
</p></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li65151264385"><span>Log in to FusionInsight Manager as a <span id="mrs_01_2348__en-us_topic_0000001219351171_text137347201779">HetuEngine</span> administrator and choose <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1218144173410">Cluster</strong> > <strong id="mrs_01_2348__en-us_topic_0000001219351171_b42181412349">Services</strong> > <strong id="mrs_01_2348__en-us_topic_0000001219351171_b5219194113416"><span id="mrs_01_2348__en-us_topic_0000001219351171_text122554231470">HetuEngine</span></strong>. The <strong id="mrs_01_2348__en-us_topic_0000001219351171_b32198419341"><span id="mrs_01_2348__en-us_topic_0000001219351171_text1601748113417">HetuEngine</span></strong> service page is displayed.</span></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1551513610389"><span>In the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b134851649133412">Basic Information</strong> area on the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b18486104919347">Dashboard</strong> page, click the link next to <strong id="mrs_01_2348__en-us_topic_0000001219351171_b448724903410">HSConsole WebUI</strong>. The HSConsole page is displayed.</span></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1551526173810"><span>Choose <strong id="mrs_01_2348__en-us_topic_0000001219351171_b124318532056">Data Source</strong> and click <strong id="mrs_01_2348__en-us_topic_0000001219351171_b112438531759">Add Data Source</strong>. Configure parameters on the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b15187125615342">Add Data Source</strong> page.</span><p><ol type="a" id="mrs_01_2348__en-us_topic_0000001219351171_ol338114219544"><li id="mrs_01_2348__en-us_topic_0000001219351171_li4381124275419">Configure parameters in the <span class="parmname" id="mrs_01_2348__en-us_topic_0000001219351171_parmname6959174183512"><b>Basic Information</b></span> area. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_table14381114211546">Table 1</a>.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table14381114211546"></a><a name="en-us_topic_0000001219351171_table14381114211546"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table14381114211546" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Basic Information</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row53814422549"><th align="left" class="cellrowborder" valign="top" width="13.14131413141314%" id="mcps1.3.3.2.6.2.1.1.3.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p93811542145412">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="69.51695169516952%" id="mcps1.3.3.2.6.2.1.1.3.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p2381114215411">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="17.341734173417343%" id="mcps1.3.3.2.6.2.1.1.3.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p3381184205417">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row33811542115418"><td class="cellrowborder" valign="top" width="13.14131413141314%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1938118424541">Name</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p2381184225418"></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.51695169516952%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p23816424547">Name of the data source to be connected.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p12436819125917">The value can contain only letters, digits, and underscores (_) and must start with a letter.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.341734173417343%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p12381242155411">hive_1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row153814429540"><td class="cellrowborder" valign="top" width="13.14131413141314%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p53811842175412">Data Source Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.51695169516952%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p18381174216546">Type of the data source to be connected. Select <strong id="mrs_01_2348__en-us_topic_0000001219351171_b8446175803513">Hive</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.341734173417343%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1438114245414">Hive</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row10284759101411"><td class="cellrowborder" valign="top" width="13.14131413141314%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p328512596148">Mode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.51695169516952%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p19285359101410">Mode of the current cluster. The default value is <strong id="mrs_01_2348__en-us_topic_0000001219351171_b128847112019">Security Mode</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.341734173417343%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p928585914141">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row4381174219542"><td class="cellrowborder" valign="top" width="13.14131413141314%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p0381124215415">Description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="69.51695169516952%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p17381194212547">Description of the data source.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p2045733013594">The value can contain only letters, digits, commas (,), periods (.), underscores (_), spaces, and line breaks.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.341734173417343%" headers="mcps1.3.3.2.6.2.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p143811642135414">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li9381134219548">Configure parameters in the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b649161817368">Hive Configuration</strong> area. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_table738124295411">Table 2</a>.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table738124295411"></a><a name="en-us_topic_0000001219351171_table738124295411"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table738124295411" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Hive Configuration</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row9381542125417"><th align="left" class="cellrowborder" valign="top" width="33.29332933293329%" id="mcps1.3.3.2.6.2.1.2.3.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p1638144211541">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.373337333733375%" id="mcps1.3.3.2.6.2.1.2.3.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p23811942125411">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.2.3.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p83814423546">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row438119429541"><td class="cellrowborder" valign="top" width="33.29332933293329%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1338144217541">Driver</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.373337333733375%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p538174214541">The default value is <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1084218338365">fi-hive-hadoop</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p438144215549">fi-hive-hadoop</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row83812421541"><td class="cellrowborder" valign="top" width="33.29332933293329%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p53811642205420">hdfs-site File</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.373337333733375%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1938111422544">Select the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b14206174183617">hdfs-site.xml</strong> configuration file obtained in <a href="#mrs_01_2348__en-us_topic_0000001219351171_li4515762384">1</a>. The file name is fixed.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p63811942195414">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row20381134285411"><td class="cellrowborder" valign="top" width="33.29332933293329%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p15381642105420">core-site File</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.373337333733375%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p14381742185413">Select the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b0463120103718">core-site.xml</strong> configuration file obtained in <a href="#mrs_01_2348__en-us_topic_0000001219351171_li4515762384">1</a>. The file name is fixed.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p16381642135412">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row63811242125413"><td class="cellrowborder" valign="top" width="33.29332933293329%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p438194211546">krb5 File</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.373337333733375%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p10520155531812">Configure this parameter when the security mode is enabled.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p1743719239196">It is the configuration file used for Kerberos authentication. Select the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b5857166193815">krb5.conf</strong> file obtained in <a href="#mrs_01_2348__en-us_topic_0000001219351171_li051517619384">2</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p13381142125410">krb5.conf</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row753164234911"><td class="cellrowborder" valign="top" width="33.29332933293329%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p7531204215494">Enable Data Source Authentication</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.373337333733375%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p87927161177">Whether to use the permission policy of the Hive data source for authentication.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p153113427498">If Ranger is disabled for the <span id="mrs_01_2348__en-us_topic_0000001219351171_text1352893742417">HetuEngine</span> service, select <strong id="mrs_01_2348__en-us_topic_0000001219351171_b14329116225">Yes</strong>. If Ranger is enabled, select <strong id="mrs_01_2348__en-us_topic_0000001219351171_b235314363221">No</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.2.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1353114424498">No</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li2038113425547">Configure parameters in the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1858015279381">MetaStore Configuration</strong> area. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_table18381204214544">Table 3</a>.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table18381204214544"></a><a name="en-us_topic_0000001219351171_table18381204214544"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table18381204214544" frame="border" border="1" rules="all"><caption><b>Table 3 </b>MetaStore Configuration</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row1938144214549"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.3.3.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p4381144275411">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.3033303330333%" id="mcps1.3.3.2.6.2.1.3.3.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p1738117428546">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.36333633363336%" id="mcps1.3.3.2.6.2.1.3.3.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p17381842155414">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row838294210547"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1438294211545">Metastore URL</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.3033303330333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p438212422542">URL of the MetaStore of the data source. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_li05151668382">3</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.36333633363336%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1382184211545">thrift://10.92.8.42:21088,thrift://10.92.8.43:21088,thrift://10.92.8.44:21088</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row15382542205412"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p438211422548">Security Authentication Mechanism</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.3033303330333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1938216425541">After the security mode is enabled, the default value is <strong id="mrs_01_2348__en-us_topic_0000001219351171_b185651653911">KERBEROS</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.36333633363336%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p11382104220548">KERBEROS</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row638294245411"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p2382142125414">Server Principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.3033303330333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1017213479112">Configure this parameter when the security mode is enabled.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p16382134215413">It specifies the username with domain name used by meta to access MetaStore. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_li05151668382">3</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.36333633363336%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1438244285410">hive/hadoop.hadoop.com@HADOOP.COM</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row1038214220543"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p14382114255412">Client Principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.3033303330333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p12781199141618">Configure this parameter when the security mode is enabled.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p1538244210547">The parameter format is as follows: <em id="mrs_01_2348__en-us_topic_0000001219351171_i14523254014">Username for accessing MetaStore</em><strong id="mrs_01_2348__en-us_topic_0000001219351171_b18574217409">@</strong><em id="mrs_01_2348__en-us_topic_0000001219351171_i13577212402">domain name (uppercase)</em><strong id="mrs_01_2348__en-us_topic_0000001219351171_b125712254012">.COM</strong>.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p0420114041218"><em id="mrs_01_2348__en-us_topic_0000001219351171_i1913944431712">Username for accessing MetaStore</em> is the user to which the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b884544741611">user.keytab</strong> file obtained in <a href="#mrs_01_2348__en-us_topic_0000001219351171_li051517619384">2</a> belongs.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.36333633363336%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p13382204225415">admintest@HADOOP.COM</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row638214429546"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1738219425541">Keytab File</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.3033303330333%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p13129131415160">Configure this parameter when the security mode is enabled.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p143826421549">It specifies the keytab credential file of the MetaStore user name. The file name is fixed. Select the <span class="filepath" id="mrs_01_2348__en-us_topic_0000001219351171_filepath920074922510"><b>user.keytab</b></span> file obtained in <a href="#mrs_01_2348__en-us_topic_0000001219351171_li051517619384">2</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.36333633363336%" headers="mcps1.3.3.2.6.2.1.3.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1738214429549">user.keytab</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li18527145012913">Configure parameters in the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b138355331934">Connection Pool Configuration</strong> area. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_table3527185012913">Table 4</a>.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table3527185012913"></a><a name="en-us_topic_0000001219351171_table3527185012913"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table3527185012913" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Connection Pool Configuration</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row165274501296"><th align="left" class="cellrowborder" valign="top" width="21.74217421742174%" id="mcps1.3.3.2.6.2.1.4.3.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p1752755010914">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="55.04550455045505%" id="mcps1.3.3.2.6.2.1.4.3.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p1152715013920">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="23.212321232123212%" id="mcps1.3.3.2.6.2.1.4.3.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p852795013918">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row55287500917"><td class="cellrowborder" valign="top" width="21.74217421742174%" headers="mcps1.3.3.2.6.2.1.4.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p175286509914">Enable Connection Pool</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="55.04550455045505%" headers="mcps1.3.3.2.6.2.1.4.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p052813501192">Whether to enable the connection pool when accessing Hive MetaStore.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.212321232123212%" headers="mcps1.3.3.2.6.2.1.4.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1952817504916">Yes/No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row105281450392"><td class="cellrowborder" valign="top" width="21.74217421742174%" headers="mcps1.3.3.2.6.2.1.4.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p5515554151213">Maximum Connections</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="55.04550455045505%" headers="mcps1.3.3.2.6.2.1.4.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p109571919102715">Maximum number of connections in the connection pool when accessing Hive MetaStore.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.212321232123212%" headers="mcps1.3.3.2.6.2.1.4.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1952817501896">50</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li39081458098">Configure parameters in <strong id="mrs_01_2348__en-us_topic_0000001219351171_b5426192913911">Hive User Information Configuration</strong>. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_table17970173051814">Table 5</a>.<div class="p" id="mrs_01_2348__en-us_topic_0000001219351171_p15856606104"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b199021655192914">Hive User Information Configuration</strong> and <strong id="mrs_01_2348__en-us_topic_0000001219351171_b52361259172920"><span id="mrs_01_2348__en-us_topic_0000001219351171_text172858121710">HetuEngine</span>-Hive User Mapping Configuration</strong> must be used together. When <span id="mrs_01_2348__en-us_topic_0000001219351171_text16685184818101">HetuEngine</span> is connected to the Hive data source, user mapping enables <span id="mrs_01_2348__en-us_topic_0000001219351171_text7321125331014">HetuEngine</span> users to have the same permissions of the mapped Hive data source user. Multiple <span id="mrs_01_2348__en-us_topic_0000001219351171_text64010810121">HetuEngine</span> users can correspond to one Hive user.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table17970173051814"></a><a name="en-us_topic_0000001219351171_table17970173051814"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table17970173051814" frame="border" border="1" rules="all"><caption><b>Table 5 </b>Hive User Information Configuration</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row497211304186"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p39721430151811">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p697283016184">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p4972113012181">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row1497217306188"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p99721130201816">Data Source User</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p5554187152415">Data source user information.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p1297210303182">The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p2574328152814">If the data source user is set to <strong id="mrs_01_2348__en-us_topic_0000001219351171_b391682614506">hiveuser1</strong>, a <span id="mrs_01_2348__en-us_topic_0000001219351171_text19384184512128">HetuEngine</span> user mapped to <strong id="mrs_01_2348__en-us_topic_0000001219351171_b13680205415502">hiveuser1</strong> must exist. For example, create <strong id="mrs_01_2348__en-us_topic_0000001219351171_b13349665418">hetuuser1</strong> and map it to <strong id="mrs_01_2348__en-us_topic_0000001219351171_b167417155413">hiveuser1</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row16972173051811"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p109726307187">Keytab File</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1797223015183">Obtain the authentication credential of the user corresponding to the data source.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.5.3.6.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p10972163015187">hiveuser1.keytab</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li5816121719206">Configure parameters in the <strong id="mrs_01_2348__en-us_topic_0000001219351171_b747613011337">HetuEngine-Hive User Mapping Configuration</strong> area. For details, see <a href="#mrs_01_2348__en-us_topic_0000001219351171_table363354362015">Table 6</a>.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table363354362015"></a><a name="en-us_topic_0000001219351171_table363354362015"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table363354362015" frame="border" border="1" rules="all"><caption><b>Table 6 </b>HetuEngine-Hive User Mapping Configuration</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row166341343162014"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.6.3.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p14634184322012">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.6.3.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p1463454320207">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.6.3.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p9634134311208">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row66341643162011"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.6.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p2634194392018">HetuEngine User</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.6.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p143401133102320"><span id="mrs_01_2348__en-us_topic_0000001219351171_text123869612516">HetuEngine</span> user information.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p20634144319200">The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.6.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p156341643122011">hetuuser1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row263494311203"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.6.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1263484352013">Data Source User</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.6.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1763444392017">Data source user information.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p9450918172411">The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.6.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p116341643102018"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b08651158125512">hiveuser1</strong> (data source user configured in <a href="#mrs_01_2348__en-us_topic_0000001219351171_table17970173051814">Table 5</a>)</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1438274211549"><a name="mrs_01_2348__en-us_topic_0000001219351171_li1438274211549"></a><a name="en-us_topic_0000001219351171_li1438274211549"></a>Modify custom configurations.<ul id="mrs_01_2348__en-us_topic_0000001219351171_ul15382542155413"><li id="mrs_01_2348__en-us_topic_0000001219351171_li1538204211545">You can click <strong id="mrs_01_2348__en-us_topic_0000001219351171_b149842016424">Add</strong> to add custom configuration parameters by referring to <a href="#mrs_01_2348__en-us_topic_0000001219351171_table16627151232810">Table 7</a>.
|
|
<div class="tablenoborder"><a name="mrs_01_2348__en-us_topic_0000001219351171_table16627151232810"></a><a name="en-us_topic_0000001219351171_table16627151232810"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2348__en-us_topic_0000001219351171_table16627151232810" frame="border" border="1" rules="all"><caption><b>Table 7 </b>Custom parameters</caption><thead align="left"><tr id="mrs_01_2348__en-us_topic_0000001219351171_row1462811220280"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.1"><p id="mrs_01_2348__en-us_topic_0000001219351171_p062810128285">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="40.244024402440246%" id="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.2"><p id="mrs_01_2348__en-us_topic_0000001219351171_p14628151215287">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="26.422642264226422%" id="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.3"><p id="mrs_01_2348__en-us_topic_0000001219351171_p86281412162813">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2348__en-us_topic_0000001219351171_row20437131111366"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1650016173361">hive.metastore.connection.pool.maxTotal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.244024402440246%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p175001717113612">Maximum number of connections in the connection pool.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.422642264226422%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p125001317103619">50 (Value range: 0-200)</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row86281212102812"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p1392112810305">hive.metastore.connection.pool.maxIdle</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.244024402440246%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p99211581301">Maximum number of idle threads in the connection pool. When the number of idle threads reaches the maximum number, new threads are not released.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p16921682308">Default value: <strong id="mrs_01_2348__en-us_topic_0000001219351171_b213015248924054">10</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.422642264226422%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p692118817302">10 (The value ranges from 0 to 200 and cannot exceed the maximum number of connections.)</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2348__en-us_topic_0000001219351171_row12628171215282"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.1 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p49215853018">hive.metastore.connection.pool.minIdle</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.244024402440246%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.2 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p99215893018">Minimum number of idle threads in the connection pool. When the number of idle threads reaches the minimum number, the thread pool does not create new threads.</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p1992115863016">Default value: <strong id="mrs_01_2348__en-us_topic_0000001219351171_b143954098524054">10</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="26.422642264226422%" headers="mcps1.3.3.2.6.2.1.7.1.1.3.2.4.1.3 "><p id="mrs_01_2348__en-us_topic_0000001219351171_p20921118183020">10 (The value ranges from 0 to 200 and cannot exceed the value of <strong id="mrs_01_2348__en-us_topic_0000001219351171_b16306132622712">hive.metastore.connection.pool.maxIdle</strong>.)</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li113821842145410">You can click <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1365343434210">Delete</strong> to delete custom configuration parameters.<div class="note" id="mrs_01_2348__en-us_topic_0000001219351171_note1984424815237"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_2348__en-us_topic_0000001219351171_ul958134293110"><li id="mrs_01_2348__en-us_topic_0000001219351171_li858118424312">You can add prefixes <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1680171512416">coordinator.</strong> and <strong id="mrs_01_2348__en-us_topic_0000001219351171_b8690915444">worker.</strong> to the preceding custom configuration items to configure coordinators and workers, respectively. For example, if <strong id="mrs_01_2348__en-us_topic_0000001219351171_b86431123104013">worker.hive.metastore.connection.pool.maxTotal</strong> is set to <strong id="mrs_01_2348__en-us_topic_0000001219351171_b03691353164319">50</strong>, a maximum number of 50 connections are allowed for workers to access Hive MetaStore. If no prefix is added, the configuration item is valid for both coordinators and workers.</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li1111412459310">By default, the maximum number of connections for coordinators to access Hive MetaStore is 5 and the maximum and minimum numbers of idle data source connections are both 10. The maximum number of connections for workers to access Hive MetaStore is 20, the maximum and minimum numbers of idle data source connections are both 0.</li></ul>
|
|
</div></div>
|
|
</li></ul>
|
|
</li><li id="mrs_01_2348__en-us_topic_0000001219351171_li738234215549">Click <strong id="mrs_01_2348__en-us_topic_0000001219351171_b51481537204215">OK</strong>.</li></ol>
|
|
</p></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li203291026162813"><span>Log in to the node where the cluster client is located and run the following commands to switch to the client installation directory and authenticate the user:</span><p><p id="mrs_01_2348__en-us_topic_0000001219351171_p28761056204315"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b687615694311">cd /opt/client</strong></p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p16876175617437"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b12876195684320">source bigdata_env</strong></p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p1087323193213"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b487152303215">kinit</strong><em id="mrs_01_2348__en-us_topic_0000001219351171_i168719236321"> User performing <span id="mrs_01_2348__en-us_topic_0000001219351171_text10875232322">HetuEngine</span> operations</em> (If the cluster is in normal mode, skip this step.)</p>
|
|
</p></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li125901127545"><span>Run the following command to log in to the catalog of the data source:</span><p><p id="mrs_01_2348__en-us_topic_0000001219351171_p13590121255419"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b1659012128543">hetu-cli --catalog </strong><em id="mrs_01_2348__en-us_topic_0000001219351171_i1590171265414">Data source name</em> <strong id="mrs_01_2348__en-us_topic_0000001219351171_b1359011126546">--schema default</strong></p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p7590412135413">For example, run the following command:</p>
|
|
<p id="mrs_01_2348__en-us_topic_0000001219351171_p8590171210542"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b1659021214546">hetu-cli --catalog </strong><strong id="mrs_01_2348__en-us_topic_0000001219351171_b105901012105411">hive</strong><strong id="mrs_01_2348__en-us_topic_0000001219351171_b5590181235410">_1</strong> <strong id="mrs_01_2348__en-us_topic_0000001219351171_b105908121548">--schema default</strong></p>
|
|
</p></li><li id="mrs_01_2348__en-us_topic_0000001219351171_li373041512542"><span>Run the following command to view the database table:</span><p><div class="p" id="mrs_01_2348__en-us_topic_0000001219351171_p67303152540"><strong id="mrs_01_2348__en-us_topic_0000001219351171_b137300154549">show tables;</strong><pre class="screen" id="mrs_01_2348__en-us_topic_0000001219351171_screen8730181585410"> Table
|
|
---------
|
|
hivetb
|
|
(1 rows)
|
|
|
|
Query 20210730_084524_00023_u3sri@default@HetuEngine, FINISHED, 3 nodes
|
|
Splits: 36 total, 36 done (100.00%)
|
|
0:00 [2 rows, 47B] [7 rows/s, 167B/s]</pre>
|
|
</div>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24174.html">Configuring a Hive Data Source</a></div>
|
|
</div>
|
|
</div>
|
|
|