forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
40 lines
9.9 KiB
HTML
40 lines
9.9 KiB
HTML
<a name="mrs_01_1711"></a><a name="mrs_01_1711"></a>
|
|
|
|
<h1 class="topictitle1">Using HetuEngine from Scratch</h1>
|
|
<div id="body32001227"><p id="mrs_01_1711__en-us_topic_0000001173471396_p14199132284015">This section describes how to use <span id="mrs_01_1711__en-us_topic_0000001173471396_text396718246614">HetuEngine</span> to connect to the Hive data source and query database tables of the Hive data source of the cluster through <span id="mrs_01_1711__en-us_topic_0000001173471396_text1213511269619">HetuEngine</span>.</p>
|
|
<div class="section" id="mrs_01_1711__en-us_topic_0000001173471396_section4223183713315"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_1711__en-us_topic_0000001173471396_ul82175415125"><li id="mrs_01_1711__en-us_topic_0000001173471396_li0558103823119">The <span id="mrs_01_1711__en-us_topic_0000001173471396_text53169182818">HetuEngine</span> and Hive services have been installed in the cluster and are running properly.</li><li id="mrs_01_1711__en-us_topic_0000001173471396_li1921125481217">If Kerberos authentication has been enabled for the cluster, you need to create a <span id="mrs_01_1711__en-us_topic_0000001173471396_text101262614715">HetuEngine</span> user and grant related permissions to the user in advance. For details, see <a href="mrs_01_1714.html">Creating a HetuEngine User</a>. In addition, you need to configure the permissions to manage the databases, tables, and columns of the data source for the user using Ranger. For details, see <a href="mrs_01_1862.html">Adding a Ranger Access Permission Policy for HetuEngine</a>.</li><li id="mrs_01_1711__en-us_topic_0000001173471396_li10710191071718">The cluster client has been installed, for example, in the <strong id="mrs_01_1711__en-us_topic_0000001173471396_b27573391462">/opt/</strong><strong id="mrs_01_1711__en-us_topic_0000001173471396_b19757139134612"></strong><strong id="mrs_01_1711__en-us_topic_0000001173471396_b1757103974611">client</strong> directory.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1711__en-us_topic_0000001173471396_section4178193551814"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1711__en-us_topic_0000001173471396_ol08761956134314"><li id="mrs_01_1711__en-us_topic_0000001173471396_li20699443131"><span>Create and start a <span id="mrs_01_1711__en-us_topic_0000001173471396_text121964931314">HetuEngine</span> compute instance.</span><p><ol type="a" id="mrs_01_1711__en-us_topic_0000001173471396_ol812410287136"><li id="mrs_01_1711__en-us_topic_0000001173471396_li158829269168">Log in to FusionInsight Manager as a <span id="mrs_01_1711__en-us_topic_0000001173471396_text888232617166">HetuEngine</span> administrator and choose <strong id="mrs_01_1711__en-us_topic_0000001173471396_b54992008183912">Cluster</strong> > <strong id="mrs_01_1711__en-us_topic_0000001173471396_b182614852483912">Services</strong> > <strong id="mrs_01_1711__en-us_topic_0000001173471396_b47793037483912"><span id="mrs_01_1711__en-us_topic_0000001173471396_text16883526151616">HetuEngine</span></strong>. The <strong id="mrs_01_1711__en-us_topic_0000001173471396_b86477980283912"><span id="mrs_01_1711__en-us_topic_0000001173471396_text38831326201610">HetuEngine</span></strong> service page is displayed.</li><li id="mrs_01_1711__en-us_topic_0000001173471396_li926324181614">In the <strong id="mrs_01_1711__en-us_topic_0000001173471396_b186589229883912">Basic Information</strong> area on the <strong id="mrs_01_1711__en-us_topic_0000001173471396_b156492261483912">Dashboard</strong> tab page, click the link next to <strong id="mrs_01_1711__en-us_topic_0000001173471396_b201162571783912">HSConsole WebUI</strong>. The HSConsole page is displayed.</li><li id="mrs_01_1711__en-us_topic_0000001173471396_li101331569596">Click <strong id="mrs_01_1711__en-us_topic_0000001173471396_b56192113783912">Create Configuration</strong> above the instance list. In the <strong id="mrs_01_1711__en-us_topic_0000001173471396_b148797231283912">Configure Instance</strong> dialog box, configure parameters.<ol class="substepthirdol" id="mrs_01_1711__en-us_topic_0000001173471396_ol1562153195212"><li id="mrs_01_1711__en-us_topic_0000001173471396_li355817317526">In the <strong id="mrs_01_1711__en-us_topic_0000001173471396_b194871431102116">Basic Configuration</strong> area, set <strong id="mrs_01_1711__en-us_topic_0000001173471396_b1427812432212">Resource Queue</strong> to the tenant queue associated with the user.</li><li id="mrs_01_1711__en-us_topic_0000001173471396_li1755912319526">Configure parameters in the <strong id="mrs_01_1711__en-us_topic_0000001173471396_b118614380236">Coordinator Container Resource Configuration</strong>, <strong id="mrs_01_1711__en-us_topic_0000001173471396_b1029105542320">Worker Container Resource Configuration</strong>, and <strong id="mrs_01_1711__en-us_topic_0000001173471396_b1854516588230">Advanced Configuration</strong> areas based on the actual resource plan. For details about the parameter configuration, see <a href="mrs_01_1731.html">Creating HetuEngine Compute Instances</a> or retain the default values.</li><li id="mrs_01_1711__en-us_topic_0000001173471396_li3562531125214">Select <strong id="mrs_01_1711__en-us_topic_0000001173471396_b058425815331">Start Now</strong> and wait until the instance configuration is complete.</li></ol>
|
|
</li></ol>
|
|
</p></li><li id="mrs_01_1711__en-us_topic_0000001173471396_li203291026162813"><span>Log in to the node where the <span id="mrs_01_1711__en-us_topic_0000001173471396_text633182615282">HetuEngine</span> client is installed and run the following command to switch to the client installation directory:</span><p><p id="mrs_01_1711__en-us_topic_0000001173471396_p28761056204315"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b687615694311">cd /opt/client</strong></p>
|
|
</p></li><li id="mrs_01_1711__en-us_topic_0000001173471396_li1387625694320"><span>Run the following command to configure environment variables:</span><p><p id="mrs_01_1711__en-us_topic_0000001173471396_p16876175617437"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b12876195684320">source bigdata_env</strong></p>
|
|
</p></li><li id="mrs_01_1711__en-us_topic_0000001173471396_li20669114113218"><span>If the cluster is in security mode, run the following command to perform security authentication. If the cluster is in normal mode, skip this step.</span><p><p id="mrs_01_1711__en-us_topic_0000001173471396_p1087323193213"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b487152303215">kinit</strong> <em id="mrs_01_1711__en-us_topic_0000001173471396_i168719236321"><span id="mrs_01_1711__en-us_topic_0000001173471396_text10875232322">HetuEngine</span> operation user</em></p>
|
|
<p id="mrs_01_1711__en-us_topic_0000001173471396_p112621733213">Example:</p>
|
|
<p id="mrs_01_1711__en-us_topic_0000001173471396_p198251146183216"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b24219483325">kinit</strong> <strong id="mrs_01_1711__en-us_topic_0000001173471396_b1675143133311">hetu_test</strong></p>
|
|
<p id="mrs_01_1711__en-us_topic_0000001173471396_p1759156143319">Enter the password as prompted and change the password upon your first login.</p>
|
|
</p></li><li id="mrs_01_1711__en-us_topic_0000001173471396_li14684226123510"><span>Run the following command to log in to the catalog of the data source:</span><p><p id="mrs_01_1711__en-us_topic_0000001173471396_p1313315013510"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b3335134123614">hetu-cli --catalog </strong><em id="mrs_01_1711__en-us_topic_0000001173471396_i13662048362">Data source name</em></p>
|
|
<p id="mrs_01_1711__en-us_topic_0000001173471396_p3931182514275">For example, run the following command:</p>
|
|
<p id="mrs_01_1711__en-us_topic_0000001173471396_p1325134062712"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b127191443132713">hetu-cli --catalog </strong><strong id="mrs_01_1711__en-us_topic_0000001173471396_b13914154752716">hive</strong></p>
|
|
<div class="note" id="mrs_01_1711__en-us_topic_0000001173471396_note13912324119"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1711__en-us_topic_0000001173471396_p19462145110352">The default name of the Hive data source of the cluster is <strong id="mrs_01_1711__en-us_topic_0000001173471396_b348718316212">hive</strong>. If you need to connect to an external data source, configure the external data source on HSConsole by referring to <a href="mrs_01_2314.html">Configuring Data Sources</a>.</p>
|
|
</div></div>
|
|
<pre class="screen" id="mrs_01_1711__en-us_topic_0000001173471396_screen5321442183715">java -Djava.security.auth.login.config=/opt/client/HetuEngine/hetuserver/conf/jaas.conf -Dzookeeper.sasl.clientconfig=Client -Dzookeeper.auth.type=kerberos -Djava.security.krb5.conf=/opt/client/KrbClient/kerberos/var/krb5kdc/krb5.conf -Djava.util.logging.config.file=/opt/client/HetuEngine/hetuserver/conf/hetuserver-client-logging.properties -jar /opt/client/HetuEngine/hetuserver/jars/hetu-cli-*-executable.jar --catalog hive --deployment-mode on_yarn --server https://10.112.17.189:24002,10.112.17.228:24002,10.112.17.150:24002?serviceDiscoveryMode=zooKeeper&zooKeeperNamespace=hsbroker --krb5-remote-service-name HTTP --krb5-config-path /opt/client/KrbClient/kerberos/var/krb5kdc/krb5.conf
|
|
hetuengine> </pre>
|
|
</p></li><li id="mrs_01_1711__en-us_topic_0000001173471396_li38761456184315"><span>Run the following command to view the database information:</span><p><div class="p" id="mrs_01_1711__en-us_topic_0000001173471396_p10104234124420"><strong id="mrs_01_1711__en-us_topic_0000001173471396_b185754224415">show schemas;</strong><pre class="screen" id="mrs_01_1711__en-us_topic_0000001173471396_screen712919018380"> Schema
|
|
--------------------
|
|
default
|
|
information_schema
|
|
(2 rows)
|
|
Query 20200730_080535_00002_ct2eg, FINISHED, 3 nodes
|
|
Splits: 36 total, 36 done (100.00%)
|
|
0:02 [2 rows, 35B] [0 rows/s, 15B/s]</pre>
|
|
</div>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1710.html">Using HetuEngine</a></div>
|
|
</div>
|
|
</div>
|
|
|