vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity

MRS COMP_Normal Doc 20240802 version

Browse Source 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
This commit is contained in:
Yang, Tong 2024-09-20 11:28:13 +00:00 committed by zuul
parent 808079c203
commit 32524d3c9e
29 changed files with 8272 additions and 2049 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7675
docs/mrs/component-operation-guide/ALL_META.TXT.json
View File

File diff suppressed because it is too large Load Diff

2370
docs/mrs/component-operation-guide/CLASS.TXT.json
View File

File diff suppressed because it is too large Load Diff

BIN
docs/mrs/component-operation-guide/en-us_image_0000001533052357.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 19 KiB

7
docs/mrs/component-operation-guide/en-us_topic_0000001351362309.html
View File

@ -8,7 +8,12 @@
</th>
</tr>
</thead>
<tbody><tr id="EN-US_TOPIC_0000001351362309__row3420927161513"><td class="cellrowborder" valign="top" width="36.480000000000004%" headers="mcps1.3.1.1.3.1.1 "><p id="EN-US_TOPIC_0000001351362309__p1024302810551">2022-1<span id="EN-US_TOPIC_0000001351362309__ph287165915457">1</span>-<span id="EN-US_TOPIC_0000001351362309__ph6965057467">01</span></p>
<tbody><tr id="EN-US_TOPIC_0000001351362309__row19823132052617"><td class="cellrowborder" valign="top" width="36.480000000000004%" headers="mcps1.3.1.1.3.1.1 "><p id="EN-US_TOPIC_0000001351362309__p2082312202266"><span id="EN-US_TOPIC_0000001351362309__ph1564762217263">2023-08-02</span></p>
</td>
<td class="cellrowborder" valign="top" width="63.519999999999996%" headers="mcps1.3.1.1.3.1.2 "><ul id="EN-US_TOPIC_0000001351362309__ul45131853142217"><li id="EN-US_TOPIC_0000001351362309__li208077182416">Removed section "GeoMesa Command Line" from HBase.</li><li id="EN-US_TOPIC_0000001351362309__li621244562411">Removed section "Hive Materialized View" from Hive.</li><li id="EN-US_TOPIC_0000001351362309__li1953215045320">Fixed link errors in some sections.</li></ul>
</td>
</tr>
<tr id="EN-US_TOPIC_0000001351362309__row3420927161513"><td class="cellrowborder" valign="top" width="36.480000000000004%" headers="mcps1.3.1.1.3.1.1 "><p id="EN-US_TOPIC_0000001351362309__p1024302810551">2022-1<span id="EN-US_TOPIC_0000001351362309__ph287165915457">1</span>-<span id="EN-US_TOPIC_0000001351362309__ph6965057467">01</span></p>
</td>
<td class="cellrowborder" valign="top" width="63.519999999999996%" headers="mcps1.3.1.1.3.1.2 "><p id="EN-US_TOPIC_0000001351362309__p11836115045520">Modified the following content:</p>
<p id="EN-US_TOPIC_0000001351362309__p159491416184613">Updated the screenshots in the operation guides for ClickHouse, Ranger, Spark2x, Tez, and Yarn.</p>

2
docs/mrs/component-operation-guide/mrs_01_0132.html
View File

@ -8,7 +8,7 @@
</div>
<div class="section" id="mrs_01_0132__s061d5c51119b4146b96b91a0495a5042"><h4 class="sectiontitle">Impact on the System</h4><p id="mrs_01_0132__aea4879b8bd36469792d5968c82656a54">Site trust must be added to the browser when you access Manager and Hue web UI for the first time. Otherwise, the Hue web UI cannot be accessed.</p>
</div>
<div class="section" id="mrs_01_0132__sbffe2dde8108458fa04cafc49371a793"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0132__aa17a01c53c86450fb84ea031ebf787a0">When Kerberos authentication is enabled, the MRS cluster administrator has assigned the permission for using Hive to the user. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0345.html" target="_blank" rel="noopener noreferrer">Creating a User</a>. For example, create a human-machine user named <span class="parmname" id="mrs_01_0132__parmname70678761062"><b>hueuser</b></span>, add the user to user groups <span class="parmname" id="mrs_01_0132__parmname1022471610628"><b>hive</b></span> (the primary group), <span class="parmname" id="mrs_01_0132__parmname72121032195920"><b>hadoop</b></span>, <span class="parmname" id="mrs_01_0132__parmname4182955185920"><b>supergroup</b></span>, and <span class="parmname" id="mrs_01_0132__parmname1113327910648"><b>System_administrator</b></span>, and assign the <span class="parmname" id="mrs_01_0132__parmname1923373608113524"><b>System_administrator</b></span> role.</p>
<div class="section" id="mrs_01_0132__sbffe2dde8108458fa04cafc49371a793"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0132__aa17a01c53c86450fb84ea031ebf787a0">When Kerberos authentication is enabled, the MRS cluster administrator has assigned the permission for using Hive to the user. For example, create a human-machine user named <span class="parmname" id="mrs_01_0132__parmname70678761062"><b>hueuser</b></span>, add the user to user groups <span class="parmname" id="mrs_01_0132__parmname1022471610628"><b>hive</b></span> (the primary group), <span class="parmname" id="mrs_01_0132__parmname72121032195920"><b>hadoop</b></span>, <span class="parmname" id="mrs_01_0132__parmname4182955185920"><b>supergroup</b></span>, and <span class="parmname" id="mrs_01_0132__parmname1113327910648"><b>System_administrator</b></span>, and assign the <span class="parmname" id="mrs_01_0132__parmname1923373608113524"><b>System_administrator</b></span> role.</p>
<p id="mrs_01_0132__p762710300410">This user is used to log in to Manager.</p>
</div>
<div class="section" id="mrs_01_0132__s03aabc6eef834965add62eb571753d1d"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0132__o7d774762b9d54bea99a7f2a2e2cc825c"><li id="mrs_01_0132__li777313202051"><span>Log in to the service page.</span><p><p id="mrs_01_0132__p553210391663">For versions earlier than MRS 3.<em id="mrs_01_0132__i5706741111217">x</em>, click the cluster name on the MRS console and choose <strong id="mrs_01_0132__b1787841961218">Components</strong> &gt; <strong id="mrs_01_0132__b1487911941216">Hue</strong>.</p>

6
docs/mrs/component-operation-guide/mrs_01_0368.html
View File

@ -134,7 +134,7 @@
<div class="section" id="mrs_01_0368__section16144125273517"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0368__p1797113171213">The client has been installed. For example, the client is installed in the <strong id="mrs_01_0368__b5461726173811">/opt</strong><strong id="mrs_01_0368__b1346112610388"></strong><strong id="mrs_01_0368__b114617267382">/client</strong> directory. The client directory in the following operations is only an example. Change it to the actual installation directory. Before using the client, download and update the client configuration file, and ensure that the active management node of Manager is available.</p>
</div>
<div class="section" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_sa545593fee0d4deda232a76a7db99db4"><h4 class="sectiontitle">Procedure</h4><p id="mrs_01_0368__p13444173135518">For versions earlier than MRS 3.x, perform the following operations:</p>
<ol id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_o4aea6ac002494d60a743261275ffd54d"><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l6b58a848ef0f4fe6a361d4ef0ac39fb8"><a name="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l6b58a848ef0f4fe6a361d4ef0ac39fb8"></a><a name="en-us_topic_0229422393_en-us_topic_0173178212_l6b58a848ef0f4fe6a361d4ef0ac39fb8"></a><span>Download the client configuration file.</span><p><ol type="a" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_o10934933c3364550a0be5195fbfc2f32"><li id="mrs_01_0368__li1799392319436">Log in to MRS Manager. For details, see <a href="mrs_01_2123.html">Accessing Manager</a>. Then, choose <strong id="mrs_01_0368__b1956411591614">Services</strong>.</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_le025c1892e7147e290dff3b94a1d0393">Click <span class="uicontrol" id="mrs_01_0368__uicontrol103217557504"><b>Download Client</b></span>.<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_a05a2dba8ee714c9490116ce98af9b812">Set <span class="parmname" id="mrs_01_0368__parmname172174244519"><b>Client Type</b></span> to <span class="parmvalue" id="mrs_01_0368__parmvalue2222224155111"><b>Only configuration files</b></span>, <span class="parmname" id="mrs_01_0368__parmname51031226112316"><b>Download To</b></span> to <span class="parmvalue" id="mrs_01_0368__parmvalue5222122414515"><b>Server</b></span>, and click <span class="uicontrol" id="mrs_01_0368__uicontrol422212475119"><b>OK</b></span> to generate the client configuration file. The generated file is saved in the <strong id="mrs_01_0368__b9222624175112">/tmp/MRS-client</strong> directory on the active management node by default. You can customize the file path.</p>
<ol id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_o4aea6ac002494d60a743261275ffd54d"><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l6b58a848ef0f4fe6a361d4ef0ac39fb8"><span>Download the client configuration file.</span><p><ol type="a" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_o10934933c3364550a0be5195fbfc2f32"><li id="mrs_01_0368__li1799392319436">Log in to MRS Manager. For details, see <a href="mrs_01_2123.html">Accessing Manager</a>. Then, choose <strong id="mrs_01_0368__b1956411591614">Services</strong>.</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_le025c1892e7147e290dff3b94a1d0393">Click <span class="uicontrol" id="mrs_01_0368__uicontrol103217557504"><b>Download Client</b></span>.<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_a05a2dba8ee714c9490116ce98af9b812">Set <span class="parmname" id="mrs_01_0368__parmname172174244519"><b>Client Type</b></span> to <span class="parmvalue" id="mrs_01_0368__parmvalue2222224155111"><b>Only configuration files</b></span>, <span class="parmname" id="mrs_01_0368__parmname51031226112316"><b>Download To</b></span> to <span class="parmvalue" id="mrs_01_0368__parmvalue5222122414515"><b>Server</b></span>, and click <span class="uicontrol" id="mrs_01_0368__uicontrol422212475119"><b>OK</b></span> to generate the client configuration file. The generated file is saved in the <strong id="mrs_01_0368__b9222624175112">/tmp/MRS-client</strong> directory on the active management node by default. You can customize the file path.</p>
</li></ol>
</p></li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l5b697da80f724b049e94ef386ccc134d"><span>Log in to the active management node of MRS Manager.</span><p><ol type="a" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_oad0dd0b610c34488815d2b07e4656c11"><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l52f4be0ff6434297a1b84c6a772a9828">On the <span class="wintitle" id="mrs_01_0368__wintitle2051224969143126"><b>Node</b></span> tab page, view the <strong id="mrs_01_0368__b842352706143150">Name</strong> parameter. The node that contains <span class="parmvalue" id="mrs_01_0368__parmvalue2050481242143221"><b>master1</b></span> in its name is the Master1 node. The node that contains <span class="parmvalue" id="mrs_01_0368__parmvalue819447924143256"><b>master2</b></span> in its name is the Master2 node.<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_a2f41d2e8a2434da2a56a6f48035e2d15">The active and standby management nodes of MRS Manager are installed on Master nodes by default. Because Master1 and Master2 are switched over in active and standby mode, Master1 is not always the active management node of MRS Manager. Run a command in Master1 to check whether Master1 is active management node of MRS Manager. For details about the command, see <a href="#mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_le8e7045cece741e8b6209b929a50ff22">2.d</a>.</p>
</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_lad0d75a7717442b68338881c7c895c81">Log in to the Master1 node using the password as user <strong id="mrs_01_0368__b1563722621918">root</strong>. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0083.html" target="_blank" rel="noopener noreferrer">Logging In to an ECS</a>.</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l96479867566a4bd59da83a86f87373ef">Run the following commands to switch to user <strong id="mrs_01_0368__b773124319532">omm</strong>:<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_adbc6f7bd4142423a89ea39d9d7347674"><strong id="mrs_01_0368__b946729183115">sudo su - root</strong></p>
@ -149,14 +149,12 @@ NodeName HostName HAVersion StartTime
</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_lde3b2ea113c34b4687592dfdc8b454c2">Log in to the active management node, for example, <span class="parmvalue" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_p4fa8933886834ca8854b5d0617f23f10"><b>192-168-0-30</b></span> of MRS Manager as user <strong id="mrs_01_0368__b15755203051811">root</strong>, and run the following command to switch to user <strong id="mrs_01_0368__b1426520364189">omm</strong>:<p id="mrs_01_0368__p646554163014"><strong id="mrs_01_0368__b277316488327">sudo su - omm</strong></p>
</li></ol>
</p></li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l187b3ef7d91249bfb5f68eec47153c36"><span>Run the following command to switch to the client installation directory, for example, <span class="filepath" id="mrs_01_0368__filepath102271523514"><b>/opt/client</b></span>:</span><p><p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p357910583196"><b><span class="cmdname" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_c80d51658ea4842c68f915afed0247987">cd /opt/client</span></b></p>
</p></li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_lc39cdd52f6ac479ab273ecabbffd083b"><a name="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_lc39cdd52f6ac479ab273ecabbffd083b"></a><a name="en-us_topic_0229422393_en-us_topic_0173178212_lc39cdd52f6ac479ab273ecabbffd083b"></a><span>Run the following command to update the client configuration for the active management node.</span><p><p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p4610751203"><strong id="mrs_01_0368__b2643183718614">sh refreshConfig.sh /opt/client</strong> <i><span class="varname" id="mrs_01_0368__varname42077435588">Full path of the client configuration file package</span></i></p>
</p></li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_lc39cdd52f6ac479ab273ecabbffd083b"><span>Run the following command to update the client configuration for the active management node.</span><p><p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p4610751203"><strong id="mrs_01_0368__b2643183718614">sh refreshConfig.sh /opt/client</strong> <i><span class="varname" id="mrs_01_0368__varname42077435588">Full path of the client configuration file package</span></i></p>
<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p461020511200">For example, run the following command:</p>
<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p556680571064"><strong id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_a016ef072f6994457ba8c4988faa524b6">sh refreshConfig.sh /opt/client /tmp/MRS-</strong><strong id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_af1eae3cb3bf449269db10c1df25194d9">c</strong><strong id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_a2b344f9e4a0c41ed8194dc65b9fbcc6f">lient/MRS_Services_Client.tar</strong></p>
<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p661075172018">If the following information is displayed, the configurations have been updated successfully.</p>
<pre class="screen" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_s38bfe3be413e4853bb9d0ee12353c8d4">ReFresh components client config is complete.
Succeed to refresh components client config.</pre>
<div class="note" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_n7db6fa88e8fe4d84916aa5e228335142"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p613218111518">You can refer to steps <a href="#mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l6b58a848ef0f4fe6a361d4ef0ac39fb8">1</a> to <a href="#mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_lc39cdd52f6ac479ab273ecabbffd083b">4</a> or Method 2 in <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a>.</p>
</div></div>
</p></li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_l6556791324fa484e91dc201b480e3115"><span>Use the client on a Master node.</span><p><ol type="a" id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_od49292eae1f845e8a383e761ce0ce2f7"><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_led80939ae6024a9ba8841d649d5132b4">On the active management node where the client is updated, for example, node <span class="parmvalue" id="mrs_01_0368__parmvalue63001169215"><b>192-168-0-30</b></span>, run the following command to go to the client directory:<p id="mrs_01_0368__p852420598612"><strong id="mrs_01_0368__b796522518712">cd /opt/client</strong></p>
</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_ldf91db8293354ac4beabdf4679eb0034">Run the following command to configure environment variables:<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_en-us_topic_0037446806_p711917124619"><a name="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_ldf91db8293354ac4beabdf4679eb0034"></a><a name="en-us_topic_0229422393_en-us_topic_0173178212_ldf91db8293354ac4beabdf4679eb0034"></a><strong id="mrs_01_0368__b09435016338">source bigdata_env</strong></p>
</li><li id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_le6f90e19da384d2689c1b5ca5896f874">If Kerberos authentication is enabled for the current cluster, run the following command to authenticate the current user. The current user must have the permission to create HBase tables. If Kerberos authentication is disabled for the current cluster, skip this step.<p id="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_aeaf76195462f4119af05c698566aba09"><a name="mrs_01_0368__en-us_topic_0229422393_en-us_topic_0173178212_le6f90e19da384d2689c1b5ca5896f874"></a><a name="en-us_topic_0229422393_en-us_topic_0173178212_le6f90e19da384d2689c1b5ca5896f874"></a><strong id="mrs_01_0368__b63065667193458">kinit</strong> <em id="mrs_01_0368__i6148471145146">MRS cluster user</em></p>

2
docs/mrs/component-operation-guide/mrs_01_0370.html
View File

@ -10,7 +10,7 @@
</div>
<div class="section" id="mrs_01_0370__s061d5c51119b4146b96b91a0495a5042"><h4 class="sectiontitle">Impact on the System</h4><p id="mrs_01_0370__aea4879b8bd36469792d5968c82656a54">Site trust must be added to the browser when you access Manager and Hue web UI for the first time. Otherwise, the Hue web UI cannot be accessed.</p>
</div>
<div class="section" id="mrs_01_0370__sbffe2dde8108458fa04cafc49371a793"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0370__aa17a01c53c86450fb84ea031ebf787a0">When Kerberos authentication is enabled, the MRS cluster administrator has assigned the permission for using Hive to the user. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0345.html" target="_blank" rel="noopener noreferrer">Creating a User</a>. For example, create a human-machine user named <span class="parmname" id="mrs_01_0370__parmname70678761062"><b>hueuser</b></span>, add the user to user groups <span class="parmname" id="mrs_01_0370__parmname1022471610628"><b>hive</b></span> (the primary group), <span class="parmname" id="mrs_01_0370__parmname72121032195920"><b>hadoop</b></span>, and <span class="parmname" id="mrs_01_0370__parmname4182955185920"><b>supergroup</b></span>, and role <span class="parmname" id="mrs_01_0370__parmname1113327910648"><b>System_administrator</b></span>.</p>
<div class="section" id="mrs_01_0370__sbffe2dde8108458fa04cafc49371a793"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0370__aa17a01c53c86450fb84ea031ebf787a0">When Kerberos authentication is enabled, the MRS cluster administrator has assigned the permission for using Hive to the user. For example, create a human-machine user named <span class="parmname" id="mrs_01_0370__parmname70678761062"><b>hueuser</b></span>, add the user to user groups <span class="parmname" id="mrs_01_0370__parmname1022471610628"><b>hive</b></span> (the primary group), <span class="parmname" id="mrs_01_0370__parmname72121032195920"><b>hadoop</b></span>, and <span class="parmname" id="mrs_01_0370__parmname4182955185920"><b>supergroup</b></span>, and role <span class="parmname" id="mrs_01_0370__parmname1113327910648"><b>System_administrator</b></span>.</p>
<p id="mrs_01_0370__p762710300410">This user is used to log in to the Hue WebUI.</p>
</div>
<div class="section" id="mrs_01_0370__s03aabc6eef834965add62eb571753d1d"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0370__o7d774762b9d54bea99a7f2a2e2cc825c"><li id="mrs_01_0370__li13806122116487"><span>Log in to the service page.</span><p><ul id="mrs_01_0370__ul1692910323486"><li id="mrs_01_0370__li792916322486">For versions earlier than MRS 1.9.2, log in to MRS Manager and choose <strong id="mrs_01_0370__b164781281897">Services</strong>.</li><li id="mrs_01_0370__li979413816488">For MRS 1.9.2 or later, click the cluster name on the MRS console and choose <strong id="mrs_01_0370__b998863012912">Components</strong>.</li></ul>

4
docs/mrs/component-operation-guide/mrs_01_0397.html
View File

@ -69,7 +69,7 @@ client.sinks.kafka_sink.kafka.bootstrap.servers = <em id="mrs_01_0397__aa6421766
client.sinks.kafka_sink.flumeBatchSize = 1000
client.sinks.kafka_sink.kafka.producer.type = sync
client.sinks.kafka_sink.kafka.security.protocol = SASL_PLAINTEXT
client.sinks.kafka_sink.kafka.kerberos.domain.name = <em id="mrs_01_0397__i11557195517119">Kafka domain name. This parameter is mandatory for a security cluster, for example, <strong id="mrs_01_0397__b048853675419">hadoop.xxx.com</strong>.</em>
client.sinks.kafka_sink.kafka.kerberos.domain.name = <em id="mrs_01_0397__i11557195517119">Kafka domain name. This parameter is mandatory for a security cluster.</em>
client.sinks.kafka_sink.requiredAcks = 0
client.sources.static_log_source.channels = static_log_channel
@ -142,7 +142,7 @@ client.sinks.kafka_sink.kafka.bootstrap.servers = <em id="mrs_01_0397__i11174692
client.sinks.kafka_sink.flumeBatchSize = 1000
client.sinks.kafka_sink.kafka.producer.type = sync
client.sinks.kafka_sink.kafka.security.protocol = SASL_PLAINTEXT
client.sinks.kafka_sink.kafka.kerberos.domain.name = <em id="mrs_01_0397__i16432172518395">Kafka domain name. This parameter is mandatory for a security cluster, for example, <strong id="mrs_01_0397__b943211252390">hadoop.xxx.com</strong>.</em>
client.sinks.kafka_sink.kafka.kerberos.domain.name = <em id="mrs_01_0397__i16432172518395">Kafka domain name. This parameter is mandatory for a security cluster.</em>
client.sinks.kafka_sink.requiredAcks = 0
client.sources.static_log_source.channels = static_log_channel

4
docs/mrs/component-operation-guide/mrs_01_0434.html
View File

@ -5,9 +5,9 @@
<p id="mrs_01_0434__p1937961181812">The Presto component of MRS 3.x does not support Kerberos authentication.</p>
<div class="section" id="mrs_01_0434__s26390c77824e48628302cd27728a109b"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_0434__ul1921432134110"><li id="mrs_01_0434__li1092113326414">The password of user <span class="parmname" id="mrs_01_0434__p168d3b622bc6437291ffb132b663ee34"><b>admin</b></span> has been obtained. The password of user <span class="parmname" id="mrs_01_0434__p65ce8b7a377247cab40e868181d0cdf4"><b>admin</b></span> is specified by the user during MRS cluster creation.</li><li id="mrs_01_0434__li10921193294111">The client has been updated.</li><li id="mrs_01_0434__li137914414587">The Presto client has been manually installed for MRS 3.<em id="mrs_01_0434__i3673678154">x</em> clusters.</li></ul>
</div>
<div class="section" id="mrs_01_0434__section15757123718144"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0434__o1a1694aafb174694a4a0e67712a47f76"><li id="mrs_01_0434__li17987144152514"><a name="mrs_01_0434__li17987144152514"></a><a name="li17987144152514"></a><span>For clusters with Kerberos authentication enabled, log in to MRS Manager and create a role with the <strong id="mrs_01_0434__b17603701465">Hive Admin Privilege</strong> permission. For details about how to create a role, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0343.html" target="_blank" rel="noopener noreferrer">Creating a Role</a>.</span></li><li id="mrs_01_0434__li9368161132311"><a name="mrs_01_0434__li9368161132311"></a><a name="li9368161132311"></a><span>Create a user that belongs to the <strong id="mrs_01_0434__b39857295519">Presto</strong> and <strong id="mrs_01_0434__b1894515535515">Hive</strong> groups, bind the role created in <a href="#mrs_01_0434__li17987144152514">1</a> to the user, and download the user authentication file. For details, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0345.html" target="_blank" rel="noopener noreferrer">Creating a User</a> and <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0352.html" target="_blank" rel="noopener noreferrer">Downloading a User Authentication File</a>.</span></li><li id="mrs_01_0434__li861292619304"><a name="mrs_01_0434__li861292619304"></a><a name="li861292619304"></a><span>Upload the downloaded<strong id="mrs_01_0434__b73161897427"> user.keytab</strong> and <strong id="mrs_01_0434__b7416201854211">krb5.conf</strong> files to the node where the MRS client resides.</span><p><div class="note" id="mrs_01_0434__note18294255302"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0434__p53012543010">For clusters with Kerberos authentication enabled, <a href="#mrs_01_0434__li9368161132311">2</a> to <a href="#mrs_01_0434__li861292619304">3</a> must be performed. For normal clusters, start from <a href="#mrs_01_0434__l6bafa992ef354ebc8c1e16387160ae24">4</a>.</p>
<div class="section" id="mrs_01_0434__section15757123718144"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0434__o1a1694aafb174694a4a0e67712a47f76"><li id="mrs_01_0434__li17987144152514"><a name="mrs_01_0434__li17987144152514"></a><a name="li17987144152514"></a><span>For clusters with Kerberos authentication enabled, log in to MRS Manager and create a role with the <strong id="mrs_01_0434__b17603701465">Hive Admin Privilege</strong> permission.</span></li><li id="mrs_01_0434__li9368161132311"><a name="mrs_01_0434__li9368161132311"></a><a name="li9368161132311"></a><span>Create a user that belongs to the <strong id="mrs_01_0434__b39857295519">Presto</strong> and <strong id="mrs_01_0434__b1894515535515">Hive</strong> groups, bind the role created in <a href="#mrs_01_0434__li17987144152514">1</a> to the user, and download the user authentication file.</span></li><li id="mrs_01_0434__li861292619304"><a name="mrs_01_0434__li861292619304"></a><a name="li861292619304"></a><span>Upload the downloaded<strong id="mrs_01_0434__b73161897427"> user.keytab</strong> and <strong id="mrs_01_0434__b7416201854211">krb5.conf</strong> files to the node where the MRS client resides.</span><p><div class="note" id="mrs_01_0434__note18294255302"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0434__p53012543010">For clusters with Kerberos authentication enabled, <a href="#mrs_01_0434__li9368161132311">2</a> to <a href="#mrs_01_0434__li861292619304">3</a> must be performed. For normal clusters, start from <a href="#mrs_01_0434__l6bafa992ef354ebc8c1e16387160ae24">4</a>.</p>
</div></div>
</p></li><li id="mrs_01_0434__l6bafa992ef354ebc8c1e16387160ae24"><a name="mrs_01_0434__l6bafa992ef354ebc8c1e16387160ae24"></a><a name="l6bafa992ef354ebc8c1e16387160ae24"></a><span>Prepare a client based on service conditions and log in to the node where the client is installed.</span><p><p id="mrs_01_0434__adf68edafed604cd99d30695b463edfa5">For example, if you have updated the client on the Master2 node, log in to the Master2 node to use the client. For details, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a>.</p>
</p></li><li id="mrs_01_0434__l6bafa992ef354ebc8c1e16387160ae24"><a name="mrs_01_0434__l6bafa992ef354ebc8c1e16387160ae24"></a><a name="l6bafa992ef354ebc8c1e16387160ae24"></a><span>Prepare a client based on service conditions and log in to the node where the client is installed.</span><p><p id="mrs_01_0434__adf68edafed604cd99d30695b463edfa5">For example, if you have updated the client on the Master2 node, log in to the Master2 node to use the client.</p>
</p></li><li id="mrs_01_0434__l6e2058aef79c4678a11786539c7d698c"><span>Run the following command to switch the user:</span><p><p id="mrs_01_0434__a1dc242a3b5a746b0b69163db94055771"><strong id="mrs_01_0434__en-us_topic_0057061151_b560210819742">sudo su - omm</strong></p>
</p></li><li id="mrs_01_0434__ldc59b65d35af40029b69d717590c5ee1"><span>Run the following command to switch to the client directory, for example, <span class="filepath" id="mrs_01_0434__f15f7cd346f484cdf97ee97e2ebfe9077"><b>/opt/client</b></span>.</span><p><p id="mrs_01_0434__a5c7abf7d609e49daa21d689801c7665d"><strong id="mrs_01_0434__en-us_topic_0057061151_b539462609946">cd /opt/client</strong></p>
</p></li><li id="mrs_01_0434__lb12ce87c6e7946bb9a848a19a34c9e35"><span>Run the following command to configure environment variables:</span><p><p id="mrs_01_0434__af0f6bc7c36064c379eed07a5509d2ba2"><strong id="mrs_01_0434__a867f14eb42d44d819707ead5ac2ca425">source bigdata_env</strong></p>

10
docs/mrs/component-operation-guide/mrs_01_0442.html
View File

@ -132,13 +132,13 @@
</table>
</div>
</div>
<div class="section" id="mrs_01_0442__section16880721113918"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0442__ol11471534103911"><li id="mrs_01_0442__l6b58a848ef0f4fe6a361d4ef0ac39fb8"><a name="mrs_01_0442__l6b58a848ef0f4fe6a361d4ef0ac39fb8"></a><a name="l6b58a848ef0f4fe6a361d4ef0ac39fb8"></a><span>Download the client configuration file.</span><p><ul id="mrs_01_0442__ul67518334192"><li id="mrs_01_0442__li1475123316190">For versions earlier than MRS 3.x, perform the following operations:<ol type="a" id="mrs_01_0442__o10934933c3364550a0be5195fbfc2f32"><li id="mrs_01_0442__li1799392319436">Log in to MRS Manager. For details, see <a href="mrs_01_2123.html">Accessing Manager</a>. Then, choose <strong id="mrs_01_0442__b138441346112116">Services</strong>.</li><li id="mrs_01_0442__le025c1892e7147e290dff3b94a1d0393">Click <span class="uicontrol" id="mrs_01_0442__uicontrol13633146836"><b>Download Client</b></span>.<p id="mrs_01_0442__p6662635987">Set <span class="parmname" id="mrs_01_0442__parmname07231071731"><b>Client Type</b></span> to <span class="parmvalue" id="mrs_01_0442__parmvalue9723671136"><b>Only configuration files</b></span>, <span class="parmname" id="mrs_01_0442__parmname1172418714314"><b>Download to</b></span> to <span class="parmvalue" id="mrs_01_0442__parmvalue772415714316"><b>Server</b></span>, and click <span class="uicontrol" id="mrs_01_0442__uicontrol11724137335"><b>OK</b></span> to generate the client configuration file. The generated file is saved in the <strong id="mrs_01_0442__b872413717315">/tmp/MRS-client</strong> directory on the active management node by default.</p>
<div class="section" id="mrs_01_0442__section16880721113918"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0442__ol11471534103911"><li id="mrs_01_0442__l6b58a848ef0f4fe6a361d4ef0ac39fb8"><span>Download the client configuration file.</span><p><ul id="mrs_01_0442__ul67518334192"><li id="mrs_01_0442__li1475123316190">For versions earlier than MRS 3.x, perform the following operations:<ol type="a" id="mrs_01_0442__o10934933c3364550a0be5195fbfc2f32"><li id="mrs_01_0442__li1799392319436">Log in to MRS Manager. For details, see <a href="mrs_01_2123.html">Accessing Manager</a>. Then, choose <strong id="mrs_01_0442__b138441346112116">Services</strong>.</li><li id="mrs_01_0442__le025c1892e7147e290dff3b94a1d0393">Click <span class="uicontrol" id="mrs_01_0442__uicontrol13633146836"><b>Download Client</b></span>.<p id="mrs_01_0442__p6662635987">Set <span class="parmname" id="mrs_01_0442__parmname07231071731"><b>Client Type</b></span> to <span class="parmvalue" id="mrs_01_0442__parmvalue9723671136"><b>Only configuration files</b></span>, <span class="parmname" id="mrs_01_0442__parmname1172418714314"><b>Download to</b></span> to <span class="parmvalue" id="mrs_01_0442__parmvalue772415714316"><b>Server</b></span>, and click <span class="uicontrol" id="mrs_01_0442__uicontrol11724137335"><b>OK</b></span> to generate the client configuration file. The generated file is saved in the <strong id="mrs_01_0442__b872413717315">/tmp/MRS-client</strong> directory on the active management node by default.</p>
</li></ol>
</li><li id="mrs_01_0442__li2066134213204">For MRS 3.<em id="mrs_01_0442__i1361819111942">x</em> or later, perform the following operations:<ol type="a" id="mrs_01_0442__ol169871359123614"><li id="mrs_01_0442__li3986195917363">Log in to FusionInsight Manager. For details, see <a href="mrs_01_2124.html">Accessing FusionInsight Manager (MRS 3.x or Later)</a>.</li><li id="mrs_01_0442__li11986125923613">Choose <strong id="mrs_01_0442__b180745429451219">Cluster</strong> &gt; <em id="mrs_01_0442__i12295953251219">Name of the desired cluster</em> &gt; <strong id="mrs_01_0442__b78877487751219">Dashboard</strong> &gt; <strong id="mrs_01_0442__b99905805451219">More</strong> &gt; <strong id="mrs_01_0442__b20987200151219">Download Client</strong>.</li><li id="mrs_01_0442__li3987145913362">Download the cluster client.<p id="mrs_01_0442__p20987105913361"><a name="mrs_01_0442__li3987145913362"></a><a name="li3987145913362"></a>Set <span class="parmname" id="mrs_01_0442__parmname89861359103617"><b>Select Client Type</b></span> to <span class="parmvalue" id="mrs_01_0442__parmvalue1198711591367"><b>Configuration Files Only </b></span>, select a platform type, and click <span class="uicontrol" id="mrs_01_0442__uicontrol1198745973617"><b>OK</b></span> to generate the client configuration file which is then saved in the <strong id="mrs_01_0442__b18808003651219">/tmp/FusionInsight-Client/</strong> directory on the active management node by default.</p>
</li></ol>
</li></ul>
</p></li><li id="mrs_01_0442__li595112919519"><span>Log in to the active management node of Manager.</span><p><ul id="mrs_01_0442__ul166431739132314"><li id="mrs_01_0442__li11643143952314">For versions earlier than MRS 3.x, perform the following operations:<ol type="a" id="mrs_01_0442__oad0dd0b610c34488815d2b07e4656c11"><li id="mrs_01_0442__l52f4be0ff6434297a1b84c6a772a9828">On the MRS console, click <strong id="mrs_01_0442__b1510935112277">Clusters</strong>, choose <strong id="mrs_01_0442__b05255516279">Active Clusters</strong>, and click a cluster name. On the <strong id="mrs_01_0442__b1641925016220">Nodes</strong> tab, view the node names. The node whose name contains <strong id="mrs_01_0442__b2101112610234">master1</strong> is the Master1 node, and the node whose name contains <strong id="mrs_01_0442__b493853692315">master2</strong> is the Master2 node.<p id="mrs_01_0442__a2f41d2e8a2434da2a56a6f48035e2d15">The active and standby management nodes of MRS Manager are installed on Master nodes by default. Because Master1 and Master2 are switched over in active and standby mode, Master1 is not always the active management node of MRS Manager. Run a command in Master1 to check whether Master1 is active management node of MRS Manager. For details about the command, see <a href="#mrs_01_0442__le8e7045cece741e8b6209b929a50ff22">2.d</a>.</p>
</li><li id="mrs_01_0442__lad0d75a7717442b68338881c7c895c81">Log in to the Master1 node using the password as user <strong id="mrs_01_0442__b1529118518202">root</strong>. For details, see <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0083.html" target="_blank" rel="noopener noreferrer">Logging In to a Cluster</a>.</li><li id="mrs_01_0442__l96479867566a4bd59da83a86f87373ef">Run the following commands to switch to user <strong id="mrs_01_0442__b1550681893417">omm</strong>:<p id="mrs_01_0442__adbc6f7bd4142423a89ea39d9d7347674"><strong id="mrs_01_0442__b2760121812416">sudo su - root</strong></p>
</li><li id="mrs_01_0442__lad0d75a7717442b68338881c7c895c81">Log in to the Master1 node using the password as user <strong id="mrs_01_0442__b1529118518202">root</strong>.</li><li id="mrs_01_0442__l96479867566a4bd59da83a86f87373ef">Run the following commands to switch to user <strong id="mrs_01_0442__b1550681893417">omm</strong>:<p id="mrs_01_0442__adbc6f7bd4142423a89ea39d9d7347674"><strong id="mrs_01_0442__b2760121812416">sudo su - root</strong></p>
<p id="mrs_01_0442__a09516e4f38e44d8085ab1e7b041fddc0"><strong id="mrs_01_0442__b1476592216414">su - omm</strong></p>
</li><li id="mrs_01_0442__le8e7045cece741e8b6209b929a50ff22"><a name="mrs_01_0442__le8e7045cece741e8b6209b929a50ff22"></a><a name="le8e7045cece741e8b6209b929a50ff22"></a>Run the following command to check the active management node of MRS Manager:<p id="mrs_01_0442__p296913416327"><a name="mrs_01_0442__le8e7045cece741e8b6209b929a50ff22"></a><a name="le8e7045cece741e8b6209b929a50ff22"></a><strong id="mrs_01_0442__b3201818417756">sh ${BIGDATA_HOME}/om-0.0.1/sbin/status-oms.sh</strong></p>
<p id="mrs_01_0442__a2e14fbc736604468bd7585d96245cd58">In the command output, the node whose <span class="parmname" id="mrs_01_0442__parmname1093914458151"><b>HAActive</b></span> is <span class="parmvalue" id="mrs_01_0442__parmvalue109409453159"><b>active</b></span> is the active management node, and the node whose <span class="parmname" id="mrs_01_0442__parmname7941154517155"><b>HAActive</b></span> is <span class="parmvalue" id="mrs_01_0442__parmvalue179421745181519"><b>standby</b></span> is the standby management node. In the following example, <strong id="mrs_01_0442__b14942345171520">mgtomsdat-sh-3-01-1</strong> is the active management node, and <strong id="mrs_01_0442__b8943845141519">mgtomsdat-sh-3-01-2</strong> is the standby management node.</p>
@ -160,19 +160,17 @@ NodeName HostName HAVersion StartTime
</li></ul>
</p></li><li id="mrs_01_0442__li1325114150135"><span>Run the following command to go to the client installation directory:</span><p><p id="mrs_01_0442__p1499713212913"><strong id="mrs_01_0442__b138661758142811">cd /opt/client</strong></p>
<p id="mrs_01_0442__p924985613282">The cluster client has been installed in advance. The following client installation directory is used as an example. Change it based on the site requirements.</p>
</p></li><li id="mrs_01_0442__li15639738131312"><a name="mrs_01_0442__li15639738131312"></a><a name="li15639738131312"></a><span>Run the following command to update the client configuration for the active management node.</span><p><p id="mrs_01_0442__p365063812136"><strong id="mrs_01_0442__b9719204219357">sh refreshConfig.sh /opt/client</strong> <i><span class="varname" id="mrs_01_0442__varname1665143881313">Full path of the client configuration file package</span></i></p>
</p></li><li id="mrs_01_0442__li15639738131312"><span>Run the following command to update the client configuration for the active management node.</span><p><p id="mrs_01_0442__p365063812136"><strong id="mrs_01_0442__b9719204219357">sh refreshConfig.sh /opt/client</strong> <i><span class="varname" id="mrs_01_0442__varname1665143881313">Full path of the client configuration file package</span></i></p>
<p id="mrs_01_0442__p86531138181318">For example, run the following command:</p>
<p id="mrs_01_0442__p3654193818136"><strong id="mrs_01_0442__b14655103818135">sh refreshConfig.sh /opt/client </strong><strong id="mrs_01_0442__b2054762513449">/tmp/FusionInsight-C</strong><strong id="mrs_01_0442__b146151621525">lient/</strong><strong id="mrs_01_0442__b139102545817">FusionInsight_Cluster_1_Services_Client.tar</strong></p>
<p id="mrs_01_0442__p9659123820139">If the following information is displayed, the configurations have been updated successfully.</p>
<pre class="screen" id="mrs_01_0442__screen1866023831317"> ReFresh components client config is complete.
Succeed to refresh components client config.</pre>
<div class="note" id="mrs_01_0442__note1971123710424"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0442__p613218111518">You can refer to Method 2 in <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a> to perform operations in steps <a href="#mrs_01_0442__l6b58a848ef0f4fe6a361d4ef0ac39fb8">1</a> to <a href="#mrs_01_0442__li15639738131312">4</a>.</p>
</div></div>
</p></li><li id="mrs_01_0442__l6556791324fa484e91dc201b480e3115"><span>Use the client on a Master node.</span><p><ol type="a" id="mrs_01_0442__od49292eae1f845e8a383e761ce0ce2f7"><li id="mrs_01_0442__led80939ae6024a9ba8841d649d5132b4">On the active management node, for example, <span class="parmvalue" id="mrs_01_0442__pb923e7c26e0f4bffb26f1521910a2c79"><b>192-168-0-30</b></span>, run the following command to switch to the client directory, for example, <strong id="mrs_01_0442__b189531649195119">/opt</strong><strong id="mrs_01_0442__b5953114910511"></strong><strong id="mrs_01_0442__b89531249105119">/client</strong>.<p id="mrs_01_0442__en-us_topic_0037446806_p11191414462"><strong id="mrs_01_0442__b1668485133617">cd /opt</strong><strong id="mrs_01_0442__b17686135173619">/client</strong></p>
</li><li id="mrs_01_0442__ldf91db8293354ac4beabdf4679eb0034">Run the following command to configure environment variables:<p id="mrs_01_0442__en-us_topic_0037446806_p711917124619"><a name="mrs_01_0442__ldf91db8293354ac4beabdf4679eb0034"></a><a name="ldf91db8293354ac4beabdf4679eb0034"></a><strong id="mrs_01_0442__b10953105010554">source bigdata_env</strong></p>
</li><li id="mrs_01_0442__li495913245475">If Kerberos authentication is enabled for the current cluster, run the following command to authenticate the current user:<p id="mrs_01_0442__aeaf76195462f4119af05c698566aba09"><a name="mrs_01_0442__li495913245475"></a><a name="li495913245475"></a><strong id="mrs_01_0442__b09391645217">kinit</strong> <em id="mrs_01_0442__i16939545217">MRS cluster user</em></p>
<p id="mrs_01_0442__en-us_topic_0037446806_p809043511651">Example: user <strong id="mrs_01_0442__b144573521725">kinit hiveuser</strong></p>
<p id="mrs_01_0442__p676363144715">The current user must have the permission to create Hive tables. To create a role with the permission, refer to <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0343.html" target="_blank" rel="noopener noreferrer">Creating a Role</a>. To bind the role to the current user, refer to <a href="https://docs.otc.t-systems.com/en-us/usermanual/mrs/mrs_01_0345.html" target="_blank" rel="noopener noreferrer">Creating a User</a>.If Kerberos authentication is disabled, skip this step.</p>
<p id="mrs_01_0442__p676363144715">The current user must have the permission to create Hive tables.If Kerberos authentication is disabled, skip this step.</p>
</li><li id="mrs_01_0442__en-us_topic_0037446806_li8119414469">Run the client command of the Hive component directly.<p id="mrs_01_0442__ae866cfe594424869bd57657b6921e1a8"><a name="mrs_01_0442__en-us_topic_0037446806_li8119414469"></a><a name="en-us_topic_0037446806_li8119414469"></a><strong id="mrs_01_0442__b181391521567">beeline</strong></p>
</li></ol>
</p></li><li id="mrs_01_0442__li2015821418148"><span>Run the Hive client command to implement service A.</span><p><p id="mrs_01_0442__p1185185714716"><strong id="mrs_01_0442__b1596817213318">Operations on internal tables</strong>:</p>

2
docs/mrs/component-operation-guide/mrs_01_0443.html
View File

@ -4,7 +4,7 @@
<div id="body1589421623830"><div class="note" id="mrs_01_0443__note157161915598"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0443__p7716149125917">The operations described in this section apply only to clusters of versions earlier than MRS 3.x.</p>
</div></div>
<p id="mrs_01_0443__p938634316933">If the default parameter settings of the MRS service cannot meet your requirements, you can modify the parameter settings as required.</p>
<ol id="mrs_01_0443__ol16662473715"><li id="mrs_01_0443__li24453319614"><span>Log in to the service page.</span><p><p id="mrs_01_0443__p1528815213240">For versions earlier than MRS 1.9.2: Log in to <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0102.html" target="_blank" rel="noopener noreferrer">MRS Manager</a>, and choose <strong id="mrs_01_0443__b56021340173016">Services</strong>.</p>
<ol id="mrs_01_0443__ol16662473715"><li id="mrs_01_0443__li24453319614"><span>Log in to the service page.</span><p><p id="mrs_01_0443__p1528815213240">For versions earlier than MRS 1.9.2: Log in to <span id="mrs_01_0443__ph381714913215">MRS Manager</span>, and choose <strong id="mrs_01_0443__b56021340173016">Services</strong>.</p>
<p id="mrs_01_0443__p463355062416">For MRS 1.9.2 or later: Click the cluster name on the MRS console and choose <strong id="mrs_01_0443__b2386950194317">Components</strong>.</p>
</p></li><li id="mrs_01_0443__li448801318714"><span>Choose <strong id="mrs_01_0443__b5206171005717">HBase</strong> &gt; <strong id="mrs_01_0443__b1636781415714">Service Configuration</strong> and switch <strong id="mrs_01_0443__b74231329183517">Basic</strong> to <strong id="mrs_01_0443__b15423129133517">All</strong>. On the displayed HBase configuration page, modify parameter settings.</span><p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0443__table430163185120" frame="border" border="1" rules="all"><caption><b>Table 1 </b>HBase parameters</caption><thead align="left"><tr id="mrs_01_0443__row2030218315519"><th align="left" class="cellrowborder" valign="top" width="39.24392439243925%" id="mcps1.3.3.2.2.1.2.4.1.1"><p id="mrs_01_0443__p13021439512">Parameter</p>

2
docs/mrs/component-operation-guide/mrs_01_0500.html
View File

@ -18,8 +18,6 @@
</li>
<li class="ulchildlink"><strong><a href="mrs_01_0510.html">Using the ReplicationSyncUp Tool</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="mrs_01_24119.html">GeoMesa Command Line</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="mrs_01_1609.html">Configuring HBase DR</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="mrs_01_24112.html">Configuring HBase Data Compression and Encoding</a></strong><br>

10
docs/mrs/component-operation-guide/mrs_01_0501.html
View File

@ -5,13 +5,13 @@
</div>
<div class="section" id="mrs_01_0501__section748518154955"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_0501__ul60630031154955"><li id="mrs_01_0501__li8799372154955">The active and standby clusters have been successfully installed and started (the cluster status is <strong id="mrs_01_0501__b19570111213419">Running</strong> on the <strong id="mrs_01_0501__b95705126414">Active Clusters</strong> page), and you have the administrator rights of the clusters.</li></ul>
</div>
<ul id="mrs_01_0501__ul12085484154955"><li id="mrs_01_0501__li41660498154955">The network between the active and standby clusters is normal and ports can be used properly.</li><li id="mrs_01_0501__li39400166154955">Cross-cluster mutual trust has been configured. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0354.html" target="_blank" rel="noopener noreferrer">Configuring Cross-Cluster Mutual Trust Relationships</a>. </li><li id="mrs_01_0501__li27466911181447">If historical data exists in the active cluster and needs to be synchronized to the standby cluster, cross-cluster replication must be configured for the active and standby clusters. For details, see <a href="mrs_01_0502.html">Enabling Cross-Cluster Copy</a>.</li><li id="mrs_01_0501__li37296868154955">Time is consistent between the active and standby clusters and the Network Time Protocol (NTP) service on the active and standby clusters uses the same time source.</li><li id="mrs_01_0501__li127496154955">Mapping relationships between the names of all hosts in the active and standby clusters and service IP addresses have been configured in the <span class="filepath" id="mrs_01_0501__filepath17461135918131"><b>/etc/hosts</b></span> file by appending <strong id="mrs_01_0501__b168494714716">192.***.***.*** host1</strong> to the <strong id="mrs_01_0501__b73931611184710">hosts</strong> file.</li><li id="mrs_01_0501__li1147471154955">The network bandwidth between the active and standby clusters is determined based on service volume, which cannot be less than the possible maximum service volume.</li></ul>
<ul id="mrs_01_0501__ul12085484154955"><li id="mrs_01_0501__li41660498154955">The network between the active and standby clusters is normal and ports can be used properly.</li><li id="mrs_01_0501__li39400166154955">Cross-cluster mutual trust has been configured.</li><li id="mrs_01_0501__li27466911181447">If historical data exists in the active cluster and needs to be synchronized to the standby cluster, cross-cluster replication must be configured for the active and standby clusters. For details, see <a href="mrs_01_0502.html">Enabling Cross-Cluster Copy</a>.</li><li id="mrs_01_0501__li37296868154955">Time is consistent between the active and standby clusters and the Network Time Protocol (NTP) service on the active and standby clusters uses the same time source.</li><li id="mrs_01_0501__li127496154955">Mapping relationships between the names of all hosts in the active and standby clusters and service IP addresses have been configured in the <span class="filepath" id="mrs_01_0501__filepath17461135918131"><b>/etc/hosts</b></span> file by appending <strong id="mrs_01_0501__b168494714716">192.***.***.*** host1</strong> to the <strong id="mrs_01_0501__b73931611184710">hosts</strong> file.</li><li id="mrs_01_0501__li1147471154955">The network bandwidth between the active and standby clusters is determined based on service volume, which cannot be less than the possible maximum service volume.</li></ul>
<div class="section" id="mrs_01_0501__section10327240154955"><h4 class="sectiontitle">Constraints</h4><ul id="mrs_01_0501__ul31200129154955"><li id="mrs_01_0501__li12365706154955">Despite that HBase cluster replication provides the real-time data replication function, the data synchronization progress is determined by several factors, such as the service loads in the active cluster and the health status of processes in the standby cluster. In normal cases, the standby cluster should not take over services. In extreme cases, system maintenance personnel and other decision makers determine whether the standby cluster takes over services according to the current data synchronization indicators.</li></ul>
</div>
<ul id="mrs_01_0501__ul44182490154955"><li id="mrs_01_0501__li62098090154955">Currently, the replication function supports only one active cluster and one standby cluster in HBase.</li><li id="mrs_01_0501__li22011899154955">Typically, do not perform operations on data synchronization tables in the standby cluster, such as modifying table properties or deleting tables. If any misoperation on the standby cluster occurs, data synchronization between the active and standby clusters will fail and data of the corresponding table in the standby cluster will be lost.</li><li id="mrs_01_0501__li63889366154955">If the replication function of HBase tables in the active cluster is enabled for data synchronization, after modifying the structure of a table in the active cluster, you need to manually modify the structure of the corresponding table in the standby cluster to ensure table structure consistency.</li></ul>
<div class="section" id="mrs_01_0501__section38133389154955"><h4 class="sectiontitle">Procedure</h4><p id="mrs_01_0501__p1796791154955"><strong id="mrs_01_0501__b210750102219">Enable the replication function for the active cluster to synchronize data written by Put.</strong></p>
</div>
<ol id="mrs_01_0501__ol11322402154955"><li id="mrs_01_0501__li155891430132615"><a name="mrs_01_0501__li155891430132615"></a><a name="li155891430132615"></a><span>Log in to the service page.</span><p><p id="mrs_01_0501__p1528815213240">For versions earlier than MRS 1.9.2: Log in to <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0102.html" target="_blank" rel="noopener noreferrer">MRS Manager</a>, and choose <strong id="mrs_01_0501__b25913815159">Services</strong>.</p>
<ol id="mrs_01_0501__ol11322402154955"><li id="mrs_01_0501__li155891430132615"><a name="mrs_01_0501__li155891430132615"></a><a name="li155891430132615"></a><span>Log in to the service page.</span><p><p id="mrs_01_0501__p1528815213240">For versions earlier than MRS 1.9.2: Log in to MRS Manager, and choose <strong id="mrs_01_0501__b25913815159">Services</strong>.</p>
<p id="mrs_01_0501__p463355062416">For MRS 1.9.2 or later: Click the cluster name on the MRS console and choose <strong id="mrs_01_0501__b046217256434">Components</strong>.</p>
</p></li><li id="mrs_01_0501__li1966213718714"><span>Go to the <strong id="mrs_01_0501__b17128165751013">All Configurations</strong> page of the HBase service. For details, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span><p><div class="note" id="mrs_01_0501__note3349205551313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p1832113512387">For clusters of MRS 1.9.2 or later:</p>
<p id="mrs_01_0501__p158007468574">If the <strong id="mrs_01_0501__b396711645817">Components</strong> tab is unavailable, complete IAM user synchronization first. (On the <strong id="mrs_01_0501__b169672016175814">Dashboard</strong> page, click <strong id="mrs_01_0501__b7967516195814">Synchronize</strong> on the right side of <span class="parmname" id="mrs_01_0501__parmname896718168583"><b>IAM User Sync</b></span> to synchronize IAM users.)</p>
@ -91,11 +91,11 @@
</div></div>
<p id="mrs_01_0501__p49575864154955">If yes, go to <a href="#mrs_01_0501__li57688977154955">6</a>.</p>
<p id="mrs_01_0501__p56222010154955">If no, go to <a href="#mrs_01_0501__li6210082154955">10</a>.</p>
</p></li><li id="mrs_01_0501__li57688977154955"><a name="mrs_01_0501__li57688977154955"></a><a name="li57688977154955"></a><span>Go to the <strong id="mrs_01_0501__b139596521126">All Configurations</strong> page of the HBase service parameters by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span></li><li id="mrs_01_0501__li42295589154955"><span>On the HBase configuration interface of the active and standby clusters, search for <strong id="mrs_01_0501__b169641627102815">hbase.replication.cluster.id</strong> and modify it. It specifies the HBase ID of the active and standby clusters. For example, the HBase ID of the active cluster is set to <strong id="mrs_01_0501__b1920923311287">replication1</strong> and the HBase ID of the standby cluster is set to <strong id="mrs_01_0501__b3325455132815">replication2</strong> for connecting the active cluster to the standby cluster. To save data overhead, the parameter value length is not recommended to exceed 30.</span></li><li id="mrs_01_0501__li3244131341713"><a name="mrs_01_0501__li3244131341713"></a><a name="li3244131341713"></a><span>On the HBase configuration interface of the standby cluster, search for <strong id="mrs_01_0501__b14277360338">hbase.replication.conf.dir</strong> and modify it. It specifies the HBase configurations of the active cluster client used by the standby cluster and is used for data replication when the bulkload data replication function is enabled. The parameter value is a path name, for example, <strong id="mrs_01_0501__b6375324151316">/home</strong>.</span><p><div class="note" id="mrs_01_0501__note0771101318917"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_0501__ul183455195611"><li id="mrs_01_0501__li1498465282715">In versions earlier than MRS 3.x, you do not need to set this parameter. Skip <a href="#mrs_01_0501__li3244131341713">8</a>.</li><li id="mrs_01_0501__li1526159185615">When bulkload replication is enabled, you need to manually place the HBase client configuration files (<strong id="mrs_01_0501__b1261104534018">core-site.xml</strong>, <strong id="mrs_01_0501__b6621345114014">hdfs-site.xml</strong>, and <strong id="mrs_01_0501__b166294574012">hbase-site.xml</strong>) in the active cluster on all RegionServer nodes in the standby cluster. The actual path for placing the configuration file is <span class="filepath" id="mrs_01_0501__filepath1535414652413"><b>${hbase.replication.conf.dir}/${hbase.replication.cluster.id}</b></span>. For example, if <strong id="mrs_01_0501__b1396611594195">hbase.replication.conf.dir</strong> of the standby cluster is set to <strong id="mrs_01_0501__b1967195991915">/home</strong> and <strong id="mrs_01_0501__b196712593194">hbase.replication.cluster.id</strong> of the active cluster is set to <strong id="mrs_01_0501__b896755917197">replication1</strong>, the actual path for placing the configuration files in the standby cluster is <strong id="mrs_01_0501__b196715596192">/home/replication1</strong>. You also need to change the corresponding directory and file permissions by running the <strong id="mrs_01_0501__b12950741172619">chown -R omm:wheel /home/replication1</strong> command.</li><li id="mrs_01_0501__li59631495619">You can obtain the client configuration files from the client in the active cluster, for example, the <span class="filepath" id="mrs_01_0501__filepath5748185916261"><b>/opt/client/HBase/hbase/conf</b></span> path. For details about how to update the configuration file, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a>.</li></ul>
</p></li><li id="mrs_01_0501__li57688977154955"><a name="mrs_01_0501__li57688977154955"></a><a name="li57688977154955"></a><span>Go to the <strong id="mrs_01_0501__b139596521126">All Configurations</strong> page of the HBase service parameters by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span></li><li id="mrs_01_0501__li42295589154955"><span>On the HBase configuration interface of the active and standby clusters, search for <strong id="mrs_01_0501__b169641627102815">hbase.replication.cluster.id</strong> and modify it. It specifies the HBase ID of the active and standby clusters. For example, the HBase ID of the active cluster is set to <strong id="mrs_01_0501__b1920923311287">replication1</strong> and the HBase ID of the standby cluster is set to <strong id="mrs_01_0501__b3325455132815">replication2</strong> for connecting the active cluster to the standby cluster. To save data overhead, the parameter value length is not recommended to exceed 30.</span></li><li id="mrs_01_0501__li3244131341713"><a name="mrs_01_0501__li3244131341713"></a><a name="li3244131341713"></a><span>On the HBase configuration interface of the standby cluster, search for <strong id="mrs_01_0501__b14277360338">hbase.replication.conf.dir</strong> and modify it. It specifies the HBase configurations of the active cluster client used by the standby cluster and is used for data replication when the bulkload data replication function is enabled. The parameter value is a path name, for example, <strong id="mrs_01_0501__b6375324151316">/home</strong>.</span><p><div class="note" id="mrs_01_0501__note0771101318917"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_0501__ul183455195611"><li id="mrs_01_0501__li1498465282715">In versions earlier than MRS 3.x, you do not need to set this parameter. Skip <a href="#mrs_01_0501__li3244131341713">8</a>.</li><li id="mrs_01_0501__li1526159185615">When bulkload replication is enabled, you need to manually place the HBase client configuration files (<strong id="mrs_01_0501__b1261104534018">core-site.xml</strong>, <strong id="mrs_01_0501__b6621345114014">hdfs-site.xml</strong>, and <strong id="mrs_01_0501__b166294574012">hbase-site.xml</strong>) in the active cluster on all RegionServer nodes in the standby cluster. The actual path for placing the configuration file is <span class="filepath" id="mrs_01_0501__filepath1535414652413"><b>${hbase.replication.conf.dir}/${hbase.replication.cluster.id}</b></span>. For example, if <strong id="mrs_01_0501__b1396611594195">hbase.replication.conf.dir</strong> of the standby cluster is set to <strong id="mrs_01_0501__b1967195991915">/home</strong> and <strong id="mrs_01_0501__b196712593194">hbase.replication.cluster.id</strong> of the active cluster is set to <strong id="mrs_01_0501__b896755917197">replication1</strong>, the actual path for placing the configuration files in the standby cluster is <strong id="mrs_01_0501__b196715596192">/home/replication1</strong>. You also need to change the corresponding directory and file permissions by running the <strong id="mrs_01_0501__b12950741172619">chown -R omm:wheel /home/replication1</strong> command.</li><li id="mrs_01_0501__li59631495619">You can obtain the client configuration files from the client in the active cluster, for example, the <span class="filepath" id="mrs_01_0501__filepath5748185916261"><b>/opt/client/HBase/hbase/conf</b></span> path.</li></ul>
</div></div>
</p></li><li id="mrs_01_0501__li49438752154955"><span>On the HBase configuration page of the active cluster, search for and change the value of <strong id="mrs_01_0501__b315633212458">hbase.replication.bulkload.enabled</strong> to <strong id="mrs_01_0501__b3652138164512">true</strong> to enable bulkload replication.</span></li></ol>
<p id="mrs_01_0501__p45115985154955"><strong id="mrs_01_0501__b205516120911">Restarting the HBase service and install the client</strong></p>
<ol start="10" id="mrs_01_0501__ol30516170154955"><li id="mrs_01_0501__li6210082154955"><a name="mrs_01_0501__li6210082154955"></a><a name="li6210082154955"></a><span>Save the configurations and restart HBase.</span></li><li id="mrs_01_0501__li11385192216347"><a name="mrs_01_0501__li11385192216347"></a><a name="li11385192216347"></a><span>In the active and standby clusters of MRS 1.9.2 or earlier, choose <strong id="mrs_01_0501__b1410344019461">Cluster</strong> &gt; <strong id="mrs_01_0501__b6103240174616">Dashboard</strong> &gt; <strong id="mrs_01_0501__b610417406465">More</strong> &gt; <strong id="mrs_01_0501__b131042405468">Download Client</strong> of MRS 1.9.2 or later, choose <strong id="mrs_01_0501__b142171950174612">Cluster</strong> &gt; <strong id="mrs_01_0501__b1521711508466">Dashboard</strong> &gt; <strong id="mrs_01_0501__b1921875084615">More</strong> &gt; <strong id="mrs_01_0501__b1621805034614">Download Client</strong>. For details about how to update the client configuration file, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0089.html" target="_blank" rel="noopener noreferrer">Updating a Client</a>.</span></li></ol>
<ol start="10" id="mrs_01_0501__ol30516170154955"><li id="mrs_01_0501__li6210082154955"><a name="mrs_01_0501__li6210082154955"></a><a name="li6210082154955"></a><span>Save the configurations and restart HBase.</span></li><li id="mrs_01_0501__li11385192216347"><a name="mrs_01_0501__li11385192216347"></a><a name="li11385192216347"></a><span>In the active and standby clusters of MRS 1.9.2 or earlier, choose <strong id="mrs_01_0501__b1410344019461">Cluster</strong> &gt; <strong id="mrs_01_0501__b6103240174616">Dashboard</strong> &gt; <strong id="mrs_01_0501__b610417406465">More</strong> &gt; <strong id="mrs_01_0501__b131042405468">Download Client</strong> of MRS 1.9.2 or later, choose <strong id="mrs_01_0501__b142171950174612">Cluster</strong> &gt; <strong id="mrs_01_0501__b1521711508466">Dashboard</strong> &gt; <strong id="mrs_01_0501__b1921875084615">More</strong> &gt; <strong id="mrs_01_0501__b1621805034614">Download Client</strong>.</span></li></ol>
<p id="mrs_01_0501__p562774110491"><strong id="mrs_01_0501__b165261018163714">Synchronize table data of the active cluster. (Skip this step if the active cluster has no data.)</strong></p>
<ol start="12" id="mrs_01_0501__ol2512178117406"><li id="mrs_01_0501__li12641483154955"><a name="mrs_01_0501__li12641483154955"></a><a name="li12641483154955"></a><span>Access the HBase shell of the active cluster as user <strong id="mrs_01_0501__b1346896175110">hbase</strong>.</span><p><ol type="a" id="mrs_01_0501__ol574666129352"><li id="mrs_01_0501__li474374619352">On the active management node where the client has been updated, run the following command to go to the client directory:<p id="mrs_01_0501__aa4134e1860f64d04b8af130e80d60c1f"><a name="mrs_01_0501__li474374619352"></a><a name="li474374619352"></a><strong id="mrs_01_0501__af7849077e90740ccbbe480ab0aa7ec10">cd /opt/client</strong></p>
</li><li id="mrs_01_0501__li533801889352">Run the following command to configure environment variables:<p class="litext" id="mrs_01_0501__p106596489352"><a name="mrs_01_0501__li533801889352"></a><a name="li533801889352"></a><strong id="mrs_01_0501__a07d0d8687a6d4a55bc7ddd4c880258a4">source bigdata_env</strong></p>
@ -116,8 +116,6 @@
</li><li id="mrs_01_0501__li1214795510491">Copy the data that has been exported to the standby cluster.<p id="mrs_01_0501__p4222273910491"><a name="mrs_01_0501__li1214795510491"></a><a name="li1214795510491"></a><strong id="mrs_01_0501__b511971519714">hadoop distcp</strong> <em id="mrs_01_0501__i165741025779">Directory for storing source data in the active cluster</em> <strong id="mrs_01_0501__b1930891919817">hdfs://</strong><em id="mrs_01_0501__i16568192320810">ActiveNameNodeIP</em>:<strong id="mrs_01_0501__b159528261081">9820/</strong> <em id="mrs_01_0501__i1126984119719">Directory for storing source data in the standby cluster</em></p>
<p id="mrs_01_0501__p6510999110491"><strong id="mrs_01_0501__b10615343778">ActiveNameNodeIP</strong> indicates the IP address of the active NameNode in the standby cluster.</p>
<p id="mrs_01_0501__p4911901110491">Example: <strong id="mrs_01_0501__b3941791810491">hadoop distcp /user/hbase/t1 hdfs://192.168.40.2:9820/user/hbase/t1</strong></p>
<div class="note" id="mrs_01_0501__note14067447129"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0501__p184064449121">In MRS 1.6.2 and earlier versions, the default port number is 25000. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0504.html" target="_blank" rel="noopener noreferrer">List of Open Source Component Ports</a>.</p>
</div></div>
</li><li id="mrs_01_0501__li1921694510491">Import data to the standby cluster as the HBase table user of the standby cluster.<p id="mrs_01_0501__p3873478110491"><a name="mrs_01_0501__li1921694510491"></a><a name="li1921694510491"></a><strong id="mrs_01_0501__b9280184811111">hbase org.apache.hadoop.hbase.mapreduce.Import</strong> <em id="mrs_01_0501__i711615384123">-Dimport.bulk.output=Directory where the output data is stored in the standby cluster Table name Directory where the source data is stored in the standby cluster</em></p>
<p id="mrs_01_0501__p5193305110491"><strong id="mrs_01_0501__b1183703811311">hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles</strong> <em id="mrs_01_0501__i1857715503137">Directory where the output data is stored in the standby cluster Table name</em></p>
<p id="mrs_01_0501__p979513510491">For example, <strong id="mrs_01_0501__b2104735110491">hbase org.apache.hadoop.hbase.mapreduce.Import -Dimport.bulk.output=/user/hbase/output_t1 t1 /user/hbase/t1</strong> and</p>

6
docs/mrs/component-operation-guide/mrs_01_0502.html
View File

@ -11,17 +11,13 @@
<p id="mrs_01_0502__p1885710212539">For versions earlier than MRS 3.x, choose <strong id="mrs_01_0502__b10434155193411">Components</strong> &gt; <strong id="mrs_01_0502__b1943511514345">HDFS</strong> &gt; <strong id="mrs_01_0502__b14435205153410">Service Configuration</strong> on the cluster details page. Switch <strong id="mrs_01_0502__b1543516519345">Basic</strong> to <strong id="mrs_01_0502__b143513533419">All</strong>, and search for <strong id="mrs_01_0502__b1643511573414">hadoop.rpc.protection</strong>.</p>
</div></div>
</div>
<div class="section" id="mrs_01_0502__section6491650916933"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0502__ol2374590516933"><li id="mrs_01_0502__li328938153213"><span>Log in to the service page.</span><p><p id="mrs_01_0502__p72818387329">For versions earlier than MRS 1.9.2: Log in to <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0102.html" target="_blank" rel="noopener noreferrer">MRS Manager</a>, and choose <strong id="mrs_01_0502__b9693174431">Services</strong>.</p>
<p id="mrs_01_0502__p428183883217">For MRS 1.9.2 or later: Click the cluster name on the MRS console and choose <strong id="mrs_01_0502__b142481044311">Components</strong>.</p>
</p></li><li id="mrs_01_0502__li1966213718714"><span>Go to the <strong id="mrs_01_0502__b2751163393918">All Configurations</strong> page of the Yarn service. For details, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span><p><div class="note" id="mrs_01_0502__note3349205551313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p158007468574">If the <strong id="mrs_01_0502__b18432028175820">Components</strong> tab is unavailable, complete IAM user synchronization first. (On the <strong id="mrs_01_0502__b19432112835811">Dashboard</strong> page, click <strong id="mrs_01_0502__b1643242815582">Synchronize</strong> on the right side of <span class="parmname" id="mrs_01_0502__parmname194321528135815"><b>IAM User Sync</b></span> to synchronize IAM users.)</p>
<div class="section" id="mrs_01_0502__section6491650916933"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0502__ol2374590516933"><li id="mrs_01_0502__li328938153213"><span>Log in to the service page.</span></li><li id="mrs_01_0502__li1966213718714"><span>Go to the <strong id="mrs_01_0502__b2751163393918">All Configurations</strong> page of the Yarn service. For details, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span><p><div class="note" id="mrs_01_0502__note3349205551313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p158007468574">If the <strong id="mrs_01_0502__b18432028175820">Components</strong> tab is unavailable, complete IAM user synchronization first. (On the <strong id="mrs_01_0502__b19432112835811">Dashboard</strong> page, click <strong id="mrs_01_0502__b1643242815582">Synchronize</strong> on the right side of <span class="parmname" id="mrs_01_0502__parmname194321528135815"><b>IAM User Sync</b></span> to synchronize IAM users.)</p>
</div></div>
</p></li><li id="mrs_01_0502__li6378706216933"><span>In the navigation pane, choose <strong id="mrs_01_0502__b1640775534813">Yarn</strong> &gt; <strong id="mrs_01_0502__b1794423334719">Distcp</strong>.</span></li><li id="mrs_01_0502__li3721265416933"><span>Set <strong id="mrs_01_0502__b1569813219450">haclusterX.remotenn1</strong> of <strong id="mrs_01_0502__b068010164452">dfs.namenode.rpc-address</strong> to the service IP address and RPC port number of one NameNode instance of the peer cluster, and set <strong id="mrs_01_0502__b124923454911">haclusterX.remotenn2</strong> to the service IP address and RPC port number of the other NameNode instance of the peer cluster. Enter a value in the <em id="mrs_01_0502__i17809946303">IP address:port</em> format.</span><p><div class="note" id="mrs_01_0502__note60675655181130"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p690124713013">For MRS 1.9.2 or later, log in to the MRS console, click the cluster name, and choose <strong id="mrs_01_0502__b6816222010">Components</strong> &gt; <strong id="mrs_01_0502__b5973547205">HDFS</strong> &gt; <strong id="mrs_01_0502__b239568172014">Instances</strong> to obtain the service IP address of the NameNode instance.</p>
<p id="mrs_01_0502__p1831135910351">You can also log in to FusionInsight Manager in MRS 3.x clusters, and choose <strong id="mrs_01_0502__b16571115211">Cluster</strong> &gt; <em id="mrs_01_0502__i964575115238">Name of the desired cluster</em> &gt; <strong id="mrs_01_0502__b1646915518210">Services</strong> &gt; <strong id="mrs_01_0502__b26321353152115">HDFS</strong> &gt; <strong id="mrs_01_0502__b2805951162112">Instance</strong> to obtain the service IP address of the NameNode instance.</p>
</div></div>
<p id="mrs_01_0502__p6647843216933"><strong id="mrs_01_0502__b625311530509">dfs.namenode.rpc-address.haclusterX.remotenn1</strong> and <strong id="mrs_01_0502__b158577145115">dfs.namenode.rpc-address.haclusterX.remotenn2</strong> do not distinguish active and standby NameNode instances. The default NameNode RPC port is 9820 and cannot be modified on MRS Manager.</p>
<p id="mrs_01_0502__p6143498316933">For example, <strong id="mrs_01_0502__b258505053117">10.1.1.1:9820</strong> and <strong id="mrs_01_0502__b16903545315">10.1.1.2:9820</strong>.</p>
<div class="note" id="mrs_01_0502__note65601137102310"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0502__p9560103742315">For MRS 1.6.2 or earlier, the default port number is <strong id="mrs_01_0502__b84561446105219">25000</strong>. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0504.html" target="_blank" rel="noopener noreferrer">List of Open Source Component Ports</a>.</p>
</div></div>
</p></li><li id="mrs_01_0502__li2449066216933"><span>Save the configuration. On the <strong id="mrs_01_0502__b169576303174">Dashboard</strong> tab page, and choose <strong id="mrs_01_0502__b169571530131719">More</strong> &gt; <strong id="mrs_01_0502__b16957123012175">Restart Service</strong> to restart the Yarn service.</span><p><p id="mrs_01_0502__p1908936716933"><strong id="mrs_01_0502__b181312576571">Operation succeeded</strong> is displayed. Click <strong id="mrs_01_0502__b177011150586">Finish</strong>. The Yarn service is started successfully.</p>
</p></li><li id="mrs_01_0502__li3758657716933"><span>Log in to the other cluster and repeat the preceding operations.</span></li></ol>
</div>

2
docs/mrs/component-operation-guide/mrs_01_0581.html
View File

@ -58,8 +58,6 @@
</li>
<li class="ulchildlink"><strong><a href="mrs_01_1750.html">Switching the Hive Execution Engine to Tez</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="mrs_01_2311.html">Hive Materialized View</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="mrs_01_0976.html">Hive Log Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="mrs_01_0977.html">Hive Performance Tuning</a></strong><br>

2
docs/mrs/component-operation-guide/mrs_01_0949.html
View File

@ -23,7 +23,7 @@
</td>
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.3.4.2.3.2.3.1.2 "><p id="mrs_01_0949__a3a16f1b645ea4893b9dbd0a7c16f56d5">In the <strong id="mrs_01_0949__b8680180133911">Permission</strong> table, click <strong id="mrs_01_0949__b1680200163910">Hive</strong> and select <strong id="mrs_01_0949__b233815217387">Hive Admin Privilege</strong>.</p>
<div class="note" id="mrs_01_0949__n768c024de71e4be5a7ca11d780550475"><span class="notetitle"> NOTE: </span><div class="notebody"><p class="textintable" id="mrs_01_0949__aeaaa930f124b470ca0f6c4f62ddfea8f">After being bound to the Hive administrator role, perform the following operations during each maintenance operation:</p>
<ol type="a" id="mrs_01_0949__o22025688792147a69a4b441a2e159ca0"><li id="mrs_01_0949__l3ec30b72298549d2b374d9add384655d">Log in to the node where the client is installed. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0091.html" target="_blank" rel="noopener noreferrer">Installing a Client</a>.</li><li id="mrs_01_0949__l53e519f4f75e40b097f9a24dc70486d0">Run the following command to configure environment variables:<p class="litext" id="mrs_01_0949__a7cb143b430624aed9fcb7f62f733aa51"><a name="mrs_01_0949__l53e519f4f75e40b097f9a24dc70486d0"></a><a name="l53e519f4f75e40b097f9a24dc70486d0"></a>For example, if the Hive client installation directory is <strong id="mrs_01_0949__b9200144514432">/opt/hiveclient</strong>, run <strong id="mrs_01_0949__a2163e7f877a9454db8ee28ea87144eeb">source /opt/hiveclient/bigdata_env</strong>.</p>
<ol type="a" id="mrs_01_0949__o22025688792147a69a4b441a2e159ca0"><li id="mrs_01_0949__l3ec30b72298549d2b374d9add384655d">Log in to the node where the client is installed.</li><li id="mrs_01_0949__l53e519f4f75e40b097f9a24dc70486d0">Run the following command to configure environment variables:<p class="litext" id="mrs_01_0949__a7cb143b430624aed9fcb7f62f733aa51"><a name="mrs_01_0949__l53e519f4f75e40b097f9a24dc70486d0"></a><a name="l53e519f4f75e40b097f9a24dc70486d0"></a>For example, if the Hive client installation directory is <strong id="mrs_01_0949__b9200144514432">/opt/hiveclient</strong>, run <strong id="mrs_01_0949__a2163e7f877a9454db8ee28ea87144eeb">source /opt/hiveclient/bigdata_env</strong>.</p>
</li><li id="mrs_01_0949__l1a40354ae9674d0da669b4fc35cc918e">Run the following command to authenticate the user:<p class="litext" id="mrs_01_0949__a48755538b3d44de38791df7ed368ab85"><a name="mrs_01_0949__l1a40354ae9674d0da669b4fc35cc918e"></a><a name="l1a40354ae9674d0da669b4fc35cc918e"></a><strong id="mrs_01_0949__b1833534919432">kinit</strong> <em id="mrs_01_0949__i2034074914318">Hive service user</em></p>
</li><li id="mrs_01_0949__l68bc41f3262f4d78852dae085cb4dd1c">Run the following command to log in to the client tool:<p class="litext" id="mrs_01_0949__a3b73ca536c184361b13af264c16b1c1d"><a name="mrs_01_0949__l68bc41f3262f4d78852dae085cb4dd1c"></a><a name="l68bc41f3262f4d78852dae085cb4dd1c"></a><strong id="mrs_01_0949__aa1e3a7d17715434ea08b2578a5591503">beeline</strong></p>
</li><li id="mrs_01_0949__le84ff71b118b4a48a9f24cbab9188f94">Run the following command to update the <span id="mrs_01_0949__ph1512102023917">Hive </span>administrator permissions:<p id="mrs_01_0949__ac94d0dcac4814326b675097bfa097f1b"><strong id="mrs_01_0949__a2707464e1e1147df8a1a6525470a5827">set role admin;</strong></p>

2
docs/mrs/component-operation-guide/mrs_01_0951.html
View File

@ -16,7 +16,7 @@
<p id="mrs_01_0951__p749713818712"><strong id="mrs_01_0951__b15509165504718">Hive over HBase Authorization in MRS Earlier than 3.x</strong></p>
<p id="mrs_01_0951__a79628b1b5f884ec99d2b2518eafb31e9">After the permissions are assigned, you can use HQL statements that are similar to SQL statements to access HBase tables from Hive. The following uses the procedure for assigning a user the rights to query HBase tables as an example.</p>
<ol id="mrs_01_0951__o0fe6d95738f9462b8122423d0815f083"><li id="mrs_01_0951__l5dcd76c4431243babfaf6cfed66ef47c"><span>On the role management page of MRS Manager, create an HBase role, for example, <strong id="mrs_01_0951__b17541549114">hive_hbase_create</strong>, and grant the permission to create HBase tables.</span><p><p id="mrs_01_0951__a05e5ac8ef9194eca883f5f0e4bfc1401">In the <strong id="mrs_01_0951__b356522114288">Permission</strong> table, choose <strong id="mrs_01_0951__b20235112522816">HBase</strong> &gt; <strong id="mrs_01_0951__b1353234102811">HBase Scope</strong> &gt; <strong id="mrs_01_0951__b1845263702810">global</strong>, select <strong id="mrs_01_0951__b8703164312283">create</strong> of the namespace <strong id="mrs_01_0951__b970094912815">default</strong>, and click <strong id="mrs_01_0951__b5695622810">OK</strong>.</p>
</p></li><li id="mrs_01_0951__l1f8708bf45424de8aee48f595ee0453d"><span>On MRS Manager, create a human-machine user, for example, <strong id="mrs_01_0951__b166527545113">hbase_creates_user</strong>, add the user to the <strong id="mrs_01_0951__b1365855415115">hive</strong> group, and bind the <strong id="mrs_01_0951__b2065915543116">hive_hbase_create</strong> role to the user so that the user can create Hive and HBase tables.</span></li><li id="mrs_01_0951__le5c89eaab339455cbfafc7dfefe99b7e"><span><span id="mrs_01_0951__ph1824832115713">Log in to the node where the client is installed. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0091.html" target="_blank" rel="noopener noreferrer">Installing a Client</a>.</span></span></li><li id="mrs_01_0951__l0f3a0529df284c05950bed4fcf7ece1b"><span>Run the following command to configure environment variables:</span><p><p id="mrs_01_0951__ac254199782ff42d5803b4235bd66d49f"><strong id="mrs_01_0951__ac649466204fe4a25a0571bc91bba4fa5">source /opt/client/bigdata_env</strong></p>
</p></li><li id="mrs_01_0951__l1f8708bf45424de8aee48f595ee0453d"><span>On MRS Manager, create a human-machine user, for example, <strong id="mrs_01_0951__b166527545113">hbase_creates_user</strong>, add the user to the <strong id="mrs_01_0951__b1365855415115">hive</strong> group, and bind the <strong id="mrs_01_0951__b2065915543116">hive_hbase_create</strong> role to the user so that the user can create Hive and HBase tables.</span></li><li id="mrs_01_0951__le5c89eaab339455cbfafc7dfefe99b7e"><span>Log in to the node where the client is installed.</span></li><li id="mrs_01_0951__l0f3a0529df284c05950bed4fcf7ece1b"><span>Run the following command to configure environment variables:</span><p><p id="mrs_01_0951__ac254199782ff42d5803b4235bd66d49f"><strong id="mrs_01_0951__ac649466204fe4a25a0571bc91bba4fa5">source /opt/client/bigdata_env</strong></p>
</p></li><li id="mrs_01_0951__l9b6ad09bfa89459cb9c3e02bfe4d50ff"><span>Run the following command to authenticate the user:</span><p><p class="litext" id="mrs_01_0951__aeb6be69711f3469f95294f9d284a7637"><strong id="mrs_01_0951__a5360d2d7a8af46a6b6ea2962565ea949">kinit hbase_creates_user</strong></p>
</p></li><li id="mrs_01_0951__l60c6415ee6f349ab85fa816f0645100e"><span>Run the following command to go to the shell environment of the Hive client:</span><p><p class="litext" id="mrs_01_0951__afbecb80aa4c54b1494c55ad9400dd684"><strong id="mrs_01_0951__a2bc438652a5a42c5a898165c802198da">beeline</strong></p>
</p></li><li id="mrs_01_0951__l7f58131545b64e078dcae8477e34ca26"><span>Run the following command to create a table in Hive and HBase, for example, the <strong id="mrs_01_0951__b19347155419306">thh</strong> table.</span><p><p id="mrs_01_0951__ab576f56741474d77876b5126173d5a1f"><strong id="mrs_01_0951__aab970834fc0d4ed19d8f5d7127fa35ad">CREATE TABLE thh(id int, name string, country string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES("hbase.columns.mapping" = "cf1:id,cf1:name,:key") TBLPROPERTIES ("hbase.table.name" = "thh");</strong></p>

4
docs/mrs/component-operation-guide/mrs_01_1183.html
View File

@ -2,8 +2,8 @@
<h1 class="topictitle1">Using the Spark Client</h1>
<div id="body1590995733628"><p id="mrs_01_1183__p19542134143214">After an MRS cluster is created, you can create and submit jobs on the client. The client can be installed on nodes inside or outside the cluster.</p>
<ul id="mrs_01_1183__ul16756144563213"><li id="mrs_01_1183__li19756184593214">Nodes inside the cluster: After an MRS cluster is created, the client has been installed on the master and core nodes in the cluster by default. For details, see <span id="mrs_01_1183__ph7215514163313"><a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0090.html" target="_blank" rel="noopener noreferrer">Using an MRS Client on Nodes Inside a Cluster</a>. Then, log in to the node where the MRS client is installed.</span>.</li><li id="mrs_01_1183__li1572012314354">Nodes outside the cluster: You can install the client on nodes outside a cluster. <span id="mrs_01_1183__ph2088172916308">For details about how to install a client, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0091.html" target="_blank" rel="noopener noreferrer">Using an MRS Client on Nodes Outside a Cluster</a>, and log in to the node where the MRS client is installed.</span>.</li></ul>
<div class="section" id="mrs_01_1183__section13206075175"><h4 class="sectiontitle">Using the Spark Client</h4><ol id="mrs_01_1183__o65a723c3f0644a5ca5bc95b50051ef8f"><li id="mrs_01_1183__li12507175635615"><span><span id="mrs_01_1183__ph243295392411">Based on the client location, log in to the node where the client is installed. For details, see <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0090.html" target="_blank" rel="noopener noreferrer">Using an MRS Client on Nodes Inside a Cluster</a>, or <a href="https://docs.otc.t-systems.com/usermanual/mrs/mrs_01_0091.html" target="_blank" rel="noopener noreferrer">Using an MRS Client on Nodes Outside a Cluster</a>.</span></span></li><li id="mrs_01_1183__l918962d747ac488fae8d369a02cf0b01"><span>Run the following command to go to the client installation directory:</span><p><p id="mrs_01_1183__ae6fabba0d23e4ae4bc35c6ef79cff605"><strong id="mrs_01_1183__a5a32ca4aacc5401ca0e8e8a12040e570">cd /opt/client</strong></p>
<ul id="mrs_01_1183__ul16756144563213"><li id="mrs_01_1183__li19756184593214">Nodes inside the cluster: After an MRS cluster is created, the client has been installed on the master and core nodes in the cluster by default.</li><li id="mrs_01_1183__li1572012314354">Nodes outside the cluster: You can install the client on nodes outside a cluster.</li></ul>
<div class="section" id="mrs_01_1183__section13206075175"><h4 class="sectiontitle">Using the Spark Client</h4><ol id="mrs_01_1183__o65a723c3f0644a5ca5bc95b50051ef8f"><li id="mrs_01_1183__li12507175635615"><span>Based on the client location, log in to the node where the client is installed.</span></li><li id="mrs_01_1183__l918962d747ac488fae8d369a02cf0b01"><span>Run the following command to go to the client installation directory:</span><p><p id="mrs_01_1183__ae6fabba0d23e4ae4bc35c6ef79cff605"><strong id="mrs_01_1183__a5a32ca4aacc5401ca0e8e8a12040e570">cd /opt/client</strong></p>
</p></li><li id="mrs_01_1183__lf110d935709c4ac48ee0ad1420665c2e"><span>Run the following command to configure environment variables:</span><p><p id="mrs_01_1183__acfff651122a74259a5e01741838c29e9"><strong id="mrs_01_1183__a468a1a99aae541fbab02686dddf0e45a">source bigdata_env</strong></p>
</p></li><li id="mrs_01_1183__lc24dad0c1fb34eec930f24087726a7fd"><span>If the cluster is in security mode, run the following command to authenticate the user. In normal mode, user authentication is not required.</span><p><p id="mrs_01_1183__a2a7d2557b3594004a61ea1dc86fb7d75"><strong id="mrs_01_1183__b1259311118179">kinit</strong> <em id="mrs_01_1183__i660141110175">Component service user</em></p>
</p></li><li id="mrs_01_1183__li6626247810502"><span>Run the Spark shell command. The following provides an example:</span><p><p id="mrs_01_1183__p15071440163915"><strong id="mrs_01_1183__b167772124113">spark-beeline</strong></p>

2
docs/mrs/component-operation-guide/mrs_01_1986.html
View File

@ -34,7 +34,7 @@
</td>
<td class="cellrowborder" valign="top" width="52.129999999999995%" headers="mcps1.3.2.3.2.3.2.4.1.3 "><p id="mrs_01_1986__a290cb3221be84ad2855408409d5e1759">Indicates the maximum value for the broadcast configuration when two tables are joined.</p>
<ul id="mrs_01_1986__u5f348fb92e444d20a94f44704f34c481"><li id="mrs_01_1986__lea2955c681ba48f9a768723d8fc9bf64">When the size of a field in a table involved in an SQL statement is less than the value of this parameter, the system broadcasts the SQL statement.</li><li id="mrs_01_1986__l00dd5db959de444b9b508ec0ba429b67">If the value is set to <strong id="mrs_01_1986__b1391494682116">-1</strong>, broadcast is not performed.</li></ul>
<p id="mrs_01_1986__ad1b84884c4784344a7fc56143fa80598">For details, visit <a href="https://spark.apache.org/docs/3.1.1/sql-programming-guide.html" target="_blank" rel="noopener noreferrer">https://spark.apache.org/docs/3.1.1/sql-programming-guide.html</a>.</p>
<p id="mrs_01_1986__ad1b84884c4784344a7fc56143fa80598">For details, visit <a href="https://archive.apache.org/dist/spark/docs/3.1.1/sql-programming-guide.html" target="_blank" rel="noopener noreferrer">https://archive.apache.org/dist/spark/docs/3.1.1/sql-programming-guide.html</a>.</p>
</td>
</tr>
</tbody>

2
docs/mrs/component-operation-guide/mrs_01_1990.html
View File

@ -3,7 +3,7 @@
<h1 class="topictitle1">Multiple JDBC Clients Concurrently Connecting to JDBCServer</h1>
<div id="body1595920218431"><div class="section" id="mrs_01_1990__se9244b68372944928499ce32dff51467"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1990__ae05b426ffd02443088f37989be8328e2">Multiple clients can be connected to JDBCServer at the same time. However, if the number of concurrent tasks is too large, the default configuration of JDBCServer must be optimized to adapt to the scenario.</p>
</div>
<div class="section" id="mrs_01_1990__sd4eb11d4a4da4433ba638ee8a1dc6c1d"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1990__o721cf9b2d8954d9c8883c62f4af0cdfd"><li id="mrs_01_1990__lcc0bc45459384d8791573751d8996a96">Set the fair scheduling policy of JDBCServer.<div class="p" id="mrs_01_1990__a28b8e936795c4ced9cd3c6481fa0cf07"><a name="mrs_01_1990__lcc0bc45459384d8791573751d8996a96"></a><a name="lcc0bc45459384d8791573751d8996a96"></a>The default scheduling policy of Spark is <strong id="mrs_01_1990__b4557603578">FIFO</strong>, which may cause a failure of short tasks in multi-task scenarios. Therefore, the fair scheduling policy must be used in multi-task scenarios to prevent task failure.<ol type="a" id="mrs_01_1990__o4a1a780875f24401a8a499525e6ab98f"><li id="mrs_01_1990__ld6f4f549733a47b09446c05e98fb4bc9">For details about how to configure Fair Scheduler in Spark, visit <a href="http://spark.apache.org/docs/3.1.1/job-scheduling.html#scheduling-within-an-application" target="_blank" rel="noopener noreferrer">http://spark.apache.org/docs/3.1.1/job-scheduling.html#scheduling-within-an-application</a>.</li><li id="mrs_01_1990__l4cab9a3de03e4eb8aefe85f396efcf04">Configure Fair Scheduler on the JDBC client.<ol class="substepthirdol" id="mrs_01_1990__od906f962b0e44f1688693ce7f120d5b5"><li id="mrs_01_1990__l5a14c99fd3a649ff96046a42237e61fe">In the Beeline command line client or the code defined by JDBC, run the following statement:<p id="mrs_01_1990__ad2fae1cfd15e438886b3d0458ee39f2a"><a name="mrs_01_1990__l5a14c99fd3a649ff96046a42237e61fe"></a><a name="l5a14c99fd3a649ff96046a42237e61fe"></a><strong id="mrs_01_1990__b179168391583">PoolName</strong> is a scheduling pool for Fair Scheduler.</p>
<div class="section" id="mrs_01_1990__sd4eb11d4a4da4433ba638ee8a1dc6c1d"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1990__o721cf9b2d8954d9c8883c62f4af0cdfd"><li id="mrs_01_1990__lcc0bc45459384d8791573751d8996a96">Set the fair scheduling policy of JDBCServer.<div class="p" id="mrs_01_1990__a28b8e936795c4ced9cd3c6481fa0cf07"><a name="mrs_01_1990__lcc0bc45459384d8791573751d8996a96"></a><a name="lcc0bc45459384d8791573751d8996a96"></a>The default scheduling policy of Spark is <strong id="mrs_01_1990__b4557603578">FIFO</strong>, which may cause a failure of short tasks in multi-task scenarios. Therefore, the fair scheduling policy must be used in multi-task scenarios to prevent task failure.<ol type="a" id="mrs_01_1990__o4a1a780875f24401a8a499525e6ab98f"><li id="mrs_01_1990__ld6f4f549733a47b09446c05e98fb4bc9">For details about how to configure Fair Scheduler in Spark, visit <a href="https://archive.apache.org/dist/spark/docs/3.1.1/job-scheduling.html#scheduling-within-an-application" target="_blank" rel="noopener noreferrer">https://archive.apache.org/dist/spark/docs/3.1.1/job-scheduling.html#scheduling-within-an-application</a>.</li><li id="mrs_01_1990__l4cab9a3de03e4eb8aefe85f396efcf04">Configure Fair Scheduler on the JDBC client.<ol class="substepthirdol" id="mrs_01_1990__od906f962b0e44f1688693ce7f120d5b5"><li id="mrs_01_1990__l5a14c99fd3a649ff96046a42237e61fe">In the Beeline command line client or the code defined by JDBC, run the following statement:<p id="mrs_01_1990__ad2fae1cfd15e438886b3d0458ee39f2a"><a name="mrs_01_1990__l5a14c99fd3a649ff96046a42237e61fe"></a><a name="l5a14c99fd3a649ff96046a42237e61fe"></a><strong id="mrs_01_1990__b179168391583">PoolName</strong> is a scheduling pool for Fair Scheduler.</p>
<pre class="screen" id="mrs_01_1990__s3f60da592e9741ac840e55dc23e9667c">SET spark.sql.thriftserver.scheduler.pool=PoolName;</pre>
</li><li id="mrs_01_1990__l1e843234bbb64356a27229f9f1369f45">Run the SQL command. The Spark task will be executed in the preceding scheduling pool.</li></ol>
</li></ol>

2
docs/mrs/component-operation-guide/mrs_01_2019.html
View File

@ -48,7 +48,7 @@ Caused by: java.lang.OutOfMemoryError: Direct buffer memory
</thead>
<tbody><tr id="mrs_01_2019__rceea590307d8479f86ff8eb46ff3c3d5"><td class="cellrowborder" valign="top" width="18.82%" headers="mcps1.3.2.7.2.4.1.1 "><p id="mrs_01_2019__a0048197656934046bdcebe69ac6bb391">GC_OPTS</p>
</td>
<td class="cellrowborder" valign="top" width="56.81%" headers="mcps1.3.2.7.2.4.1.2 "><p id="mrs_01_2019__a6a894f55124f4026846edcf69a66dd10">The GC parameter of YARN NodeManger.</p>
<td class="cellrowborder" valign="top" width="56.81%" headers="mcps1.3.2.7.2.4.1.2 "><p id="mrs_01_2019__a6a894f55124f4026846edcf69a66dd10">The GC parameter of YARN NodeMan<span id="mrs_01_2019__ph5841154745414">a</span>ger.</p>
</td>
<td class="cellrowborder" valign="top" width="24.37%" headers="mcps1.3.2.7.2.4.1.3 "><p id="mrs_01_2019__a309ebedfb6854460a758a4eb3a3bbaf4">128M</p>
</td>

2
docs/mrs/component-operation-guide/mrs_01_2028.html
View File

@ -10,7 +10,7 @@
<p id="mrs_01_2028__a056ff100a29f44b0bacb4065351b8e9b"><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_2028__c1255f1e9d1dc4a079e70a7205293198c">REFRESH TABLE table_name;</span></b></i></p>
<p id="mrs_01_2028__a5046a8e8970d4ee5b89ddfde4a33b8dc"><i><span class="varname" id="mrs_01_2028__v39b941cf88ee46898b3103ea7cc205ca">table_name</span></i> indicates the name of the table to be updated. The table must exist. Otherwise, an error is reported.</p>
<p id="mrs_01_2028__a0140f0b252394bca94563e548a5dca2f">When the query statement is executed, the latest inserted data can be obtained.</p>
<p id="mrs_01_2028__a474b3754466644d3a310d974e2b714b8">For details, visit <a href="https://spark.apache.org/docs/3.1.1/sql-programming-guide.html#metadata-refreshing" target="_blank" rel="noopener noreferrer">https://spark.apache.org/docs/3.1.1/sql-programming-guide.html#metadata-refreshing</a>.</p>
<p id="mrs_01_2028__a474b3754466644d3a310d974e2b714b8">For details, visit <a href="https://archive.apache.org/dist/spark/docs/3.1.1/sql-programming-guide.html#metadata-refreshing" target="_blank" rel="noopener noreferrer">https://archive.apache.org/dist/spark/docs/3.1.1/sql-programming-guide.html#metadata-refreshing</a>.</p>
</div>
</div>
<div>

12
docs/mrs/component-operation-guide/mrs_01_2127.html
View File

@ -35,18 +35,6 @@
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.1.2.2.4.1.2 "><p id="mrs_01_2127__en-us_topic_0264269828_p12811549123617">CentOS 7.6</p>
</td>
</tr>
<tr id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_row176952056184517"><td class="cellrowborder" rowspan="2" valign="top" width="15%" headers="mcps1.3.2.2.1.2.2.4.1.1 "><p id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_p15246194518374">Kunpeng computing (Arm)</p>
</td>
<td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.1.2.2.4.1.2 "><p id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_p126771821194219">Euler</p>
</td>
<td class="cellrowborder" valign="top" width="60%" headers="mcps1.3.2.2.1.2.2.4.1.3 "><p id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_p1329205172516">EulerOS 2.8</p>
</td>
</tr>
<tr id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_row7244555103511"><td class="cellrowborder" valign="top" headers="mcps1.3.2.2.1.2.2.4.1.1 "><p id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_p11246555203516">CentOS</p>
</td>
<td class="cellrowborder" valign="top" headers="mcps1.3.2.2.1.2.2.4.1.2 "><p id="mrs_01_2127__en-us_topic_0264269828_en-us_topic_0270713152_en-us_topic_0264269418_p776320021018">CentOS 7.6</p>
</td>
</tr>
</tbody>
</table>
</div>

87
docs/mrs/component-operation-guide/mrs_01_2311.html
View File

@ -1,87 +0,0 @@
<a name="mrs_01_2311"></a><a name="mrs_01_2311"></a>
<h1 class="topictitle1">Hive Materialized View</h1>
<div id="body1600949903677"><div class="section" id="mrs_01_2311__section059850111710"><h4 class="sectiontitle">Introduction</h4><p id="mrs_01_2311__p1039381719187">A Hive materialized view is a special table obtained based on the query results of Hive internal tables. A materialized view can be considered as an intermediate table that stores actual data and occupies physical space. The tables on which a materialized view depends are called the base tables of the materialized view.</p>
<p id="mrs_01_2311__p1139311716186">Materialized views are used to pre-compute and save the results of time-consuming operations such as table joining or aggregation. When executing a query, you can rewrite the query statement based on the base tables to the query statement based on materialized views. In this way, you do not need to perform time-consuming operations such as join and group by, thereby quickly obtaining the query result.</p>
</div>
<div class="note" id="mrs_01_2311__note1033293241911"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_2311__ul174322569197"><li id="mrs_01_2311__li4432145611911">A materialized view is a special table that stores actual data and occupies physical space.</li><li id="mrs_01_2311__li144321356171916">Before deleting a base table, you must delete the materialized view created based on the base table.</li><li id="mrs_01_2311__li1743211564194">The materialized view creation statement is atomic, which means that other users cannot see the materialized view until all query results are populated.</li><li id="mrs_01_2311__li174321056171912">A materialized view cannot be created based on the query results of another materialized view.</li><li id="mrs_01_2311__li343215569195">A materialized view cannot be created based on the results of a tableless query.</li><li id="mrs_01_2311__li1143265613199">You cannot insert, update, delete, load, or merge materialized views.</li><li id="mrs_01_2311__li16432656171919">You can perform complex query operations on materialized views, because they are special tables in nature.</li><li id="mrs_01_2311__li343215569192">When the data of a base table is updated, you need to manually update the materialized view. Otherwise, the materialized view will retain the old data. That is, the materialized view expires.</li><li id="mrs_01_2311__li643295681917">You can use the describe syntax to check whether the materialized view created based on ACID tables has expired.</li><li id="mrs_01_2311__li17432105651915">The describe statement cannot be used to check whether a materialized view created based on non-ACID tables has expired.</li><li id="mrs_01_2311__li1812415163221">A materialized view can store only ORC files. You can use TBLPROPERTIES ('transactional'='true') to create a transactional Hive internal table.</li></ul>
</div></div>
<div class="section" id="mrs_01_2311__section16112616142011"><h4 class="sectiontitle">Creating a Materialized View</h4><p id="mrs_01_2311__p1892121292118"><strong id="mrs_01_2311__b1889618312206">Syntax</strong></p>
<pre class="screen" id="mrs_01_2311__screen137941843123917">CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db_name.]materialized_view_name
[COMMENT materialized_view_comment]
DISABLE REWRITE
[ROW FORMAT row_format]
[STORED AS file_format]
| STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)]
]
[LOCATION hdfs_path]
[TBLPROPERTIES (property_name=property_value, ...)]
AS
&lt;query&gt;;</pre>
<div class="note" id="mrs_01_2311__note459105802311"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_2311__ul53921213141318"><li id="mrs_01_2311__li20392713131317">Currently, the following materialized view file formats are supported: PARQUET, TextFile, SequenceFile, RCfile, and ORC. If <strong id="mrs_01_2311__b97381537389">STORED AS</strong> is not specified in the creation statement, the default file format is ORC.</li><li id="mrs_01_2311__li82191166130">Names of materialized views must be unique in the same database. Otherwise, you cannot create a new materialized view, and data files of the original materialized view will be overwritten by the data files queried based on the base table in the new one. As a result, data may be tampered with. (After being tampered with, the materialized view can be restored by re-creating the materialized view.).</li></ul>
</div></div>
<p id="mrs_01_2311__p2078165642415"><strong id="mrs_01_2311__b121263818216">Cases</strong></p>
<ol id="mrs_01_2311__ol1560418121252"><li id="mrs_01_2311__li9605181232512"><span>Log in to the Hive client and run the following command to enable the following parameters. For details, see <a href="mrs_01_0952.html">Using a Hive Client</a>.</span><p><p id="mrs_01_2311__p178661236162515"><strong id="mrs_01_2311__b1789812913471">set hive.support.concurrency=true;</strong></p>
<p id="mrs_01_2311__p18665360254"><strong id="mrs_01_2311__b16930112924710">set hive.exec.dynamic.partition.mode=nonstrict;</strong></p>
<p id="mrs_01_2311__p128661360257"><strong id="mrs_01_2311__b193116293475">set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;</strong></p>
</p></li><li id="mrs_01_2311__li19360193116255"><span>Create a base table and insert data.</span><p><pre class="screen" id="mrs_01_2311__screen14110659113915">create table tb_emp(
empno int,ename string,job string,mgr int,hiredate TIMESTAMP,sal float,comm float,deptno int
)stored as orc
tblproperties('transactional'='true');
insert into tb_emp values(7369, 'SMITH', 'CLERK',7902, '1980-12-17 08:30:09',800.00,NULL,20),
(7499, 'ALLEN', 'SALESMAN',7698, '1981-02-20 17:12:00',1600.00,300.00,30),
(7521, 'WARD', 'SALESMAN',7698, '1981-02-22 09:05:34',1250.00,500.00,30),
(7566, 'JONES', 'MANAGER', 7839, '1981-04-02 10:14:13',2975.00,NULL,20),
(7654, 'MARTIN', 'SALESMAN',7698, '1981-09-28 08:36:17',1250.00,1400.00,30),
(7698, 'BLAKE', 'MANAGER',7839, '1981-05-01 11:12:55',2850.00,NULL,30),
(7782, 'CLARK', 'MANAGER',7839, '1981-06-09 15:45:28',2450.00,NULL,10),
(7788, 'SCOTT', 'ANALYST',7566, '1987-04-19 14:05:34',3000.00,NULL,20),
(7839, 'KING', 'PRESIDENT',NULL, '1981-11-17 10:18:25',5000.00,NULL,10),
(7844, 'TURNER', 'SALESMAN',7698, '1981-09-08 09:05:34',1500.00,0.00,30),
(7876, 'ADAMS', 'CLERK',7788, '1987-05-23 15:07:44',1100.00,NULL,20),
(7900, 'JAMES', 'CLERK',7698, '1981-12-03 16:23:56',950.00,NULL,30),
(7902, 'FORD', 'ANALYST',7566, '1981-12-03 08:48:17',3000.00,NULL,20),
(7934, 'MILLER', 'CLERK',7782, '1982-01-23 11:45:29',1300.00,NULL,10);</pre>
</p></li><li id="mrs_01_2311__li20298125819258"><span>Create a materialized view based on the results of the <strong id="mrs_01_2311__b1633934182211">tb_emp</strong> query.</span><p><pre class="screen" id="mrs_01_2311__screen1247817143405">create materialized view group_mv disable rewrite
row format serde 'org.apache.hadoop.hive.serde2.JsonSerDe'
stored as textfile
tblproperties('mv_content'='Total compensation of each department')
as select deptno,sum(sal) sum_sal from tb_emp group by deptno;</pre>
</p></li></ol>
</div>
<div class="section" id="mrs_01_2311__section196123753012"><h4 class="sectiontitle">Applying a Materialized View</h4><p id="mrs_01_2311__p162944544307">Rewrite the query statement based on base tables to the query statement based on materialized views to improve the query efficiency.</p>
<p id="mrs_01_2311__p1910219351805"><strong id="mrs_01_2311__b34962227236">Cases</strong></p>
<p id="mrs_01_2311__p62941754123010">Execute the following query statement:</p>
<p id="mrs_01_2311__p329485453011"><strong id="mrs_01_2311__b362557164810">select deptno,sum(sal) from tb_emp group by deptno having sum(sal)&gt;10000;</strong></p>
<p id="mrs_01_2311__p3294125419306">Based on the created materialized view, rewrite the query statement:</p>
<p id="mrs_01_2311__p429425419303"><strong id="mrs_01_2311__b1499011011480">select deptno, sum_sal from group_mv where sum_sal&gt;10000;</strong></p>
</div>
<div class="section" id="mrs_01_2311__section1320977143114"><h4 class="sectiontitle">Checking a Materialized View</h4><p id="mrs_01_2311__p1431041818318"><strong id="mrs_01_2311__b864151162411">Syntax</strong></p>
<p id="mrs_01_2311__p6763134344912"><strong id="mrs_01_2311__b13814218246">SHOW MATERIALIZED VIEWS [IN database_name] ['identifier_with_wildcards'];</strong></p>
<p id="mrs_01_2311__p14763154354914"><strong id="mrs_01_2311__b12763174318493">DESCRIBE [EXTENDED | FORMATTED] [db_name.]materialized_view_name;</strong></p>
<p id="mrs_01_2311__p15617122211329"><strong id="mrs_01_2311__b1153217303249">Cases</strong></p>
<p id="mrs_01_2311__p96074084811"><strong id="mrs_01_2311__b397144714485">show materialized views;</strong></p>
<p id="mrs_01_2311__p2059184054818"><strong id="mrs_01_2311__b0100114713487">describe formatted group_mv;</strong></p>
</div>
<div class="section" id="mrs_01_2311__section37855329331"><h4 class="sectiontitle">Deleting a Materialized View</h4><p id="mrs_01_2311__p1632118719347"><strong id="mrs_01_2311__b424412404246">Syntax</strong></p>
<p id="mrs_01_2311__p58653219494"><strong id="mrs_01_2311__b392514614499">DROP MATERIALIZED VIEW [db_name.]materialized_view_name;</strong></p>
<p id="mrs_01_2311__p188689513344"><strong id="mrs_01_2311__b8946144212241">Cases</strong></p>
<p id="mrs_01_2311__p17157101614914"><strong id="mrs_01_2311__b89375209490">drop materialized view group_mv;</strong></p>
</div>
<div class="section" id="mrs_01_2311__section648719587351"><h4 class="sectiontitle">Rebuilding a Materialized View</h4><p id="mrs_01_2311__p777231020361">When a materialized view is created, the base table data is filled in the materialized view. However, the data that is added, deleted, or modified in the base table is not automatically synchronized to the materialized view. Therefore, you need to manually rebuild the view after updating the data.</p>
<p id="mrs_01_2311__p6441322133613"><strong id="mrs_01_2311__b17113349255">Syntax</strong></p>
<p id="mrs_01_2311__p9397235185016"><strong id="mrs_01_2311__b18397193565012">ALTER MATERIALIZED VIEW [db_name.]materialized_view_name REBUILD;</strong></p>
<p id="mrs_01_2311__p1452045810363"><strong id="mrs_01_2311__b28907522256">Cases</strong></p>
<p id="mrs_01_2311__p157263436507"><strong id="mrs_01_2311__b4726134325012">alter materialized view group_mv rebuild;</strong></p>
<div class="note" id="mrs_01_2311__note18500812154010"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_2311__p10365121919401">When the base table data is updated but the materialized view data is not updated, the materialized view is in the expired state by default.</p>
<p id="mrs_01_2311__p20365719144018">The describe statement can be used to check whether a materialized view created based on transaction tables has expired. If the value of <strong id="mrs_01_2311__b1712304952611">Outdated for Rewriting</strong> is <strong id="mrs_01_2311__b8209052102617">Yes</strong>, the license has expired. If the value of <strong id="mrs_01_2311__b99287112714">Outdated for Rewriting</strong> is <strong id="mrs_01_2311__b397795132713">No</strong>, the license has not expired.</p>
</div></div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0581.html">Using Hive</a></div>
</div>
</div>

1
docs/mrs/component-operation-guide/mrs_01_2398.html
View File

@ -3,7 +3,6 @@
<h1 class="topictitle1">Creating a ClickHouse Table</h1>
<div id="body0000001080812946"><p id="mrs_01_2398__p208931159813">ClickHouse implements the replicated table mechanism based on the ReplicatedMergeTree engine and ZooKeeper. When creating a table, you can specify an engine to determine whether the table is highly available. Shards and replicas of each table are independent of each other.</p>
<p id="mrs_01_2398__p175761923215">ClickHouse also implements the distributed table mechanism based on the Distributed engine. Views are created on all shards (local tables) for distributed query, which is easy to use. ClickHouse has the concept of data sharding, which is one of the features of distributed storage. That is, parallel read and write are used to improve efficiency.</p>
<p id="mrs_01_2398__p554421512177">The ClickHouse cluster table engine that uses Kunpeng as the CPU architecture does not support HDFS and Kafka.</p>
<div class="section" id="mrs_01_2398__section1386435625"><a name="mrs_01_2398__section1386435625"></a><a name="section1386435625"></a><h4 class="sectiontitle">Viewing cluster and Other Environment Parameters of ClickHouse</h4><ol id="mrs_01_2398__ol1957054154819"><li id="mrs_01_2398__li538035105918"><span>Use the ClickHouse client to connect to the ClickHouse server by referring to <a href="mrs_01_2345.html">Using ClickHouse from Scratch</a>.</span></li><li id="mrs_01_2398__li5153155032517"><a name="mrs_01_2398__li5153155032517"></a><a name="li5153155032517"></a><span>Query the cluster identifier and other information about the environment parameters.</span><p><div class="p" id="mrs_01_2398__p9611951122514"><strong id="mrs_01_2398__b129808844916">select cluster,shard_num,replica_num,host_name from system.clusters;</strong><pre class="screen" id="mrs_01_2398__screen8405017103510">SELECT
cluster,
shard_num,

2
docs/mrs/component-operation-guide/mrs_01_24049.html
View File

@ -27,7 +27,7 @@
</tbody>
</table>
</div>
</p></li><li id="mrs_01_24049__l979a717f376c4ea6990d9e6534d00895"><span>Click <strong id="mrs_01_24049__b8682921272149">OK</strong>. Return to role management page.</span><p><div class="note" id="mrs_01_24049__note264083116421"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_24049__p538616673918">After the FlinkServer role is created, create a FlinkServer user and bind the user to the role and user group. For details, see <span id="mrs_01_24049__ph1294413550112"><a href="https://docs.otc.t-systems.com/usermanual/mrs/admin_guide_000137.html" target="_blank" rel="noopener noreferrer">Creating a User</a></span>.</p>
</p></li><li id="mrs_01_24049__l979a717f376c4ea6990d9e6534d00895"><span>Click <strong id="mrs_01_24049__b8682921272149">OK</strong>. Return to role management page.</span><p><div class="note" id="mrs_01_24049__note264083116421"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_24049__p538616673918">After the FlinkServer role is created, create a FlinkServer user and bind the user to the role and user group.</p>
</div></div>
</p></li></ol>
</div>

75
docs/mrs/component-operation-guide/mrs_01_24119.html
View File

@ -1,75 +0,0 @@
<a name="mrs_01_24119"></a><a name="mrs_01_24119"></a>
<h1 class="topictitle1">GeoMesa Command Line</h1>
<div id="body0000001108144836"><div class="note" id="mrs_01_24119__note743381417182"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_24119__p1043317141187">This section applies only to MRS 3.1.0 or later.</p>
</div></div>
<p id="mrs_01_24119__p8060118">This section describes common GeoMesa commands. For more GeoMesa commands, visit <a href="https://www.geomesa.org/documentation/user/accumulo/commandline.html" target="_blank" rel="noopener noreferrer">https://www.geomesa.org/documentation/user/accumulo/commandline.html</a>.</p>
<p id="mrs_01_24119__p180543719182">After installing the HBase client and loading environment variables, you can use the geomesa-hbase command line.</p>
<ul id="mrs_01_24119__ul4991861483"><li id="mrs_01_24119__li129913617482">Viewing <strong id="mrs_01_24119__b842352706164734">classpath</strong><p id="mrs_01_24119__p115301919183917">After you run the <strong id="mrs_01_24119__b842352706164748">classpath</strong> command, all <strong id="mrs_01_24119__b12208122122414">classpath</strong> information of the current command line tool will be returned.</p>
<p id="mrs_01_24119__p55302019193915"><strong id="mrs_01_24119__b1179383434018">bin/geomesa-hbase classpath</strong></p>
</li><li id="mrs_01_24119__li14991766486">Creating a table<p id="mrs_01_24119__p11523191933916"><a name="mrs_01_24119__li14991766486"></a><a name="li14991766486"></a>Run the <strong id="mrs_01_24119__b842352706164833">create-schema</strong> command to create a table. When creating a table, you need to specify the directory name, table name, and table specifications at least.</p>
<p id="mrs_01_24119__p19436191414016"><strong id="mrs_01_24119__b9564123124011">bin/geomesa-hbase create-schema -c geomesa -f test -s Who:String,What:java.lang.Long,When:Date,*Where:Point:srid=4326,Why:String</strong></p>
</li></ul>
<ul id="mrs_01_24119__ul193931545194719"><li id="mrs_01_24119__li1039384514719">Describing a table<p id="mrs_01_24119__p1652351993917"><a name="mrs_01_24119__li1039384514719"></a><a name="li1039384514719"></a>Run the <strong id="mrs_01_24119__b842352706165041">describe-schema</strong> command to obtain table descriptions. When describing a table, you need to specify the directory name and table name.</p>
<p id="mrs_01_24119__p65231519133915"><strong id="mrs_01_24119__b1523719183917">bin/geomesa-hbase describe-schema -c geomesa -f test</strong></p>
</li><li id="mrs_01_24119__li43931045134718">Importing data in batches<p id="mrs_01_24119__p1852371910393"><a name="mrs_01_24119__li43931045134718"></a><a name="li43931045134718"></a>Run the <strong id="mrs_01_24119__b842352706165232">ingest</strong> command to import data in batches. When importing data, you need to specify the directory name, table name, table specifications, and the related data converter.</p>
<p id="mrs_01_24119__p17523819193915">The data in the <strong id="mrs_01_24119__b9929491883">data.csv</strong> file contains license plate number, vehicle color, longitude, latitude, and time. Save the data table to the folder.</p>
<pre class="screen" id="mrs_01_24119__screen13523319183919">AAA,red,113.918417,22.505892,2017-04-09 18:03:46
BBB,white,113.960719,22.556511,2017-04-24 07:38:47
CCC,blue,114.088333,22.637222,2017-04-23 15:07:54
DDD,yellow,114.195456,22.596103,2017-04-21 21:27:06
EEE,black,113.897614,22.551331,2017-04-09 09:34:48</pre>
<p id="mrs_01_24119__p5523181917394">Table structure definition: <strong id="mrs_01_24119__b84235270614151">myschema.sft</strong>. Save <strong id="mrs_01_24119__b842352706141523">myschema.sft</strong> to the <strong id="mrs_01_24119__b842352706141532">conf</strong> folder of the GeoMesa command line tool.</p>
<pre class="screen" id="mrs_01_24119__screen4524141923910">geomesa.sfts.cars = {
attributes = [
{ name = "carid", type = "String", index = true }
{ name = "color", type = "String", index = false }
{ name = "time", type = "Date", index = false }
{ name = "geom", type = "Point", index = true,srid = 4326,default = true }
]
}</pre>
<p id="mrs_01_24119__p1952431915392">Converter definition: <strong id="mrs_01_24119__b842352706141556">myconvertor.convert</strong> Save <strong id="mrs_01_24119__b842352706141610">myconvertor.convert</strong> to the <strong id="mrs_01_24119__b842352706141616">conf</strong> folder of the GeoMesa command line tool.</p>
<pre class="screen" id="mrs_01_24119__screen20525161933911">geomesa.converters.cars= {
type = "delimited-text",
format = "CSV",
id-field = "$fid",
fields = [
{ name = "fid", transform = "concat($1,$5)" }
{ name = "carid", transform = "$1::string" }
{ name = "color", transform = "$2::string" }
{ name = "lon", transform = "$3::double" }
{ name = "lat", transform = "$4::double" }
{ name = "geom", transform = "point($lon,$lat)" }
{ name = "time", transform = "date('YYYY-MM-dd HH:mm:ss',$5)" }
]
}</pre>
<p id="mrs_01_24119__p18525131983913">Run the following command to import data:</p>
<p id="mrs_01_24119__p4525919153916"><strong id="mrs_01_24119__b15525719173910">bin/geomesa-hbase ingest -c geomesa -C conf/myconvertor.convert -s conf/myschema.sft data/data.csv</strong></p>
<p id="mrs_01_24119__p15608182054117">For details about other parameters for importing data, visit <a href="https://www.geomesa.org/documentation/user/accumulo/examples.html#ingesting-data" target="_blank" rel="noopener noreferrer">https://www.geomesa.org/documentation/user/accumulo/examples.html#ingesting-data</a>.</p>
</li><li id="mrs_01_24119__li10394545164719">Querying explanations<p id="mrs_01_24119__p20529181920393"><a name="mrs_01_24119__li10394545164719"></a><a name="li10394545164719"></a>Run the <strong id="mrs_01_24119__b84235270616588">explain</strong> command to obtain execution plan explanations of the specified query statement. You need to specify the directory name, table name, and query statement.</p>
<p id="mrs_01_24119__p325383694210"><strong id="mrs_01_24119__b13970439184210">bin/geomesa-hbase explain -c geomesa -f cars -q "carid = 'BBB'"</strong></p>
</li><li id="mrs_01_24119__li13394144514712">Analyzing statistics<p id="mrs_01_24119__p1552991933912"><a name="mrs_01_24119__li13394144514712"></a><a name="li13394144514712"></a>Run the <strong id="mrs_01_24119__b84235270617543">stats-analyze</strong> command to conduct statistical analysis on the data table. In addition, you can run the <strong id="mrs_01_24119__b84235270617627">stats-bounds</strong>, <strong id="mrs_01_24119__b84235270617632">stats-count</strong>, <strong id="mrs_01_24119__b84235270617638">stats-histogram</strong>, and <strong id="mrs_01_24119__b84235270617644">stats-top-k</strong> commands to collect more detailed statistics on the data table.</p>
<p id="mrs_01_24119__p031911224417"><strong id="mrs_01_24119__b8992238134419">bin/geomesa-hbase stats-analyze -c geomesa -f cars</strong></p>
<p id="mrs_01_24119__p1291811134418"><strong id="mrs_01_24119__b79999386447">bin/geomesa-hbase stats-bounds -c geomesa -f cars</strong></p>
<p id="mrs_01_24119__p23966183442"><strong id="mrs_01_24119__b711399444">bin/geomesa-hbase stats-count -c geomesa -f cars</strong></p>
<p id="mrs_01_24119__p928714266449"><strong id="mrs_01_24119__b173939184410">bin/geomesa-hbase stats-histogram -c geomesa -f cars</strong></p>
<p id="mrs_01_24119__p108243328449"><strong id="mrs_01_24119__b13513934410">bin/geomesa-hbase stats-top-k -c geomesa -f cars</strong></p>
</li><li id="mrs_01_24119__li4394545114715">Exporting a feature<p id="mrs_01_24119__p2529181953917"><a name="mrs_01_24119__li4394545114715"></a><a name="li4394545114715"></a>Run the <strong id="mrs_01_24119__b84235270617810">export</strong> command to export a feature. When exporting the feature, you must specify the directory name and table name. In addition, you can specify a query statement to export the feature.</p>
<p id="mrs_01_24119__p1199734184510"><strong id="mrs_01_24119__b12961191453">bin/geomesa-hbase export -c geomesa -f cars -q "carid = 'BBB'"</strong></p>
</li><li id="mrs_01_24119__li183942455473">Deleting a feature<p id="mrs_01_24119__p25291919193914"><a name="mrs_01_24119__li183942455473"></a><a name="li183942455473"></a>Run the <strong id="mrs_01_24119__b200895014">delete-features</strong> command to delete a feature. When deleting the feature, you must specify the directory name and table name. In addition, you can specify a query statement to delete the feature.</p>
<p id="mrs_01_24119__p10711181134611"><strong id="mrs_01_24119__b1642641564619">bin/geomesa-hbase delete-features -c geomesa -f cars -q "carid = 'BBB'"</strong></p>
</li><li id="mrs_01_24119__li739417455473">Obtain the names of all tables in the directory.<p id="mrs_01_24119__p1352941943917"><a name="mrs_01_24119__li739417455473"></a><a name="li739417455473"></a>Run the <strong id="mrs_01_24119__b842352706171034">get-type-names</strong> command to obtain the names of tables in the specified directory.</p>
<p id="mrs_01_24119__p514444618468"><strong id="mrs_01_24119__b104311849114610">bin/geomesa-hbase get-type-names -c geomesa</strong></p>
</li><li id="mrs_01_24119__li19394164515474">Deleting a table<p id="mrs_01_24119__p1053012199392"><a name="mrs_01_24119__li19394164515474"></a><a name="li19394164515474"></a>Run the <strong id="mrs_01_24119__b842352706171115">remove-schema</strong> command to delete a table. You need to specify the directory name and table name at least.</p>
<p id="mrs_01_24119__p17339537478"><strong id="mrs_01_24119__b1739114134710">bin/geomesa-hbase remove-schema -c geomesa -f test</strong></p>
<p id="mrs_01_24119__p1269516924720"><strong id="mrs_01_24119__b154331414720">bin/geomesa-hbase remove-schema -c geomesa -f cars</strong></p>
</li><li id="mrs_01_24119__li53943455472">Deleting a catalog<p id="mrs_01_24119__p125308191393"><a name="mrs_01_24119__li53943455472"></a><a name="li53943455472"></a>Run the <strong id="mrs_01_24119__b842352706171238">delete-catalog</strong> command to delete the specified catalog.</p>
<p id="mrs_01_24119__p13752193620477"><strong id="mrs_01_24119__b18314144044720">bin/geomesa-hbase delete-catalog -c geomesa</strong></p>
</li></ul>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0500.html">Using HBase</a></div>
</div>
</div>

2
docs/mrs/component-operation-guide/mrs_01_24198.html
View File

@ -4,7 +4,7 @@
<div id="body0000001136546900"><p id="mrs_01_24198__p176811527111117">The ClickHouse data migration tool can migrate some partitions of one or more partitioned MergeTree tables on several ClickHouseServer nodes to the same tables on other ClickHouseServer nodes. In the capacity expansion scenario, you can use this tool to migrate data from an original node to a new node to balance data after capacity expansion.</p>
<div class="section" id="mrs_01_24198__section135233106402"><h4 class="sectiontitle">Prerequisites</h4><ul id="mrs_01_24198__ul1366411569378"><li id="mrs_01_24198__li16664115614371">The ClickHouse and Zookeeper services are running properly. The ClickHouseServer instances on the source and destination nodes are normal.</li><li id="mrs_01_24198__li13180716392">The destination node has the data table to be migrated and the table is a partitioned MergeTree table.</li><li id="mrs_01_24198__li68491131104015">Before creating a migration task, ensure that all tasks for writing data to a table to be migrated have been stopped. After the task is started, you can only query the table to be migrated and cannot write data to or delete data from the table. Otherwise, data may be inconsistent before and after the migration.</li><li id="mrs_01_24198__li1631232264013">The ClickHouse data directory on the destination node has sufficient space.</li></ul>
</div>
<div class="section" id="mrs_01_24198__section1041216526381"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24198__ol466517515449"><li id="mrs_01_24198__li17664185174415"><span>Log in to Manager and choose <strong id="mrs_01_24198__b36417385543741">Cluster</strong> &gt; <strong id="mrs_01_24198__b16015138843741">Services</strong> &gt; <strong id="mrs_01_24198__b25756644243741">ClickHouse</strong>. On the ClickHouse service page, click the <strong id="mrs_01_24198__b954828643741">Data Migration</strong> tab.</span><p><p id="mrs_01_24198__p2664135164419"></p>
<div class="section" id="mrs_01_24198__section1041216526381"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24198__ol466517515449"><li id="mrs_01_24198__li17664185174415"><span>Log in to Manager and choose <strong id="mrs_01_24198__b36417385543741">Cluster</strong> &gt; <strong id="mrs_01_24198__b16015138843741">Services</strong> &gt; <strong id="mrs_01_24198__b25756644243741">ClickHouse</strong>. On the ClickHouse service page, click the <strong id="mrs_01_24198__b954828643741">Data Migration</strong> tab.</span><p><p id="mrs_01_24198__p2664135164419"><span><img id="mrs_01_24198__image8865109236" src="en-us_image_0000001533052357.png"></span></p>
</p></li><li id="mrs_01_24198__li106653518442"><span>Click <strong id="mrs_01_24198__b185335751243741">Add Task</strong>.</span><p><p id="mrs_01_24198__p4665251184418"></p>
</p></li></ol><ol start="3" id="mrs_01_24198__ol266495154417"><li id="mrs_01_24198__li18211183015203"><span>On the page for creating a migration task, set the migration task parameters. For details, see <a href="#mrs_01_24198__table1724256152117">Table 1</a>.</span><p><p id="mrs_01_24198__p131077571306"></p>