forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
30 lines
9.9 KiB
HTML
30 lines
9.9 KiB
HTML
<a name="mrs_01_1750"></a><a name="mrs_01_1750"></a>
|
|
|
|
<h1 class="topictitle1">Switching the Hive Execution Engine to Tez</h1>
|
|
<div id="body1594977624481"><div class="section" id="mrs_01_1750__section897072914375"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1750__p162521931153713">Hive can use the Tez engine to process data computing tasks. Before executing a task, you can manually switch the execution engine to Tez.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1750__section13562315184415"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_1750__p0312171854420">The TimelineServer role of the Yarn service has been installed in the cluster and is running properly.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1750__section846520364375"><h4 class="sectiontitle">Switching the Execution Engine on the Client to Tez</h4><ol id="mrs_01_1750__ol793491613427"><li id="mrs_01_1750__li993314161422"><span>Install and log in to the Hive client. For details, see <a href="mrs_01_0952.html">Using a Hive Client</a>.</span></li><li id="mrs_01_1750__li148931227185116"><span>Run the following commands to switch the engine and enable the <strong id="mrs_01_1750__b113785319154">yarn.timeline-service.enabled</strong> parameter:</span><p><p id="mrs_01_1750__p13611171212523"><strong id="mrs_01_1750__b94461916155212">set hive.execution.engine=tez</strong>;</p>
|
|
<div class="p" id="mrs_01_1750__p17662022135220"><strong id="mrs_01_1750__b980111259523">set yarn.timeline-service.enabled=true</strong>;<div class="note" id="mrs_01_1750__note1793421624218"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1750__ul9879133013507"><li id="mrs_01_1750__li68798303503">After <strong id="mrs_01_1750__b21906718152059">yarn.timeline-service.enabled</strong> is enabled, you can view the details about the tasks executed by the Tez engine on TezUI. After this function is enabled, task information will be reported to TimelineServer. If the TimelineServer instance is faulty, the task will fail.</li><li id="mrs_01_1750__li8631534145017">Tez uses the ApplicationMaster buffer pool. Therefore, <strong id="mrs_01_1750__b119630607052059">yarn.timeline-service.enabled</strong> must be enabled before Tez tasks are submitted. Otherwise, this parameter cannot take effect and you need to log in to the client again to configure it.</li><li id="mrs_01_1750__li9702193715506">When the execution engine needs to be switched to another engine, you need to run the <strong id="mrs_01_1750__b23780482152059">set yarn.timeline-service.enabled=false</strong> command on the client to disable the <strong id="mrs_01_1750__b175673868352059">yarn.timeline-service.enabled</strong> parameter.</li><li id="mrs_01_1750__li11456418151310">To specify a Yarn running queue, run the <strong id="mrs_01_1750__b1917919231">set tez.queue.name=default</strong> command on the client.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
</p></li><li id="mrs_01_1750__li293415164427"><span>Submit and execute the Tez tasks.</span></li><li id="mrs_01_1750__li10244134214144"><span>Log in to FusionInsight Manager. For details, see <a href="mrs_01_2124.html">Accessing FusionInsight Manager (MRS 3.x or Later)</a>. Choose <strong id="mrs_01_1750__b1769815119396">Cluster</strong> > <em id="mrs_01_1750__i15161142923919">Name of the desired cluster</em> > <strong id="mrs_01_1750__b2662193613916">Services</strong> > <strong id="mrs_01_1750__b16851738113915">Tez</strong> > <strong id="mrs_01_1750__b19146842183918">TezUI</strong><em id="mrs_01_1750__i4683736115112">(host name)</em> to view the task execution status on the TezUI page.</span><p><p id="mrs_01_1750__p7860946121614">For versions earlier than MRS 3.x, log in to MRS Manager, choose <strong id="mrs_01_1750__b17805185031119">Services</strong>, and click <strong id="mrs_01_1750__b9681154171116">Tez</strong>. On the displayed page, click the link next to <strong id="mrs_01_1750__b177921184131">Tez WebUI</strong> to view the task execution status on the TezUI page.</p>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="mrs_01_1750__section64712291238"><h4 class="sectiontitle">Switching the Default Execution Engine of Hive to Tez</h4><ol id="mrs_01_1750__ol15825141020156"><li id="mrs_01_1750__li168251010121510"><span>Log in to FusionInsight Manager. For details, see <a href="mrs_01_2124.html">Accessing FusionInsight Manager (MRS 3.x or Later)</a>. Choose <strong id="mrs_01_1750__b152634501316">Cluster</strong> > <em id="mrs_01_1750__i1931153322">Name of the desired cluster</em> > <strong id="mrs_01_1750__b10469171053215">Services</strong> > <strong id="mrs_01_1750__b660211273216">Hive</strong> > <strong id="mrs_01_1750__b202594182326">Configurations</strong> > <strong id="mrs_01_1750__b10984220163215">All Configurations</strong> > <strong id="mrs_01_1750__b1572622318323">HiveServer(Role)</strong>, and search for <strong id="mrs_01_1750__b1282162912326">hive.execution.engine</strong>.</span><p><p id="mrs_01_1750__p1198191115393">For versions earlier than MRS 3.x, log in to MRS Manager, choose <strong id="mrs_01_1750__b874372992419">Services</strong>, and click <strong id="mrs_01_1750__b989313192417">Hive</strong>. On the displayed page, click the <strong id="mrs_01_1750__b17485195292416">Service Configuration</strong> tab, select <strong id="mrs_01_1750__b20303171319250">All</strong> from the <strong id="mrs_01_1750__b893712324016">Type</strong> drop-down list. On the navigation pane on the left, choose <strong id="mrs_01_1750__b2054229152518">HiveServer</strong> and search for <strong id="mrs_01_1750__b18337124116251">hive.execution.engine</strong>.</p>
|
|
</p></li><li id="mrs_01_1750__li1882531071512"><span>Set <strong id="mrs_01_1750__b22244592352059">hive.execution.engine</strong> to <strong id="mrs_01_1750__b80044796152059">tez</strong>.</span></li><li id="mrs_01_1750__li464525410155"><span>Choose <strong id="mrs_01_1750__b1993359121512">Hive(Service)</strong> > <strong id="mrs_01_1750__b46157571613">Customization</strong> and search for <strong id="mrs_01_1750__b53540288166">yarn.site.customized.configs</strong>.</span></li><li id="mrs_01_1750__li17324913111616"><span>Add custom parameter <strong id="mrs_01_1750__b49933271196">yarn.timeline-service.enabled</strong> to <strong id="mrs_01_1750__b151061644161914">yarn.site.customized.configs</strong> and set it to <strong id="mrs_01_1750__b1387514817194">true</strong>.</span><p><div class="note" id="mrs_01_1750__note15486172920166"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1750__ul712184482315"><li id="mrs_01_1750__li10131044142310">After <strong id="mrs_01_1750__b199338915752059">yarn.timeline-service.enabled</strong> is enabled, you can view the details about the tasks executed by the Tez engine on TezUI. After this function is enabled, task information will be reported to TimelineServer. If the TimelineServer instance is faulty, the task will fail.</li><li id="mrs_01_1750__li613174422315">Tez uses the ApplicationMaster buffer pool. Therefore, <strong id="mrs_01_1750__b149005368652059">yarn.timeline-service.enabled</strong> must be enabled before Tez tasks are submitted. Otherwise, this parameter cannot take effect and you need to log in to the client again to configure it.</li><li id="mrs_01_1750__li2131445239">When the execution engine needs to be switched to another one, you need to set the value of parameter <strong id="mrs_01_1750__b48245653552059">yarn.timeline-service.enabled</strong> to <strong id="mrs_01_1750__b182103597552059">false</strong>.</li></ul>
|
|
</div></div>
|
|
</p></li><li id="mrs_01_1750__li731417703218"><span>Click <strong id="mrs_01_1750__b104472399852059">Save</strong>. In the displayed confirmation dialog box, click <strong id="mrs_01_1750__b194062795052059">OK</strong>. </span><p><p id="mrs_01_1750__p38466132488">For versions earlier than MRS 3.x, click <strong id="mrs_01_1750__b28071249172519">Save Configuration</strong> and click <strong id="mrs_01_1750__b976575610255">Yes</strong> in the displayed dialog box.</p>
|
|
</p></li><li id="mrs_01_1750__li9486529191619"><span>Choose <strong id="mrs_01_1750__b3451223338">Dashboard</strong> > <strong id="mrs_01_1750__b1978911241031">More</strong> > <strong id="mrs_01_1750__b963113267314">Restart Service</strong> to restart the Hive service. Enter the password to restart the service.</span><p><p id="mrs_01_1750__p066395014495">For versions earlier than MRS 3.x, Click the <strong id="mrs_01_1750__b11421135616268">Service Status</strong> tab and choose <strong id="mrs_01_1750__b8975121102619">More</strong> > <strong id="mrs_01_1750__b392813247266">Restart Service</strong> to restart the Hive service.</p>
|
|
</p></li><li id="mrs_01_1750__li61597871717"><span>Install and log in to the Hive client. For details, see <a href="mrs_01_0952.html">Using a Hive Client</a>.</span></li><li id="mrs_01_1750__li1196182121714"><span>Submit and execute the Tez tasks.</span></li><li id="mrs_01_1750__li54945290176"><span>Log in to FusionInsight Manager and choose <strong id="mrs_01_1750__b66646603952059">Cluster</strong> > <em id="mrs_01_1750__i213196265152059">Name of the desired cluster</em> > <strong id="mrs_01_1750__b103139696052059">Services</strong> > <strong id="mrs_01_1750__b87745831352059">Tez</strong> > <strong id="mrs_01_1750__b147942328952059">TezUI</strong><em id="mrs_01_1750__i1471315113527">(host name)</em>. On the displayed TezUI page, view the task execution status.</span><p><p id="mrs_01_1750__p59422515115">For versions earlier than MRS 3.x, log in to MRS Manager, choose <strong id="mrs_01_1750__b08014302410">Services</strong>, and click <strong id="mrs_01_1750__b1080193017419">Tez</strong>. On the displayed page, click the link next to <strong id="mrs_01_1750__b13800308414">Tez WebUI</strong> to view the task execution status on the TezUI page.</p>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0581.html">Using Hive</a></div>
|
|
</div>
|
|
</div>
|
|
|