forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
23 lines
3.5 KiB
HTML
23 lines
3.5 KiB
HTML
<a name="mrs_01_0873"></a><a name="mrs_01_0873"></a>
|
|
|
|
<h1 class="topictitle1">Setting the Task Priority</h1>
|
|
<div id="body1590130748024"><div class="section" id="mrs_01_0873__s830e96ae382f4a5b9084c15688c7034f"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0873__a3e568f794823438385e8a976f8d86a50">The resource contention scenarios of a cluster are as follows:</p>
|
|
<ol id="mrs_01_0873__oa3f6b6ce788d4d5b8ba0e7c2dea31655"><li id="mrs_01_0873__l8e26cf9fa23e4d82bd92b3d95d450818">Submit two jobs (Job 1 and Job 2) with lower priorities.</li><li id="mrs_01_0873__l18167a60b6d14733bada188a79edf2de">Some tasks of running Job 1 and Job 2 are in the running state. However, some tasks are pending due to resource deficiency because the capacity of cluster or queue resources is limited.</li><li id="mrs_01_0873__l83378b1d9a7d4c0fbd4c456797ae113f">Submit a job (Job 3) with a higher priority. In this case, after the running tasks of Job 1 and Job 2 are complete, their resources will be released and then allocated to the pending tasks of Job 3.</li><li id="mrs_01_0873__ld16626506f9644cd9ef0604d1d1c1e76">After Job 3 is complete, its resources will be released and then allocated to Job 1 and Job 2.</li></ol>
|
|
<p id="mrs_01_0873__ad6b4878912394d25ac80177f30c7bfd7">Users can use capacity scheduler of ResourceManager to set the task priority in Yarn because the task priority is implemented by the scheduler of ResourceManager.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0873__s5a3a6e13efc64f11a5e75ab75ec0ef5e"><h4 class="sectiontitle">Procedure</h4><p id="mrs_01_0873__a5ca2f5a0a6b74368939853aa4c6020b8">Set the <span class="parmname" id="mrs_01_0873__p31ab55893f17469d9f3615469db0e801"><b>mapreduce.job.priority</b></span> parameter and use CLI or API to set the task priority.</p>
|
|
<ul id="mrs_01_0873__ueac9cc967e3d4d44bfa98efb063e95e0"><li id="mrs_01_0873__l7c5667427b2546e0990a1049770bb9a4">Through the CLI<p id="mrs_01_0873__a734e1910b2e8416286da553f3406b30c"><a name="mrs_01_0873__l7c5667427b2546e0990a1049770bb9a4"></a><a name="l7c5667427b2546e0990a1049770bb9a4"></a>When submitting tasks, add the <strong id="mrs_01_0873__b1259171524818">-Dmapreduce.job.priority=</strong><em id="mrs_01_0873__i430141114717"><priority></em> parameter.</p>
|
|
<p id="mrs_01_0873__add3d758ca813435daaba56053b17581b"><em id="mrs_01_0873__i1240417105"><priority></em> can be set to any of the following values:</p>
|
|
<ul id="mrs_01_0873__udf1e2db1ba68484dac2429e159c24704"><li id="mrs_01_0873__l755bd04fb1e547c9b5c01879964439f9">VERY_HIGH</li><li id="mrs_01_0873__la7aa109eef13496f944fe8d195246898">HIGH</li><li id="mrs_01_0873__ld13c0e27bd034d20baad7c17911df4f7">NORMAL</li><li id="mrs_01_0873__le1b5a71b89dd46b794214755fd28eeab">LOW</li><li id="mrs_01_0873__l01ab45ebbe8f4b6b9fe99edcb7602719">VERY_LOW</li></ul>
|
|
</li><li id="mrs_01_0873__l0cbeea36cf0145a58428fd51f4c51138">Through the API<p id="mrs_01_0873__a230c46e950c8499db279beb3e470d869"><a name="mrs_01_0873__l0cbeea36cf0145a58428fd51f4c51138"></a><a name="l0cbeea36cf0145a58428fd51f4c51138"></a>You can also set the task priority through the API.</p>
|
|
<p id="mrs_01_0873__ae6ca094e2ef84d69af93f060b1953ffd">Set <strong id="mrs_01_0873__ac7559d04f7cd4c8b9ec4d34e97e8cd01">Configuration.set("mapreduce.job.priority", <priority>)</strong> or <strong id="mrs_01_0873__a69fe821c945d4fca94101efd5ad4c235">Job.setPriority(JobPriority priority)</strong>.</p>
|
|
</li></ul>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0871.html">Yarn Performance Tuning</a></div>
|
|
</div>
|
|
</div>
|
|
|