doc-exports/docs/mrs/component-operation-guide/mrs_01_0858.html

<a name="mrs_01_0858"></a><a name="mrs_01_0858"></a>

<h1 class="topictitle1">Configuring Container Log Aggregation</h1>
<div id="body1590130746693"><div class="section" id="mrs_01_0858__s84af02b3a1434387b5b90b62364a2ef2"><h4 class="sectiontitle">Scenario</h4><div class="p" id="mrs_01_0858__a4f68a44f4ad84643be1dddfe8dafef3f">Yarn provides the container log aggregation function to collect logs generated by containers on each node to HDFS to release local disk space. You can collect logs in either of the following ways:<ul id="mrs_01_0858__uae80f2b5994d43eda36e2e24f6de36db"><li id="mrs_01_0858__l5f9709ad91354596bfae51342ce2f78c">After the application is complete, collect container logs to HDFS at a time.</li><li id="mrs_01_0858__lb75a48b95c8a4bea863d764b0beedeb9">During application running, periodically collect log segments generated by containers and save them to HDFS.</li></ul>
</div>
</div>
<div class="section" id="mrs_01_0858__s687c781bede440e887c069f6a5d4064e"><h4 class="sectiontitle">Configuration Description</h4><p id="mrs_01_0858__a92550ce6bd694b7db0faeb54173387c0"><strong id="mrs_01_0858__b7638753135612">Navigation path for setting parameters:</strong></p>
<p id="mrs_01_0858__p11156202312364">Go to the <strong id="mrs_01_0858__b7180911708298">All Configurations</strong> tab page of YARN, enter the parameters listed in <a href="#mrs_01_0858__te1c6a71343c044a9a63faeb4a644fb5a">Table 1</a> in the search box, modify the parameters by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>, and save the configuration. On the <strong id="mrs_01_0858__b926963998298">Dashboard</strong> tab page, choose <strong id="mrs_01_0858__b21379769208298">More</strong> &gt; <strong id="mrs_01_0858__b18550597588298">Synchronize Configuration</strong>. After the synchronization is complete, restart the YARN service.</p>
<p id="mrs_01_0858__a9eab391ff22f449e84a893fc1d4916bf">The <span class="parmname" id="mrs_01_0858__p75825fad85c447fbbd5a9ebce41816f3"><b>yarn.nodemanager.remote-app-log-dir-suffix</b></span> parameter must be configured on the Yarn client. The configurations on the ResourceManager, NodeManager, and JobHistory nodes must be the same as those on the Yarn client.</p>
<p id="mrs_01_0858__a1b55436b8f4b456f87ef519ce13cabd9">The periodic log collection function applies only to MapReduce applications, for which rolling output of log files must be configured. <a href="#mrs_01_0858__t969e9ceee95b43149610e8f72cf9d91f">Table 3</a> describes the configurations in the <span class="filepath" id="mrs_01_0858__ff039160e974143f4b615e35ee59b41af"><b>mapred-site.xml</b></span> configuration file on the MapReduce client node.</p>

<div class="tablenoborder"><a name="mrs_01_0858__te1c6a71343c044a9a63faeb4a644fb5a"></a><a name="te1c6a71343c044a9a63faeb4a644fb5a"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0858__te1c6a71343c044a9a63faeb4a644fb5a" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_0858__r95f761897937458f85ea80c6af07ee56"><th align="left" class="cellrowborder" valign="top" width="28.16%" id="mcps1.3.2.6.2.4.1.1"><p id="mrs_01_0858__a0bf9605f26ce45889adb508d293dd0fc">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="56.67%" id="mcps1.3.2.6.2.4.1.2"><p id="mrs_01_0858__a5600d03210644c0f919c0244fe148eaf">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.17%" id="mcps1.3.2.6.2.4.1.3"><p id="mrs_01_0858__aee2c3b64b53b49f28bcbfd7d42704c3f">Default Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0858__r38356de382164685997daa173de63d3e"><td class="cellrowborder" valign="top" width="28.16%" headers="mcps1.3.2.6.2.4.1.1 "><p id="mrs_01_0858__ad83df65a6a0a436e9bbcb588c8e24a88">yarn.log-aggregation-enable</p>
</td>
<td class="cellrowborder" valign="top" width="56.67%" headers="mcps1.3.2.6.2.4.1.2 "><p id="mrs_01_0858__ae12830b272584222a0980980af2c1d36">Whether to enable container log aggregation</p>
<ul id="mrs_01_0858__u8542f5f0f09f45609ee4a47550bd40df"><li id="mrs_01_0858__l124384e338e94ef5ab550a63c19d180b">If this parameter is set to <span class="parmvalue" id="mrs_01_0858__pd672c2609bb444cdbc769e9bfb2ef143"><b>true</b></span>, logs are collected to the HDFS directory.</li><li id="mrs_01_0858__lb975d56d2c8348a0967bd74f0c433c0b">If this parameter is set to <span class="parmvalue" id="mrs_01_0858__p4ae98e68aed54cf88e9774830861cf68"><b>false</b></span>, the function is disabled, and logs are not collected to HDFS.</li></ul>
<p id="mrs_01_0858__a4e345c977dec4972af0e5f56c0edac15">After changing the parameter value, restart the Yarn service for the setting to take effect.</p>
<div class="note" id="mrs_01_0858__n227ba0aa068c4dab896125a992978f9f"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="mrs_01_0858__en-us_topic_0039590391_ul7724921798"><li id="mrs_01_0858__en-us_topic_0039590391_li99858521798">The container logs that are generated before the parameter is set to <span class="parmvalue" id="mrs_01_0858__p8ed8ff2f7dba42b4a852b01687c78bd5"><b>false</b></span> and the setting takes effect cannot be obtained from the web UI.</li><li id="mrs_01_0858__l920623ce9a7d48c487e1179c9ad6f546">If you need to view the logs generated before on the web UI, you are advised to set this parameter to <span class="parmvalue" id="mrs_01_0858__p8a2ea582fa174cbaa7df1b7289c6d5e0"><b>true</b></span>.</li></ul>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="15.17%" headers="mcps1.3.2.6.2.4.1.3 "><p id="mrs_01_0858__abfbdb96c60464259b8db07b47ecca81c">true</p>
</td>
</tr>
<tr id="mrs_01_0858__r3fcd7b2b5f814ff8ba54ca70feacfcf4"><td class="cellrowborder" valign="top" width="28.16%" headers="mcps1.3.2.6.2.4.1.1 "><p id="mrs_01_0858__a539f6727d3034e7baefca7408dfc6ecd">yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds</p>
</td>
<td class="cellrowborder" valign="top" width="56.67%" headers="mcps1.3.2.6.2.4.1.2 "><p id="mrs_01_0858__aaac3c9173b6940ec90d2d9534cbd621b">Interval for NodeManager to periodically collect logs</p>
<ul id="mrs_01_0858__u983ef8b1e1b54f1ab4002b53e244232f"><li id="mrs_01_0858__l5417ce48b91f4c86b3d988d6817fdb3d">If this parameter is set to <strong id="mrs_01_0858__b36873333153">-1</strong> or <strong id="mrs_01_0858__b13786143561520">0</strong>, periodic log collection is disabled. Logs are collected at a time after application running is complete.</li><li id="mrs_01_0858__l2c11b28c2c97412bab00e867c4c80366">The minimum collection interval can be set to 3,600 seconds. If this parameter is set to a value greater than 0 and less than 3,600, the collection interval is 3,600 seconds.</li></ul>
<p id="mrs_01_0858__p11141183941015">Interval for NodeManager to wake up and upload logs. If this parameter is set to <strong id="mrs_01_0858__b194932314413">-1</strong> or <strong id="mrs_01_0858__b11550174904414">0</strong>, rolling monitoring is disabled and logs are aggregated when the application task is complete. The value must be greater than or equal to <strong id="mrs_01_0858__b836615004617">-1</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="15.17%" headers="mcps1.3.2.6.2.4.1.3 "><p id="mrs_01_0858__ab18ffba2e7ef47f9900dea046b0fd065">-1</p>
</td>
</tr>
<tr id="mrs_01_0858__rfe008daa2521474485c362a06f48bb6f"><td class="cellrowborder" valign="top" width="28.16%" headers="mcps1.3.2.6.2.4.1.1 "><p id="mrs_01_0858__aa9b8a923ba5f464e955c98c38e05c78f">yarn.nodemanager.disk-health-checker.log-dirs.max-disk-utilization-per-disk-percentage</p>
</td>
<td class="cellrowborder" valign="top" width="56.67%" headers="mcps1.3.2.6.2.4.1.2 "><p id="mrs_01_0858__a0572be89384240cb9644bb7363bd9e7f">Maximum percentage of the Yarn disk quota that can be occupied by the container log directory on each disk. When the space occupied by the log directory exceeds the value of this parameter, the periodic log collection service is triggered to start a log collection activity beyond the period to release the local disk space. Maximum space for container logs that can be provided on each disk. If the disk space occupied by container logs exceeds this threshold, data aggregation in rolling mode is triggered.</p>
<ul id="mrs_01_0858__ul146111316175113"><li id="mrs_01_0858__li1961131617518">For clusters of versions earlier than MRS 3.x: The valid value range of the maximum disk quota percentage is 0 to 100. If the value is less than or equal to <strong id="mrs_01_0858__b1727521124714">0</strong>, it is forcibly reset to <strong id="mrs_01_0858__b14271921154710">25</strong>. If the value is greater than 100, the value is forcibly reset to <strong id="mrs_01_0858__b52852134713">25</strong>.</li><li id="mrs_01_0858__li1770116197514">For clusters of MRS 3.<em id="mrs_01_0858__i913515518544">x</em> or later: The valid value range of the maximum disk quota percentage is –1 to 100. If the value is less than –1, it is forcibly reset to <strong id="mrs_01_0858__b138531813135415">25</strong>. If the value is greater than 100, the value is forcibly reset to <strong id="mrs_01_0858__b9853161317543">25</strong>. If you set the value to <strong id="mrs_01_0858__b1597495413453">–1</strong>, the disk capacity detection function for Container log directory is disabled.</li></ul>
<div class="note" id="mrs_01_0858__n2920423f2045493bb0a21b74e03a3413"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="mrs_01_0858__u665550358be2490e9dcf9d70634f2799"><li id="mrs_01_0858__lcba7bebdd524499ebe028df74ed08685">Percentage of the available disk space of the container log directory = Percentage of the available disk space of Yarn (<span class="parmname" id="mrs_01_0858__p812a2114503f4cc58b65916f0e6de6a5"><b>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</b></span>) x Percentage of the available disk space of the container log directory (<span class="parmname" id="mrs_01_0858__pc58c65166f574c5199276e89d3027428"><b>yarn.nodemanager.disk-health-checker.log-dirs.max-disk-utilization-per-disk-percentage</b></span>)</li><li id="mrs_01_0858__ld5ccb980bb3e406cb12a60af2a0444b3">Only applications with the periodic log collection function enabled can trigger log collection when the disk quota of the log directory exceeds the threshold.</li></ul>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="15.17%" headers="mcps1.3.2.6.2.4.1.3 "><p id="mrs_01_0858__a499c19c4b6b7491eb535de9f93fe047d">25</p>
</td>
</tr>
<tr id="mrs_01_0858__r51b583abfb5f4bf8b8435fdffac85dd6"><td class="cellrowborder" valign="top" width="28.16%" headers="mcps1.3.2.6.2.4.1.1 "><p id="mrs_01_0858__ab59101bd8cb440eabe87663eaf17f973">yarn.nodemanager.remote-app-log-dir-suffix</p>
</td>
<td class="cellrowborder" valign="top" width="56.67%" headers="mcps1.3.2.6.2.4.1.2 "><p id="mrs_01_0858__a58ae55e00f8c4522818f7ac7b8b64541">Name of the HDFS folder in which container logs are to be stored. This parameter and <span class="parmname" id="mrs_01_0858__p7b48f8bb1cab4dffbef541546d309f46"><b>yarn.nodemanager.remote-app-log-dir</b></span> form the full path for storing container logs. That is, <span class="filepath" id="mrs_01_0858__fc5ff335d73ef4468b9e131bfd644fe94"><b>{yarn.nodemanager.remote-app-log-dir}/${user}/{yarn.nodemanager.remote-app-log-dir-suffix}</b></span>.</p>
<div class="note" id="mrs_01_0858__ne2e2c5a774aa4972b52b941187ca0e94"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0858__abb4370d553254652b435c00690019667"><i><span class="varname" id="mrs_01_0858__v7619f4a355994162920b9d1f1022a06d">{user}</span></i> indicates the username for running the task.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="15.17%" headers="mcps1.3.2.6.2.4.1.3 "><p id="mrs_01_0858__a84fa84f7801e44a8b6d52b389c12b1d7">logs</p>
</td>
</tr>
<tr id="mrs_01_0858__r7dcf03dc6f1d4722a298a205ebcb0450"><td class="cellrowborder" valign="top" width="28.16%" headers="mcps1.3.2.6.2.4.1.1 "><p id="mrs_01_0858__a188c09ce001f42aa93df5ac2e6dc393f">yarn.nodemanager.log-aggregator.on-fail.remain-log-in-sec</p>
</td>
<td class="cellrowborder" valign="top" width="56.67%" headers="mcps1.3.2.6.2.4.1.2 "><p id="mrs_01_0858__a7c5ad2d331194764a3d5ddc8e4b06a7e">Duration for retaining container logs on the local host after the logs fail to be collected, in second</p>
<ul id="mrs_01_0858__udcc64cb94dc743989187f9b34bb6bbc8"><li id="mrs_01_0858__l1bc7b19709b141818aed661bbdda817f">If this parameter is set to <strong id="mrs_01_0858__b193865132712">0</strong>, local logs are deleted immediately.</li><li id="mrs_01_0858__la0cae2ab793142e1981ace3bbed7e48f">If this parameter is set to a positive number, local logs are retained for this period.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="15.17%" headers="mcps1.3.2.6.2.4.1.3 "><p id="mrs_01_0858__aa92547c825144c59acf626614c6cafd7">604800</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="mrs_01_0858__p641352912811">Go to the <strong id="mrs_01_0858__b4797204519335">All Configurations</strong> page of MapReduce and enter a parameter name in <a href="#mrs_01_0858__table11598435102812">Table 2</a> in the search box by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</p>

<div class="tablenoborder"><a name="mrs_01_0858__table11598435102812"></a><a name="table11598435102812"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0858__table11598435102812" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_0858__row2598123519288"><th align="left" class="cellrowborder" valign="top" width="28.26%" id="mcps1.3.2.8.2.4.1.1"><p id="mrs_01_0858__p175981135102819">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="56.45%" id="mcps1.3.2.8.2.4.1.2"><p id="mrs_01_0858__p859813352288">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.290000000000001%" id="mcps1.3.2.8.2.4.1.3"><p id="mrs_01_0858__p12598123518287">Default Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0858__row959873518285"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.8.2.4.1.1 "><p id="mrs_01_0858__p1459963542815">yarn.log-aggregation.retain-seconds</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.8.2.4.1.2 "><p id="mrs_01_0858__p759953562815">Duration for retaining aggregated logs, in second</p>
<ul id="mrs_01_0858__ul1138474311396"><li id="mrs_01_0858__li7384143133919">If this parameter is set to <strong id="mrs_01_0858__b108226124474">–1</strong>, the container logs will be retained permanently in the HDFS.</li></ul>
<ul id="mrs_01_0858__ul859963511288"><li id="mrs_01_0858__li1859920357284">If this parameter is set to <strong id="mrs_01_0858__b5400336183813">0</strong> or a positive integer, container logs will be stored for such a period and deleted after the period expires.<div class="note" id="mrs_01_0858__note1059923511289"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0858__p4599835162817">A short period may increase load of the NameNode. Therefore, you are advised to set this parameter to a proper value.</p>
</div></div>
</li></ul>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.8.2.4.1.3 "><p id="mrs_01_0858__p195991835162815">1296000</p>
</td>
</tr>
<tr id="mrs_01_0858__row459943532812"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.8.2.4.1.1 "><p id="mrs_01_0858__p45991835182819">yarn.log-aggregation.retain-check-interval-seconds</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.8.2.4.1.2 "><p id="mrs_01_0858__p9599835142812">Interval for storing container logs in HDFS, in second</p>
<ul id="mrs_01_0858__ul359919359282"><li id="mrs_01_0858__li125991135112820">If this parameter is set to <strong id="mrs_01_0858__b788154313915">-1</strong> or <strong id="mrs_01_0858__b1585794514393">0</strong>, the interval will be one tenth of the period specified by <span class="parmname" id="mrs_01_0858__parmname260020354287"><b>yarn.log-aggregation.retain-seconds</b></span>.<div class="note" id="mrs_01_0858__note260033522815"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0858__p1760053552819">If this parameter is set to <strong id="mrs_01_0858__b36564157407">-1</strong> or <strong id="mrs_01_0858__b1832841718403">0</strong>, <span class="parmname" id="mrs_01_0858__parmname3600435162819"><b>yarn.log-aggregation.retain-seconds</b></span> cannot be set to <strong id="mrs_01_0858__b99471126184020">0</strong>.</p>
</div></div>
</li><li id="mrs_01_0858__li1060043522812">If this parameter is set to a positive number, container logs in HDFS will be scanned at such an interval.<div class="note" id="mrs_01_0858__note1660013518281"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0858__p10600123520284">A short interval may increase load of the NameNode. Therefore, you are advised to set this parameter to a proper value.</p>
</div></div>
</li></ul>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.8.2.4.1.3 "><p id="mrs_01_0858__p16600123582818">86400</p>
</td>
</tr>
</tbody>
</table>
</div>
<p id="mrs_01_0858__p1110836195517">Go to the <strong id="mrs_01_0858__b1710218053419">All Configurations</strong> page of Yarn and enter a parameter name list in <a href="#mrs_01_0858__t969e9ceee95b43149610e8f72cf9d91f">Table 3</a> in the search box by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</p>

<div class="tablenoborder"><a name="mrs_01_0858__t969e9ceee95b43149610e8f72cf9d91f"></a><a name="t969e9ceee95b43149610e8f72cf9d91f"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0858__t969e9ceee95b43149610e8f72cf9d91f" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Configuring rolling output of MapReduce application log files</caption><thead align="left"><tr id="mrs_01_0858__r282dcbbab2e949c3b333a965ca69fb47"><th align="left" class="cellrowborder" valign="top" width="28.26%" id="mcps1.3.2.10.2.4.1.1"><p id="mrs_01_0858__a117e3ed290794e6da7bfa8ccbb5401ce">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="56.45%" id="mcps1.3.2.10.2.4.1.2"><p id="mrs_01_0858__a54cde717c6234435add34408ea4d0895">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="15.290000000000001%" id="mcps1.3.2.10.2.4.1.3"><p id="mrs_01_0858__a972798a64fed4f61b1de03c44a3a9126">Default Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0858__rb773a6826362428a936b293b53f6dab6"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.10.2.4.1.1 "><p id="mrs_01_0858__a2fb9eb1df3db4b5aa5aafd9ea7fdd256">mapreduce.task.userlog.limit.kb</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.10.2.4.1.2 "><p id="mrs_01_0858__ab4f6a795c9ac43e0b4a3b5945e66eb25">Maximum size of a single task log file of the MapReduce application. When the maximum size of the log file has been reached, a new log file is generated. The value <strong id="mrs_01_0858__b13021656104213">0</strong> indicates that the size of the log file is not limited.</p>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.10.2.4.1.3 "><p id="mrs_01_0858__a634fc555c85b413ca95869f2a939341e">51200</p>
</td>
</tr>
<tr id="mrs_01_0858__r41e828982312464391e4db66dc914737"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.10.2.4.1.1 "><p id="mrs_01_0858__abd29fad4ed424fe5b233714eb1cebfd8">yarn.app.mapreduce.task.container.log.backups</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.10.2.4.1.2 "><p id="mrs_01_0858__a8315eb9b1d534608a6cc49642df819e4">Maximum number of task logs that can be retained for the MapReduce application.</p>
<p id="mrs_01_0858__a8cfe7a8801cb4839aca91ce806ce01aa">If this parameter is set to <strong id="mrs_01_0858__b1635114115444">0</strong>, rolling output is disabled.</p>
<p id="mrs_01_0858__p1214952412330">Number of task log backup files when ContainerRollingLogAppender (CRLA) is used. By default, ContainerLogAppender (CLA) is used and container logs are not rolled back.</p>
<p id="mrs_01_0858__p18699115753215">When both <strong id="mrs_01_0858__b11216125517448">mapreduce.task.userlog.limit.kb</strong> and <strong id="mrs_01_0858__b18962145915444">yarn.app.mapreduce.task.container.log.backups</strong> are greater than 0, CRLA is enabled. The value ranges from 0 to 999.</p>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.10.2.4.1.3 "><p id="mrs_01_0858__af8a84b696b594fa1b70b9a9dc7e232e3">10</p>
</td>
</tr>
<tr id="mrs_01_0858__r0e049a63268b42db9a827ca668f95d22"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.10.2.4.1.1 "><p id="mrs_01_0858__a46499f59153b44849d2e6a26b59856aa">yarn.app.mapreduce.am.container.log.limit.kb</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.10.2.4.1.2 "><p id="mrs_01_0858__a5bb2355e5e294c6c880e0b216d22fa7a">Maximum size of a single ApplicationMaster log file of the MapReduce application, in KB. When the maximum size of the log file has been reached, a new log file is generated. The value <strong id="mrs_01_0858__b74681825204610">0</strong> indicates that the size of a single ApplicationMaster log file is not limited.</p>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.10.2.4.1.3 "><p id="mrs_01_0858__a3d1334c4594d4afba7562ffee4e2b453">51200</p>
</td>
</tr>
<tr id="mrs_01_0858__r8e81f396f78d4e129e405dc93b167263"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.10.2.4.1.1 "><p id="mrs_01_0858__afb01ff95ddb84dcb9615e10408d0f2de">yarn.app.mapreduce.am.container.log.backups</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.10.2.4.1.2 "><p id="mrs_01_0858__p17114202833615">Maximum number of ApplicationMaster logs that can be retained for the MapReduce application. If this parameter is set to <strong id="mrs_01_0858__b51562714717">0</strong>, rolling output is disabled. Number of ApplicationMaster log backup files when CRLA is used. By default, CLA is used and container logs are not rolled back.</p>
<p id="mrs_01_0858__a644cf3d2ebc7441089a0c2a670308ba5">When both <strong id="mrs_01_0858__b787411754817">yarn.app.mapreduce.am.container.log.limit.kb</strong> and <strong id="mrs_01_0858__b0276101884814">yarn.app.mapreduce.am.container.log.backups</strong> are greater than 0, CRLA is enabled for the ApplicationMaster. The value ranges from 0 to 999.</p>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.10.2.4.1.3 "><p id="mrs_01_0858__a46f1a26d3e1f49dcab21a8d383c373f7">20</p>
</td>
</tr>
<tr id="mrs_01_0858__rb4adcbd4ce634d84a934560391fc4698"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.10.2.4.1.1 "><p id="mrs_01_0858__a25ed55b22e5e47f4bc9a866e970ae5d3">yarn.app.mapreduce.shuffle.log.backups</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.10.2.4.1.2 "><p id="mrs_01_0858__ac6525540ba4c46caa5275dab6ffc6e1d"><span id="mrs_01_0858__ph6605579511650">Maximum number of shuffle logs that can be retained for the MapReduce application. </span>If this parameter is set to <strong id="mrs_01_0858__b168321030194913">0</strong>, <span id="mrs_01_0858__ph4927147111654">rolling output is disabled</span>.</p>
<p id="mrs_01_0858__p614951315371">When both <strong id="mrs_01_0858__b318811405015">yarn.app.mapreduce.shuffle.log.limit.kb</strong> and <strong id="mrs_01_0858__b78141592505">yarn.app.mapreduce.shuffle.log.backups</strong> are greater than 0, <strong id="mrs_01_0858__b428219196507">syslog.shuffle</strong> uses CRLA. The value ranges from 0 to 999.</p>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.10.2.4.1.3 "><p id="mrs_01_0858__affbcd5376b9e437c984d7921cb11e401">10</p>
</td>
</tr>
<tr id="mrs_01_0858__r174a2ec23a414bf9b3d341df2b486ab6"><td class="cellrowborder" valign="top" width="28.26%" headers="mcps1.3.2.10.2.4.1.1 "><p id="mrs_01_0858__a67d10e85948a4e66a535328d7d213d5d">yarn.app.mapreduce.shuffle.log.limit.kb</p>
</td>
<td class="cellrowborder" valign="top" width="56.45%" headers="mcps1.3.2.10.2.4.1.2 "><p id="mrs_01_0858__a7568bc0709f14f69b28c7c1983b28f0b"><span id="mrs_01_0858__ph501540101178">Maximum size of a single shuffle log file of the MapReduce application, in KB. When the maximum size of the log file has been reached, a new log file is generated. </span>If this parameter is set to <strong id="mrs_01_0858__b19358173405110">0</strong>, <span id="mrs_01_0858__ph3299369711710">the size of a single shuffle log file is not limited</span>. The value must be greater than or equal to <strong id="mrs_01_0858__b94695813522">0</strong>.</p>
</td>
<td class="cellrowborder" valign="top" width="15.290000000000001%" headers="mcps1.3.2.10.2.4.1.3 "><p id="mrs_01_0858__aef4d587d12cc47168d1909cfdecc8dc4">51200</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0851.html">Using Yarn</a></div>
</div>
</div>