forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
151 lines
15 KiB
HTML
151 lines
15 KiB
HTML
<a name="mrs_01_1081"></a><a name="mrs_01_1081"></a>
|
|
|
|
<h1 class="topictitle1">Introduction to Flume Logs</h1>
|
|
<div id="body1590374796595"><div class="section" id="mrs_01_1081__s7c10a70635c744399005f0b474e3cc1f"><h4 class="sectiontitle">Log Description</h4><p id="mrs_01_1081__en-us_topic_0046736730_p7703550"><strong id="mrs_01_1081__b31431933133314">Log path</strong>: The default path of Flume log files is <strong id="mrs_01_1081__b22321829143417">/var/log/Bigdata/</strong><em id="mrs_01_1081__i19852103110345">Role name</em>.</p>
|
|
<ul id="mrs_01_1081__en-us_topic_0046736730_ul20007789"><li id="mrs_01_1081__en-us_topic_0046736730_li45852376">FlumeServer: <strong id="mrs_01_1081__b1781133793416">/var/log/Bigdata/flume/flume</strong></li><li id="mrs_01_1081__en-us_topic_0046736730_li10018201">FlumeClient: <strong id="mrs_01_1081__b13805843123418">/var/log/Bigdata/flume-client-n/flume</strong></li><li id="mrs_01_1081__en-us_topic_0046736730_li23054945">MonitorServer: <strong id="mrs_01_1081__b124415483346">/var/log/Bigdata/flume/monitor</strong></li></ul>
|
|
<p id="mrs_01_1081__en-us_topic_0046736730_p6167913"><strong id="mrs_01_1081__b10113191359">Log archive rule</strong>: The automatic Flume log compression function is enabled. By default, when the size of logs exceeds 50 MB , logs are automatically compressed into a log file named in the following format: <em id="mrs_01_1081__i1929658133619"><Original log file name>-<yyyy-mm-dd_hh-mm-ss>.[ID]</em><strong id="mrs_01_1081__b836673143716">.log.zip</strong>. A maximum of 20 latest compressed files are reserved. The number of compressed files can be configured on the Manager portal.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1081__en-us_topic_0046736730_table29838917" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Flume log list</caption><thead align="left"><tr id="mrs_01_1081__en-us_topic_0046736730_row9770605"><th align="left" class="cellrowborder" valign="top" width="29.592959295929592%" id="mcps1.3.1.5.2.4.1.1"><p id="mrs_01_1081__en-us_topic_0046736730_p53221504"><strong id="mrs_01_1081__b1054775773811">Type</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="35.713571357135706%" id="mcps1.3.1.5.2.4.1.2"><p id="mrs_01_1081__en-us_topic_0046736730_p9553720"><strong id="mrs_01_1081__b683845910384">Name</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="34.69346934693469%" id="mcps1.3.1.5.2.4.1.3"><p id="mrs_01_1081__en-us_topic_0046736730_p52449071"><strong id="mrs_01_1081__b196340219397">Description</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1081__en-us_topic_0046736730_row50429634"><td class="cellrowborder" rowspan="10" valign="top" width="29.592959295929592%" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p58268531">Run logs</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35.713571357135706%" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p22130602">/flume/flumeServer.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="34.69346934693469%" headers="mcps1.3.1.5.2.4.1.3 "><p id="mrs_01_1081__en-us_topic_0046736730_p47748300">Log file that records FlumeServer running environment information.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row27081522"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p46119650">/flume/install.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p44704179">FlumeServer installation log file</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row66793295"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p41547828">/flume/flumeServer-gc.log.<em id="mrs_01_1081__i894417454012"><No.></em></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p9930893">GC log file of the FlumeServer process</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row22269177"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p58972892">/flume/prestartDvietail.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p12074919">Work log file before the FlumeServer startup</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row41565411"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p11355167">/flume/startDetail.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p47353302">Startup log file of the Flume process</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row23526540"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p26601610">/flume/stopDetail.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p7246776">Shutdown log file of the Flume process</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row65220984"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p48408370">/monitor/monitorServer.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p28763888">Log file that records MonitorServer running environment information</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row57548404"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p30909172">/monitor/startDetail.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p20615021">Startup log file of the MonitorServer process</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row51317462"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p63073718">/monitor/stopDetail.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p8697506">Shutdown log file of the MonitorServer process</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row11168697"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p32249251">function.log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p62052558">External function invoking log file</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1081__section1744547204712"><h4 class="sectiontitle">Log Level</h4><p id="mrs_01_1081__en-us_topic_0046736730_p60201314"><a href="#mrs_01_1081__tc09b739e3eb34797a6da936a37654e97">Table 2</a> describes the log levels supported by Flume.</p>
|
|
<p id="mrs_01_1081__en-us_topic_0046736730_p44468229">Levels of run logs are FATAL, ERROR, WARN, INFO, and DEBUG from the highest to the lowest priority. Run logs of equal or higher levels are recorded. The higher the specified log level, the fewer the logs recorded.</p>
|
|
|
|
<div class="tablenoborder"><a name="mrs_01_1081__tc09b739e3eb34797a6da936a37654e97"></a><a name="tc09b739e3eb34797a6da936a37654e97"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1081__tc09b739e3eb34797a6da936a37654e97" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Log level</caption><thead align="left"><tr id="mrs_01_1081__en-us_topic_0046736730_row63443418"><th align="left" class="cellrowborder" valign="top" width="33.333333333333336%" id="mcps1.3.2.4.2.4.1.1"><p id="mrs_01_1081__en-us_topic_0046736730_p38643240"><strong id="mrs_01_1081__b1597183554812">Type</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.333333333333336%" id="mcps1.3.2.4.2.4.1.2"><p id="mrs_01_1081__en-us_topic_0046736730_p52308663"><strong id="mrs_01_1081__b13832339164811">Level</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.333333333333336%" id="mcps1.3.2.4.2.4.1.3"><p id="mrs_01_1081__en-us_topic_0046736730_p15180599"><strong id="mrs_01_1081__b543683815484">Description</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1081__row9239421124812"><td class="cellrowborder" rowspan="5" valign="top" width="33.333333333333336%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1081__p4240192119486">Run log</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1081__p19240021134811">FATAL</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.333333333333336%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1081__p11240142113485">Logs of this level record critical error information about system running.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row60803124"><td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p34212241">ERROR</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p19728122">Logs of this level record error information about system running.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row43335374"><td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p20504385">WARN</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p50242460">Logs of this level record exception information about the current event processing.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row49528960"><td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p52422860">INFO</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p18393235">Logs of this level record normal running status information about the system and events.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row31321393"><td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p54004927">DEBUG</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p12322941">Logs of this level record the system information and system debugging information.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_1081__en-us_topic_0046736730_p43797605">To modify log levels, perform the following operations:</p>
|
|
<ol id="mrs_01_1081__ol2049469418545"><li id="mrs_01_1081__li4027424618545"><span>Go to the <strong id="mrs_01_1081__b139326477334">All Configurations</strong> page of Flume by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</span></li><li id="mrs_01_1081__li2692390118545"><span>On the menu bar on the left, select the log menu of the target role.</span></li><li id="mrs_01_1081__li4098852518545"><span>Select a desired log level.</span></li><li id="mrs_01_1081__li1719026918545"><span>Save the configuration. In the displayed dialog box, click <strong id="mrs_01_1081__b145575133419">OK</strong> to make the configurations take effect.</span></li></ol>
|
|
<div class="note" id="mrs_01_1081__note199281245114918"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1081__p1292912455494">The configurations take effect immediately without the need to restart the service.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1081__s70eeb581418c4885b24bf52a5037ed2a"><h4 class="sectiontitle">Log Format</h4><p id="mrs_01_1081__en-us_topic_0046736730_p51681963">The following table lists the Flume log formats.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1081__en-us_topic_0046736730_table62484483" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Log format</caption><thead align="left"><tr id="mrs_01_1081__en-us_topic_0046736730_row6057523"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.3.2.4.1.1"><p id="mrs_01_1081__en-us_topic_0046736730_p20897367"><strong id="mrs_01_1081__b16314153619515">Type</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.3.2.4.1.2"><p id="mrs_01_1081__en-us_topic_0046736730_p468450"><strong id="mrs_01_1081__b8769539195115">Format</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.3.2.4.1.3"><p id="mrs_01_1081__en-us_topic_0046736730_p5956164"><strong id="mrs_01_1081__b13900154715116">Example</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1081__en-us_topic_0046736730_row47076455"><td class="cellrowborder" rowspan="2" valign="top" width="33.33333333333333%" headers="mcps1.3.3.3.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p55096476">Run logs</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.3.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p33629549"><<em id="mrs_01_1081__i7488855215">yyyy-MM-dd HH:mm:ss,SSS</em>>|<<em id="mrs_01_1081__i65468115217">Log level</em>>|<<em id="mrs_01_1081__i145428175220">Name of the thread that generates the log</em>>|<<em id="mrs_01_1081__i195513814524">Message in the log</em>>|<<em id="mrs_01_1081__i1455086522">Location where the log event occurs</em>></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.3.2.4.1.3 "><p id="mrs_01_1081__en-us_topic_0046736730_p39638980">2014-12-12 11:54:57,316 | INFO | [main] | log4j dynamic load is start. | org.apache.flume.tools.LogDynamicLoad.start(LogDynamicLoad.java:59)</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1081__en-us_topic_0046736730_row21206500"><td class="cellrowborder" valign="top" headers="mcps1.3.3.3.2.4.1.1 "><p id="mrs_01_1081__en-us_topic_0046736730_p40004924"><<em id="mrs_01_1081__i74183315218">yyyy-MM-dd HH:mm:ss,SSS</em>><<em id="mrs_01_1081__i12373195015219">Username</em>><<em id="mrs_01_1081__i97533541527">User IP</em>><<em id="mrs_01_1081__i6862173205320">Time</em>><<em id="mrs_01_1081__i71281185312">Operation</em>><<em id="mrs_01_1081__i14362201535312">Resource</em>><<em id="mrs_01_1081__i1813741915316">Result</em>><<em id="mrs_01_1081__i9649152245314">Detail></em></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.3.3.2.4.1.2 "><p id="mrs_01_1081__en-us_topic_0046736730_p19173388">2014-12-12 23:04:16,572 | INFO | [SinkRunner-PollingRunner-DefaultSinkProcessor] | SRCIP=null OPERATION=close</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0390.html">Using Flume</a></div>
|
|
</div>
|
|
</div>
|
|
|