forked from docs/doc-exports
gtema
72909318a9
Reviewed-by: Goncharov, Artem <artem.goncharov@t-systems.com> Co-authored-by: gtema <artem.goncharov@gmail.com> Co-committed-by: gtema <artem.goncharov@gmail.com>
27 lines
4.6 KiB
HTML
27 lines
4.6 KiB
HTML
<a name="css_02_0018"></a><a name="css_02_0018"></a>
|
|
|
|
<h1 class="topictitle1">How Can I Improve Filebeat Performance?</h1>
|
|
<div id="body1556000179283"><div class="section" id="css_02_0018__section58176298578"><h4 class="sectiontitle">Symptom</h4><p id="css_02_0018__p0116172455211">Filebeat is a high-performance file collection tool. By default, one core is allocated to Filebeat, and it writes 1 MB data to Elasticsearch per second. However, in practice, when a large number of service logs are generated, Filebeat cannot promptly collect and write them to Elasticsearch. In this case, you can optimize parameter settings in the <strong id="css_02_0018__b1899964194720">filebeat.yml</strong> file to improve the Filebeat performance.</p>
|
|
</div>
|
|
<div class="section" id="css_02_0018__section1758212551564"><h4 class="sectiontitle">Procedure</h4><ol id="css_02_0018__ol058255511619"><li id="css_02_0018__li1088533813714">Optimize the parameters involved in <strong id="css_02_0018__b20917184314457">input</strong> of the <strong id="css_02_0018__b13328125615458">filebeat.yml</strong> configuration file.<p id="css_02_0018__p1518711556496">Increase the value of <strong id="css_02_0018__b12802133017461">harvester_buffer_size</strong> based on actual requirements. This parameter defines the buffer size used by every <strong id="css_02_0018__b376082964215">harvester</strong>.</p>
|
|
<p id="css_02_0018__p16250171410259"><strong id="css_02_0018__b129137154124">harvester_buffer_size</strong>: <strong id="css_02_0018__b1234217184121">40,960,000</strong></p>
|
|
<p id="css_02_0018__p171331257124911">Increase the value of <strong id="css_02_0018__b174612599487">filebeat.spool_size</strong> based on actual requirements. This parameter defines the number of log records that can be uploaded by the <strong id="css_02_0018__b02739365468">spooler</strong> at a time.</p>
|
|
<p id="css_02_0018__p1251201412259"><strong id="css_02_0018__b4335192651318">filebeat.spool_size</strong>: <strong id="css_02_0018__b964882814133">250,000</strong></p>
|
|
<p id="css_02_0018__p15415587506">Adjust the value of <strong id="css_02_0018__b352011396516">filebeat.idle_timeout</strong> based on actual requirements. This parameter defines how often the <strong id="css_02_0018__b1828712916479">spooler</strong> is flushed. After the <strong id="css_02_0018__b9354194285517">idle_timeout</strong> is reached, the <strong id="css_02_0018__b16711163211476">spooler</strong> is flushed regardless of whether the <strong id="css_02_0018__b19404185935510">spool_size</strong> has been reached.</p>
|
|
<p id="css_02_0018__p483114586490"><strong id="css_02_0018__b1642434710135">filebeat.idle_timeout</strong>: <strong id="css_02_0018__b1919174981317">1s</strong></p>
|
|
</li><li id="css_02_0018__li13511621172513">Optimize the parameters involved in <strong id="css_02_0018__b392121217521">output.elasticsearch</strong> in the <strong id="css_02_0018__b792461245219">filebeat.yml</strong> configuration file.<p id="css_02_0018__p201161422105316">Set the value of <strong id="css_02_0018__b168801240185212">worker</strong> to the number of Elasticsearch clusters based on actual requirements. This parameter indicates the number of Elasticsearch clusters. The default value is <strong id="css_02_0018__b154721310205314">1</strong>.</p>
|
|
<p id="css_02_0018__p10644162533811"><strong id="css_02_0018__b1016958141312">worker</strong>: <strong id="css_02_0018__b1999913592138">1</strong></p>
|
|
<p id="css_02_0018__p7535377548">Increase the value of <strong id="css_02_0018__b281033135319">bulk_max_size</strong> based on actual requirements. This parameter defines the maximum number of events to bulk in a single Elasticsearch bulk API index request. The default is <strong id="css_02_0018__b148322535514">50</strong>.</p>
|
|
<p id="css_02_0018__p9600115182717"><strong id="css_02_0018__b375810717148">bulk_max_size</strong>: <strong id="css_02_0018__b163991999148">15,000</strong></p>
|
|
<p id="css_02_0018__p1441014416220">Adjust the value of <strong id="css_02_0018__b586416411201">flush_interval</strong> based on the actual requirements. This parameter defines the number of seconds to wait for new events between two bulk API index requests. If <strong id="css_02_0018__b152411913341">bulk_max_size</strong> is reached before this interval expires, additional bulk index requests are made.</p>
|
|
<p id="css_02_0018__p103721840115417"><strong id="css_02_0018__b146008221146">flush_interval</strong>: <strong id="css_02_0018__b694422319141">1s</strong></p>
|
|
</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_02_0001.html">FAQs</a></div>
|
|
</div>
|
|
</div>
|
|
|