forked from docs/doc-exports
Chen, Junjie
dd8a3a658b
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Chen, Junjie <chenjunjie@huawei.com> Co-committed-by: Chen, Junjie <chenjunjie@huawei.com>
24 lines
4.3 KiB
HTML
24 lines
4.3 KiB
HTML
<a name="kafka-faq-200426036"></a><a name="kafka-faq-200426036"></a>
|
|
|
|
<h1 class="topictitle1">Why Does Message Poll Often Fail During Rebalancing?</h1>
|
|
<div id="body1587865606409"><p id="kafka-faq-200426036__p1725520119127">Rebalancing is a process where partitions of topics are re-allocated for a consumer group.</p>
|
|
<p id="kafka-faq-200426036__p53438191906">In normal cases, rebalancing occurs inevitably when a consumer is added to or removed from a consumer group. However, if a consumer is regarded as abnormal and removed from the consumer group, message retrieval may fail.</p>
|
|
<p id="kafka-faq-200426036__p13485711201313">This may happen in the following scenarios:</p>
|
|
<ol id="kafka-faq-200426036__ol2526181561317"><li id="kafka-faq-200426036__li199542355108">Heartbeat requests are not sent in time.<p id="kafka-faq-200426036__p1345715521015"><a name="kafka-faq-200426036__li199542355108"></a><a name="li199542355108"></a>A consumer sends heartbeat requests to the broker at the interval specified by <strong id="kafka-faq-200426036__b108784412249">heartbeat.interval.ms</strong>. If the broker does not receive any heartbeat request from the consumer within the period specified by <strong id="kafka-faq-200426036__b3429601251">session.timeout.ms</strong>, the broker considers that the consumer is abnormal and removes the consumer from the consumer group, triggering rebalancing.</p>
|
|
</li><li id="kafka-faq-200426036__li1996711206112">The interval between retrievals is too long.<p id="kafka-faq-200426036__p6441715121211"><a name="kafka-faq-200426036__li1996711206112"></a><a name="li1996711206112"></a>The maximum number of messages that a consumer can retrieve at a time is specified by <strong id="kafka-faq-200426036__b474129122620">max.poll.records</strong>. In most cases, a client processes the retrieved data before starting the next retrieval. The processing may be prolonged when a large number of messages are retrieved at a time and cannot be processed within the time specified by <strong id="kafka-faq-200426036__b3671164433819">max.poll.interval.ms</strong>, or when an exception occurs during the process (for example, data needs to be written to the backend database, but the backend database pressure is too high, resulting in high latency). If the consumer does not send the next retrieval request within the time specified by <strong id="kafka-faq-200426036__b6766115074113">max.poll.interval.ms</strong>, the broker considers that the consumer is inactive and removes it from the consumer group, triggering rebalancing.</p>
|
|
</li></ol>
|
|
<div class="section" id="kafka-faq-200426036__section332175810228"><h4 class="sectiontitle">Solutions and Troubleshooting Methods</h4><p id="kafka-faq-200426036__p11613418234"><strong id="kafka-faq-200426036__b96685152241">Scenario 1:</strong> Heartbeat requests are not sent in time.</p>
|
|
<p id="kafka-faq-200426036__p622111437232"><strong id="kafka-faq-200426036__b15252059104210">Solution</strong>: Set the value of <strong id="kafka-faq-200426036__b16895811164317">session.timeout.ms</strong> to three times the value of <strong id="kafka-faq-200426036__b14793216144313">heartbeat.interval.ms</strong>.</p>
|
|
<p id="kafka-faq-200426036__p102340216255"><strong id="kafka-faq-200426036__b1190272216245">Scenario 2:</strong> The interval between retrievals is too long.</p>
|
|
<p id="kafka-faq-200426036__p19470410102516"><strong id="kafka-faq-200426036__b3971721121714">Troubleshooting methods:</strong></p>
|
|
<ol id="kafka-faq-200426036__ol9421141152710"><li id="kafka-faq-200426036__li142164117279">Check the time required for processing a single message and whether the time required for processing a specified number (<strong id="kafka-faq-200426036__b7669172113444">max.poll.records</strong>) of messages exceeds the time specified by <strong id="kafka-faq-200426036__b11851529144419">max.poll.interval.ms</strong>.</li><li id="kafka-faq-200426036__li167481113103217">Check whether message processing requires network connections, such as writing data to the database and calling backend APIs, and whether the backend is normal in rebalancing scenarios.</li></ol>
|
|
</div>
|
|
<p id="kafka-faq-200426036__p1913832716255"></p>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-faq-190416001.html">Messages</a></div>
|
|
</div>
|
|
</div>
|
|
|