Yang, Tong 6182f91ba8 MRS component operation guide_normal 2.0.38.SP20 version
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2022-12-09 14:55:21 +00:00

230 lines
20 KiB
HTML

<a name="mrs_01_1033"></a><a name="mrs_01_1033"></a>
<h1 class="topictitle1">Kafka Common Parameters</h1>
<div id="body1590133653336"><p id="mrs_01_1033__p97034173518">This section applies to MRS 3.x or later.</p>
<div class="section" id="mrs_01_1033__section12987170151213"><h4 class="sectiontitle">Navigation path for setting parameters:</h4><p id="mrs_01_1033__p11526143493219">For details about how to set parameters, see <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</p>
</div>
<div class="section" id="mrs_01_1033__s584f59065857469f905512880dc5fb7d"><h4 class="sectiontitle">Common Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1033__t6be371425c5c43bdb59c1579fc4d0613" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_1033__r0dab1abbb04e4b68936487aae94542c0"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.4.1.1"><p id="mrs_01_1033__a8674b95ef1f64311b550b9f535f87f01">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.4.1.2"><p id="mrs_01_1033__ae68798057813463886d95560ea64ed59">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.4.1.3"><p id="mrs_01_1033__a8126d37f52b6435fa8858d5f83de3172">Default Value</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1033__ra518df23f43a4f08be47df4a6496b75e"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.1 "><p id="mrs_01_1033__a31ed5c81b9f946cd844b3b5514e817c3">log.dirs</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.2 "><p id="mrs_01_1033__ad2141a9e4bce4069928ebc38d3242123">List of Kafka data storage directories. Use commas (,) to separate multiple directories.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.3 "><p id="mrs_01_1033__ad22744b7bbf04812b3a391230e2a1dbb">%{@auto.detect.datapart.bk.log.logs}</p>
</td>
</tr>
<tr id="mrs_01_1033__r7bf563b34d3a47b8873b134c7065c9c4"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.1 "><p id="mrs_01_1033__a7bd9d326376a4eb2a853ca7a4a77b66f">KAFKA_HEAP_OPTS</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.2 "><p id="mrs_01_1033__ac3d2eee9c9a14c57bda9e25651a664f5">Specifies the JVM option used for Kafka to start broker. It is recommended that you set this parameter based on service requirements.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.3 "><p id="mrs_01_1033__a7625765ffa334a3b8a227a8c00337a79">-Xmx6G -Xms6G</p>
</td>
</tr>
<tr id="mrs_01_1033__rb81971ab2e724c418e6905b677bc7dc1"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.1 "><p id="mrs_01_1033__a0cfc8fc2bf0e4988b009c205c1871ca7">auto.create.topics.enable</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.2 "><p id="mrs_01_1033__a820a13a022fe41a7b5ad0ed53aeeb3b8">Indicates whether a topic is automatically created. If this parameter is set to <strong id="mrs_01_1033__b1220711614918">false</strong>, you need to run a command to create a topic before sending a message.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.3 "><p id="mrs_01_1033__a481d2d673b1a48ee9923818db4489097">true</p>
</td>
</tr>
<tr id="mrs_01_1033__rd646ebc8ffcc465c9de364dee5bef6bb"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.1 "><p id="mrs_01_1033__a80e93bc636c54b1ca1a5ff80d7a2a437">default.replication.factor</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.2 "><p id="mrs_01_1033__ac7fbb9549f524ce98f5d91e15c2cee72">Default number of replicas of a topic is automatically created.</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.3 "><p id="mrs_01_1033__a25c11f871d8144cab475a5545fd40f18">2</p>
</td>
</tr>
<tr id="mrs_01_1033__rd39f64d662004f40bb96dcdc14949038"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.1 "><p id="mrs_01_1033__aaf28d74a508d4f39a9ddedd54e8f6b6e">monitor.preInitDelay</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.2 "><p id="mrs_01_1033__a1629ea18f325414bb6534c6f5e030e76">Delay of the first health check after the server is started. If the startup takes a long time, increase the value of the parameter. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.4.1.3 "><p id="mrs_01_1033__a3cada012bd5f4be19ada8644618035b3">600,000</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_1033__section141646135013"><h4 class="sectiontitle">Timeout Parameters</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1033__table4135737103619" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Broker-related timeout parameters</caption><thead align="left"><tr id="mrs_01_1033__row141361337113612"><th align="left" class="cellrowborder" valign="top" width="19.881988198819883%" id="mcps1.3.4.2.2.5.1.1"><p id="mrs_01_1033__p326148113916">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="40.89408940894089%" id="mcps1.3.4.2.2.5.1.2"><p id="mrs_01_1033__p102938173911">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="7.630763076307631%" id="mcps1.3.4.2.2.5.1.3"><p id="mrs_01_1033__p132168153914">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="31.593159315931597%" id="mcps1.3.4.2.2.5.1.4"><p id="mrs_01_1033__p235198183917">Impact</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1033__row21361937113612"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p15389803915">controller.socket.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p84111833914">Specifies the timeout for connecting controller to broker. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p1243108103915">30,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p16458873915">Generally, retain the default value of this parameter.</p>
</td>
</tr>
<tr id="mrs_01_1033__row1213610372365"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p248118153918">group.max.session.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p13507812395">Specifies the maximum session timeout during the consumer registration. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p125218818390">1800000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p05568123916">The configured value must be less than the value of this parameter.</p>
</td>
</tr>
<tr id="mrs_01_1033__row1413633733611"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p7573863912">group.min.session.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p05916813392">Specifies the minimum session timeout during the consumer registration. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p18616843913">6,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p11651485399">The configured value must be greater than the value of this parameter.</p>
</td>
</tr>
<tr id="mrs_01_1033__row213614371367"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p14689813397">offsets.commit.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p77108113910">Specifies the timeout for the Offset to submit requests. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p20731088391">5,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p075178193915">This parameter specifies the maximum delay for processing an Offset request.</p>
</td>
</tr>
<tr id="mrs_01_1033__row5137537153618"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p4783803916">replica.socket.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p108013813393">Specifies the timeout of the request for synchronizing replica data. Its value must be greater than or equal to that of the <strong id="mrs_01_1033__b183306294105">replica.fetch.wait.max.ms</strong> parameter. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p16822085397">30,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p1884198173916">Specifies the maximum timeout for establishing a channel before the synchronization thread sends a synchronization request. The value must be greater than that of the <strong id="mrs_01_1033__b250457121017">replica.fetch.wait.max.ms</strong> parameter.</p>
</td>
</tr>
<tr id="mrs_01_1033__row131371737143616"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p128838113915">request.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p12918863910">Specifies the timeout for waiting for a response after the client sends a connection request. If no response is received within the timeout, the client resends the request. A request failure is returned after the maximum retry times is reached. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p0933819393">30,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p1396168113918">This parameter is configured when the networkclient connection is transferred in the controller and replica threads on the broker node.</p>
</td>
</tr>
<tr id="mrs_01_1033__row61378371364"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p13981780391">transaction.max.timeout.ms</p>
<p id="mrs_01_1033__p9203171912282"></p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p8101148133916">Specifies the maximum timeout allowed by the transaction. If the client request time exceeds the value of this parameter, broker returns an error in InitProducerIdRequest. This prevents a long client request timeout, ensuring that consumer can receive topics. Unit: millisecond</p>
<p id="mrs_01_1033__p14201519162812"></p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p1210315893917">900,000</p>
<p id="mrs_01_1033__p1419961913286"></p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p1210511810399">Specifies the maximum timeout for transactions.</p>
<p id="mrs_01_1033__p2195151932816"></p>
</td>
</tr>
<tr id="mrs_01_1033__row4639124617380"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p1610758173913">user.group.cache.timeout.sec</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p611019833915">Specifies the time when the user group information is stored in the cache. Unit: second</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p41123853911">300</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p1411578123915">Specifies the time for caching the mapping between users and user groups. If time exceeds the threshold, the system automatically runs the <strong id="mrs_01_1033__b136721827181813">id -Gn</strong> command to query the user information. During this period, the mapping in the cache is used.</p>
</td>
</tr>
<tr id="mrs_01_1033__row186401346103819"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p16119128193917">zookeeper.connection.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p71214812396">Specifies the timeout for connecting to ZooKeeper. Unit: millisecond</p>
<p id="mrs_01_1033__p7123138103914"></p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p712620818390">45,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p161284818399">This parameter specifies the duration for connecting the ZooKeeper and zkclient for the first time. If the duration exceeds the value of this parameter, the zkclient automatically disconnects the connection.</p>
</td>
</tr>
<tr id="mrs_01_1033__row764114466386"><td class="cellrowborder" valign="top" width="19.881988198819883%" headers="mcps1.3.4.2.2.5.1.1 "><p id="mrs_01_1033__p513320833915">zookeeper.session.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="40.89408940894089%" headers="mcps1.3.4.2.2.5.1.2 "><p id="mrs_01_1033__p191353873920">Specifies the ZooKeeper session timeout duration. During this period, ZooKeeper disconnects the connection if broker does not report its heartbeats to ZooKeeper. Unit: millisecond</p>
</td>
<td class="cellrowborder" valign="top" width="7.630763076307631%" headers="mcps1.3.4.2.2.5.1.3 "><p id="mrs_01_1033__p1513878123918">45,000</p>
</td>
<td class="cellrowborder" valign="top" width="31.593159315931597%" headers="mcps1.3.4.2.2.5.1.4 "><p id="mrs_01_1033__p191401283396">ZooKeeper session timeout has the following functions:</p>
<p id="mrs_01_1033__p114078163911">1) Based on value of this parameter and the number of ZooKeeper URLs in ZKURL, if the connection duration exceeds the node timeout value (sessionTimeout/Number of transferred ZooKeeper URLs), the connection fails and the system attempts to connect to the next node.</p>
<p id="mrs_01_1033__p161421386391">2) After the connection is established, a session (for example, the temporary BrokerId node registered on the ZooKeeper) is cleared by the ZooKeeper a session timeout later if the broker is stopped.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1033__table8581059153910" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Producer-related timeout parameters</caption><thead align="left"><tr id="mrs_01_1033__row9583195923910"><th align="left" class="cellrowborder" valign="top" width="19.79%" id="mcps1.3.4.3.2.5.1.1"><p id="mrs_01_1033__p42841967406">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="41.15%" id="mcps1.3.4.3.2.5.1.2"><p id="mrs_01_1033__p02875617403">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.459999999999999%" id="mcps1.3.4.3.2.5.1.3"><p id="mrs_01_1033__p82896684019">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="30.599999999999998%" id="mcps1.3.4.3.2.5.1.4"><p id="mrs_01_1033__p3292366404">Impact</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1033__row14583145933919"><td class="cellrowborder" valign="top" width="19.79%" headers="mcps1.3.4.3.2.5.1.1 "><p id="mrs_01_1033__p63111460406">request.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="41.15%" headers="mcps1.3.4.3.2.5.1.2 "><p id="mrs_01_1033__p531213624010">Specifies the timeout of a message request.</p>
</td>
<td class="cellrowborder" valign="top" width="8.459999999999999%" headers="mcps1.3.4.3.2.5.1.3 "><p id="mrs_01_1033__p1331536114015">30,000</p>
</td>
<td class="cellrowborder" valign="top" width="30.599999999999998%" headers="mcps1.3.4.3.2.5.1.4 "><p id="mrs_01_1033__p131916654010">If a network fault occurs, increase the value of this parameter. If the value is too small, the Batch Expire occurs.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1033__table1319720794216" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Consumer-related timeout parameters</caption><thead align="left"><tr id="mrs_01_1033__row41984724212"><th align="left" class="cellrowborder" valign="top" width="19.7%" id="mcps1.3.4.4.2.5.1.1"><p id="mrs_01_1033__p111133994219">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="41.5%" id="mcps1.3.4.4.2.5.1.2"><p id="mrs_01_1033__p316173944219">Description</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.459999999999999%" id="mcps1.3.4.4.2.5.1.3"><p id="mrs_01_1033__p719539174211">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="30.34%" id="mcps1.3.4.4.2.5.1.4"><p id="mrs_01_1033__p11222394421">Impact</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_1033__row11982704216"><td class="cellrowborder" valign="top" width="19.7%" headers="mcps1.3.4.4.2.5.1.1 "><p id="mrs_01_1033__p10541539154214">connections.max.idle.ms</p>
</td>
<td class="cellrowborder" valign="top" width="41.5%" headers="mcps1.3.4.4.2.5.1.2 "><p id="mrs_01_1033__p257173924216">Specifies the maximum retention period for idle connections.</p>
</td>
<td class="cellrowborder" valign="top" width="8.459999999999999%" headers="mcps1.3.4.4.2.5.1.3 "><p id="mrs_01_1033__p106013915427">600,000</p>
</td>
<td class="cellrowborder" valign="top" width="30.34%" headers="mcps1.3.4.4.2.5.1.4 "><p id="mrs_01_1033__p1363103918422">If the idle connection time is greater than this parameter value, this connection is disconnected. If necessary, a new connection is created.</p>
</td>
</tr>
<tr id="mrs_01_1033__row11987794213"><td class="cellrowborder" valign="top" width="19.7%" headers="mcps1.3.4.4.2.5.1.1 "><p id="mrs_01_1033__p2815398426">request.timeout.ms</p>
</td>
<td class="cellrowborder" valign="top" width="41.5%" headers="mcps1.3.4.4.2.5.1.2 "><p id="mrs_01_1033__p8851339114219">Specifies the timeout for consumer requests.</p>
<p id="mrs_01_1033__p124471017134912"></p>
</td>
<td class="cellrowborder" valign="top" width="8.459999999999999%" headers="mcps1.3.4.4.2.5.1.3 "><p id="mrs_01_1033__p58773934212">30,000</p>
</td>
<td class="cellrowborder" valign="top" width="30.34%" headers="mcps1.3.4.4.2.5.1.4 "><p id="mrs_01_1033__p691123964213">If the request times out, the request will fail and be sent again.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0375.html">Using Kafka</a></div>
</div>
</div>