forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
83 lines
6.8 KiB
HTML
83 lines
6.8 KiB
HTML
<a name="mrs_01_1570"></a><a name="mrs_01_1570"></a>
|
|
|
|
<h1 class="topictitle1">Network communication (via Netty)</h1>
|
|
<div id="body1596163871729"><div class="section" id="mrs_01_1570__s9913e294b2db488f9794cd4756718a18"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1570__a4ef97b90086b4f488585ec87bb180233">When Flink runs a job, data transmission and reverse pressure detection between tasks depend on Netty. In certain environments, <strong id="mrs_01_1570__b16326501693">Netty</strong> parameters should be configured.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1570__s5590e08b2b0d4d4ab4ffefd05b4caf86"><h4 class="sectiontitle">Configuration Description</h4><p id="mrs_01_1570__a1d97c9f449ad4934aeac5c9f542cc093">For advanced optimization, you can modify the following Netty configuration items. The default configuration can meet the requirements of tasks of large-scale clusters with high concurrent throughput.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1570__t9898777c01174dbf86dd53f3258870ef" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_1570__r09dcad5eef99400e8d23366a0c1b000d"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.2.3.2.5.1.1"><p id="mrs_01_1570__a86273343f29c404fad555aa34bb21b56">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="42.29%" id="mcps1.3.2.3.2.5.1.2"><p id="mrs_01_1570__a7939868039d24d97a0596264d50347cf">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="15.659999999999998%" id="mcps1.3.2.3.2.5.1.3"><p id="mrs_01_1570__aeeafd36a6b83436cb7aba87adbc4e66a">Default Value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="17.05%" id="mcps1.3.2.3.2.5.1.4"><p id="mrs_01_1570__a3a92517b28d5471ba17280f85a33692d">Mandatory</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1570__r0ed7c84c006447fbbe1756ca3126db00"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1570__a570ca836573a4a8f8c543ea0da180f70">taskmanager.network.netty.num-arenas</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.29%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1570__a9279514da70b4681ad340979d88c7fc2">Number of Netty memory blocks.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1570__ac4cb4f4c680244d2a54f7597e52c131a">1</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.05%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1570__abb419d3d16ed4fce853c944a5d200819">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1570__r67d03beecc9346fba21862621f2014f9"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1570__a2ab2ad50080e442d84c356fd29ad5666">taskmanager.network.netty.server.numThreads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.29%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1570__a5543c660abc94fa0b5c1bc893ff8abc8">Number of Netty server threads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1570__aa63b73a65bce41b4a25f0bf0af978084">1</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.05%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1570__a791c86940196430781aa6a2b6eae52b1">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1570__rcf5944a12d274090b60f46dad044252d"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1570__a8c20a48e06d84197b31382ae66d984d2">taskmanager.network.netty.client.numThreads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.29%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1570__a3676aabda7d34fb3bc07780ed10af695">Number of Netty client threads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1570__p613124171416">1</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.05%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1570__a64b0324a12184c2d805d0d760e99fa05">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1570__r7504a27837e2427e8c85658c6a937daa"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1570__a5716beac0c05411caa80706b00a471b4">taskmanager.network.netty.client.connectTimeoutSec</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.29%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1570__a540eef0092d14984a5e3f8d271e0b0c3">Netty client connection timeout duration. Unit: second</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1570__a2087c988b0fd47788ff9a74fbdea620e">120</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.05%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1570__aaaf3404cae3a4298814e0979c00fcfb5">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1570__r9d2d50b95bf64affbc12bab30db6aa11"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1570__a54fa2e3670964ffda7f995ba5ede286e">taskmanager.network.netty.sendReceiveBufferSize</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.29%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1570__a100566b3e3614cb1b9bf7eb80521ab11">Size of Netty sending and receiving buffers. This defaults to the system buffer size (<strong id="mrs_01_1570__b16667142891215">cat /proc/sys/net/ipv4/tcp_[rw]mem</strong>) and is 4 MB in modern Linux. Unit: byte</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1570__a290e9b2d39da4c8391f0d6746756c7e6">4096</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.05%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1570__aaf3bb39e9629483eab20e4b64c57f84c">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1570__r7608c815ae7f4fb99a0b0e43cc401627"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1570__a4789d2c447904a43b69dca779b8ee69f">taskmanager.network.netty.transport</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="42.29%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1570__a5501138e73ab483da82a7555a055d9dc">Netty transport type, either <strong id="mrs_01_1570__b3987149151218">nio</strong> or <strong id="mrs_01_1570__b15991649181214">epoll</strong></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="15.659999999999998%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1570__adac1c821cf744dd59009aeeeeaec7c25">nio</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.05%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1570__a2b2d9457a13945a6b6aef3acb01d602d">No</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0592.html">Flink Configuration Management</a></div>
|
|
</div>
|
|
</div>
|
|
|