Scenarios
The communication of Flink is based on Netty network. The network performance determines the data switching speed and task execution efficiency. Therefore, the performance of Flink can be optimized by optimizing the Netty network.
Procedure
In the conf/flink-conf.yaml file on the client, change configurations as required. Exercise caution when changing default values, because default values are optimal.
- taskmanager.network.netty.num-arenas: Specifies the number of arenas of Netty. The default value is taskmanager.numberOfTaskSlots.
- taskmanager.network.netty.server.numThreads and taskmanager.network.netty.client.numThreads: specify the number of threads on the client and server. The default value is taskmanager.numberOfTaskSlots.
- taskmanager.network.netty.client.connectTimeoutSec: specifies the timeout interval for connection of TaskManager client. The default value is 120s.
- taskmanager.network.netty.sendReceiveBufferSize: specifies the buffer size of the Netty network. The default value is the buffer size (cat /proc/sys/net/ipv4/tcp_[rw]mem) of the system and the value is usually 4 MB.
- taskmanager.network.netty.transport: specifies the transmission method of the Netty network. The default value is nio. The value can only be nio and epoll.