mikacur/MRS-operation-guide-lts
1
0
Fork 0
forked from docs/mapreduce-service
Code Pull Requests Activity
MRS-operation-guide-lts/doc/component-operation-guide/source/using_flink/flink_configuration_management/ha.rst
proposalbot 3ccfbdc0f0 Changes to mrs_operation-guide from docs/doc-exports#475 (MRS component operatio 
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com>
Co-authored-by: proposalbot <proposalbot@otc-service.com>
Co-committed-by: proposalbot <proposalbot@otc-service.com>
2022-12-09 14:50:38 +00:00

13 KiB
Raw Blame History

original_name

mrs_01_1575.html

HA

Scenarios

The Flink HA mode depends on ZooKeeper. Therefore, ZooKeeper-related configuration items must be set.

Configuration Description

Configuration items include the ZooKeeper address, path, and security certificate.

Table 1 Parameters
Parameter Description Default Value Mandatory
high-availability

Whether HA is enabled. Only the following two modes are supported currently:

  1. none: Only a single JobManager is running. The checkpoint is disabled for JobManager.
  2. ZooKeeper:
    • In non-Yarn mode, multiple JobManagers are supported and the leader JobManager is elected.
    • In Yarn mode, only one JobManager exists.
 zookeeper No
high-availability.zookeeper.quorum ZooKeeper quorum address. Automatic configuration No
high-availability.zookeeper.path.root Root directory that Flink creates on ZooKeeper, storing metadata required in HA mode. /flink No
high-availability.storageDir Directory for storing JobManager metadata of state backend. ZooKeeper stores only pointers to actual data. hdfs:///flink/recovery No
high-availability.zookeeper.client.session-timeout Session timeout duration on the ZooKeeper client. The unit is millisecond. 60000 No
high-availability.zookeeper.client.connection-timeout Connection timeout duration on the ZooKeeper client. The unit is millisecond. 15000 No
high-availability.zookeeper.client.retry-wait Retry waiting time on the ZooKeeper client. The unit is millisecond. 5000 No
high-availability.zookeeper.client.max-retry-attempts Maximum retry times on the ZooKeeper client. 3 No
high-availability.job.delay Delay of job restart when JobManager recovers. The default value is the same as the value of akka.ask.timeout. No
high-availability.zookeeper.client.acl ACL (open creator) of the ZooKeeper node. For ACL options, see https://zookeeper.apache.org/doc/r3.5.1-alpha/zookeeperProgrammers.html#sc_BuiltinACLSchemes.

This parameter is configured automatically according to the cluster installation mode.

  • Security mode: The default value is creator.
  • Non-security mode: The default value is open.
 Yes
zookeeper.sasl.disable Simple authentication and security layer (SASL)-based certificate enable switch.

This parameter is configured automatically according to the cluster installation mode.

  • Security mode: The default value is false.
  • Non-security mode: The default value is true.
 Yes
zookeeper.sasl.service-name
  • If the ZooKeeper server configures a service whose name is different from ZooKeeper, this configuration item can be set.
  • If service names on the client and server are inconsistent, authentication fails.
 zookeeper Yes

Note

For versions earlier than MRS 3.x, the high-availability.job.delay parameter is not supported.