forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
143 lines
14 KiB
HTML
143 lines
14 KiB
HTML
<a name="mrs_01_1575"></a><a name="mrs_01_1575"></a>
|
|
|
|
<h1 class="topictitle1">HA</h1>
|
|
<div id="body1596163871758"><div class="section" id="mrs_01_1575__sa9997b680e4c47d196a217137d3a82f1"><h4 class="sectiontitle">Scenarios</h4><p id="mrs_01_1575__a5b6ceb6bb6a34411a2b08187a1ec7633">The Flink HA mode depends on ZooKeeper. Therefore, ZooKeeper-related configuration items must be set.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1575__s07be7718764e491ab6454b506bb0506a"><h4 class="sectiontitle">Configuration Description</h4><p id="mrs_01_1575__a423bbb824aaf4c68af86bfcc617957e3">Configuration items include the ZooKeeper address, path, and security certificate.</p>
|
|
|
|
<div class="tablenoborder"><a name="mrs_01_1575__ta903d6a9c6d24f72abdf46625096cd8c"></a><a name="ta903d6a9c6d24f72abdf46625096cd8c"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1575__ta903d6a9c6d24f72abdf46625096cd8c" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="mrs_01_1575__r97bf10df33a34c99a6500196e8d03baf"><th align="left" class="cellrowborder" valign="top" width="21.22%" id="mcps1.3.2.3.2.5.1.1"><p id="mrs_01_1575__ad886d214a2bc4e34b0969f6bb0fdadc1">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="43.6%" id="mcps1.3.2.3.2.5.1.2"><p id="mrs_01_1575__a4169e94da8934565ab86f6822cdd55a8">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="22.38%" id="mcps1.3.2.3.2.5.1.3"><p id="mrs_01_1575__abb8dd8fda9234cda88b23b21980a8821">Default Value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="12.8%" id="mcps1.3.2.3.2.5.1.4"><p id="mrs_01_1575__ab0fc35cf3dde49c4931c2464f0ed9767">Mandatory</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1575__r7b2d7551c25d4c7696d54e390624f892"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a075dd6f7ff274e5f9e0a7c7fd64e5bca">high-availability</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a9ec08f53a6624e09a3a5ddc2601316f5">Whether HA is enabled. Only the following two modes are supported currently:</p>
|
|
<ol id="mrs_01_1575__o3d917f1f0df14eba81dc8ec78fdc3ca9"><li id="mrs_01_1575__l0ea8eef7dba641d6a10369d35be1f132">none: Only a single JobManager is running. The checkpoint is disabled for JobManager.</li><li id="mrs_01_1575__l69abac0c2aa84e3b826a4a4eff99b583">ZooKeeper:<ul id="mrs_01_1575__ufa266da3ae664701bf6d892535866b50"><li id="mrs_01_1575__lb236a4b0e4ff41188795047f35e68dbb">In non-Yarn mode, multiple JobManagers are supported and the leader JobManager is elected.</li><li id="mrs_01_1575__l274e25971e744cbf8dfe0aa473e2fb9b">In Yarn mode, only one JobManager exists.</li></ul>
|
|
</li></ol>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a719807502286464f885d1ceabbe95eed">zookeeper</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a99df5eeb04c047908fa8c971e2746cc4">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__rf711b396b7b44d6ebbec36761f0960c7"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__aebe58a4fa4494f08af58427cafc0f9ad">high-availability.zookeeper.quorum</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a35ecf0be7d8644a38e869484f6e06f6a">ZooKeeper quorum address.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__adc1b6b80e57d4c27be69bd4fb576aedc">Automatic configuration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a92e7751f31f9496c968bac59d4f48934">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__r81767e596ab340a8bb8579fe4c72df66"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__ae09640f8ce664134a61861ba83574ef2">high-availability.zookeeper.path.root</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a71cabacaa71f43a18ced480cba982f13">Root directory that Flink creates on ZooKeeper, storing metadata required in HA mode.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a08ccdbb7244948e6bb5bbf22cf9965d5">/flink</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a9187e77c97da4bc8b04274f4541992df">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__r430d779098bc4c1681498f0ef248b97a"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a4bb56b67c6a84da38f16983057510aa7">high-availability.storageDir</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a5e9271eacc0044e4b5cce755366849d2">Directory for storing JobManager metadata of state backend. ZooKeeper stores only pointers to actual data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a86fa295f9dc349548802e8ba1a0d1c2d">hdfs:///flink/recovery</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__ae063841aad4146c6aa0ff64ba23032da">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__re497ba8540924cf5b0216db7a42fa9f2"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a2ef2b3916af84caa87103ddf4ba6b74f">high-availability.zookeeper.client.session-timeout</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__aa8837f3ce5bf40be99bc4c46c9093cff">Session timeout duration on the ZooKeeper client. The unit is millisecond.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__ab49c867583a34135b8f43c4f7b30d95b">60000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a1219e1f0b15e40989a15e14add50e821">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__r636790cf4c554c61ae090ee835e18ce6"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a5710b91fae3d4dcda963f4bd4ee87f34">high-availability.zookeeper.client.connection-timeout</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a5d221e6b7a2c428fad21118e964e5297">Connection timeout duration on the ZooKeeper client. The unit is millisecond.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a7d6bce9a73f74c3f9a8d8d4ae53fa333">15000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a9076fe6bc18f4ddd98a913fec01304ec">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__rf9a65bf9f0b7406ea1c4560317144a20"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__af5703309569f49e792cfb8b904a916bc">high-availability.zookeeper.client.retry-wait</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a63e98a795f394a7f800ef8632689ed29">Retry waiting time on the ZooKeeper client. The unit is millisecond.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__adfae27b6125b4a25bb32a12332f8f36e">5000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a8e399d96fd794723824d47fd68af9c99">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__r7ed5a3c08813452e99361dafb41b6f7e"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a424bd651fe964b4892cc03e5b7fbf649">high-availability.zookeeper.client.max-retry-attempts</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a0a2ee51e66cc403bb50d6908986b3b44">Maximum retry times on the ZooKeeper client.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a092d3380044948e0a904e830044eff78">3</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a73b3e933b8954c78adc50bc1e27e1470">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__rc810f3cb742943fc8c3918a50b43a855"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__aec08a8d893334da3bd91ca334a6e25e5">high-availability.job.delay</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__abe418a8427e340d3b686c814d98656f6">Delay of job restart when JobManager recovers.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a1fe8ca778279450a839c299212fdf0b7">The default value is the same as the value of <strong id="mrs_01_1575__b182111317531">akka.ask.timeout</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__aedae206af8c6446f9e1924f14e64ffb9">No</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__r6a52a715dcf14d00a91e639d2052e463"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a9f2e4ebddf10408b984902f0c4dee646">high-availability.zookeeper.client.acl</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__afbdecc6e2df643f295a93766fb218053">ACL (open creator) of the ZooKeeper node. For ACL options, see <a href="https://zookeeper.apache.org/doc/r3.5.1-alpha/zookeeperProgrammers.html#sc_BuiltinACLSchemes" target="_blank" rel="noopener noreferrer">https://zookeeper.apache.org/doc/r3.5.1-alpha/zookeeperProgrammers.html#sc_BuiltinACLSchemes</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__ab994dea9db5e420bb6f7eb92259764d4">This parameter is configured automatically according to the cluster installation mode.</p>
|
|
<ul id="mrs_01_1575__en-us_topic_0085562725_ul15724245598"><li id="mrs_01_1575__la281cb3e9ad74044bf043c6bf9b153cf">Security mode: The default value is <strong id="mrs_01_1575__b83203017189">creator</strong>.</li><li id="mrs_01_1575__ldd97cc0b24024ad39357351ba26f401d">Non-security mode: The default value is <strong id="mrs_01_1575__b619162871101217">open</strong>.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a7f0bcdb30986406ea24b778a011a20a8">Yes</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__rd11963af790b476297767f574c4372c5"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a024bb4a6f2784dd8a44aad5ef10ae853">zookeeper.sasl.disable</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><p id="mrs_01_1575__a46ebedf0004d45f8a6cc0a5d826be31a">Simple authentication and security layer (SASL)-based certificate enable switch.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__a9e412df4e4f247aea08f4765a5fd41b1">This parameter is configured automatically according to the cluster installation mode.</p>
|
|
<ul id="mrs_01_1575__ua22b0900789341a59828cf3418329e53"><li id="mrs_01_1575__l75c5f00db38f426fa37c486ee53f6d15">Security mode: The default value is <strong id="mrs_01_1575__b414581218196">false</strong>.</li><li id="mrs_01_1575__l8711e50152514ca196905c91a88f586d">Non-security mode: The default value is <strong id="mrs_01_1575__b1815596866101217">true</strong>.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__ab4117fb77426414b8e06e0ee6b6751d6">Yes</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1575__rbff56588212e4d57ab6762321842cb57"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.2.3.2.5.1.1 "><p id="mrs_01_1575__a88d6ec71d7a94547b9ba36b1eba6e700">zookeeper.sasl.service-name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.6%" headers="mcps1.3.2.3.2.5.1.2 "><ul id="mrs_01_1575__uf54849655d074d7e96c30cbe305c6e68"><li id="mrs_01_1575__lb7d59c2b7036449096af96138ae9025a">If the ZooKeeper server configures a service whose name is different from <span class="parmname" id="mrs_01_1575__pf0fe742449ea4dce99669263c6025901"><b>ZooKeeper</b></span>, this configuration item can be set.</li><li id="mrs_01_1575__l9bf4f61a76f5434cb109a6b696c80827">If service names on the client and server are inconsistent, authentication fails.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="22.38%" headers="mcps1.3.2.3.2.5.1.3 "><p id="mrs_01_1575__ae85eb66b8b0447e099b9f8bed88e2a53">zookeeper</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="12.8%" headers="mcps1.3.2.3.2.5.1.4 "><p id="mrs_01_1575__a362f2e97c1d94b558fcbef75d523574b">Yes</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div class="note" id="mrs_01_1575__note1714552293913"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1575__p111461522153916">For versions earlier than MRS 3.x, the <strong id="mrs_01_1575__b1359073452012">high-availability.job.delay</strong> parameter is not supported.</p>
|
|
</div></div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0592.html">Flink Configuration Management</a></div>
|
|
</div>
|
|
</div>
|
|
|