forked from docs/doc-exports
Wuwan, Qi
050b395397
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Wuwan, Qi <wuwanqi1@noreply.gitea.eco.tsi-dev.otc-service.com> Co-committed-by: Wuwan, Qi <wuwanqi1@noreply.gitea.eco.tsi-dev.otc-service.com>
123 lines
13 KiB
HTML
123 lines
13 KiB
HTML
<a name="css_01_0142"></a><a name="css_01_0142"></a>
|
||
|
||
<h1 class="topictitle1">Memory Flow Control</h1>
|
||
<div id="body0000001273212041"><div class="section" id="css_01_0142__en-us_topic_0000001273212041_section2344204318530"><h4 class="sectiontitle">Context</h4><p id="css_01_0142__en-us_topic_0000001273212041_p7185363169">Elasticsearch provides a circuit breaker, which will terminate requests if the memory usage exceeds its threshold. However, Elasticsearch does not check the heap memory usage when an API is called, and does not allow users to configure the threshold for a single request. In this case, memory usage can only be calculated during request processing, which may lead to frequent circuit breaking and cannot avoid heap memory waste. To solve this problem, CSS checks the heap memory usage when receiving REST requests, blocking excess API requests and protecting nodes. You can configure global memory flow control, or configure the request path and heap memory threshold for a specific request path. Before a request is processed, the system checks the configured heap memory threshold. If the threshold is exceeded, the request path will be blocked.</p>
|
||
<div class="note" id="css_01_0142__en-us_topic_0000001273212041_note1774592819415"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="css_01_0142__en-us_topic_0000001273212041_ul54871944416"><li id="css_01_0142__en-us_topic_0000001273212041_li548794412111">Memory flow control may affect request processing performance.</li><li id="css_01_0142__en-us_topic_0000001273212041_li154871444010">If the memory flow control is enabled, some Kibana search requests may fail.</li><li id="css_01_0142__en-us_topic_0000001273212041_li17487144316">If memory flow control is enabled in Elasticsearch 5.5.1, _mget requests will be blocked and Kibana access will be abnormal. You can add _mget requests to the request whitelist to avoid this problem.</li></ul>
|
||
</div></div>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p1327218103711">The following table describes memory flow control parameters.</p>
|
||
|
||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="css_01_0142__en-us_topic_0000001273212041_table51107567182" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Memory flow control parameters</caption><thead align="left"><tr id="css_01_0142__en-us_topic_0000001273212041_row1611015564187"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.1.5.2.4.1.1"><p id="css_01_0142__en-us_topic_0000001273212041_p10110105617188">Parameter</p>
|
||
</th>
|
||
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.1.5.2.4.1.2"><p id="css_01_0142__en-us_topic_0000001273212041_p81101456141815">Type</p>
|
||
</th>
|
||
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.1.5.2.4.1.3"><p id="css_01_0142__en-us_topic_0000001273212041_p411085651815">Description</p>
|
||
</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody><tr id="css_01_0142__en-us_topic_0000001273212041_row1711035641816"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.1 "><p id="css_01_0142__en-us_topic_0000001273212041_p511095615180">flowcontrol.memory.enabled</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.2 "><p id="css_01_0142__en-us_topic_0000001273212041_p71102056121819">Boolean</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.3 "><p id="css_01_0142__en-us_topic_0000001273212041_p4110125631819">Whether to enable memory flow control. This function is disabled by default. Enabling memory flow control may slightly affect node performance.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p973514386510">Value: <strong id="css_01_0142__en-us_topic_0000001273212041_b127239925531655">true</strong> or <strong id="css_01_0142__en-us_topic_0000001273212041_b209477388231655">false</strong></p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p815195415510">Default value: <strong id="css_01_0142__en-us_topic_0000001273212041_b124273273831655">false</strong></p>
|
||
</td>
|
||
</tr>
|
||
<tr id="css_01_0142__en-us_topic_0000001273212041_row811035611820"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.1 "><p id="css_01_0142__en-us_topic_0000001273212041_p111019565188">flowcontrol.memory.allow_path</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.2 "><p id="css_01_0142__en-us_topic_0000001273212041_p15110205631817">List<String></p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.3 "><p id="css_01_0142__en-us_topic_0000001273212041_p1636619409316">Request path whitelist for memory flow control.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p21101356111815">Whitelisted paths are blocked in memory flow control. Wildcard characters are supported. By default, query APIs controlled by the cluster are not blocked in memory flow control. This prevents the failure to query cluster information when the memory usage reaches the threshold.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p3898131722317">Example:</p>
|
||
<ul id="css_01_0142__en-us_topic_0000001273212041_ul153051554236"><li id="css_01_0142__en-us_topic_0000001273212041_li183057546319">"flowcontrol.memory.allow_path": "/index/_search",</li><li id="css_01_0142__en-us_topic_0000001273212041_li53055547316">"flowcontrol.memory.allow_path": "/index*/_search",</li><li id="css_01_0142__en-us_topic_0000001273212041_li12305195411319">"flowcontrol.memory.allow_path": ["/index/_search", "/index1/_bulk"],</li></ul>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p18641117206">A maximum of 10 paths can be configured. A path can contain up to 32 characters.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p154576111162">The default value is null.</p>
|
||
</td>
|
||
</tr>
|
||
<tr id="css_01_0142__en-us_topic_0000001273212041_row1011035651815"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.1 "><p id="css_01_0142__en-us_topic_0000001273212041_p171101456171820">flowcontrol.memory.heap_limit</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.2 "><p id="css_01_0142__en-us_topic_0000001273212041_p611012562183">String</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.3 "><p id="css_01_0142__en-us_topic_0000001273212041_p10110156181811">Maximum global heap memory usage of a node. The value cannot be less than 10% of the heap memory.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p149741557373">Value range: 10%–100%</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p59459111814">Default value: <strong id="css_01_0142__en-us_topic_0000001273212041_b184540297131655">90%</strong></p>
|
||
</td>
|
||
</tr>
|
||
<tr id="css_01_0142__en-us_topic_0000001273212041_row12110195651817"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.1 "><p id="css_01_0142__en-us_topic_0000001273212041_p68159811202">flowcontrol.memory.*.filter_path</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.2 "><p id="css_01_0142__en-us_topic_0000001273212041_p31106569187">String</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.3 "><p id="css_01_0142__en-us_topic_0000001273212041_p1429793752612">Paths under memory flow control.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p4388542586">The default value is <strong id="css_01_0142__en-us_topic_0000001273212041_b2501522122418">**</strong>, indicating all paths. If <strong id="css_01_0142__en-us_topic_0000001273212041_b1148310262519">flowcontrol.memory.heap_limit</strong> is configured and <strong id="css_01_0142__en-us_topic_0000001273212041_b143385942513">flowcontrol.memory.*.filter_path</strong> is not, it indicates that all the paths, except those in the whitelist, are under control. The whitelist takes precedence over the single-path rule. If a path is specified in both <strong id="css_01_0142__en-us_topic_0000001273212041_b198853812711">flowcontrol.memory.allow_path</strong> and <strong id="css_01_0142__en-us_topic_0000001273212041_b101143424279">flowcontrol.memory.*.filter_path</strong>, the requests from the path will be allowed.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p1634629165712">For example, if <strong id="css_01_0142__en-us_topic_0000001273212041_b1139123252916">flowcontrol.memory.allow_path</strong> and <strong id="css_01_0142__en-us_topic_0000001273212041_b576811472309">flowcontrol.memory.*.filter_path</strong> are both set to <strong id="css_01_0142__en-us_topic_0000001273212041_b11152451143011">abc/_search</strong>, then <strong id="css_01_0142__en-us_topic_0000001273212041_b18740139103112">abc/_search</strong> will not be under flow control.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p185154152198">Maximum length: <strong id="css_01_0142__en-us_topic_0000001273212041_b51931858122511">32</strong> characters</p>
|
||
</td>
|
||
</tr>
|
||
<tr id="css_01_0142__en-us_topic_0000001273212041_row1211085651816"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.1 "><p id="css_01_0142__en-us_topic_0000001273212041_p14341120122018">flowcontrol.memory.*.heap_limit</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.2 "><p id="css_01_0142__en-us_topic_0000001273212041_p181101856101819">String</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.1.5.2.4.1.3 "><p id="css_01_0142__en-us_topic_0000001273212041_p139661844135915">Heap memory usage threshold of request paths. If the heap memory usage exceeds the threshold, flow control will be triggered.</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p16110175615180">Value range: 0–100%</p>
|
||
<p id="css_01_0142__en-us_topic_0000001273212041_p1368317917519">Default value: <strong id="css_01_0142__en-us_topic_0000001273212041_b615141073314">90%</strong></p>
|
||
</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="css_01_0142__en-us_topic_0000001273212041_section11982148192117"><h4 class="sectiontitle">Procedure</h4><ol id="css_01_0142__en-us_topic_0000001273212041_ol927111214106"><li id="css_01_0142__en-us_topic_0000001273212041_li1274916552817">Log in to the CSS management console.</li><li id="css_01_0142__en-us_topic_0000001273212041_li1274935516811">Choose <strong id="css_01_0142__en-us_topic_0000001273212041_b1290191555419">Clusters</strong> in the navigation pane. On the <strong id="css_01_0142__en-us_topic_0000001273212041_b185921524102917">Clusters</strong> page, locate the target cluster and click <strong id="css_01_0142__en-us_topic_0000001273212041_b14593224192911">Access Kibana</strong> in the <strong id="css_01_0142__en-us_topic_0000001273212041_b165935246299">Operation</strong> column.</li><li id="css_01_0142__en-us_topic_0000001273212041_li927171291011">In the navigation pane on the left, choose <strong id="css_01_0142__en-us_topic_0000001273212041_b179531348332">Dev Tools</strong> and run commands to enable or disable memory flow control.<ul id="css_01_0142__en-us_topic_0000001273212041_ul176348454417"><li id="css_01_0142__en-us_topic_0000001273212041_li1563417414445">Enabling memory flow control<pre class="screen" id="css_01_0142__en-us_topic_0000001273212041_screen1857516140228">PUT /_cluster/settings
|
||
{
|
||
"persistent": {
|
||
"flowcontrol.memory.enabled": true,
|
||
"flowcontrol.memory.allow_path": "/index/_search",
|
||
"flowcontrol.memory.heap_limit": "85%"
|
||
}
|
||
}</pre>
|
||
</li><li id="css_01_0142__en-us_topic_0000001273212041_li148087316239">Enabling memory flow control for a request path<p id="css_01_0142__en-us_topic_0000001273212041_p48087317232"><a name="css_01_0142__en-us_topic_0000001273212041_li148087316239"></a><a name="en-us_topic_0000001273212041_li148087316239"></a>Configure the heap memory usage threshold for a request path. You can configure the priorities of such threshold rules.</p>
|
||
<pre class="screen" id="css_01_0142__en-us_topic_0000001273212041_screen198083319239">PUT /_cluster/settings
|
||
{
|
||
"persistent": {
|
||
"flowcontrol.memory.enabled": true,
|
||
"flowcontrol.memory": {
|
||
"flowcontrol_search": {
|
||
"filter_path": "index1/_search",
|
||
"heap_limit": "50%"
|
||
},
|
||
"flowcontrol_bulk": {
|
||
"filter_path": "index*/_bulk",
|
||
"heap_limit": "50%"
|
||
}
|
||
}
|
||
}
|
||
}</pre>
|
||
</li><li id="css_01_0142__en-us_topic_0000001273212041_li1380843172317">Deleting the memory flow control configuration of a request path<pre class="screen" id="css_01_0142__en-us_topic_0000001273212041_screen128083352312">PUT /_cluster/settings
|
||
{
|
||
"persistent": {
|
||
"flowcontrol.memory.enabled": true,
|
||
"flowcontrol.memory": {
|
||
"flowcontrol_search": {
|
||
"filter_path": null,
|
||
"heap_limit": null
|
||
}
|
||
}
|
||
}
|
||
}</pre>
|
||
</li><li id="css_01_0142__en-us_topic_0000001273212041_li680915352319">Disabling cluster memory flow control<pre class="screen" id="css_01_0142__en-us_topic_0000001273212041_screen38095315235">PUT /_cluster/settings
|
||
{
|
||
"persistent": {
|
||
"flowcontrol.memory.enabled": false
|
||
}
|
||
}</pre>
|
||
</li></ul>
|
||
</li></ol>
|
||
</div>
|
||
</div>
|
||
<div>
|
||
<div class="familylinks">
|
||
<div class="parentlink"><strong>Parent topic:</strong> <a href="css_01_0139.html">Flow Control 1.0</a></div>
|
||
</div>
|
||
</div>
|
||
|