forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
35 lines
3.0 KiB
HTML
35 lines
3.0 KiB
HTML
<a name="mrs_01_1947"></a><a name="mrs_01_1947"></a>
|
||
|
||
<h1 class="topictitle1">Configuring Executor Off-Heap Memory</h1>
|
||
<div id="body1595920206811"><div class="section" id="mrs_01_1947__s58b91abedccd418a83f24d854d77675c"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1947__a3b3d5ce6b42845118f5a6b3f85275bbb">When the executor off-heap memory is too small, or processes with higher priority preempt resources, the physical memory usage will exceed the maximal value. To prevent the physical memory usage from exceeding, set the following parameter.</p>
|
||
</div>
|
||
<div class="section" id="mrs_01_1947__sd7ef7f618ea8436f9a32b9ad56042837"><h4 class="sectiontitle">Configuration</h4><p id="mrs_01_1947__afb4983eb6997447b891ac1f013fafdea"><strong id="mrs_01_1947__b1878324225519">Navigation path for setting parameters:</strong></p>
|
||
<p id="mrs_01_1947__a11a6d681b63f4231b3d718cfb66ba388">When submitting an application, set the following parameter using <span class="parmname" id="mrs_01_1947__parmname11360105535512"><b>--conf</b></span> or adjust the parameter in the <span class="filepath" id="mrs_01_1947__filepath133651155175511"><b>spark-defaults.conf</b></span> configuration file on the client.</p>
|
||
|
||
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1947__tfaf6bf89c678496d9495e263777b4fa0" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_1947__r0d8c1e8c9d1e49479304c18d7dbfa02e"><th align="left" class="cellrowborder" valign="top" width="20.18%" id="mcps1.3.2.4.2.4.1.1"><p id="mrs_01_1947__a7c135258b3f74b3aae04647b544b265f">Parameter</p>
|
||
</th>
|
||
<th align="left" class="cellrowborder" valign="top" width="63.970000000000006%" id="mcps1.3.2.4.2.4.1.2"><p id="mrs_01_1947__ad353ac8cb5cd4efb892e88e1a06a5a06">Description</p>
|
||
</th>
|
||
<th align="left" class="cellrowborder" valign="top" width="15.85%" id="mcps1.3.2.4.2.4.1.3"><p id="mrs_01_1947__afaecfdedf90141bc8da11af0e8e9e1b0">Default Value</p>
|
||
</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody><tr id="mrs_01_1947__rafd648d1e3874d86b171ee1a4039f302"><td class="cellrowborder" valign="top" width="20.18%" headers="mcps1.3.2.4.2.4.1.1 "><p id="mrs_01_1947__a7c220f4a9d7f4ac6909e0d046040a482">spark.executor.memoryOverhead</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="63.970000000000006%" headers="mcps1.3.2.4.2.4.1.2 "><p id="mrs_01_1947__a0cc173494d4543ff8f8b77f684da5fdc">Indicates the off-heap memory of each executor, in MB. Increasing the value of this parameter prevents the physical memory usage from exceeding the maximal value. The value is calculated based on max(384, Executor – Memory x 0.1). The minimal value is 384.</p>
|
||
</td>
|
||
<td class="cellrowborder" valign="top" width="15.85%" headers="mcps1.3.2.4.2.4.1.3 "><p id="mrs_01_1947__a605d75ef92bf4ee58bbb7303790f0872">1024</p>
|
||
</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div>
|
||
<div class="familylinks">
|
||
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1941.html">Scenario-Specific Configuration</a></div>
|
||
</div>
|
||
</div>
|
||
|