forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
18 lines
1.3 KiB
HTML
18 lines
1.3 KiB
HTML
<a name="mrs_01_2025"></a><a name="mrs_01_2025"></a>
|
|
|
|
<h1 class="topictitle1">How to Assign a Parameter Value in a Spark Command?</h1>
|
|
<div id="body1595920220420"><div class="section" id="mrs_01_2025__s55fad1d7c11d4e4e8b3f259564db9ec8"><h4 class="sectiontitle">Question</h4><p id="mrs_01_2025__a3be4aa975ecb4a57ae4ef43a34c75936">Is it possible to assign parameter values through Spark commands, in addition to through a user interface or a configuration file?</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_2025__se14fda3442194e248686a55c12c49574"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_2025__a46012f41e4ac4749b5d2d73392fcc5b2">Spark configuration options can be defined either in a configuration file or in Spark commands.</p>
|
|
<p id="mrs_01_2025__ab99e492c08354a88840bc63e655758c3">To assign a parameter value, run the --conf command on a Spark client. The parameter value takes effect immediately after the command is run.</p>
|
|
<p id="mrs_01_2025__a3adddb6577e8440283deeb7d79d26085">The command format is --conf + parameter name + parameter value. Example command:</p>
|
|
<p id="mrs_01_2025__afdbafe2b4910431e878ba837c36ccca0">--conf spark.eventQueue.size=50000</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_2022.html">Spark SQL and DataFrame</a></div>
|
|
</div>
|
|
</div>
|
|
|