forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
86 lines
7.1 KiB
HTML
86 lines
7.1 KiB
HTML
<a name="mrs_01_24255"></a><a name="mrs_01_24255"></a>
|
|
|
|
<h1 class="topictitle1">Managing ENV</h1>
|
|
<div id="body0000001532950752"><div class="section" id="mrs_01_24255__section768174512333"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24255__p126201948193313">To capture data to or from Hudi, create and manage Hudi environment variables by performing the operations in this section.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24255__section296352011367"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_24255__p12142142311392">A user with the CDL management permission has been created for the cluster with Kerberos authentication enabled.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24255__section09718589438"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_24255__ol4252101010395"><li id="mrs_01_24255__li2138162063820"><span>Access the CDLService web UI as a user with the CDL management permissions or the <strong id="mrs_01_24255__b510619504578">admin</strong> user (for the cluster where Kerberos authentication is not enabled). For details, see <a href="mrs_01_24236.html">Logging In to the CDLService WebUI</a>.</span></li><li id="mrs_01_24255__li1713802043818"><span>Choose <strong id="mrs_01_24255__b360411126223">ENV Management</strong> and click <strong id="mrs_01_24255__b5101840192219">Add Env</strong>. In the displayed dialog box, set related parameters.</span><p>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24255__table162589447374" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters for adding an ENV</caption><thead align="left"><tr id="mrs_01_24255__row17256444103713"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.2.1.2.4.1.1"><p id="mrs_01_24255__p92557448375">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.2.1.2.4.1.2"><p id="mrs_01_24255__p182569448374">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.2.2.2.1.2.4.1.3"><p id="mrs_01_24255__p11256144123712">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24255__row52561644143710"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p15854172318813">Name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p128537230812">ENV name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p1485042317813">spark-env</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row16256174416371"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p1725634412377">Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p1256114411379">ENV type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p72561044173713">spark</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row125694414376"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p72567441378">Driver Memory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p925634413377">Memory for the driver process, in GB ( by default).</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p125624419377">1 GB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row2025624414373"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p16256194463714">Executor Memory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p825618444375">Memory size for each Executor process, in GB by default. Its string format is the same as that of JVM.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p11256164433716">1 GB</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row325664419376"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p52564442371">Executor Cores</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p122561544133718">Number of CPU cores occupied by each Executor</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p152566448373">1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row0257144418374"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p172575445378">Number Executors</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p11257124413712">Number of Executors</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p8257134413720">1</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row18257044143719"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p7257114463717">Queue</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p7257104463710">Name of the Yarn tenant queue. Jobs are submitted to the default queue if this parameter is not specified.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p225754418377">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24255__row1425816441370"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="mrs_01_24255__p525734473713">Description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><p id="mrs_01_24255__p14258124419373">ENV description</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="mrs_01_24255__p42582044133719">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</p></li><li id="mrs_01_24255__li1942598479"><span>Click <strong id="mrs_01_24255__b5944951187222">OK</strong>.</span><p><p id="mrs_01_24255__p062635934719">After the ENV is created, you can click <strong id="mrs_01_24255__b16872180125010">Edit</strong> or <strong id="mrs_01_24255__b7654812145017">Delete</strong> in the <strong id="mrs_01_24255__b328014242509">Operation</strong> column to edit or delete the ENV, respectively.</p>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24235.html">Preparing for Creating a CDL Job</a></div>
|
|
</div>
|
|
</div>
|
|
|