forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
44 lines
5.0 KiB
HTML
44 lines
5.0 KiB
HTML
<a name="mrs_01_24056"></a><a name="mrs_01_24056"></a>
|
|
|
|
<h1 class="topictitle1">Spark on OBS Tuning</h1>
|
|
<div id="body32001227"><div class="section" id="mrs_01_24056__en-us_topic_0000001173631392_section1963647104716"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_24056__en-us_topic_0000001173631392_p2066714782412">In the scenario where a small number of requests are frequently sent from Spark on OBS to OBS, you can disable OBS monitoring to improve performance.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24056__en-us_topic_0000001173631392_section10560181714617"><h4 class="sectiontitle">Configuration</h4><div class="p" id="mrs_01_24056__en-us_topic_0000001173631392_p478872024612">Modify the configuration in the <span class="filepath" id="mrs_01_24056__en-us_topic_0000001173631392_f389335c59bc940f3880bdc2e4e07b73e"><b>core-site.xml</b></span> file on the Spark client.
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24056__en-us_topic_0000001173631392_t0f01c19bf3d342d69b1e43165d3651c4" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_24056__en-us_topic_0000001173631392_rb9ef34d486ad4457b7c140ac7c5243e8"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.2.2.2.2.4.1.1"><p id="mrs_01_24056__en-us_topic_0000001173631392_a05b274f73c3041f6a34dd69c90341c4f"><strong id="mrs_01_24056__en-us_topic_0000001173631392_a2b8ad43fef16448cbae98294af82f75f">Parameter</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.2.2.2.2.4.1.2"><p id="mrs_01_24056__en-us_topic_0000001173631392_a8e83292fff7743b490d6d0d245bd6738"><strong id="mrs_01_24056__en-us_topic_0000001173631392_a439a37b8371a4f64a0930d0c033d213c">Description</strong></p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.2.2.2.2.4.1.3"><p id="mrs_01_24056__en-us_topic_0000001173631392_a6d9950c769ba499b8b18f1556640b762"><strong id="mrs_01_24056__en-us_topic_0000001173631392_a993e58aa44114f2196c326c77f6329b2">Default Value</strong></p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24056__en-us_topic_0000001173631392_rb515ca4fb5d94847b8c421e7a81db874"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.2.4.1.1 "><p id="mrs_01_24056__en-us_topic_0000001173631392_a073693eb4a7441568d003eed9ed60697">fs.obs.metrics.switch</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.2.2.4.1.2 "><p id="mrs_01_24056__en-us_topic_0000001173631392_af4ddd8ddb91944dca86307f5f2e9b23a">Specifies whether to report OBS monitoring metrics.</p>
|
|
<ul id="mrs_01_24056__en-us_topic_0000001173631392_uea6421717be94a15bd8509940df3c474"><li id="mrs_01_24056__en-us_topic_0000001173631392_l4d3160acde444f3aa96b5543b28839af"><strong id="mrs_01_24056__en-us_topic_0000001173631392_b2795567759044">true</strong>: enable</li><li id="mrs_01_24056__en-us_topic_0000001173631392_l47ff8e44339d4850980ac5c63fdb415d"><strong id="mrs_01_24056__en-us_topic_0000001173631392_b9871992839044">false</strong>: disable</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.2.4.1.3 "><p id="mrs_01_24056__en-us_topic_0000001173631392_a4bc2f3c6b42d4ecdb7a5e5a417f43ae3">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24056__en-us_topic_0000001173631392_r20c07787337549d3942c7d5b9a663e0b"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.2.4.1.1 "><p id="mrs_01_24056__en-us_topic_0000001173631392_a2d8493df5d9d420e8bc79f89eb4919b3">fs.obs.metrics.consumer</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.2.2.2.2.4.1.2 "><p id="mrs_01_24056__en-us_topic_0000001173631392_acc47cf33f68c42a58b1d7a20216d5c0d">Specifies the processing mode of OBS monitoring metrics.</p>
|
|
<ul id="mrs_01_24056__en-us_topic_0000001173631392_u107898c36a0b43d18ea673a6af8f60f1"><li id="mrs_01_24056__en-us_topic_0000001173631392_l2f8b505484fe4b21bc4fd03b7216c024"><strong id="mrs_01_24056__en-us_topic_0000001173631392_b16944181542716">org.apache.hadoop.fs.obs.metrics.OBSAMetricsProvider</strong>: indicates that OBS monitoring metrics are collected.</li><li id="mrs_01_24056__en-us_topic_0000001173631392_lc51aa79d44384e17bfb8a575899a22a9"><strong id="mrs_01_24056__en-us_topic_0000001173631392_b183141243192717">org.apache.hadoop.fs.obs.DefaultMetricsConsumer</strong>: indicates that OBS monitoring metrics are not collected.</li></ul>
|
|
<p id="mrs_01_24056__en-us_topic_0000001173631392_a45b93eff1fc04058acd31f5f525e7389">To use the OBS monitoring function, ensure that the function of reporting OBS monitoring metrics is enabled.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.2.4.1.3 "><p id="mrs_01_24056__en-us_topic_0000001173631392_adc1a7e5d65b24052a58e6e641c4ebd02">org.apache.hadoop.fs.obs.metrics.OBSAMetricsProvider</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1974.html">Spark2x Performance Tuning</a></div>
|
|
</div>
|
|
</div>
|
|
|