vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24056.html
Yang, Tong 3f5759eed2 MRS comp-lts 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2023-01-19 17:08:45 +00:00

5.0 KiB
Raw Permalink Blame History

Spark on OBS Tuning

Scenario

In the scenario where a small number of requests are frequently sent from Spark on OBS to OBS, you can disable OBS monitoring to improve performance.

Configuration

Modify the configuration in the core-site.xml file on the Spark client.
Table 1 Parameter description

Parameter

Description

Default Value

fs.obs.metrics.switch

Specifies whether to report OBS monitoring metrics.

  • true: enable
  • false: disable

true

fs.obs.metrics.consumer

Specifies the processing mode of OBS monitoring metrics.

  • org.apache.hadoop.fs.obs.metrics.OBSAMetricsProvider: indicates that OBS monitoring metrics are collected.
  • org.apache.hadoop.fs.obs.DefaultMetricsConsumer: indicates that OBS monitoring metrics are not collected.

To use the OBS monitoring function, ensure that the function of reporting OBS monitoring metrics is enabled.

org.apache.hadoop.fs.obs.metrics.OBSAMetricsProvider