In Spark WebUI, the Executor page can display information about Lost Executor. Executors are dynamically recycled. If the JDBCServer tasks are large, there may be too many lost executors displayed in WebUI. Therefore, the number of displayed lost executors can be configured.
Configure the following parameter in the spark-defaults.conf file on Spark client.
Parameter |
Description |
Default Value |
|---|---|---|
spark.ui.retainedDeadExecutors |
The maximum number of Lost Executors displayed in Spark WebUI. |
100 |