This command is used to dynamically add, update, display, or reset Hudi parameters without restarting the driver.
SET parameter_name=parameter_value
This command is used to add or update the value of parameter_name.
This command is used to display the value of parameter_name.
This command is used to display all supported session parameters.
This command is used to display all supported session parameters and their usage details.
This command is used to reset all session parameters.
Parameter |
Description |
---|---|
parameter_name |
Name of the parameter to be dynamically added, updated, or displayed. |
parameter_value |
New value to be set for parameter_name. |
The following table lists the properties to be used in the SET or RESET commands.
Property |
Description |
---|---|
hoodie.insert.shuffle.parallelism |
Degree of parallelism (DOP) of Spark shuffle for writing data in insert mode. |
hoodie.upsert.shuffle.parallelism |
DOP of Spark shuffle for writing data in upsert mode. |
hoodie.delete.shuffle.parallelism |
DOP of Spark shuffle for deleting data in delete mode. |
hoodie.sql.insert.mode |
Insert mode. The value can be strict, non-strict, or upsert. |
hoodie.sql.bulk.insert.enable |
Whether to enable bulk insert. |
spark.sql.hive.convertMetastoreParquet |
Converts the parquet table into a data source table for reading. If the provider of Hudi is Hive and Spark SQL or Spark Beeline is used to read data, set this parameter to false. |
set hoodie.insert.shuffle.parallelism = 100; set hoodie.upsert.shuffle.parallelism = 100; set hoodie.delete.shuffle.parallelism = 100;
RESET