Hudi consumes much space in a temporary folder during upsert.
Hudi will spill part of input data to disk if the maximum memory for merge is reached when much input data is upserted.
If the memory is sufficient, increase the memory of the Spark executor and add the hoodie.memory.merge.fraction option, for example, option("hoodie.memory.merge.fraction", "0.8").