forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
21 lines
2.1 KiB
HTML
21 lines
2.1 KiB
HTML
<a name="mrs_01_24504"></a><a name="mrs_01_24504"></a>
|
|
|
|
<h1 class="topictitle1">Hudi Fails to Write Decimal Data with Lower Precision</h1>
|
|
<div id="body0000001208420474"><div class="section" id="mrs_01_24504__section1453810451898"><h4 class="sectiontitle">Question</h4><p id="mrs_01_24504__p16821131013553">Decimal data is initially written to a Hudi table using the <strong id="mrs_01_24504__b263581081817">BULK_INSERT</strong> command. Then when data is subsequently written using <strong id="mrs_01_24504__b722155217187">UPSERT</strong>, the following error is reported:</p>
|
|
<pre class="screen" id="mrs_01_24504__screen63515448334">java.lang.UnsupportedOperationException: org.apache.parquet.avro.AvroConverters$FieldFixedConverter</pre>
|
|
</div>
|
|
<div class="section" id="mrs_01_24504__section11310239101"><h4 class="sectiontitle">Answer</h4><p id="mrs_01_24504__p179911820134310"><strong id="mrs_01_24504__b10509927184312">Cause:</strong></p>
|
|
<p id="mrs_01_24504__p53358525561">The Hudi table contains decimal data.</p>
|
|
<p id="mrs_01_24504__p106921123105712">The initial bulk insert of data is implemented using the Spark class for writing Parquet files. However, Spark processes the decimal data with different precisions differently.</p>
|
|
<p id="mrs_01_24504__p10846113375816">When data is written using the <strong id="mrs_01_24504__b14858681221">UPSERT</strong> command, Hudi uses the Avro-compliant class for writing Parquet files, which is incompatible with the Spark class.</p>
|
|
<p id="mrs_01_24504__p34891548165917"><strong id="mrs_01_24504__b173631623124213">Solutions:</strong></p>
|
|
<p id="mrs_01_24504__p32764811019">When executing the <strong id="mrs_01_24504__b125681322811">BULK_INSERT</strong> command, set <strong id="mrs_01_24504__b157921817914">hoodie.datasource.write.row.writer.enable</strong> to <strong id="mrs_01_24504__b15440171615910">false</strong> to enable Hoodie to use the Avro-compliant class for writing Parquet files.</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24070.html">Data Write</a></div>
|
|
</div>
|
|
</div>
|
|
|