forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
64 lines
5.7 KiB
HTML
64 lines
5.7 KiB
HTML
<a name="mrs_01_24781"></a><a name="mrs_01_24781"></a>
|
|
|
|
<h1 class="topictitle1">CLEAN_FILE</h1>
|
|
<div id="body0000001585034877"><div class="section" id="mrs_01_24781__section143972345248"><h4 class="sectiontitle">Function</h4><p id="mrs_01_24781__p19553184312246">Cleans invalid data files from the Hudi table directory.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24781__section27787498240"><h4 class="sectiontitle">Syntax</h4><p id="mrs_01_24781__p163011553182412"><strong id="mrs_01_24781__b105984985815">call clean_file</strong>(table => '[table_name]', mode=>'[op_type]', backup_path=>'[backup_path]', start_instant_time=>'[start_time]', end_instant_time=>'[end_time]');</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24781__section96316555241"><h4 class="sectiontitle">Parameter Description</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24781__table213494418406" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="mrs_01_24781__row121345446404"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.2.3.1.1"><p id="mrs_01_24781__p17134184474014">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.3.2.2.3.1.2"><p id="mrs_01_24781__p4134174415408">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24781__row20134114424016"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24781__p813474424017">table_name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24781__p1513417441406">Mandatory. Name of the Hudi table from which invalid data files are to be deleted.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24781__row1613417445409"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24781__p313410443408">op_type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24781__p1753218644316">Optional. Command running mode. The default value is <strong id="mrs_01_24781__b14643163111411">dry_run</strong>. Value options are <strong id="mrs_01_24781__b022575441413">dry_run</strong>, <strong id="mrs_01_24781__b13292856151410">repair</strong>, <strong id="mrs_01_24781__b14155859141412">undo</strong>, and <strong id="mrs_01_24781__b178491054152">query</strong>.</p>
|
|
<p id="mrs_01_24781__p1038722844410"><strong id="mrs_01_24781__b38961910191515">dry_run</strong>: displays invalid data files to be cleaned.</p>
|
|
<p id="mrs_01_24781__p1831695815446"><strong id="mrs_01_24781__b10799152731518">repair</strong>: displays and cleans invalid data files.</p>
|
|
<p id="mrs_01_24781__p1142121817455"><strong id="mrs_01_24781__b20389183961517">undo</strong>: restores deleted data files.</p>
|
|
<p id="mrs_01_24781__p1637119342455"><strong id="mrs_01_24781__b1956364631518">query</strong>: displays the backup directories that have been cleaned.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24781__row1513418445408"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24781__p1813454413400">backup_path</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24781__p113464414016">Mandatory. Backup directory of the data files to be restored. This parameter is available only when the running mode is <strong id="mrs_01_24781__b1230695021816">undo</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24781__row31341844184010"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24781__p1313411441406">start_time</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24781__p19134144124017">Optional. Start time for generating invalid data files. This parameter is available only when the running mode is <strong id="mrs_01_24781__b1071117219229">dry_run</strong> or <strong id="mrs_01_24781__b841319513228">repair</strong>. The start time is not limited by default.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24781__row19134544144014"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_24781__p17134154411405">end_time</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_24781__p101341644164015">Optional. End time for generating invalid data files. This parameter is available only when the running mode is <strong id="mrs_01_24781__b1614873182218">dry_run</strong> or <strong id="mrs_01_24781__b171611031182213">repair</strong>. The end time is not limited by default.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_24781__section16795334255"><h4 class="sectiontitle">Example</h4><pre class="screen" id="mrs_01_24781__screen17680191075716">call clean_file(table => 'h1', mode=>'repair');
|
|
call clean_file(table => 'h1', mode=>'dry_run');
|
|
call clean_file(table => 'h1', mode=>'query');
|
|
call clean_file(table => 'h1', mode=>'undo', backup_path=>'/tmp/hudi/h1/.hoodie/.cleanbackup/hoodie_repair_backup_20220222222222');</pre>
|
|
</div>
|
|
<div class="section" id="mrs_01_24781__section1142451142514"><h4 class="sectiontitle">Precautions</h4><p id="mrs_01_24781__p1791516173253">The command cleans only invalid Parquet files.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_24781__section7720121912517"><h4 class="sectiontitle">System Response</h4><p id="mrs_01_24781__p4696926125913">You can view command execution results in the driver log or on the client.</p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24739.html">CALL COMMAND</a></div>
|
|
</div>
|
|
</div>
|
|
|