vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide-lts/mrs_01_24040.html
Yang, Tong 3f5759eed2 MRS comp-lts 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2023-01-19 17:08:45 +00:00

86 lines
23 KiB
HTML
Raw Blame History

<a name="mrs_01_24040"></a><a name="mrs_01_24040"></a>
<h1 class="topictitle1">Hive Supporting Reading Hudi Tables</h1>
<div id="body8662426"><div class="section" id="mrs_01_24040__en-us_topic_0000001173631436_section97683581468"><h4 class="sectiontitle">Hive External Tables Corresponding to Hudi Tables</h4><p id="mrs_01_24040__en-us_topic_0000001173631436_p797116188720">A Hudi source table corresponds to a copy of HDFS data. The Hudi table data can be mapped to a Hive external table through the Spark component, Flink component, or Hudi client. Based on the external table, Hive can easily perform real-time query, read-optimized view query, and incremental view query.</p>
<div class="note" id="mrs_01_24040__en-us_topic_0000001173631436_note13633037577"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_24040__en-us_topic_0000001173631436_ul55821712814"><li id="mrs_01_24040__en-us_topic_0000001173631436_li95822117819">Different view queries are provided for different types of Hudi source tables:<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul4319522283"><li id="mrs_01_24040__en-us_topic_0000001173631436_li2052412181588">When the Hudi source table is a Copy-On-Write (COW) table, it can be mapped to a Hive external table. The table supports real-time query and incremental view query.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li1552516181986">When the Hudi source table is a Merge-On-Read (MOR) table, it can be mapped to two Hive external tables (RO table and RT table). The RO table supports read-optimized view query, and the RT table supports real-time view query and incremental view query.</li></ul>
</li></ul>
<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul31831145473"><li id="mrs_01_24040__en-us_topic_0000001173631436_li7183345272">Hive external tables cannot be added, deleted, or modified (including <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1365259316">insert</strong>, <strong id="mrs_01_24040__en-us_topic_0000001173631436_b17141227230">update</strong>, <strong id="mrs_01_24040__en-us_topic_0000001173631436_b93113295315">delete</strong>, <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1113132638">load</strong>, <strong id="mrs_01_24040__en-us_topic_0000001173631436_b114051338317">merge</strong>, <strong id="mrs_01_24040__en-us_topic_0000001173631436_b199212377316">alter</strong> and <strong id="mrs_01_24040__en-us_topic_0000001173631436_b10379153818319">msck</strong>). Only the query operation (<strong id="mrs_01_24040__en-us_topic_0000001173631436_b297310481631">select</strong>) is supported.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li4947113119561">Granting table permissions: The update, alter, write, and all permissions cannot be modified.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li965492515017">Backup and restoration: The RO and RT tables are mapped from the same Hudi source table. When one table is backed up, the other table is also backed up. The same applies to restoration. Therefore, only one table needs to be backed up.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li34481223165815">Hudi bootstrap tables: The storage mode of bootstrap tables is different from that of common Hudi tables. Therefore, you need to configure <strong id="mrs_01_24040__en-us_topic_0000001173631436_b183722912211">hive.user.aux.jars.path</strong> for the Tez query engine. Log in to FusionInsight Manager, choose <strong id="mrs_01_24040__en-us_topic_0000001173631436_b18655102319301">Cluster</strong> &gt; <strong id="mrs_01_24040__en-us_topic_0000001173631436_b3798112853015">Services</strong> &gt; <strong id="mrs_01_24040__en-us_topic_0000001173631436_b14711931133015">Hive</strong> &gt; <strong id="mrs_01_24040__en-us_topic_0000001173631436_b172551358193016">Configurations</strong> &gt; <strong id="mrs_01_24040__en-us_topic_0000001173631436_b17901182173118">All Configurations</strong>, and add the following paths to the value of <strong id="mrs_01_24040__en-us_topic_0000001173631436_b817442010319">hive.user.aux.jars.path</strong> (use commas (,) as separators):<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul387955075812"><li id="mrs_01_24040__en-us_topic_0000001173631436_li6370709592">${BIGDATA_HOME}/FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text5834327516">8.1.2.2</span>/install/FusionInsight-Hive-3.1.0/hive-3.1.0/lib/hbase-shaded-miscellaneous-xxx.jar</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li126189375920">${BIGDATA_HOME}/FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text32859331817">8.1.2.2</span>/install/FusionInsight-Hive-3.1.0/hive-3.1.0/lib/hbase-metrics-api-xxx.jar</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li443865105916">${BIGDATA_HOME}/FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text52910362018">8.1.2.2</span>/install/FusionInsight-Hive-3.1.0/hive-3.1.0/lib/hbase-metrics-xxx.jar</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li14836071593">${BIGDATA_HOME}/FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text4608238612">8.1.2.2</span>/install/FusionInsight-Hive-3.1.0/hive-3.1.0/lib/hbase-protocol-shaded-xxx.jar</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li1398013915592">${BIGDATA_HOME}/FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text33714411515">8.1.2.2</span>/install/FusionInsight-Hive-3.1.0/hive-3.1.0/lib/hbase-shaded-protobuf-xxx.jar</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li616111313598">${BIGDATA_HOME}/FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text18638144319111">8.1.2.2</span>/install/FusionInsight-Hive-3.1.0/hive-3.1.0/lib/htrace-core4-4.2.0-incubatin-xxx.jar</li></ul>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p157101634175517">Replace <em id="mrs_01_24040__en-us_topic_0000001173631436_i20587103672819">xxx</em> with the actual version in the path of the HiveServer node. For example, if the name of <strong id="mrs_01_24040__en-us_topic_0000001173631436_b71688464296">hbase-shaded-miscellaneous-</strong><em id="mrs_01_24040__en-us_topic_0000001173631436_i696715210299">xxx</em><strong id="mrs_01_24040__en-us_topic_0000001173631436_b1389419493290">.jar</strong> in the actual environment is <strong id="mrs_01_24040__en-us_topic_0000001173631436_b16438113122914">hbase-shaded-miscellaneous-3.3.0.jar</strong>, replace <em id="mrs_01_24040__en-us_topic_0000001173631436_i18788123642910">xxx</em> with <strong id="mrs_01_24040__en-us_topic_0000001173631436_b92902452315">3.3.0</strong>.</p>
</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li132766817518">Component versions:<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul1342092012514"><li id="mrs_01_24040__en-us_topic_0000001173631436_li412315131515">Hive: FusionInsight_HD_<span id="mrs_01_24040__en-us_topic_0000001173631436_text1683784519120">8.1.2.2</span>; Hive kernel version 3.1.0.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li13183184516711">Spark2x: FusionInsight_Spark2x_<span id="mrs_01_24040__en-us_topic_0000001173631436_text187949471017">8.1.2.2</span>; Hudi kernel version: 0.9.0.</li></ul>
</li></ul>
</div></div>
</div>
<div class="section" id="mrs_01_24040__en-us_topic_0000001173631436_section432613121191"><h4 class="sectiontitle">Creating Hive External Tables Corresponding to Hudi Tables</h4><p id="mrs_01_24040__en-us_topic_0000001173631436_p15715203215919">Generally, Hudi table data is synchronized to Hive external tables when the data is imported to the lake. In this case, you can directly query the corresponding Hive external tables in Beeline. If the data is not synchronized to the Hive external tables, you can use the Hudi client tool <strong id="mrs_01_24040__en-us_topic_0000001173631436_b960411394177">run_hive_sync_tool.sh</strong> to synchronize data manually.</p>
</div>
<div class="section" id="mrs_01_24040__en-us_topic_0000001173631436_section175403382913"><h4 class="sectiontitle">Querying Hive External Tables Corresponding to Hudi Tables</h4><p id="mrs_01_24040__en-us_topic_0000001173631436_p112671611101"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b9126161614106">Prerequisites</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p182162791019">Set <strong id="mrs_01_24040__en-us_topic_0000001173631436_b81779383164">hive.input.format</strong> before using Hive to query a Hudi table.</p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p11214271104">In addition, you need to set another three parameters for incremental query. The three parameters are table-level parameters. Each Hudi source table corresponds to three parameters, where <strong id="mrs_01_24040__en-us_topic_0000001173631436_b9832019151616">hudisourcetablename</strong> indicates the name of the Hudi source table (not the name of the Hive external table).</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24040__en-us_topic_0000001173631436_table115710119115" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_24040__en-us_topic_0000001173631436_row14588101116"><th align="left" class="cellrowborder" valign="top" width="33.33333333333333%" id="mcps1.3.3.5.2.4.1.1"><p id="mrs_01_24040__en-us_topic_0000001173631436_p205812101112">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="8.61086108610861%" id="mcps1.3.3.5.2.4.1.2"><p id="mrs_01_24040__en-us_topic_0000001173631436_p125815110110">Default Value</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="58.055805580558065%" id="mcps1.3.3.5.2.4.1.3"><p id="mrs_01_24040__en-us_topic_0000001173631436_p85816119113">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_24040__en-us_topic_0000001173631436_row105817110119"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.5.2.4.1.1 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p10581912110">hoodie. hudisourcetablename.consume.mode</p>
</td>
<td class="cellrowborder" valign="top" width="8.61086108610861%" headers="mcps1.3.3.5.2.4.1.2 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p1758121121112">None</p>
</td>
<td class="cellrowborder" valign="top" width="58.055805580558065%" headers="mcps1.3.3.5.2.4.1.3 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p18692741210">Query mode of the Hudi table.</p>
<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul640632111127"><li id="mrs_01_24040__en-us_topic_0000001173631436_li54066218122">Incremental query: Set it to <strong id="mrs_01_24040__en-us_topic_0000001173631436_b17589103561812">INCREMENTAL</strong>.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li8407621121211">Non-incremental query: Do not set this parameter or set it to <strong id="mrs_01_24040__en-us_topic_0000001173631436_b137935431813">SNAPSHOT</strong>.</li></ul>
</td>
</tr>
<tr id="mrs_01_24040__en-us_topic_0000001173631436_row1858201151117"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.5.2.4.1.1 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p25819120118">hoodie. hudisourcetablename.consume.start.timestamp</p>
</td>
<td class="cellrowborder" valign="top" width="8.61086108610861%" headers="mcps1.3.3.5.2.4.1.2 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p13581112116">None</p>
</td>
<td class="cellrowborder" valign="top" width="58.055805580558065%" headers="mcps1.3.3.5.2.4.1.3 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p19513272122">Start time of the incremental query on the Hudi table.</p>
<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul5652134712126"><li id="mrs_01_24040__en-us_topic_0000001173631436_li765224721218">Incremental query: start time of the incremental query.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li1865214478123">Non-incremental query: Do not set this parameter.</li></ul>
</td>
</tr>
<tr id="mrs_01_24040__en-us_topic_0000001173631436_row1858171171116"><td class="cellrowborder" valign="top" width="33.33333333333333%" headers="mcps1.3.3.5.2.4.1.1 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p105815111119">hoodie. hudisourcetablename.consume.max.commits</p>
</td>
<td class="cellrowborder" valign="top" width="8.61086108610861%" headers="mcps1.3.3.5.2.4.1.2 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p9583114118">None</p>
</td>
<td class="cellrowborder" valign="top" width="58.055805580558065%" headers="mcps1.3.3.5.2.4.1.3 "><p id="mrs_01_24040__en-us_topic_0000001173631436_p9746436181211">The incremental query on the Hudi table is based on the number of commits after <strong id="mrs_01_24040__en-us_topic_0000001173631436_b17829420811">hoodie.hudisourcetablename.consume.start.timestamp</strong>.</p>
<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul796114548121"><li id="mrs_01_24040__en-us_topic_0000001173631436_li1496113544129">Incremental query: number of commits. For example, if this parameter is set to <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1913975911304">3</strong>, data after three commits from the specified start time is queried. If this parameter is set to <strong id="mrs_01_24040__en-us_topic_0000001173631436_b8693114853414">-1</strong>, all data committed after the specified start time is queried.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li59629542123">Non-incremental query: Do not set this parameter.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p178491725111419"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b5871165117495">Querying a Hudi COW Table</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p194121537111416">For example, the name of a Hudi source table of the COW type is <strong id="mrs_01_24040__en-us_topic_0000001173631436_b711102216384">hudicow</strong>, and the name of the mapped Hive external table is <strong id="mrs_01_24040__en-us_topic_0000001173631436_b4711154175014">hudicow</strong>.</p>
<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul56371316191715"><li id="mrs_01_24040__en-us_topic_0000001173631436_li9637201610174">Real-time view query on the COW table: After <strong id="mrs_01_24040__en-us_topic_0000001173631436_b5301162011426">hive.input.format</strong> is set, query the COW table as a common Hive table.<p id="mrs_01_24040__en-us_topic_0000001173631436_p811672912448"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b81711533114414">set hive.input.format= org.apache.hadoop.hive.ql.io.HiveInputFormat;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p113681339171616"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b2355843101618">Select * from hudicow;</strong></p>
</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li13751824161711">Incremental query on the COW table: In addition to <strong id="mrs_01_24040__en-us_topic_0000001173631436_b6433636185016">hive.input.format</strong>, you need to set three incremental query parameters based on the name of the Hudi source table. The <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1243393613504">where</strong> clause of the incremental query statements must contain <strong id="mrs_01_24040__en-us_topic_0000001173631436_b4434536125019">`_hoodie_commit_time`&gt;'</strong><em id="mrs_01_24040__en-us_topic_0000001173631436_i16434336115011">xxx</em><strong id="mrs_01_24040__en-us_topic_0000001173631436_b5434203612504">'</strong>, where <em id="mrs_01_24040__en-us_topic_0000001173631436_i15434153613508">xxx</em> indicates the value of <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1943518363507">hoodie.hudisourcetablename.consume.start.timestamp</strong>.<p id="mrs_01_24040__en-us_topic_0000001173631436_p7499939184516"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b7695544184516">set hive.input.format= org.apache.hadoop.hive.ql.io.HiveInputFormat;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p2096914531918"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b127224134198">set hoodie.</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b2969175171917">hudicow</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b472211132197">.consume.mode= INCREMENTAL;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p1896910517192"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b0724111391910">set hoodie.</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b159691515194">hudicow</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b3724101313195">.consume.max.commits=3;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p13969205181911"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b3726131310193">set hoodie.</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b119695561917">hudicow</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b272611139198">.consume.start.timestamp= 20200427114546;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p69692516195"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b272751341913">select count(*) from hudicow where `_hoodie_commit_time`&gt;'20200427114546';</strong></p>
</li></ul>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p61313681913"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b8625569509">Querying a Hudi MOR Table</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p116671628202">For example, the name of a Hudi source table of the MOR type is <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1178934112535">hudimor</strong>, and the two mapped Hive external tables are <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1242814491547">hudimor_ro</strong> (RO table) and <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1854225795418">hudimor_rt</strong> (RT table).</p>
<ul id="mrs_01_24040__en-us_topic_0000001173631436_ul55181316172216"><li id="mrs_01_24040__en-us_topic_0000001173631436_li145187167225">Read-optimized view query on the RO table: After <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1385310225557">hive.input.format</strong> is set, query the COW table as a common Hive table.<p id="mrs_01_24040__en-us_topic_0000001173631436_p1249305211473"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b38671555164715">set hive.input.format= org.apache.hadoop.hive.ql.io.HiveInputFormat;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p329582512115"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b18351921182218">Select * from hudicow_ro;</strong></p>
</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li0514525152210">Real-time view query on the RT table: After <strong id="mrs_01_24040__en-us_topic_0000001173631436_b11870101579">hive.input.format</strong> is set, the latest data of the Hudi source table can be queried.<p id="mrs_01_24040__en-us_topic_0000001173631436_p1450853710489"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b711175611233">set hive.input.format= org.apache.hadoop.hive.ql.io.HiveInputFormat;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p2017017437218"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b2051213367228">Select * from hudicow_rt;</strong></p>
</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li764312426222">Incremental query on the RT table: In addition to <strong id="mrs_01_24040__en-us_topic_0000001173631436_b87461480210">hive.input.format</strong>, you need to set three incremental query parameters based on the name of the Hudi source table. The <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1874716481221">where</strong> clause of the incremental query statements must contain <strong id="mrs_01_24040__en-us_topic_0000001173631436_b3747348424">`_hoodie_commit_time`&gt;'</strong><em id="mrs_01_24040__en-us_topic_0000001173631436_i147476481922">xxx</em><strong id="mrs_01_24040__en-us_topic_0000001173631436_b1747174814218">'</strong>, where <em id="mrs_01_24040__en-us_topic_0000001173631436_i9747548921">xxx</em> indicates the value of <strong id="mrs_01_24040__en-us_topic_0000001173631436_b17485481326">hoodie.hudisourcetablename.consume.start.timestamp</strong>.<p id="mrs_01_24040__en-us_topic_0000001173631436_p12758101320229"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b63141621112319">set hive.input.format=org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p1758121352210"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b6316221132315">set hoodie.</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b7758213122211">hudimor</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b113163218239">.consume.mode=INCREMENTAL;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p10758113142219"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b53192217237">set hoodie.</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b1975812138227">hudimor</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b18319152182318">.consume.max.commits=-1;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p10758131317224"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b143212021132313">set hoodie.</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b075813135226">hudimor</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b8321172111231">.consume.start.timestamp=20210207144611;</strong></p>
<p id="mrs_01_24040__en-us_topic_0000001173631436_p57581813132213"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b11323521182317">select * from hudimor_rt where `_hoodie_commit_time`&gt;'20210207144611';</strong></p>
<div class="note" id="mrs_01_24040__en-us_topic_0000001173631436_note92451527906"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_24040__en-us_topic_0000001173631436_ul576219265559"><li id="mrs_01_24040__en-us_topic_0000001173631436_li67621266558"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b10262842142918">set hive.input.format=org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat;</strong> is used only for the incremental query on the RT table and cannot be used for other tables. Therefore, after the incremental query on the RT table is complete, run <strong id="mrs_01_24040__en-us_topic_0000001173631436_b1262174272914">set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;</strong>. Alternatively, run <strong id="mrs_01_24040__en-us_topic_0000001173631436_b226414213292">set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;</strong> to change the value to the default value for the queries on other tables.</li><li id="mrs_01_24040__en-us_topic_0000001173631436_li14762192675515"><strong id="mrs_01_24040__en-us_topic_0000001173631436_b155201049202914">set hoodie.hudisourcetablename.consume.mode=INCREMENTAL</strong><strong id="mrs_01_24040__en-us_topic_0000001173631436_b352664912294">;</strong> is used only for the incremental query on the table. To switch to another query mode, run <strong id="mrs_01_24040__en-us_topic_0000001173631436_b16526149122919">set hoodie.hudisourcetablename.consume.mode=SNAPSHOT;</strong>.</li></ul>
</div></div>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0581.html">Using Hive</a></div>
</div>
</div>
View Git Blame Copy Permalink