forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
33 lines
7.2 KiB
HTML
33 lines
7.2 KiB
HTML
<a name="mrs_01_24118"></a><a name="mrs_01_24118"></a>
|
|
|
|
<h1 class="topictitle1">Hive Supporting Cold and Hot Storage of Partitioned Metadata</h1>
|
|
<div id="body32001227"><div class="section" id="mrs_01_24118__en-us_topic_0000001219230515_section6813124153710"><h4 class="sectiontitle">Cold and Hot Storage of Partitioned Metadata</h4><ul id="mrs_01_24118__en-us_topic_0000001219230515_ul19741731143815"><li id="mrs_01_24118__en-us_topic_0000001219230515_li1374123173810">The metadata that have not been used for a long time is moved to a backup table to reduce the pressure on metadata databases. This process is called partitioned data freezing. The partitions in which data is moved are cold partitions, partitions that are not frozen are hot partitions. A table with a cold partition is a frozen table. Moving the frozen data back to the original metadata table is called partitioned data unfreezing.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li1974183115383">When a partition is changed from a hot partition to a cold partition, only metadata is identified. The partition path and data file content on the HDFS service side do not change.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_24118__en-us_topic_0000001219230515_section328552019556"><h4 class="sectiontitle">Freezing a Partition</h4><p id="mrs_01_24118__en-us_topic_0000001219230515_p20262841131010">The user who creates the table can freeze one or more partitions based on filter criteria. The format is <strong id="mrs_01_24118__en-us_topic_0000001219230515_b3925113354519">freeze partitions</strong> <em id="mrs_01_24118__en-us_topic_0000001219230515_i192523315458">Database name Table name</em> <strong id="mrs_01_24118__en-us_topic_0000001219230515_b8925633174516">where</strong> <em id="mrs_01_24118__en-us_topic_0000001219230515_i69260338458">Filter criteria</em>.</p>
|
|
<p id="mrs_01_24118__en-us_topic_0000001219230515_p669490528">Example:</p>
|
|
<pre class="screen" id="mrs_01_24118__en-us_topic_0000001219230515_screen41391919536">freeze partitions testdb.test where year <= 2021;
|
|
freeze partitions testdb.test where year<=2021 and month <= 5;
|
|
freeze partitions testdb.test where year<=2021 and month <= 5 and day <= 27;</pre>
|
|
</div>
|
|
<div class="section" id="mrs_01_24118__en-us_topic_0000001219230515_section0151834205616"><h4 class="sectiontitle">Unfreezing a Partition</h4><p id="mrs_01_24118__en-us_topic_0000001219230515_p240384475713">The user who creates the table can unfreeze one or more partitions based on filter criteria. The format is <strong id="mrs_01_24118__en-us_topic_0000001219230515_b81441940183617">unfreeze partitions</strong> <em id="mrs_01_24118__en-us_topic_0000001219230515_i181462040103616">Database name Table name</em> <strong id="mrs_01_24118__en-us_topic_0000001219230515_b1114884063615">where</strong> <em id="mrs_01_24118__en-us_topic_0000001219230515_i18150134010367">Filter criteria</em>. Example:</p>
|
|
<pre class="screen" id="mrs_01_24118__en-us_topic_0000001219230515_screen1450662695314">unfreeze partitions testdb.test where year <= 2021;
|
|
unfreeze partitions testdb.test where year<=2021 and month <= 5;
|
|
unfreeze partitions testdb.test where year<=2021 and month <= 5 and day <= 27;</pre>
|
|
</div>
|
|
<div class="section" id="mrs_01_24118__en-us_topic_0000001219230515_section328311563586"><h4 class="sectiontitle">Querying Tables with Frozen Data</h4><ul id="mrs_01_24118__en-us_topic_0000001219230515_ul14454193115917"><li id="mrs_01_24118__en-us_topic_0000001219230515_li145483125910">Querying all frozen tables in the current database<p id="mrs_01_24118__en-us_topic_0000001219230515_p6622195845313"><a name="mrs_01_24118__en-us_topic_0000001219230515_li145483125910"></a><a name="en-us_topic_0000001219230515_li145483125910"></a><strong id="mrs_01_24118__en-us_topic_0000001219230515_b8128175785316">show frozen tables</strong>;</p>
|
|
</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li1245473125916">Querying all frozen tables in the <strong id="mrs_01_24118__en-us_topic_0000001219230515_b168179315186">dbname</strong> database<p id="mrs_01_24118__en-us_topic_0000001219230515_p340931819549"><strong id="mrs_01_24118__en-us_topic_0000001219230515_b1983816125410">show frozen tables in dbname</strong>;</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_24118__en-us_topic_0000001219230515_section1718153615599"><h4 class="sectiontitle">Querying Frozen Partitions of a Frozen Table</h4><p id="mrs_01_24118__en-us_topic_0000001219230515_p7931171211112">Querying frozen partitions</p>
|
|
<p id="mrs_01_24118__en-us_topic_0000001219230515_p151811546575"><strong id="mrs_01_24118__en-us_topic_0000001219230515_b0924101411588">show frozen partitions table</strong>;</p>
|
|
<div class="note" id="mrs_01_24118__en-us_topic_0000001219230515_note18781131904"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_24118__en-us_topic_0000001219230515_ul54191115123310"><li id="mrs_01_24118__en-us_topic_0000001219230515_li7419201511333">By default, only partitions of the int, string, varchar, date, or timestamp type can be frozen in the metadata database.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li24741345204015">For external metadata databases, only the Postgres database is supported, and only partitions of the int, string, varchar, or timestamp type can be frozen.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li15991102711016">You need to unfreeze data to restore the metadata of a frozen table using MSCK. If a frozen table has been backed up, you can run <strong id="mrs_01_24118__en-us_topic_0000001219230515_b96454422250">msck repair</strong> to restore the table, and you can only run this command to unfreeze the table.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li05631358144616">You need to unfreeze data before renaming a frozen partition. Otherwise, a message indicating that the partition does not exist is displayed.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li109918271013">When a table that contains frozen data is deleted, the frozen data is also deleted.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li1899162720011">When a partition that contains frozen data is deleted, information about the frozen partition and HDFS service data are not deleted.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li1599112279016">When you run the <strong id="mrs_01_24118__en-us_topic_0000001219230515_b7451816113018">select</strong> command to query data, the criteria for filtering the data in cold partitions is automatically added. The query result does not contain the data in cold partitions.</li><li id="mrs_01_24118__en-us_topic_0000001219230515_li1799116271003">When you run the <strong id="mrs_01_24118__en-us_topic_0000001219230515_b2957331153516">show partitions table</strong> command to query the partitioned data in the table, the query result does not contain the data in cold partitions. You can run the <strong id="mrs_01_24118__en-us_topic_0000001219230515_b7511171123710">show frozen partitions table</strong> command to query frozen partitions.</li></ul>
|
|
</div></div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0581.html">Using Hive</a></div>
|
|
</div>
|
|
</div>
|
|
|