forked from docs/doc-exports
Su, Xiaomeng
76a5b1ee83
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com> Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
51 lines
5.5 KiB
HTML
51 lines
5.5 KiB
HTML
<a name="dli_08_0359"></a><a name="dli_08_0359"></a>
|
|
|
|
<h1 class="topictitle1">Updating Table Metadata with REFRESH TABLE</h1>
|
|
<div id="body0000001223686511"><div class="section" id="dli_08_0359__en-us_topic_0114776173_en-us_topic_0093946988_s961c878200f14a358c5aea6c61a2e797"><h4 class="sectiontitle">Function</h4><div class="p" id="dli_08_0359__p77407559254">Spark caches Parquet metadata to improve performance. If you update a Parquet table, the cached metadata is not updated. Spark SQL cannot find the newly inserted data and an error similar with the following is reported:<pre class="screen" id="dli_08_0359__screen1434180192616">DLI.0002: FileNotFoundException: getFileStatus on error message</pre>
|
|
</div>
|
|
<p id="dli_08_0359__en-us_topic_0114776173_en-us_topic_0093946988_a16897d215347400b9b4674708e9d6bdc">You can use REFRESH TABLE to solve this problem. REFRESH TABLE reorganizes files of a partition and reuses the original table metadata information to detect the increase or decrease of table fields. This statement is mainly used when the metadata in a table is not modified but the table data is modified.</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0359__en-us_topic_0114776173_en-us_topic_0093946988_s9997a4a51ee54377b5391ca21cf697ae"><h4 class="sectiontitle">Syntax</h4><div class="codecoloring" codetype="Sql" id="dli_08_0359__en-us_topic_0114776173_screen1648333311472"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="n">REFRESH</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="p">[</span><span class="n">db_name</span><span class="p">.]</span><span class="k">table_name</span><span class="p">;</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0359__en-us_topic_0114776175_en-us_topic_0093946864_s0c845eb1e41148d79029d4f419b4fc4f"><h4 class="sectiontitle">Keywords</h4><p id="dli_08_0359__p16891125119211">None</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0359__en-us_topic_0114776188_en-us_topic_0093946905_s95436b2cebbc4c34b0e6ca8427d4f1d9"><h4 class="sectiontitle">Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_08_0359__en-us_topic_0114776188_en-us_topic_0103084433_t2fa8597e0dde4884bd6412ac207ebea8" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dli_08_0359__en-us_topic_0114776188_en-us_topic_0103084433_r16c75fd8fdbf41c79f9aee82b032351c"><th align="left" class="cellrowborder" valign="top" width="18.759999999999998%" id="mcps1.3.4.2.2.3.1.1"><p id="dli_08_0359__en-us_topic_0114776188_en-us_topic_0103084433_a9dd69379a3f54904a9c510fb817659ad">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="81.24%" id="mcps1.3.4.2.2.3.1.2"><p id="dli_08_0359__en-us_topic_0114776188_en-us_topic_0103084433_a7ede0d62539542baa7e367cf902bf2b2">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dli_08_0359__row9456161015224"><td class="cellrowborder" valign="top" width="18.759999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0359__en-us_topic_0114776177_en-us_topic_0103084434_a1cdeda4790a8452d8619c797147bf9cf">db_name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="81.24%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0359__en-us_topic_0114776177_en-us_topic_0103084434_p4729195274115">Database name that contains letters, digits, and underscores (_). It cannot contain only digits or start with an underscore (_).</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dli_08_0359__row160712191411"><td class="cellrowborder" valign="top" width="18.759999999999998%" headers="mcps1.3.4.2.2.3.1.1 "><p id="dli_08_0359__p3608320149">table_name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="81.24%" headers="mcps1.3.4.2.2.3.1.2 "><p id="dli_08_0359__p160812219141">Table name of a database that contains letters, digits, and underscores (_). It cannot contain only digits or start with an underscore (_). The matching rule is <strong id="dli_08_0359__b11143822084280">^(?!_)(?![0-9]+$)[A-Za-z0-9_$]*$</strong>. If special characters are required, use single quotation marks ('') to enclose them.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dli_08_0359__en-us_topic_0114776173_en-us_topic_0093946988_sa26295793f714ee3b039534eb041370c"><h4 class="sectiontitle">Precautions</h4><p id="dli_08_0359__p12091449202313">None</p>
|
|
</div>
|
|
<div class="section" id="dli_08_0359__en-us_topic_0114776177_en-us_topic_0093946793_sbd247a258fdd490a988c980dd33504e6"><h4 class="sectiontitle">Example</h4><p id="dli_08_0359__en-us_topic_0114776177_en-us_topic_0093946793_a07d135d6e32641898620af93b0a267c2">Update metadata of the <strong id="dli_08_0359__b1532842918181">test</strong> table.</p>
|
|
<div class="codecoloring" codetype="Sql" id="dli_08_0359__en-us_topic_0114776177_screen361074812321"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="n">REFRESH</span><span class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span class="n">test</span><span class="p">;</span>
|
|
</pre></div></td></tr></table></div>
|
|
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_08_0080.html">Syntax for Partitioning a Table</a></div>
|
|
</div>
|
|
</div>
|
|
|