vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide/mrs_01_0372.html
Yang, Tong 6182f91ba8 MRS component operation guide_normal 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2022-12-09 14:55:21 +00:00

66 lines
22 KiB
HTML
Raw Blame History

<a name="mrs_01_0372"></a><a name="mrs_01_0372"></a>
<h1 class="topictitle1">Using the Metadata Browser on the Hue Web UI</h1>
<div id="body1589421629062"><div class="section" id="mrs_01_0372__s005f823b53fb448aa190d542f069040f"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0372__p113531582010">Users can use the Hue web UI to manage Hive metadata in an MRS cluster.</p>
<p id="mrs_01_0372__p18497646204210">For versions earlier than MRS 1.9.2, MRS clusters with Kerberos authentication enabled support this function.</p>
</div>
<div class="section" id="mrs_01_0372__sd71c303869ac441398c55ef7b5ccf200"><h4 class="sectiontitle">Using Metastore Manager</h4><p id="mrs_01_0372__p1295134953817">Access the Hue web UI. For details, see <a href="mrs_01_0370.html">Accessing the Hue Web UI</a>.</p>
<p id="mrs_01_0372__p18607140143212">Choose <strong id="mrs_01_0372__b12639141193718">Data Browsers</strong> &gt; <strong id="mrs_01_0372__b113684493710">Metastore Tables</strong>, and access <span class="wintitle" id="mrs_01_0372__wintitle752610547221"><b>Metastore Manager</b></span>.</p>
<ul id="mrs_01_0372__u6e49bf15e77b46d8b6e6280902d75119"><li id="mrs_01_0372__le04aa16f50d044ad9529de5c94b3d2b0">Viewing metadata of Hive tables<p id="mrs_01_0372__a64364b5dcfed448983d871e881eeb634"><a name="mrs_01_0372__le04aa16f50d044ad9529de5c94b3d2b0"></a><a name="le04aa16f50d044ad9529de5c94b3d2b0"></a>In the left navigation pane, move the cursor to a table and click <span><img id="mrs_01_0372__i9b4027dac080433c8a596cb7f830ccf4" src="en-us_image_0000001295930292.png"></span> on the right. The metadata of the Hive table is displayed.</p>
</li><li id="mrs_01_0372__l382b3c11a3354a369f9e9292d0d45d00">Managing metadata of Hive tables<p id="mrs_01_0372__ae897276f6167445c8850d8d0272dab93"><a name="mrs_01_0372__l382b3c11a3354a369f9e9292d0d45d00"></a><a name="l382b3c11a3354a369f9e9292d0d45d00"></a>On the metadata page of a Hive table, you can click <span><img id="mrs_01_0372__i39da232baccf437488ed894a68ad767a" src="en-us_image_0000001295770328.png"></span> in the upper right corner to import data, click <span><img id="mrs_01_0372__i177bfa39398e43a7bd7f70b9de4cf142" src="en-us_image_0000001349169853.png"></span> to browse data, and click <span><img id="mrs_01_0372__i37959c57bcaa454ab19fd4281f1bd79f" src="en-us_image_0000001349289433.png"></span> to view the location of the table file.</p>
<div class="caution" id="mrs_01_0372__note2228538205219"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="mrs_01_0372__p02781117181714">The Hue page is used to view and analyze data such as files and tables. Do not perform high-risk management operations such as deleting objects on the page. If an operation is required, you are advised to perform the operation on each component after confirming that the operation has no impact on services. For example, you can use the HDFS client to perform operations on HDFS files and use the Hive client to perform operations on Hive tables.</p>
</div></div>
</li><li id="mrs_01_0372__lace0c3ea8a674b31a306d25ee0a5dac6">Managing Hive metadata tables<p id="mrs_01_0372__a9670c1752c67416fa36f4ae1ef20bfcf"><a name="mrs_01_0372__lace0c3ea8a674b31a306d25ee0a5dac6"></a><a name="lace0c3ea8a674b31a306d25ee0a5dac6"></a>Click <span><img id="mrs_01_0372__image685114610205" src="en-us_image_0000001349169861.png"></span> in the upper right corner to create a table in the database based on the uploaded files. Or click <span><img id="mrs_01_0372__image08529469208" src="en-us_image_0000001295770332.png"></span> in the upper right corner to manually create a table.</p>
</li></ul>
</div>
<div class="section" id="mrs_01_0372__section2565199014569"><h4 class="sectiontitle">Accessing Metastore Manager</h4><ol id="mrs_01_0372__ol1603633814582"><li id="mrs_01_0372__li4744865614582"><span>Access the Hue web UI. For details, see <a href="mrs_01_0370.html">Accessing the Hue Web UI</a>.</span></li><li id="mrs_01_0372__li5972161614582"><span>Choose <strong id="mrs_01_0372__b740412113388">Data Browsers</strong> &gt; <strong id="mrs_01_0372__b640501153813">Metastore Tables</strong>, and access <span class="wintitle" id="mrs_01_0372__wintitle5405181113816"><b>Metastore Manager</b></span>.</span><p><p id="mrs_01_0372__p6140156714588"><span class="wintitle" id="mrs_01_0372__wintitle62051717182215"><b>Metastore Manager</b></span> supports the following functions:</p>
<ul id="mrs_01_0372__ul1578427145813"><li id="mrs_01_0372__li11633199145813">Creating a Hive table from a file</li><li id="mrs_01_0372__li22644589145813">Manually creating a Hive table</li><li id="mrs_01_0372__li63357319145813">Viewing Hive table metadata</li></ul>
</p></li></ol>
</div>
<div class="section" id="mrs_01_0372__section46959017145822"><h4 class="sectiontitle">Creating a Hive table from a File</h4><ol id="mrs_01_0372__ol33343268145839"><li id="mrs_01_0372__li52913031145839"><span>Access <span class="wintitle" id="mrs_01_0372__wintitle159661118152315"><b>Metastore Manager</b></span> and select a database in <span class="parmname" id="mrs_01_0372__parmname43161927145839"><b>Databases</b></span>.</span><p><p id="mrs_01_0372__p11415356145857">The default database is <span class="parmvalue" id="mrs_01_0372__parmvalue6455235145839"><b>default</b></span>.</p>
</p></li><li id="mrs_01_0372__li27507478145839"><span>Click <span><img id="mrs_01_0372__image1451314581701" src="en-us_image_0000001349289429.jpg"></span>. The <span class="wintitle" id="mrs_01_0372__wintitle11110151502515"><b>Create a new table from a file</b></span> page is displayed.</span></li><li id="mrs_01_0372__li6175625145839"><span>Select a file.</span><p><ol type="a" id="mrs_01_0372__ol30512342145839"><li id="mrs_01_0372__li54510264145839">In <span class="parmname" id="mrs_01_0372__parmname7427191122615"><b>Table Name</b></span>, enter a Hive table name.<p id="mrs_01_0372__p13513236145839">A Hive table name contains no more than 128 characters, including letters, numbers, or underscores (_), and must start with a letter or number. </p>
</li><li id="mrs_01_0372__li53255288145839">In <span class="parmname" id="mrs_01_0372__parmname12323104020265"><b>Description</b></span>, enter description about the Hive table as required.</li><li id="mrs_01_0372__li21343403145839">In <span class="parmname" id="mrs_01_0372__parmname1080657202713"><b>Input File or Location</b></span>, click <span><img id="mrs_01_0372__image32631549102213" src="en-us_image_0000001349169857.jpg"></span> and select a Hive table file from HDFS. The file is used to store new data of the Hive table.<p id="mrs_01_0372__p17284570145839">If the file is not stored in HDFS, click <span class="uicontrol" id="mrs_01_0372__uicontrol7360910283"><b>Upload a file</b></span> to upload the file from the local directory to HDFS. Multiple files can be simultaneously uploaded. The files cannot be empty.</p>
</li><li id="mrs_01_0372__li19465682145839">If you need to import the data in the file to the Hive table, select <span class="parmvalue" id="mrs_01_0372__parmvalue93751232122912"><b>Import data</b></span> as <span class="parmname" id="mrs_01_0372__parmname7308953132912"><b>Load method</b></span>. By default, <span class="parmvalue" id="mrs_01_0372__parmvalue358311243019"><b>Import data</b></span> is selected.<p id="mrs_01_0372__p231823711663">If you select <span class="parmvalue" id="mrs_01_0372__parmvalue18261114710304"><b>Create External Table</b></span>, a Hive external table is created.</p>
<div class="note" id="mrs_01_0372__note3201372716710"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0372__p1968809116710">If you select <span class="parmvalue" id="mrs_01_0372__parmvalue18438458173015"><b>Create External Table</b></span>, set <span class="parmname" id="mrs_01_0372__parmname1339262363119"><b>Input File or Location</b></span> to a path.</p>
</div></div>
<p id="mrs_01_0372__p2162853145839">If you select <span class="parmvalue" id="mrs_01_0372__parmvalue2054611497316"><b>Leave Empty</b></span>, an empty Hive table is created.</p>
</li><li id="mrs_01_0372__li33216422145839">Click <span class="uicontrol" id="mrs_01_0372__uicontrol712255133314"><b>Next</b></span>.</li></ol>
</p></li><li id="mrs_01_0372__li10043372145839"><span>Set a delimiter.</span><p><ol type="a" id="mrs_01_0372__ol16029011145839"><li id="mrs_01_0372__li51636899145839">In <span class="uicontrol" id="mrs_01_0372__uicontrol4430012123512"><b>Delimiter</b></span>, select one.<p id="mrs_01_0372__p5737433145839">If your desired delimiter is not in the list, select <span class="parmvalue" id="mrs_01_0372__parmvalue30463654145839"><b>Other..</b></span> and enter a delimiter.</p>
</li><li id="mrs_01_0372__li21839272145839">Click <span class="uicontrol" id="mrs_01_0372__uicontrol1043916653619"><b>Preview</b></span> to preview data processing.</li><li id="mrs_01_0372__li24150622145839">Click <span class="uicontrol" id="mrs_01_0372__uicontrol29681915113617"><b>Next</b></span>.</li></ol>
</p></li><li id="mrs_01_0372__li49479839145839"><span>Define a column. </span><p><ol type="a" id="mrs_01_0372__ol27867381145839"><li id="mrs_01_0372__li6752731145839">If you click <span><img id="mrs_01_0372__image623111309264" src="en-us_image_0000001296249764.jpg"></span> on the right side of <span class="uicontrol" id="mrs_01_0372__uicontrol67185834110"><b>Use first row as column names</b></span>, the first row of data in the file is used as a column name. If you do not click it, the first row of data is not used as the column name.</li><li id="mrs_01_0372__li16240092145839">In <span class="uicontrol" id="mrs_01_0372__uicontrol15390104718433"><b>Column name</b></span>, set a name for each column.<p id="mrs_01_0372__p10100301145839">A Hive table name contains no more than 128 characters, including letters, numbers, or underscores (_), and must start with a letter or number. </p>
<div class="note" id="mrs_01_0372__note31630616145839"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0372__p48253755145839">You can rename columns in batches by clicking <span><img id="mrs_01_0372__image196873201426" src="en-us_image_0000001348770157.jpg"></span> on the right side of <span class="uicontrol" id="mrs_01_0372__uicontrol1356919519468"><b>Bulk edit column names</b></span>. Enter all column names and separate them by commas (,).</p>
</div></div>
</li><li id="mrs_01_0372__li40379077145839">In <span class="uicontrol" id="mrs_01_0372__uicontrol1025343319475"><b>Column Type</b></span>, select a type for each column. </li></ol>
</p></li><li id="mrs_01_0372__li48444050145839"><span>Click <span class="uicontrol" id="mrs_01_0372__uicontrol19133142664819"><b>Create Table</b></span> to create the table. Wait for Hue to display information about the Hive table.</span></li></ol>
</div>
<div class="section" id="mrs_01_0372__section33909292145956"><h4 class="sectiontitle">Manually Creating a Hive Table</h4><ol id="mrs_01_0372__ol19533881509"><li id="mrs_01_0372__li502345971509"><span>Access <span class="wintitle" id="mrs_01_0372__wintitle2016392919589"><b>Metastore Manager</b></span> and select a database in <span class="uicontrol" id="mrs_01_0372__uicontrol351012605918"><b>Databases</b></span>.</span><p><p id="mrs_01_0372__p557745615034">The default database is <span class="parmvalue" id="mrs_01_0372__parmvalue494581971509"><b>default</b></span>.</p>
</p></li><li id="mrs_01_0372__li636706081509"><span>Click <span><img id="mrs_01_0372__image1301871313" src="en-us_image_0000001295930296.jpg"></span>. The <span class="uicontrol" id="mrs_01_0372__uicontrol323816501818"><b>Create a new table manually</b></span> page is displayed.</span></li><li id="mrs_01_0372__li42359601509"><span>Set a table name.</span><p><ol type="a" id="mrs_01_0372__ol377533641509"><li id="mrs_01_0372__li436488471509">In <span class="parmname" id="mrs_01_0372__parmname1124042119513"><b>Table Name</b></span>, enter a Hive table name.<p id="mrs_01_0372__p570456551509">A Hive table name contains no more than 128 characters, including letters, numbers, or underscores (_), and must start with a letter or number. </p>
</li><li id="mrs_01_0372__li458957031509">In <span class="parmname" id="mrs_01_0372__parmname544817489519"><b>Description</b></span>, enter description about the Hive table as required. </li><li id="mrs_01_0372__li265644391509">Click <span class="uicontrol" id="mrs_01_0372__uicontrol16812391268"><b>Next</b></span>.</li></ol>
</p></li><li id="mrs_01_0372__li318382411509"><span>Select a data storage format.</span><p><ul id="mrs_01_0372__ul333637441509"><li id="mrs_01_0372__li10074731509">If data needs to be separated by delimiters, select <span class="parmvalue" id="mrs_01_0372__parmvalue45318131572"><b>Delimited</b></span> and perform <a href="#mrs_01_0372__li379246041509">5</a>.</li><li id="mrs_01_0372__li633594061509">If data needs to be stored in serialization format, select <span class="parmvalue" id="mrs_01_0372__parmvalue90672591509"><b>SerDe</b></span> and perform <a href="#mrs_01_0372__li119336551509">6</a>.</li></ul>
</p></li><li id="mrs_01_0372__li379246041509"><a name="mrs_01_0372__li379246041509"></a><a name="li379246041509"></a><span>Set a delimiter.</span><p><ol type="a" id="mrs_01_0372__ol414965471509"><li id="mrs_01_0372__li479153681509">In <span class="parmname" id="mrs_01_0372__parmname456620389716"><b>Field terminator</b></span>, set a column delimiter. <p id="mrs_01_0372__p575197121509">If your desired delimiter is not in the list, select <span class="parmvalue" id="mrs_01_0372__parmvalue287607001509"><b>Other..</b></span> and enter a delimiter.</p>
</li><li id="mrs_01_0372__li118601201509">In <span class="parmname" id="mrs_01_0372__parmname139417213816"><b>Collection terminator</b></span>, set a delimiter to separate the data set of columns of the <span class="parmvalue" id="mrs_01_0372__parmvalue559395961509"><b>array</b></span> type in Hive. For example, the type of a column is array. A value needs to store <span class="parmvalue" id="mrs_01_0372__parmvalue336943231509"><b>employee</b></span> and <span class="parmvalue" id="mrs_01_0372__parmvalue348134571509"><b>manager</b></span>. The user specifies a colon (<span class="parmvalue" id="mrs_01_0372__parmvalue448856631509"><b>:</b></span>) as the delimiter. Therefore, the final value is <span class="parmvalue" id="mrs_01_0372__parmvalue13177911509"><b>employee:manager</b></span>.</li><li id="mrs_01_0372__li520214651509">In <span class="parmname" id="mrs_01_0372__parmname668871414916"><b>Map key terminator</b></span>, set a delimiter to separate the data set of columns of the <span class="parmvalue" id="mrs_01_0372__parmvalue211456691509"><b>map</b></span> type in Hive. For example, the type of a column is map. A value needs to store <span class="parmvalue" id="mrs_01_0372__en-us_topic_0050240812_parmvalue80812455174651"><b>home</b></span> of <span class="parmvalue" id="mrs_01_0372__en-us_topic_0050240812_parmvalue41145818717476"><b>aaa</b></span> and <span class="parmvalue" id="mrs_01_0372__en-us_topic_0050240812_parmvalue1006837477174712"><b>company</b></span> of <span class="parmvalue" id="mrs_01_0372__en-us_topic_0050240812_parmvalue903832812174727"><b>bbb</b></span>. The user defines <span class="parmvalue" id="mrs_01_0372__en-us_topic_0050240812_parmvalue19069717017484"><b>|</b></span> as the delimiter. Therefore, the final value is <span class="parmvalue" id="mrs_01_0372__parmvalue281497841509"><b>home|aaa:company|bbb</b></span>.</li><li id="mrs_01_0372__li642630511509">Click <span class="uicontrol" id="mrs_01_0372__uicontrol46761245993"><b>Next</b></span> and perform <a href="#mrs_01_0372__li564987691509">7</a>.</li></ol>
</p></li><li id="mrs_01_0372__li119336551509"><a name="mrs_01_0372__li119336551509"></a><a name="li119336551509"></a><span>Set serialization properties.</span><p><ol type="a" id="mrs_01_0372__ol162390421509"><li id="mrs_01_0372__li507707761509">In <span class="parmname" id="mrs_01_0372__parmname2051203212104"><b>SerDe Name</b></span>, enter the class name of the serialization format: <span class="parmvalue" id="mrs_01_0372__parmvalue519940781509"><b>org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe</b></span><p id="mrs_01_0372__p652935201509">Users can expand Hive to support more customized serialization classes.</p>
</li><li id="mrs_01_0372__li349121761509">In <span class="parmname" id="mrs_01_0372__parmname542838061509"><b>Serde properties</b></span>, enter the value of the serialization format: <span class="parmvalue" id="mrs_01_0372__parmvalue187922111509"><b>"field.delim"="," "colelction.delim"=":" "mapkey.delim"="|"</b></span></li><li id="mrs_01_0372__li167174181509">Click <span class="uicontrol" id="mrs_01_0372__uicontrol560394891010"><b>Next</b></span> and perform <a href="#mrs_01_0372__li564987691509">7</a>.</li></ol>
</p></li><li id="mrs_01_0372__li564987691509"><a name="mrs_01_0372__li564987691509"></a><a name="li564987691509"></a><span>Select a data table format and click <span class="uicontrol" id="mrs_01_0372__uicontrol402940341509"><b>Next</b></span>.</span><p><ul id="mrs_01_0372__ul659299641509"><li id="mrs_01_0372__li425913021509"><span class="parmname" id="mrs_01_0372__parmname271019881509"><b>TextFile</b></span>: indicates that data is stored in text files.</li><li id="mrs_01_0372__li273434311509"><span class="parmname" id="mrs_01_0372__parmname477774011509"><b>SequenceFile</b></span>: indicates that data is stored in binary files.</li><li id="mrs_01_0372__li446082531509"><span class="parmname" id="mrs_01_0372__parmname447642871509"><b>InputFormat</b></span>: indicates that data in files is used in the customized input and output formats.<div class="p" id="mrs_01_0372__p646087961509">Users can expand Hive to support more customized formatting classes.<ol type="a" id="mrs_01_0372__ol668310781509"><li id="mrs_01_0372__li182577411509">In <span class="parmname" id="mrs_01_0372__parmname2254041509"><b>InputFormat Class</b></span>, enter the class used by input data: <span class="parmvalue" id="mrs_01_0372__parmvalue20286371509"><b>org.apache.hadoop.hive.ql.io.RCFileInputFormat</b></span></li><li id="mrs_01_0372__li223387561509">In <span class="parmname" id="mrs_01_0372__parmname301019481509"><b>OutputFormat Class</b></span>, enter the class used by output data: <span class="parmvalue" id="mrs_01_0372__parmvalue24820841509"><b>org.apache.hadoop.hive.ql.io.RCFileOutputFormat</b></span></li></ol>
</div>
</li></ul>
</p></li><li id="mrs_01_0372__li129975771509"><span>Select a file storage location and click <span class="uicontrol" id="mrs_01_0372__uicontrol19315123219128"><b>Next</b></span>.</span><p><p id="mrs_01_0372__p5178341415057"><span class="parmvalue" id="mrs_01_0372__parmvalue2076618152135"><b>Use default location</b></span> is selected by default. If you want to customize a storage location, deselect the default value and specify a file storage location in <span class="parmname" id="mrs_01_0372__parmname461708301509"><b>External location</b></span> by clicking <span><img id="mrs_01_0372__image10371155216148" src="en-us_image_0000001349289425.jpg"></span>. </p>
</p></li><li id="mrs_01_0372__li173092171509"><span>Set columns of the Hive table.</span><p><ol type="a" id="mrs_01_0372__ol466624891509"><li id="mrs_01_0372__li369944351509">In <span class="parmname" id="mrs_01_0372__parmname11158132811154"><b>Column name</b></span>, set a column name.<p id="mrs_01_0372__p488497351509">A Hive table name contains no more than 128 characters, including letters, numbers, or underscores (_), and must start with a letter or number. </p>
</li><li id="mrs_01_0372__li548373611509">In <span class="parmname" id="mrs_01_0372__parmname74162503151"><b>Column type</b></span>, select a type for each column.<p id="mrs_01_0372__p582888231509">Click <span class="uicontrol" id="mrs_01_0372__uicontrol78554814160"><b>Add a column</b></span> to add a new column. </p>
</li><li id="mrs_01_0372__li126412611509">Click <span class="uicontrol" id="mrs_01_0372__uicontrol982892681616"><b>Add a partition</b></span> to add a new partition for the Hive table to improve the query efficiency.</li></ol>
</p></li><li id="mrs_01_0372__li598693661509"><span>Click <span class="uicontrol" id="mrs_01_0372__uicontrol1659884611611"><b>Create Table</b></span> to create a new table. Wait for Hue to display information about the Hive table.</span></li></ol>
</div>
<div class="section" id="mrs_01_0372__section1010060115237"><h4 class="sectiontitle">Managing the Hive Table</h4><ol id="mrs_01_0372__ol3900545415244"><li id="mrs_01_0372__li5498443515244"><span>Access <span class="wintitle" id="mrs_01_0372__wintitle976891181716"><b>Metastore Manager</b></span> and select a database in <span class="uicontrol" id="mrs_01_0372__uicontrol12396123421716"><b>Databases</b></span>. All tables in the database are displayed on the page.</span><p><p id="mrs_01_0372__p2719071315247">The default database is <span class="parmvalue" id="mrs_01_0372__parmvalue2509786815244"><b>default</b></span>.</p>
</p></li><li id="mrs_01_0372__li4907318215244"><span>Click a table name in the database to view table details.</span><p><p id="mrs_01_0372__p2455422615244">The following operations are supported: importing data, browsing data,, or viewing file storage location. When viewing all tables in the database, you can select tables and perform the following operations such as viewing tables and browsing data.</p>
<div class="caution" id="mrs_01_0372__note4273527815244"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="mrs_01_0372__en-us_topic_0264266662_p02781117181714">The Hue page is used to view and analyze data such as files and tables. Do not perform high-risk management operations such as deleting objects on the page. If an operation is required, you are advised to perform the operation on each component after confirming that the operation has no impact on services. For example, you can use the HDFS client to perform operations on HDFS files and use the Hive client to perform operations on Hive tables.</p>
</div></div>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0369.html">Using Hue (Versions Earlier Than MRS 3.x)</a></div>
</div>
</div>
View Git Blame Copy Permalink