forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
57 lines
6.9 KiB
HTML
57 lines
6.9 KiB
HTML
<a name="mrs_01_0961"></a><a name="mrs_01_0961"></a>
|
|
|
|
<h1 class="topictitle1">Using Hive to Read Data in a Relational Database</h1>
|
|
<div id="body1590395283792"><div class="section" id="mrs_01_0961__s15376514ec274083a3330c6fe0a2cd50"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0961__aa84249dad734482b801d5a8a7fb31a60">Hive allows users to create external tables to associate with other relational databases. External tables read data from associated relational databases and support Join operations with other tables in Hive.</p>
|
|
<div class="p" id="mrs_01_0961__a065ac46b2fac45d08e6f31fb58be9c19">Currently, the following relational databases can use Hive to read data:<ul id="mrs_01_0961__u54bb0b513ce34e3fbd61b6014e63ba0f"><li id="mrs_01_0961__l5a1d8a6084d844b4a26d35f1388f4e48">DB2</li><li id="mrs_01_0961__l5f670b8d98e3443a8fd07d3896d2a88e">Oracle</li></ul>
|
|
<div class="note" id="mrs_01_0961__note835285022717"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_0961__p12363135052715">This section applies to MRS 3.<em id="mrs_01_0961__i3369193620582">x</em> or later clusters.</p>
|
|
</div></div>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_0961__s61a69bc86614401ba9baec328ef0bfad"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_0961__af505fa83304d469e804c221b79452faf">The Hive client has been installed.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0961__s0fcad6112a844cc3bc2816f7e2af5ec3"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_0961__o4cd7a9d555884c499c78107c13d7fc55"><li id="mrs_01_0961__li795618393387"><span>Log in to the node where the Hive client is installed as the Hive client installation user .</span></li><li id="mrs_01_0961__l4175d420fd7945bda95bcb396d089f03"><span>Run the following command to go to the client installation directory:</span><p><p id="mrs_01_0961__a50b43b3cfd774498b4a110c7bd98b7a9"><strong id="mrs_01_0961__b12695352513">cd </strong><em id="mrs_01_0961__i1832553132511">Client installation directory</em></p>
|
|
<p id="mrs_01_0961__a7c9e5857e32b4e36972568454525e0cf">For example, if the client installation directory is <strong id="mrs_01_0961__b1250318451746">/opt</strong><strong id="mrs_01_0961__b650317455411"></strong><strong id="mrs_01_0961__b13503204512412">/client</strong>, run the following command:</p>
|
|
<p id="mrs_01_0961__ada23c984bea548ee94fa405676074150"><strong id="mrs_01_0961__a7b39cd1a697f4f7696f866a3c50143dc">cd /opt/client</strong></p>
|
|
</p></li><li id="mrs_01_0961__l48d0f7d873cf4d5e944ef1463ba124e8"><span>Run the following command to configure environment variables:</span><p><p id="mrs_01_0961__a19e2f356a910457c82494c8780f533b5"><strong id="mrs_01_0961__aa8f1392c848f461a924c4097852609de">source bigdata_env</strong></p>
|
|
</p></li><li id="mrs_01_0961__lee840f4d696c4bbea71444ec60cff4e8"><span>Check whether the cluster authentication mode is Security.</span><p><ul id="mrs_01_0961__ufe677221a3ac490492680756275ebc89"><li id="mrs_01_0961__l24ba98b8ca2b4fdcb2763529ccd9c350">If yes, run the following command to authenticate the user:<p id="mrs_01_0961__acf97ef8eab8c4d739336fc0f82c0c6e9"><a name="mrs_01_0961__l24ba98b8ca2b4fdcb2763529ccd9c350"></a><a name="l24ba98b8ca2b4fdcb2763529ccd9c350"></a><strong id="mrs_01_0961__b3950165782616">kinit</strong> <em id="mrs_01_0961__i11955175772619">Hive service user</em> </p>
|
|
</li><li id="mrs_01_0961__l612296ab7b964236874aaac640a32bb3">If no, go to <a href="#mrs_01_0961__lad1ae23de6bb407cb136beb10cc94ad2">5</a>.</li></ul>
|
|
</p></li><li id="mrs_01_0961__lad1ae23de6bb407cb136beb10cc94ad2"><a name="mrs_01_0961__lad1ae23de6bb407cb136beb10cc94ad2"></a><a name="lad1ae23de6bb407cb136beb10cc94ad2"></a><span>Run the following command to upload the driver JAR package of the relational database to be associated to an HDFS directory.</span><p><p id="mrs_01_0961__a6a76b678c7e64f0d8002aca187162c6f"><strong id="mrs_01_0961__a486376c61b904a3d970676f6832c0915">hdfs dfs -put </strong><i><span class="varname" id="mrs_01_0961__v168766128e0b471fb19f4a220859acde">directory where the JAR package is located</span></i> <i><span class="varname" id="mrs_01_0961__v4b32aecef9f2480a841ee4329851f8bf">HDFS directory to which the JAR is uploaded</span></i></p>
|
|
<p id="mrs_01_0961__acf06717eed6e4d179f3aa2277e399ba4">For example, to upload the Oracle driver JAR package in <span class="filepath" id="mrs_01_0961__f0315083761224f4bb3a4b8790f69736e"><b>/opt</b></span> to the <span class="filepath" id="mrs_01_0961__fe218425d1c8b4c5f892d25cc60090bd5"><b>/tmp</b></span> directory in HDFS, run the following command:</p>
|
|
<p class="litext" id="mrs_01_0961__a2f715cd7560f411abee63a721e3ca954"><strong id="mrs_01_0961__ac7415f18ec184046ad9a4154593df3ee">hdfs dfs -put /opt/ojdbc6.jar /tmp</strong></p>
|
|
</p></li><li id="mrs_01_0961__lc46432fcae354e5d842697ddbbf7f4ca"><span>Create an external table on the Hive client to associate with the relational database, as shown in the following example.</span><p><div class="note" id="mrs_01_0961__na8342ef9c6394c70837bbc3895170dd3"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><div class="p" id="mrs_01_0961__a7e26f87a193d4470b13eeab836003e79">If the security mode is used, the user who creates the table must have the <strong id="mrs_01_0961__b975351519295">ADMIN</strong> permission. The <strong id="mrs_01_0961__b17781322152911">ADD JAR</strong> path is subject to the actual path.<pre class="screen" id="mrs_01_0961__sd9564e4bf96f45d5b6e5ac48304dab1a">-- Example of associating with an Oracle Linux 6 database
|
|
-- In security mode, set the <strong id="mrs_01_0961__b084510093011">admin</strong> permission.
|
|
set role admin;
|
|
-- Upload the driver JAR package of the relational database to be associated. The driver JAR packages vary according to databases.
|
|
ADD JAR hdfs:///tmp/ojdbc6.jar;
|
|
|
|
CREATE EXTERNAL TABLE ora_test
|
|
-- The Hive table must have one more column than the database return result. This column is used for paging query.
|
|
(id STRING,rownum string)
|
|
STORED BY 'com.qubitproducts.hive.storage.jdbc.JdbcStorageHandler'
|
|
TBLPROPERTIES (
|
|
-- Relational database table type
|
|
"qubit.sql.database.type" = "ORACLE",
|
|
-- Connect to the URL of the relational database through JDBC. (The URL formats vary according to databases.)
|
|
"qubit.sql.jdbc.url" = "jdbc:oracle:thin:@//10.163.0.1:1521/mydb",
|
|
-- Relational database driver class type
|
|
"qubit.sql.jdbc.driver" = "oracle.jdbc.OracleDriver",
|
|
-- SQL statement queried in the relational database. The result is returned to the Hive table.
|
|
"qubit.sql.query" = "select name from aaa",
|
|
-- (Optional) Match the Hive table columns to the relational database table columns.
|
|
"qubit.sql.column.mapping" = "id=name",
|
|
-- Relational database user
|
|
"qubit.sql.dbcp.username" = "test",
|
|
-- Relational database password
|
|
"qubit.sql.dbcp.password" = "<em id="mrs_01_0961__i92773504156">xxx</em>");</pre>
|
|
</div>
|
|
</div></div>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0581.html">Using Hive</a></div>
|
|
</div>
|
|
</div>
|
|
|