forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
381 lines
46 KiB
HTML
381 lines
46 KiB
HTML
<a name="mrs_01_1164"></a><a name="mrs_01_1164"></a>
|
|
|
|
<h1 class="topictitle1">Example for Using the Open-Source sqoop-shell Tool (Oracle-HBase)</h1>
|
|
<div id="body8662426"><div class="section" id="mrs_01_1164__en-us_topic_0000001173631484_sa1ae0a4725e347738b9f2b7ffb7551fa"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1164__en-us_topic_0000001173631484_aceafe602bd8e4f719d6bd23e552b37d1">Taking <strong id="mrs_01_1164__en-us_topic_0000001173631484_b30241942391946">Importing Data from Oracle to HBase</strong> as an example, this section introduces how to use the sqoop-shell tool to create and start Loader jobs in the interaction mode and batch mode.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1164__en-us_topic_0000001173631484_se03b1436971648b8a53f605367c21961"><h4 class="sectiontitle">Prerequisites</h4><p id="mrs_01_1164__en-us_topic_0000001173631484_a0978460ec8454ee1a8b46c17ab729500">The Loader client has been installed and configured.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1164__en-us_topic_0000001173631484_section17594095597"><h4 class="sectiontitle">Example for the Interaction Mode</h4><ol id="mrs_01_1164__en-us_topic_0000001173631484_ol20864112014234"><li id="mrs_01_1164__en-us_topic_0000001173631484_li68541720182313"><span>Log in to the node where the Loader client is installed as the user who installs the client.</span></li><li id="mrs_01_1164__en-us_topic_0000001173631484_li985472010235"><span>Run the following command to go to the <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_filepath1376014183517"><b>conf</b></span> directory of the sqoop-shell tool. For example, if the Loader client installation directory is <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_filepath14199124433514"><b>/opt/hadoopclient/Loader</b></span>, run the following command:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_p3854120152313"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b168541720162318">cd /opt/hadoopclient/Loader/loader-tools-1.99.3/sqoop-shell/conf</strong></p>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_li2863152022312"><span>Run the following command to configure authentication information:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_p1185592014238"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b1085592019234">vi client.properties</strong></p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_screen1685610202230">server.url=<em id="mrs_01_1164__en-us_topic_0000001173631484_i38555200234">10.0.0.1:21351</em>
|
|
# simple or kerberos
|
|
authentication.type=simple
|
|
# true or false
|
|
use.keytab=true
|
|
|
|
authentication.user=
|
|
authentication.password=
|
|
|
|
client.principal=oracle/hadoop<em id="mrs_01_1164__en-us_topic_0000001173631484_i385522015232">@<system domain name></em>
|
|
|
|
# keytab file
|
|
client.keytab.file=./conf/login/oracle.keytab</pre>
|
|
<div class="note" id="mrs_01_1164__en-us_topic_0000001173631484_note48581520102317"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1164__en-us_topic_0000001173631484_p3555163415514">Log in to FusionInsight Manager and choose <strong id="mrs_01_1164__en-us_topic_0000001173631484_b203532481759">System</strong> > <strong id="mrs_01_1164__en-us_topic_0000001173631484_b10354154812514">Permission</strong> > <strong id="mrs_01_1164__en-us_topic_0000001173631484_b335454817518">Domain and Mutual Trust</strong>. The value of <strong id="mrs_01_1164__en-us_topic_0000001173631484_b1935464819510">Local Domain</strong> is the current system domain name.</p>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1164__en-us_topic_0000001173631484_table158635204232" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Configuration parameters</caption><thead align="left"><tr id="mrs_01_1164__en-us_topic_0000001173631484_row16858162019230"><th align="left" class="cellrowborder" valign="top" width="31.39%" id="mcps1.3.3.2.3.2.4.2.4.1.1"><p id="mrs_01_1164__en-us_topic_0000001173631484_p38589205237">Configuration parameters</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="44.55%" id="mcps1.3.3.2.3.2.4.2.4.1.2"><p id="mrs_01_1164__en-us_topic_0000001173631484_p8858192015232">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="24.060000000000002%" id="mcps1.3.3.2.3.2.4.2.4.1.3"><p id="mrs_01_1164__en-us_topic_0000001173631484_p785832014234">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1164__en-us_topic_0000001173631484_row12859172032316"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p2085812018230">server.url</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p5858122012320">Floating IP address and port (21351) for Loader.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1859820102312">For compatibility, multiple IP addresses and ports can be configured and need to be separated by <span class="parmvalue" id="mrs_01_1164__en-us_topic_0000001173631484_parmvalue38581420172310"><b>commas (,)</b></span>. The first IP address and port must be those of Loader (21351). The others can be configured based on service requirements.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p1985920209235">10.0.0.1:21351</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_row986012207236"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p2859112013234">authentication.type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p08599202233">Login authentication mode.</p>
|
|
<ul id="mrs_01_1164__en-us_topic_0000001173631484_ul98591520132315"><li id="mrs_01_1164__en-us_topic_0000001173631484_li78594205238"><span class="parmvalue" id="mrs_01_1164__en-us_topic_0000001173631484_parmvalue1785932014235"><b>kerberos</b></span> indicates that the security mode is used and Kerberos authentication is performed. Kerberos authentication provides two authentication modes: the password mode and the keytab file mode.</li><li id="mrs_01_1164__en-us_topic_0000001173631484_li17859102019235"><span class="parmvalue" id="mrs_01_1164__en-us_topic_0000001173631484_parmvalue7859172010234"><b>simple</b></span> indicates that the normal mode is used and Kerberos authentication is not performed.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p20859152002314">kerberos</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_row2860152042316"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p98601820122310">authentication.user</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p1586002042318">User for login when the normal mode or password authentication is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p78601120162315">In the keytab login mode, this parameter does not need to be set.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p38607201237">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_row20862132013236"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p10860162016232">authentication.password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p086052016230">User password for login when the password authentication mode is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p38603209236">In the normal mode or keytab login mode, this parameter does not need to be set.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1860920102313">The password needs to be encrypted. The encryption method is described as follows:</p>
|
|
<ol type="a" id="mrs_01_1164__en-us_topic_0000001173631484_ol108621120172311"><li id="mrs_01_1164__en-us_topic_0000001173631484_li886118201230">Go to the directory where <span class="parmname" id="mrs_01_1164__en-us_topic_0000001173631484_parmname1486032032314"><b>encrypt_tool</b></span> is located. For example, if the Loader client installation directory is <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_filepath1186042016231"><b>/opt/hadoopclient/Loader</b></span>, run the following command:<p id="mrs_01_1164__en-us_topic_0000001173631484_p158601020172319"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b1886012202231">cd /opt/hadoopclient/Loader/loader-tools-1.99.3</strong></p>
|
|
</li><li id="mrs_01_1164__en-us_topic_0000001173631484_li178619208234">Run the following command to encrypt the non-encrypted password:<p id="mrs_01_1164__en-us_topic_0000001173631484_p1861820122316"><a name="mrs_01_1164__en-us_topic_0000001173631484_li178619208234"></a><a name="en-us_topic_0000001173631484_li178619208234"></a><strong id="mrs_01_1164__en-us_topic_0000001173631484_b9671165671312">./encrypt_tool </strong><em id="mrs_01_1164__en-us_topic_0000001173631484_i267605616135">Unencrypted password</em></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p5861182082317">The obtained encrypted password is used as the value of <span class="parmname" id="mrs_01_1164__en-us_topic_0000001173631484_parmname1486122010238"><b>authentication.password</b></span>.</p>
|
|
<div class="note" id="mrs_01_1164__en-us_topic_0000001173631484_note16861720202317"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1164__en-us_topic_0000001173631484_p198616209235">If a non-encrypted password contains special characters, the special characters must be escaped. For example, the dollar sign ($) is a special character and can be escaped using single quotation marks ('), for example, <strong id="mrs_01_1164__en-us_topic_0000001173631484_b1193812534319">'1q2w#e$r'</strong>. If a non-encrypted password contains single quotation marks, use double quotation marks to escape the single quotation marks. If a non-encrypted password contains double quotation marks, use backslashes (\) to escape the double quotation marks. For details, see the shell escape character rules.</p>
|
|
</div></div>
|
|
</li></ol>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p6862152082320">43B80E33A96DF3D203ABBDFD1050C041</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_row19862122062318"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p148623204235">use.keytab</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p198621320152312">Whether to use the keytab mode to log in.</p>
|
|
<ul id="mrs_01_1164__en-us_topic_0000001173631484_ul17862120152318"><li id="mrs_01_1164__en-us_topic_0000001173631484_li186218209233"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b191381023791946">true</strong> indicates using the keytab file to log in.</li><li id="mrs_01_1164__en-us_topic_0000001173631484_li15862152010236"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b158479021091946">false</strong> indicates using the password to log in.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p386272002316">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_row1086214209235"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p78626208236">client.principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p11862112010235">User principal for accessing the Loader service when the keytab authentication mode is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p28621320142317">In the normal mode or password login mode, this parameter does not need to be set.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p286292015233">loader/hadoop</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_row286362019239"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.3.2.3.2.4.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p6862142082315">client.keytab.file</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.3.2.3.2.4.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p18862182082317">Directory where the used keytab file is located when the keytab authentication mode is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p6862202012230">In the normal mode or password login mode, this parameter does not need to be set.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.3.2.3.2.4.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_p1863142022312">/opt/client/conf/loader.keytab</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_li128642020202319"><span>Run the following command to go to the interaction mode:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_p6863182014231"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b286322022313">source /opt/hadoopclient/bigdata_env</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1686312201237"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b9863142013239">cd /opt/hadoopclient/Loader/loader-tools-1.99.3/sqoop-shell</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1686332072320"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b98633206231">./sqoop2-shell</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p158631206234">The preceding commands obtain authentication information by reading the configuration file.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p2863122013234">Alternatively, you can also use the password or Kerberos authentication.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1786392010232">Run the following command to authenticate login using the password mode:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p9863112010231"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b0863520122314">./sqoop2-shell -uk false -u username -p encryptedPassword</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1586315204232">Run the following command to authenticate login using the Kerberos mode:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p16863182010234"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b7863152092310">./sqoop2-shell -uk true -k user.keytab -s userPrincipal</strong></p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_screen3864132020234">Welcome to sqoop client
|
|
Use the username and password authentication mode
|
|
Authentication success.
|
|
Sqoop Shell: Type 'help' or '\h' for help.
|
|
|
|
sqoop:000> </pre>
|
|
</p></li></ol><ol start="5" id="mrs_01_1164__en-us_topic_0000001173631484_ol16853192012234"><li id="mrs_01_1164__en-us_topic_0000001173631484_li11853820152319"><span>Run the following command to view the corresponding ID of the current connector:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_p1285302019235"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b68531820142312">show connector</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p178531020132311">The following information is displayed:</p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173789890_sef03db952be54fd7855ffd5b4209b2eb">+----+----------------------------+----------------+----------------------------------------------------------------------+
|
|
| Id | Name | Version | Class |
|
|
+----+----------------------------+----------------+----------------------------------------------------------------------+
|
|
| 1 | generic-jdbc-connector | XXX | org.apache.sqoop.connector.jdbc.GenericJdbcConnector |
|
|
| 2 | ftp-connector | XXX | org.apache.sqoop.connector.ftp.FtpConnector |
|
|
| 3 | hdfs-connector | XXX | org.apache.sqoop.connector.hdfs.HdfsConnector |
|
|
| 4 | oracle-connector | XXX | org.apache.sqoop.connector.oracle.OracleConnector |
|
|
| 5 | mysql-fastpath-connector | XXX | org.apache.sqoop.connector.mysql.MySqlConnector |
|
|
| 6 | sftp-connector | XXX | org.apache.sqoop.connector.sftp.SftpConnector |
|
|
| 7 | oracle-partition-connector | XXX | org.apache.sqoop.connector.oracle.partition.OraclePartitionConnector |
|
|
+----+----------------------------+----------------+----------------------------------------------------------------------+</pre>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a7f4afe2e03bb43118c2ece29c7048b7a">The preceding information indicates that the Oracle connector ID is 4.</p>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_l46393bc70c6c4102a11e0c098fcbd2a0"><span>Run the following command to create connectors and enter the specific connector information as prompted:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_a510048cff86845d4b79912f946a77614"><strong id="mrs_01_1164__en-us_topic_0000001173631484_afea020c6c4224ec5a5e1956237648115">create connection -c </strong><i><span class="varname" id="mrs_01_1164__en-us_topic_0000001173631484_v34521b5db4ae4a9fb8360f82ad973545">connector ID</span></i></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a451eb655414a480d9cc0aad69e08411b">For example, if the connector ID is 4, run the following command:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a50790e78a9344970a0092d18ff57d30d"><strong id="mrs_01_1164__en-us_topic_0000001173631484_aa864a861f01946eb92d1571c29ae9424">create connection -c 4</strong></p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_screen191726612615">sqoop:000> create connection -c 4
|
|
Creating connection for connector with id 4
|
|
Please fill following values to create new connection object
|
|
Name: oracle14
|
|
|
|
Oracle connection configuration
|
|
|
|
JDBC connection string: jdbc:oracle:thin:@189.120.84.106:1521:orcl
|
|
Username: oracledba
|
|
Password: **********
|
|
JDBC connection properties:
|
|
There are currently 0 values in the map:
|
|
entry#
|
|
New connection was successfully created with validation status FINE and persistent id 3
|
|
sqoop:000></pre>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p848744913566">The preceding information indicates that the connection ID is 3.</p>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_l8a09a2d7ba4842f9accaea1b38288ff3"><span>Based on the connection ID, run the following command to create jobs:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_adedafafbf4f14390b09116a2512cfe69"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b91993183391946">create job -x </strong><i><span class="varname" id="mrs_01_1164__en-us_topic_0000001173631484_varname150441608591946">connection ID</span></i><strong id="mrs_01_1164__en-us_topic_0000001173631484_b5750513691946"> -t impor</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b182419765991946">t</strong> <strong id="mrs_01_1164__en-us_topic_0000001173631484_b192396363891946">--trans </strong><em id="mrs_01_1164__en-us_topic_0000001173631484_i200548679991946">absolute path of job-config</em><strong id="mrs_01_1164__en-us_topic_0000001173631484_b52267871591946">/oracle-hbase.json</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a2ff3ca31b3fb4b86b618660db5ebfbd5">For example, if the connection ID is 3, run the following command:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_ab70dc7058d5541bebbc9b46ef210e2fd"><strong id="mrs_01_1164__en-us_topic_0000001173631484_a79eef615ad8d4444aafefd4590a46be8">create job -x 3 -t import</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b9262519194311"> --trans /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-confi</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b159311019174314">g</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b9192516114319">/</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b1496614884316">oracle-hbase.json</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_ae168f57b31ca4ea6b9ba4ef5f802748c">The following information is displayed:</p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_screen1795102373019">sqoop:000> create job -x 3 -t import --trans /opt/hadoopclient/Loader/loader-tools-1.99.3/loader-tool/job-config/oracle-to-hbase.json
|
|
Creating job for connection with id 3
|
|
Please fill following values to create new job object
|
|
Name: run
|
|
|
|
Database target
|
|
|
|
Table name: test
|
|
Columns:
|
|
Conditions:
|
|
Data split method:
|
|
0 : ROWID
|
|
1 : PARTITION
|
|
Choose:
|
|
Table Partitions:
|
|
Data split allocation method:
|
|
0 : ROUNDROBIN
|
|
1 : SEQUENTIAL
|
|
2 : RANDOM
|
|
Choose:
|
|
JDBC fetch size:
|
|
|
|
Output configuration
|
|
|
|
Storage type:
|
|
0 : HDFS
|
|
1 : HBASE_BULKLOAD
|
|
2 : HBASE_PUTLIST
|
|
3 : HIVE
|
|
4 : SPARK
|
|
Choose: 1
|
|
HBase instance: HBase
|
|
Clear data before import : false
|
|
|
|
Throttling resources
|
|
|
|
Extractors: 10
|
|
Extractor size:
|
|
New job was successfully created with validation status FINE and persistent id 7
|
|
sqoop:000></pre>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p12870471384">The preceding information indicates that the job ID is 7.</p>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_li17104142743319"><span>Run the following command to start the job:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_p36329598337"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b137879262191946">start job -j</strong> <em id="mrs_01_1164__en-us_topic_0000001173631484_i4921614991946">job ID</em><strong id="mrs_01_1164__en-us_topic_0000001173631484_b163458449191946"> -s</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p1996210255348">For example, if the job ID is 7, run the following command:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p163021644193413"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b869180153515">start job -j 7 -s</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_p8729101918351">Displaying the <span class="menucascade" id="mrs_01_1164__en-us_topic_0000001173631484_menucascade88898133616"><b><span class="uicontrol" id="mrs_01_1164__en-us_topic_0000001173631484_uicontrol1488817113618">SUCCEEDED</span></b></span> information indicates that the job is started successfully.</p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_screen4203314395">Submission details
|
|
Job ID: 7
|
|
Server URL: https://10.0.0.0:21351/loader/
|
|
Created by: admintest
|
|
Creation date: 2019-12-04 16:37:34 CST
|
|
Lastly updated by: admintest
|
|
2019-12-04 16:37:34 CST: BOOTING - Progress is not available
|
|
2019-12-04 16:37:42 CST: BOOTING - 0.00 %
|
|
2019-12-04 16:37:42 CST: BOOTING - 0.00 %
|
|
2019-12-04 16:37:57 CST: RUNNING - 0.00 %
|
|
2019-12-04 16:38:12 CST: RUNNING - 45.00 %
|
|
2019-12-04 16:38:12 CST: RUNNING - 45.00 %
|
|
2019-12-04 16:38:27 CST: SUCCEEDED</pre>
|
|
</p></li></ol>
|
|
</div>
|
|
<div class="section" id="mrs_01_1164__en-us_topic_0000001173631484_section1866810514386"><h4 class="sectiontitle">Example for the Batch Mode</h4><ol id="mrs_01_1164__en-us_topic_0000001173631484_o64ee352dbec4403c8d61a41ade429584"><li id="mrs_01_1164__en-us_topic_0000001173631484_l4699e82751e345d184d4082c063a4204"><span>Log in to the node where the Loader client is installed as the user who installs the client.</span></li><li id="mrs_01_1164__en-us_topic_0000001173631484_la9176ee35923415396eaa699f24b19e0"><span>Run the following command to go to the <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_f831559174c6b40fab9a65e1f6822a994"><b>conf</b></span> directory of the sqoop-shell tool. For example, if the Loader client installation directory is <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_fc944def6c6784cde86421bcc49d5f4f6"><b>/opt/hadoopclient/Loader</b></span>, run the following command:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_a688fae95c8cf41cd9447ae8f30f94ef9"><strong id="mrs_01_1164__en-us_topic_0000001173631484_af585064bb2a147918bbd5e257ff1892d">cd /opt/hadoopclient/Loader/loader-tools-1.99.3/sqoop-shell/conf</strong></p>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_lc1599bbd7e724a339e9c9d2df27932ec"><span>Run the following command to configure authentication information:</span><p><div class="p" id="mrs_01_1164__en-us_topic_0000001173631484_a026bd479e1824637aecff835577ea5c9"><strong id="mrs_01_1164__en-us_topic_0000001173631484_a319d8ad5b6bf415f8d298fb6c2093a8d">vi client.properties</strong><pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_s9518e7a7a31944af9bb6c75685b9fa51">server.url=<em id="mrs_01_1164__en-us_topic_0000001173631484_a4c3c7caadf3648738007b72ad72fef8b">10.0.0.1:21351</em>
|
|
# simple or kerberos
|
|
authentication.type=simple
|
|
# true or false
|
|
use.keytab=true
|
|
|
|
authentication.user=
|
|
authentication.password=
|
|
|
|
client.principal=hdfs/hadoop.@<em id="mrs_01_1164__en-us_topic_0000001173631484_i1488441395115"><system domain name>@<system domain name></em>
|
|
|
|
# keytab file
|
|
client.keytab.file=./conf/login/hdfs.keytab</pre>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1164__en-us_topic_0000001173631484_te2c2382e2a6f420e9a9cbf72d79be725" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Configuration parameters</caption><thead align="left"><tr id="mrs_01_1164__en-us_topic_0000001173631484_r9f3d7657dee844a99cee7a0485dbb6a3"><th align="left" class="cellrowborder" valign="top" width="31.39%" id="mcps1.3.4.2.3.2.1.3.2.4.1.1"><p id="mrs_01_1164__en-us_topic_0000001173631484_a42929e9fd81d470eb3bd030635ffe635">Configuration parameters</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="44.55%" id="mcps1.3.4.2.3.2.1.3.2.4.1.2"><p id="mrs_01_1164__en-us_topic_0000001173631484_a64ff8f85aee84a3ba51e7884f76faa2c">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="24.060000000000002%" id="mcps1.3.4.2.3.2.1.3.2.4.1.3"><p id="mrs_01_1164__en-us_topic_0000001173631484_a3cfb4d165a6e440daa402889e95dee2e">Example Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1164__en-us_topic_0000001173631484_r36d04151cc614a958316d333927cd4a5"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_acad91ed7a43c45ebbb5d3e6b03100c08">server.url</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a95d0994a8681412babb2fa201b1a5bfd">Floating IP address and port (21351) for Loader.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a3d0ff9e026954ada8fe2a7501c5e4ca2">For compatibility, multiple IP addresses and ports can be configured and need to be separated by <span class="parmvalue" id="mrs_01_1164__en-us_topic_0000001173631484_p49db6fb154a0460c85cdccb35dfcd6b4"><b>commas (,)</b></span>. The first IP address and port must be those of Loader (21351). The others can be configured based on service requirements.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a3dc9044e57da4f0c84b0d7a548967373">10.0.0.1:21351</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_rf5a63b11689347d3a97009eddd2b0e71"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a68a2025d6ba84ea0a9d1296d98ae600a">authentication.type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a6629feab921041feb083cba6bc5958f0">Login authentication mode.</p>
|
|
<ul id="mrs_01_1164__en-us_topic_0000001173631484_u9ff1484828b141598de56da18ebc405a"><li id="mrs_01_1164__en-us_topic_0000001173631484_l90f474ac1fbf4a1ca9557dde00439b8d"><span class="parmvalue" id="mrs_01_1164__en-us_topic_0000001173631484_pf4850465c3d44257bd2e6df973989f43"><b>kerberos</b></span> indicates that the security mode is used and Kerberos authentication is performed. Kerberos authentication provides two authentication modes: the password mode and the keytab file mode.</li><li id="mrs_01_1164__en-us_topic_0000001173631484_l8ae4ad3f7c0640eb9ce0e0f5aa8ba0d0"><span class="parmvalue" id="mrs_01_1164__en-us_topic_0000001173631484_pdde26e499e9247cb8724427e7fa39aab"><b>simple</b></span> indicates that the normal mode is used and Kerberos authentication is not performed.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_aaf0bd5dcd32341b291a3ff7ff2e6f10c">kerberos</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_r47edd6d398df474bb55d4007bf415d7b"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_ac753839765f44946995e34efd911a4b9">authentication.user</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a082daea23abb41e2a98deba60bbe04f6">User for login when the normal mode or password authentication is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_aa32f893a118642a8bc3eab55185c73e3">In the keytab login mode, this parameter does not need to be set.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_ace2987ca3afc44d38f2ddd1655d0302f">bar</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_r797bb50134da4ba38fe05cca56ec7c63"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a9845a19e132d4f379d2c032b767332d5">authentication.password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_aecf651d74cab462db634e0205ccd24f6">User password for login when the password authentication mode is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a63b9c1b995604132a8fcfe36cac2a768">In the normal mode or keytab login mode, this parameter does not need to be set.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a323f1218a6b14e48a978c1f261373519">The password needs to be encrypted. The encryption method is described as follows:</p>
|
|
<ol type="a" id="mrs_01_1164__en-us_topic_0000001173631484_ofdd1434b3de24b78a4a773ccb45707c2"><li id="mrs_01_1164__en-us_topic_0000001173631484_l75912f0cc47c474e96394309ee62be84">Go to the directory where <span class="parmname" id="mrs_01_1164__en-us_topic_0000001173631484_pb281722dc35c493190a7e015e4fa738d"><b>encrypt_tool</b></span> is located. For example, if the Loader client installation directory is <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_f5b9468f8e0864488a5cd22695c28caeb"><b>/opt/hadoopclient/Loader</b></span>, run the following command:<p id="mrs_01_1164__en-us_topic_0000001173631484_aba77549b2cb9446fb2e7c5b608e4a889"><strong id="mrs_01_1164__en-us_topic_0000001173631484_a540b2228ab2a4aa89d7a8937f39a754f">cd /opt/hadoopclient/Loader/loader-tools-1.99.3</strong></p>
|
|
</li><li id="mrs_01_1164__en-us_topic_0000001173631484_l1261d65f35324abc8767bcde16d776d9">Run the following command to encrypt the non-encrypted password:<p id="mrs_01_1164__en-us_topic_0000001173631484_a77ce502de8ed41d58c837d54bbe44600"><a name="mrs_01_1164__en-us_topic_0000001173631484_l1261d65f35324abc8767bcde16d776d9"></a><a name="en-us_topic_0000001173631484_l1261d65f35324abc8767bcde16d776d9"></a><strong id="mrs_01_1164__en-us_topic_0000001173631484_b173711011147">./encrypt_tool </strong><em id="mrs_01_1164__en-us_topic_0000001173631484_i57381809141">Unencrypted password</em></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_ae057e9ce998844409d007d878ade79a6">The obtained encrypted password is used as the value of <span class="parmname" id="mrs_01_1164__en-us_topic_0000001173631484_p00a8e863a7a84fd5917cd377b7c4b7f0"><b>authentication.password</b></span>.</p>
|
|
<div class="note" id="mrs_01_1164__en-us_topic_0000001173631484_note29755391976"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_1164__en-us_topic_0000001173631484_p66471928976">If a non-encrypted password contains special characters, the special characters must be escaped. For example, the dollar sign ($) is a special character and can be escaped using single quotation marks ('), for example, <strong id="mrs_01_1164__en-us_topic_0000001173631484_b201941825144312">'1q2w#e$r'</strong>. If a non-encrypted password contains single quotation marks, use double quotation marks to escape the single quotation marks. If a non-encrypted password contains double quotation marks, use backslashes (\) to escape the double quotation marks. For details, see the shell escape character rules.</p>
|
|
</div></div>
|
|
</li></ol>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a518143b5907640d8b0650f291c2a67a6">43B80E33A96DF3D203ABBDFD1050C041</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_r6ebe5291b0e34c31bea6624c74614b93"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_ab5933841d5c241b69f4311cf72d6c83e">use.keytab</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a29ba208f8ca440faae5ca8c94ffee866">Whether to use the keytab mode to log in.</p>
|
|
<ul id="mrs_01_1164__en-us_topic_0000001173631484_u0be9df9ff2874d36b2db7b8d7593cead"><li id="mrs_01_1164__en-us_topic_0000001173631484_la42666961c08457f824dfc2599876aa4"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b45672577391946">true</strong> indicates using the keytab file to log in.</li><li id="mrs_01_1164__en-us_topic_0000001173631484_l44249ab879e24f1baa8366419bf3abe8"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b188498075991946">false</strong> indicates using the password to log in.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a2d4d3d5d2d3b4de48428041eedfee1f9">true</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_ra766d8dee1e0453e86821c50371e9621"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a186f234ac7c748578d0915f7f698e5c6">client.principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a9b14585060254ab8a5d7bb6bf1b43b07">User principal for accessing the Loader service when the keytab authentication mode is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a3aed084b2d074a71bcba84ead4d5cdcf">In the normal mode or password login mode, this parameter does not need to be set.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a2550c21beb71454ea5ed5351c1320501">loader/hadoop</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1164__en-us_topic_0000001173631484_r2ff08e334de64b63ad4df6cc0c2a7f5b"><td class="cellrowborder" valign="top" width="31.39%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.1 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a5a9507278a1342ebb2232b93dcfa0203">client.keytab.file</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.55%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.2 "><p id="mrs_01_1164__en-us_topic_0000001173631484_a487cd4e9d1fa4722b3af4b0abd36c6a0">Directory where the used keytab file is located when the keytab authentication mode is used.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_ac40634ffaa7841c895c293da906093f7">In the normal mode or password login mode, this parameter does not need to be set.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="24.060000000000002%" headers="mcps1.3.4.2.3.2.1.3.2.4.1.3 "><p id="mrs_01_1164__en-us_topic_0000001173631484_acaa065c9230045498b0fb2f16e1d03fc">/opt/client/conf/loader.keytab</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_lf3d8495af2c749a799130af34b5689b1"><span>Run the following command to go to the directory where the <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_filepath89213404091946"><b>sqoop2-shell</b></span> script is located and create a text file in the directory, such as <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_filepath8124398791946"><b>batchCommand.sh</b></span>:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_a2f12a5db93cb4de58cf140982437b816"><strong id="mrs_01_1164__en-us_topic_0000001173631484_a30bcd8d9cbb748c98ee1c9e6eb28e18f">cd /opt/hadoopclient/Loader/loader-tools-1.99.3/sqoop-shell</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_abe8ba56175ba436f8e17fddcf634a066"><strong id="mrs_01_1164__en-us_topic_0000001173631484_aeea5f67b3cb54e7ba5856bd43949f2ff">vi batchCommand.sh</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_aefacf0d703ee4d6eb07f5b99e0192728">An example of <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_f0f86fdc5a58749fd98b2926cc22af09f"><b>batchCommand.sh</b></span> is displayed as follows:</p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_s7d3cc62d03544f24bd73a4a8573eec2e">View parameters
|
|
create connection -c 4 --help
|
|
|
|
// Create a connector
|
|
create connection -c 4 -name oracle-connection --connector-connection-oracleServerIp 10.0.0.1 --connector-connection-oracleServerPort 22 --connector-connection-oracleUser root --connector-connection-oraclePassword xxxxx
|
|
|
|
Create a job
|
|
create job -t import -x 3 --connector-file-inputPath /opt/tempfile --connector-file-fileFilter * --framework-output-outputDirectory /user/loader/1 --framework-output-storageType HBase --framework-throttling-extractorSize 120 --framework-output-fileType TEXT_FILE --connector-file-splitType FILE -name test
|
|
|
|
Start a job
|
|
start job -j 7 -s</pre>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a74a53ca2995a46ffaba619d8f832c639">xxxxx is the password for the connector.</p>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_l9988d2405f5e4abebcb3f692663108b6"><span>Run the following command and the sqoop-shell tool will run the preceding commands in sequence:</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_ada9247a157ec44ceac1fc1f67b1a0487"><strong id="mrs_01_1164__en-us_topic_0000001173631484_a3ad10dcf8ab44010a0e5d9e963782e57">./sqoop2-shell batchCommand.sh</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_aa994564182be4b67b455a921aac1a3dc">The commands above authenticate login by reading configuration files. Alternatively, you can attach the authentication information to the command, that is, use the password mode or Kerberos mode to authenticate login.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_abb185c7221b249a2b8b2226f5ed5facf">Run the following command to authenticate login using the password mode:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a6e484dc359414e60b4d29bca3a29d6aa"><strong id="mrs_01_1164__en-us_topic_0000001173631484_a41a715724d5f4d84804ec0b5e4e67be1">./sqoop2-shell -uk false -u username -p encryptedPassword batchCommand.sh</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_acd49ffa2cfb8456a8c40d444dc471d78">Run the following command to authenticate login using the Kerberos mode:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a04bbd49191204d948686e53722380380"><strong id="mrs_01_1164__en-us_topic_0000001173631484_ae8a23ccd3be04cad9f0f5ff3f526e5ce">./sqoop2-shell -uk true -k user.keytab -s userPrincipal batchCommand.sh</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a4a4d90a3ccda4d72aa38a21d83e2f702">Displaying the <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_f2824cc816d8f43948e995a6a4cbf2483"><b>SUCCEEDED</b></span> information indicates that the job is started successfully.</p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_s4e592bd0568d44c495bdb2c07f087ce0">Welcome to sqoop client
|
|
Use the username and password authentication mode
|
|
Authentication success.
|
|
sqoop:000> create connection -c 4 --help
|
|
usage: Show connection viparameters:
|
|
--connector-connection-oraclePassword <arg>
|
|
--connector-connection-oracleServerIp <arg>
|
|
--connector-connection-oracleServerPort <arg>
|
|
--connector-connection-oracleUser <arg>
|
|
--framework-security-maxConnections <arg>
|
|
--name <arg>
|
|
===> FINE
|
|
sqoop:000> create connection -c 4 -name oracle-connection --connector-connection-oracleServerIp 10.0.0.1 --connector-connection-oracleServerPort 22 --connector-connection-oracleUser root --connector-connection-oraclePassword xxxxx
|
|
Creating connection for connector with id 4
|
|
New connection was successfully created with validation status FINE and persistent id 3
|
|
===> FINE
|
|
sqoop:000> create job -t import -x 3 --connector-file-inputPath /opt/tempfile --connector-file-fileFilter * --framework-output-outputDirectory /user/loader/1 --framework-output-storageType HDFS --framework-throttling-extractorSize 120 --framework-output-fileType TEXT_FILE --connector-file-splitType FILE -name test
|
|
Creating job for connection with id 3
|
|
New job was successfully created with validation status FINE and persistent id 7
|
|
===> FINE
|
|
Submission details
|
|
Job ID: 7
|
|
Server URL: https://10.0.0.0:21351/loader/
|
|
Created by: admintest
|
|
Creation date: 2019-12-04 16:37:34 CST
|
|
Lastly updated by: admintest
|
|
2019-12-04 16:37:34 CST: BOOTING - Progress is not available
|
|
2019-12-04 16:37:42 CST: BOOTING - 0.00 %
|
|
2019-12-04 16:37:42 CST: BOOTING - 0.00 %
|
|
2019-12-04 16:37:57 CST: RUNNING - 0.00 %
|
|
2019-12-04 16:38:12 CST: RUNNING - 45.00 %
|
|
2019-12-04 16:38:12 CST: RUNNING - 45.00 %
|
|
2019-12-04 16:38:27 CST: SUCCEEDED</pre>
|
|
</p></li><li id="mrs_01_1164__en-us_topic_0000001173631484_l5720e3f798cc44baab11f4ac39c82c95"><span>In the batch mode, the <strong id="mrs_01_1164__en-us_topic_0000001173631484_b68909754191946">-c</strong> parameter can be used to attach a command. sqoop-shell can execute only the attached command at a time.</span><p><p id="mrs_01_1164__en-us_topic_0000001173631484_a222621aafc3246a3b44bbcbf1a112b4d">Run the following command to create a connection:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a55a312380ecf4dd69d910a4514d1316a"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b525434471033">./sqoop2-shell -c "create connection -c 4 -name oracle-connection --connector-connection-oracleServerIp 10.0.0.1 --connector-connection-oracleServerPort 22 --connector-connection-oracleUser root --connector-connection-oraclePassword xxxxx</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b520191791611">"</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a6b039e8d9251462d938539f2abbd9f32">You can also use the password mode or Kerberos mode to attach the authentication information to the command.</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a21106f811637435bbe101a0977943775">Run the following command to authenticate login using the password mode:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_afe6b3cac4e28430ab534d86cf0eaeb5f"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b353836481033">./sqoop2-shell -uk false -u username -p encryptedPassword -c "create connection -c 4 -name oracle-connection --connector-connection-oracleerverIp 10.0.0.1 --connector-connection-oracleServerPort 22 --connector-connection-oracleUser root --connector-connection-oraclePassword xxxxx</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b3827842716114">"</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a65e4053092964ea9ae698643b918d04d">Run the following command to authenticate login using the Kerberos mode:</p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_a1e38adca25ef4e69a6978a6d013f7285"><strong id="mrs_01_1164__en-us_topic_0000001173631484_b498170361033">./sqoop2-shell -uk true -k user.keytab -s userPrincipal -c "create connection -c 4 -name oracle-connection --connector-connection-oracleServerIp 10.0.0.1 --connector-connection-oracleServerPort 22 --connector-connection-oracleUser root --connector-connection-oraclePassword xxxxx</strong><strong id="mrs_01_1164__en-us_topic_0000001173631484_b5694288516127">"</strong></p>
|
|
<p id="mrs_01_1164__en-us_topic_0000001173631484_aa223e86b2ae74b57808352c28ee8d3bb">Displaying the <span class="filepath" id="mrs_01_1164__en-us_topic_0000001173631484_f42db2ff026264b07bac114fb1a85effc"><b>FINE</b></span> information indicates the connection is created successfully.</p>
|
|
<pre class="screen" id="mrs_01_1164__en-us_topic_0000001173631484_s97391dd5429b490284a089c41b110860">Welcome to sqoop client
|
|
Use the username and password authentication mode
|
|
Authentication success.
|
|
sqoop:000> create connection -c 4 -name oracle-connection --connector-connection-oracleServerIp 10.0.0.1 --connector-connection-oracleServerPort 22 --connector-connection-oracleUser root --connector-connection-oraclePassword xxxxx
|
|
Creating connection for connector with id 4
|
|
New connection was successfully created with validation status FINE and persistent id 3
|
|
===> FINE</pre>
|
|
</p></li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1155.html">Client Tool Description</a></div>
|
|
</div>
|
|
</div>
|
|
|