forked from docs/doc-exports
Xiong, Chen Xiao
14a6d65e8c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com> Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
193 lines
18 KiB
HTML
193 lines
18 KiB
HTML
<a name="dataartsstudio_02_0269"></a><a name="dataartsstudio_02_0269"></a>
|
|
|
|
<h1 class="topictitle1">Link to Hive</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_02_0269__en-us_topic_0108272815_section621837"><h4 class="sectiontitle">Description</h4><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p5584437022119">By creating a Hive link, you can extract data from or load data to Hive of MRS.</p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0269__en-us_topic_0108272815_section6163607716523"><h4 class="sectiontitle">Sample Link</h4><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1995319452492">The following is the message body of a sample link. You are advised to store the AK and SK in ciphertext in the configuration file or environment variables and decrypt them when needed to ensure security.</p>
|
|
<pre class="screen" id="dataartsstudio_02_0269__en-us_topic_0108272815_screen51442492191358">{
|
|
"links": [
|
|
{
|
|
"link-config-values": {
|
|
"configs": [
|
|
{
|
|
"inputs": [
|
|
{
|
|
"name": "linkConfig.host",
|
|
"value": "10.120.205.230"
|
|
},
|
|
{
|
|
"name": "linkConfig.authType",
|
|
"value": "KERBEROS"
|
|
},
|
|
{
|
|
"name": "linkConfig.user",
|
|
"value": "cdm"
|
|
},
|
|
{
|
|
"name": "linkConfig.password",
|
|
"value": "<em id="dataartsstudio_02_0269__en-us_topic_0108272815_i1149155918485">Add password here</em>"
|
|
}
|
|
],
|
|
"name": "linkConfig"
|
|
}
|
|
]
|
|
},
|
|
"name": "hive_link",
|
|
"connector-name": "hive-connector"
|
|
}
|
|
]
|
|
}</pre>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0269__en-us_topic_0108272815_section5035508012043"><h4 class="sectiontitle">Link Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_02_0269__en-us_topic_0108272815_table52350397173911" frame="border" border="1" rules="all"><thead align="left"><tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row42443862173911"><th align="left" class="cellrowborder" valign="top" width="19.84%" id="mcps1.3.3.2.1.5.1.1"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p15400793173911">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20.34%" id="mcps1.3.3.2.1.5.1.2"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p39504729173911">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="16.68%" id="mcps1.3.3.2.1.5.1.3"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p45766461173911">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="43.14%" id="mcps1.3.3.2.1.5.1.4"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p16095854173911">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row66924219173911"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p52152668173911">llinkConfig.host</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p63616581173911">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p52669471173911">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p117731251522">IP address of MRS Manager</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row159351483434"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p17935138204310">linkConfig.authType</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p43871655175124">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p32281241175124">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><div class="p" id="dataartsstudio_02_0269__en-us_topic_0108272815_p121015210521">Authentication method of MRS. The options are as follows:<ul id="dataartsstudio_02_0269__en-us_topic_0108272815_ul131011722520"><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li2101122175214"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b84235270611123">SIMPLE</strong>: for non-security mode</li><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li3101426524"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b842352706183921_3">KERBEROS</strong>: for security mode</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row1247610113513"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p176731282356">linkConfig.principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p198011521353">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1298065218357">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1450092103611">Account <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b844619173491">principal</strong> required for Kerveros authentication. You can contact the administrator to obtain the account. Before using the cluster configuration, you must set this parameter in cluster configuration management.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row192261613133517"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p12940816193514">linkConfig.keytab</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p2098025217352">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p8980145203519">FileContent</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p69402160358">Local absolute path of the <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1127015425917">keytab</strong> file required for Kerveros authentication. You can also contact the administrator to obtain the file. Before using the cluster configuration, you must set this parameter in cluster configuration management.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row10200192435319"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1220162415533">linkConfig.hiveVersion</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p92011224105312">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p191422417542">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p132011524135310">Hive version</p>
|
|
<ul id="dataartsstudio_02_0269__en-us_topic_0108272815_ul15916412550"><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li17599419558">HIVE_1_X</li><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li8597455513">HIVE_3_X</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row91261657116"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1567019719436">linkConfig.user</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p154701718362">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1347051143620">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p120393117135">Username for logging in to Manager. This parameter is not required if the cluster configuration is used.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row3460168101116"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1234310844315">linkConfig.password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p4221926369">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1022116213365">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p18429123121418">Password for logging in to Manager. This parameter is not required if the cluster configuration is used.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row772434015310"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p30068829175124">linkConfig.uri</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p379375175124">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p46443378175124">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p38035211748">NameNode URI required for the link to Apache Hadoop. The format is <em id="dataartsstudio_02_0269__en-us_topic_0108272815_i18185422421">ip:port</em>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row1696304417315"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1335419511036">linkConfig.hiveMsUris</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p103541514316">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p103547510314">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p45098918617">Hive metadata address for connecting to Apache Hadoop. For details, see the <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1428754210218">hive.metastore.uris</strong> configuration item. Example: thrift://host-192-168-1-212:9083</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row115761427115316"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p10577327165314">linkConfig.obsSupport</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1857762705312">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p16577112712531">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p9577627155311">The server must support OBS storage. When creating a Hive table, you can store the table in OBS.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row8709615599"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p47101415094">linkConfig.runMode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p47105151496">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p177101156911">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1679911515118">This parameter is used only when the Hive version is <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b974215231357">HIVE_3_</strong><em id="dataartsstudio_02_0269__en-us_topic_0108272815_i1759525459">X</em>. Possible values are:</p>
|
|
<ul id="dataartsstudio_02_0269__en-us_topic_0108272815_ul5799105141116"><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li179919561113"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1495119313516">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li1679914517118"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b14730333957">STANDALONE</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, select <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1259513616516">STANDALONE</strong> or configure different agents.<p id="dataartsstudio_02_0269__en-us_topic_0108272815_p27991759118">Note: The <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b102371046451">STANDALONE</strong> mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row11707619792"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p7204562141730">linkConfig.accessKey</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p46698681141730">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p24496779141730">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1585198617219">AK. This parameter is mandatory when OBS is supported. You are advised to store it in ciphertext in the configuration file or an environment variable and decrypt it when needed to ensure security.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row39949228915"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p45889226141730">linkConfig.securityKey</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p26039827141730">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p28851277141730">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1549749917230">SK. This parameter is mandatory when OBS is supported. You are advised to store it in ciphertext in the configuration file or an environment variable and decrypt it when needed to ensure security.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row45101345515"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p843281310380">linkConfig.properties</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1196925110380">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p2998524510380">Map</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1288576310380">Properties. You can add configuration properties of the client. Each property must contain a name and a value.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="link_parameter.html">Link Parameter Description</a></div>
|
|
</div>
|
|
</div>
|
|
|