forked from laiweijian4/doc-exports
Xiong, Chen Xiao
f2733178cc
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com> Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
192 lines
17 KiB
HTML
192 lines
17 KiB
HTML
<a name="dataartsstudio_02_0269"></a><a name="dataartsstudio_02_0269"></a>
|
|
|
|
<h1 class="topictitle1">Link to Hive</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_02_0269__en-us_topic_0108272815_section621837"><h4 class="sectiontitle">Description</h4><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p5584437022119">By creating a Hive link, you can extract data from or load data to Hive of MRS.</p>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0269__en-us_topic_0108272815_section6163607716523"><h4 class="sectiontitle">Sample Link</h4><pre class="screen" id="dataartsstudio_02_0269__en-us_topic_0108272815_screen51442492191358">{
|
|
"links": [
|
|
{
|
|
"link-config-values": {
|
|
"configs": [
|
|
{
|
|
"inputs": [
|
|
{
|
|
"name": "linkConfig.host",
|
|
"value": "10.120.205.230"
|
|
},
|
|
{
|
|
"name": "linkConfig.authType",
|
|
"value": "KERBEROS"
|
|
},
|
|
{
|
|
"name": "linkConfig.user",
|
|
"value": "cdm"
|
|
},
|
|
{
|
|
"name": "linkConfig.password",
|
|
"value": "<em id="dataartsstudio_02_0269__en-us_topic_0108272815_i1149155918485">Add password here</em>"
|
|
}
|
|
],
|
|
"name": "linkConfig"
|
|
}
|
|
]
|
|
},
|
|
"name": "hive_link",
|
|
"connector-name": "hive-connector"
|
|
}
|
|
]
|
|
}</pre>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0269__en-us_topic_0108272815_section5035508012043"><h4 class="sectiontitle">Link Parameters</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_02_0269__en-us_topic_0108272815_table52350397173911" frame="border" border="1" rules="all"><thead align="left"><tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row42443862173911"><th align="left" class="cellrowborder" valign="top" width="19.84%" id="mcps1.3.3.2.1.5.1.1"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p15400793173911">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20.34%" id="mcps1.3.3.2.1.5.1.2"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p39504729173911">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="16.68%" id="mcps1.3.3.2.1.5.1.3"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p45766461173911">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="43.14%" id="mcps1.3.3.2.1.5.1.4"><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p16095854173911">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row66924219173911"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p52152668173911">llinkConfig.host</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p63616581173911">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p52669471173911">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p117731251522">IP address of MRS Manager</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row159351483434"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p17935138204310">linkConfig.authType</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p43871655175124">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p32281241175124">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><div class="p" id="dataartsstudio_02_0269__en-us_topic_0108272815_p121015210521">Authentication method of MRS. The options are as follows:<ul id="dataartsstudio_02_0269__en-us_topic_0108272815_ul131011722520"><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li2101122175214"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b84235270611123">SIMPLE</strong>: for non-security mode</li><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li3101426524"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b842352706183921_3">KERBEROS</strong>: for security mode</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row1247610113513"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p176731282356">linkConfig.principal</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p198011521353">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1298065218357">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1450092103611">Account <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b10925115015918">principal</strong> required for Kerveros authentication. You can contact the administrator to obtain the account. Before using the cluster configuration, you must set this parameter in cluster configuration management.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row192261613133517"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p12940816193514">linkConfig.keytab</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p2098025217352">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p8980145203519">FileContent</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p69402160358">Local absolute path of the <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1127015425917">keytab</strong> file required for Kerveros authentication. You can also contact the administrator to obtain the file. Before using the cluster configuration, you must set this parameter in cluster configuration management.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row10200192435319"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1220162415533">linkConfig.hiveVersion</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p92011224105312">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p191422417542">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p132011524135310">Hive version</p>
|
|
<ul id="dataartsstudio_02_0269__en-us_topic_0108272815_ul15916412550"><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li17599419558">HIVE_1_X</li><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li8597455513">HIVE_3_X</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row91261657116"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1567019719436">linkConfig.user</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p154701718362">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1347051143620">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p120393117135">Username for logging in to Manager. This parameter is not required if the cluster configuration is used.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row3460168101116"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1234310844315">linkConfig.password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p4221926369">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1022116213365">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p18429123121418">Password for logging in to Manager. This parameter is not required if the cluster configuration is used.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row772434015310"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p30068829175124">linkConfig.uri</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p379375175124">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p46443378175124">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p38035211748">NameNode URI required for the link to Apache Hadoop. The format is <em id="dataartsstudio_02_0269__en-us_topic_0108272815_i18185422421">ip:port</em>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row1696304417315"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1335419511036">linkConfig.hiveMsUris</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p103541514316">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p103547510314">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p45098918617">Hive metadata address for connecting to Apache Hadoop. For details, see the <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1428754210218">hive.metastore.uris</strong> configuration item. Example: thrift://host-192-168-1-212:9083</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row115761427115316"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p10577327165314">linkConfig.obsSupport</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1857762705312">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p16577112712531">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p9577627155311">The server must support OBS storage. When creating a Hive table, you can store the table in OBS.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row8709615599"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p47101415094">linkConfig.runMode</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p47105151496">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p177101156911">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1679911515118">This parameter is used only when the Hive version is <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b974215231357">HIVE_3_</strong><em id="dataartsstudio_02_0269__en-us_topic_0108272815_i1759525459">X</em>. Possible values are:</p>
|
|
<ul id="dataartsstudio_02_0269__en-us_topic_0108272815_ul5799105141116"><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li179919561113"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1495119313516">EMBEDDED</strong>: The link instance runs with CDM. This mode delivers better performance.</li><li id="dataartsstudio_02_0269__en-us_topic_0108272815_li1679914517118"><strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b14730333957">STANDALONE</strong>: The link instance runs in an independent process. If CDM needs to connect to multiple Hadoop data sources (MRS, Hadoop, or CloudTable) with both Kerberos and Simple authentication modes, select <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b1259513616516">STANDALONE</strong> or configure different agents.<p id="dataartsstudio_02_0269__en-us_topic_0108272815_p27991759118">Note: The <strong id="dataartsstudio_02_0269__en-us_topic_0108272815_b102371046451">STANDALONE</strong> mode is used to solve the version conflict problem. If the connector versions of the source and destination ends of the same link are different, a JAR file conflict occurs. In this case, you need to place the source or destination end in the STANDALONE process to prevent the migration failure caused by the conflict.</p>
|
|
</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row11707619792"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p7204562141730">linkConfig.accessKey</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p46698681141730">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p24496779141730">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1585198617219">AK This parameter is mandatory when OBS is supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row39949228915"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p45889226141730">linkConfig.securityKey</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p26039827141730">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p28851277141730">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1549749917230">SK This parameter is mandatory when OBS is supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0269__en-us_topic_0108272815_row45101345515"><td class="cellrowborder" valign="top" width="19.84%" headers="mcps1.3.3.2.1.5.1.1 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p843281310380">linkConfig.properties</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20.34%" headers="mcps1.3.3.2.1.5.1.2 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1196925110380">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.1.5.1.3 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p2998524510380">Map</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="43.14%" headers="mcps1.3.3.2.1.5.1.4 "><p id="dataartsstudio_02_0269__en-us_topic_0108272815_p1288576310380">Properties. You can add configuration properties of the client. Each property must contain a name and a value.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_02_0261.html">Link Parameter Description</a></div>
|
|
</div>
|
|
</div>
|
|
|