forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
80 lines
13 KiB
HTML
80 lines
13 KiB
HTML
<a name="mrs_01_1125"></a><a name="mrs_01_1125"></a>
|
|
|
|
<h1 class="topictitle1">HBase Input</h1>
|
|
<div id="body8662426"><div class="section" id="mrs_01_1125__en-us_topic_0000001219350823_s70fac88f578d4cc58433d53d433c074d"><h4 class="sectiontitle">Overview</h4><p id="mrs_01_1125__en-us_topic_0000001219350823_a29a9f44124874eaca372121c33499d60">The <span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_pf5ed7e00ac96460b93f4c2ae1d375e6c"><b>HBase Input</b></span> operator converts specified columns in an HBase table into input fields of the same quantity.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_1125__en-us_topic_0000001219350823_s0b31047ebe764ba482c0b18e0e3116ce"><h4 class="sectiontitle">Input and Output</h4><ul id="mrs_01_1125__en-us_topic_0000001219350823_uec5b389bf44e4c69986180bc2e962862"><li id="mrs_01_1125__en-us_topic_0000001219350823_l6b27c19037034dbd8827c7fae639056e">Input: HBase table columns</li><li id="mrs_01_1125__en-us_topic_0000001219350823_l3d55ce273dc34ba3bfc614da30aed834">Output: fields</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1125__en-us_topic_0000001219350823_sfed1bd5ee8574c3da5d4115d7a8fa973"><h4 class="sectiontitle">Parameter Description</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1125__en-us_topic_0000001219350823_t6f4875aad0ff4253b2b76c21a94ed438" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Operator parameter description</caption><thead align="left"><tr id="mrs_01_1125__en-us_topic_0000001219350823_re7647161b98f4b2492d3ad6ccac81b2d"><th align="left" class="cellrowborder" valign="top" width="16.68%" id="mcps1.3.3.2.2.6.1.1"><p id="mrs_01_1125__en-us_topic_0000001219350823_a4e0a2d1d7fe442cd827e7ddd28c26b70">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="44.019999999999996%" id="mcps1.3.3.2.2.6.1.2"><p id="mrs_01_1125__en-us_topic_0000001219350823_ae55e7f9ba9d747a0b80b47582430ab55">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="9.959999999999999%" id="mcps1.3.3.2.2.6.1.3"><p id="mrs_01_1125__en-us_topic_0000001219350823_afcae78a397a849b9ac4ecc9db7de390f">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="16.27%" id="mcps1.3.3.2.2.6.1.4"><p id="mrs_01_1125__en-us_topic_0000001219350823_a39d0f4092f994913a9cb84b72aa35b1c">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="13.07%" id="mcps1.3.3.2.2.6.1.5"><p id="mrs_01_1125__en-us_topic_0000001219350823_a19affef49a4440df87606ee77a160d34">Default Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1125__en-us_topic_0000001219350823_r62e862a48c65499cb02d25b0276641e5"><td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.2.6.1.1 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a1559d65aa06a496090e273142bddfb23">Hbase Table Type</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.019999999999996%" headers="mcps1.3.3.2.2.6.1.2 "><p id="mrs_01_1125__en-us_topic_0000001219350823_abaf79d5469fa4088b732c78b6e1d7312">HBase table type. The options include <strong id="mrs_01_1125__en-us_topic_0000001219350823_a3f984a6e0ac6408da01396f41220e7db">normal</strong> (common HBase table) and <strong id="mrs_01_1125__en-us_topic_0000001219350823_a26b6e5faf1fd433a8177aac199fa988f">phoenix</strong>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.959999999999999%" headers="mcps1.3.3.2.2.6.1.3 "><p id="mrs_01_1125__en-us_topic_0000001219350823_aaf0e7fa550c440fc881ed925eb145e5d">enum</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.27%" headers="mcps1.3.3.2.2.6.1.4 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a15bd9b0b936b46429e6f25d79eeac924">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.07%" headers="mcps1.3.3.2.2.6.1.5 "><p id="mrs_01_1125__en-us_topic_0000001219350823_acb99e05e09cc4706af40d7270e3fb12c">normal</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1125__en-us_topic_0000001219350823_r8165c8260a1142369e2a574b9078fb79"><td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.2.6.1.1 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a65ea7a14fbfb4b34a6a33bcccfd398a4">HBase table name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.019999999999996%" headers="mcps1.3.3.2.2.6.1.2 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a12b06907dd504dc4980b327f18dcd8de">HBase table name. Only one HBase table is supported.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.959999999999999%" headers="mcps1.3.3.2.2.6.1.3 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a85c303b72e1b43b49e495994762cb01b">string</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.27%" headers="mcps1.3.3.2.2.6.1.4 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a1c220a6a737e4e3dbd80aa844b366d73">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.07%" headers="mcps1.3.3.2.2.6.1.5 "><p id="mrs_01_1125__en-us_topic_0000001219350823_aa3e7d3b4efc042e88b1d387ff5719816">None</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1125__en-us_topic_0000001219350823_rd7d0c97fe5d945688b888c263dd11f8f"><td class="cellrowborder" valign="top" width="16.68%" headers="mcps1.3.3.2.2.6.1.1 "><p id="mrs_01_1125__en-us_topic_0000001219350823_abf397fd4ec624dc3a3b884160519188b">HBase input fields</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="44.019999999999996%" headers="mcps1.3.3.2.2.6.1.2 "><p id="mrs_01_1125__en-us_topic_0000001219350823_afde6561b26024d3b87f5c1745bcdca6d">HBase input information:</p>
|
|
<ul id="mrs_01_1125__en-us_topic_0000001219350823_ubaea43b943df40668dd38591d5dfb4ab"><li id="mrs_01_1125__en-us_topic_0000001219350823_l0da88de1bf064ad38ca470ca1d0e67d7"><span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p81c328c9abd34924a61bc828060bd846"><b>family name</b></span>: HBase column family name.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_l4a6f057ef6e748e383c1cf2314f99245"><span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_pa9a2d1043ee6455fa214f549fc86c9ca"><b>column name</b></span>: HBase column name.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_lbda33c90e35149a8b25217623fc87110"><span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p5ca571681abd4752b216e512ab971703"><b>field name</b></span>: Names of input fields.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_l438dde124c4c436d931fd6b1ab3b1163"><span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p33008e31e7c9450da9ea6cebac2e2bd7"><b>type</b></span>: Field type.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_lfb81836d485c435db1def5865c9d6d0f"><span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_pa5ef02e761c8417e974e421d9cebe7ce"><b>length</b></span>: Field value length. If the actual field value is excessively long, the value is cut based on the configured length. When <span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p8578816ab0f24d51ab023fd0ef78887a"><b>type</b></span> is set to <span class="parmvalue" id="mrs_01_1125__en-us_topic_0000001219350823_pe3ea18270e8648a28ed26369afb2bd91"><b>CHAR</b></span>, spaces are added to the field value for supplement if the actual field value length is less than the configured length. When <span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p05ffdfb5988e4e40a2fbf10755192d99"><b>type</b></span> is set to <span class="parmvalue" id="mrs_01_1125__en-us_topic_0000001219350823_p27fca5ca5ef746e6b6986114bccb4759"><b>VARCHAR</b></span>, no space is added to the field value for supplement if the actual field value length is less than the configured length.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_lce98d80e9f1d4c168d4ecbbd182a7ec7"><span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p433a36e2c7c240ffa6d2cd5e59645301"><b>is rowkey</b></span>: Indicates whether a column is a primary key column. A common HBase table can have only one primary key, while a phoenix table can have multiple primary keys. If multiple primary keys are configured, they are combined according to the configuration sequence. At least one primary key column must be configured.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="9.959999999999999%" headers="mcps1.3.3.2.2.6.1.3 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a289d59c5748140949b03bca5691c697e">map</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="16.27%" headers="mcps1.3.3.2.2.6.1.4 "><p id="mrs_01_1125__en-us_topic_0000001219350823_ac5b3926346c84cfdb4a767dff99b4103">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="13.07%" headers="mcps1.3.3.2.2.6.1.5 "><p id="mrs_01_1125__en-us_topic_0000001219350823_a02eec390b988424aa0181f85c6f0d455">None</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1125__en-us_topic_0000001219350823_s99aa3e90fcfe441281e46356a963cb18"><h4 class="sectiontitle">Data Processing Rule</h4><ul id="mrs_01_1125__en-us_topic_0000001219350823_ub7fab121430848939d3c87789d0b690d"><li id="mrs_01_1125__en-us_topic_0000001219350823_lf186d6609a644b828e29151fa8934455">If the HBase table name does not exist, the job fails to be submitted.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_l87b8f540fad54691abf19494e7c6734c">If the configured column names are inconsistent with the HBase table column names, the data cannot be read and the number of imported data records is 0.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_lcbb6aa6b483a40e68528823223b7a3f3">If the number of input field columns is greater than the number of field columns actually included in the original data, all data becomes dirty data.</li><li id="mrs_01_1125__en-us_topic_0000001219350823_l8dac622c30ad4ed2b2de4d3ebc0dbb0c">If the field value does not match the actual type, the data in the line will become dirty data.</li></ul>
|
|
</div>
|
|
<div class="section" id="mrs_01_1125__en-us_topic_0000001219350823_sabee1f44246148c58b11ac8e0ea7278f"><h4 class="sectiontitle">Example</h4><p id="mrs_01_1125__en-us_topic_0000001219350823_a472e0c713c8d4e9ebdf5aa8f255fe55a">Use the data export from HBase to sqlserver2014 as an example.</p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_a59502e254c3b4a3d9f11cd0df4afca55">In sqlserver2014, run the following statement to create an empty data test_1 for storing HBase data:</p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_aab0ff76452a14042b74935bca807e834"><strong id="mrs_01_1125__en-us_topic_0000001219350823_abb2f6520a21845c2b31c399adc2fcc57">create table test_1 (id int, name text, value text);</strong></p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_a22b8abfa7727426a8eeb7c5de68c4730">Configure the <span class="parmname" id="mrs_01_1125__en-us_topic_0000001219350823_p220be9fa8ef341aba18731b290423c57"><b>HBase Input</b></span> operator to generate fields A, B, and C.</p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_add76abb41fe340e3be9d08f1f1c0381a">After the database connection is set up, click <span class="menucascade" id="mrs_01_1125__en-us_topic_0000001219350823_m28831cf8cc474b6684f05f9dfd729622"><b><span class="uicontrol" id="mrs_01_1125__en-us_topic_0000001219350823_u37dae6950a5d4a4490cabeaecdda7327">autoRecognition</span></b></span>. The system will automatically read fields in the database and select required fields for adding. You only need to optimize or modify the fields manually based on service scenarios.</p>
|
|
<div class="note" id="mrs_01_1125__en-us_topic_0000001219350823_n909d5ea58cae4eba8a4623ff19318d69"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="mrs_01_1125__en-us_topic_0000001219350823_a9b1dbe610aa94f02b78c4d400f76b793">Performing this operation will overwrite existing data in the table.</p>
|
|
</div></div>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_aebb9caf332f74f7e8539097b25a84430"><span><img id="mrs_01_1125__en-us_topic_0000001219350823_i476a3e5543664f2b89a220d210d1b5b0" src="en-us_image_0000001349139569.png"></span></p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_a70905a341cb84d649c6d1c892de43e2f">Use the <strong id="mrs_01_1125__en-us_topic_0000001219350823_a2fbfa657b03348a5baf0224fad6ab44d">Table Out</strong> operator to export A, B, and C to the test_1 table.</p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_a8fbefab3dd6d421dbfc8d0f540812c17"><strong id="mrs_01_1125__en-us_topic_0000001219350823_a11b26f807b2c404aba5904464a663ed2">select * from test_1;</strong></p>
|
|
<p id="mrs_01_1125__en-us_topic_0000001219350823_a83bb10f517894e80a81568c89ee50ab0"><span><img id="mrs_01_1125__en-us_topic_0000001219350823_i00477d03f944468c856dbc7ba080424e" src="en-us_image_0000001348739881.jpg"></span></p>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1121.html">Input Operators</a></div>
|
|
</div>
|
|
</div>
|
|
|