forked from docs/doc-exports
Xiong, Chen Xiao
14a6d65e8c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com> Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
165 lines
15 KiB
HTML
165 lines
15 KiB
HTML
<a name="dataartsstudio_02_0300"></a><a name="dataartsstudio_02_0300"></a>
|
|
|
|
<h1 class="topictitle1">To HBase/CloudTable</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_02_0300__en-us_topic_0108272851_section33401108172339"><h4 class="sectiontitle">Sample JSON File</h4><pre class="screen" id="dataartsstudio_02_0300__en-us_topic_0108272851_screen8420958101454">"to-config-values": {
|
|
"configs": [
|
|
{
|
|
"inputs": [
|
|
{
|
|
"name": "toJobConfig.table",
|
|
"value": "rf_to"
|
|
},
|
|
{
|
|
"name": "toJobConfig.storageType",
|
|
"value": "PUTLIST"
|
|
},
|
|
{
|
|
"name": "toJobConfig.columns",
|
|
"value": "AA:AA&BB:BB&CC:CC&DD:DD"
|
|
},
|
|
{
|
|
"name": "toJobConfig.rowKeyColumn",
|
|
"value": "AA:AA"
|
|
},
|
|
{
|
|
"name": "toJobConfig.isOverride",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"name": "toJobConfig.isRowkeyRedundancy",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"name": "toJobConfig.algorithm",
|
|
"value": "NONE"
|
|
},
|
|
{
|
|
"name": "toJobConfig.writeToWAL",
|
|
"value": "true"
|
|
},
|
|
{
|
|
"name": "toJobConfig.transType",
|
|
"value": "false"
|
|
}
|
|
],
|
|
"name": "toJobConfig"
|
|
}
|
|
]
|
|
}</pre>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0300__en-us_topic_0108272851_section48102183111156"><h4 class="sectiontitle">Parameter Description</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_02_0300__en-us_topic_0108272851_table13922888141527" frame="border" border="1" rules="all"><thead align="left"><tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row229143141527"><th align="left" class="cellrowborder" valign="top" width="22.657734226577343%" id="mcps1.3.2.2.1.5.1.1"><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p66756185141527">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.96800319968003%" id="mcps1.3.2.2.1.5.1.2"><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p38541938141527">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="17.23827617238276%" id="mcps1.3.2.2.1.5.1.3"><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p34889279141527">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="40.135986401359865%" id="mcps1.3.2.2.1.5.1.4"><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p7459369141527">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row62628929141527"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p29098284165959">toJobConfig.table</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p27015713141527">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p50167516142856">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p2430205141527">Name of the table to which data is written. For example, <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue15419570101029"><b>TBL_EXAMPLE</b></span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row64475722155949"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p32328561155955">toJobConfig.storageType</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p1367823155955">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p43684849155955">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><div class="p" id="dataartsstudio_02_0300__en-us_topic_0108272851_p52791884154748">Mode for writing data to an HBase table. The options are as follows:<ul id="dataartsstudio_02_0300__en-us_topic_0108272851_ul54155538155037"><li id="dataartsstudio_02_0300__en-us_topic_0108272851_li5641490220549"><strong id="dataartsstudio_02_0300__en-us_topic_0108272851_b28691531163917">PUTLIST</strong>: Data is written using the putList() function.</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row1211632319404"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p29681527141527">toJobConfig.columns</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p55393515141527">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p57689731141527">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p42356648141527">Columns to be extracted. Use <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue803971483171345_1"><b>&</b></span> to separate column numbers and <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue556591817149_1"><b>:</b></span> to separate column families and columns. For example, <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue460980378171446_1"><b>cf1:c1&cf2:c2</b></span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row5807461019222"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p642293319222">toJobConfig.rowKeyColumn</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p5049560219222">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p6361195119222">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p5229438819222">Columns serve as rowkeys. Use <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue803971483171345_3"><b>&</b></span> to separate column numbers and <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue556591817149_3"><b>:</b></span> to separate column families and columns. For example, <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue460980378171446_3"><b>cf1:c1&cf2:c2</b></span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row45367709192212"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p50905776192212">toJobConfig.isOverride</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p29727229192212">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p59095338192212">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p21993055192212">Whether to clear data when data is imported in BULKLOAD mode. For example, <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue27697732017547"><b>true</b></span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row44180181192225"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p21824946192225">toJobConfig.delimiter</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p22990191192225">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p50266164192225">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p45027498192225">Delimiter used for separating columns when multiple columns are used as rowkeys. For example, vertical bars (|).</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row40951843151814"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p28764986151814">toJobConfig.isRowkeyRedundancy</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p48262521151814">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p16950153151814">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p30785170151814">Whether to write rowkey data to the HBase column at the same time</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row870435217434"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p1770415524430">toJobConfig.algorithm</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p3704155284317">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p1704752114317">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p1870405219437">Compression algorithm used when a new HBase table is created. The Snappy and GZ algorithms are supported. The default value is <span class="parmvalue" id="dataartsstudio_02_0300__en-us_topic_0108272851_parmvalue1060204763154538"><b>None</b></span>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row98514123112"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p12850133114">toJobConfig.writeToWAL</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p585617317">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p1291535313115">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p791673125616">Whether to enable Write Ahead Log (WAL) of HBase. The options are as follows:</p>
|
|
<ul id="dataartsstudio_02_0300__en-us_topic_0108272851_ul1899132517107"><li id="dataartsstudio_02_0300__en-us_topic_0108272851_li1199114259100"><strong id="dataartsstudio_02_0300__en-us_topic_0108272851_b1272202062211">Yes</strong>: If the HBase server breaks down after the function is enabled, you can replay the operations that have not been performed in WAL.</li><li id="dataartsstudio_02_0300__en-us_topic_0108272851_li1382192814103"><strong id="dataartsstudio_02_0300__en-us_topic_0108272851_b37382052192218">No</strong>: If you set this parameter to <strong id="dataartsstudio_02_0300__en-us_topic_0108272851_b157381352162219">No</strong>, the write performance is improved. However, if the HBase server breaks down, data may be lost.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0300__en-us_topic_0108272851_row1253573222917"><td class="cellrowborder" valign="top" width="22.657734226577343%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p05351632182912">toJobConfig.transType</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.96800319968003%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p1535163262915">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.23827617238276%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0300__en-us_topic_0108272851_p953563272917">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.135986401359865%" headers="mcps1.3.2.2.1.5.1.4 "><ul id="dataartsstudio_02_0300__en-us_topic_0108272851_ul4509123194219"><li id="dataartsstudio_02_0300__en-us_topic_0108272851_li125091936429"><strong id="dataartsstudio_02_0300__en-us_topic_0108272851_b161911218192319">true</strong>: Data of the Short, Int, Long, Float, Double, and Decimal columns in the source database is converted into Byte[] arrays (binary) and written into HBase. Other types of data are written as character strings. If several types of data mentioned above are combined as rowkeys, they will be written as character strings.<p id="dataartsstudio_02_0300__en-us_topic_0108272851_p12731115251120">This function saves storage space. In specific scenarios, the rowkey distribution is evener.</p>
|
|
</li><li id="dataartsstudio_02_0300__en-us_topic_0108272851_li18571121654717"><strong id="dataartsstudio_02_0300__en-us_topic_0108272851_b78511293257">false</strong>: All types of data in the source database are written into HBase as character strings.</li></ul>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="destination_job_parameters.html">Destination Job Parameters</a></div>
|
|
</div>
|
|
</div>
|
|
|