forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
106 lines
9.7 KiB
HTML
106 lines
9.7 KiB
HTML
<a name="mrs_01_2341"></a><a name="mrs_01_2341"></a>
|
|
|
|
<h1 class="topictitle1">Usage Guide of HetuEngine Cross-Source Function</h1>
|
|
<div id="body32001227"><ol id="mrs_01_2341__en-us_topic_0000001219149043_ol0761234176"><li id="mrs_01_2341__en-us_topic_0000001219149043_li5768232173"><span>Register the data source by referring to <a href="mrs_01_1719.html">Configuring a HetuEngine Data Source</a>.</span></li><li id="mrs_01_2341__en-us_topic_0000001219149043_li8769235176"><span>If the HBase data source is used, you need to create a structured mapping table.</span><p><ul id="mrs_01_2341__en-us_topic_0000001219149043_ul584689161815"><li id="mrs_01_2341__en-us_topic_0000001219149043_li38461593189">The format of the statement for creating a mapping table is as follows:<pre class="screen" id="mrs_01_2341__en-us_topic_0000001219149043_screen111700911915">CREATE TABLE schemaName.tableName (
|
|
rowId VARCHAR,
|
|
qualifier1 TINYINT,
|
|
qualifier2 SMALLINT,
|
|
qualifier3 INTEGER,
|
|
qualifier4 BIGINT,
|
|
qualifier5 DOUBLE,
|
|
qualifier6 BOOLEAN,
|
|
qualifier7 TIME,
|
|
qualifier8 DATE,
|
|
qualifier9 TIMESTAMP
|
|
)
|
|
WITH (
|
|
column_mapping = 'qualifier1:f1:q1,qualifier2:f1:q2,qualifier3:f2:q3,qualifier4:f2:q4,qualifier5:f2:q5,qualifier6:f3:q1,qualifier7:f3:q2,qualifier8:f3:q3,qualifier9:f3:q4',
|
|
row_id = 'rowId',
|
|
hbase_table_name = 'hbaseNamespace:hbaseTable',
|
|
external = true
|
|
);
|
|
-- Note: The value of <strong id="mrs_01_2341__en-us_topic_0000001219149043_b1790955213203">schemaName</strong> must be the same as the value of <strong id="mrs_01_2341__en-us_topic_0000001219149043_b1393928219">hbaseNamespace</strong> in <strong id="mrs_01_2341__en-us_topic_0000001219149043_b1458624192115">hbase_table_name</strong>. Otherwise, the table fails to be created.</pre>
|
|
</li><li id="mrs_01_2341__en-us_topic_0000001219149043_li2422162601816">Supported mapping tables: Mapping tables can be directly associated with tables in the HBase data source or created and associated with new tables that do not exist in the HBase data source.</li><li id="mrs_01_2341__en-us_topic_0000001219149043_li989715074120">Supported data types in a mapping table: VARCHAR, TINYINT, SMALLINT, INTEGER, BIGINT, DOUBLE, BOOLEAN, TIME, DATE, and TIMESTAMP</li><li id="mrs_01_2341__en-us_topic_0000001219149043_li890134711810">The following table describes the keywords in the statements for creating mapping tables.
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_2341__en-us_topic_0000001219149043_table6927130102519" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Keywords in the statements for creating mapping tables</caption><thead align="left"><tr id="mrs_01_2341__en-us_topic_0000001219149043_row1792763019251"><th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.1.2.2.1.4.1.2.6.1.1"><p id="mrs_01_2341__en-us_topic_0000001219149043_p1292793016254">Keyword</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.1.2.2.1.4.1.2.6.1.2"><p id="mrs_01_2341__en-us_topic_0000001219149043_p392718308252">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.1.2.2.1.4.1.2.6.1.3"><p id="mrs_01_2341__en-us_topic_0000001219149043_p13927143010253">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.1.2.2.1.4.1.2.6.1.4"><p id="mrs_01_2341__en-us_topic_0000001219149043_p6927130172516">Default Value</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.1.2.2.1.4.1.2.6.1.5"><p id="mrs_01_2341__en-us_topic_0000001219149043_p1392723082512">Remarks</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_2341__en-us_topic_0000001219149043_row1892712305254"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.1 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p79270302255">column_mapping</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.2 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p169271730142510">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.3 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p15927163020259">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.4 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p11927143092520">All columns belong to the same Family column family.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.5 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p2928143082519">Specify the mapping between columns in the mapping table and column families in the HBase data source table. If a table in the HBase data source needs to be associated, the value of <strong id="mrs_01_2341__en-us_topic_0000001219149043_b45496556253">column_mapping</strong> must be the same as that in the HBase data source. If you create a table that does not exist in the HBase data source, you need to specify <strong id="mrs_01_2341__en-us_topic_0000001219149043_b6758101482620">column_mapping</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2341__en-us_topic_0000001219149043_row17928430142518"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.1 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p18928103010254">row_id</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.2 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p1892803082516">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.3 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p1492853014257">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.4 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p1392853011257">First column in the mapping table</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.5 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p792823012517">Column name corresponding to the rowkey table in the HBase data source</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2341__en-us_topic_0000001219149043_row1992833032519"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.1 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p1792863092516">hbase_table_name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.2 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p199281630132515">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.3 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p209282030162511">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.4 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p20928183013256">Null</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.5 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p7928173012513">Tablespace and table name of the HBase data source to be associated. Use a colon (:) to separate them. The default tablespace is <strong id="mrs_01_2341__en-us_topic_0000001219149043_b3220034132819">default</strong>. If a new table that does not exist in the HBase data source is created, <strong id="mrs_01_2341__en-us_topic_0000001219149043_b494764892819">hbase_table_name</strong> does not need to be specified.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_2341__en-us_topic_0000001219149043_row4928113016252"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.1 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p892833002515">external</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.2 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p1928133017251">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.3 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p17928153013251">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.4 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p39283302257">true</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.1.2.2.1.4.1.2.6.1.5 "><p id="mrs_01_2341__en-us_topic_0000001219149043_p492873013259">If <strong id="mrs_01_2341__en-us_topic_0000001219149043_b43101156192819">external</strong> is set to <strong id="mrs_01_2341__en-us_topic_0000001219149043_b4651358132819">true</strong>, the table is a mapping table in the HBase data source and the original table in the HBase data source cannot be deleted. If <strong id="mrs_01_2341__en-us_topic_0000001219149043_b1685487202911">external</strong> is set to false, the table in the HBase data source is deleted when the <strong id="mrs_01_2341__en-us_topic_0000001219149043_b549715182297">Hetu-HBase</strong> table is deleted.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li></ul>
|
|
</p></li><li id="mrs_01_2341__en-us_topic_0000001219149043_li07622318175"><span>Use cross-source collaborative analysis.</span><p><pre class="screen" id="mrs_01_2341__en-us_topic_0000001219149043_screen13192193631212">// 1. Register three types of data sources, including Hive, Elasticsearch, and GaussDB A.
|
|
hetuengine> show catalogs;
|
|
Catalog
|
|
----------
|
|
dws
|
|
es
|
|
hive
|
|
hive_dg
|
|
system
|
|
systemremote
|
|
(6 rows)
|
|
|
|
// 2. Compile SQL statements for cross-source collaborative analysis.
|
|
select * from hive_dg.schema1.table1 t1 join es.schema3.table3 t2 join dws.schema02.table4 t3 on t1.name = t2.item and t2.id = t3.cardNo;</pre>
|
|
</p></li></ol>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1738.html">Using the HetuEngine Cross-Source Function</a></div>
|
|
</div>
|
|
</div>
|
|
|