vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/mrs/component-operation-guide/mrs_01_0404.html
Yang, Tong 6182f91ba8 MRS component operation guide_normal 2.0.38.SP20 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Yang, Tong <yangtong2@huawei.com>
Co-committed-by: Yang, Tong <yangtong2@huawei.com>
2022-12-09 14:55:21 +00:00

219 lines
22 KiB
HTML
Raw Blame History

<a name="mrs_01_0404"></a><a name="mrs_01_0404"></a>
<h1 class="topictitle1">Source Link Configurations of Loader Jobs</h1>
<div id="body1589421630552"><div class="section" id="mrs_01_0404__se787b8d017e44b12aff2f6e2f1b4e627"><h4 class="sectiontitle">Overview</h4><p id="mrs_01_0404__aa302f2771c3743b5b20e3efdb24987ae">When Loader jobs obtain data from different data sources, a link corresponding to a data source type needs to be selected and the link properties need to be configured.</p>
<p id="mrs_01_0404__p85814576378">This section applies to versions earlier than MRS 3.x.</p>
</div>
<div class="section" id="mrs_01_0404__sdd455438f59c455d868736ad52d1097c"><a name="mrs_01_0404__sdd455438f59c455d868736ad52d1097c"></a><a name="sdd455438f59c455d868736ad52d1097c"></a><h4 class="sectiontitle">obs-connector</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0404__t965be05e8bc445ceb862e8444f43702d" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Data source link properties of <strong id="mrs_01_0404__b84235270612316">obs-connector</strong></caption><thead align="left"><tr id="mrs_01_0404__r89610f38a3bd4777bd5e9ca134b222e5"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.2.2.2.3.1.1"><p id="mrs_01_0404__a8072335d76834b18b0f44b7f9c2d21a1">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75%" id="mcps1.3.2.2.2.3.1.2"><p id="mrs_01_0404__adc3b79b34b2d4c05ad3783fe1724431e">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0404__rf0eae9e862f14c50ad7b3ad004b45b49"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__af0e31907c16b4f3ab5df0206def965ef">Bucket Name</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><p id="mrs_01_0404__a485981e1bcdc42828eeaed6308d4a663">OBS file system for storing source data.</p>
</td>
</tr>
<tr id="mrs_01_0404__r9e0c98cd97f549c98fd562c1fb8ce2ce"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__a202f05431f8e4102ad9a9fb58f7f6e34">Source Directory/File</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><p id="mrs_01_0404__a95f75beacfc5450581e99147c1f7f892">Actual storage form of source data. It can be either all data files in a directory or a single data file contained in the file system.</p>
</td>
</tr>
<tr id="mrs_01_0404__rcf91c52be99d428ea9d9d0418b409504"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__a49a663116b1e4a29a383fa8104f0c818">File Format</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><p id="mrs_01_0404__a15457bb719344f6e880a4319fa42c522">Loader supports the following file formats of data stored in OBS:</p>
<ul id="mrs_01_0404__ue5bdb47f32be4b6e9883eadcc42874d5"><li id="mrs_01_0404__l5272ed0b438449e5ae712b46e50385c0"><strong id="mrs_01_0404__b842352706173625">CSV_FILE</strong>: Specifies a text file. When the destination link is a database link, only the text file is supported.</li><li id="mrs_01_0404__lb4580a987e0646ba9cfabefe5b1e2851"><strong id="mrs_01_0404__b842352706173630">BINARY_FILE</strong>: Specifies binary files excluding text files.</li></ul>
</td>
</tr>
<tr id="mrs_01_0404__rc2a65a9a1f9d45a889a2b7d892065e33"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__en-us_topic_0071084972_p547205051766">Line Separator</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><p id="mrs_01_0404__en-us_topic_0071084972_p527477181766">Identifier of each line end of source data</p>
</td>
</tr>
<tr id="mrs_01_0404__r2c130675a2be4d11a2ddf516de4413e3"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__a06fc20d8943048638481cac86001b285">Field Separator</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><p id="mrs_01_0404__a8a4d1140c3a143c7b1766f0995064d1d">Identifier of each field end of source data</p>
</td>
</tr>
<tr id="mrs_01_0404__rcde61c44288b43ac9c08c0a24cb1eea3"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__a8e482078c4934372826fe1b572c9a290">Encoding Type</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><p id="mrs_01_0404__aad20a27929c142f7b8f3b7fd77de565f">Text encoding type of source data. It takes effect on text files only.</p>
</td>
</tr>
<tr id="mrs_01_0404__r4b8fe32f452747bd9f0c984691398638"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.2.2.2.3.1.1 "><p id="mrs_01_0404__en-us_topic_0071084972_p179695571766">File Split Type</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.2.2.2.3.1.2 "><div class="p" id="mrs_01_0404__en-us_topic_0071084972_p98470941766">The following types are supported:<ul id="mrs_01_0404__u9354c314b2a94b48b61fb993acf15d96"><li id="mrs_01_0404__en-us_topic_0071084972_li94552341785"><strong id="mrs_01_0404__b84235270614438">File</strong>: The number of files is assigned to a map task by the total number of files. The calculation formula is <span class="parmvalue" id="mrs_01_0404__parmvalue725802544142242"><b>Total number of files/Extractors</b></span>.</li><li id="mrs_01_0404__lf2d0c21e8edf4a3683ffbc953037dfd6"><strong id="mrs_01_0404__b84235270614441">Size</strong>: A file size is assigned to a map task by the total file size. The calculation formula is <span class="parmvalue" id="mrs_01_0404__parmvalue244487834"><b>Total file size/Extractors</b></span>.</li></ul>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_0404__sd84e31a43f93483d8f37f23ccdbb3c1e"><h4 class="sectiontitle">generic-jdbc-connector</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0404__tf66599c326bf46bb841b6a108b17321e" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Data source link properties of <strong id="mrs_01_0404__b1757259257">generic-jdbc-connector</strong></caption><thead align="left"><tr id="mrs_01_0404__rd407a71a8db34b509b61a8d99dbc56bb"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.3.2.2.3.1.1"><p id="mrs_01_0404__a43d562c8796944ce94ea7e5a8b7e27da">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75%" id="mcps1.3.3.2.2.3.1.2"><p id="mrs_01_0404__a792bdea57e474891b6489f864801ed05">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0404__r3cbc3267179045eea5bcb44d9260a94f"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_0404__a2cc54c2a2da242c794767cfcfafbc346">Schema/Tablespace</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_0404__a99e448b09d484d3f89d2d391ac5eeac3">Name of the database storing source data. You can query and select it on the interface.</p>
</td>
</tr>
<tr id="mrs_01_0404__r6e9c2a3e9d1e4dfc93132e217e84ea7d"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_0404__a8fa190bea6bf4cf0bf8e086bbe79840e">Table Name</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_0404__abaaa2b7a360a4569a8b0f856f0cf02e5">Data table storing the source data. You can query and select it on the interface.</p>
</td>
</tr>
<tr id="mrs_01_0404__ra31516f8d3de41d2b2e350d4eb3b740c"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_0404__a8bb598fba7764f55ae931fa979fedd42">Partition Column</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_0404__a9f254e46e7504c0e984e8fdd87e4e12e">If multiple columns need to be read, use this column to split the result and obtain data.</p>
</td>
</tr>
<tr id="mrs_01_0404__r58dad8320ec34e29b3280d73bd844969"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.3.2.2.3.1.1 "><p id="mrs_01_0404__a48e8ee41ce2d4010ab5976b1c7bf5260">Where Clause</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.3.2.2.3.1.2 "><p id="mrs_01_0404__aaa8d4660374743a492dc69465e398c6a">Query statement used when accessing the database</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_0404__s033d5edc10164032b9ea23d01081beae"><a name="mrs_01_0404__s033d5edc10164032b9ea23d01081beae"></a><a name="s033d5edc10164032b9ea23d01081beae"></a><h4 class="sectiontitle">ftp-connector or sftp-connector</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0404__t48b1b68e9563446d8689be9e4db828e0" frame="border" border="1" rules="all"><caption><b>Table 3 </b>Data source link properties of <strong id="mrs_01_0404__b84235270694830">ftp-connector</strong> or <strong id="mrs_01_0404__b84235270694827">sftp-connector</strong></caption><thead align="left"><tr id="mrs_01_0404__rbac17e98fe924d9c9170415379a10b36"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.4.2.2.3.1.1"><p id="mrs_01_0404__a67a7341ae37a4161b65189b65b8fb3d8">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75%" id="mcps1.3.4.2.2.3.1.2"><p id="mrs_01_0404__ad08f3f4f9f924eae8e3c405db1a05f45">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0404__ra42b9b35b6b741ac978f2dfe198c8651"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.4.2.2.3.1.1 "><p id="mrs_01_0404__a36540afb5ffa49de8aa79acd1f597bfb">Source Directory/File</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.4.2.2.3.1.2 "><p id="mrs_01_0404__ad6f49542380a44adac6e02a34e5d2149">Actual storage form of source data. It can be either all data files in a directory or single data file contained in the file server.</p>
</td>
</tr>
<tr id="mrs_01_0404__rd80060b048364dea95f430ef620c1045"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.4.2.2.3.1.1 "><p id="mrs_01_0404__a464198f64d3b4357a725fe95d6b32545">File Format</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.4.2.2.3.1.2 "><p id="mrs_01_0404__aa6bfc9fcb4fd4459a74516700b94b1f5">Loader supports the following file formats of data stored in the file server:</p>
<ul id="mrs_01_0404__uc1fb4b67fa7a40b8802fe069c1c4a5e1"><li id="mrs_01_0404__lc9a70e726d2f400d8f8b4be8166c479c"><strong id="mrs_01_0404__b1175892062716">CSV_FILE</strong>: Specifies a text file. When the destination link is a database link, only the text file is supported.</li><li id="mrs_01_0404__l49b160f7a70746c8b6c2f2af11bff4ca"><strong id="mrs_01_0404__b147022312712">BINARY_FILE</strong>: Specifies binary files excluding text files.</li></ul>
</td>
</tr>
<tr id="mrs_01_0404__r6075ad2b6e7e4aaf895d7d93651a40d3"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.4.2.2.3.1.1 "><p id="mrs_01_0404__a39cbaf2d9d17449d9e1834255d231318">Line Separator</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.4.2.2.3.1.2 "><p id="mrs_01_0404__a9dc23093b3b54e709a62c2aee1b183b7">Identifier of each line end of source data</p>
<div class="note" id="mrs_01_0404__n14452d7c4e10432f96e42859bb7b0860"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0404__ad46fbccebc4d439eaf1eeb910b99b781">If FTP or SFTP serves as a source link and <strong id="mrs_01_0404__b14494105062712">File Format</strong> is set to <strong id="mrs_01_0404__b82297592276">BINARY_FILE</strong>, the value of <strong id="mrs_01_0404__b172391728145410">Line Separator</strong> in the advanced properties is invalid.</p>
</div></div>
</td>
</tr>
<tr id="mrs_01_0404__r9461de09b93c48248b8fb56358b3fb03"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.4.2.2.3.1.1 "><p id="mrs_01_0404__ac9f0f153056148faa8df41590deaa5a2">Field Separator</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.4.2.2.3.1.2 "><p id="mrs_01_0404__aec1a8685dd7d4d409a65df82132a7897">Identifier of each field end of source data</p>
<div class="note" id="mrs_01_0404__n3a78d39b9ff241e2ad62dac2e8d81a63"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0404__a8760c2ab5b794168a5a39db4e3a7622a">If FTP or SFTP serves as a source link and <strong id="mrs_01_0404__b1484135213545">File Format</strong> is set to <strong id="mrs_01_0404__b984845213546">BINARY_FILE</strong>, the value of <strong id="mrs_01_0404__b198491652145413">Field Separator</strong> in the advanced properties is invalid.</p>
</div></div>
</td>
</tr>
<tr id="mrs_01_0404__re66c91fd98c74fe6b4390b4125d9a09d"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.4.2.2.3.1.1 "><p id="mrs_01_0404__ab5055adc799e472d92b78746524524ec">Encoding Type</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.4.2.2.3.1.2 "><p id="mrs_01_0404__aea6207e7e660498b8eba02c82dfc8b98">Text encoding type of source data. It takes effect on text files only.</p>
</td>
</tr>
<tr id="mrs_01_0404__rce09f49b2e224a7a90d3b485dffa0054"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.4.2.2.3.1.1 "><p id="mrs_01_0404__ae61c499e201f4ecf9ea8ff99281d6d1e">File Split Type</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.4.2.2.3.1.2 "><div class="p" id="mrs_01_0404__acc3a803d73fa45559033f392c885fbab">The following types are supported:<ul id="mrs_01_0404__ub3e19f90832e4366a4d2c81f8ebb0431"><li id="mrs_01_0404__l9f52def2a6544848823ba7c0ebb4939c"><strong id="mrs_01_0404__b399610252308">File</strong>: The number of files is assigned to a map task by the total number of files. The calculation formula is <span class="parmvalue" id="mrs_01_0404__parmvalue14252633017"><b>Total number of files/Extractors</b></span>.</li><li id="mrs_01_0404__ld1c13aa7c64c4302a6a4e3ec6d6ef998"><strong id="mrs_01_0404__b164851027153015">Size</strong>: A file size is assigned to a map task by the total file size. The calculation formula is <span class="parmvalue" id="mrs_01_0404__parmvalue54851827113013"><b>Total file size/Extractors</b></span>.</li></ul>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_0404__s14d5b5fb6b6d4767847a0b8ea25b3da7"><h4 class="sectiontitle">hbase-connector</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0404__t4b650ca517a146df971786eaa4af2ad2" frame="border" border="1" rules="all"><caption><b>Table 4 </b>Data source link properties of <strong id="mrs_01_0404__b663692564">hbase-connector</strong></caption><thead align="left"><tr id="mrs_01_0404__r560adc7480434d588126b2a821b29d87"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.5.2.2.3.1.1"><p id="mrs_01_0404__ac373b4206d484352a38b18378af82b7a">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75%" id="mcps1.3.5.2.2.3.1.2"><p id="mrs_01_0404__adf019a29b0b14801b6992a81625c917d">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0404__r3cca16159e92414f87fc8a3cdaa2b3ba"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.5.2.2.3.1.1 "><p id="mrs_01_0404__a7b07040abad3438bad3550996b27ee5c">Table Name</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.5.2.2.3.1.2 "><p id="mrs_01_0404__ac0d12bdb2af548ad8f59e896260085ef">HBase table storing source data</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_0404__sf7d0abeee0644e4396f4d4666ba879f3"><h4 class="sectiontitle">hdfs-connector</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0404__tb77070175c9049beaeb9b100dc5df454" frame="border" border="1" rules="all"><caption><b>Table 5 </b>Data source link properties of <strong id="mrs_01_0404__b14975546">hdfs-connector</strong></caption><thead align="left"><tr id="mrs_01_0404__r7558ba2fe7dc4b14a76a7c98a7cf05ae"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.6.2.2.3.1.1"><p id="mrs_01_0404__a2dc201da49a941eca588fb1da4e4165d">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75%" id="mcps1.3.6.2.2.3.1.2"><p id="mrs_01_0404__a67507f5c293842699f1f33667113faa4">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0404__r7867d379e7734023b4798166ce8a1b76"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.6.2.2.3.1.1 "><p id="mrs_01_0404__en-us_topic_0071084972_p385045217275">Source Directory/File</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.6.2.2.3.1.2 "><p id="mrs_01_0404__ad29a9fced04f418ba257fcff25a3e2ce">Actual storage form of source data. It can be either all data files in a directory or single data file contained in HDFS.</p>
</td>
</tr>
<tr id="mrs_01_0404__red833ae7f38d4f1cab5c68222dc7d048"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.6.2.2.3.1.1 "><p id="mrs_01_0404__a4e6624234a364bbcbe6b3b52332ae441">File Format</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.6.2.2.3.1.2 "><p id="mrs_01_0404__a4fff3fc891ba44e2aac6847f32f2ad82">Loader supports the following file formats of data stored in HDFS:</p>
<ul id="mrs_01_0404__uacba7d5c84d84514ba2c61eaa36805a4"><li id="mrs_01_0404__l4f1702bdcecc4ada8fbd153c2bad40b0"><strong id="mrs_01_0404__b537112611317">CSV_FILE</strong>: Specifies a text file. When the destination link is a database link, only the text file is supported.</li><li id="mrs_01_0404__lcf74adb0c0f6483daf20d433cba86fcd"><strong id="mrs_01_0404__b1832193063119">BINARY_FILE</strong>: Specifies binary files excluding text files.</li></ul>
</td>
</tr>
<tr id="mrs_01_0404__r0dd938cd29d24e958b3bfc840a28dab3"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.6.2.2.3.1.1 "><p id="mrs_01_0404__a661a6871ff7d42e1ade037600f140172">Line Separator</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.6.2.2.3.1.2 "><p id="mrs_01_0404__a6c6b3edaa22f4fd79591332f6483b11b">Identifier of each line end of source data</p>
<div class="note" id="mrs_01_0404__n891b951ef4c7493a9b9025e9ed2e817c"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0404__ade7060074ac149d6b337f1ff8d92e899">If HDFS serves as a source link and <strong id="mrs_01_0404__b29190243157">File Format</strong> is set to <strong id="mrs_01_0404__b14919172416157">BINARY_FILE</strong>, the value of <strong id="mrs_01_0404__b8920224101520">Line Separator</strong> in the advanced properties is invalid.</p>
</div></div>
</td>
</tr>
<tr id="mrs_01_0404__r9ec8ef25ff2647dc8465aefc2a075c4a"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.6.2.2.3.1.1 "><p id="mrs_01_0404__a34ba003afd8e4c52bc3d8ae1888172b1">Field Separator</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.6.2.2.3.1.2 "><p id="mrs_01_0404__ae8acbfb210504a93a6fb4dc368887d05">Identifier of each field end of source data</p>
<div class="note" id="mrs_01_0404__n84553400b67d4b7baa520df16c9a5079"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="mrs_01_0404__ae27fd36befe640fe89bb27bdfbc88ea8">If HDFS serves as a source link and <strong id="mrs_01_0404__b1494919110564">File Format</strong> is set to <strong id="mrs_01_0404__b9954712564">BINARY_FILE</strong>, the value of <strong id="mrs_01_0404__b1295416111562">Field Separator</strong> in the advanced properties is invalid.</p>
</div></div>
</td>
</tr>
<tr id="mrs_01_0404__r89e2cb7caae045f5952e9fb06d16423e"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.6.2.2.3.1.1 "><p id="mrs_01_0404__a4d90c32d837140c5bcb8355995d227cd">File Split Type</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.6.2.2.3.1.2 "><div class="p" id="mrs_01_0404__ad91cb084f4fd4c199075a3310e9e63d3">The following types are supported:<ul id="mrs_01_0404__en-us_topic_0071084972_en-us_topic_0071084972_ul200684591785"><li id="mrs_01_0404__en-us_topic_0071084972_en-us_topic_0071084972_li94552341785"><strong id="mrs_01_0404__b17787161711332">File</strong>: The number of files is assigned to a map task by the total number of files. The calculation formula is <span class="parmvalue" id="mrs_01_0404__parmvalue979351793314"><b>Total number of files/Extractors</b></span>.</li><li id="mrs_01_0404__en-us_topic_0071084972_en-us_topic_0071084972_li22998232171132"><strong id="mrs_01_0404__b15331202333">Size</strong>: A file size is assigned to a map task by the total file size. The calculation formula is <span class="parmvalue" id="mrs_01_0404__parmvalue4533220123315"><b>Total file size/Extractors</b></span>.</li></ul>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="section" id="mrs_01_0404__s792acd5598e84c68bd801ad8f80e728c"><h4 class="sectiontitle">hive-connector</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0404__t907d02227ef5419eb1d3871e25546a5c" frame="border" border="1" rules="all"><caption><b>Table 6 </b>Data source link properties of <strong id="mrs_01_0404__b694172477">hive-connector</strong></caption><thead align="left"><tr id="mrs_01_0404__r5c7b95c178894c9d9e546ce6fc22c3bc"><th align="left" class="cellrowborder" valign="top" width="25%" id="mcps1.3.7.2.2.3.1.1"><p id="mrs_01_0404__a26825152cbe548bea9d6598ced7a9565">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="75%" id="mcps1.3.7.2.2.3.1.2"><p id="mrs_01_0404__aba7826d72809468dbe05dc94c62a9a50">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="mrs_01_0404__rdf7f60b2307a4e4eaa7e8d2b24e9f906"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.7.2.2.3.1.1 "><p id="mrs_01_0404__ae2bd3f3c804148abb83107bafd536a38">Database Name</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.7.2.2.3.1.2 "><p id="mrs_01_0404__aa18a4f696ab14f8d870e5402be551b02">Name of the Hive database storing the data source. You can query and select it on the interface.</p>
</td>
</tr>
<tr id="mrs_01_0404__ra1f979146752439cbbde30a256fac209"><td class="cellrowborder" valign="top" width="25%" headers="mcps1.3.7.2.2.3.1.1 "><p id="mrs_01_0404__a2b51c5d42b24478292f0f13d42bc7e25">Table</p>
</td>
<td class="cellrowborder" valign="top" width="75%" headers="mcps1.3.7.2.2.3.1.2 "><p id="mrs_01_0404__a1bc7c9d129444cf3995d9487fe10eb3c">Name of the Hive table storing the data source. You can query and select it on the interface.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0400.html">Using Loader</a></div>
</div>
</div>
View Git Blame Copy Permalink