forked from docs/doc-exports
Yang, Tong
3f5759eed2
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
126 lines
22 KiB
HTML
126 lines
22 KiB
HTML
<a name="mrs_01_1154"></a><a name="mrs_01_1154"></a>
|
|
|
|
<h1 class="topictitle1">Operator Data Processing Rules</h1>
|
|
<div id="body8662426"><p id="mrs_01_1154__en-us_topic_0000001219149647_a884a191796694b348cc64f32c57e524e">In Loader data import and export tasks, each operator defines different processing rules for null values and empty strings in raw data. Dirty data cannot be imported or exported.</p>
|
|
<div class="p" id="mrs_01_1154__en-us_topic_0000001219149647_a39e17d61fbc6455bb4cb9cb9cb894aa3">The following table describes the operator data processing rules for each conversion procedure.
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1154__en-us_topic_0000001219149647_tce9979184ac043f6afd8e1049d411ed1" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Data processing rules</caption><thead align="left"><tr id="mrs_01_1154__en-us_topic_0000001219149647_r937e0746ba64466fa0f76ac55ed66eff"><th align="left" class="cellrowborder" valign="top" width="22%" id="mcps1.3.2.1.2.3.1.1"><p id="mrs_01_1154__en-us_topic_0000001219149647_a957d4cc5d1ff40fc857273230cfed10a">Procedure</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="78%" id="mcps1.3.2.1.2.3.1.2"><p id="mrs_01_1154__en-us_topic_0000001219149647_a30e7ad68b71f48f891641cc51aac2700">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1154__en-us_topic_0000001219149647_re405a11b7c544739834d9ddd9fcc55d5"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a5d9ce7066cd54be1ba43da416ac9a462">CSV file input</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_uc7c620cb8f71497f8202ac76052322ce"><li id="mrs_01_1154__en-us_topic_0000001219149647_l98cec08667b143be9c759fbc778f98fc">If a delimiter appears twice consecutively in the original data, an empty string field is generated.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_le30d277a1aad4c30856edde18b62e0cc">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l706131c04c0f496caa6c103677cbbb98">If a type conversion error occurs, the current data is saved as dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r6e7d3f55e0b743f59137a3b7d680894d"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_aec1aa0b0e0324a38b25f964c8d1439ec">Fixed file input</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u35a5aeca6f0746b48a3e923d2991b507"><li id="mrs_01_1154__en-us_topic_0000001219149647_l23304cb70800404483841fa712a09697">If the original data includes null values, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lf5e93bb25e6c453e9994e2e7362d7dc9">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l2c3415dad2e34bb381bbfc3574d7555b">If the configured field conversion type is different from the actual type of the original data, all data becomes dirty data. For example, convert the string type to the numeric type.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lc2dc182e90ed4b5abfa1fa8b3201e46e">If the configured field split length is greater than the length of the original field value, the data split fails and the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rb01a415ec71c4afa9b33f1818ba3272b"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a4b201e31166446e7bd3de16a24c3bb65">Table input</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u2f147ae569d04803a34ec266986c7dba"><li id="mrs_01_1154__en-us_topic_0000001219149647_l77ee63c75b56422dba3880c9975ec042">If the original data includes null values, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l98872d4b1d5c4c799073730e6aa68ffd">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l40adeb45f3c94dbc93c77d06c64ccc9a">If the configured field conversion type is different from the actual type of the original data, all data becomes dirty data. For example, convert the string type to the numeric type.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r63d1183e86504a729a7a6d8dfc6e8c61"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_abc4a2147bdc743db920b5f1a68a71229">HBase input</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u5b8fcd7d0a494c0b9fe5c28cd7bb5e9d"><li id="mrs_01_1154__en-us_topic_0000001219149647_l6c38d35ccfc1497ab52c293009c273bd">If the original data includes null values, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lc458981e8edd432eabbd695d22f4b1dd">If the HBase table name is incorrect, all data becomes dirty data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l39435909e31e49c4889829e922cb6e5b">If the primary key column is not configured in <strong id="mrs_01_1154__en-us_topic_0000001219149647_b96321588935143">Is rowkey</strong>, all data becomes dirty data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_ld0e7f0deb1484bbd968427b58275c353">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l76b85b5858b546068b5b6fc8290fad89">If the configured field conversion type is different from the actual type of the original data, all data becomes dirty data. For example, convert the string type to the numeric type.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r147085c77a414042b82c87883f1ad0c4"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a8c0663558ea74e15b2a1f975935475ed">Long integer time conversion</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ub9b8038239ee4ec9b8e9d6edb5dc3f86"><li id="mrs_01_1154__en-us_topic_0000001219149647_l82f9d38cb2c949a1a9ee171c6bf13a2f">If the original data includes null values, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l783ec27bbd1f4319a54ff633ef3dd379">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l5c3aae8f45cd4e2b88fdd7668f7efffe">If a type conversion error occurs, the current data is saved as dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r095e05beb5d64c348e0ac6451e14342c"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_ae53f6a4e36be48b19befcdfb53a83594">Null value conversion</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_uad9632ceecb6421fa54cf39eb652a301"><li id="mrs_01_1154__en-us_topic_0000001219149647_l105dbdfe2709494093b4fc5cd8ae69a2">If the original data contains null values, data is converted to a specified value.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l555128bc0aea45698f2dd9d26e4dac19">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rb16adc6d1f3141c386b61ecece21bd0a"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_aa47dcd39f9944e2d9bd1d32b74c87b30">Random value conversion</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a21172b1e6d814325b017ffb7dfb6e242">Processing of null value and empty string is not involved, and dirty data is not generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_ra99a74d5830f4c4e845521c246689662"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_aadd924931b3c407ba74cad2dfe262364">Constant field addition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><p id="mrs_01_1154__en-us_topic_0000001219149647_p845881041920">Processing of null value and empty string is not involved, and dirty data is not generated.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r97cec1ed1cff40d2b461f86808f293e4"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a9154e0d506ee47e8be5379bf79196413">Concat fields</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ued4e70ca520049da8c945fb73e734325"><li id="mrs_01_1154__en-us_topic_0000001219149647_l14b77861678a410bac13f661a132333c">If the original data contains null values, data is converted to empty string.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l321059376d1042b99d40651560353252">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rfc7479ae754d41b9a6d5cb7abb4657cf"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a84efdb566cbd40a891f1cdaa8b473642">Extracts fields</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_uf6f0246233aa4381b38acf49ae8a6541"><li id="mrs_01_1154__en-us_topic_0000001219149647_l96a8f58da16743ceb0f897f82caad29d">If the original data contains null values, the current line becomes dirty data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l35b4a7f32eff4217a1210a8163a70aba">If the number of field columns after separation is greater than the actual number allowed by the original data, the line will become dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rd4678193f186407aaf8891df85a1321c"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_ac2b267a77a9d4b9fbf334c1ce06af33b">Modulo integer</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ue72a57d15fdd44fda029db1707d3f32b"><li id="mrs_01_1154__en-us_topic_0000001219149647_l488c0c14abcd4fca9557f69ec97839e6">If the original data contains null values, the current line becomes dirty data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lb6c4c242048840e89e56cda897dba41c">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lbc51d434e5814224961052a0ee97f540">If data type conversion fails, the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rce6b492bd08e4123a9779973b3bf9c29"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_aa1c3747b79f54999a85a5d68aecc317e">String cut</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u381adbefec5147359e2570c1cd602f52"><li id="mrs_01_1154__en-us_topic_0000001219149647_ld13d567db2d94923b6e3b009e8e389a6">If the input data is null, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l12e1f47364914c3cacadfb536b2acc5a">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l31987dd88a264bc7801f3f4e8c77fab0">If the start or end position of the string to be truncated is greater than the length of the input field, the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r5cb272157a6448de856cd18e3c1b559f"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a2faadf65818645d9b496f48bf9f32711">EL operation</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u0ad5a91b20b440bea727b3c084c60f3a"><li id="mrs_01_1154__en-us_topic_0000001219149647_la7fa1930371847da97221b79db523901">If the input data is null, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l07cbbea11223444596e3885b45d3b3b0">Enter the value of one or more fields and output the calculation result.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l3070c6bc94734b19a5000fa606c58347">When the input type is incompatible with the operator, the current row is dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_re082fd917a7648f284bf1da15f51a965"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_ad8c462a9da2948c5be454c15201cd5cc">String case conversion</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ufbb17bd5f1d4419da61a6cd9110d9bdd"><li id="mrs_01_1154__en-us_topic_0000001219149647_l79bcf96b2d3244c6b4edf8d1f6544804">If the input data is null, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l865174fa2f2441a8bcb586223afe3640">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r9894cafa02cc49ea814b9a4db8794dbe"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_af0ce3b593a034f4597a2f8f222cfbe39">String reverse</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ul117906232014"><li id="mrs_01_1154__en-us_topic_0000001219149647_li579022122010">If the input data is null, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_li107901826201">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r4e0619719f94458396a0c856d48f1f2b"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a6619f7609bd7477ea4354ed50bc3f325">String trim</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ul1667910492013"><li id="mrs_01_1154__en-us_topic_0000001219149647_li167944202016">If the input data is null, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_li1567916492011">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rc8d12ff0e9f04fb38830a61513caedb6"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_ad4c709fd4ef242f088b1d41feef02b5f">Filter rows</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_ua837b569ed264206b184ff4ca9e841db"><li id="mrs_01_1154__en-us_topic_0000001219149647_l9a27e41dcbd5478da6405eab7b1be104">When the condition logic is <strong id="mrs_01_1154__en-us_topic_0000001219149647_b1345915314310">AND</strong>, if no filter condition is added, all data becomes dirty data; if the original data meets all the added filter conditions, the current line becomes dirty data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_ld3aa7b55235b486cacffc49a8ab3cf47">When the condition logic is <strong id="mrs_01_1154__en-us_topic_0000001219149647_b147745866935143">OR</strong>, if no filter condition is added, all data becomes dirty data; if the original data meets all the added filter conditions, the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_re6167cbb2c2d4bc5ac90028ca54838cc"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_af46ecdace28f434882c06d573bac7cd4">File output</p>
|
|
</td>
|
|
<td class="cellrowborder" rowspan="2" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u7eceb4c1681b4968b946ab8c8524e6b2"><li id="mrs_01_1154__en-us_topic_0000001219149647_l5c34a1f96d0b4474a2331f021106f832">If the input data is null, no conversion is performed.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lb1985f49530c4140a71c0fb44a4c5134">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lb636783c22b447c9b7a03ed5be30c38b">If data type conversion fails, the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_ree9e874b63b84b77b906912bc9700bc9"><td class="cellrowborder" valign="top" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_adafee6edec444aca8db2b9a3f6896b03">Table output </p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_rc3e378657e874d3bb79b618e8435a259"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_aadbcc55a03a644b3af967042abef9f0c">HBase output</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u928b2641bf9240fbbf683c253c371bcd"><li id="mrs_01_1154__en-us_topic_0000001219149647_l22bf7ab0fc7f4712839a243e89e42703">If the original data contains null values and <strong id="mrs_01_1154__en-us_topic_0000001219149647_b189025328935143">Store null column</strong> is set to <strong id="mrs_01_1154__en-us_topic_0000001219149647_b18997838135143">true</strong>, data is converted to empty string and saved. If <strong id="mrs_01_1154__en-us_topic_0000001219149647_b175584373335143">Store null column</strong> is set to <strong id="mrs_01_1154__en-us_topic_0000001219149647_b8036998535143">false</strong>, data will not be saved.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l33b32136cb3c4792a5e5330c03fbd1ad">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l5c8e7e7174c94014ab0e2415bf621388">If data type conversion fails, the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1154__en-us_topic_0000001219149647_r327d822e581543f9b1fd8e0c9627cd28"><td class="cellrowborder" valign="top" width="22%" headers="mcps1.3.2.1.2.3.1.1 "><p id="mrs_01_1154__en-us_topic_0000001219149647_a3bd5d560f30e4e62af94b60175cfd6cb">Hive output</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="78%" headers="mcps1.3.2.1.2.3.1.2 "><ul id="mrs_01_1154__en-us_topic_0000001219149647_u5b2ce710533a48978327e9060e20b427"><li id="mrs_01_1154__en-us_topic_0000001219149647_l25a5eba992d84d5cb13408cf9b610b48">If one or more columns are designated as partition columns, the <strong id="mrs_01_1154__en-us_topic_0000001219149647_b47707114135143">Partition Handlers</strong> feature is displayed on the <strong id="mrs_01_1154__en-us_topic_0000001219149647_b131033883735143">To </strong>page. <strong id="mrs_01_1154__en-us_topic_0000001219149647_b105709601035143">Partition Handlers</strong> specifies the number of handlers for processing data partitioning.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l8b5a99252dff4923b9f83b238a61e21d">If no column is designated as partition columns, input data does not need to be partitioned, and <span class="parmvalue" id="mrs_01_1154__en-us_topic_0000001219149647_parmvalue143079761535143"><b>Partition Handlers</b></span> is hidden by default.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_l84cb87b3d32442b6832ed67ce71d5352">It can be configured that all data becomes dirty data when the number of input field columns is greater than the number of field columns actually included in the original data.</li><li id="mrs_01_1154__en-us_topic_0000001219149647_lf1c69d03123f4283b28c81be5462a4d7">If data type conversion fails, the current line becomes dirty data.</li></ul>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1119.html">Operator Help</a></div>
|
|
</div>
|
|
</div>
|
|
|