forked from docs/doc-exports
Xiong, Chen Xiao
f2733178cc
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com> Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
149 lines
15 KiB
HTML
149 lines
15 KiB
HTML
<a name="dataartsstudio_02_0288"></a><a name="dataartsstudio_02_0288"></a>
|
|
|
|
<h1 class="topictitle1">From HTTP/HTTPS</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_02_0288__en-us_topic_0115187940_section33401108172339"><h4 class="sectiontitle">Sample JSON File</h4><pre class="screen" id="dataartsstudio_02_0288__en-us_topic_0115187940_screen18422129192527">"from-config-values": {
|
|
"configs": [
|
|
{
|
|
"inputs": [
|
|
{
|
|
"name": "fromJobConfig.inputDirectory",
|
|
"value": "http://10.114.196.186:8080/httpfrom/symbol.txt"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.inputFormat",
|
|
"value": "BINARY_FILE"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.fromCompression",
|
|
"value": "TARGZ"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.compressedFileSuffix",
|
|
"value": "*"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.fileSeparator",
|
|
"value": "|"
|
|
}
|
|
],
|
|
"name": "fromJobConfig"
|
|
}
|
|
]
|
|
}</pre>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0288__en-us_topic_0115187940_section46589520174555"><h4 class="sectiontitle">Parameter Description</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_02_0288__en-us_topic_0115187940_table13922888141527" frame="border" border="1" rules="all"><thead align="left"><tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row229143141527"><th align="left" class="cellrowborder" valign="top" width="22.66%" id="mcps1.3.2.2.1.5.1.1"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p66756185141527">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.78%" id="mcps1.3.2.2.1.5.1.2"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p38541938141527">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="17.419999999999998%" id="mcps1.3.2.2.1.5.1.3"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p34889279141527">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="40.14%" id="mcps1.3.2.2.1.5.1.4"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p7459369141527">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row62628929141527"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p18560626141527">fromJobConfig.inputDirectory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p27015713141527">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p50167516142856">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p15904104063219">URL of the file to be extracted</p>
|
|
<p id="dataartsstudio_02_0288__en-us_topic_0115187940_p0123115172917">These connectors are used to read files with an HTTP/HTTPS URL, such as reading public files on the third-party object storage system and web disks.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row42094113141527"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p26789449141527">fromJobConfig.inputFormat</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p22461756141527">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p7462956141527">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1465533914361">File format required for data transmission. Currently, only the binary format is supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row9447745533"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p4025721311518">fromJobConfig.fromCompression</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p3960883811518">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p5419927911518">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><div class="p" id="dataartsstudio_02_0288__en-us_topic_0115187940_p2806544011518">Compression format of the source files. The options are as follows:<ul id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_ul64234801103023"><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li41242297103023"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b842352706143522">NONE</strong>: Files in all formats are transferred.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li52291733103023"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b842352706143558">GZIP</strong>: Files in gzip format are transferred.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li1713102714591"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b986486914">ZIP</strong>: Files in Zip format are transferred.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_li264515201048"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b4322172761013">TAR.GZ</strong>: Files in TAR.GZ format are transferred.</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row81878171452"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p11187517355">fromJobConfig.compressedFileSuffix</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p101873176517">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1418731718517">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p13187191714515">Extension of the files to be decompressed. The decompression operation is performed only when the file name extension is used in a batch of files. Otherwise, files are transferred in the original format. If you enter <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b137951822111514">*</strong> or leave the parameter blank, all files are decompressed.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row24617461835"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p104644610314">fromJobConfig.fileSeparator</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p38965081218">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p589710191214">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p8813175612198">File separator. When multiple files are transferred, CDM uses the file separator to separate files. The default value is <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b667111194207">|</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row882482611141"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p7825026201418">fromJobConfig.useQuery</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1482611260145">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p382620264145">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><ul id="dataartsstudio_02_0288__en-us_topic_0115187940_ul887516311069"><li id="dataartsstudio_02_0288__en-us_topic_0115187940_li9875431666">If you set this parameter to <span class="parmvalue" id="dataartsstudio_02_0288__en-us_topic_0115187940_parmvalue16178810145918"><b>true</b></span>, the name of the objects uploaded to OBS does not carry the <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b8253175811110">query</strong> parameter.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_li76907159154">If you set this parameter to <span class="parmvalue" id="dataartsstudio_02_0288__en-us_topic_0115187940_parmvalue1801559135914"><b>false</b></span>, the name of the objects uploaded to OBS carries the <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b881520423">query</strong> parameter.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row1844163653611"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p5622115931110">fromJobConfig.decryption</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p10622125919118">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1762214595111">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><div class="p" id="dataartsstudio_02_0288__en-us_topic_0115187940_p268631112373">Whether to decrypt the encrypted file before export and the decryption method. The options are as follows:<ul id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_ul193603764111"><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li443953725020"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_b9211811104110">NONE</strong>: Do not decrypt but directly export the file.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li599585720411"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_b15585121604110">AES-256-GCM</strong>: Use the AES-256-GCM (NoPadding) algorithm to decrypt the file and then export the file.</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row163413714362"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p167122217132">fromJobConfig.dek</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p74741127142115">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p2474627102111">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p23281415172211">Data decryption key. The key is a string of 64-bit hexadecimal numbers and must be the same as the data encryption key <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_b11865175422817">toJobConfig.dek</strong> configured during encryption. If the encryption and decryption keys are inconsistent, the system does not report an exception, but the decrypted data is incorrect.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row259514378360"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p26851918151311">fromJobConfig.iv</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1114152632113">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1414162617212">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p24771354512">Initialization vector required for decryption. The initialization vector is a string of 32-bit hexadecimal numbers and must be the same as the initialization vector <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_b9484124621918">toJobConfig.iv</strong> configured during encryption. If the initialization vectors are inconsistent, the system does not report an exception, but the decrypted data is incorrect.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row83947524118"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1724715172115">fromJobConfig.md5FileSuffix</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p32471517121116">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1424720170114">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p72470172110">Check whether the files extracted by CDM are consistent with those in the migration source.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_02_0281.html">Source Job Parameters</a></div>
|
|
</div>
|
|
</div>
|
|
|