forked from docs/doc-exports
Xiong, Chen Xiao
14a6d65e8c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com> Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
121 lines
11 KiB
HTML
121 lines
11 KiB
HTML
<a name="dataartsstudio_02_0288"></a><a name="dataartsstudio_02_0288"></a>
|
|
|
|
<h1 class="topictitle1">From HTTP/HTTPS</h1>
|
|
<div id="body8662426"><div class="section" id="dataartsstudio_02_0288__en-us_topic_0115187940_section33401108172339"><h4 class="sectiontitle">Sample JSON File</h4><pre class="screen" id="dataartsstudio_02_0288__en-us_topic_0115187940_screen18422129192527">"from-config-values": {
|
|
"configs": [
|
|
{
|
|
"inputs": [
|
|
{
|
|
"name": "fromJobConfig.inputDirectory",
|
|
"value": "http://10.114.196.186:8080/httpfrom/symbol.txt"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.inputFormat",
|
|
"value": "BINARY_FILE"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.fromCompression",
|
|
"value": "TARGZ"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.compressedFileSuffix",
|
|
"value": "*"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.fileSeparator",
|
|
"value": "|"
|
|
}
|
|
],
|
|
"name": "fromJobConfig"
|
|
}
|
|
]
|
|
}</pre>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0288__en-us_topic_0115187940_section46589520174555"><h4 class="sectiontitle">Parameter Description</h4>
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_02_0288__en-us_topic_0115187940_table13922888141527" frame="border" border="1" rules="all"><thead align="left"><tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row229143141527"><th align="left" class="cellrowborder" valign="top" width="22.66%" id="mcps1.3.2.2.1.5.1.1"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p66756185141527">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.78%" id="mcps1.3.2.2.1.5.1.2"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p38541938141527">Mandatory</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="17.419999999999998%" id="mcps1.3.2.2.1.5.1.3"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p34889279141527">Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="40.14%" id="mcps1.3.2.2.1.5.1.4"><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p7459369141527">Description</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row62628929141527"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p18560626141527">fromJobConfig.inputDirectory</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p27015713141527">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p50167516142856">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p15904104063219">URL of the file to be extracted</p>
|
|
<p id="dataartsstudio_02_0288__en-us_topic_0115187940_p0123115172917">These connectors are used to read files with an HTTP/HTTPS URL, such as reading public files on the third-party object storage system and web disks.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row42094113141527"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p26789449141527">fromJobConfig.inputFormat</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p22461756141527">Yes</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p7462956141527">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1465533914361">File format required for data transmission. Currently, only the binary format is supported.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row9447745533"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p4025721311518">fromJobConfig.fromCompression</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p3960883811518">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p5419927911518">Enumeration</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><div class="p" id="dataartsstudio_02_0288__en-us_topic_0115187940_p2806544011518">Compression format of the source files. The options are as follows:<ul id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_ul64234801103023"><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li41242297103023"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b842352706143522">NONE</strong>: Files in all formats are transferred.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li52291733103023"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b842352706143558">GZIP</strong>: Files in gzip format are transferred.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_en-us_topic_0108272831_li1713102714591"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b268666090">ZIP</strong>: Files in Zip format are transferred.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_li264515201048"><strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b4322172761013">TAR.GZ</strong>: Files in TAR.GZ format are transferred.</li></ul>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row81878171452"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p11187517355">fromJobConfig.compressedFileSuffix</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p101873176517">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1418731718517">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p13187191714515">Extension of the files to be decompressed. The decompression operation is performed only when the file name extension is used in a batch of files. Otherwise, files are transferred in the original format. If you enter <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b137951822111514">*</strong> or leave the parameter blank, all files are decompressed.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row24617461835"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p104644610314">fromJobConfig.fileSeparator</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p38965081218">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p589710191214">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p8813175612198">File separator. When multiple files are transferred, CDM uses the file separator to separate files. The default value is <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b667111194207">|</strong>.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row882482611141"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p7825026201418">fromJobConfig.useQuery</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1482611260145">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p382620264145">Boolean</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><ul id="dataartsstudio_02_0288__en-us_topic_0115187940_ul887516311069"><li id="dataartsstudio_02_0288__en-us_topic_0115187940_li9875431666">If you set this parameter to <span class="parmvalue" id="dataartsstudio_02_0288__en-us_topic_0115187940_parmvalue16178810145918"><b>true</b></span>, the name of the objects uploaded to OBS does not carry the <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b8253175811110">query</strong> parameter.</li><li id="dataartsstudio_02_0288__en-us_topic_0115187940_li76907159154">If you set this parameter to <span class="parmvalue" id="dataartsstudio_02_0288__en-us_topic_0115187940_parmvalue1801559135914"><b>false</b></span>, the name of the objects uploaded to OBS carries the <strong id="dataartsstudio_02_0288__en-us_topic_0115187940_b881520423">query</strong> parameter.</li></ul>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0288__en-us_topic_0115187940_row83947524118"><td class="cellrowborder" valign="top" width="22.66%" headers="mcps1.3.2.2.1.5.1.1 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1724715172115">fromJobConfig.md5FileSuffix</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.78%" headers="mcps1.3.2.2.1.5.1.2 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p32471517121116">No</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="17.419999999999998%" headers="mcps1.3.2.2.1.5.1.3 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p1424720170114">String</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="40.14%" headers="mcps1.3.2.2.1.5.1.4 "><p id="dataartsstudio_02_0288__en-us_topic_0115187940_p72470172110">Check whether the files extracted by CDM are consistent with those in the migration source.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="source_job_parameters.html">Source Job Parameters</a></div>
|
|
</div>
|
|
</div>
|
|
|