forked from docs/doc-exports
Lai, Weijian
6aa966a79a
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Lai, Weijian <laiweijian4@huawei.com> Co-committed-by: Lai, Weijian <laiweijian4@huawei.com>
163 lines
16 KiB
HTML
163 lines
16 KiB
HTML
<a name="EN-US_TOPIC_0000002079104377"></a><a name="EN-US_TOPIC_0000002079104377"></a>
|
|
|
|
<h1 class="topictitle1">Introduction to Importing Data from OBS</h1>
|
|
<div id="body0000001162584000"><div class="section" id="EN-US_TOPIC_0000002079104377__section6593442101711"><h4 class="sectiontitle">Import Modes</h4><p id="EN-US_TOPIC_0000002079104377__p14400184315121">You can import data from OBS through an OBS path or a manifest file.</p>
|
|
<ul id="EN-US_TOPIC_0000002079104377__ul36131558299"><li id="EN-US_TOPIC_0000002079104377__li1061413582911">OBS path: indicates that the dataset to be imported has been stored in an OBS path. In this case, select an OBS path that you can access. In addition, the directory structure in the OBS path must comply with the specifications. For details, see <a href="dataprepare-modelarts-0013.html">Specifications for Importing Data from an OBS Directory</a>. This import mode is available only for the following types of datasets: <strong id="EN-US_TOPIC_0000002079104377__b44044903292735">Image classification</strong>, <strong id="EN-US_TOPIC_0000002079104377__b208264716292735">Object detection</strong>, <strong id="EN-US_TOPIC_0000002079104377__b23001850692735">Text classification</strong>, <strong id="EN-US_TOPIC_0000002079104377__b199496413792735">Table</strong>, and <strong id="EN-US_TOPIC_0000002079104377__b177785101292735">Sound classification</strong>. For other types of datasets, data can be imported only through a manifest file.</li><li id="EN-US_TOPIC_0000002079104377__li176135514298">Manifest file: indicates that the dataset file is in the manifest format and the manifest file has been uploaded to OBS. The manifest file defines the mapping between labeling objects and content. For details about the specifications of manifest files, see <a href="dataprepare-modelarts-0015.html">Specifications for Importing a Manifest File</a>.</li></ul>
|
|
<div class="note" id="EN-US_TOPIC_0000002079104377__note179351454219"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="EN-US_TOPIC_0000002079104377__p69358541112">Before importing an object detection dataset, ensure that the labeling range of the labeling file does not exceed the size of the original image. Otherwise, the import may fail.</p>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="EN-US_TOPIC_0000002079104377__table11677122420123" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Import modes supported by datasets</caption><thead align="left"><tr id="EN-US_TOPIC_0000002079104377__row156781824161219"><th align="left" class="cellrowborder" valign="top" width="8.081676518001075%" id="mcps1.3.1.5.2.5.1.1"><p id="EN-US_TOPIC_0000002079104377__p57691931185318">Dataset Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="14.959699086512629%" id="mcps1.3.1.5.2.5.1.2"><p id="EN-US_TOPIC_0000002079104377__p86783240129">Labeling Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="38.5599140247179%" id="mcps1.3.1.5.2.5.1.3"><p id="EN-US_TOPIC_0000002079104377__p14678112421219">From an OBS Path</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="38.398710370768406%" id="mcps1.3.1.5.2.5.1.4"><p id="EN-US_TOPIC_0000002079104377__p19678202414121">From a Manifest File</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="EN-US_TOPIC_0000002079104377__row18678524201214"><td class="cellrowborder" rowspan="3" valign="top" width="8.081676518001075%" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p1033512484537">Image</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="14.959699086512629%" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p567817249129">Image classification</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.5599140247179%" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p10888637440">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p26789247128">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p179319363187">Format specifications of labeled data: <a href="dataprepare-modelarts-0013.html#EN-US_TOPIC_0000002043025328__en-us_topic_0000001194052681_section570816190577">Image Classification</a></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.398710370768406%" headers="mcps1.3.1.5.2.5.1.4 "><p id="EN-US_TOPIC_0000002079104377__p3331198204410">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p12336162015016">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p256393813199">Format specifications of labeled data: <a href="dataprepare-modelarts-0013.html#EN-US_TOPIC_0000002043025328__en-us_topic_0000001194052681_section570816190577">Image Classification</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row86781224101212"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p567852411128">Object detection</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p1379151211447">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p7822174614433">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p56161049105511">Format specifications of labeled data: <a href="dataprepare-modelarts-0013.html#EN-US_TOPIC_0000002043025328__en-us_topic_0000001194052681_section570816190577">Image Classification</a></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p1992261519449">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p9896750174319">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p14867133517199">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section1571582442114">Object Detection</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row20678182413124"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p1097420211547">Image segmentation</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p1067852491212">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1516783265611">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1753595341919">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section1571582442114">Object Detection</a></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p967822411127">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p34521339184418">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1922723401914">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section1571582442114">Object Detection</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row13678102471218"><td class="cellrowborder" rowspan="3" valign="top" width="8.081676518001075%" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p1077043115310">Audio</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="14.959699086512629%" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p883814481946">Sound classification</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.5599140247179%" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p1567892471213">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p107017819576">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p17177151111197">Follow the format specifications described in <a href="dataprepare-modelarts-0013.html#EN-US_TOPIC_0000002043025328__en-us_topic_0000001194052681_section1683314458578">Sound Classification</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.398710370768406%" headers="mcps1.3.1.5.2.5.1.4 "><p id="EN-US_TOPIC_0000002079104377__p6678924131214">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1695711339566">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p9516193291912">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section2373122922115">Sound Classification</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row567822413124"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p158393481341">Speech labeling</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p12941162183819">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p36781244122">You can import unlabeled data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p166780246127">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p7715399015">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p33611330111918">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section10586153472113">Speech Labeling</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row1667819245125"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p158399481647">Speech paragraph labeling</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p18828744183817">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p7828174453813">You can import unlabeled data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p176781424201211">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p0772011905">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p147116151175">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section1260563812219">Speech Paragraph Labeling</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row26781124181216"><td class="cellrowborder" rowspan="3" valign="top" width="8.081676518001075%" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p43915745412">Text</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="14.959699086512629%" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p484016481842">Text classification</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.5599140247179%" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p146787249122">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p747110035117">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1121815131191">Format specifications of labeled data: <a href="dataprepare-modelarts-0013.html#EN-US_TOPIC_0000002043025328__en-us_topic_0000001194052681_section163641141195713">Text Classification</a></p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.398710370768406%" headers="mcps1.3.1.5.2.5.1.4 "><p id="EN-US_TOPIC_0000002079104377__p1267872441217">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p64921144155516">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p9171151810193">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section8593163192118">Text Classification</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row1075415405137"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p97541140171312">Named entity recognition</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p250910933917">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1950919915396">You can import unlabeled data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p1875514031313">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p11701151015">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p163471320151915">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section335761812211">Named Entity Recognition</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row107554405139"><td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p284115484416">Text triplet</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p11678101116399">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p66781311103917">You can import unlabeled data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p3755840121311">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p49219171203">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1775112251913">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section29512198">Text Triplet</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row8755104019136"><td class="cellrowborder" valign="top" width="8.081676518001075%" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p1277013315535">Video</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="14.959699086512629%" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p20755540191314">Video</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.5599140247179%" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p887175115407">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p787185114014">You can import unlabeled data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.398710370768406%" headers="mcps1.3.1.5.2.5.1.4 "><p id="EN-US_TOPIC_0000002079104377__p07551840201319">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p15569199590">You can import unlabeled or labeled data.</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p253195220226">Format specifications of labeled data: <a href="dataprepare-modelarts-0015.html#EN-US_TOPIC_0000002043025324__en-us_topic_0000001148092878_section1269454020180">Video Labeling</a></p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row026613555136"><td class="cellrowborder" valign="top" width="8.081676518001075%" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p877053125314">Other</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="14.959699086512629%" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p1826675511132">Free format</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.5599140247179%" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p15266105511312">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p1340311431362">You can import unlabeled data.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.398710370768406%" headers="mcps1.3.1.5.2.5.1.4 "><p id="EN-US_TOPIC_0000002079104377__p1266555131315">-</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="EN-US_TOPIC_0000002079104377__row437864215110"><td class="cellrowborder" valign="top" width="8.081676518001075%" headers="mcps1.3.1.5.2.5.1.1 "><p id="EN-US_TOPIC_0000002079104377__p1377015318533">Tables</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="14.959699086512629%" headers="mcps1.3.1.5.2.5.1.2 "><p id="EN-US_TOPIC_0000002079104377__p12577746312">Tables</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.5599140247179%" headers="mcps1.3.1.5.2.5.1.3 "><p id="EN-US_TOPIC_0000002079104377__p1357712461714">Supported</p>
|
|
<p id="EN-US_TOPIC_0000002079104377__p35771646110">Follow the format specifications described in <a href="dataprepare-modelarts-0013.html#EN-US_TOPIC_0000002043025328__en-us_topic_0000001194052681_section118011361754">Tables</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="38.398710370768406%" headers="mcps1.3.1.5.2.5.1.4 "><p id="EN-US_TOPIC_0000002079104377__p1157717461915">-</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataprepare-modelarts-0010.html">Importing Data from OBS</a></div>
|
|
</div>
|
|
</div>
|
|
|