Changes to ma_umn from docs/doc-exports#26
Reviewed-by: Jiang, Beibei <beibei.jiang@t-systems.com> Co-authored-by: proposalbot <proposalbot@otc-service.com> Co-committed-by: proposalbot <proposalbot@otc-service.com>
BIN
umn/source/_static/images/en-us_image_0000001110761054.png
Normal file
|
After Width: | Height: | Size: 12 KiB |
|
Before Width: | Height: | Size: 165 KiB |
|
Before Width: | Height: | Size: 260 B |
|
Before Width: | Height: | Size: 375 B |
|
Before Width: | Height: | Size: 1.7 MiB |
|
Before Width: | Height: | Size: 351 B |
BIN
umn/source/_static/images/en-us_image_0000001110761582.png
Normal file
|
After Width: | Height: | Size: 3.2 KiB |
BIN
umn/source/_static/images/en-us_image_0000001110920960.png
Normal file
|
After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 462 B |
|
Before Width: | Height: | Size: 447 B |
|
Before Width: | Height: | Size: 4.2 KiB |
|
Before Width: | Height: | Size: 390 B |
|
Before Width: | Height: | Size: 395 B |
BIN
umn/source/_static/images/en-us_image_0000001156920825.png
Normal file
|
After Width: | Height: | Size: 28 KiB |
BIN
umn/source/_static/images/en-us_image_0000001156920933.png
Normal file
|
After Width: | Height: | Size: 19 KiB |
BIN
umn/source/_static/images/en-us_image_0000001156920935.png
Normal file
|
After Width: | Height: | Size: 8.4 KiB |
BIN
umn/source/_static/images/en-us_image_0000001156920939.png
Normal file
|
After Width: | Height: | Size: 7.3 KiB |
|
Before Width: | Height: | Size: 264 B |
|
Before Width: | Height: | Size: 921 KiB |
|
Before Width: | Height: | Size: 545 B |
|
Before Width: | Height: | Size: 250 B |
|
Before Width: | Height: | Size: 297 B |
|
Before Width: | Height: | Size: 3.8 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |
BIN
umn/source/_static/images/en-us_image_0000001156921451.png
Normal file
|
After Width: | Height: | Size: 11 KiB |
BIN
umn/source/_static/images/en-us_image_0000001157080841.png
Normal file
|
After Width: | Height: | Size: 8.0 KiB |
BIN
umn/source/_static/images/en-us_image_0000001157080843.png
Normal file
|
After Width: | Height: | Size: 10 KiB |
BIN
umn/source/_static/images/en-us_image_0000001157080903.png
Normal file
|
After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 259 KiB |
|
Before Width: | Height: | Size: 32 KiB |
BIN
umn/source/_static/images/en-us_image_0000001157080915.png
Normal file
|
After Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 312 B |
BIN
umn/source/_static/images/en-us_image_0000001157081267.png
Normal file
|
After Width: | Height: | Size: 12 KiB |
|
Before Width: | Height: | Size: 59 KiB |
|
Before Width: | Height: | Size: 54 KiB |
BIN
umn/source/_static/images/en-us_image_0000001278234781.png
Normal file
|
After Width: | Height: | Size: 48 KiB |
BIN
umn/source/_static/images/en-us_image_0000001281686748.png
Normal file
|
After Width: | Height: | Size: 25 KiB |
BIN
umn/source/_static/images/en-us_image_0000001290603082.png
Normal file
|
After Width: | Height: | Size: 37 KiB |
BIN
umn/source/_static/images/en-us_image_0000001340184197.png
Normal file
|
After Width: | Height: | Size: 141 KiB |
BIN
umn/source/_static/images/en-us_image_0000001340265309.png
Normal file
|
After Width: | Height: | Size: 151 KiB |
@ -24,6 +24,6 @@ Creating and Uploading a Custom Image
|
||||
|
||||
#. Purchase a cloud server or use a local host to set up the Docker environment.
|
||||
#. Obtain the basic image from the local environment.
|
||||
#. Compile a Dockerfile based on your requirements to build a custom image. For details about how to efficiently compile a Dockerfile, see .
|
||||
#. Compile a Dockerfile based on your requirements to build a custom image.
|
||||
|
||||
4. After customizing an image, upload the image to SWR by referring to .
|
||||
4. After customizing an image, upload the image to SWR by referring to "`Uploading an Image Through a Container Engine Client <https://docs.otc.t-systems.com/usermanual/swr/swr_01_0011.html>`__" in *Software Repository for Container User Guide*.
|
||||
|
||||
@ -11,7 +11,7 @@ Prerequisites
|
||||
-------------
|
||||
|
||||
- You have created a custom image package based on ModelArts specifications. For details about the specifications you need to comply with when using a custom image to create training jobs, see :ref:`Specifications for Custom Images Used for Training Jobs <modelarts_23_0217>`.
|
||||
- You have uploaded the custom image to SWR. For details, see .
|
||||
- You have uploaded the custom image to SWR. For details, see :ref:`Creating and Uploading a Custom Image <modelarts_23_0085>`.
|
||||
|
||||
Creating a Training Job
|
||||
-----------------------
|
||||
|
||||
@ -39,10 +39,6 @@ There are two import modes: **OBS path** and **Manifest file**.
|
||||
| | | |
|
||||
| | Follow the format specifications described in :ref:`Object Detection <modelarts_23_0008__en-us_topic_0170886816_section1371122614572>`. | Follow the format specifications described in :ref:`Object Detection <modelarts_23_0009__en-us_topic_0170886817_section1571582442114>`. |
|
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Image segmentation | Supported | Supported |
|
||||
| | | |
|
||||
| | Follow the format specifications described in :ref:`Image Segmentation <modelarts_23_0008__en-us_topic_0170886816_section1363851815518>`. | Follow the format specifications described in :ref:`Image Segmentation <modelarts_23_0009__en-us_topic_0170886817_section6459163044216>`. |
|
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Sound classification | Supported | Supported |
|
||||
| | | |
|
||||
| | Follow the format specifications described in :ref:`Sound Classification <modelarts_23_0008__en-us_topic_0170886816_section1683314458578>`. | Follow the format specifications described in :ref:`Sound Classification <modelarts_23_0009__en-us_topic_0170886817_section2373122922115>`. |
|
||||
@ -67,10 +63,6 @@ There are two import modes: **OBS path** and **Manifest file**.
|
||||
| | | |
|
||||
| | | Follow the format specifications described in :ref:`Text Triplet <modelarts_23_0009__en-us_topic_0170886817_section29512198>`. |
|
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Video | N/A | Supported |
|
||||
| | | |
|
||||
| | | Follow the format specifications described in :ref:`Video Labeling <modelarts_23_0009__en-us_topic_0170886817_section1269454020180>`. |
|
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| Free format | N/A | N/A |
|
||||
+---------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
|
||||
|
||||
@ -5,65 +5,6 @@
|
||||
Data Management
|
||||
===============
|
||||
|
||||
ModelArts is easy to use for users with different experience.
|
||||
|
||||
- For service developers without AI development experience, you can use ExeML of ModelArts to build AI models without coding.
|
||||
- For developers who are familiar with code compilation, debugging, and common AI engines, ModelArts provides online code compiling environments as well as AI development lifecycle that covers data preparation, model training, model management, and service deployment, helping the developers build models efficiently and quickly.
|
||||
|
||||
ExeML
|
||||
-----
|
||||
|
||||
ExeML is a customized code-free model development tool that helps users start AI application development from scratch with high flexibility. ExeML automates model design, parameter tuning and training, and model compression and deployment with the labeled data. Developers do not need to develop basic and encoding capabilities, but only to upload data and complete model training and deployment as prompted by ExeML.
|
||||
|
||||
For details about how to use ExeML, see :ref:`Introduction to ExeML <modelarts_21_0001>`.
|
||||
|
||||
AI Development Lifecycle
|
||||
------------------------
|
||||
|
||||
The AI development lifecycle on ModelArts takes developers' habits into consideration and provides a variety of engines and scenarios for developers to choose. The following describes the entire process from data preparation to service development using ModelArts.
|
||||
|
||||
.. _modelarts_23_0002__en-us_topic_0171857760_fig6769203518431:
|
||||
|
||||
.. figure:: /_static/images/en-us_image_0000001278130605.png
|
||||
:alt: **Figure 1** Process of using ModelArts
|
||||
|
||||
|
||||
**Figure 1** Process of using ModelArts
|
||||
|
||||
.. table:: **Table 1** Process description
|
||||
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| Task | Sub Task | Description | Reference |
|
||||
+==================+==================================================+==========================================================================================================================================================================================================================+==========================================================================================+
|
||||
| Data preparation | Creating a dataset | Create a dataset in ModelArts to manage and preprocess your business data. | :ref:`Creating a Dataset <modelarts_23_0004>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | Labeling data | Label and preprocess the data in your dataset based on the business logic to facilitate subsequent training. Data labeling affects the model training performance. | :ref:`Labeling Data <modelarts_23_0010>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | Publishing a dataset | After labeling data, publish the database to generate a dataset version that can be used for model training. | :ref:`Publishing a Dataset <modelarts_23_0018>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| Development | Creating a notebook instance | Create a notebook instance as the development environment. | :ref:`Creating a Notebook Instance <modelarts_23_0034>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | Compiling code | Compile code in an existing notebook to directly build a model. | :ref:`Opening a Notebook Instance <modelarts_23_0325>` |
|
||||
| | | | |
|
||||
| | | | :ref:`Common Operations on Jupyter Notebook <modelarts_23_0120>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | Exporting the **.py** file | Export the compiled training script as a **.py** file for subsequent operations, such as model training and management. | :ref:`Using the Convert to Python File Function <modelarts_23_0037>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| Model training | Creating a training job | Create a training job, upload and use the compiled training script. After the training is complete, a model is generated and stored in OBS. | :ref:`Introduction to Training Jobs <modelarts_23_0046>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | (Optional) Creating a visualization job | Create a visualization job (TensorBoard type) to view the model training process, learn about the model, and adjust and optimize the model. Currently, visualization jobs only support the MXNet and TensorFlow engines. | :ref:`Managing Visualization Jobs <modelarts_23_0050>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| Model management | Compiling inference code and configuration files | Following the model package specifications provided by ModelArts, compile inference code and configuration files for your model, and save the inference code and configuration files to the training output location. | :ref:`Model Package Specifications <modelarts_23_0090>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | Importing a model | Import the training model to ModelArts to facilitate service deployment. | :ref:`Introduction to Model Management <modelarts_23_0052>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| Model deployment | Deploying a model as a service | Deploy a model as a real-time or batch service. | - :ref:`Deploying a Model as a Real-Time Service <modelarts_23_0060>` |
|
||||
| | | | - :ref:`Deploying a Model as a Batch Service <modelarts_23_0066>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
| | Accessing the service | After the service is deployed, access the real-time service, or view the prediction result of the batch service. | - :ref:`Accessing a Real-Time Service (Token-based Authentication) <modelarts_23_0063>` |
|
||||
| | | | - :ref:`Viewing the Batch Service Prediction Result <modelarts_23_0067>` |
|
||||
+------------------+--------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
|
||||
|
||||
- :ref:`Introduction to Data Management <modelarts_23_0003>`
|
||||
- :ref:`Creating a Dataset <modelarts_23_0004>`
|
||||
- :ref:`Labeling Data <modelarts_23_0010>`
|
||||
@ -73,6 +14,7 @@ The AI development lifecycle on ModelArts takes developers' habits into consider
|
||||
- :ref:`Publishing a Dataset <modelarts_23_0018>`
|
||||
- :ref:`Deleting a Dataset <modelarts_23_0021>`
|
||||
- :ref:`Managing Dataset Versions <modelarts_23_0019>`
|
||||
- :ref:`Team Labeling <modelarts_23_0180>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
@ -87,3 +29,4 @@ The AI development lifecycle on ModelArts takes developers' habits into consider
|
||||
publishing_a_dataset
|
||||
deleting_a_dataset
|
||||
managing_dataset_versions
|
||||
team_labeling/index
|
||||
|
||||
@ -45,6 +45,8 @@ The following filter criteria are supported. You can set one or more filter crit
|
||||
- **File Name** or **Path**: Filter files by file name or file storage path.
|
||||
- **Labeled By**: Select the name of the user who performs the labeling operation.
|
||||
|
||||
.. _modelarts_23_0011__en-us_topic_0170889731_section888019266174:
|
||||
|
||||
Labeling Images (Manually)
|
||||
--------------------------
|
||||
|
||||
|
||||
@ -7,14 +7,12 @@ Labeling Data
|
||||
|
||||
- :ref:`Image Classification <modelarts_23_0011>`
|
||||
- :ref:`Object Detection <modelarts_23_0012>`
|
||||
- :ref:`Image Segmentation <modelarts_23_0345>`
|
||||
- :ref:`Text Classification <modelarts_23_0013>`
|
||||
- :ref:`Named Entity Recognition <modelarts_23_0014>`
|
||||
- :ref:`Text Triplet <modelarts_23_0211>`
|
||||
- :ref:`Sound Classification <modelarts_23_0015>`
|
||||
- :ref:`Speech Labeling <modelarts_23_0016>`
|
||||
- :ref:`Speech Paragraph Labeling <modelarts_23_0017>`
|
||||
- :ref:`Video Labeling <modelarts_23_0282>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
@ -22,11 +20,9 @@ Labeling Data
|
||||
|
||||
image_classification
|
||||
object_detection
|
||||
image_segmentation
|
||||
text_classification
|
||||
named_entity_recognition
|
||||
text_triplet
|
||||
sound_classification
|
||||
speech_labeling
|
||||
speech_paragraph_labeling
|
||||
video_labeling
|
||||
|
||||