vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/modelarts/umn/modelarts_23_0332.html
Jiang, Beibei 781e07249c ModelArts 2021430 (GA) UMN 25072022 provided by R&D (third review) 
Reviewed-by: gtema <artem.goncharov@gmail.com>
Co-authored-by: Jiang, Beibei <beibei.jiang@t-systems.com>
Co-committed-by: Jiang, Beibei <beibei.jiang@t-systems.com>
2022-09-06 10:45:57 +00:00

4.3 KiB
Raw Permalink Blame History

Uploading Data to JupyterLab

On the JupyterLab page, click Upload Files to upload a file. For details, see Uploading a File in Introduction to JupyterLab and Common Operations. If a message is displayed indicating that the size of the files to be uploaded exceeds the upper limit when uploading files to notebook instances or JupyterLab, you can upload the files to OBS and then download them to notebook instances.

Step 1: Uploading Files to OBS

Use the OBS API to upload large files because OBS Console has restrictions on the file size and quantity.

Step 2: Downloading Files from OBS to Notebook Instances

A notebook instance can be mounted to OBS or EVS as the storage location. The operation method varies depending on the instance types.

  • Downloading files to notebook instances with EVS attached
    • Read an OBS file. For example, if you read the obs://bucket_name/obs_file.txt file, the content is returned as strings.
      file_str = mox.file.read('obs://bucket_name/obs_file.txt')
      You can also open the file object and read data from it. Both methods are equivalent.
      with mox.file.File('obs://bucket_name/obs_file.txt', 'r') as f:
  file_str = f.read()
    • Use the OBS API in the ModelArts SDK to download data to notebook instances.

      If the size of a single file exceeds 5 GB, the file cannot be uploaded in this mode. Use the MoXing API to upload large files.

      Sample code:

      from modelarts.session import Session
session = Session()
session.download_data(bucket_path="/bucket-name/dir1/sdk.txt", path="/home/user/sdk/obs.txt")
  • Downloading files to notebook instances using OBS for data storage

    Upload files to the OBS path specified during notebook instance creation and synchronize the files from OBS to the notebook instances using Sync OBS.