Multiple environments have been installed in ModelArts notebook instances, including TensorFlow. You can use pip install to install external libraries from a Jupyter notebook or terminal to facilitate use.
Assume that you want to install Shapely from a notebook instance. Follow the following instructions:
Assume that you want to install Shapely from the terminal of a notebook instance by using pip. Follow the following instructions:
/opt/conda/envs/python27_tf/bin/pip install Shapely
source /home/ma-user/anaconda3/bin/activate TensorFlow-1.13.1 pip install shapely
Table 1 lists the Python paths of the TensorFlow, MXNet, PyTorch, Caffe, Scikit-learn & XGBoost, and Spark engines in the terminal. The pip system is also installed in the same directory as the related engine. For details about the engines used by Multi-Engine notebook instances, refer to the README file.
AI Engine |
Version |
Python Path |
---|---|---|
TensorFlow |
TF-1.8.0-python2.7 |
/opt/conda/envs/python27_tf/bin/python |
TensorFlow |
TF-1.8.0-python3.6 |
/opt/conda/envs/python36_tf/bin/python |
MXNet |
MXNet-1.2.1-python2.7 |
/opt/conda/envs/python27_mxnet/bin/python |
MXNet |
MXNet-1.2.1-python3.6 |
/opt/conda/envs/python36_mxnet/bin/python |
PyTorch |
PyTorch-1.0.0-python2.7 |
/opt/conda/envs/python27_pytorch/bin/python |
PyTorch |
PyTorch-1.0.0-python3.6 |
/opt/conda/envs/python36_pytorch/bin/python |
Caffe |
Caffe-1.0.0-python2.7 |
/opt/conda/envs/python27_caffe/bin/python |
Scikit-learn & XGBoost |
ML-1.0.0-python2.7 |
/opt/notebook/anaconda2/bin/python |
Spark |
Spark-2.2.0-python2.7 |
|
Scikit-learn & XGBoost |
ML-1.0.0-python3.6 |
/opt/notebook/anaconda3/bin/python |
Spark |
Spark-2.2.0-python3.6 |
When you create a ModelArts training job, a new independent running environment is started, which is not associated with the packages installed in the Notebook environment. Therefore, add os.system('pip install xxx') to the startup code before importing the installation package.
For example, if you need to use the Shapely dependency in the training job, add the following code to the startup code:
os.system('pip install Shapely') import Shapely