vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/modelarts/sdk-ref/modelarts_04_0201.html
Lai, Weijian efd08778d3 modelarts sdk-ref version 21.430 update 
Reviewed-by: Jiang, Beibei <beibei.jiang@t-systems.com>
Reviewed-by: Rechenburg, Matthias <matthias.rechenburg@t-systems.com>
Co-authored-by: Lai, Weijian <laiweijian4@huawei.com>
Co-committed-by: Lai, Weijian <laiweijian4@huawei.com>
2023-04-28 11:36:58 +00:00

671 lines
73 KiB
HTML
Raw Blame History

<a name="modelarts_04_0201"></a><a name="modelarts_04_0201"></a>
<h1 class="topictitle1">Deploying a Real-Time Service</h1>
<div id="body8662426"><p id="modelarts_04_0201__en-us_topic_0180094086_p227012427208">Real-time service deployment covers the following aspects:</p>
<ul id="modelarts_04_0201__en-us_topic_0180094086_ul91661445122017"><li id="modelarts_04_0201__en-us_topic_0180094086_li20238165212014">Initialize a real-time service.</li><li id="modelarts_04_0201__en-us_topic_0180094086_li13149150112115">Deploy a real-time service predictor.</li><li id="modelarts_04_0201__en-us_topic_0180094086_li12166145182017">Deploy a batch service transformer.</li></ul>
<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p964517401512">The service object predictor is returned after deployment. The attributes of the service object include all functions described in this chapter.</p>
<div class="section" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_section20261580353"><h4 class="sectiontitle">Sample Code</h4><p id="modelarts_04_0201__en-us_topic_0180094086_p1756782517483">In ModelArts notebook, you do not need to enter authentication parameters for session authentication. For details about session authentication of other development environments, see <a href="modelarts_04_0123.html">Session Authentication</a>.</p>
<ul id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_ul7683131718489"><li id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li1268341712483">Method 1: Initialize the predictor that has been deployed as a real-time service.<div class="codecoloring" codetype="Python" id="modelarts_04_0201__en-us_topic_0180094086_screen530532718210"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span></pre></div></td><td class="code"><div><pre><span></span><span class="kn">from</span> <span class="nn">modelarts.session</span> <span class="kn">import</span> <span class="n">Session</span>
<span class="kn">from</span> <span class="nn">modelarts.model</span> <span class="kn">import</span> <span class="n">Predictor</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">Session</span><span class="p">()</span>
<span class="n">predictor_instance</span> <span class="o">=</span> <span class="n">Predictor</span><span class="p">(</span><span class="n">session</span><span class="p">,</span> <span class="n">service_id</span><span class="o">=</span><span class="s2">&quot;your_service_id&quot;</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
<ul id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_ul1288181835013"><li id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li5881201812504">Method 2: Deploy a real-time service predictor.<div class="codecoloring" codetype="Python" id="modelarts_04_0201__en-us_topic_0180094086_screen1742233462120"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span>
<span class="normal">19</span>
<span class="normal">20</span>
<span class="normal">21</span>
<span class="normal">22</span></pre></div></td><td class="code"><div><pre><span></span><span class="kn">from</span> <span class="nn">modelarts.session</span> <span class="kn">import</span> <span class="n">Session</span>
<span class="kn">from</span> <span class="nn">modelarts.model</span> <span class="kn">import</span> <span class="n">Model</span>
<span class="kn">from</span> <span class="nn">modelarts.config.model_config</span> <span class="kn">import</span> <span class="n">ServiceConfig</span><span class="p">,</span> <span class="n">TransformerConfig</span><span class="p">,</span> <span class="n">Schedule</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">Session</span><span class="p">()</span>
<span class="n">model_instance</span> <span class="o">=</span> <span class="n">Model</span><span class="p">(</span><span class="n">session</span><span class="p">,</span> <span class="n">model_id</span><span class="o">=</span><span class="s1">'your_model_id'</span><span class="p">)</span>
<span class="n">vpc_id</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) ID of the VPC where the real-time service instance is deployed. This parameter is left blank by default.</span>
<span class="n">subnet_network_id</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) Subnet ID. This parameter is left blank by default.</span>
<span class="n">security_group_id</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) Security group. This parameter is left blank by default.</span>
<span class="n">configs</span> <span class="o">=</span> <span class="p">[</span><span class="n">ServiceConfig</span><span class="p">(</span><span class="n">model_id</span><span class="o">=</span><span class="n">model_instance</span><span class="o">.</span><span class="n">model_id</span><span class="p">,</span>
<span class="n">weight</span><span class="o">=</span><span class="s2">&quot;100&quot;</span><span class="p">,</span>
<span class="n">instance_count</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
<span class="n">specification</span><span class="o">=</span><span class="s2">&quot;modelarts.vm.cpu.2u&quot;</span><span class="p">)]</span> <span class="c1"># For details, see specification.</span>
<span class="n">predictor_instance</span> <span class="o">=</span> <span class="n">model_instance</span><span class="o">.</span><span class="n">deploy_predictor</span><span class="p">(</span>
<span class="n">service_name</span><span class="o">=</span><span class="s2">&quot;service_predictor_name&quot;</span><span class="p">,</span>
<span class="n">infer_type</span><span class="o">=</span><span class="s2">&quot;real-time&quot;</span><span class="p">,</span>
<span class="n">vpc_id</span><span class="o">=</span><span class="n">vpc_id</span><span class="p">,</span>
<span class="n">subnet_network_id</span><span class="o">=</span><span class="n">subnet_network_id</span><span class="p">,</span>
<span class="n">security_group_id</span><span class="o">=</span><span class="n">security_group_id</span><span class="p">,</span>
<span class="n">configs</span><span class="o">=</span><span class="n">configs</span><span class="p">,</span> <span class="c1"># predictor configuration parameter. For details, see configs.</span>
<span class="n">schedule</span> <span class="o">=</span> <span class="p">[</span><span class="n">Schedule</span><span class="p">(</span><span class="n">op_type</span><span class="o">=</span><span class="s1">'stop'</span><span class="p">,</span> <span class="n">time_unit</span><span class="o">=</span><span class="s1">'HOURS'</span><span class="p">,</span> <span class="n">duration</span><span class="o">=</span><span class="mi">1</span><span class="p">)]</span> <span class="c1"># (Optional) Specify the runtime duration for a real-time service.</span>
<span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p64151923185419">The <span class="parmname" id="modelarts_04_0201__en-us_topic_0180094086_parmname134541648202116"><b>model_id</b></span> parameter specifies the model to be deployed as a real-time service. You can obtain the value by calling the API described in <a href="modelarts_04_0195.html">Obtaining the Model List</a> or from the ModelArts management console.</p>
</li></ul>
<ul id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_ul175654514519"><li id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li10567114011171"><a name="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li10567114011171"></a><a name="en-us_topic_0180094086_en-us_topic_0160619034_li10567114011171"></a>Method 3: Deploy a batch service transformer.<div class="codecoloring" codetype="Python" id="modelarts_04_0201__en-us_topic_0180094086_screen232195912110"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span></pre></div></td><td class="code"><div><pre><span></span><span class="n">vpc_id</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) ID of the VPC where the real-time service instance is deployed. This parameter is left blank by default.</span>
<span class="n">subnet_network_id</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) Subnet ID. This parameter is left blank by default.</span>
<span class="n">security_group_id</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) Security group. This parameter is left blank by default.</span>
<span class="n">transformer</span> <span class="o">=</span> <span class="n">model_instance</span><span class="o">.</span><span class="n">deploy_transformer</span><span class="p">(</span>
<span class="n">service_name</span><span class="o">=</span><span class="s2">&quot;service_transformer_name&quot;</span><span class="p">,</span>
<span class="n">infer_type</span><span class="o">=</span><span class="s2">&quot;batch&quot;</span><span class="p">,</span>
<span class="n">vpc_id</span><span class="o">=</span><span class="n">vpc_id</span><span class="p">,</span>
<span class="n">subnet_network_id</span><span class="o">=</span><span class="n">subnet_network_id</span><span class="p">,</span>
<span class="n">security_group_id</span><span class="o">=</span><span class="n">security_group_id</span><span class="p">,</span>
<span class="n">configs</span><span class="o">=</span><span class="n">configs</span> <span class="c1"># transformer configuration parameter. For details, see configs.</span>
<span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
<ul id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_ul992619266537"><li id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li17681415143919">Definition formats of the <strong id="modelarts_04_0201__en-us_topic_0180094086_b399813529310">configs</strong> parameter group used for deploying a real-time service predictor and a batch service transformer<ul id="modelarts_04_0201__en-us_topic_0180094086_ul12623551231"><li id="modelarts_04_0201__en-us_topic_0180094086_li462312542318">Deploying a real-time service predictor:<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p6238199191511"><a name="modelarts_04_0201__en-us_topic_0180094086_li462312542318"></a><a name="en-us_topic_0180094086_li462312542318"></a>The SDK provides the definition of the <strong id="modelarts_04_0201__en-us_topic_0180094086_b13844155620357">configs</strong> parameter. The type of <strong id="modelarts_04_0201__en-us_topic_0180094086_b11845105623516">configs</strong> is list, and those of the tuple objects in the list are ServiceConfig. The code is as follows:</p>
<pre class="screen" id="modelarts_04_0201__screen12415582114">configs = []
envs = {"model_name":"mxnet-model-1", "load_epoch":"0"}
service_config1 = ServiceConfig(
model_id="model_id1", # <strong id="modelarts_04_0201__b20855114214397">model_id1</strong> and <strong id="modelarts_04_0201__b126564543919">model_id2</strong> must be different versions of the same model.
weight="70",
specification="modelarts.vm.cpu.2u", # For details, see <strong id="modelarts_04_0201__b53672818406">specification</strong>.
instance_count=2,
envs=envs) # (Optional) Configure the environment variable, for example, <strong id="modelarts_04_0201__b127514318405">envs = {"model_name":"mxnet-model-1", "load_epoch":"0"}</strong>.
service_config2 = ServiceConfig(
model_id='model_id2',
weight="30",
specification="modelarts.vm.cpu.2u", # For details, see <strong id="modelarts_04_0201__b6376015124018">specification</strong>.
instance_count=2,
envs=envs) # (Optional) Configure the environment variable, for example, <strong id="modelarts_04_0201__b1626684914010">envs = {"model_name":"mxnet-model-1", "load_epoch":"0"}</strong>.
configs.append(service_config1)
configs.append(service_config2)</pre>
</li><li id="modelarts_04_0201__en-us_topic_0180094086_li18623145122320">Deploying a batch service transformer:<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p3767205418407"><a name="modelarts_04_0201__en-us_topic_0180094086_li18623145122320"></a><a name="en-us_topic_0180094086_li18623145122320"></a>The SDK provides the definition of the <strong id="modelarts_04_0201__en-us_topic_0180094086_b1312913673716">configs</strong> parameter. The type of <strong id="modelarts_04_0201__en-us_topic_0180094086_b1513016193717">configs</strong> is list, and those of the tuple objects in the list are TransformerConfig. The code is as follows:</p>
<div class="codecoloring" codetype="Python" id="modelarts_04_0201__en-us_topic_0180094086_screen647102610237"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span></pre></div></td><td class="code"><div><pre><span></span><span class="n">configs</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">mapping_rule</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># (Optional) Mapping between input parameters and CSV data</span>
<span class="n">mapping_type</span><span class="o">=</span> <span class="s2">&quot;file&quot;</span> <span class="c1"># file or CSV</span>
<span class="n">envs</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;model_name&quot;</span><span class="p">:</span><span class="s2">&quot;mxnet-model-1&quot;</span><span class="p">,</span> <span class="s2">&quot;load_epoch&quot;</span><span class="p">:</span><span class="s2">&quot;0&quot;</span><span class="p">}</span>
<span class="n">transformer_config1</span> <span class="o">=</span> <span class="n">TransformerConfig</span><span class="p">(</span>
<span class="n">model_id</span><span class="o">=</span><span class="s2">&quot;model_id&quot;</span><span class="p">,</span>
<span class="n">specification</span><span class="o">=</span><span class="s2">&quot;modelarts.vm.cpu.2u&quot;</span><span class="p">,</span> <span class="c1"># For details, see specification.</span>
<span class="n">instance_count</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span>
<span class="n">src_path</span><span class="o">=</span><span class="s2">&quot;/shp-cn4/sdk-demo/&quot;</span><span class="p">,</span> <span class="c1"># OBS path to the input of the batch task</span>
<span class="n">dest_path</span><span class="o">=</span><span class="s2">&quot;/shp-cn4/data-out/&quot;</span><span class="p">,</span> <span class="c1"># OBS path to the output of the batch task</span>
<span class="n">req_uri</span><span class="o">=</span><span class="s2">&quot;/&quot;</span><span class="p">,</span>
<span class="n">mapping_type</span><span class="o">=</span><span class="n">mapping_type</span><span class="p">,</span>
<span class="n">mapping_rule</span><span class="o">=</span><span class="n">mapping_rule</span><span class="p">,</span>
<span class="n">envs</span><span class="o">=</span><span class="n">envs</span><span class="p">)</span> <span class="c1"># (Optional) Configure the environment variable, for example, envs = {&quot;model_name&quot;:&quot;mxnet-model-1&quot;, &quot;load_epoch&quot;:&quot;0&quot;}.</span>
<span class="n">configs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">transformer_config1</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</li></ul>
</div>
<div class="section" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_section18683951918"><h4 class="sectiontitle">Parameter Description</h4>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_table16518993181628" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row41524868181628"><th align="left" class="cellrowborder" valign="top" width="15.151515151515152%" id="mcps1.3.5.2.2.5.1.1"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p30350749181639">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.121212121212121%" id="mcps1.3.5.2.2.5.1.2"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p42491604181639">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="13.131313131313133%" id="mcps1.3.5.2.2.5.1.3"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p19267891181639">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.59595959595959%" id="mcps1.3.5.2.2.5.1.4"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p17195363181639">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row6770181628"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.2.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p8780885181639">service_id</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.2.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p40163046181639">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.2.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p31981288181639">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.2.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p40347522181639">Service ID, which can be obtained from the real-time service on the ModelArts management console</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row399025208"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.2.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1992242017">session</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.2.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p49910292012">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.2.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p69922102018">Object</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.2.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1689152543">Session object. For details about the initialization method, see <a href="modelarts_04_0123.html">Session Authentication</a>.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_table37462542238" frame="border" border="1" rules="all"><caption><b>Table 2 </b>Parameters for deploying the predictor and transformer</caption><thead align="left"><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row57501354122312"><th align="left" class="cellrowborder" valign="top" width="15.151515151515152%" id="mcps1.3.5.3.2.5.1.1"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p14751165412319">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.121212121212121%" id="mcps1.3.5.3.2.5.1.2"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p575315411231">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="13.131313131313133%" id="mcps1.3.5.3.2.5.1.3"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1275435419238">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.59595959595959%" id="mcps1.3.5.3.2.5.1.4"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p10756175492314">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row4758654192310"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p47591654132311">service_name</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p17761854142313">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p8762155482314">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p15674133152710">Service name, which consists of 1 to 64 characters. It must start with a letter. Only letters, digits, hyphens (-), and underscores (_) are allowed.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row197641954112315"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p176625410236">description</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p12768115419231">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p15770145472313">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p178591840172714">Service description, which contains a maximum of 100 characters. By default, this parameter is left blank.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row8773165432319"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1477414540232">infer_type</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1977655415234">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p12777135420239">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p11246114246">Inference mode. The value can be <strong id="modelarts_04_0201__en-us_topic_0180094086_b122545191615">real-time</strong> or <strong id="modelarts_04_0201__en-us_topic_0180094086_b42675161611">batch</strong>. The default value is <strong id="modelarts_04_0201__en-us_topic_0180094086_b1229210549392">real-time</strong>.</p>
<ul id="modelarts_04_0201__en-us_topic_0180094086_ul1765415422419"><li id="modelarts_04_0201__en-us_topic_0180094086_li29781971243"><strong id="modelarts_04_0201__en-us_topic_0180094086_b13715815525">real-time</strong>: real-time service. A model is deployed as a web service and provides real-time test UI and monitoring capabilities. The service keeps running.</li><li id="modelarts_04_0201__en-us_topic_0180094086_li72091215142417"><strong id="modelarts_04_0201__en-us_topic_0180094086_b956651825212">batch</strong>: batch service. A batch service can perform inference on batch data and automatically stops after data processing is completed.</li></ul>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row13340112615168"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p23406263166">vpc_id</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p934052616164">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1340726101620">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p9565710261">ID of the VPC to which a real-time service instance is deployed. By default, this parameter is left blank. In this case, ModelArts allocates a dedicated VPC to each user, and users are isolated from each other. To access other service components in the VPC of the service instance, set this parameter to the ID of the corresponding VPC.</p>
<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p23404269166">Once a VPC is configured, it cannot be modified. When <strong id="modelarts_04_0201__en-us_topic_0180094086_b2044018525517">vpc_id</strong> and <strong id="modelarts_04_0201__en-us_topic_0180094086_b044120565516">cluster_id</strong> are configured, only the dedicated cluster parameter takes effect.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row957220334161"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p16572153315160">subnet_network_id</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p3572163313168">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p5572143391611">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p2572933181610">ID of a subnet. By default, this parameter is left blank. This parameter is mandatory when <strong id="modelarts_04_0201__en-us_topic_0180094086_b4777191413554">vpc_id</strong> is configured. Enter the network ID displayed in the subnet details on the VPC management console. A subnet provides dedicated network resources that are isolated from other networks.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row295516219164"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p8956132151619">security_group_id</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p15956122161615">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p295632112162">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1595642161617">Security group. By default, this parameter is left blank. This parameter is mandatory when <strong id="modelarts_04_0201__en-us_topic_0180094086_b660674118557">vpc_id</strong> is configured. A security group is a virtual firewall that provides secure network access control policies for service instances. A security group must contain at least one inbound rule to permit the requests whose protocol is TCP, source address is <strong id="modelarts_04_0201__en-us_topic_0180094086_b4176124317">0.0.0.0/0</strong>, and port number is <strong id="modelarts_04_0201__en-us_topic_0180094086_b71881134311">8080</strong>.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row1477965462320"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1678119545237">configs</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p12784165413231">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1378505482320"><strong id="modelarts_04_0201__en-us_topic_0180094086_b648888174318">configs</strong> parameters of <strong id="modelarts_04_0201__en-us_topic_0180094086_b448812854313">predictor</strong> and <strong id="modelarts_04_0201__en-us_topic_0180094086_b11489382431">transformer</strong></p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p171811636162510">Model running configurations</p>
<ul id="modelarts_04_0201__en-us_topic_0180094086_ul4744163702515"><li id="modelarts_04_0201__en-us_topic_0180094086_li195751241172511">When <strong id="modelarts_04_0201__en-us_topic_0180094086_b611391804319">infer_type</strong> is set to <strong id="modelarts_04_0201__en-us_topic_0180094086_b1911410188432">batch</strong>, only one model can be configured.</li><li id="modelarts_04_0201__en-us_topic_0180094086_li274411372258">When <strong id="modelarts_04_0201__en-us_topic_0180094086_b9747203016437">infer_type</strong> is set to <strong id="modelarts_04_0201__en-us_topic_0180094086_b15748153016439">real-time</strong>, you can configure multiple models and assign traffic weights based on service requirements. The version numbers of the models must be different.</li></ul>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_row1907759164415"><td class="cellrowborder" valign="top" width="15.151515151515152%" headers="mcps1.3.5.3.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_p16306103103120">schedule</p>
</td>
<td class="cellrowborder" valign="top" width="12.121212121212121%" headers="mcps1.3.5.3.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_p23067311315">No</p>
</td>
<td class="cellrowborder" valign="top" width="13.131313131313133%" headers="mcps1.3.5.3.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_p13306173173111"><strong id="modelarts_04_0201__en-us_topic_0180094086_b672485612393">schedule</strong> array</p>
</td>
<td class="cellrowborder" valign="top" width="59.59595959595959%" headers="mcps1.3.5.3.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p33061130310">Service scheduling configuration, which can be configured only for real-time services. By default, this parameter is not used. Services run for a long time. For details, see <a href="#modelarts_04_0201__en-us_topic_0180094086_table1892915349285">Table 6</a>.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_table6841887256" frame="border" border="1" rules="all"><caption><b>Table 3 </b><strong id="modelarts_04_0201__en-us_topic_0180094086_b4490640144314">configs</strong> parameters of <strong id="modelarts_04_0201__en-us_topic_0180094086_b154911940194316">predictor</strong></caption><thead align="left"><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row4846788252"><th align="left" class="cellrowborder" valign="top" width="14.85148514851485%" id="mcps1.3.5.4.2.5.1.1"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p884819812515">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.871287128712872%" id="mcps1.3.5.4.2.5.1.2"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1985013812252">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.871287128712872%" id="mcps1.3.5.4.2.5.1.3"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1852188192520">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.4059405940594%" id="mcps1.3.5.4.2.5.1.4"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p38538832514">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row1856380258"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.4.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1786010872513">model_id</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p386268152512">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1386419814252">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.4.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p9866118172511">Model ID. You can obtain the value by calling the API described in <a href="modelarts_04_0195.html">Obtaining the Model List</a> or from the ModelArts management console.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row1086798112511"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.4.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p986916812517">weight</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p687111819259">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p138731787259">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.4.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p19875118162511">Weight of traffic allocated to a model. This parameter is mandatory only when <strong id="modelarts_04_0201__en-us_topic_0180094086_b8377348144313">infer_type</strong> is set to <strong id="modelarts_04_0201__en-us_topic_0180094086_b1637884812438">real-time</strong>. The sum of multiple weights must be equal to 100. If multiple model versions are configured in a real-time service and different traffic weights are set, ModelArts continuously accesses the prediction API of the service and forwards prediction requests to the model instances of the corresponding versions based on the weights.</p>
<pre class="screen" id="modelarts_04_0201__en-us_topic_0180094086_screen1095381218159">{
"service_name": "mnist",
"description": "mnist service",
"infer_type": "real-time",
"config": [
{
"model_id": "xxxmodel-idxxx",
"weight": "70",
"specification": "modelarts.vm.cpu.2u",
"instance_count": 1,
"envs":
{
"model_name": "mxnet-model-1",
"load_epoch": "0"
}
},
{
"model_id": "xxxxxx",
"weight": "30",
"specification": "modelarts.vm.cpu.2u",
"instance_count": 1
}
]
}</pre>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row1687613842514"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.4.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1187828182513">specification</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p38801282251">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p08821882254">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.4.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p152111045125813">Resource specifications. </p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row988558102512"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.4.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1588814812519">instance_count</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p68902872512">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p18926812256">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.4.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1253075612589">Number of instances deployed in a model. The maximum number of instances is 5. To use more instances, submit a service ticket.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row208952088256"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.4.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p48972820252">envs</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1889910820252">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.4.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p119011483251">Map&lt;String, String&gt;</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.4.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p590416818251">(Optional) Environment variable key-value pair required for running a model. By default, this parameter is left blank.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_table6420928063" frame="border" border="1" rules="all"><caption><b>Table 4 </b><strong id="modelarts_04_0201__en-us_topic_0180094086_b16524850144413">configs</strong> parameters of <strong id="modelarts_04_0201__en-us_topic_0180094086_b75251250144411">transformer</strong></caption><thead align="left"><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row12420102810614"><th align="left" class="cellrowborder" valign="top" width="14.85148514851485%" id="mcps1.3.5.5.2.5.1.1"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p164282280612">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.871287128712872%" id="mcps1.3.5.5.2.5.1.2"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1442892817613">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.871287128712872%" id="mcps1.3.5.5.2.5.1.3"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p942814289610">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.4059405940594%" id="mcps1.3.5.5.2.5.1.4"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p342819281067">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row242862813615"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1042811281364">model_id</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1343542810611">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p643515281612">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p144351289614">Model ID</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row644311283615"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1444316281614">specification</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1745022815613">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p124501281565">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1192195463517">Resource flavor. Currently, <strong id="modelarts_04_0201__en-us_topic_0180094086_b279255519449">modelarts.vm.cpu.2u</strong> and <strong id="modelarts_04_0201__en-us_topic_0180094086_b20793185564415">modelarts.vm.gpu.p4</strong> are available.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row104508285618"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p645020286620">instance_count</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p104501028868">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p545910281362">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1745912811612">Number of instances deployed in a model. The value range during the closed beta test is [1, 2].</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row245911284618"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p174598287617">envs</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p194590281468">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p204591928064">Map&lt;String, String&gt;</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p14459202819615">(Optional) Environment variable key-value pair required for running a model. By default, this parameter is left blank.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row27917187920"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p27919181917">src_path</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p37921815919">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p2791818799">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p137911181899">OBS path of the input data of a batch job</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row924312220920"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p13250162218914">dest_path</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p152505229917">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p122507220919">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p647112917104">OBS path of the output data of a batch job</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row62328563176"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1823215618175">req_uri</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p16234125681718">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p18234205618176">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p15991750134316">Inference API called in a batch task, that is, the RESTful API exposed in the model image. You must select an API URL from the <strong id="modelarts_04_0201__en-us_topic_0180094086_b4602161284511">config.json</strong> file of the model for inference. If a built-in inference image of ModelArts is used, the API is displayed as <strong id="modelarts_04_0201__en-us_topic_0180094086_b860315121454">/</strong>.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row285695891717"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1085645820173">mapping_type</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p18856115816179">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p585614581175">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p18859132072910">Mapping type of the input data. The value can be <strong id="modelarts_04_0201__en-us_topic_0180094086_b474115194514">file</strong> or <strong id="modelarts_04_0201__en-us_topic_0180094086_b187551517454">csv</strong>.</p>
<ul id="modelarts_04_0201__en-us_topic_0180094086_ul9992152211292"><li id="modelarts_04_0201__en-us_topic_0180094086_li83931127102918">If you select <strong id="modelarts_04_0201__en-us_topic_0180094086_b10719421144515">file</strong>, each inference request corresponds to a file in the input data path. When this mode is used, <strong id="modelarts_04_0201__en-us_topic_0180094086_b87202021134514">req_uri</strong> of a model can have only one input parameter and the type of this parameter is <strong id="modelarts_04_0201__en-us_topic_0180094086_b972118217453">file</strong>.</li><li id="modelarts_04_0201__en-us_topic_0180094086_li1099311220293">If you select <strong id="modelarts_04_0201__en-us_topic_0180094086_b18759145316457">csv</strong>, each inference request corresponds to a row of data in the CSV file. When this mode is used, the files in the input data path can only be in CSV format and <strong id="modelarts_04_0201__en-us_topic_0180094086_b4760165324518">mapping_rule</strong> needs to be configured to map the index of each parameter in the inference request body to the CSV file.</li></ul>
<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1567374319574">The following shows how to create a batch service whose <strong id="modelarts_04_0201__en-us_topic_0180094086_b113665184617">mapping_type</strong> is set to <strong id="modelarts_04_0201__en-us_topic_0180094086_b151379564612">file</strong>:</p>
<pre class="screen" id="modelarts_04_0201__en-us_topic_0180094086_screen14511328131218">{
"service_name": "batchservicetest",
"description": "",
"infer_type": "batch",
"config": [{
"model_id": "598b913a-af3e-41ba-a1b5-bf065320f1e2",
"specification": "modelarts.vm.cpu.2u",
"instance_count": 1,
"src_path": "https://infers-data.obs.xxx.com/xgboosterdata/",
"dest_path": "https://infers-data.obs.xxx.com/output/",
"req_uri": "/",
"mapping_type": "file"
}]
}</pre>
<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p17786115845719">The following shows how to create a batch service whose <strong id="modelarts_04_0201__en-us_topic_0180094086_b192514912463">mapping_type</strong> is set to <strong id="modelarts_04_0201__en-us_topic_0180094086_b182759194610">csv</strong>:</p>
<pre class="screen" id="modelarts_04_0201__en-us_topic_0180094086_screen35651039171215">{
"service_name": "batchservicetest",
"description": "",
"infer_type": "batch",
"config": [{
"model_id": "598b913a-af3e-41ba-a1b5-bf065320f1e2",
"specification": "modelarts.vm.cpu.2u",
"instance_count": 1,
"src_path": "https://infers-data.obs.xxx.com/xgboosterdata/",
"dest_path": "https://infers-data.obs.xxx.com/output/",
"req_uri": "/",
"mapping_type": "csv",
"mapping_rule": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"req_data": {
"type": "array",
"items": [{
"type": "object",
"properties": {
"input5": {
"type": "number",
"index": 0
},
"input4": {
"type": "number",
"index": 1
},
"input3": {
"type": "number",
"index": 2
},
"input2": {
"type": "number",
"index": 3
},
"input1": {
"type": "number",
"index": 4
}
}
}]
}
}
}
}
}
}]
}</pre>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row158141118183"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.5.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p481416131817">mapping_rule</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p188141813180">No</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.5.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p178147119189">Map</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.5.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p12814018180">Mapping between input parameters and CSV data. This parameter is mandatory only when <strong id="modelarts_04_0201__en-us_topic_0180094086_b045410154469">mapping_type</strong> is set to <strong id="modelarts_04_0201__en-us_topic_0180094086_b104559158460">csv</strong>. The mapping rule is similar to the input parameter definition in the <strong id="modelarts_04_0201__en-us_topic_0180094086_b4436720184612">config.json</strong> model configuration file. You only need to configure the index parameters under each parameter of the string, number, integer, or boolean type, and the value of this parameter to the values of the index parameters in the CSV file to send an inference request. Use commas (,) to separate multiple pieces of CSV data. The values of the index parameters start from <strong id="modelarts_04_0201__en-us_topic_0180094086_b10466152895616">0</strong>. If the value of the index parameter is <strong id="modelarts_04_0201__en-us_topic_0180094086_b16771142575616">-1</strong>, ignore this parameter. For details, see the <a href="#modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li10567114011171">sample code of deploying transformer</a>.</p>
<p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p16246141718588">The format of the inference request body described in <strong id="modelarts_04_0201__en-us_topic_0180094086_b840451213472">mapping_rule</strong> is as follows:</p>
<pre class="screen" id="modelarts_04_0201__en-us_topic_0180094086_screen14553182221617">{
"data": {
"req_data": [{
"input1": 1,
"input2": 2,
"input3": 3,
"input4": 4,
"input5": 5
}]
}
}</pre>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_table19248816291" frame="border" border="1" rules="all"><caption><b>Table 5 </b>Parameters in the response to the request for deploying <strong id="modelarts_04_0201__en-us_topic_0180094086_b1540522184720">predictor</strong> and <strong id="modelarts_04_0201__en-us_topic_0180094086_b164215229473">transformer</strong></caption><thead align="left"><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row092468182915"><th align="left" class="cellrowborder" valign="top" width="15%" id="mcps1.3.5.6.2.5.1.1"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p119241181292">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12%" id="mcps1.3.5.6.2.5.1.2"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p189248810294">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="13%" id="mcps1.3.5.6.2.5.1.3"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p39246812913">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="60%" id="mcps1.3.5.6.2.5.1.4"><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p16924148142918">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_row1392418102915"><td class="cellrowborder" valign="top" width="15%" headers="mcps1.3.5.6.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p1892488192918">predictor</p>
</td>
<td class="cellrowborder" valign="top" width="12%" headers="mcps1.3.5.6.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p492438142911">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="13%" headers="mcps1.3.5.6.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p11924108162919">Predictor object</p>
</td>
<td class="cellrowborder" valign="top" width="60%" headers="mcps1.3.5.6.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_p19924118192917">Predictor object. Its attributes include all functions described in this chapter.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="tablenoborder"><a name="modelarts_04_0201__en-us_topic_0180094086_table1892915349285"></a><a name="en-us_topic_0180094086_table1892915349285"></a><table cellpadding="4" cellspacing="0" summary="" id="modelarts_04_0201__en-us_topic_0180094086_table1892915349285" frame="border" border="1" rules="all"><caption><b>Table 6 </b><strong id="modelarts_04_0201__en-us_topic_0180094086_b16912373402">schedule</strong> parameters</caption><thead align="left"><tr id="modelarts_04_0201__en-us_topic_0180094086_row9929133418283"><th align="left" class="cellrowborder" valign="top" width="14.85148514851485%" id="mcps1.3.5.7.2.5.1.1"><p id="modelarts_04_0201__en-us_topic_0180094086_p7929113442813">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.871287128712872%" id="mcps1.3.5.7.2.5.1.2"><p id="modelarts_04_0201__en-us_topic_0180094086_p49301534192811">Mandatory</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="12.871287128712872%" id="mcps1.3.5.7.2.5.1.3"><p id="modelarts_04_0201__en-us_topic_0180094086_p199305340285">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="59.4059405940594%" id="mcps1.3.5.7.2.5.1.4"><p id="modelarts_04_0201__en-us_topic_0180094086_p7930173472812">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="modelarts_04_0201__en-us_topic_0180094086_row1893015341284"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.7.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_p15930834142811">op_type</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.7.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_p19930434162810">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.7.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_p15930193412820">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.7.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p1393083414285">Scheduling type. Currently, only the value <strong id="modelarts_04_0201__en-us_topic_0180094086_b19716124117408">stop</strong> is supported.</p>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_row179301634112811"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.7.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_p1593053482815">time_unit</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.7.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_p10930133402815">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.7.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_p129301334162818">String</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.7.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p171491712114110">Scheduling time unit. The options are as follows:</p>
<ul id="modelarts_04_0201__en-us_topic_0180094086_ul205311014184113"><li id="modelarts_04_0201__en-us_topic_0180094086_li385618194119">DAYS</li><li id="modelarts_04_0201__en-us_topic_0180094086_li8387621184118">HOURS</li><li id="modelarts_04_0201__en-us_topic_0180094086_li1553171411411">MINUTES</li></ul>
</td>
</tr>
<tr id="modelarts_04_0201__en-us_topic_0180094086_row89301634162818"><td class="cellrowborder" valign="top" width="14.85148514851485%" headers="mcps1.3.5.7.2.5.1.1 "><p id="modelarts_04_0201__en-us_topic_0180094086_p39301334172816">duration</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.7.2.5.1.2 "><p id="modelarts_04_0201__en-us_topic_0180094086_p1293093442817">Yes</p>
</td>
<td class="cellrowborder" valign="top" width="12.871287128712872%" headers="mcps1.3.5.7.2.5.1.3 "><p id="modelarts_04_0201__en-us_topic_0180094086_p12930734202819">Integer</p>
</td>
<td class="cellrowborder" valign="top" width="59.4059405940594%" headers="mcps1.3.5.7.2.5.1.4 "><p id="modelarts_04_0201__en-us_topic_0180094086_p753864762914">Value that maps to the time unit. For example, if the task stops after two hours, set <strong id="modelarts_04_0201__en-us_topic_0180094086_b1468718454111">time_unit</strong> to <strong id="modelarts_04_0201__en-us_topic_0180094086_b36871540416">HOURS</strong> and <strong id="modelarts_04_0201__en-us_topic_0180094086_b1968844174117">duration</strong> to <strong id="modelarts_04_0201__en-us_topic_0180094086_b14688184204119">2</strong>.</p>
</td>
</tr>
</tbody>
</table>
</div>
<div class="note" id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_note166451649111611"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_ul17321155131915"><li id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li665292162115">Example of deploying a real-time <strong id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_b629314390238">predictor</strong> instance in the handwritten digit recognition project implemented by MXNet:<div class="codecoloring" codetype="Python" id="modelarts_04_0201__en-us_topic_0180094086_screen9192125243010"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span></pre></div></td><td class="code"><div><pre><span></span><span class="kn">from</span> <span class="nn">modelarts.session</span> <span class="kn">import</span> <span class="n">Session</span>
<span class="kn">from</span> <span class="nn">modelarts.model</span> <span class="kn">import</span> <span class="n">Model</span>
<span class="kn">from</span> <span class="nn">modelarts.config.model_config</span> <span class="kn">import</span> <span class="n">ServiceConfig</span><span class="p">,</span><span class="n">TransformerConfig</span>
<span class="n">model_instance</span> <span class="o">=</span> <span class="n">Model</span><span class="p">(</span><span class="n">session</span><span class="p">,</span> <span class="n">model_id</span> <span class="o">=</span> <span class="s2">&quot;input your model id&quot;</span><span class="p">)</span>
<span class="n">configs</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">config1</span> <span class="o">=</span> <span class="n">ServiceConfig</span><span class="p">(</span><span class="n">model_id</span><span class="o">=</span><span class="s2">&quot;input your model id&quot;</span><span class="p">,</span>
<span class="n">weight</span><span class="o">=</span><span class="s2">&quot;100&quot;</span><span class="p">,</span>
<span class="n">instance_count</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
<span class="n">specification</span><span class="o">=</span><span class="s2">&quot;modelarts.vm.cpu.2u&quot;</span><span class="p">,</span>
<span class="n">envs</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;input_data_name&quot;</span><span class="p">:</span><span class="s2">&quot;images&quot;</span><span class="p">,</span>
<span class="s2">&quot;input_data_shape&quot;</span><span class="p">:</span><span class="s2">&quot;0,1,28,28&quot;</span><span class="p">,</span>
<span class="s2">&quot;output_data_shape&quot;</span><span class="p">:</span><span class="s2">&quot;0,10&quot;</span><span class="p">})</span>
<span class="n">configs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">config1</span><span class="p">)</span>
<span class="n">predictor</span> <span class="o">=</span> <span class="n">model_instance</span><span class="o">.</span><span class="n">deploy_predictor</span><span class="p">(</span><span class="n">service_name</span><span class="o">=</span><span class="s2">&quot;DigitRecognition&quot;</span><span class="p">,</span> <span class="n">configs</span><span class="o">=</span><span class="n">configs</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</li><li id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_li1332195161911">Example of deploying a <strong id="modelarts_04_0201__en-us_topic_0180094086_en-us_topic_0160619034_b9988163452318">transformer</strong> instance (batch processing) in a handwritten digit recognition project implemented by MXNet:<div class="codecoloring" codetype="Python" id="modelarts_04_0201__en-us_topic_0180094086_screen173777815311"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span></pre></div></td><td class="code"><div><pre><span></span><span class="kn">from</span> <span class="nn">modelarts.session</span> <span class="kn">import</span> <span class="n">Session</span>
<span class="kn">from</span> <span class="nn">modelarts.model</span> <span class="kn">import</span> <span class="n">Model</span>
<span class="kn">from</span> <span class="nn">modelarts.config.model_config</span> <span class="kn">import</span> <span class="n">ServiceConfig</span><span class="p">,</span><span class="n">TransformerConfig</span>
<span class="n">model_instance</span> <span class="o">=</span> <span class="n">Model</span><span class="p">(</span><span class="n">session</span><span class="p">,</span> <span class="n">model_id</span> <span class="o">=</span> <span class="s2">&quot;input your model id&quot;</span><span class="p">)</span>
<span class="n">configs</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">config1</span> <span class="o">=</span> <span class="n">TransformerConfig</span><span class="p">(</span><span class="n">model_id</span><span class="o">=</span><span class="s2">&quot;input your model id&quot;</span><span class="p">,</span>
<span class="n">specification</span><span class="o">=</span><span class="s2">&quot;modelarts.vm.cpu.2u&quot;</span><span class="p">,</span>
<span class="n">instance_count</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
<span class="n">envs</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;input_data_name&quot;</span><span class="p">:</span><span class="s2">&quot;images&quot;</span><span class="p">,</span><span class="s2">&quot;input_data_shape&quot;</span><span class="p">:</span><span class="s2">&quot;0,1,28,28&quot;</span><span class="p">,</span><span class="s2">&quot;output_data_shape&quot;</span><span class="p">:</span><span class="s2">&quot;0,10&quot;</span><span class="p">},</span>
<span class="n">src_path</span><span class="o">=</span><span class="s2">&quot;/w0403/testdigitrecognition/inferimages/&quot;</span><span class="p">,</span>
<span class="n">dest_path</span><span class="o">=</span><span class="s2">&quot;/w0403/testdigitrecognition/&quot;</span> <span class="p">,</span>
<span class="n">req_uri</span> <span class="o">=</span> <span class="s2">&quot;/&quot;</span><span class="p">,</span>
<span class="n">mapping_type</span> <span class="o">=</span> <span class="s2">&quot;file&quot;</span><span class="p">)</span>
<span class="n">configs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">config1</span><span class="p">)</span>
<span class="n">predictor</span> <span class="o">=</span> <span class="n">model_instance</span><span class="o">.</span><span class="n">deploy_transformer</span><span class="p">(</span><span class="n">service_name</span><span class="o">=</span><span class="s2">&quot;DigitRecognition&quot;</span><span class="p">,</span> <span class="n">infer_type</span><span class="o">=</span><span class="s2">&quot;batch&quot;</span><span class="p">,</span> <span class="n">configs</span><span class="o">=</span><span class="n">configs</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div></div>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="modelarts_04_0199.html">Service Management</a></div>
</div>
</div>
View Git Blame Copy Permalink