forked from docs/doc-exports
Xiong, Chen Xiao
14a6d65e8c
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com> Co-authored-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com> Co-committed-by: Xiong, Chen Xiao <chenxiaoxiong@huawei.com>
492 lines
55 KiB
HTML
492 lines
55 KiB
HTML
<a name="dataartsstudio_02_0013"></a><a name="dataartsstudio_02_0013"></a>
|
|
|
|
<h1 class="topictitle1">Example of Using DataArts Migration APIs</h1>
|
|
<div id="en-us_topic_0000001716156281_en-us_topic_0108272822_body1516069052917"><div class="p" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_en-us_topic_0091558317_p262185013913">This section describes how to use cURL to call CDM APIs to migrate data from a local MySQL database to DWS in the cloud.<ol id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_o211811989cff42d2a71dd4166a5fa520"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li3586915127"><a href="#dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_s80117f7397924f1eaaa579614623e6ba">Obtaining a Token</a><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p535232010217">Call the API to obtain the user token, which will be put into the request header for authentication in a subsequent request.</p>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li23611324139"><a href="#dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section858213116488">Creating a CDM Cluster</a><ul id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ul593632067"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li19318326615">If you have created a CDM cluster, skip this step and directly use the ID of the created cluster.</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1857463410620">If you want to use a new cluster for migration, call the API in <a href="CreateCluster_0.html">Creating a Cluster</a> to create a CDM cluster.</li></ul>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1162917481842"><a href="#dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section1543119195119">Creating Links</a><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p13246231455">Call the API in <a href="CreateLink_0.html">Creating a Link</a> to create the MySQL and DWS links.</p>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1185132114511"><a href="#dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section07183115314">Creating a Migration Job</a><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p178084310517">Call the API in <a href="CreateJob_0.html">Creating a Job in a Specified Cluster</a> to create a job for migrating data from MySQL to DWS.</p>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li9833125516513"><a href="#dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section179778188581">Viewing Job Result</a><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p10250151020616">Call the API in <a href="StartJob_0.html">Starting a Job</a> to execute the job.</p>
|
|
</li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_s58e76c9e26c2479393009757dc08a139"><h4 class="sectiontitle">Preparing Data</h4><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1874912233513">Before calling an API, prepare the following data.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_table15616145115144" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Preparing data</caption><thead align="left"><tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row1561865131419"><th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.2.3.2.5.1.1"><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p8618135117144">Item</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="20%" id="mcps1.3.2.3.2.5.1.2"><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1761816510143">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.2.3.2.5.1.3"><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p461835171410">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="30%" id="mcps1.3.2.3.2.5.1.4"><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p16618135161410">Example</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row4618451181414"><td class="cellrowborder" rowspan="5" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p16618145115141">Account information</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p13618155110146">Project name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p13618205111410">Name of the project where CDM resides</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1961885171414">Project Name</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row11618651181417"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p961845114142">Project ID</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p126181251151419">ID of the project where CDM resides</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p5618115119144">1551c7f6c808414d8e9f3c514a170f2e</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row20618155115141"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p8618151131416">Account name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p561855118145">Name of an enterprise account to which a user belongs</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1361818518145">Account Name</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row15474132117307"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p34741021163014">Username</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p6474142113016">Username for using a cloud service. The user must have operation permissions on CDM.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p2474521183018">Username</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row8377172711303"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p43781527183018">Password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p193781527173010">User password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p337872773017">password</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row1261955191416"><td class="cellrowborder" rowspan="3" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p461935131416">VPC information</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p2619651161418">VPC ID</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p17619051131413">The VPC where CDM resides must be the same as that of DWS.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p861985181415">6b47302a-bf79-4b20-bf7a-80987408e196</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row14619451141410"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1761935141417">Subnet ID</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p16858324114117">The subnet where CDM resides must be the same as that of DWS.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p146191851111412">63bdc3cb-a4e7-486f-82ee-d9bf208c8f8c</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row9619195120147"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p11619051121419">Security group ID</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p8881152416419">The security group where CDM resides must be the same as that of DWS.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p16619135119147">005af77a-cce5-45ac-99c7-2ea50ea8addf</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row1561985101414"><td class="cellrowborder" rowspan="2" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p461913511147">Endpoint</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p461945120146">IAM endpoint</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1139145614105">An endpoint is the <strong id="dataartsstudio_02_0013__en-us_topic_0000001082451448_b176481237152913">request address</strong> for calling an API. Endpoints vary depending on services and regions. You can obtain endpoints from <a href="https://docs.otc.t-systems.com/en-us/endpoint/index.html" target="_blank" rel="noopener noreferrer">Regions and Endpoints</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p76199517141">iam_endpoint</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row12381202619205"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p8382626202015">CDM endpoint</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p115721651114712">An endpoint is the <strong id="dataartsstudio_02_0013__en-us_topic_0000001082451448_b16300195371816">request address</strong> for calling an API. Endpoints vary depending on services and regions. You can obtain endpoints of the service from <a href="https://docs.otc.t-systems.com/en-us/endpoint/index.html" target="_blank" rel="noopener noreferrer">Regions and Endpoints</a>.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1538214265202">cdm_endpoint</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row964415269207"><td class="cellrowborder" rowspan="5" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p16644182662019">MySQL database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p146445268204">IP address</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p5644126132020">IP address of the local MySQL database, which allows CDM to access the MySQL database using a public IP address</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1164416260209">1xx.120.85.24</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row16846126202013"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p10846152642018">Port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p28466263207">MySQL database port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p168461526112018">3306</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row11171271207"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p151719276205">Database name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p71752710208">Name of the MySQL database from which data is to be exported</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p143549127530">DB_name</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row161789272201"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p19179327202017">Username</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1617910275201">Username for accessing the MySQL database. The user must have the read, write, and delete permissions on the MySQL database.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1617918279204">username</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row536611275200"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p63661327122010">Password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p4366927152016">Password for accessing the MySQL database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p436612270200">DB_password</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row45171527102010"><td class="cellrowborder" rowspan="5" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p151782792012">DWS database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p118161558142516">IP address</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p18517827172013">IP address of the DWS database. CDM can access the IP address through the internal network.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="30%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p23710211545">10.120.85.24</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row3613195162518"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p108161858152514">Port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1961416522518">DWS database port</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p66141592517">3306</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row784616582517"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p68161158192512">Database name</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p58472512255">Name of the DWS database to which data is written</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1284755112513">DWS</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row13528672516"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p8816158172516">Username</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p3531763255">Username for accessing the DWS database. The user must have the read, write, and delete permissions on the DWS database.</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p175356112515">user_dws</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_row14312176132511"><td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.1 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p11816145812520">Password</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.2 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p17312666257">Password for accessing the DWS database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.2.3.2.5.1.3 "><p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1931211632516">dws_password</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_s80117f7397924f1eaaa579614623e6ba"><a name="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_s80117f7397924f1eaaa579614623e6ba"></a><a name="en-us_topic_0000001716156281_en-us_topic_0108272822_s80117f7397924f1eaaa579614623e6ba"></a><h4 class="sectiontitle">Obtaining a Token</h4><ol id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_en-us_topic_0091558317_ol06001248159"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_l0598a9cff3af494dbce5b7023e9577d9">Before calling other APIs, obtain the token and set it as an environment variable.<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_s7a0014144434479eaccfccd5913553e3">curl -H "Content-Type:application/json" https://{<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_en-us_topic_0091558317_b166555491133">iam_endpoint</strong>}/v3/auth/tokens -X POST -d '
|
|
{
|
|
"auth": {
|
|
"identity": {
|
|
"methods": [
|
|
"password"
|
|
],
|
|
"password": {
|
|
"user": {
|
|
"name": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b581516113484">Username</strong>",
|
|
"password": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b159553266436">password</strong>",
|
|
"domain": {
|
|
"name": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b101861029104816">Account Name</strong>"
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"scope": {
|
|
"project": {
|
|
"id": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b713662516421">1551c7f6c808414d8e9f3c514a170f2e</strong>"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
' -v -k</pre>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_a6921b0724a604c3687593489efc81bd9">The value of <span class="parmname" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_parmname398555921203948"><b>X-Subject-Token</b></span> in the response header is the token.</p>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_s3fecce5be9d247a4bae83b13b42b90bf"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_en-us_topic_0091558317_b753340142114">X-Subject-Token</strong>:MIIDkgYJKoZIhvcNAQcCoIIDgzCCA38CAQExDTALBglghkgBZQMEAgEwgXXXXX...</pre>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1589418554412">Run the following command to set the token as an environment variable for future use:<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen139081854447">export Token = MIIDkgYJKoZIhvcNAQcCoIIDgzCCA38CAQExDTALBglghkgBZQMEAgEwgXXXXX...</pre>
|
|
</li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section858213116488"><a name="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section858213116488"></a><a name="en-us_topic_0000001716156281_en-us_topic_0108272822_section858213116488"></a><h4 class="sectiontitle">Creating a CDM Cluster</h4><ol id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ol1897011164816"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li169531518484">Call the API in <a href="CreateCluster_0.html">Creating a Cluster</a> to create a cluster. The following values are examples:<ul id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ul095118154814"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li19951161134817">Cluster name: <span class="uicontrol" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_uicontrol198179057204037"><b>cdm-ab82</b></span></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li119511817484">Cluster flavor: <span class="uicontrol" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_uicontrol425606777204046"><b>cdm.medium</b></span></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li39511215484">The VPC, subnet, and security group are the same as those of DWS, and the EIP is automatically bound.</li></ul>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p7268516155611">If status code <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b8423527069410">200</strong> is returned, the cluster is successfully created.</p>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen139527114480">curl -X POST -H 'Content-Type:application/json;charset=utf-8' -H "X-Auth-Token:$Token" -d '
|
|
{
|
|
"cluster": {
|
|
"name": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1895111113481">cdm-ab82</strong>",
|
|
"vpcId": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b77999914113">6b47302a-bf79-4b20-bf7a-80987408e196</strong>",
|
|
"instances": [{
|
|
"flavorRef": "fb8fe666-6734-4b11-bc6c-43d11db3c745",
|
|
"nics": [{
|
|
"net-id": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b11528941151119">63bdc3cb-a4e7-486f-82ee-d9bf208c8f8c</strong>",
|
|
"securityGroupId": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b927145531115">005af77a-cce5-45ac-99c7-2ea50ea8addf</strong>"
|
|
}],
|
|
"availability_zone": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b12272143020122">Project Name</strong>",
|
|
"type": "cdm"
|
|
}],
|
|
"datastore": {
|
|
"version": "1.8.5",
|
|
"type": "cdm"
|
|
},
|
|
"isScheduleBootOff": false,
|
|
"scheduleBootTime": "null",
|
|
"scheduleOffTime": "null",
|
|
"isAutoOff": false,
|
|
"sys_tags": [{
|
|
"key": "_sys_enterprise_project_id",
|
|
"value": "1ce45885-4033-40d2-bdde-d4dbaceb387d"
|
|
}]
|
|
},
|
|
"autoRemind": false,
|
|
"phoneNum": "null",
|
|
"email": "null"
|
|
}'
|
|
https://<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b5952216487">{cdm_endpoint}</strong>/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1195217110483">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters -v -k</pre>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li295514117489">Call the API in <a href="ListClusters_0.html">Querying the Cluster List</a> to query cluster information, obtain the cluster ID, and set the cluster ID to a global variable.<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen1595318144812">curl -X GET -H 'Content-Type:application/json;charset=utf-8' -H "X-Auth-Token:$Token" https://<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b17953181184819">{</strong><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b3953911488">cdm_endpoint</strong><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b11953714486">}</strong>/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1395351104817">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters -k -v</pre>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p8953413482">The response is as follows:</p>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen10954141194810">{
|
|
"clusters": [{
|
|
"version": "x.x.x",
|
|
"updated": "2018-09-05T08:38:25",
|
|
"name": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b19547134816">cdm-ab82</strong>",
|
|
"created": "2018-09-05T08:38:25",
|
|
"id": "<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b12954101134815">bae65496-643e-47ca-84af-948672de7eeb</strong>",
|
|
"status": "200",
|
|
"isFrozen": "0",
|
|
"statusDetail": "Normal",
|
|
"actionProgress": {},
|
|
"config_status": "In-Sync"
|
|
}]
|
|
}</pre>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p295461154814">If the value of <span class="parmname" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_parmname727153392104556"><b>status</b></span> is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1275002353204518">200</strong>, the cluster is successfully created. The cluster ID is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b277694512204536">bae65496-643e-47ca-84af-948672de7eeb</strong>.</p>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li124718310578">Run the following command to set the cluster ID to a global variable for future use:<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen194962310578">export ID = <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b74961313571">bae65496-643e-47ca-84af-948672de7eeb</strong></pre>
|
|
</li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section1543119195119"><a name="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section1543119195119"></a><a name="en-us_topic_0000001716156281_en-us_topic_0108272822_section1543119195119"></a><h4 class="sectiontitle">Creating Links</h4><ol id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ol1944310917512"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li443339185118">Call the API in <a href="CreateLink_0.html">Creating a Link</a> to create the MySQL link <span class="parmvalue" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_parmvalue1850375516911"><b>mysql_link</b></span>. The following values are examples:<ul id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ul64305919512"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1242918912516">IP address: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1385033412118">1</strong><em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i95271530524">xx</em><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b486503962118">.120.85.24</strong></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li114292905113">Port number: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b84235270614570">3306</strong></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1242912985113">Database name: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b079955616205">DB_</strong><em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i37901259172010">name</em></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li642912925118">Login username: <em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i165277717217">username</em></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li6430697516">Password: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b3721815172113">DB_</strong><em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i18776181815212">password</em></li></ul>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p59531516484">If status code <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b197741618135214">200</strong> is returned, the link is successfully created.</p>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen12433396515">curl -X POST -H "Content-Type:application/json" -H "X-Auth-Token:$Token" -d '{
|
|
"links": [{
|
|
"enabled": true,
|
|
"update-user": null,
|
|
"name": "mysql_link",
|
|
"link-config-values": {
|
|
"configs": [
|
|
{
|
|
"name": "linkConfig",
|
|
"inputs": [
|
|
{
|
|
"name": "linkConfig.databaseType",
|
|
"value": "MYSQL"
|
|
},
|
|
{
|
|
"name": "linkConfig.host",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i174301895515"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b743016916514">1xx.120.85.24</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.port",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i04311914513"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b2043119915111">3306</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.database",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i1643111985115"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b843117916519">DB_name</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.username",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i643114935114"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b343169145115">username</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.password",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i14432119165118"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b19432191512">DB_password</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.fetchSize",
|
|
"value": "100000"
|
|
},
|
|
{
|
|
"name": "linkConfig.usingNative",
|
|
"value": "true"
|
|
}
|
|
]
|
|
}
|
|
]
|
|
},
|
|
"connector-name": "generic-jdbc-connector",
|
|
"creation-date": 1536654788622,
|
|
"update-date": 1536654788622,
|
|
"creation-user": null
|
|
}]
|
|
}'
|
|
https://{<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b84333918510">cdm_endpoint</strong>}/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b64331905110">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1743369115110">bae65496-643e-47ca-84af-948672de7eeb</strong>/cdm/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1543319910511">link</strong> -k -v</pre>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li34376975120">Call the API in <a href="CreateLink_0.html">Creating a Link</a> to create the DWS link <span class="parmvalue" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_parmvalue258814519103"><b>dws_link</b></span>. The following values are examples:<ul id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ul54341596511"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li34338935113">IP address of the database: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706145834">10.120.85.24</strong></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1343315915513">Port number: <strong id="dataartsstudio_02_0013__b2096577961">3306</strong></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li64349916510">Database name: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706145715">DWS</strong></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li2043410917519">Login username: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706145736">user_dws</strong></li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li64341995118">Password: <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706145810">dws_password</strong></li></ul>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen1043717910516">curl -X POST -H "Content-Type:application/json" -H "X-Auth-Token:$Token" -d '{
|
|
"links": [{
|
|
"enabled": true,
|
|
"update-user": null,
|
|
"name": "dws_link",
|
|
"link-config-values": {
|
|
"configs": [
|
|
{
|
|
"name": "linkConfig",
|
|
"inputs": [
|
|
{
|
|
"name": "linkConfig.databaseType",
|
|
"value": "DWS"
|
|
},
|
|
{
|
|
"name": "linkConfig.host",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i643514919517"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b743569195115">10.120.85.24</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.port",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i243513919512"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b154352912511">3306</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.database",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i16435599511"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1943514919510">DWS</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.username",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i14351598515"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b74353911514">user_dws</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.password",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i343619105119"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b104364985119">dws_password</strong></em>"
|
|
},
|
|
{
|
|
"name": "linkConfig.fetchSize",
|
|
"value": "100000"
|
|
},
|
|
{
|
|
"name": "linkConfig.usingNative",
|
|
"value": "true"
|
|
}
|
|
]
|
|
}
|
|
]
|
|
},
|
|
"connector-name": "generic-jdbc-connector",
|
|
"creation-date": 1536654788622,
|
|
"update-date": 1536654788622,
|
|
"creation-user": null
|
|
}]
|
|
}'
|
|
https://{<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b11437109105119">cdm_endpoint</strong>}/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1343717910514">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b9437092511">bae65496-643e-47ca-84af-948672de7eeb</strong>/cdm/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b34379916518">link</strong> -k -v</pre>
|
|
</li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section07183115314"><a name="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section07183115314"></a><a name="en-us_topic_0000001716156281_en-us_topic_0108272822_section07183115314"></a><h4 class="sectiontitle">Creating a Migration Job</h4><ol id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ol13547211533"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1451182165317">After the links are created, call the API in <a href="CreateJob_0.html">Creating a Job in a Specified Cluster</a> to create a migration job. The following is a sample job:<ul id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ul1047428534"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li174612218539">The job name is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b186561641131119">mysql2dws</strong>.</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li194612285318">The name of the MySQL database from which data is exported is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706205841">default</strong>, and the name of the exported table is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706205931">mysql_tbl</strong>. The job is split into multiple tasks by <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706205942">id</strong> and the tasks are executed concurrently.</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li94715225314">The name of the database on DWS to which the data is imported is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b8423527062123">public</strong>, and the table name is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b84235270621242">cdm_all_type</strong>. Do not clear the data in the table before import.</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li04732135317">If no table in the local MySQL database exists in the database on DWS, CDM automatically creates the table on DWS.</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li174718216531">The field list loaded to DWS is <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b84235270621359">id&gid&name</strong>.</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1947162115316">When the job extracts data, three extractors are concurrently executed.</li></ul>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p1651102125318">If status code <strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b483914169547">200</strong> is returned, the job is successfully created.</p>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen14515265311">curl -X POST -H "Content-Type:application/json" -H "X-Cluster-ID:$ID" -H "X-Auth-Token:$Token" -d '{
|
|
"jobs": [{
|
|
"job_type": "NORMAL_JOB",
|
|
"name": "mysql2dws",
|
|
"from-link-name": "mysql_link",
|
|
"from-connector-name": "generic-jdbc-connector",
|
|
"to-link-name": "dws_link",
|
|
"to-connector-name": "generic-jdbc-connector",
|
|
"from-config-values": {
|
|
"configs": [{
|
|
"name": "fromJobConfig",
|
|
"inputs": [{
|
|
"name": "fromJobConfig.schemaName",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i24820245317">default</em>"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.tableName",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i848826539">mysql_tbl</em>"
|
|
},
|
|
{
|
|
"name": "fromJobConfig.partitionColumn",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i194992175318">id</em>"
|
|
}]
|
|
}]
|
|
},
|
|
"to-config-values": {
|
|
"configs": [
|
|
{
|
|
"inputs": [
|
|
{
|
|
"name": "toJobConfig.schemaName",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i44910218535">public</em>"
|
|
},
|
|
{
|
|
"name": "toJobConfig.tablePreparation",
|
|
"value": "CREATE_WHEN_NOT_EXIST"
|
|
},
|
|
{
|
|
"name": "toJobConfig.tableName",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i165016213532">cdm_all_type</em>"
|
|
},
|
|
{
|
|
"name": "toJobConfig.columnList",
|
|
"value": "<em id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_i65015217534">id&gid&name</em>"
|
|
},
|
|
{
|
|
"name": "toJobConfig.shouldClearTable",
|
|
"value": "false"
|
|
}
|
|
],
|
|
"name": "toJobConfig"
|
|
}
|
|
]
|
|
},
|
|
"driver-config-values": {
|
|
"configs": [{
|
|
"name": "throttlingConfig",
|
|
"inputs": [{
|
|
"name": "throttlingConfig.numExtractors",
|
|
"value": "3"
|
|
}]
|
|
}]
|
|
}
|
|
}]
|
|
}' https://<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b135114217532">{cdm_endpoint}</strong>/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b105192145316">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b75117219531">bae65496-643e-47ca-84af-948672de7eeb</strong>/cdm/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1651152165316">job</strong> -k -v</pre>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li15242145316">Call the API in <a href="StartJob_0.html">Starting a Job</a> to execute the job.<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen55210245317">curl -X GET -H 'Content-Type:application/json;charset=utf-8' -H "X-Cluster-ID:$ID" -H "X-Auth-Token:$Token" https://<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1511921538">{cdm_endpoint}</strong>/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b14521213535">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1452152115317">bae65496-643e-47ca-84af-948672de7eeb</strong>/cdm/job/mysql2dws/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b2527211539">start</strong> -k -v</pre>
|
|
<p id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_p155262115313">The response is as follows:</p>
|
|
<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen9524212538">{
|
|
"submissions": [{
|
|
"progress": 1,
|
|
"job-name": "mysql2dws",
|
|
"status": "BOOTING",
|
|
"creation-date": 1536654788622,
|
|
"creation-user": "cdm"
|
|
}]
|
|
}</pre>
|
|
</li></ol>
|
|
</div>
|
|
<div class="section" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section179778188581"><a name="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_section179778188581"></a><a name="en-us_topic_0000001716156281_en-us_topic_0108272822_section179778188581"></a><h4 class="sectiontitle">Viewing Job Result</h4><ol id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ol161631519135817"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li2163161955815">Call the API in <a href="ShowJobStatus_0.html">Querying Job Status</a> to query the job status.<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen01617199589">curl -X GET -H 'Content-Type:application/json;charset=utf-8' -H "X-Cluster-ID:$ID" -H "X-Auth-Token:$Token" https://<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b131615198584">{cdm_endpoint}</strong>/v1.1/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b21618196580">1551c7f6c808414d8e9f3c514a170f2e</strong>/clusters/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b316116194580">6ec9a0a4-76be-4262-8697-e7af1fac7920</strong>/cdm/job/mysql2dws/<strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b121612198585">status</strong> -k -v</pre>
|
|
</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li96151739306">View the job execution result. The response to successful job execution is as follows:<pre class="screen" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_screen667716391800">{
|
|
"submissions": [{
|
|
"progress": 0,
|
|
"job-name": "mysql2dws",
|
|
"status": "SUCCEEDED",
|
|
"creation-date": 1536654788622,
|
|
"creation-user": "cdm",
|
|
"isStopingIncrement": "",
|
|
"last-update-date": 1536654888622,
|
|
"is-execute-auto": false,
|
|
"last-update-user": "cdm",
|
|
"isDeleteJob": false,
|
|
"isIncrementing": false,
|
|
"external-id": "job_local1127970451_0009",
|
|
"counters": {
|
|
"org.apache.sqoop.submission.counter.SqoopCounters": {
|
|
"BYTES_WRITTEN": -1,
|
|
"TOTAL_FILES": -1,
|
|
"BYTES_READ": -1,
|
|
"FILES_WRITTEN": -1,
|
|
"TOTAL_SIZE": -1,
|
|
"FILES_READ": -1,
|
|
"ROWS_WRITTEN": 80,
|
|
"ROWS_READ": 80
|
|
}
|
|
}
|
|
}]
|
|
}</pre>
|
|
<div class="note" id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_note18679113912010"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_ul3679203918019"><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li1367733911018"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b26771849152520">BYTES_WRITTEN</strong>: number of written bytes</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li2067818392019"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1099184512262">BYTES_READ</strong>: number of read bytes</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li15679339308"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b14335155516261">TOTAL_FILES</strong>: total number of files</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li46791239905"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b1277311103275">FILES_WRITTEN</strong>: number of written files</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li26798392013"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b13361640172718">FILES_READ</strong>: number of read files</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li46794391004"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706211511">ROWS_WRITTEN</strong>: number of rows that are successfully written</li><li id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_li116792391408"><strong id="dataartsstudio_02_0013__en-us_topic_0000001716156281_en-us_topic_0108272822_b842352706211523">ROWS_READ</strong>: number of rows that are successfully read</li></ul>
|
|
</div></div>
|
|
</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="dataartsstudio_02_0012.html">Application Cases</a></div>
|
|
</div>
|
|
</div>
|
|
|