vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity

dli_dev_0104_version

Browse Source 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-committed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
This commit is contained in:
Hasko, Vladimir 2024-05-06 09:14:57 +00:00 committed by zuul
parent 4edaa4a4cc
commit cfc48b3aed
164 changed files with 12519 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1304
docs/dli/dev/ALL_META.TXT.json Normal file
View File

File diff suppressed because it is too large Load Diff

560
docs/dli/dev/CLASS.TXT.json Normal file
View File

File diff suppressed because it is too large Load Diff

3
docs/dli/dev/PARAMETERS.txt Normal file
View File

@ -0,0 +1,3 @@
version=""
language="en-us"
type=""

175
docs/dli/dev/dli_05_0044.html Normal file
View File

File diff suppressed because it is too large Load Diff

250
docs/dli/dev/dli_05_0062.html Normal file
View File

File diff suppressed because it is too large Load Diff

32
docs/dli/dev/dli_09_00001.html Normal file
View File

@ -0,0 +1,32 @@
<a name="dli_09_00001"></a><a name="dli_09_00001"></a>
<h1 class="topictitle1">Change History</h1>
<div id="body0000001762602254">
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_09_00001__table7273930123717" frame="border" border="1" rules="all"><thead align="left"><tr id="dli_09_00001__row18273153018375"><th align="left" class="cellrowborder" valign="top" width="31.31%" id="mcps1.3.1.1.3.1.1"><p id="dli_09_00001__p1227312301379"><strong id="dli_09_00001__b102731430103716">Release Date</strong></p>
</th>
<th align="left" class="cellrowborder" valign="top" width="68.69%" id="mcps1.3.1.1.3.1.2"><p id="dli_09_00001__p52738307372"><strong id="dli_09_00001__b14378427510">What's New</strong></p>
</th>
</tr>
</thead>
<tbody><tr id="dli_09_00001__row138391724164218"><td class="cellrowborder" valign="top" width="31.31%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_09_00001__p7491122717422">2024-04-30</p>
</td>
<td class="cellrowborder" valign="top" width="68.69%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_09_00001__p4491132714424">Modified the following section:</p>
<p id="dli_09_00001__p3491182716423">In <a href="dli_09_0113.html">Connecting to Mongo</a>, modified "mongo" in the sample table name and added additional clarification that DDS is compatible with the MongoDB protocol.</p>
</td>
</tr>
<tr id="dli_09_00001__row1673713139618"><td class="cellrowborder" valign="top" width="31.31%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_09_00001__p37726171462">2024-02-27</p>
</td>
<td class="cellrowborder" valign="top" width="68.69%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_09_00001__p877211171368">Modified the following section:</p>
<p id="dli_09_00001__p117787291869">Modified the description of the <strong id="dli_09_00001__b58731839111115">url</strong> parameter in the sample code in <a href="dli_09_0114.html">Scala Example Code</a>.</p>
</td>
</tr>
<tr id="dli_09_00001__row7273153020376"><td class="cellrowborder" valign="top" width="31.31%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_09_00001__p1327312306374">2024-01-05</p>
</td>
<td class="cellrowborder" valign="top" width="68.69%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_09_00001__p3164451183711">This issue is the first official release.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>

22
docs/dli/dev/dli_09_0006.html Normal file
View File

@ -0,0 +1,22 @@
<a name="dli_09_0006"></a><a name="dli_09_0006"></a>
<h1 class="topictitle1">Flink OpenSource SQL Jobs</h1>
<div id="body1547114988859"><p id="dli_09_0006__p8060118"></p>
</div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0009.html">Reading Data from Kafka and Writing Data to RDS</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0010.html">Reading Data from Kafka and Writing Data to GaussDB(DWS)</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0011.html">Reading Data from Kafka and Writing Data to Elasticsearch</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0012.html">Reading Data from MySQL CDC and Writing Data to GaussDB(DWS)</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0013.html">Reading Data from PostgreSQL CDC and Writing Data to GaussDB(DWS)</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0207.html">Configuring High-Reliability Flink Jobs (Automatic Restart upon Exceptions)</a></strong><br>
</li>
</ul>
</div>

143
docs/dli/dev/dli_09_0009.html Normal file
View File

File diff suppressed because it is too large Load Diff

124
docs/dli/dev/dli_09_0010.html Normal file
View File

File diff suppressed because it is too large Load Diff

202
docs/dli/dev/dli_09_0011.html Normal file
View File

File diff suppressed because it is too large Load Diff

129
docs/dli/dev/dli_09_0012.html Normal file
View File

File diff suppressed because it is too large Load Diff

245
docs/dli/dev/dli_09_0013.html Normal file
View File

File diff suppressed because it is too large Load Diff

29
docs/dli/dev/dli_09_0019.html Normal file
View File

@ -0,0 +1,29 @@
<a name="dli_09_0019"></a><a name="dli_09_0019"></a>
<h1 class="topictitle1">Using Spark Jobs to Access Data Sources of Datasource Connections</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0020.html">Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0089.html">Connecting to CSS</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0086.html">Connecting to GaussDB(DWS)</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0077.html">Connecting to HBase</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0080.html">Connecting to OpenTSDB</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0083.html">Connecting to RDS</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0093.html">Connecting to Redis</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0113.html">Connecting to Mongo</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0203.html">Spark Jar Jobs</a></div>
</div>
</div>

13
docs/dli/dev/dli_09_0020.html Normal file
View File

@ -0,0 +1,13 @@
<a name="dli_09_0020"></a><a name="dli_09_0020"></a>
<h1 class="topictitle1">Overview</h1>
<div id="body8662426"><p id="dli_09_0020__p1697165194619">DLI supports the native Spark DataSource capability and other extended capabilities. You can use SQL statements or Spark jobs to access other data storage services and import, query, analyze, and process data. Currently, DLI supports the following datasource access services: CloudTable, Cloud Search Service (CSS), Distributed Cache Service (DCS), Document Database Service (DDS), GaussDB(DWS), MapReduce Service (MRS), and Relational Database Service (RDS). To use the datasource capability of DLI, you need to create a datasource connection first.</p>
<p id="dli_09_0020__en-us_topic_0149356083_en-us_topic_0142920068_p17615162014209">When you use Spark jobs to access other data sources, you can use Scala, PySpark, or Java to program functions for the jobs.</p>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

759
docs/dli/dev/dli_09_0061.html Normal file
View File

File diff suppressed because it is too large Load Diff

450
docs/dli/dev/dli_09_0063.html Normal file
View File

File diff suppressed because it is too large Load Diff

274
docs/dli/dev/dli_09_0065.html Normal file
View File

File diff suppressed because it is too large Load Diff

577
docs/dli/dev/dli_09_0067.html Normal file
View File

File diff suppressed because it is too large Load Diff

515
docs/dli/dev/dli_09_0069.html Normal file
View File

File diff suppressed because it is too large Load Diff

23
docs/dli/dev/dli_09_0077.html Normal file
View File

@ -0,0 +1,23 @@
<a name="dli_09_0077"></a><a name="dli_09_0077"></a>
<h1 class="topictitle1">Connecting to HBase</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0196.html">MRS Configuration</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0063.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0078.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0197.html">Java Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0198.html">Troubleshooting</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

299
docs/dli/dev/dli_09_0078.html Normal file
View File

File diff suppressed because it is too large Load Diff

21
docs/dli/dev/dli_09_0080.html Normal file
View File

@ -0,0 +1,21 @@
<a name="dli_09_0080"></a><a name="dli_09_0080"></a>
<h1 class="topictitle1">Connecting to OpenTSDB</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0065.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0081.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0193.html">Java Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0195.html">Troubleshooting</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

167
docs/dli/dev/dli_09_0081.html Normal file
View File

File diff suppressed because it is too large Load Diff

19
docs/dli/dev/dli_09_0083.html Normal file
View File

@ -0,0 +1,19 @@
<a name="dli_09_0083"></a><a name="dli_09_0083"></a>
<h1 class="topictitle1">Connecting to RDS</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0067.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0084.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0187.html">Java Example Code</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

220
docs/dli/dev/dli_09_0084.html Normal file
View File

File diff suppressed because it is too large Load Diff

19
docs/dli/dev/dli_09_0086.html Normal file
View File

@ -0,0 +1,19 @@
<a name="dli_09_0086"></a><a name="dli_09_0086"></a>
<h1 class="topictitle1">Connecting to GaussDB(DWS)</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0069.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0087.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0199.html">Java Example Code</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

298
docs/dli/dev/dli_09_0087.html Normal file
View File

File diff suppressed because it is too large Load Diff

21
docs/dli/dev/dli_09_0089.html Normal file
View File

@ -0,0 +1,21 @@
<a name="dli_09_0089"></a><a name="dli_09_0089"></a>
<h1 class="topictitle1">Connecting to CSS</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0189.html">CSS Security Cluster Configuration</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0061.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0090.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0190.html">Java Example Code</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

546
docs/dli/dev/dli_09_0090.html Normal file
View File

File diff suppressed because it is too large Load Diff

21
docs/dli/dev/dli_09_0093.html Normal file
View File

@ -0,0 +1,21 @@
<a name="dli_09_0093"></a><a name="dli_09_0093"></a>
<h1 class="topictitle1">Connecting to Redis</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0094.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0097.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0100.html">Java Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0188.html">Troubleshooting</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

603
docs/dli/dev/dli_09_0094.html Normal file
View File

File diff suppressed because it is too large Load Diff

217
docs/dli/dev/dli_09_0097.html Normal file
View File

File diff suppressed because it is too large Load Diff

157
docs/dli/dev/dli_09_0100.html Normal file
View File

File diff suppressed because it is too large Load Diff

202
docs/dli/dev/dli_09_0110.html Normal file
View File

File diff suppressed because it is too large Load Diff

19
docs/dli/dev/dli_09_0113.html Normal file
View File

@ -0,0 +1,19 @@
<a name="dli_09_0113"></a><a name="dli_09_0113"></a>
<h1 class="topictitle1">Connecting to Mongo</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0114.html">Scala Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0117.html">PySpark Example Code</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0110.html">Java Example Code</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0019.html">Using Spark Jobs to Access Data Sources of Datasource Connections</a></div>
</div>
</div>

263
docs/dli/dev/dli_09_0114.html Normal file
View File

File diff suppressed because it is too large Load Diff

200
docs/dli/dev/dli_09_0117.html Normal file
View File

File diff suppressed because it is too large Load Diff

19
docs/dli/dev/dli_09_0120.html Normal file
View File

@ -0,0 +1,19 @@
<a name="dli_09_0120"></a><a name="dli_09_0120"></a>
<h1 class="topictitle1">SQL Jobs</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_05_0044.html">Using Spark SQL Jobs to Analyze OBS Data</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0171.html">Calling UDFs in Spark SQL Jobs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0204.html">Calling UDTFs in Spark SQL Jobs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_05_0062.html">Calling UDAFs in Spark SQL Jobs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0123.html">Submitting a Spark SQL Job Using JDBC</a></strong><br>
</li>
</ul>
</div>

46
docs/dli/dev/dli_09_0121.html Normal file
View File

@ -0,0 +1,46 @@
<a name="dli_09_0121"></a><a name="dli_09_0121"></a>
<h1 class="topictitle1">Performing Authentication</h1>
<div id="body8662426"><div class="section" id="dli_09_0121__en-us_topic_0093946854_section2073076515120"><h4 class="sectiontitle">Scenario</h4><p id="dli_09_0121__en-us_topic_0093946854_p2214320115124">You need to be authenticated when using JDBC to create DLI driver connections.</p>
</div>
<div class="section" id="dli_09_0121__en-us_topic_0093946854_section18149062151222"><h4 class="sectiontitle">Procedure</h4><p id="dli_09_0121__en-us_topic_0093946854_p2577325517947">Currently, the JDBC supports authentication using the Access Key/Secret Key (AK/SK) or token.</p>
<ul id="dli_09_0121__en-us_topic_0093946854_ul16758061123310"><li id="dli_09_0121__en-us_topic_0093946854_li66078932151559">(Recommended) Generating an AK/SK<ol id="dli_09_0121__en-us_topic_0093946854_ol50793300151559"><li id="dli_09_0121__en-us_topic_0093946854_li54486522151559">Log in to the DLI management console.</li><li id="dli_09_0121__en-us_topic_0093946854_li20616653151559">Click the username in the upper right corner and select <span class="uicontrol" id="dli_09_0121__uicontrol6520707591112"><b>My Credentials</b></span> from the drop-down list.</li><li id="dli_09_0121__en-us_topic_0093946854_li59336166151559">On the displayed <span class="uicontrol" id="dli_09_0121__uicontrol805792508184342"><b>My Credentials</b></span> page, click <span class="wintitle" id="dli_09_0121__wintitle37956610151559"><b>Access Keys</b></span>. By default, the <span class="wintitle" id="dli_09_0121__wintitle4955132514402"><b>Project List</b></span> page is displayed.</li><li id="dli_09_0121__en-us_topic_0093946854_li6065177151559">Click <span class="uicontrol" id="dli_09_0121__uicontrol54586595151559"><b>Add Access Key</b></span>. In the displayed <strong id="dli_09_0121__b842352706184524">Add Access Key</strong> dialog box, specify <span class="parmname" id="dli_09_0121__parmname1729726471184548"><b>Login Password</b></span> and <span class="parmname" id="dli_09_0121__parmname822559648184548"><b>SMS Verification Code</b></span>.</li><li id="dli_09_0121__en-us_topic_0093946854_li21517315151559">Click <span class="uicontrol" id="dli_09_0121__uicontrol59438113151559"><b>OK</b></span> to download the certificate.</li><li id="dli_09_0121__en-us_topic_0093946854_li65180974151559">After the certificate is downloaded, you can obtain the AK and SK information in the <strong id="dli_09_0121__b5202851719139">credentials</strong> file.</li></ol>
<div class="note" id="dli_09_0121__note196905137167"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_09_0121__p414493171019">Hard coding AKs and SKs or storing them in code in plaintext poses significant security risks. You are advised to store them in encrypted form in configuration files or environment variables and decrypt them when needed to ensure security.</p>
</div></div>
</li><li id="dli_09_0121__en-us_topic_0093946854_li66327556123310">Obtain the token.<p id="dli_09_0121__en-us_topic_0093946854_p758818412234"><a name="dli_09_0121__en-us_topic_0093946854_li66327556123310"></a><a name="en-us_topic_0093946854_li66327556123310"></a>When using token authentication, you need to obtain the user token and configure the token information in the JDBC connection parameters. You can obtain the token as follows:</p>
<ol id="dli_09_0121__en-us_topic_0093946854_ol1120012248335"><li id="dli_09_0121__en-us_topic_0093946854_li920012483318">Send <i><b><span class="cmdname" style="font-family:Arial" id="dli_09_0121__en-us_topic_0093946854_cmdname1854033311311">POST https://&lt;IAM_Endpoint&gt;/v3/auth/tokens</span></b></i>. To obtain the IAM endpoint, contact the administrator to obtain the region and endpoint information.<div class="p" id="dli_09_0121__en-us_topic_0093946854_p120019249335">An example request message is as follows:<div class="note" id="dli_09_0121__en-us_topic_0093946854_note1420019242332"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_09_0121__en-us_topic_0093946854_p820014246337">Replace content in italic in the sample code with the actual values. For details, see .</p>
</div></div>
<pre class="screen" id="dli_09_0121__en-us_topic_0093946854_screen20200132433310">{
"auth": {
"identity": {
"methods": [
"password"
],
"password": {
"user": {
"name": "<em id="dli_09_0121__en-us_topic_0093946854_i7200124103310">username</em>",
"password": "<em id="dli_09_0121__en-us_topic_0093946854_i18200024203314">password</em>",
"domain": {
"name": "<em id="dli_09_0121__en-us_topic_0093946854_i1420032415330">domainname</em>"
}
}
}
},
"scope": {
"project": {
"id": "0aa253a31a2f4cfda30eaa073fee6477" //Assume that <strong id="dli_09_0121__b3786124883116">project_id</strong> is <strong id="dli_09_0121__b478614480313">0aa253a31a2f4cfda30eaa073fee6477</strong>.
}
}
}
}</pre>
</div>
</li><li id="dli_09_0121__en-us_topic_0093946854_li07732031141510">After the request is processed, the value of <span class="parmvalue" id="dli_09_0121__parmvalue860919211214"><b>X-Subject-Token</b></span> in the response header is the token value.</li></ol>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0123.html">Submitting a Spark SQL Job Using JDBC</a></div>
</div>
</div>

209
docs/dli/dev/dli_09_0122.html Normal file
View File

File diff suppressed because it is too large Load Diff

23
docs/dli/dev/dli_09_0123.html Normal file
View File

@ -0,0 +1,23 @@
<a name="dli_09_0123"></a><a name="dli_09_0123"></a>
<h1 class="topictitle1">Submitting a Spark SQL Job Using JDBC</h1>
<div id="body8662426"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_09_0124.html">Obtaining the Server Connection Address</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0125.html">Downloading the JDBC Driver Package</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0121.html">Performing Authentication</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0127.html">Submitting a Job Using JDBC</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_09_0129.html">Introduction to RDS</a></strong><br>
</li>
</ul>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0120.html">SQL Jobs</a></div>
</div>
</div>

15
docs/dli/dev/dli_09_0124.html Normal file
View File

@ -0,0 +1,15 @@
<a name="dli_09_0124"></a><a name="dli_09_0124"></a>
<h1 class="topictitle1">Obtaining the Server Connection Address</h1>
<div id="body8662426"><div class="section" id="dli_09_0124__en-us_topic_0093946851_section26616723151647"><h4 class="sectiontitle">Scenario</h4><p id="dli_09_0124__en-us_topic_0093946851_p19197496151650">On DLI, you can connect to the server for data query in the Internet environment. In this case, you need to first obtain the connection information, including the endpoint and project ID by following the following procedure.</p>
</div>
<div class="section" id="dli_09_0124__en-us_topic_0093946851_section5539467151713"><h4 class="sectiontitle">Procedure</h4><p id="dli_09_0124__en-us_topic_0093946851_p880312216531">The format of the address for connecting to DLI is jdbc:dli://&lt;endPoint&gt;/&lt;projectId&gt;. Therefore, you need to obtain the endpoint and project ID.</p>
<p id="dli_09_0124__en-us_topic_0093946851_p33898414152138">Obtain the DLI endpoint from the administrator. Specifically, log in to the cloud, click your username, and choose <strong id="dli_09_0124__b86924584169">My Credentials</strong> from the short-cut menu. You can obtain the project ID on the <strong id="dli_09_0124__b669265814164">My Credentials</strong> page.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0123.html">Submitting a Spark SQL Job Using JDBC</a></div>
</div>
</div>

18
docs/dli/dev/dli_09_0125.html Normal file
View File

@ -0,0 +1,18 @@
<a name="dli_09_0125"></a><a name="dli_09_0125"></a>
<h1 class="topictitle1">Downloading the JDBC Driver Package</h1>
<div id="body8662426"><div class="section" id="dli_09_0125__en-us_topic_0093946838_section44958387161653"><h4 class="sectiontitle">Scenario</h4><p id="dli_09_0125__en-us_topic_0093946838_p55664307161655">To connect to DLI, JDBC is utilized. You can obtain the JDBC installation package from Maven or download the JDBC driver file from the DLI management console.</p>
</div>
<div class="section" id="dli_09_0125__section051344617132"><h4 class="sectiontitle">Downloading the JDBC Driver File from the DLI Management Console</h4><ol id="dli_09_0125__ol166351352111318"><li id="dli_09_0125__li1963595271312"><span>Log in to the DLI management console.</span></li><li id="dli_09_0125__li1263511522131"><span>Ask the administrator for the download link to the SDK driver package.</span></li><li id="dli_09_0125__li763545221312"><span>On the <strong id="dli_09_0125__b868548146">DLI SDK DOWNLOAD</strong> page, select a driver and download it.</span><p><ul id="dli_09_0125__ul116354524136"><li id="dli_09_0125__li3635165221310">JDBC driver package<p id="dli_09_0125__p1763595210138"><a name="dli_09_0125__li3635165221310"></a><a name="li3635165221310"></a>For example, click <strong id="dli_09_0125__b13178105912612">dli-jdbc-1.2.5</strong> to download the JDBC driver package of version 1.2.5.</p>
<div class="note" id="dli_09_0125__note1063505241314"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_09_0125__p13635175291313">The JDBC driver package is named <strong id="dli_09_0125__b4845321875">dli-jdbc-</strong><em id="dli_09_0125__i1784511218718">&lt;version&gt;</em><strong id="dli_09_0125__b1084613211715">.zip</strong>. It can be used in all versions of all platforms (such as Linux and Windows) and depends on JDK 1.7 or later.</p>
</div></div>
</li></ul>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0123.html">Submitting a Spark SQL Job Using JDBC</a></div>
</div>
</div>

463
docs/dli/dev/dli_09_0127.html Normal file
View File

File diff suppressed because it is too large Load Diff

14
docs/dli/dev/dli_09_0129.html Normal file
View File

@ -0,0 +1,14 @@
<a name="dli_09_0129"></a><a name="dli_09_0129"></a>
<h1 class="topictitle1">Introduction to RDS</h1>
<div id="body8662426"><div class="section" id="dli_09_0129__en-us_topic_0117736646_section1164215581616"><h4 class="sectiontitle">Basic Concepts</h4><p id="dli_09_0129__en-us_topic_0117736646_p363211526179">Relational Database Service (RDS) is a cloud-based web service that is reliable, scalable, easy to manage, and immediately ready for use. It can be deployed in single-node, active/standby, or cluster mode.</p>
</div>
<div class="section" id="dli_09_0129__en-us_topic_0117736646_section7322141516348"><h4 class="sectiontitle">Features</h4><ul id="dli_09_0129__en-us_topic_0117736646_ul77572012543"><li id="dli_09_0129__en-us_topic_0117736646_li187577095418">Hosting: RDS provides upgrade, patch, and backup hosting services.</li><li id="dli_09_0129__en-us_topic_0117736646_li157573017542">Compatibility: RDS is compatible with MySQL, SQL Server, and PostgreSQL.</li><li id="dli_09_0129__en-us_topic_0117736646_li117571606548">Scalability: RDS provides elastic capacity expansion to deliver one-click CPU, memory, and disk capacity expansion</li><li id="dli_09_0129__en-us_topic_0117736646_li4757160185414">Security: RDS supports creation of VPCs, subnets, and security groups.</li><li id="dli_09_0129__en-us_topic_0117736646_li8757601542">Stability: RDS supports automatic or manual backup.</li><li id="dli_09_0129__en-us_topic_0117736646_li147575011546">High availability: RDS can work in active/standby mode.</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0123.html">Submitting a Spark SQL Job Using JDBC</a></div>
</div>
</div>

180
docs/dli/dev/dli_09_0150.html Normal file
View File

File diff suppressed because it is too large Load Diff

20
docs/dli/dev/dli_09_0162.html Normal file
View File

@ -0,0 +1,20 @@
<a name="dli_09_0162"></a><a name="dli_09_0162"></a>
<h1 class="topictitle1">Stream Ecosystem</h1>
<div id="body1575018104651"><div class="section" id="dli_09_0162__section1855218487123"><h4 class="sectiontitle">Overview</h4><p id="dli_09_0162__p11165751163414">Built on Flink and Spark, the stream ecosystem is fully compatible with the open-source Flink, Storm, and Spark APIs. It is enhanced in features and improved in performance to provide easy-to-use DLI with low latency and high throughput.</p>
<div class="p" id="dli_09_0162__p14165195113419">The DLI stream ecosystem includes the cloud service ecosystem, open source ecosystem, and custom stream ecosystem.<ul id="dli_09_0162__ul2165175183418"><li id="dli_09_0162__li1616575193411">Cloud service ecosystems<p id="dli_09_0162__p11656519340"><a name="dli_09_0162__li1616575193411"></a><a name="li1616575193411"></a>DLI can be interconnected with other services by using Stream SQLs. You can directly use SQL statements to read and write data from various cloud services, such as Data Ingestion Service (DIS), Object Storage Service (OBS), CloudTable Service (CloudTable), MapReduce Service (MRS), Relational Database Service (RDS), Simple Message Notification (SMN), and Distributed Cache Service (DCS).</p>
</li><li id="dli_09_0162__li516515133412">Open-source ecosystems<p id="dli_09_0162__p9165135173419"><a name="dli_09_0162__li516515133412"></a><a name="li516515133412"></a>After connections to other VPCs are established through VPC peering connections, you can access all data sources and output targets (such as Kafka, HBase, and Elasticsearch) supported by Flink and Spark in your dedicated DLI queues.</p>
</li><li id="dli_09_0162__li1250335412473">Custom stream ecosystems<p id="dli_09_0162__p1849919716482"><a name="dli_09_0162__li1250335412473"></a><a name="li1250335412473"></a>You can compile code to obtain data from the desired cloud ecosystem or open-source ecosystem as the input data of Flink jobs.</p>
</li></ul>
</div>
</div>
<div class="section" id="dli_09_0162__section1822193412152"><h4 class="sectiontitle">Supported Data Formats</h4><p id="dli_09_0162__p1018126165813">DLI Flink jobs support the following data formats:</p>
<p id="dli_09_0162__p7323644191518">Avro, Avro_merge, BLOB, CSV, EMAIL, JSON, ORC, Parquet, and XML.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_09_0202.html">Flink Jar Jobs</a></div>
</div>
</div>

153
docs/dli/dev/dli_09_0171.html Normal file
View File

File diff suppressed because it is too large Load Diff

372
docs/dli/dev/dli_09_0176.html Normal file
View File

File diff suppressed because it is too large Load Diff

124
docs/dli/dev/dli_09_0187.html Normal file
View File

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More