dli_umn_0519_version

Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
This commit is contained in:
Su, Xiaomeng 2023-05-20 10:20:05 +00:00 committed by zuul
parent 8d0cd12846
commit f701254745
256 changed files with 4317 additions and 2449 deletions

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -8,7 +8,13 @@
</th>
</tr>
</thead>
<tbody><tr id="dli_01_00006__row01821691115"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p1518161613115">2023-01-30</p>
<tbody><tr id="dli_01_00006__row33702917592"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p1528713120599">2023-05-11</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p16287151205919">Modified the following content:</p>
<ul id="dli_01_00006__ul209491872110"><li id="dli_01_00006__li99491719111">Added the dependency package information of Spark 2.4.5 and Flink 1.12 to <a href="dli_01_0397.html">Built-in Dependencies</a>.</li><li id="dli_01_00006__li119492716114">Modified the description of log buckets in <a href="dli_01_0320.html">SQL Editor</a>.</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row01821691115"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p1518161613115">2023-01-30</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p14331173193510">This issue is the first official release.</p>
</td>

View File

@ -1,40 +1,35 @@
<a name="dli_01_0002"></a><a name="dli_01_0002"></a>
<h1 class="topictitle1">Creating and Submitting a Spark SQL Job</h1>
<div id="body1501038213757"><p id="dli_01_0002__p8060118">You can use DLI to submit a Spark SQL job to query data. The general procedure is as follows:</p>
<p id="dli_01_0002__p313355315401"><a href="#dli_01_0002__section3751181910618">Step 1: Logging in to the Cloud Platform</a></p>
<p id="dli_01_0002__p1391181313415"><a href="#dli_01_0002__section61379418181550">Step 2: Uploading Data to OBS</a></p>
<p id="dli_01_0002__p10892641184110"><a href="#dli_01_0002__section19012773105034">Step 3: Logging In to the DLI Management Console</a></p>
<p id="dli_01_0002__p146371751184314"><a href="#dli_01_0002__section10742144985011">Step 4: Creating a Queue</a></p>
<p id="dli_01_0002__p13187162154420"><a href="#dli_01_0002__section21433273112656">Step 5: Creating a Database</a></p>
<p id="dli_01_0002__p266214107449"><a href="#dli_01_0002__section21590507141153">Step 6: Creating a Table</a></p>
<p id="dli_01_0002__p9606134314412"><a href="#dli_01_0002__section37788816112733">Step 7: Querying Data</a></p>
<p id="dli_01_0002__p3654323610322">The following illustrates how to query OBS data using DLI. Operations to query DLI data are similar.</p>
<div class="section" id="dli_01_0002__section3751181910618"><a name="dli_01_0002__section3751181910618"></a><a name="section3751181910618"></a><h4 class="sectiontitle">Step 1: Logging in to the Cloud Platform</h4><ol id="dli_01_0002__ol77610191561"><li id="dli_01_0002__li1968871183717">Open the DLI homepage.</li><li id="dli_01_0002__li1597319241563">On the login page, enter the username and password, and click <strong id="dli_01_0002__b380955624716">Log In</strong>.</li></ol>
<div id="body1501038213757"><div class="section" id="dli_01_0002__section76281123162520"><h4 class="sectiontitle">Scenarios</h4><p id="dli_01_0002__p11420171102612">DLI can query data stored in OBS. This section describes how to us a Spark SQL job on DLI to query OBS data.</p>
</div>
<div class="section" id="dli_01_0002__section61379418181550"><a name="dli_01_0002__section61379418181550"></a><a name="section61379418181550"></a><h4 class="sectiontitle">Step 2: Uploading Data to OBS</h4><p id="dli_01_0002__p45348185145915">DLI allows you to query data stored on OBS. Before querying the data, you need to upload data to OBS.</p>
<ol id="dli_01_0002__ol19047014145745"><li id="dli_01_0002__li18770142812720">In the services displayed, click <strong id="dli_01_0002__b16911338132720">Object Storage Service (OBS)</strong> in <strong id="dli_01_0002__b1679352125611">Storage</strong>.</li><li id="dli_01_0002__li2478348216220">The OBS console page is displayed.</li><li id="dli_01_0002__li1385211252717">Create a bucket. The bucket name must be globally unique. In this example, assume that the bucket name is <strong id="dli_01_0002__b349033115613">obs1</strong>.<ol type="a" id="dli_01_0002__ol182612310711"><li id="dli_01_0002__li462934116719">Click <span class="uicontrol" id="dli_01_0002__uicontrol481933104114"><b>Create Bucket</b></span> in the upper right corner.</li><li id="dli_01_0002__li183491949872">On the displayed <strong id="dli_01_0002__b997218337576">Create Bucket</strong> page, enter the <strong id="dli_01_0002__b41931653205710">Bucket Name</strong>. Retain the default values for other parameters or set them as required.</li><li id="dli_01_0002__li3583138155449">Click <strong id="dli_01_0002__b55859547573">Create Now</strong>.</li></ol>
<div class="section" id="dli_01_0002__section136122917251"><h4 class="sectiontitle">Procedure</h4><p id="dli_01_0002__p8060118">You can use DLI to submit a Spark SQL job to query data. The general procedure is as follows:</p>
<p id="dli_01_0002__p1391181313415"><a href="#dli_01_0002__section61379418181550">Step 1: Upload Data to OBS</a></p>
<p id="dli_01_0002__p146371751184314"><a href="#dli_01_0002__section10742144985011">Step 2: Create a Queue</a></p>
<p id="dli_01_0002__p13187162154420"><a href="#dli_01_0002__section21433273112656">Step 3: Create a Database</a></p>
<p id="dli_01_0002__p266214107449"><a href="#dli_01_0002__section21590507141153">Step 4: Create a Table</a></p>
<p id="dli_01_0002__p9606134314412"><a href="#dli_01_0002__section37788816112733">Step 5: Query Data</a></p>
</div>
<div class="section" id="dli_01_0002__section61379418181550"><a name="dli_01_0002__section61379418181550"></a><a name="section61379418181550"></a><h4 class="sectiontitle">Step 1: Upload Data to OBS</h4><p id="dli_01_0002__p161899822612">Before you use DLI to query and analyze data, upload data files to OBS.</p>
<ol id="dli_01_0002__ol19047014145745"><li id="dli_01_0002__li17461001806">Go to the DLI console.</li><li id="dli_01_0002__li18770142812720">In the service list, click <strong id="dli_01_0002__b1075523782712">Object Storage Service</strong> under <strong id="dli_01_0002__b11498104092711">Storage</strong>. The OBS console page is displayed.</li><li id="dli_01_0002__li1385211252717">Create a bucket. In this example, the bucket name is <strong id="dli_01_0002__b2533143112814">obs1</strong>.<ol type="a" id="dli_01_0002__ol182612310711"><li id="dli_01_0002__li462934116719">Click <span class="uicontrol" id="dli_01_0002__uicontrol481933104114"><b>Create Bucket</b></span> in the upper right corner.</li><li id="dli_01_0002__li183491949872">On the displayed <strong id="dli_01_0002__b997218337576">Create Bucket</strong> page, enter the <strong id="dli_01_0002__b41931653205710">Bucket Name</strong>. Retain the default values for other parameters or set them as required.<div class="note" id="dli_01_0002__note1145175833410"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0002__p7146205812341">You must select the same region as the DLI management console.</p>
</div></div>
</li><li id="dli_01_0002__li3583138155449">Click <strong id="dli_01_0002__b55859547573">Create Now</strong>.</li></ol>
</li><li id="dli_01_0002__li17845354105919">Click <strong id="dli_01_0002__b1314010449132">obs1</strong> to switch to the <strong id="dli_01_0002__b22171556131317">Overview</strong> page.</li><li id="dli_01_0002__li9783175118519">In the left navigation pane, click <strong id="dli_01_0002__b1083044015274">Objects</strong>. Click <strong id="dli_01_0002__b1830164072711">Upload Object</strong>. In the displayed dialog box, drag files or folders to upload or add file, for example, <span class="parmname" id="dli_01_0002__parmname11831040102710"><b>sampledata.csv</b></span> to the file upload box. Then, click <span class="uicontrol" id="dli_01_0002__uicontrol13831134072715"><b>Upload</b></span>.<div class="p" id="dli_01_0002__p1366220522517">You can create a <strong id="dli_01_0002__b19103192713017">sampledata.txt</strong> file, copy the following content separated by commas (,), and save the file as <strong id="dli_01_0002__b1597217291805">sampledata.csv</strong>.<pre class="screen" id="dli_01_0002__screen14345174815219">12,test</pre>
</div>
<p id="dli_01_0002__p1540915717295">After the file is uploaded successfully, the file path is <span class="filepath" id="dli_01_0002__filepath82411311208"><b>obs://obs1/sampledata.csv</b></span>.</p>
<p id="dli_01_0002__p34103573290">For more information about OBS operations, see the <em id="dli_01_0002__i15721531173210">Object Storage Service Console Operation Guide</em>.</p>
<p id="dli_01_0002__p17560428164613">For more information about the tool, see the <em id="dli_01_0002__i142921634153219">OBS Tool Guide</em>.</p>
<div class="note" id="dli_01_0002__note11115319367"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0002__p43881963392">You are advised to use an OBS tool, such as OBS Browser+, to upload large files because OBS Console has restrictions on the file size and quantity.</p>
<ul id="dli_01_0002__ul1234018993918"><li id="dli_01_0002__li17525130154011">OBS Browser+ is a graphical tool that provides complete functions for managing your buckets and objects in OBS.</li></ul>
<div class="note" id="dli_01_0002__note11115319367"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0002__ul1234018993918"><li id="dli_01_0002__li71801921936">For more information about OBS operations, see the <em id="dli_01_0002__i15721531173210">Object Storage Service Console Operation Guide</em>.</li><li id="dli_01_0002__li21801214311">For more information about the tool, see the <em id="dli_01_0002__i142921634153219">OBS Tool Guide</em>.</li><li id="dli_01_0002__li1418042110310">You are advised to use an OBS tool, such as OBS Browser+, to upload large files because OBS Console has restrictions on the file size and quantity.<ul id="dli_01_0002__ul569116231937"><li id="dli_01_0002__li17525130154011">OBS Browser+ is a graphical tool that provides complete functions for managing your buckets and objects in OBS.</li></ul>
</li></ul>
</div></div>
</li></ol>
</div>
<div class="section" id="dli_01_0002__section19012773105034"><a name="dli_01_0002__section19012773105034"></a><a name="section19012773105034"></a><h4 class="sectiontitle">Step 3: Logging In to the DLI Management Console</h4><ol id="dli_01_0002__ol33542655111116"><li id="dli_01_0002__li2096805415940">.</li><li id="dli_01_0002__li38964926151831">The DLI management console page is displayed. If you log in to the DLI management console for the first time, you need to be authorized to access OBS.<p id="dli_01_0002__p180191065412"><a name="dli_01_0002__li38964926151831"></a><a name="li38964926151831"></a>Note: For this guide, you need at least the <strong id="dli_01_0002__b8797105363018">Tenant Administrator (Global service)</strong> permission.</p>
</li></ol>
</div>
<div class="section" id="dli_01_0002__section10742144985011"><a name="dli_01_0002__section10742144985011"></a><a name="section10742144985011"></a><h4 class="sectiontitle">Step 4: Creating a Queue</h4><p id="dli_01_0002__p313914434915">A queue is the basis for using DLI. Before executing an SQL job, you need to create a queue.</p>
<ul id="dli_01_0002__ul17202247195010"><li id="dli_01_0002__li131931114812">An available queue <span class="parmvalue" id="dli_01_0002__parmvalue1369741014488"><b>default</b></span> is preset in DLI.</li><li id="dli_01_0002__li1131916110481">You can also create queues as needed.<ol id="dli_01_0002__ol15578144671515"><li id="dli_01_0002__li2578546131517">On the DLI management console, click <strong id="dli_01_0002__b16611756192915">SQL Editor</strong> in the navigation pane on the left. The <strong id="dli_01_0002__b1120919212301">SQL Editor</strong> page is displayed.</li><li id="dli_01_0002__li844914584169">On the left pane, select the <strong id="dli_01_0002__b8902351334">Queues</strong> tab, and click <span><img id="dli_01_0002__image17450135891614" src="en-us_image_0276441461.png"></span> next to <strong id="dli_01_0002__b1478271612">Queues</strong>.<p id="dli_01_0002__p17288510177">For details, see Creating a Queue.</p>
<div class="section" id="dli_01_0002__section10742144985011"><a name="dli_01_0002__section10742144985011"></a><a name="section10742144985011"></a><h4 class="sectiontitle">Step 2: Create a Queue</h4><p id="dli_01_0002__p313914434915">A queue is the basis for using DLI. Before executing an SQL job, you need to create a queue.</p>
<ul id="dli_01_0002__ul17202247195010"><li id="dli_01_0002__li131931114812">An available queue <span class="parmvalue" id="dli_01_0002__parmvalue1369741014488"><b>default</b></span> is preset in DLI.</li><li id="dli_01_0002__li1131916110481">You can also create queues as needed.<ol id="dli_01_0002__ol15578144671515"><li id="dli_01_0002__li13332820164119">Log in to the DLI management console.</li><li id="dli_01_0002__li2578546131517">On the DLI management console, click <strong id="dli_01_0002__b16611756192915">SQL Editor</strong> in the navigation pane on the left. The <strong id="dli_01_0002__b1120919212301">SQL Editor</strong> page is displayed.</li><li id="dli_01_0002__li844914584169">On the left pane, select the <strong id="dli_01_0002__b8902351334">Queues</strong> tab, and click <span><img id="dli_01_0002__image17450135891614" src="en-us_image_0276441461.png"></span> next to <strong id="dli_01_0002__b1478271612">Queues</strong>.<p id="dli_01_0002__p17288510177">For details, see Creating a Queue.</p>
<p id="dli_01_0002__p8128252161516"></p>
</li></ol>
</li></ul>
</div>
<div class="section" id="dli_01_0002__section21433273112656"><a name="dli_01_0002__section21433273112656"></a><a name="section21433273112656"></a><h4 class="sectiontitle">Step 5: Creating a Database</h4><p id="dli_01_0002__p498774501214">Before querying data, create a database, for example, <strong id="dli_01_0002__b84235270619941">db1</strong>.</p>
<div class="note" id="dli_01_0002__note23711139194010"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0002__p143724399404">The <strong id="dli_01_0002__b842352706172053">default</strong> database is a built-in database. You cannot create the database named <strong id="dli_01_0002__b842352706172112">default</strong>.</p>
<div class="section" id="dli_01_0002__section21433273112656"><a name="dli_01_0002__section21433273112656"></a><a name="section21433273112656"></a><h4 class="sectiontitle">Step 3: Create a Database</h4><p id="dli_01_0002__p498774501214">Before querying data, create a database, for example, <strong id="dli_01_0002__b84235270619941">db1</strong>.</p>
<div class="note" id="dli_01_0002__note23711139194010"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0002__p143724399404">The <strong id="dli_01_0002__b842352706172053">default</strong> database is a built-in database. You cannot create the <strong id="dli_01_0002__b842352706172112">default</strong>. database.</p>
</div></div>
<ol id="dli_01_0002__ol1216817105472"><li id="dli_01_0002__li11682102472">On the DLI management console, click <strong id="dli_01_0002__b1635163117400">SQL Editor</strong> in the navigation pane on the left. The <strong id="dli_01_0002__b12483735174016">SQL Editor</strong> page is displayed.</li><li id="dli_01_0002__li47441514124717">In the editing window on the right of the <strong id="dli_01_0002__b16368113112419">SQL Editor</strong> page, enter the following SQL statement and click <strong id="dli_01_0002__b1136814317247">Execute</strong>. Read and agree to the privacy agreement, and click <strong id="dli_01_0002__b2057914102316">OK</strong>.<pre class="screen" id="dli_01_0002__screen161415843616">create database db1;</pre>
<p id="dli_01_0002__p38732969153456">After database <strong id="dli_01_0002__b842352706161813">db1</strong> is successfully created, <strong id="dli_01_0002__b842352706161823">db1</strong> will be displayed in the <strong id="dli_01_0002__b842352706161834">Database</strong> list.</p>
@ -42,12 +37,12 @@
</div></div>
</li></ol>
</div>
<div class="section" id="dli_01_0002__section21590507141153"><a name="dli_01_0002__section21590507141153"></a><a name="section21590507141153"></a><h4 class="sectiontitle">Step 6: Creating a Table</h4><p id="dli_01_0002__p35220739144319">After database <strong id="dli_01_0002__b1827512525545">db1</strong> is created, create a table (for example, <strong id="dli_01_0002__b8275145215544">table1</strong>) containing data in the sample file <span class="filepath" id="dli_01_0002__filepath427515215419"><b>obs://obs1/sampledata.csv</b></span> stored on OBS in <strong id="dli_01_0002__b4277135245414">db1</strong>.</p>
<div class="section" id="dli_01_0002__section21590507141153"><a name="dli_01_0002__section21590507141153"></a><a name="section21590507141153"></a><h4 class="sectiontitle">Step 4: Create a Table</h4><p id="dli_01_0002__p35220739144319">After database <strong id="dli_01_0002__b1827512525545">db1</strong> is created, create a table (for example, <strong id="dli_01_0002__b8275145215544">table1</strong>) containing data in the sample file <span class="filepath" id="dli_01_0002__filepath427515215419"><b>obs://obs1/sampledata.csv</b></span> stored on OBS in <strong id="dli_01_0002__b4277135245414">db1</strong>.</p>
<ol id="dli_01_0002__ol39318463153849"><li id="dli_01_0002__li23198164153933">In the SQL editing window of the<span class="wintitle" id="dli_01_0002__wintitle1132135205811"><b> SQL Editor</b></span> page, select the <strong id="dli_01_0002__b9133145155814">default</strong> queue and database <strong id="dli_01_0002__b31330515816">db1</strong>.</li><li id="dli_01_0002__li22227453153849">Enter the following SQL statement in the job editor window and click <strong id="dli_01_0002__b8418103081114">Execute</strong>:<pre class="screen" id="dli_01_0002__screen114301935133717">create table table1 (id int, name string) using csv options (path 'obs://obs1/sampledata.csv');</pre>
<p id="dli_01_0002__p6761159153849">After the table is created, click the <strong id="dli_01_0002__b924464317219">Databases</strong> tab and then select <strong id="dli_01_0002__b16240727145816">db1</strong>. The created table <strong id="dli_01_0002__b122531636185811">table1</strong> is displayed in the <span class="parmname" id="dli_01_0002__parmname1926113527581"><b>Table</b></span> area.</p>
</li></ol>
</div>
<div class="section" id="dli_01_0002__section37788816112733"><a name="dli_01_0002__section37788816112733"></a><a name="section37788816112733"></a><h4 class="sectiontitle">Step 7: Querying Data</h4><p id="dli_01_0002__p489584411557">After performing the preceding steps, you can start querying data.</p>
<div class="section" id="dli_01_0002__section37788816112733"><a name="dli_01_0002__section37788816112733"></a><a name="section37788816112733"></a><h4 class="sectiontitle">Step 5: Query Data</h4><p id="dli_01_0002__p489584411557">After performing the preceding steps, you can start querying data.</p>
<ol id="dli_01_0002__ol6245542914546"><li id="dli_01_0002__li146528414546">In the <span class="parmname" id="dli_01_0002__parmname1356301610013"><b>Table</b></span> tab on the <span class="wintitle" id="dli_01_0002__wintitle556341610020"><b>SQL Editor</b></span> page, double-click the created table <strong id="dli_01_0002__b856419161100">table1</strong>. The SQL statement is automatically displayed in the SQL job editing window in the right pane. Run following statement to query 1,000 records in the <strong id="dli_01_0002__b1456411165017">table1</strong> table:<pre class="screen" id="dli_01_0002__screen18201181093815">select * from db1.table1 limit 1000;</pre>
</li><li id="dli_01_0002__li19582472145233">Click <strong id="dli_01_0002__b126126091410">Execute</strong>. The system starts the query.<p id="dli_01_0002__p2694027411345">After the SQL statement is executed successfully, you can view the query result in <strong id="dli_01_0002__b294691411285">View Result</strong> under the SQL job editing window.</p>
</li></ol>

View File

@ -25,7 +25,7 @@
<tbody><tr id="dli_01_0005__row55596491188"><td class="cellrowborder" valign="top" width="13%" headers="mcps1.3.3.2.2.2.1.2.4.1.1 "><p id="dli_01_0005__p1355884915188">Database Name</p>
</td>
<td class="cellrowborder" valign="top" width="72%" headers="mcps1.3.3.2.2.2.1.2.4.1.2 "><ul id="dli_01_0005__ul8558154951816"><li id="dli_01_0005__li175581049171817">The database name can contain only digits, letters, and underscores (_), but cannot contain only digits or start with an underscore (_).</li><li id="dli_01_0005__li3558849141817">The database name is case insensitive and cannot be left blank.</li><li id="dli_01_0005__li13558174914181">The length of the database name cannot exceed 128 characters.</li></ul>
<div class="note" id="dli_01_0005__note105591449171815"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0005__p195593494182">The <strong id="dli_01_0005__b842352706172053">default</strong> database is a built-in database. You cannot create the database named <strong id="dli_01_0005__b842352706172112">default</strong>.</p>
<div class="note" id="dli_01_0005__note105591449171815"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0005__p195593494182">The <strong id="dli_01_0005__b28441763415">default</strong> database is a built-in database. You cannot create the <strong id="dli_01_0005__b384414618417">default</strong>. database.</p>
</div></div>
</td>
<td class="cellrowborder" valign="top" width="15%" headers="mcps1.3.3.2.2.2.1.2.4.1.3 "><p id="dli_01_0005__p11558749171819">DB01</p>
@ -45,9 +45,9 @@
</p></li></ol>
</div>
<div class="section" id="dli_01_0005__section46221651175510"><h4 class="sectiontitle">Creating a Table</h4><p id="dli_01_0005__p56362480175534">Before creating a table, ensure that a database has been created.</p>
<ol id="dli_01_0005__ol12637600103738"><li id="dli_01_0005__li59972234153426"><span>You can create a table on either the <span class="wintitle" id="dli_01_0005__wintitle681473279"><b>Databases and Tables</b></span> page or the <span class="wintitle" id="dli_01_0005__wintitle892027565"><b>SQL Editor</b></span> page.</span><p><div class="note" id="dli_01_0005__note159813598457"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0005__p4599175984510">Datasource connection tables, such as View tables, HBase (MRS) tables, OpenTSDB (MRS) tables, DWS tables, RDS tables, and CSS tables, cannot be created. You can use SQL to create views and datasource connection tables. For details, see sections <strong id="dli_01_0005__b168941558103516">Creating a View</strong> and <strong id="dli_01_0005__b1989435873520">Creating a Datasource Connection Table</strong> in the <em id="dli_01_0005__i12894358133518">Data Lake Insight SQL Syntax Reference</em>.</p>
<ol id="dli_01_0005__ol12637600103738"><li id="dli_01_0005__li59972234153426"><span>You can create a table on either the <span class="wintitle" id="dli_01_0005__wintitle1824439563"><b>Databases and Tables</b></span> page or the <span class="wintitle" id="dli_01_0005__wintitle144676590"><b>SQL Editor</b></span> page.</span><p><div class="note" id="dli_01_0005__note159813598457"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0005__p4599175984510">Datasource connection tables, such as View tables, HBase (MRS) tables, OpenTSDB (MRS) tables, DWS tables, RDS tables, and CSS tables, cannot be created. You can use SQL to create views and datasource connection tables. For details, see sections <strong id="dli_01_0005__b168941558103516">Creating a View</strong> and <strong id="dli_01_0005__b1989435873520">Creating a Datasource Connection Table</strong> in the <em id="dli_01_0005__i12894358133518">Data Lake Insight SQL Syntax Reference</em>.</p>
</div></div>
<ul id="dli_01_0005__ul22405571153537"><li id="dli_01_0005__li323555153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle220605428"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0005__ol2911995153537"><li id="dli_01_0005__li26207957153537">On the left of the management console, choose <span class="uicontrol" id="dli_01_0005__uicontrol1559686145"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0005__uicontrol8597462415"><b>Databases and Tables</b></span>.</li><li id="dli_01_0005__li17500832153537">On the <strong id="dli_01_0005__b1211154675314">Databases and Tables</strong> page, select the database for which you want to create a table. In the <strong id="dli_01_0005__b10345811102717">Operation</strong> column, click <strong id="dli_01_0005__b12918326165019">More</strong> &gt; <strong id="dli_01_0005__b63873822719">Create Table</strong> to create a table in the current database.</li></ol>
<ul id="dli_01_0005__ul22405571153537"><li id="dli_01_0005__li323555153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle1498676755"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0005__ol2911995153537"><li id="dli_01_0005__li26207957153537">On the left of the management console, choose <span class="uicontrol" id="dli_01_0005__uicontrol1559686145"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0005__uicontrol8597462415"><b>Databases and Tables</b></span>.</li><li id="dli_01_0005__li17500832153537">On the <strong id="dli_01_0005__b1211154675314">Databases and Tables</strong> page, select the database for which you want to create a table. In the <strong id="dli_01_0005__b10345811102717">Operation</strong> column, click <strong id="dli_01_0005__b12918326165019">More</strong> &gt; <strong id="dli_01_0005__b63873822719">Create Table</strong> to create a table in the current database.</li></ol>
</li><li id="dli_01_0005__li8281288153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle8700317181735"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0005__ol7422733153537"><li id="dli_01_0005__li66804605153537">On the left of the management console, click <span class="uicontrol" id="dli_01_0005__uicontrol11208161745"><b>SQL Editor</b></span>.</li><li id="dli_01_0005__li17512154252514">In the navigation pane of the displayed <span class="wintitle" id="dli_01_0005__wintitle13614162313185"><b>SQL Editor</b></span> page, click <strong id="dli_01_0005__b4615162311183">Databases</strong>. You can create a table in either of the following ways:<ul id="dli_01_0005__ul782474512255"><li id="dli_01_0005__li1893119202619">Click a database name. In the <strong id="dli_01_0005__b11582159133716">Tables</strong> area, click <span><img id="dli_01_0005__image15644111617506" src="en-us_image_0237539075.png"></span> on the right to create a table in the current database.</li><li id="dli_01_0005__li16824154518252">Click <span><img id="dli_01_0005__image1322473935113" src="en-us_image_0237532018.png"></span> on the right of the database and choose <strong id="dli_01_0005__b135378142190">Create Table</strong> from the shortcut menu to create a table in the current database.</li></ul>
</li></ol>
</li></ul>
@ -71,7 +71,7 @@
</td>
<td class="cellrowborder" valign="top" width="71.45%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.2 "><p id="dli_01_0005__p41514219173521">Data storage location. Currently, DLI and OBS are supported.</p>
</td>
<td class="cellrowborder" valign="top" width="14.17%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.3 "><p id="dli_01_0005__p7208591173521">OBS</p>
<td class="cellrowborder" valign="top" width="14.17%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.3 "><p id="dli_01_0005__p7208591173521">DLI</p>
</td>
</tr>
<tr id="dli_01_0005__row16142559103738"><td class="cellrowborder" valign="top" width="14.38%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.1 "><p id="dli_01_0005__p32478935103738">Description</p>
@ -101,7 +101,7 @@
<tr id="dli_01_0005__row16943758105944"><td class="cellrowborder" valign="top" width="14.38%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.1 "><p id="dli_01_0005__p30267119105944">Type</p>
</td>
<td class="cellrowborder" valign="top" width="71.45%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.2 "><p id="dli_01_0005__p35717569105944">Data type of a column. This parameter corresponds to <span class="parmname" id="dli_01_0005__parmname4155795592938"><b>Column Name</b></span>.</p>
<ul id="dli_01_0005__ul2957726293032"><li id="dli_01_0005__li159154616209"><strong id="dli_01_0005__b12861523131117">string</strong>: The data is of the string type.</li><li id="dli_01_0005__li5035776893042"><strong id="dli_01_0005__b84235270617190">int</strong>: Each integer is stored on four bytes.</li><li id="dli_01_0005__li197172662015"><strong id="dli_01_0005__b71123713114">date</strong>: The value ranges from 0000-01-01 to 9999-12-31.</li><li id="dli_01_0005__li15276123414204"><strong id="dli_01_0005__b2073283811110">double</strong>: Each number is stored on eight bytes.</li><li id="dli_01_0005__li122191918102110"><strong id="dli_01_0005__b87406418113">boolean</strong>: Each value is stored on one byte.</li><li id="dli_01_0005__li616884422117"><strong id="dli_01_0005__b842352706172140">decimal</strong>: The valid bits are positive integers between 1 to 38, including 1 and 38. The decimal digits are integers less than 10.</li><li id="dli_01_0005__li6039945193127"><strong id="dli_01_0005__b1416819636">smallint/short</strong>: The number is stored on two bytes.</li><li id="dli_01_0005__li157282298387"><strong id="dli_01_0005__b1271693340">bigint/long</strong>: The number is stored on eight bytes.</li><li id="dli_01_0005__li4930135112217"><strong id="dli_01_0005__b1251694319111">timestamp</strong>: The data indicates a date and time. The value can be accurate to six decimal points.</li><li id="dli_01_0005__li482517952312"><strong id="dli_01_0005__b161442457112">float</strong>: Each number is stored on four bytes.</li><li id="dli_01_0005__li246310417404"><strong id="dli_01_0005__b0368141417399">tinyint</strong>: Each number is stored on one byte. Only OBS tables support this data type.</li></ul>
<ul id="dli_01_0005__ul2957726293032"><li id="dli_01_0005__li159154616209"><strong id="dli_01_0005__b12861523131117">string</strong>: The data is of the string type.</li><li id="dli_01_0005__li5035776893042"><strong id="dli_01_0005__b84235270617190">int</strong>: Each integer is stored on four bytes.</li><li id="dli_01_0005__li197172662015"><strong id="dli_01_0005__b71123713114">date</strong>: The value ranges from 0000-01-01 to 9999-12-31.</li><li id="dli_01_0005__li15276123414204"><strong id="dli_01_0005__b2073283811110">double</strong>: Each number is stored on eight bytes.</li><li id="dli_01_0005__li122191918102110"><strong id="dli_01_0005__b87406418113">boolean</strong>: Each value is stored on one byte.</li><li id="dli_01_0005__li616884422117"><strong id="dli_01_0005__b842352706172140">decimal</strong>: The valid bits are positive integers between 1 to 38, including 1 and 38. The decimal digits are integers less than 10.</li><li id="dli_01_0005__li6039945193127"><strong id="dli_01_0005__b1606967536">smallint/short</strong>: The number is stored on two bytes.</li><li id="dli_01_0005__li157282298387"><strong id="dli_01_0005__b461113533">bigint/long</strong>: The number is stored on eight bytes.</li><li id="dli_01_0005__li4930135112217"><strong id="dli_01_0005__b1251694319111">timestamp</strong>: The data indicates a date and time. The value can be accurate to six decimal points.</li><li id="dli_01_0005__li482517952312"><strong id="dli_01_0005__b161442457112">float</strong>: Each number is stored on four bytes.</li><li id="dli_01_0005__li246310417404"><strong id="dli_01_0005__b0368141417399">tinyint</strong>: Each number is stored on one byte. Only OBS tables support this data type.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="14.17%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.3 "><p id="dli_01_0005__p7441963105944">string</p>
</td>
@ -137,7 +137,7 @@
<tbody><tr id="dli_01_0005__row3015013618314"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p3709008118455">Data Format</p>
</td>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p5150660918455">DLI supports the following data formats:</p>
<ul id="dli_01_0005__ul4173603518455"><li id="dli_01_0005__li4007999518455"><strong id="dli_01_0005__b84235270614338">Parquet</strong>: DLI can read non-compressed data or data that is compressed using Snappy and gzip.</li><li id="dli_01_0005__li2517563618455"><strong id="dli_01_0005__b1059454224">CSV</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li2525413518455"><strong id="dli_01_0005__b496524445">ORC</strong>: DLI can read non-compressed data or data that is compressed using Snappy.</li><li id="dli_01_0005__li2596063018455"><strong id="dli_01_0005__b1424812439">JSON</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li20203144172320"><strong id="dli_01_0005__b14575133533814">Avro</strong>: DLI can read uncompressed Avro data.</li></ul>
<ul id="dli_01_0005__ul4173603518455"><li id="dli_01_0005__li4007999518455"><strong id="dli_01_0005__b84235270614338">Parquet</strong>: DLI can read non-compressed data or data that is compressed using Snappy and gzip.</li><li id="dli_01_0005__li2517563618455"><strong id="dli_01_0005__b1981815355">CSV</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li2525413518455"><strong id="dli_01_0005__b1789934416">ORC</strong>: DLI can read non-compressed data or data that is compressed using Snappy.</li><li id="dli_01_0005__li2596063018455"><strong id="dli_01_0005__b1180375596">JSON</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li20203144172320"><strong id="dli_01_0005__b14575133533814">Avro</strong>: DLI can read uncompressed Avro data.</li></ul>
</td>
<td class="cellrowborder" valign="top" width="30.330000000000002%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.3 "><p id="dli_01_0005__p2243627218455">CSV</p>
</td>
@ -170,7 +170,7 @@
</tr>
<tr id="dli_01_0005__row21739275111826"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p39917803111836">User-defined Quotation Character</p>
</td>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p056181715520">This parameter is valid only when <span class="parmname" id="dli_01_0005__parmname121957196"><b>Data Format</b></span> is set to <span class="parmvalue" id="dli_01_0005__parmvalue1415365822"><b>CSV</b></span> and you select <strong id="dli_01_0005__b446068245">User-defined Quotation Character</strong>.</p>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p056181715520">This parameter is valid only when <span class="parmname" id="dli_01_0005__parmname917259724"><b>Data Format</b></span> is set to <span class="parmvalue" id="dli_01_0005__parmvalue968546622"><b>CSV</b></span> and you select <strong id="dli_01_0005__b1515618674">User-defined Quotation Character</strong>.</p>
<p id="dli_01_0005__p57404199154711">The following quotation characters are supported:</p>
<ul id="dli_01_0005__ul41920423111836"><li id="dli_01_0005__li41739488111836">Single quotation mark (')</li><li id="dli_01_0005__li40111075111836">Double quotation marks (")</li><li id="dli_01_0005__li25455362111836">Others: Enter a user-defined quotation character.</li></ul>
</td>
@ -189,7 +189,7 @@
<tr id="dli_01_0005__row14602987103855"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p56877187104214">Date Format</p>
</td>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p1033432005318">This parameter is valid only when <span class="parmname" id="dli_01_0005__parmname101493527346"><b>Data Format</b></span> is set to <span class="parmvalue" id="dli_01_0005__parmvalue1149205243420"><b>CSV</b></span> or <span class="parmvalue" id="dli_01_0005__parmvalue1115045243420"><b>JSON</b></span>.</p>
<p id="dli_01_0005__p43649441104214">This parameter specifies the format of the date in the table and is valid only <span class="parmvalue" id="dli_01_0005__parmvalue193439967115458"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0005__parmname181884393215542"><b>yyyy-MM-dd</b></span>. For definition of characters involved in the date pattern, see Table 3 in the </p>
<p id="dli_01_0005__p43649441104214">This parameter specifies the format of the date in the table and is valid only <span class="parmvalue" id="dli_01_0005__parmvalue193439967115458"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0005__parmname181884393215542"><b>yyyy-MM-dd</b></span>. For definition of characters involved in the date pattern, see Table 3 in .</p>
</td>
<td class="cellrowborder" valign="top" width="30.330000000000002%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.3 "><p id="dli_01_0005__p10841672104214">2000-01-01</p>
</td>
@ -197,7 +197,7 @@
<tr id="dli_01_0005__row5732746810392"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p51841824104214">Timestamp Format</p>
</td>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p138917486534">This parameter is valid only when <span class="parmname" id="dli_01_0005__parmname1399715783416"><b>Data Format</b></span> is set to <span class="parmvalue" id="dli_01_0005__parmvalue9998145714349"><b>CSV</b></span> or <span class="parmvalue" id="dli_01_0005__parmvalue39981357133420"><b>JSON</b></span>.</p>
<p id="dli_01_0005__p38438192104214">This parameter specifies the format of the timestamp in the table and is valid only <span class="parmvalue" id="dli_01_0005__parmvalue10399413104214"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0005__parmname36794602143645"><b>yyyy-MM-dd HH:mm:ss</b></span>. For definition of characters involved in the time pattern, see Table 3 in the </p>
<p id="dli_01_0005__p38438192104214">This parameter specifies the format of the timestamp in the table and is valid only <span class="parmvalue" id="dli_01_0005__parmvalue10399413104214"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0005__parmname36794602143645"><b>yyyy-MM-dd HH:mm:ss</b></span>. For definition of characters involved in the time pattern, see Table 3 in .</p>
</td>
<td class="cellrowborder" valign="top" width="30.330000000000002%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.3 "><p id="dli_01_0005__p37046125104214">2000-01-01 09:00:00</p>
</td>
@ -206,7 +206,7 @@
</table>
</div>
</li></ul>
</p></li><li id="dli_01_0005__li6436679692218"><span>Click <span class="uicontrol" id="dli_01_0005__uicontrol1346028235151847"><b>OK</b></span>.</span><p><p id="dli_01_0005__p2255887315231">After a table is created, you can view and select the table for use on the <span class="wintitle" id="dli_01_0005__wintitle67811743"><b>Data Management</b></span> page or <strong id="dli_01_0005__b2075207989">SQL Editor</strong> page.</p>
</p></li><li id="dli_01_0005__li6436679692218"><span>Click <span class="uicontrol" id="dli_01_0005__uicontrol1346028235151847"><b>OK</b></span>.</span><p><p id="dli_01_0005__p2255887315231">After a table is created, you can view and select the table for use on the <span class="wintitle" id="dli_01_0005__wintitle1228761699"><b>Data Management</b></span> page or <strong id="dli_01_0005__b1747557251">SQL Editor</strong> page.</p>
</p></li><li id="dli_01_0005__li66082750104919"><span>(Optional) After a DLI table is created, you can decide whether to directly import data to the table.</span></li></ol>
</div>
</div>

View File

@ -6,40 +6,42 @@
</div></div>
<ol id="dli_01_0006__ol194665011112"><li id="dli_01_0006__li322142705519">Apply for a cluster in MRS.<p id="dli_01_0006__p1166656182213"><a name="dli_01_0006__li322142705519"></a><a name="li322142705519"></a>If a cluster is available, you do not need to apply for one.</p>
</li><li id="dli_01_0006__li918675917143">In the navigation pane of the DLI management console, choose <span class="wintitle" id="dli_01_0006__wintitle7647752205418"><b>Datasource Connections</b></span>.</li><li id="dli_01_0006__li16324435150">Click the <strong id="dli_01_0006__b196273113384">Enhanced</strong> tab and click <strong id="dli_01_0006__b1263315183820">Create</strong> in the upper left corner.<p id="dli_01_0006__p114601031134519">Enter the <strong id="dli_01_0006__b899536143813">Connection Name</strong>, select the <strong id="dli_01_0006__b099519611386">Bind Queue</strong> (optional), <strong id="dli_01_0006__b2099613612386">VPC</strong>, and <strong id="dli_01_0006__b1399614615382">Subnet</strong>, and enter the <strong id="dli_01_0006__b999614616388">Host Information</strong> (optional). For details about the parameters, see <a href="#dli_01_0006__table24931148155220">Table 1</a>.</p>
<p id="dli_01_0006__p155812303175"></p>
<p id="dli_01_0006__p158349011310"></p>
<div class="tablenoborder"><a name="dli_01_0006__table24931148155220"></a><a name="table24931148155220"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__table24931148155220" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0006__row1149712486527"><th align="left" class="cellrowborder" valign="top" width="15.920000000000002%" id="mcps1.3.1.4.3.5.2.3.1.1"><p id="dli_01_0006__p349916487526">Parameter</p>
<div class="tablenoborder"><a name="dli_01_0006__table24931148155220"></a><a name="table24931148155220"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0006__table24931148155220" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters</caption><thead align="left"><tr id="dli_01_0006__row1149712486527"><th align="left" class="cellrowborder" valign="top" width="15.920000000000002%" id="mcps1.3.1.4.3.6.2.3.1.1"><p id="dli_01_0006__p349916487526">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="84.08%" id="mcps1.3.1.4.3.5.2.3.1.2"><p id="dli_01_0006__p115011548105211">Description</p>
<th align="left" class="cellrowborder" valign="top" width="84.08%" id="mcps1.3.1.4.3.6.2.3.1.2"><p id="dli_01_0006__p115011548105211">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0006__row1350324845215"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.5.2.3.1.1 "><p id="dli_01_0006__p8504184814524">Connection Name</p>
<tbody><tr id="dli_01_0006__row1350324845215"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.6.2.3.1.1 "><p id="dli_01_0006__p8504184814524">Connection Name</p>
</td>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.5.2.3.1.2 "><p id="dli_01_0006__p1550604814528">Name of the created datasource connection.</p>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.6.2.3.1.2 "><p id="dli_01_0006__p1550604814528">Name of the created datasource connection.</p>
<ul id="dli_01_0006__ul185072486523"><li id="dli_01_0006__li650717489523">The name can contain only letters, digits, and underscores (_). The parameter must be specified.</li><li id="dli_01_0006__li1350913489529">A maximum of 64 characters are allowed.</li></ul>
</td>
</tr>
<tr id="dli_01_0006__row113282120419"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.5.2.3.1.1 "><p id="dli_01_0006__p632112174118">Bound Queue </p>
<tr id="dli_01_0006__row3401103533514"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.6.2.3.1.1 "><p id="dli_01_0006__p64021535183519">Resource Pool</p>
</td>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.5.2.3.1.2 "><p id="dli_01_0006__p133212111414">(Optional) Bind a queue that requires datasource connections.</p>
<div class="note" id="dli_01_0006__note13421165512496"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__p943095519492">Before using an enhanced datasource connection, you must bind a queue and ensure that the VPC peering connection is in the <strong id="dli_01_0006__b11412161014431">Active</strong> state.</p>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.6.2.3.1.2 "><p id="dli_01_0006__p12402335203516">It binds an elastic resource pool or queue that uses a datasource connection. This parameter is optional.</p>
<p id="dli_01_0006__p2025312143719">In regions where this function is available, a resource pool with the same name is created by default for the queue created in "Creating a Queue." </p>
<div class="note" id="dli_01_0006__note14747353115719"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__p2074812538571">Before using an enhanced datasource connection, you must bind a queue and ensure that the VPC peering connection is in the <strong id="dli_01_0006__b51961843131119">Active</strong> state.</p>
</div></div>
</td>
</tr>
<tr id="dli_01_0006__row7764655142317"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.5.2.3.1.1 "><p id="dli_01_0006__p16764105532311">VPC</p>
<tr id="dli_01_0006__row7764655142317"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.6.2.3.1.1 "><p id="dli_01_0006__p16764105532311">VPC</p>
</td>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.5.2.3.1.2 "><p id="dli_01_0006__p1676416559237">VPC used by the destination data source.</p>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.6.2.3.1.2 "><p id="dli_01_0006__p1676416559237">VPC used by the destination data source.</p>
</td>
</tr>
<tr id="dli_01_0006__row1095810374248"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.5.2.3.1.1 "><p id="dli_01_0006__p1495903712415">Subnet</p>
<tr id="dli_01_0006__row1095810374248"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.6.2.3.1.1 "><p id="dli_01_0006__p1495903712415">Subnet</p>
</td>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.5.2.3.1.2 "><p id="dli_01_0006__p1795918371243">Subnet used by the destination data source.</p>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.6.2.3.1.2 "><p id="dli_01_0006__p1795918371243">Subnet used by the destination data source.</p>
</td>
</tr>
<tr id="dli_01_0006__row11771019513"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.5.2.3.1.1 "><p id="dli_01_0006__p41891015518">Host Information</p>
<tr id="dli_01_0006__row11771019513"><td class="cellrowborder" valign="top" width="15.920000000000002%" headers="mcps1.3.1.4.3.6.2.3.1.1 "><p id="dli_01_0006__p41891015518">Host Information</p>
</td>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.5.2.3.1.2 "><p id="dli_01_0006__p1184104510">(Optional) When connecting to the HBase cluster of MRS, enter the host name and IP address of the ZooKeeper instance. Enter one record in each line. The format is as follows: <strong id="dli_01_0006__b6308101634416">IP address Host name/Domain name</strong>.</p>
<td class="cellrowborder" valign="top" width="84.08%" headers="mcps1.3.1.4.3.6.2.3.1.2 "><p id="dli_01_0006__p1184104510">(Optional) When connecting to the HBase cluster of MRS, enter the host name and IP address of the ZooKeeper instance. Enter one record in each line. The format is as follows: <strong id="dli_01_0006__b6308101634416">IP address Host name/Domain name</strong>.</p>
<p id="dli_01_0006__p2011023525">To obtain the host name and IP address of the MRS cluster, perform the following steps (with MRS3.x as an example):</p>
<ol type="a" id="dli_01_0006__ol34131830476"><li id="dli_01_0006__li24136334712">Log in to the MRS management console.</li><li id="dli_01_0006__li14136317472">In the navigation pane, choose <strong id="dli_01_0006__b20689123710445">Clusters</strong> &gt; <strong id="dli_01_0006__b7695737114411">Active Clusters</strong>. Click the target cluster name to access the cluster details page.</li><li id="dli_01_0006__li741363194711">Click <span class="wintitle" id="dli_01_0006__wintitle499972615559"><b>Component Management</b></span>.</li><li id="dli_01_0006__li174136317476">Click <strong id="dli_01_0006__b196843014459">Zookeeper</strong>.</li><li id="dli_01_0006__li114131237471">Click the <span class="wintitle" id="dli_01_0006__wintitle16169762457"><b>Instance</b></span> tab to view the corresponding service IP address. You can select any service IP address.</li></ol>
<div class="note" id="dli_01_0006__note1426252593"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0006__p142655145920">If the MRS cluster has multiple IP addresses, enter any service IP address when creating a datasource connection.</p>
@ -50,7 +52,7 @@
</table>
</div>
</li><li id="dli_01_0006__li9142144141616">Click <span class="uicontrol" id="dli_01_0006__uicontrol1787622894613"><b>OK</b></span>.</li><li id="dli_01_0006__li137440184715">To connect to Kafka, DWS, and RDS instances, add security group rules for the DLI network segment to the security group where the instances belong. The following uses RDS as an example:<ol type="a" id="dli_01_0006__ol89018481616"><li id="dli_01_0006__li3500123095914">Choose <strong id="dli_01_0006__b35631153204514">Resources</strong> &gt; <strong id="dli_01_0006__b103961815165711">Queue Management</strong> in the navigation pane. Select the target queue, and click <span><img id="dli_01_0006__image105154539180" src="en-us_image_0000001259009999.png"></span> to expand the row containing the target queue to view its CIDR block.</li><li id="dli_01_0006__li15781102518551">On the <strong id="dli_01_0006__b87522541614">Instance Management</strong> page of the RDS console, click the instance name. In the <strong id="dli_01_0006__b3276633424">Connection Information</strong> area, view the port number of the RDS database instance.</li><li id="dli_01_0006__li88331728195516">In the <strong id="dli_01_0006__b17744122491213">Connection Information</strong> area locate the<strong id="dli_01_0006__b5747714161312"> Security Group</strong> and click the group name to switch to the security group management page. Select the <strong id="dli_01_0006__b446210532137">Inbound Rules</strong> tab and click <strong id="dli_01_0006__b196062591418">Add Rule</strong>. Set the priority to 1, protocol to TCP, port to the database port number, and source to the CIDR block of the DLI queue. Click <strong id="dli_01_0006__b4416755131517">OK</strong>.</li></ol>
</li><li id="dli_01_0006__li68345507185">Test the connectivity between the DLI queue and the connection instance. The following example describes how to test the connectivity between DLI and an RDS DB instance.<ol type="a" id="dli_01_0006__ol19285185311814"><li id="dli_01_0006__li112310149192">On the <strong id="dli_01_0006__b1673917298175">Instance Management</strong> page, click the target DB instance. On the displayed page, locate the <strong id="dli_01_0006__b177458291174">Connection Information</strong> pane and view the floating IP address. In the <strong id="dli_01_0006__b1434312191914">Connection Information</strong> pane, locate the <strong id="dli_01_0006__b11459142202015">Database Port</strong> to view the port number of the RDS DB instance.</li><li id="dli_01_0006__li10880157142314">Go back to the DLI console. On the <strong id="dli_01_0006__b737654518310">Resources</strong> &gt; <strong id="dli_01_0006__b5653513132015">Queue Management</strong> page, locate the target queue. In the <strong id="dli_01_0006__b19660201310208">Operation</strong> column, click <strong id="dli_01_0006__b1366081302018">More</strong> &gt; <strong id="dli_01_0006__b266061312019">Test Address Connectivity</strong>.</li><li id="dli_01_0006__li15869145571811">Enter the connection address of the RDS DB instance and port number in the format of <strong id="dli_01_0006__b0259173692111">IP address</strong><strong id="dli_01_0006__b10485193932113">:port</strong> to test the network connectivity.</li></ol>
</li><li id="dli_01_0006__li68345507185">Test the connectivity between the DLI queue and the connection instance. The following example describes how to test the connectivity between DLI and an RDS DB instance.<ol type="a" id="dli_01_0006__ol19285185311814"><li id="dli_01_0006__li112310149192">On the <strong id="dli_01_0006__b1673917298175">Instance Management</strong> page, click the target DB instance. On the displayed page, locate the <strong id="dli_01_0006__b177458291174">Connection Information</strong> pane and view the floating IP address. In the <strong id="dli_01_0006__b1434312191914">Connection Information</strong> pane, locate the <strong id="dli_01_0006__b11459142202015">Database Port</strong> to view the port number of the RDS DB instance.</li><li id="dli_01_0006__li10880157142314">Go back to the DLI console. On the <strong id="dli_01_0006__b737654518310">Resources</strong> &gt; <strong id="dli_01_0006__b5653513132015">Queue Management</strong> page, locate the target queue. In the <strong id="dli_01_0006__b19660201310208">Operation</strong> column, click <strong id="dli_01_0006__b1366081302018">More</strong> and select <strong id="dli_01_0006__b266061312019">Test Address Connectivity</strong>.</li><li id="dli_01_0006__li15869145571811">Enter the connection address of the RDS DB instance and port number in the format of <strong id="dli_01_0006__b0259173692111">IP address</strong><strong id="dli_01_0006__b10485193932113">:port</strong> to test the network connectivity.</li></ol>
</li></ol>
</div>
<div class="section" id="dli_01_0006__section9644161019415"><h4 class="sectiontitle">Querying an Enhanced Datasource Connection</h4><p id="dli_01_0006__p049105611378">On the <span class="wintitle" id="dli_01_0006__wintitle1323715521461"><b>Enhanced</b></span> tab page, you can enter the keyword of a connection name in the search box to search for the matching connection.</p>

View File

@ -5,61 +5,61 @@
<div class="section" id="dli_01_0010__section181111466568"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0010__ul12225587299"><li id="dli_01_0010__li0222105872918">The exported file can be in JSON format, and the text format can only be UTF-8.</li><li id="dli_01_0010__li6222205822916">Only the data in the DLI table (the table type is <strong id="dli_01_0010__b17552458537">Managed</strong>) can be exported to the OBS bucket, and the export path must be specified to the folder level.</li><li id="dli_01_0010__li675153163611">Data can be exported across accounts. That is, after account B authorizes account A, account A can export data to the OBS path of account B if account A has the permission to read the metadata and permission information about the OBS bucket of account B and read and write the path.</li></ul>
</div>
<div class="section" id="dli_01_0010__section42958999144515"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0010__ol16716195143659"><li id="dli_01_0010__li2078303115103"><span>You can export data on either the <span class="wintitle" id="dli_01_0010__wintitle189235461532"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0010__wintitle392464655315"><b>SQL Editor</b></span> page.</span><p><ul id="dli_01_0010__ul13952860151010"><li id="dli_01_0010__li20669721151337">To export data on the <span class="wintitle" id="dli_01_0010__wintitle4564762181553"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0010__ol65658627151340"><li id="dli_01_0010__li16879536151342">On the left of the management console, choose <span class="uicontrol" id="dli_01_0010__uicontrol1582814278819"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0010__uicontrol882811271784"><b>Databases and Tables</b></span>.</li><li id="dli_01_0010__li29714355165922">Click the name of the database corresponding to the table where data is to be exported to switch to the <strong id="dli_01_0010__b681715115537">Manage Tables</strong> page.</li><li id="dli_01_0010__li14811172414512">Select the corresponding table (DLI table) and choose <strong id="dli_01_0010__b03118415457">More</strong> &gt; <strong id="dli_01_0010__b731104154514">Export</strong> in the <strong id="dli_01_0010__b123220419453">Operation</strong> column. The <span class="wintitle" id="dli_01_0010__wintitle63234104516"><b>Export Data</b></span> page is displayed.</li></ol>
</li><li id="dli_01_0010__li10384371151118">To export data on the <span class="wintitle" id="dli_01_0010__wintitle134925579488"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0010__ol39382464151459"><li id="dli_01_0010__li26745450151459">On the left of the management console, click <span class="uicontrol" id="dli_01_0010__uicontrol7578153418814"><b>SQL Editor</b></span>.</li><li id="dli_01_0010__li1773513617043">In the navigation tree on the left, click <strong id="dli_01_0010__b20683530717">Databases</strong> to see all databases. Click the database name corresponding to the table to which data is to be exported. The tables are displayed.</li><li id="dli_01_0010__li2614201531">Click <span><img id="dli_01_0010__image10601210194513" src="en-us_image_0237994910.png"></span> on the right of the managed table (DLI table) whose data is to be exported, and choose <strong id="dli_01_0010__b1860815103455">Export</strong> from the shortcut menu.<p id="dli_01_0010__p584671624616"></p>
</li></ol>
</li><li id="dli_01_0010__li10384371151118">To export data on the <span class="wintitle" id="dli_01_0010__wintitle134925579488"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0010__ol39382464151459"><li id="dli_01_0010__li26745450151459">On the left of the management console, click <span class="uicontrol" id="dli_01_0010__uicontrol7578153418814"><b>SQL Editor</b></span>.</li><li id="dli_01_0010__li1773513617043">In the navigation tree on the left, click <strong id="dli_01_0010__b20683530717">Databases</strong> to see all databases. Click the database name corresponding to the table to which data is to be exported. The tables are displayed.</li><li id="dli_01_0010__li2614201531">Click <span><img id="dli_01_0010__image10601210194513" src="en-us_image_0237994910.png"></span> on the right of the managed table (DLI table) whose data is to be exported, and choose <strong id="dli_01_0010__b1860815103455">Export</strong> from the shortcut menu.</li></ol>
</li></ul>
</p></li><li id="dli_01_0010__li10037634143659"><span>In the displayed <span class="wintitle" id="dli_01_0010__wintitle4856260395119"><b>Export Data</b></span> dialog box, specify parameters by referring to <a href="#dli_01_0010__table7742063143659">Table 1</a>.</span><p>
<div class="tablenoborder"><a name="dli_01_0010__table7742063143659"></a><a name="table7742063143659"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0010__table7742063143659" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0010__row48986708143659"><th align="left" class="cellrowborder" valign="top" width="18.57%" id="mcps1.3.3.2.2.2.1.2.3.1.1"><p id="dli_01_0010__p8500389143659">Parameter</p>
</p></li><li id="dli_01_0010__li10037634143659"><span>In the displayed <span class="wintitle" id="dli_01_0010__wintitle1477194264819"><b>Export Data</b></span> dialog box, specify parameters by referring to <a href="#dli_01_0010__table1903113115020">Table 1</a>.</span><p><div class="p" id="dli_01_0010__p25213331808">
<div class="tablenoborder"><a name="dli_01_0010__table1903113115020"></a><a name="table1903113115020"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0010__table1903113115020" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0010__row18963311404"><th align="left" class="cellrowborder" valign="top" width="13.8%" id="mcps1.3.3.2.2.2.1.1.2.3.1.1"><p id="dli_01_0010__p4896631504">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="81.43%" id="mcps1.3.3.2.2.2.1.2.3.1.2"><p id="dli_01_0010__p17442940143659">Description</p>
<th align="left" class="cellrowborder" valign="top" width="86.2%" id="mcps1.3.3.2.2.2.1.1.2.3.1.2"><p id="dli_01_0010__p389619311604">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0010__row55162434145333"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p21307823145337">Databases</p>
<tbody><tr id="dli_01_0010__row390033110014"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p8896183120014">Databases</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p48212085145337">Database where the current table is located.</p>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p1789763113015">Database where the current table is located.</p>
</td>
</tr>
<tr id="dli_01_0010__row54786783145255"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p5670283814532">Table Name</p>
<tr id="dli_01_0010__row1490043115016"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p1190073115015">Table Name</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p2952718314532">Name of the current table.</p>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p79001931203">Name of the current table.</p>
</td>
</tr>
<tr id="dli_01_0010__row59287839143659"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p37585653143659">Data Format</p>
<tr id="dli_01_0010__row1290013117017"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p39003313012">Data Format</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p24539023143659">Format of the file storing data to be exported. Formats other than JSON will be supported in later versions.</p>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p129002031005">Format of the file storing data to be exported. Formats other than JSON will be supported in later versions.</p>
</td>
</tr>
<tr id="dli_01_0010__row33984858114535"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p1310090114535">Queue</p>
<tr id="dli_01_0010__row1390016314017"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p139003311011">Queue</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p39008475114535">Select a queue.</p>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p2900931201">Select a queue.</p>
</td>
</tr>
<tr id="dli_01_0010__row1774342414552"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p2547309214552">Compression Format</p>
<tr id="dli_01_0010__row290212314015"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p119001331308">Compression Format</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p5005459614552">Compression format of the data to be exported. The following compression formats are supported:</p>
<ul id="dli_01_0010__ul35000658144913"><li id="dli_01_0010__li26407599145154">none</li><li id="dli_01_0010__li15808987144913">bzip2</li><li id="dli_01_0010__li39569454144913">deflate</li><li id="dli_01_0010__li7424986144913">gzip</li></ul>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p109005311408">Compression format of the data to be exported. The following compression formats are supported:</p>
<ul id="dli_01_0010__ul1490212318013"><li id="dli_01_0010__li890118311107">none</li><li id="dli_01_0010__li149019311702">bzip2</li><li id="dli_01_0010__li7901133117017">deflate</li><li id="dli_01_0010__li29018313018">gzip</li></ul>
</td>
</tr>
<tr id="dli_01_0010__row6367025143659"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p3346061614541">Storage Path</p>
<tr id="dli_01_0010__row10902131809"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p10902113112020">Storage Path</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><ul id="dli_01_0010__ul194291955145519"><li id="dli_01_0010__li74291755195510">Enter or select an OBS path.</li><li id="dli_01_0010__li94291755205513">The export path must be a folder that does not exist in the OBS bucket. Specifically, you need to create a folder in the target OBS directory.</li><li id="dli_01_0010__li194294550559">The folder name cannot contain the special characters of <strong id="dli_01_0010__b198865712220">\/:*?</strong> <strong id="dli_01_0010__b52895136212">"&lt;&gt;|</strong>, and cannot start or end with a dot (.).</li></ul>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><ul id="dli_01_0010__ul9902331502"><li id="dli_01_0010__li3902103116018">Enter or select an OBS path.</li><li id="dli_01_0010__li1690212313018">The export path must be a folder that does not exist in the OBS bucket. Specifically, you need to create a folder in the target OBS directory.</li><li id="dli_01_0010__li1590215311802">The folder name cannot contain the special characters of <strong id="dli_01_0010__b198865712220">\/:*?</strong> <strong id="dli_01_0010__b52895136212">"&lt;&gt;|</strong>, and cannot start or end with a dot (.).</li></ul>
</td>
</tr>
<tr id="dli_01_0010__row48430784114641"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p30579455114641">Export Mode</p>
<tr id="dli_01_0010__row109032318019"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p179020313017">Export Mode</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p61016786114641">Storage mode of the data to be exported.</p>
<ul id="dli_01_0010__ul625034191496"><li id="dli_01_0010__li1968673914926"><strong id="dli_01_0010__b1864119122549">New OBS directory</strong>: If the specified export directory exists, an error is reported and the export operation cannot be performed.</li><li id="dli_01_0010__li7475124010252"><strong id="dli_01_0010__b1138281495413">Existing OBS directory (Overwritten)</strong>: If you create a file in the specified directory, the existing file will be overwritten.</li></ul>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p59028311707">Storage mode of the data to be exported.</p>
<ul id="dli_01_0010__ul1090373111018"><li id="dli_01_0010__li890220311402"><strong id="dli_01_0010__b1864119122549">New OBS directory</strong>: If the specified export directory exists, an error is reported and the export operation cannot be performed.</li><li id="dli_01_0010__li690318315014"><strong id="dli_01_0010__b1138281495413">Existing OBS directory (Overwritten)</strong>: If you create a file in the specified directory, the existing file will be overwritten.</li></ul>
</td>
</tr>
<tr id="dli_01_0010__row1218154413337"><td class="cellrowborder" valign="top" width="18.57%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0010__p44720296144515">Table Header: No/Yes</p>
<tr id="dli_01_0010__row79031731005"><td class="cellrowborder" valign="top" width="13.8%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.1 "><p id="dli_01_0010__p19039310010">Table Header: No/Yes</p>
</td>
<td class="cellrowborder" valign="top" width="81.43%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><p id="dli_01_0010__p15245155210525">Whether the data to be exported contains the table header.</p>
<td class="cellrowborder" valign="top" width="86.2%" headers="mcps1.3.3.2.2.2.1.1.2.3.1.2 "><p id="dli_01_0010__p10903531805">Whether the data to be exported contains the table header.</p>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</p></li><li id="dli_01_0010__li48215562113558"><span>Click <span class="uicontrol" id="dli_01_0010__uicontrol1346108317431"><b>OK</b></span>.</span></li><li id="dli_01_0010__li7840124181212"><span>(Optional) You can view the job status (indicated by <span class="parmname" id="dli_01_0010__parmname158399481215"><b>Status</b></span>), statements (indicated by <span class="parmname" id="dli_01_0010__parmname883964151211"><b>Statement</b></span>), and other information about exporting jobs on the <span class="wintitle" id="dli_01_0010__wintitle283934191210"><b>SQL Jobs</b></span> page.</span><p><ol type="a" id="dli_01_0010__ol484010416125"><li id="dli_01_0010__li9839540126">Select <span class="parmvalue" id="dli_01_0010__parmvalue1650172515298"><b>EXPORT</b></span> from the <span class="parmname" id="dli_01_0010__parmname13789193911292"><b>Job Type</b></span> drop-down list box and specify the time range for exporting data. The jobs meeting the requirements are displayed in the job list.</li><li id="dli_01_0010__li28407441211">Click <span><img id="dli_01_0010__image1166513199136" src="en-us_image_0206789824.png"></span> to view details about an exporting job.</li></ol>
</p></li></ol>
</div>

View File

@ -56,7 +56,7 @@
</li><li id="dli_01_0017__li821851916016"><strong id="dli_01_0017__b8637165181412">Re-execute</strong>: Execute the job again.</li><li id="dli_01_0017__li12875164791814"><strong id="dli_01_0017__b10242547150">SparkUI</strong>: Display the Spark job execution page.<div class="note" id="dli_01_0017__note151024818191"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0017__ul55667208198"><li id="dli_01_0017__li956713205198">When you execute a job on a created queue, the cluster is restarted. It takes about 10 minutes. If you click <strong id="dli_01_0017__b1274243225511">SparkUI</strong> before the cluster is created, an empty <strong id="dli_01_0017__b12742203215512">projectID</strong> will be cached. The SparkUI page cannot be displayed. You are advised to use a dedicated queue so that the cluster will not be released. Alternatively, wait for a while after the job is submitted (the cluster is created), and then check <strong id="dli_01_0017__b1381493675517">SparkUI</strong>.</li><li id="dli_01_0017__li1359102371912">Currently, only the latest 100 job information records are displayed on the SparkUI of DLI.</li></ul>
</div></div>
</li></ul>
<ul id="dli_01_0017__ul1875134719182"><li id="dli_01_0017__li3883129255">In addition to the preceding operations, the following operations are available for QUERY jobs and asynchronous DDL jobs.<ul id="dli_01_0017__ul14392109145620"><li id="dli_01_0017__li38468323121"><strong id="dli_01_0017__b53531757153820">View Result</strong>: View the job running result.</li><li id="dli_01_0017__li37817583123"><strong id="dli_01_0017__b55953114394">Export Result</strong>: Export the job running result to the created OBS bucket. For details, see section <a href="#dli_01_0017__section1152211221244">Exporting Query Results</a>.</li></ul>
<ul id="dli_01_0017__ul1875134719182"><li id="dli_01_0017__li3883129255">In addition to the preceding operations, the following operations are available for QUERY jobs and asynchronous DDL jobs.<ul id="dli_01_0017__ul14392109145620"><li id="dli_01_0017__li38468323121"><strong id="dli_01_0017__b53531757153820">View Result</strong>: View the job running result.</li><li id="dli_01_0017__li37817583123"><strong id="dli_01_0017__b55953114394">Export Result</strong>: Export the job running result to the created OBS bucket. For details, see <a href="#dli_01_0017__section1152211221244">Exporting Query Results</a>.</li></ul>
</li><li id="dli_01_0017__li98421757121616">In addition to the preceding operations, the EXPORT job also includes the following operations:<ul id="dli_01_0017__ul1384275714169"><li id="dli_01_0017__li1684214571164"><strong id="dli_01_0017__b1972253413916">Download</strong></li></ul>
</li><li id="dli_01_0017__li16809489135"><strong id="dli_01_0017__b674619324407">View Log</strong>: Save job logs to the temporary OBS bucket created by DLI.</li><li id="dli_01_0017__li113222516517"><strong id="dli_01_0017__b5645835124010">Export Log</strong>: Export logs to the created OBS bucket. If the job is in the <strong id="dli_01_0017__b19611619124114">Running</strong> state, logs cannot be exported.</li></ul>
<div class="note" id="dli_01_0017__note9317379552"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0017__p1633103716553">To export the logs, you need to obtain the permission to create an OBS bucket.</p>

View File

@ -1,12 +1,11 @@
<a name="dli_01_0021"></a><a name="dli_01_0021"></a>
<h1 class="topictitle1">SQL Template Management</h1>
<h1 class="topictitle1">Managing SQL Templates</h1>
<div id="body1501038213757"><p id="dli_01_0021__p60316341134840">To facilitate SQL operation execution, DLI allows you to customize query templates or save the SQL statements in use as templates. After templates are saved, you do not need to compile SQL statements. Instead, you can directly perform the SQL operations using the templates.</p>
<p id="dli_01_0021__p141731510133114">SQL templates include sample templates and custom templates. The default sample template contains 22 standard TPC-H query statements, which can meet most TPC-H test requirements. For details, see <a href="dli_01_05111.html">TPC-H Sample Data in the SQL Template</a>.</p>
<p id="dli_01_0021__p835610562472">SQL template management provides the following functions:</p>
<ul id="dli_01_0021__ul32249213483"><li id="dli_01_0021__li208256193516"><a href="#dli_01_0021__section1039614583412">Sample Templates</a></li><li id="dli_01_0021__li829193111513"><a href="#dli_01_0021__section1616314111518">Custom Templates</a></li><li id="dli_01_0021__li64311610480"><a href="#dli_01_0021__section73391334165211">Creating a Template</a></li><li id="dli_01_0021__li1522410210481"><a href="#dli_01_0021__section1936164995213">Executing the Template</a></li><li id="dli_01_0021__li8722114154811"><a href="#dli_01_0021__section1045610354536">Searching for a Template</a></li><li id="dli_01_0021__li339093964717"><a href="#dli_01_0021__section08698165316">Modifying a Template</a></li><li id="dli_01_0021__li229014717477"><a href="#dli_01_0021__section1317681345320">Deleting a Template</a></li></ul>
<div class="section" id="dli_01_0021__section18354320771"><h4 class="sectiontitle">Table Settings</h4><p id="dli_01_0021__p95181419112418">In the upper right corner of the <strong id="dli_01_0021__b5641183711342">SQL Template</strong> page, click <strong id="dli_01_0021__b127917579344">Set Property</strong> to determine whether to display templates by group.</p>
<p id="dli_01_0021__p113871648193610"></p>
<p id="dli_01_0021__p1922162613714">If you select <strong id="dli_01_0021__b15906720113518">Display by Group</strong>, the following display modes are available:</p>
<ul id="dli_01_0021__ul5104175292412"><li id="dli_01_0021__li18104185213240">Expand the first group</li><li id="dli_01_0021__li1999935892413">Expand all</li><li id="dli_01_0021__li159834397299">Collapse All</li></ul>
</div>
@ -71,7 +70,7 @@
</tr>
<tr id="dli_01_0021__row1662880815250"><td class="cellrowborder" valign="top" width="14.09%" headers="mcps1.3.7.3.2.3.1.1 "><p id="dli_01_0021__p475621615250">Operation</p>
</td>
<td class="cellrowborder" valign="top" width="85.91%" headers="mcps1.3.7.3.2.3.1.2 "><ul id="dli_01_0021__ul15800707615"><li id="dli_01_0021__li178003014613"><strong id="dli_01_0021__b317417175">Execute</strong>: After you click this button, the system switches to the <strong id="dli_01_0021__b618725154">SQL Editor</strong> page, where you can modify or directly perform the statement as required. For details, see <a href="#dli_01_0021__section1936164995213">Executing the Template</a>.</li><li id="dli_01_0021__li3859410272"><strong id="dli_01_0021__b69311255148">Modify</strong>: Click <strong id="dli_01_0021__b1635885817144">Modify</strong>. In the displayed <span class="wintitle" id="dli_01_0021__wintitle197439472448"><b>Modify Template</b></span> dialog box, modify the template information as required. For details, see <a href="#dli_01_0021__section08698165316">Modifying a Template</a>.</li></ul>
<td class="cellrowborder" valign="top" width="85.91%" headers="mcps1.3.7.3.2.3.1.2 "><ul id="dli_01_0021__ul15800707615"><li id="dli_01_0021__li178003014613"><strong id="dli_01_0021__b2116946048">Execute</strong>: After you click this button, the system switches to the <strong id="dli_01_0021__b810727484">SQL Editor</strong> page, where you can modify or directly perform the statement as required. For details, see <a href="#dli_01_0021__section1936164995213">Executing the Template</a>.</li><li id="dli_01_0021__li3859410272"><strong id="dli_01_0021__b69311255148">Modify</strong>: Click <strong id="dli_01_0021__b1635885817144">Modify</strong>. In the displayed <span class="wintitle" id="dli_01_0021__wintitle197439472448"><b>Modify Template</b></span> dialog box, modify the template information as required. For details, see <a href="#dli_01_0021__section08698165316">Modifying a Template</a>.</li></ul>
</td>
</tr>
</tbody>

View File

@ -36,7 +36,6 @@
<tr id="dli_01_0228__row15599739103713"><td class="cellrowborder" valign="top" width="19.35%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_01_0228__p47271145191218">Enterprise Project</p>
</td>
<td class="cellrowborder" valign="top" width="80.65%" headers="mcps1.3.4.3.2.3.1.2 "><p id="dli_01_0228__p96527411337">Enterprise project to which the database belongs. An enterprise project facilitates project-level management and grouping of cloud resources and users.</p>
<p id="dli_01_0228__p196152313331">For details about how to set enterprise projects, see .</p>
</td>
</tr>
<tr id="dli_01_0228__row1662880815250"><td class="cellrowborder" valign="top" width="19.35%" headers="mcps1.3.4.3.2.3.1.1 "><p id="dli_01_0228__p475621615250">Operation</p>

View File

@ -6,9 +6,8 @@
</div>
<div class="section" id="dli_01_0253__section46923850144935"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_01_0253__p2990805493216">The data to be imported has been stored on OBS.</p>
</div>
<div class="section" id="dli_01_0253__section1935812127104"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0253__ol64023642144515"><li id="dli_01_0253__li2078303115103"><span>You can import data on either the <span class="wintitle" id="dli_01_0253__wintitle6968736183919"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0253__wintitle296873615397"><b>SQL Editor</b></span> page.</span><p><ul id="dli_01_0253__ul13952860151010"><li id="dli_01_0253__li20669721151337">To import data on the <span class="wintitle" id="dli_01_0253__wintitle718703911393"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0253__ol65658627151340"><li id="dli_01_0253__li16879536151342">On the left of the management console, choose <span class="uicontrol" id="dli_01_0253__uicontrol1861322375215"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0253__uicontrol8614122385211"><b>Databases and Tables</b></span>.</li><li id="dli_01_0253__li29714355165922">Click the name of the database corresponding to the table where data is to be imported to switch to the table management page.</li><li id="dli_01_0253__li14811172414512">Locate the row where the target table resides and choose <strong id="dli_01_0253__b1434424713920">More</strong> &gt; <strong id="dli_01_0253__b834444717393">Import</strong> in the <strong id="dli_01_0253__b113451847143918">Operation</strong> column. The <span class="wintitle" id="dli_01_0253__wintitle9345164715393"><b>Import</b></span> dialog box is displayed.<p id="dli_01_0253__p1475334133411"></p>
</li></ol>
</li><li id="dli_01_0253__li10384371151118">To import data on the <span class="wintitle" id="dli_01_0253__wintitle163710502399"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0253__ol39382464151459"><li id="dli_01_0253__li26745450151459">On the left of the management console, click <span class="uicontrol" id="dli_01_0253__uicontrol1351715612720"><b>SQL Editor</b></span>.</li><li id="dli_01_0253__li1773513617043">In the navigation tree on the left of <span class="uicontrol" id="dli_01_0253__uicontrol1274112477320"><b>SQL Editor</b></span>, click <strong id="dli_01_0253__b19748144733213">Databases</strong> to see all databases. Click the database where the target table belongs. The table list is displayed.</li><li id="dli_01_0253__li2614201531">Click <span><img id="dli_01_0253__image12666523113" src="en-us_image_0237990324.png"></span> on the right of the table and choose <strong id="dli_01_0253__b1726611527114">Import</strong> from the shortcut menu. The <span class="wintitle" id="dli_01_0253__wintitle526612521613"><b>Import</b></span> page is displayed.<p id="dli_01_0253__p133265173711"></p>
<div class="section" id="dli_01_0253__section1935812127104"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0253__ol64023642144515"><li id="dli_01_0253__li2078303115103"><span>You can import data on either the <span class="wintitle" id="dli_01_0253__wintitle6968736183919"><b>Data Management</b></span> page or the <span class="wintitle" id="dli_01_0253__wintitle296873615397"><b>SQL Editor</b></span> page.</span><p><ul id="dli_01_0253__ul13952860151010"><li id="dli_01_0253__li20669721151337">To import data on the <span class="wintitle" id="dli_01_0253__wintitle718703911393"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0253__ol65658627151340"><li id="dli_01_0253__li16879536151342">On the left of the management console, choose <span class="uicontrol" id="dli_01_0253__uicontrol1861322375215"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0253__uicontrol8614122385211"><b>Databases and Tables</b></span>.</li><li id="dli_01_0253__li29714355165922">Click the name of the database corresponding to the table where data is to be imported to switch to the table management page.</li><li id="dli_01_0253__li14811172414512">Locate the row where the target table resides and choose <strong id="dli_01_0253__b1434424713920">More</strong> &gt; <strong id="dli_01_0253__b834444717393">Import</strong> in the <strong id="dli_01_0253__b113451847143918">Operation</strong> column. The <span class="wintitle" id="dli_01_0253__wintitle9345164715393"><b>Import</b></span> dialog box is displayed.</li></ol>
</li><li id="dli_01_0253__li10384371151118">To import data on the <span class="wintitle" id="dli_01_0253__wintitle163710502399"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0253__ol39382464151459"><li id="dli_01_0253__li26745450151459">On the left of the management console, click <span class="uicontrol" id="dli_01_0253__uicontrol1351715612720"><b>SQL Editor</b></span>.</li><li id="dli_01_0253__li1773513617043">In the navigation tree on the left of <span class="uicontrol" id="dli_01_0253__uicontrol1274112477320"><b>SQL Editor</b></span>, click <strong id="dli_01_0253__b19748144733213">Databases</strong> to see all databases. Click the database where the target table belongs. The table list is displayed.</li><li id="dli_01_0253__li2614201531">Click <span><img id="dli_01_0253__image12666523113" src="en-us_image_0237990324.png"></span> on the right of the table and choose <strong id="dli_01_0253__b1726611527114">Import</strong> from the shortcut menu. The <span class="wintitle" id="dli_01_0253__wintitle526612521613"><b>Import</b></span> page is displayed.<p id="dli_01_0253__p186911222355"></p>
</li></ol>
</li></ul>
</p></li><li id="dli_01_0253__li88586820311"><span>In the <strong id="dli_01_0253__b1393765403">Import</strong> dialog box, set the parameters based on <a href="#dli_01_0253__table48581581434">Table 1</a>.</span><p>
@ -96,7 +95,7 @@
<tr id="dli_01_0253__row88514818311"><td class="cellrowborder" valign="top" width="20.3%" headers="mcps1.3.4.2.2.2.1.2.4.1.1 "><p id="dli_01_0253__p98507815314">Date Format</p>
</td>
<td class="cellrowborder" valign="top" width="64.2%" headers="mcps1.3.4.2.2.2.1.2.4.1.2 "><p id="dli_01_0253__p1285020810315">This parameter is valid only when <span class="parmname" id="dli_01_0253__parmname101493527346"><b>File Format</b></span> is set to <span class="parmvalue" id="dli_01_0253__parmvalue1149205243420"><b>CSV</b></span> or <span class="parmvalue" id="dli_01_0253__parmvalue1115045243420"><b>JSON</b></span>.</p>
<p id="dli_01_0253__p3851128437">This parameter specifies the format of the date in the table and is valid only <span class="parmvalue" id="dli_01_0253__parmvalue193439967115458"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0253__parmname181884393215542"><b>yyyy-MM-dd</b></span>. </p>
<p id="dli_01_0253__p3851128437">This parameter specifies the format of the date in the table and is valid only <span class="parmvalue" id="dli_01_0253__parmvalue193439967115458"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0253__parmname181884393215542"><b>yyyy-MM-dd</b></span>. For definition of characters involved in the date pattern, see Table 3 in .</p>
</td>
<td class="cellrowborder" valign="top" width="15.5%" headers="mcps1.3.4.2.2.2.1.2.4.1.3 "><p id="dli_01_0253__p15851582314">2000-01-01</p>
</td>
@ -104,7 +103,7 @@
<tr id="dli_01_0253__row1685317817310"><td class="cellrowborder" valign="top" width="20.3%" headers="mcps1.3.4.2.2.2.1.2.4.1.1 "><p id="dli_01_0253__p1585178432">Timestamp Format</p>
</td>
<td class="cellrowborder" valign="top" width="64.2%" headers="mcps1.3.4.2.2.2.1.2.4.1.2 "><p id="dli_01_0253__p13852181332">This parameter is valid only when <span class="parmname" id="dli_01_0253__parmname6195716193619"><b>File Format</b></span> is set to <span class="parmvalue" id="dli_01_0253__parmvalue319571619366"><b>CSV</b></span> or <span class="parmvalue" id="dli_01_0253__parmvalue1719521614365"><b>JSON</b></span>.</p>
<p id="dli_01_0253__p16852168533">This parameter specifies the format of the timestamp in the table and is valid only <span class="parmvalue" id="dli_01_0253__parmvalue10399413104214"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0253__parmname36794602143645"><b>yyyy-MM-dd HH:mm:ss</b></span>. For definition of characters involved in the time pattern, see Table 3 in the </p>
<p id="dli_01_0253__p16852168533">This parameter specifies the format of the timestamp in the table and is valid only <span class="parmvalue" id="dli_01_0253__parmvalue10399413104214"><b>Advanced Settings</b></span> is selected. The default value is <span class="parmname" id="dli_01_0253__parmname36794602143645"><b>yyyy-MM-dd HH:mm:ss</b></span>. For definition of characters involved in the time pattern, see Table 3 in .</p>
</td>
<td class="cellrowborder" valign="top" width="15.5%" headers="mcps1.3.4.2.2.2.1.2.4.1.3 "><p id="dli_01_0253__p148526811314">2000-01-01 09:00:00</p>
</td>

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -24,7 +24,7 @@
</tr>
<tr id="dli_01_0363__row82981752094"><td class="cellrowborder" valign="top" width="14.77%" headers="mcps1.3.3.2.2.2.1.2.3.1.1 "><p id="dli_01_0363__p629715213917">Type</p>
</td>
<td class="cellrowborder" valign="top" width="85.22999999999999%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><ul id="dli_01_0363__ul1029875217914"><li id="dli_01_0363__li1229745218914"><strong id="dli_01_0363__b3471630195419">For SQL</strong>: compute resources used for SQL jobs.</li><li id="dli_01_0363__li1829818521296"><strong id="dli_01_0363__b8738194132314">For general purpose</strong>: compute resources used for Spark and Flink jobs.<div class="note" id="dli_01_0363__note1829815214917"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0363__p329815212917">In <strong id="dli_01_0363__b114679913236">Dedicated Resource Mode</strong>, you can create enhanced datasource connections. </p>
<td class="cellrowborder" valign="top" width="85.22999999999999%" headers="mcps1.3.3.2.2.2.1.2.3.1.2 "><ul id="dli_01_0363__ul1029875217914"><li id="dli_01_0363__li1229745218914"><strong id="dli_01_0363__b3471630195419">For SQL</strong>: compute resources used for SQL jobs.</li><li id="dli_01_0363__li1829818521296"><strong id="dli_01_0363__b8738194132314">For general purpose</strong>: compute resources used for Spark and Flink jobs.</li><li id="dli_01_0363__li1970512436542"><div class="note" id="dli_01_0363__note1829815214917"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0363__p329815212917">In <strong id="dli_01_0363__b114679913236">Dedicated Resource Mode</strong>, you can create enhanced datasource connections. </p>
</div></div>
</li></ul>
</td>

View File

@ -1,43 +1,36 @@
<a name="dli_01_0375"></a><a name="dli_01_0375"></a>
<h1 class="topictitle1">Creating and Submitting a Spark Jar Job</h1>
<div id="body1531966492787"><p id="dli_01_0375__p934442315541">You can use DLI to submit Spark jobs for real-time computing. The general procedure is as follows:</p>
<p id="dli_01_0375__p117221010204913"><a href="#dli_01_0375__section3751181910618">Step 1: Logging in to the Cloud</a></p>
<p id="dli_01_0375__p5746512114914"><a href="#dli_01_0375__section10891114913473">Step 2: Uploading Data to OBS</a></p>
<p id="dli_01_0375__p8291815134918"><a href="#dli_01_0375__section19012773105034">Step 3: Logging In to the DLI Management Console</a></p>
<p id="dli_01_0375__p07722016154920"><a href="#dli_01_0375__section122981023152710">Step 4: Creating a Queue</a></p>
<p id="dli_01_0375__p13369418124916"><a href="#dli_01_0375__section21433273112656">Step 5: Creating a Package</a></p>
<p id="dli_01_0375__p132924161969"><a href="#dli_01_0375__section21590507141153">Step 6: Submitting a Spark Job</a></p>
<div class="section" id="dli_01_0375__section3751181910618"><a name="dli_01_0375__section3751181910618"></a><a name="section3751181910618"></a><h4 class="sectiontitle">Step 1: Logging in to the Cloud</h4><p id="dli_01_0375__p79731024469">To use DLI, you need to log in to the cloud.</p>
<div id="body1531966492787"><div class="section" id="dli_01_0375__section18602102002716"><h4 class="sectiontitle">Scenarios</h4><p id="dli_01_0375__p2824922152712">DLI can query data stored in OBS. This section describes how to use a Spark Jar job on DLI to query OBS data in real time.</p>
</div>
<ol id="dli_01_0375__ol77610191561"><li id="dli_01_0375__li1873417233111">Open the DLI home page.</li><li id="dli_01_0375__li1597319241563">On the login page, enter the username and password, and click <strong id="dli_01_0375__b87301033104820">Log In</strong>.</li></ol>
<div class="section" id="dli_01_0375__section10891114913473"><a name="dli_01_0375__section10891114913473"></a><a name="section10891114913473"></a><h4 class="sectiontitle">Step 2: Uploading Data to OBS</h4><p id="dli_01_0375__p551741275618">Write a Spark Jar job program , and compile and package it as <strong id="dli_01_0375__b281881725913">spark-examples.jar</strong>. Perform the following steps to submit the job:</p>
<div class="section" id="dli_01_0375__section14374132910279"><h4 class="sectiontitle">Procedure</h4><p id="dli_01_0375__p934442315541">You can use DLI to submit Spark jobs for real-time computing. The general procedure is as follows:</p>
<p id="dli_01_0375__p5746512114914"><a href="#dli_01_0375__section10891114913473">Step 1: Upload Data to OBS</a></p>
<p id="dli_01_0375__p07722016154920"><a href="#dli_01_0375__section122981023152710">Step 2: Create a Queue</a></p>
<p id="dli_01_0375__p13369418124916"><a href="#dli_01_0375__section21433273112656">Step 3: Create a Package</a></p>
<p id="dli_01_0375__p132924161969"><a href="#dli_01_0375__section21590507141153">Step 4: Submit a Spark Job</a></p>
</div>
<div class="section" id="dli_01_0375__section10891114913473"><a name="dli_01_0375__section10891114913473"></a><a name="section10891114913473"></a><h4 class="sectiontitle">Step 1: Upload Data to OBS</h4><p id="dli_01_0375__p551741275618">Write a Spark Jar job program , and compile and package it as <strong id="dli_01_0375__b281881725913">spark-examples.jar</strong>. Perform the following steps to submit the job:</p>
<p id="dli_01_0375__p45348185145915">Before submitting Spark Jar jobs, upload data files to OBS.</p>
<ol id="dli_01_0375__ol18891449164710"><li id="dli_01_0375__li18770142812720">In the services displayed, click <strong id="dli_01_0375__b10681912113315">Object Storage Service (OBS)</strong> in <strong id="dli_01_0375__b86819122333">Storage</strong>.</li><li id="dli_01_0375__li108923494479">The OBS console page is displayed.</li><li id="dli_01_0375__li1385211252717">Create a bucket. The bucket name must be globally unique. In this example, assume that the bucket name is <strong id="dli_01_0375__b349033115613">dli-test-obs01</strong>.<ol type="a" id="dli_01_0375__ol182612310711"><li id="dli_01_0375__li462934116719">Click <span class="uicontrol" id="dli_01_0375__uicontrol3515111010568"><b>Create Bucket</b></span>.</li><li id="dli_01_0375__li183491949872">On the displayed <strong id="dli_01_0375__b1736513156263">Create Bucket</strong> page, enter the <strong id="dli_01_0375__b153711815192617">Bucket Name</strong>. Retain the default values for other parameters or set them as required.</li><li id="dli_01_0375__li1089354984713">Click <strong id="dli_01_0375__b55859547573">Create Now</strong>.</li></ol>
<ol id="dli_01_0375__ol18891449164710"><li id="dli_01_0375__li12281133046">Log in to the DLI console.</li><li id="dli_01_0375__li18770142812720">In the service list, click <strong id="dli_01_0375__b912513195911">Object Storage Service</strong> under <strong id="dli_01_0375__b8125151910912">Storage</strong>. The OBS console page is displayed.</li><li id="dli_01_0375__li1385211252717">Create a bucket. In this example, name it <strong id="dli_01_0375__b41010361919">dli-test-obs01</strong>.<ol type="a" id="dli_01_0375__ol182612310711"><li id="dli_01_0375__li462934116719">Click <span class="uicontrol" id="dli_01_0375__uicontrol3515111010568"><b>Create Bucket</b></span>.</li><li id="dli_01_0375__li183491949872">On the displayed <strong id="dli_01_0375__b1736513156263">Create Bucket</strong> page, enter the <strong id="dli_01_0375__b153711815192617">Bucket Name</strong>. Retain the default values for other parameters or set them as required.<div class="note" id="dli_01_0375__note1145175833410"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0375__p7146205812341">When creating an OBS bucket, you must select the same region as the DLI management console.</p>
</div></div>
</li><li id="dli_01_0375__li1089354984713">Click <strong id="dli_01_0375__b55859547573">Create Now</strong>.</li></ol>
</li><li id="dli_01_0375__li13893749154715">Click <strong id="dli_01_0375__b198121931182012">dli-test-obs01</strong> to switch to the <strong id="dli_01_0375__b168121831102014">Overview</strong> page.</li><li id="dli_01_0375__li1129115358610">From the left navigation tree, click <strong id="dli_01_0375__b17105173492019">Objects</strong>. Click <strong id="dli_01_0375__b310514347204">Upload Object</strong>. In the displayed dialog box, drag files or folders to upload or add file to the file upload box, for example, <span class="parmname" id="dli_01_0375__parmname477111272516"><b>spark-examples.jar</b></span>. Then, click <span class="uicontrol" id="dli_01_0375__uicontrol1160514822611"><b>Upload</b></span>.<p id="dli_01_0375__p53510371013">After the file is uploaded successfully, the file path is <span class="filepath" id="dli_01_0375__filepath8679102112119"><b>obs://dli-test-obs01/spark-examples.jar</b></span>.</p>
<p id="dli_01_0375__p9375852182812">For more information about OBS operations, see the <em id="dli_01_0375__i1410111914262">Object Storage Service Console Operation Guide</em>.</p>
<p id="dli_01_0375__p1644914312293">For more information about the tool, see the <em id="dli_01_0375__i1218132242612">OBS Tool Guide</em>.</p>
<div class="note" id="dli_01_0375__note121534311285"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0375__p277224772810">You are advised to use an OBS tool, such as OBS Browser+, to upload large files because OBS Console has restrictions on the file size and quantity.</p>
<ul id="dli_01_0375__ul12727649112820"><li id="dli_01_0375__li182202533286">OBS Browser+ is a graphical tool that provides complete functions for managing your buckets and objects in OBS. You are advised to use this tool to create buckets or upload objects.</li></ul>
<div class="note" id="dli_01_0375__note121534311285"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0375__ul12727649112820"><li id="dli_01_0375__li1990710154514">For more information about OBS operations, see the <em id="dli_01_0375__i1410111914262">Object Storage Service Console Operation Guide</em>.</li><li id="dli_01_0375__li1090710151850">For more information about the tool, see the <em id="dli_01_0375__i1218132242612">OBS Tool Guide</em>.</li><li id="dli_01_0375__li8907315659">You are advised to use an OBS tool, such as OBS Browser+, to upload large files because OBS Console has restrictions on the file size and quantity.<ul id="dli_01_0375__ul1195116186510"><li id="dli_01_0375__li182202533286">OBS Browser+ is a graphical tool that provides complete functions for managing your buckets and objects in OBS. You are advised to use this tool to create buckets or upload objects.</li></ul>
</li></ul>
</div></div>
</li></ol>
</div>
<div class="section" id="dli_01_0375__section19012773105034"><a name="dli_01_0375__section19012773105034"></a><a name="section19012773105034"></a><h4 class="sectiontitle">Step 3: Logging In to the DLI Management Console</h4><p id="dli_01_0375__p170111716265">To submit Spark jobs, you need to enter the Spark job creation page first.</p>
<ol id="dli_01_0375__ol33542655111116"><li id="dli_01_0375__li2096805415940">In the list, choose .</li><li id="dli_01_0375__li38964926151831">The DLI management console page is displayed.<div class="note" id="dli_01_0375__note231512513316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0375__p231925193114">If you log in to the DLI management console for the first time, you need to be authorized to access OBS. For details, see "Service Authorization".</p>
</div></div>
</li></ol>
</div>
<div class="section" id="dli_01_0375__section122981023152710"><a name="dli_01_0375__section122981023152710"></a><a name="section122981023152710"></a><h4 class="sectiontitle">Step 4: Creating a Queue</h4><p id="dli_01_0375__p5551648414">If you submit a Spark job for the first time, you need to create a queue first. For example, create a queue named <strong id="dli_01_0375__b18940163725910">sparktest</strong> and set <strong id="dli_01_0375__b9941203755913">Queue Type</strong> to <strong id="dli_01_0375__b07631213164717">General Queue</strong>.</p>
<ol id="dli_01_0375__ol1977217571226"><li id="dli_01_0375__li1059116511492">In the navigation pane of the DLI management console, choose <span class="uicontrol" id="dli_01_0375__uicontrol510284210593"><b>Queue Management</b></span>.</li><li id="dli_01_0375__li759275184915">In the upper right corner of the <span class="wintitle" id="dli_01_0375__wintitle4186197182915"><b>Queue Management</b></span> page, click <strong id="dli_01_0375__b16186107192918">Create Queue</strong> to create a queue.</li><li id="dli_01_0375__li550904418312">Create a queue, name it <strong id="dli_01_0375__b4533124655914">sparktest</strong>, and set the queue usage to for general purpose. For details, see Creating a Queue.<p id="dli_01_0375__p86371522489"></p>
<div class="section" id="dli_01_0375__section122981023152710"><a name="dli_01_0375__section122981023152710"></a><a name="section122981023152710"></a><h4 class="sectiontitle">Step 2: Create a Queue</h4><p id="dli_01_0375__p5551648414">If you submit a Spark job for the first time, you need to create a queue first. For example, create a queue named <strong id="dli_01_0375__b18940163725910">sparktest</strong> and set <strong id="dli_01_0375__b9941203755913">Queue Type</strong> to <strong id="dli_01_0375__b07631213164717">General Queue</strong>.</p>
<ol id="dli_01_0375__ol1977217571226"><li id="dli_01_0375__li13332820164119">Log in to the DLI management console.</li><li id="dli_01_0375__li1059116511492">In the navigation pane of the DLI management console, choose <span class="uicontrol" id="dli_01_0375__uicontrol510284210593"><b>Queue Management</b></span>.</li><li id="dli_01_0375__li759275184915">In the upper right corner of the <span class="wintitle" id="dli_01_0375__wintitle4186197182915"><b>Queue Management</b></span> page, click <strong id="dli_01_0375__b16186107192918">Create Queue</strong> to create a queue.</li><li id="dli_01_0375__li550904418312">Create a queue, name it <strong id="dli_01_0375__b4533124655914">sparktest</strong>, and set the queue usage to for general purpose. For details, see Creating a Queue.<p id="dli_01_0375__p86371522489"></p>
</li><li id="dli_01_0375__li647261910292">Click <span class="uicontrol" id="dli_01_0375__uicontrol338141308"><b>Create Now</b></span> to create a queue.</li></ol>
</div>
<div class="section" id="dli_01_0375__section21433273112656"><a name="dli_01_0375__section21433273112656"></a><a name="section21433273112656"></a><h4 class="sectiontitle">Step 5: Creating a Package</h4><p id="dli_01_0375__p498774501214">Before submitting a Spark job, you need to create a package, for example, <strong id="dli_01_0375__b17591192235312">spark-examples.jar</strong>.</p>
<ol id="dli_01_0375__ol6181725181410"><li id="dli_01_0375__li0727165414248">In the navigation pane on the left of the DLI console, choose <span class="uicontrol" id="dli_01_0375__uicontrol16727106102"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0375__uicontrol772715611012"><b>Package Management</b></span>.</li><li id="dli_01_0375__li13727135411249">On the <span class="wintitle" id="dli_01_0375__wintitle1923015915014"><b>Package Management</b></span> page, click <strong id="dli_01_0375__b42312091008">Create</strong> in the upper right corner to create a package.</li><li id="dli_01_0375__li10728165402418">In the <strong id="dli_01_0375__b51515151302">Create Package</strong> dialog box, set <strong id="dli_01_0375__b151614152015">Type</strong> to <strong id="dli_01_0375__b1316131513010">JAR</strong>, <strong id="dli_01_0375__b17169152002">OBS Path</strong> to the path of the spark-examples.jar package in <a href="#dli_01_0375__section10891114913473">Step 2: Uploading Data to OBS</a>, and <strong id="dli_01_0375__b15175152002">Group</strong> to <strong id="dli_01_0375__b2172151509">Do not use</strong>.<p id="dli_01_0375__p1853715711915"></p>
<div class="section" id="dli_01_0375__section21433273112656"><a name="dli_01_0375__section21433273112656"></a><a name="section21433273112656"></a><h4 class="sectiontitle">Step 3: Create a Package</h4><p id="dli_01_0375__p498774501214">Before submitting a Spark job, you need to create a package, for example, <strong id="dli_01_0375__b17591192235312">spark-examples.jar</strong>.</p>
<ol id="dli_01_0375__ol6181725181410"><li id="dli_01_0375__li0727165414248">In the navigation pane on the left of the DLI console, choose <span class="uicontrol" id="dli_01_0375__uicontrol16727106102"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0375__uicontrol772715611012"><b>Package Management</b></span>.</li><li id="dli_01_0375__li13727135411249">On the <span class="wintitle" id="dli_01_0375__wintitle1923015915014"><b>Package Management</b></span> page, click <strong id="dli_01_0375__b42312091008">Create</strong> in the upper right corner to create a package.</li><li id="dli_01_0375__li10728165402418">In the <strong id="dli_01_0375__b51515151302">Create Package</strong> dialog box, set <strong id="dli_01_0375__b151614152015">Type</strong> to <strong id="dli_01_0375__b1316131513010">JAR</strong>, <strong id="dli_01_0375__b17169152002">OBS Path</strong> to the path of the spark-examples.jar package in <a href="#dli_01_0375__section10891114913473">Step 1: Upload Data to OBS</a>, and <strong id="dli_01_0375__b15175152002">Group</strong> to <strong id="dli_01_0375__b2172151509">Do not use</strong>.<p id="dli_01_0375__p1853715711915"></p>
</li><li id="dli_01_0375__li472825420248">Click <span class="uicontrol" id="dli_01_0375__uicontrol13383131711010"><b>OK</b></span>.<p id="dli_01_0375__en-us_topic_0122016946_en-us_topic_0093946917_p1543112413517">You can view and select the package on the <span class="wintitle" id="dli_01_0375__wintitle7855171912016"><b>Package Management</b></span> page.</p>
</li></ol>
<p id="dli_01_0375__p152403475316">For details about how to create a package, see "Creating a Package".</p>
</div>
<div class="section" id="dli_01_0375__section21590507141153"><a name="dli_01_0375__section21590507141153"></a><a name="section21590507141153"></a><h4 class="sectiontitle">Step 6: Submitting a Spark Job</h4><ol id="dli_01_0375__ol13334841116"><li id="dli_01_0375__li787014474310">On the DLI management console, choose <strong id="dli_01_0375__b252635684513">Job Management &gt; Spark Jobs</strong> in the navigation pane on the left. On the displayed page, click <strong id="dli_01_0375__b11359182644">Create Job</strong>.</li><li id="dli_01_0375__li68657434228">On the Spark job editing page, set <strong id="dli_01_0375__b11433131752">Queues</strong> to the queue created in <a href="#dli_01_0375__section122981023152710">Step 4: Creating a Queue</a> and <strong id="dli_01_0375__b1112910211056">Application</strong> to the package created in <a href="#dli_01_0375__section21433273112656">Step 5: Creating a Package</a>.<p id="dli_01_0375__p8407123292413">For details about other parameters, see the description of the Spark job editing page in "Creating a Spark Job".</p>
<div class="section" id="dli_01_0375__section21590507141153"><a name="dli_01_0375__section21590507141153"></a><a name="section21590507141153"></a><h4 class="sectiontitle">Step 4: Submit a Spark Job</h4><ol id="dli_01_0375__ol13334841116"><li id="dli_01_0375__li787014474310">On the DLI management console, choose <strong id="dli_01_0375__b252635684513">Job Management &gt; Spark Jobs</strong> in the navigation pane on the left. On the displayed page, click <strong id="dli_01_0375__b11359182644">Create Job</strong>.</li><li id="dli_01_0375__li68657434228">On the Spark job editing page, set <strong id="dli_01_0375__b11433131752">Queues</strong> to the queue created in <a href="#dli_01_0375__section122981023152710">Step 2: Create a Queue</a> and <strong id="dli_01_0375__b1112910211056">Application</strong> to the package created in <a href="#dli_01_0375__section21433273112656">Step 3: Create a Package</a>.<p id="dli_01_0375__p8407123292413">For details about other parameters, see the description of the Spark job editing page in "Creating a Spark Job".</p>
</li><li id="dli_01_0375__li793921414487">Click <strong id="dli_01_0375__b1648118125414">Execute</strong> in the upper right corner of the Spark job editing window, read and agree to the privacy agreement, and click <strong id="dli_01_0375__b19493885411">OK</strong>. Submit the job. A message is displayed, indicating that the job is submitted.<p id="dli_01_0375__p171961850152716"></p>
</li><li id="dli_01_0375__li91513419184">(Optional) Switch to the <strong id="dli_01_0375__b774111583518">Job Management &gt; Spark Jobs</strong> page to view the status and logs of the submitted Spark job.<div class="note" id="dli_01_0375__note8365749164313"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0375__p19786105514174">When you click <strong id="dli_01_0375__b3208155475413">Execute</strong> on the DLI management console for the first time, you need to read the privacy agreement. Once agreed to the agreement, you will not receive any privacy agreement messages for subsequent operations.</p>
</div></div>

File diff suppressed because it is too large Load Diff

View File

@ -1,12 +1,12 @@
<a name="dli_01_0378"></a><a name="dli_01_0378"></a>
<h1 class="topictitle1">DLI Introduction</h1>
<div id="body39451090"><div class="section" id="dli_01_0378__section118410576246"><h4 class="sectiontitle">What Is Data Lake Insight?</h4><p id="dli_01_0378__p104391318592">Data Lake Insight (DLI) is a serverless data processing and analysis service fully compatible with <a href="https://spark.apache.org/" target="_blank" rel="noopener noreferrer">Apache Spark</a> and <a href="https://flink.apache.org/" target="_blank" rel="noopener noreferrer">Apache Flink</a> ecosystems. It frees you from managing any server.</p>
<h1 class="topictitle1">What Is Data Lake Insight?</h1>
<div id="body39451090"><div class="section" id="dli_01_0378__section118410576246"><h4 class="sectiontitle">DLI Introduction</h4><p id="dli_01_0378__p104391318592">Data Lake Insight (DLI) is a serverless data processing and analysis service fully compatible with <a href="https://spark.apache.org/" target="_blank" rel="noopener noreferrer">Apache Spark</a> and <a href="https://flink.apache.org/" target="_blank" rel="noopener noreferrer">Apache Flink</a> ecosystems. It frees you from managing any servers.</p>
<p id="dli_01_0378__p35691490142221">DLI supports standard SQL and is compatible with Spark SQL and Flink SQL. It also supports multiple access modes, and is compatible with mainstream data formats. DLI supports SQL statements and Spark applications for heterogeneous data sources, including CloudTable, RDS, GaussDB(DWS), CSS, OBS, custom databases on ECSs, and offline databases.</p>
</div>
<div class="section" id="dli_01_0378__section189215415182"><h4 class="sectiontitle">Function Description</h4><p id="dli_01_0378__p3872545124817">You can query and analyze heterogeneous data sources such as RDS, and DWS on the cloud using access methods, such as visualized interface, RESTful API, JDBC, ODBC, and Beeline. The data format is compatible with five mainstream data formats: CSV, JSON, Parquet, and ORC.</p>
<ul id="dli_01_0378__ul95252494910"><li id="dli_01_0378__li37017520555">Basic functions<ul id="dli_01_0378__ul33508814556"><li id="dli_01_0378__li4884112014444">You can use standard SQL statements to query in SQL jobs. </li><li id="dli_01_0378__li583064563216">Flink jobs support Flink SQL online analysis. Aggregation functions such as Window and Join, geographic functions, and CEP functions are supported. SQL is used to express service logic, facilitating service implementation. For details, see .</li><li id="dli_01_0378__li652124194912">For spark jobs, fully-managed Spark computing can be performed. You can submit computing tasks through interactive sessions or in batch to analyze data in the fully managed Spark queues. For details, see .</li></ul>
</li><li id="dli_01_0378__li13705141165713">Federated analysis of heterogeneous data sources<ul id="dli_01_0378__ul14754404579"><li id="dli_01_0378__li68911658193117">Spark datasource connection: Data sources such as DWS, RDS, and CSS can be accessed through DLI. For details, see .</li><li id="dli_01_0378__li9919175613471">Interconnection with multiple cloud services is supported in Flink jobs to form a rich stream ecosystem. The DLI stream ecosystem consists of cloud service ecosystems and open source ecosystems.<ul id="dli_01_0378__ul4978181016481"><li id="dli_01_0378__li10978151011487">Cloud service ecosystem: DLI can interconnect with other services in Flink SQL. You can directly use SQL to read and write data from cloud services.</li><li id="dli_01_0378__li4978610154816">Open-source ecosystems: After connections to other VPCs are established through datasource connections, you can access all data sources and output targets (such as Kafka, HBase, and Elasticsearch) supported by Flink and Spark in your exclusive DLI queue.</li></ul>
<div class="section" id="dli_01_0378__section189215415182"><h4 class="sectiontitle">Functions</h4><p id="dli_01_0378__p3872545124817">You can query and analyze heterogeneous data sources such as RDS, and DWS on the cloud using access methods, such as visualized interface, RESTful API, JDBC, ODBC, and Beeline. The data format is compatible with five mainstream data formats: CSV, JSON, Parquet, and ORC.</p>
<ul id="dli_01_0378__ul95252494910"><li id="dli_01_0378__li37017520555">Basic functions<ul id="dli_01_0378__ul33508814556"><li id="dli_01_0378__li4884112014444">You can use standard SQL statements to query in SQL jobs. </li><li id="dli_01_0378__li583064563216">Flink jobs support Flink SQL online analysis. Aggregation functions such as Window and Join, geographic functions, and CEP functions are supported. SQL is used to express service logic, facilitating service implementation. For details, see .</li><li id="dli_01_0378__li652124194912">For spark jobs, fully-managed Spark computing can be performed. You can submit computing tasks through interactive sessions or in batch to analyze data in the fully managed Spark queues. For details, see .</li></ul>
</li><li id="dli_01_0378__li13705141165713">Federated analysis of heterogeneous data sources<ul id="dli_01_0378__ul14754404579"><li id="dli_01_0378__li68911658193117">Spark datasource connection: Data sources such as DWS, RDS, and CSS can be accessed through DLI. For details, see .</li><li id="dli_01_0378__li9919175613471">Interconnection with multiple cloud services is supported in Flink jobs to form a rich stream ecosystem. The DLI stream ecosystem consists of cloud service ecosystems and open source ecosystems.<ul id="dli_01_0378__ul4978181016481"><li id="dli_01_0378__li10978151011487">Cloud service ecosystem: DLI can interconnect with other services in Flink SQL. You can directly use SQL to read and write data from cloud services.</li><li id="dli_01_0378__li4978610154816">Open-source ecosystems: After connections to other VPCs are established through datasource connections, you can access all data sources and output targets (such as Kafka, HBase, and Elasticsearch) supported by Flink and Spark in your dedicated DLI queue.</li></ul>
<p id="dli_01_0378__p11978131024820">For details, see .</p>
</li></ul>
</li><li id="dli_01_0378__li620462561618">Storage-compute decoupling<p id="dli_01_0378__p12236173712168"><a name="dli_01_0378__li620462561618"></a><a name="li620462561618"></a>DLI is interconnected with OBS for data analysis. In this architecture where storage and compute are decoupled, resources of these two types are charged separately, helping you reduce costs and improving resource utilization.</p>

View File

@ -4,9 +4,11 @@
<div id="body1532683323732"></div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0021.html">SQL Template Management</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0021.html">Managing SQL Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0464.html">Flink Template Management</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0464.html">Managing Flink Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0551.html">Managing Spark SQL Templates</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_05110.html">Appendix</a></strong><br>
</li>

File diff suppressed because it is too large Load Diff

View File

@ -12,7 +12,7 @@
</li>
<li class="ulchildlink"><strong><a href="dli_01_0498.html">(Recommended) Creating a Flink OpenSource SQL Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0455.html">Creating a Flink SQL job</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0455.html">Creating a Flink SQL Job</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0457.html">Creating a Flink Jar Job</a></strong><br>
</li>

View File

@ -6,9 +6,9 @@
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0228.html">Overview</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0447.html">Database Permission Management</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0447.html">Managing Database Permissions</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0448.html">Table Permission Management</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0448.html">Managing Table Permissions</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0005.html">Creating a Database or a Table</a></strong><br>
</li>

File diff suppressed because one or more lines are too long

View File

@ -1,7 +1,8 @@
<a name="dli_01_0410"></a><a name="dli_01_0410"></a>
<h1 class="topictitle1">Datasource Connection and Cross-Source Analysis</h1>
<div id="body1568974021769"><p id="dli_01_0410__p82341101018">DLI supports the datasource capability of the native Spark and extends it. With DLI datasource connection, you can access other data storage services through SQL statements, Spark jobs, and Flink jobs and import, query, analyze, and process data in the services.</p>
<div id="body1568974021769"><p id="dli_01_0410__p244516400579"></p>
<p id="dli_01_0410__p82341101018">DLI supports the datasource capability of the native Spark and extends it. With DLI datasource connection, you can access other data storage services through SQL statements, Spark jobs, and Flink jobs and import, query, analyze, and process data in the services.</p>
<div class="section" id="dli_01_0410__section390162410466"><h4 class="sectiontitle">Datasource Connections</h4><p id="dli_01_0410__p879575781920">Before using DLI to perform cross source analysis, you need to set up a datasource connection to enable the network between data sources.</p>
<p id="dli_01_0410__p1593528944">The enhanced datasource connection uses VPC peering at the bottom layer to directly connect the VPC network between the DLI queue and the destination datasource. Data is exchanged in point-to-point mode.</p>
<div class="note" id="dli_01_0410__note6225712410"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0410__ul9456100513"><li id="dli_01_0410__li54651010513">Datasource connections cannot be created for the default queue.</li><li id="dli_01_0410__li18641201213518"><strong id="dli_01_0410__b1054013291417">VPC Administrator</strong> permissions are required to use the VPC, subnet, route, VPC peering connection, and port for DLI datasource connections. You can set this parameter in <a href="dli_01_0486.html">Service Authorization</a>.</li></ul>

View File

@ -67,6 +67,13 @@
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.2.2.4.1.3 "><p id="dli_01_0417__dli_01_0451_p266618125317">datasourceauth.name</p>
</td>
</tr>
<tr id="dli_01_0417__dli_01_0451_row1671920220218"><td class="cellrowborder" valign="top" width="17.77%" headers="mcps1.3.2.2.4.1.1 "><p id="dli_01_0417__dli_01_0451_p1671919252112">edsconnections</p>
</td>
<td class="cellrowborder" valign="top" width="22.74%" headers="mcps1.3.2.2.4.1.2 "><p id="dli_01_0417__dli_01_0451_p2071912122110">Enhanced datasource connection</p>
</td>
<td class="cellrowborder" valign="top" width="59.489999999999995%" headers="mcps1.3.2.2.4.1.3 "><p id="dli_01_0417__dli_01_0451_p771917217213">edsconnections.<em id="dli_01_0417__dli_01_0451_i18646151818515">connection ID</em></p>
</td>
</tr>
</tbody>
</table>
</div>

View File

@ -12,7 +12,7 @@
<ol id="dli_01_0418__ol139492810017"><li id="dli_01_0418__li895020818018"><a name="dli_01_0418__li895020818018"></a><a name="li895020818018"></a>Create a user group and grant the permission to it.<p id="dli_01_0418__p99501985019"><a name="dli_01_0418__li895020818018"></a><a name="li895020818018"></a>Create a user group on the IAM console, and assign the <strong id="dli_01_0418__b163744195013">DLI ReadOnlyAccess</strong> permission to the group.</p>
</li><li id="dli_01_0418__li59503817014">Create a user and add the user to the user group.<p id="dli_01_0418__p4950882013"><a name="dli_01_0418__li59503817014"></a><a name="li59503817014"></a>Create a user on the IAM console and add the user to the group created in <a href="#dli_01_0418__li895020818018">1</a>.</p>
</li><li id="dli_01_0418__li159506814017">Log in and verify the permission.<p id="dli_01_0418__p29911201511"><a name="dli_01_0418__li159506814017"></a><a name="li159506814017"></a>Log in to the management console using the newly created user, and verify that the user's permissions.</p>
<ul id="dli_01_0418__ul9992304111"><li id="dli_01_0418__li149921301015">Choose <strong id="dli_01_0418__b1538390723">Service List</strong> &gt; <strong id="dli_01_0418__b2428344">Data Lake Insight</strong>. The DLI management console is displayed. If you can view the queue list on the <strong id="dli_01_0418__b86423619315">Queue Management</strong> page but cannot buy DLI queues by clicking <strong id="dli_01_0418__b1701367313">Buy Queue</strong> in the upper right corner (assume that the current permission contains only <strong id="dli_01_0418__b1070193615314">DLI ReadOnlyAccess</strong>), the <strong id="dli_01_0418__b7710363316">DLI ReadOnlyAccess</strong> permission has taken effect.</li><li id="dli_01_0418__li29921500115">Choose any other service in <strong id="dli_01_0418__b6827231507">Service List</strong>. If a message appears indicating that you have insufficient permissions to access the service, the <strong id="dli_01_0418__b148281231700">DLI ReadOnlyAccess</strong> permission has already taken effect.</li></ul>
<ul id="dli_01_0418__ul9992304111"><li id="dli_01_0418__li149921301015">Choose <strong id="dli_01_0418__b903531909">Service List</strong> &gt; <strong id="dli_01_0418__b1933557329">Data Lake Insight</strong>. The DLI management console is displayed. If you can view the queue list on the <strong id="dli_01_0418__b86423619315">Queue Management</strong> page but cannot buy DLI queues by clicking <strong id="dli_01_0418__b1701367313">Buy Queue</strong> in the upper right corner (assume that the current permission contains only <strong id="dli_01_0418__b1070193615314">DLI ReadOnlyAccess</strong>), the <strong id="dli_01_0418__b7710363316">DLI ReadOnlyAccess</strong> permission has taken effect.</li><li id="dli_01_0418__li29921500115">Choose any other service in <strong id="dli_01_0418__b6827231507">Service List</strong>. If a message appears indicating that you have insufficient permissions to access the service, the <strong id="dli_01_0418__b148281231700">DLI ReadOnlyAccess</strong> permission has already taken effect.</li></ul>
</li></ol>
<p id="dli_01_0418__p05981420125912"></p>
</div>

View File

@ -0,0 +1,23 @@
<a name="dli_01_0420"></a><a name="dli_01_0420"></a>
<h1 class="topictitle1">Importing Data to a DLI Table</h1>
<div id="body1562569148234"><div class="section" id="dli_01_0420__section133251856125910"><h4 class="sectiontitle">Importing Data Using OBS</h4><p id="dli_01_0420__p1323701709">On the DLI management console, you can import data stored on OBS to DLI tables from <span class="wintitle" id="dli_01_0420__wintitle89211315417"><b>Data Management &gt; Databases and Tables &gt; Table Management</b></span> and <span class="wintitle" id="dli_01_0420__wintitle892143164111"><b>SQL Editor</b></span> pages. For details, see <a href="dli_01_0253.html">Importing Data to the Table</a>.</p>
</div>
<div class="section" id="dli_01_0420__section28435121978"><h4 class="sectiontitle">Importing Data Using CDM</h4><p id="dli_01_0420__p194281108614">Use the Cloud Data Migration (CDM) service to import data from OBS to DLI. You need to create a CDM queue first.</p>
<p id="dli_01_0420__p194387111764"></p>
<p id="dli_01_0420__p432410177">For details about how to create the queue, see "Migrating Data from OBS to DLI" in the <em id="dli_01_0420__i176611119191112">Cloud Data Migration User Guide</em>.</p>
<p id="dli_01_0420__p64282955162347">Pay attention to the following configurations:</p>
<ul id="dli_01_0420__ul64528752175931"><li id="dli_01_0420__li40214787175931">The VPC to which the DLI account belongs is the same as the VPC of the CDM queue.</li><li id="dli_01_0420__li2079024818231">You need to create two links, including a DLI link and an OBS link.</li><li id="dli_01_0420__li45910066182646">The format of the file to be transmitted can be <span class="parmvalue" id="dli_01_0420__parmvalue183613429422"><b>CSV</b></span> or <span class="parmvalue" id="dli_01_0420__parmvalue188361842134219"><b>JSON</b></span>.</li></ul>
</div>
<div class="section" id="dli_01_0420__section85731517984"><h4 class="sectiontitle">Importing Data Using DIS</h4><p id="dli_01_0420__p0948926193816">Use the Data Ingestion Service (DIS) service to import data to DLI. You need to create a DIS stream.</p>
<p id="dli_01_0420__p996411571081"></p>
<p id="dli_01_0420__p871914891">For details, see "Creating a DIS Stream" in <em id="dli_01_0420__i1756492817137">Data Ingestion Service User Guide</em>.</p>
<p id="dli_01_0420__p4665193704510">When configuring the DIS stream, set the <span class="parmname" id="dli_01_0420__parmname460764714217"><b>Dump Destination</b></span> to <span class="parmvalue" id="dli_01_0420__parmvalue1460774774210"><b>DLI</b></span> and select the database and table in DLI.</p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0513.html">Other Common Operations</a></div>
</div>
</div>

View File

@ -1,11 +1,14 @@
<a name="dli_01_0421"></a><a name="dli_01_0421"></a>
<h1 class="topictitle1">Creating a Message Notification Topic</h1>
<div id="body1566197386597"><p id="dli_01_0421__p95711691043">Once you have created a message notification topic, you can <strong id="dli_01_0421__b11606173513119">Add subscription</strong> of the topic on the <strong id="dli_01_0421__b1213852219334">Topic Management</strong> page of the Simple Message Notification service. You can select different ways (such as text messages or emails) to subscribe. After the subscription succeeds, any job failure will automatically be sent to your subscription endpoints. The operation is as follows:</p>
<ol id="dli_01_0421__ol25920582507"><li id="dli_01_0421__li9592558175016">On the <strong id="dli_01_0421__b1990823713261">Queue Management</strong> page, click <strong id="dli_01_0421__b1914153715265">Create SMN Topic</strong> on the upper left side. The <strong id="dli_01_0421__b15914113762619">Create SMN Topic</strong> dialog box is displayed.</li><li id="dli_01_0421__li144161849165115">Select a queue and click <strong id="dli_01_0421__b8335131614183">OK</strong>. The dialog box is displayed.<div class="note" id="dli_01_0421__note1848145063719"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0421__ul731513171718"><li id="dli_01_0421__li04613151711">You can select a single queue or all queues. If the subscription endpoints of a single queue are different from those of all queues, when a single queue and all queues are both selected, the SMN of all queues does not include the message of the single queue.</li><li id="dli_01_0421__li174141311718">After a notification topic is created, you will receive a notification only when a session or a batch job fails to be created.</li></ul>
<div id="body1566197386597"><div class="section" id="dli_01_0421__section5242489132"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0421__p1153362612142">Once you have created a message notification topic, you can <strong id="dli_01_0421__b596222242318">Add subscription</strong> of the topic on the <strong id="dli_01_0421__b179621022142319">Topic Management</strong> page of the Simple Message Notification service. You can select different ways (such as text messages or emails) to subscribe. After the subscription succeeds, any job failure will automatically be sent to your subscription endpoints.</p>
</div>
<div class="section" id="dli_01_0421__section742629201411"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0421__ol25920582507"><li id="dli_01_0421__li9592558175016">On the <strong id="dli_01_0421__b1990823713261">Queue Management</strong> page, click <strong id="dli_01_0421__b1914153715265">Create SMN Topic</strong> on the upper left side. The <strong id="dli_01_0421__b15914113762619">Create SMN Topic</strong> dialog box is displayed.</li><li id="dli_01_0421__li144161849165115">Select a queue and click <strong id="dli_01_0421__b8335131614183">OK</strong>.<div class="note" id="dli_01_0421__note1848145063719"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0421__ul731513171718"><li id="dli_01_0421__li780591161620">You can select a single queue or all queues.</li><li id="dli_01_0421__li04613151711">If you create a topic for a queue and another topic for all queues, the SMN of all queues does not include the message of the single queue.</li><li id="dli_01_0421__li174141311718">After a notification topic is created, you will receive a notification only when a session or a batch job fails to be created.</li></ul>
</div></div>
</li><li id="dli_01_0421__li10453153145415">Click <strong id="dli_01_0421__b159641031271">Topic Management</strong> to go to the <strong id="dli_01_0421__b6970236273">Topic Management</strong> page of the SMN service.</li><li id="dli_01_0421__li1720834116576">In the <strong id="dli_01_0421__b9213191510275">Operation</strong> column of the corresponding topic in , click <strong id="dli_01_0421__b422011514271">Add Subscription</strong>. Select <strong id="dli_01_0421__b7220141562711">Protocol</strong> to determine the subscription mode.<ul id="dli_01_0421__ul161132031182414"><li id="dli_01_0421__li765194572414">If you select <strong id="dli_01_0421__b1433681216278">Email</strong>, enter the email address for receiving the confirmation email in the <strong id="dli_01_0421__b13530125172710">Endpoint</strong> text box.</li></ul>
</li><li id="dli_01_0421__li105037206332">After you click the link in the email, you will receive a message indicating that the subscription is successful.</li><li id="dli_01_0421__li165897201922">Return to the <strong id="dli_01_0421__b2096810231281">Topic Management</strong> page and click <strong id="dli_01_0421__b596982314284">Subscriptions</strong> on the left. If the subscription status is <strong id="dli_01_0421__b096982315283">Confirmed</strong>, the subscription is successful.</li></ol>
<p id="dli_01_0421__p0261153411191"></p>
</li><li id="dli_01_0421__li10453153145415">Click <strong id="dli_01_0421__b955534693012">Topic Management</strong> in the prompt for a successfully created topic to go to the <strong id="dli_01_0421__b258111280317">Topic Management</strong> page of SMN.</li><li id="dli_01_0421__li1720834116576">In the <strong id="dli_01_0421__b9213191510275">Operation</strong> column of the topic, click <strong id="dli_01_0421__b422011514271">Add Subscription</strong>. Select <strong id="dli_01_0421__b7220141562711">Protocol</strong> to determine the subscription mode.<p id="dli_01_0421__p11811112417238"></p>
</li><li id="dli_01_0421__li105037206332">After you click the link in the email, you will receive a message indicating that the subscription is successful.</li><li id="dli_01_0421__li165897201922">Go to the <strong id="dli_01_0421__b123031745123314">Subscriptions</strong> page of SMN, and check that subscription status is <strong id="dli_01_0421__b2101122153418">Confirmed</strong>.</li></ol>
</div>
</div>
<div>
<div class="familylinks">

File diff suppressed because it is too large Load Diff

View File

@ -90,7 +90,7 @@
</td>
<td class="cellrowborder" valign="top" width="15.409999999999998%" headers="mcps1.3.4.3.2.6.1.2 "><p id="dli_01_0440__p35344449817">Database permissions</p>
</td>
<td class="cellrowborder" rowspan="3" valign="top" width="20.21%" headers="mcps1.3.4.3.2.6.1.3 "><p id="dli_01_0440__p145341344285">For details, see <a href="dli_01_0447.html">Database Permission Management</a> and <a href="dli_01_0448.html">Table Permission Management</a>.</p>
<td class="cellrowborder" rowspan="3" valign="top" width="20.21%" headers="mcps1.3.4.3.2.6.1.3 "><p id="dli_01_0440__p145341344285">For details, see <a href="dli_01_0447.html">Managing Database Permissions</a> and <a href="dli_01_0448.html">Managing Table Permissions</a>.</p>
</td>
<td class="cellrowborder" rowspan="3" valign="top" width="25.900000000000002%" headers="mcps1.3.4.3.2.6.1.4 "><p id="dli_01_0440__p85341944382">For details, see <strong id="dli_01_0440__b136361332281">SQL Syntax of Batch Jobs</strong> &gt; <strong id="dli_01_0440__b196945378287">Data Permissions Management</strong> &gt; <strong id="dli_01_0440__b1674340192815">Data Permissions List</strong> in the <em id="dli_01_0440__i192565412284">Data Lake Insight SQL Syntax Reference</em>.</p>
</td>

View File

@ -1,7 +1,7 @@
<a name="dli_01_0441"></a><a name="dli_01_0441"></a>
<h1 class="topictitle1">Common Operations Supported by DLI System Policy</h1>
<div id="body1569396916430"><p id="dli_01_0441__p14436205111210"><a href="#dli_01_0441__table168060107500">Table 1</a> lists the common operations supported by each system policy of DLI. Choose proper system policies according to this table. For details about the SQL statement permission matrix in DLI in terms of permissions on databases, tables, and roles, see <strong id="dli_01_0441__b1753410206019">SQL Syntax of Batch Jobs</strong> &gt; <strong id="dli_01_0441__b1553511201109">Data Permissions Management</strong> &gt; <strong id="dli_01_0441__b105365203011">Data Permissions List</strong> in the <em id="dli_01_0441__i19717256145416">Data Lake Insight SQL Syntax Reference</em>.</p>
<div id="body1569396916430"><p id="dli_01_0441__p14436205111210"><a href="#dli_01_0441__table168060107500">Table 1</a> lists the common operations supported by each system policy of DLI. Choose proper system policies according to this table. For details about the SQL statement permission matrix in DLI in terms of permissions on databases, tables, and roles, see <strong id="dli_01_0441__b111571242314">SQL Syntax of Batch Jobs</strong> &gt; <strong id="dli_01_0441__b11575420116">Data Permissions Management</strong> &gt; <strong id="dli_01_0441__b1415764216112">Data Permissions List</strong> in the <em id="dli_01_0441__i01571242711">Data Lake Insight SQL Syntax Reference</em>.</p>
<div class="tablenoborder"><a name="dli_01_0441__table168060107500"></a><a name="table168060107500"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0441__table168060107500" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Common operations supported by each system policy</caption><thead align="left"><tr id="dli_01_0441__en-us_topic_0206791772_row167851710165011"><th align="left" class="cellrowborder" valign="top" width="7.689231076892311%" id="mcps1.3.2.2.8.1.1"><p id="dli_01_0441__en-us_topic_0206791772_p4783161010506">Resources</p>
</th>

File diff suppressed because it is too large Load Diff

View File

@ -1,9 +1,9 @@
<a name="dli_01_0447"></a><a name="dli_01_0447"></a>
<h1 class="topictitle1">Database Permission Management</h1>
<h1 class="topictitle1">Managing Database Permissions</h1>
<div id="body1571736789703"><div class="section" id="dli_01_0447__section2883543204514"><h4 class="sectiontitle">Operation Scenarios</h4><ul id="dli_01_0447__ul6751144710454"><li id="dli_01_0447__li18752174718455">You can isolate databases allocated to different users by setting permissions to ensure data query performance.</li><li id="dli_01_0447__li16752047104516">The administrator and database owner have all permissions, which cannot be set or modified by other users.</li></ul>
</div>
<div class="section" id="dli_01_0447__section13393554152612"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0447__ul16414185422619"><li id="dli_01_0447__li44137546264">Lower-level objects automatically inherit permissions granted to upper-level objects. The hierarchical relationship is database &gt; table &gt; column.</li><li id="dli_01_0447__li971149172410">The database owner, table owner, and <span class="parmname" id="dli_01_0447__parmname13746731125417"><b>authorized</b></span> users can assign permissions on the database and tables.</li><li id="dli_01_0447__li10413145418269">Columns can only inherit the query permission. For details about <span class="parmname" id="dli_01_0447__parmname15888124214540"><b>Inheritable Permissions</b></span>, see <a href="dli_01_0447.html">Database Permission Management</a>.</li><li id="dli_01_0447__li16413175413264">The permissions can be revoked only at the initial level to which the permissions are granted. You need to grant and revoke permissions at the same level. You need to grant and revoke permissions at the same level. For example, after you are granted the insertion permission on a database, you can obtain the insertion permission on the tables in the database. Your insertion permission can be revoked only at the database level.</li><li id="dli_01_0447__li29052232217">If you create a database with the same name as a deleted database, the database permissions will not be inherited. In this case, you need to grant the database permissions to users or projects.<p id="dli_01_0447__p08812692719"><a name="dli_01_0447__li29052232217"></a><a name="li29052232217"></a>For example, user A is granted with the permission to delete the <strong id="dli_01_0447__b20228161610456">testdb</strong> database. Delete the database and create another one with the same name. You need to grant user A the deletion permission of the <strong id="dli_01_0447__b76882037184612">testdb</strong> database again.</p>
<div class="section" id="dli_01_0447__section13393554152612"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0447__ul16414185422619"><li id="dli_01_0447__li44137546264">Lower-level objects automatically inherit permissions granted to upper-level objects. The hierarchical relationship is database &gt; table &gt; column.</li><li id="dli_01_0447__li971149172410">The database owner, table owner, and <span class="parmname" id="dli_01_0447__parmname13746731125417"><b>authorized</b></span> users can assign permissions on the database and tables.</li><li id="dli_01_0447__li10413145418269">Columns can only inherit the query permission. For details about <span class="parmname" id="dli_01_0447__parmname15888124214540"><b>Inheritable Permissions</b></span>, see <a href="dli_01_0447.html">Managing Database Permissions</a>.</li><li id="dli_01_0447__li16413175413264">The permissions can be revoked only at the initial level to which the permissions are granted. You need to grant and revoke permissions at the same level. You need to grant and revoke permissions at the same level. For example, after you are granted the insertion permission on a database, you can obtain the insertion permission on the tables in the database. Your insertion permission can be revoked only at the database level.</li><li id="dli_01_0447__li29052232217">If you create a database with the same name as a deleted database, the database permissions will not be inherited. In this case, you need to grant the database permissions to users or projects.<p id="dli_01_0447__p08812692719"><a name="dli_01_0447__li29052232217"></a><a name="li29052232217"></a>For example, user A is granted with the permission to delete the <strong id="dli_01_0447__b20228161610456">testdb</strong> database. Delete the database and create another one with the same name. You need to grant user A the deletion permission of the <strong id="dli_01_0447__b76882037184612">testdb</strong> database again.</p>
</li></ul>
</div>
<div class="section" id="dli_01_0447__section634510674716"><h4 class="sectiontitle">Viewing Database Permissions</h4><ol id="dli_01_0447__ol935515484613"><li id="dli_01_0447__li43554544462">On the left of the management console, choose <span class="uicontrol" id="dli_01_0447__uicontrol6899187203016"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0447__uicontrol148994793012"><b>Databases and Tables</b></span>.</li><li id="dli_01_0447__li2356754154613">Locate the row where the target database resides and click <strong id="dli_01_0447__b2937183285519">Manage Permissions</strong> in the <strong id="dli_01_0447__b4937133220554">Operation</strong> column.<p id="dli_01_0447__p136751243476">Permissions can be granted to new users or projects, modified for users or projects with existing permissions, or revoked from a user or project.</p>

View File

@ -1,6 +1,6 @@
<a name="dli_01_0448"></a><a name="dli_01_0448"></a>
<h1 class="topictitle1">Table Permission Management</h1>
<h1 class="topictitle1">Managing Table Permissions</h1>
<div id="body1571736789703"><div class="section" id="dli_01_0448__section2883543204514"><h4 class="sectiontitle">Operation Scenario</h4><ul id="dli_01_0448__ul6751144710454"><li id="dli_01_0448__li18752174718455">You can isolate databases allocated to different users by setting permissions to ensure data query performance.</li><li id="dli_01_0448__li16752047104516">The administrator and database owner have all permissions, which cannot be set or modified by other users.</li><li id="dli_01_0448__li475264711458">When setting database permissions for a new user, ensure that the user group to which the user belongs has the <strong id="dli_01_0448__b17434485377">Tenant Guest</strong> permission. </li></ul>
</div>
<div class="section" id="dli_01_0448__section9719284599"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0448__ul849031211304"><li id="dli_01_0448__li29052232217">If you create a table with the same name as a deleted table, the table permissions will not be inherited. In this case, you need to grant the table permissions to users or projects.<p id="dli_01_0448__p08812692719"><a name="dli_01_0448__li29052232217"></a><a name="li29052232217"></a>For example, user A is granted with the permission to delete the <strong id="dli_01_0448__b1382451218915">testTable</strong> table. Delete the table and create another one with the same name. You need to grant user A the deletion permission of the <strong id="dli_01_0448__b165131017912">testTable</strong> table again.</p>
@ -55,21 +55,21 @@
</thead>
<tbody><tr id="dli_01_0448__row565224044819"><td class="cellrowborder" valign="top" width="18.75%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.1 "><p id="dli_01_0448__p9652144074816">Authorization Object</p>
</td>
<td class="cellrowborder" valign="top" width="81.25%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.2 "><p id="dli_01_0448__p14652174017486">Select <strong id="dli_01_0448__b1179828778">User</strong> or <strong id="dli_01_0448__b988846992">Project</strong>.</p>
<td class="cellrowborder" valign="top" width="81.25%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.2 "><p id="dli_01_0448__p14652174017486">Select <strong id="dli_01_0448__b1908672938">User</strong> or <strong id="dli_01_0448__b746750941">Project</strong>.</p>
</td>
</tr>
<tr id="dli_01_0448__row166532040114818"><td class="cellrowborder" valign="top" width="18.75%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.1 "><p id="dli_01_0448__p565219406486">Username/Project</p>
</td>
<td class="cellrowborder" valign="top" width="81.25%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.2 "><ul id="dli_01_0448__ul126532403482"><li id="dli_01_0448__li16652440194810">If you select <strong id="dli_01_0448__b863505281">User</strong>, enter the IAM username when granting table permissions to the user.<div class="note" id="dli_01_0448__note42264017164"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0448__p2232130161618">The username is an existing IAM user name and has logged in to the DLI management console.</p>
<td class="cellrowborder" valign="top" width="81.25%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.2 "><ul id="dli_01_0448__ul126532403482"><li id="dli_01_0448__li16652440194810">If you select <strong id="dli_01_0448__b1182990332">User</strong>, enter the IAM username when granting table permissions to the user.<div class="note" id="dli_01_0448__note42264017164"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0448__p2232130161618">The username is an existing IAM user name and has logged in to the DLI management console.</p>
</div></div>
</li><li id="dli_01_0448__li765312406486">If you select <strong id="dli_01_0448__b1921239249">Project</strong>, select the project to be authorized in the current region.<div class="note" id="dli_01_0448__note5437185910377"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0448__p3437159143716">If you select <strong id="dli_01_0448__b222310053715">Project</strong>, you can only view information about the authorized tables and their databases.</p>
</li><li id="dli_01_0448__li765312406486">If you select <strong id="dli_01_0448__b1209073757">Project</strong>, select the project to be authorized in the current region.<div class="note" id="dli_01_0448__note5437185910377"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0448__p3437159143716">If you select <strong id="dli_01_0448__b222310053715">Project</strong>, you can only view information about the authorized tables and their databases.</p>
</div></div>
</li></ul>
</td>
</tr>
<tr id="dli_01_0448__row116546401484"><td class="cellrowborder" valign="top" width="18.75%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.1 "><p id="dli_01_0448__p0653184018489">Non-inheritable Permissions</p>
</td>
<td class="cellrowborder" valign="top" width="81.25%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.2 "><div class="p" id="dli_01_0448__p26531940154817">Select a permission to grant it to the user, or deselect a permission to revoke it.<ul id="dli_01_0448__ul186541940194819"><li id="dli_01_0448__li1037614422018">The following permissions are applicable to both user and project authorization:<ul id="dli_01_0448__ul11414111319209"><li id="dli_01_0448__li17280185220227"><strong id="dli_01_0448__b1475611842911">View Table Creation Statement</strong>: This permission allows you to view the statement for creating the current table.</li><li id="dli_01_0448__li6385363379"><strong id="dli_01_0448__b88661213112914">View Table Information</strong>: This permission allows you to view information about the current table.</li><li id="dli_01_0448__li1365364034816"><strong id="dli_01_0448__b1580061494">Select Table</strong>: This permission allows you to query data of the current table.</li><li id="dli_01_0448__li9653240154816"><strong id="dli_01_0448__b565748211">Drop Table</strong>: This permission allows you to delete the current table.</li><li id="dli_01_0448__li56531540144810"><strong id="dli_01_0448__b1500987642">Rename Table</strong>: Rename the current table.</li><li id="dli_01_0448__li1465374011487"><strong id="dli_01_0448__b1012148532">Insert</strong>: This permission allows you to insert data into the current table.</li><li id="dli_01_0448__li1965364014817"><strong id="dli_01_0448__b1487197387">Overwrite</strong>: This permission allows you to insert data to overwrite the data in the current table.</li><li id="dli_01_0448__li6653194011481"><strong id="dli_01_0448__b1868414236293">Add Column</strong>: This permission allows you to add columns to the current table.</li><li id="dli_01_0448__li186531240124810"><strong id="dli_01_0448__b11229173814372">Grant Permission</strong>: This permission allows you to grant table permissions to other users or projects.</li><li id="dli_01_0448__li66531440174811"><strong id="dli_01_0448__b1752995116373">Revoke Permission</strong>: This permission allows you to revoke the table's permissions that other users or projects have but cannot revoke the table owner's permissions.</li><li id="dli_01_0448__li8654194024817"><strong id="dli_01_0448__b814732953819">View Other Users' Permissions</strong>: This permission allows you to query other users' permission on the current table.</li></ul>
<td class="cellrowborder" valign="top" width="81.25%" headers="mcps1.3.4.2.1.2.2.2.3.2.3.1.2 "><div class="p" id="dli_01_0448__p26531940154817">Select a permission to grant it to the user, or deselect a permission to revoke it.<ul id="dli_01_0448__ul186541940194819"><li id="dli_01_0448__li1037614422018">The following permissions are applicable to both user and project authorization:<ul id="dli_01_0448__ul11414111319209"><li id="dli_01_0448__li17280185220227"><strong id="dli_01_0448__b1475611842911">View Table Creation Statement</strong>: This permission allows you to view the statement for creating the current table.</li><li id="dli_01_0448__li6385363379"><strong id="dli_01_0448__b88661213112914">View Table Information</strong>: This permission allows you to view information about the current table.</li><li id="dli_01_0448__li1365364034816"><strong id="dli_01_0448__b2011496059">Select Table</strong>: This permission allows you to query data of the current table.</li><li id="dli_01_0448__li9653240154816"><strong id="dli_01_0448__b614804474">Drop Table</strong>: This permission allows you to delete the current table.</li><li id="dli_01_0448__li56531540144810"><strong id="dli_01_0448__b1808897767">Rename Table</strong>: Rename the current table.</li><li id="dli_01_0448__li1465374011487"><strong id="dli_01_0448__b340672502">Insert</strong>: This permission allows you to insert data into the current table.</li><li id="dli_01_0448__li1965364014817"><strong id="dli_01_0448__b254284073">Overwrite</strong>: This permission allows you to insert data to overwrite the data in the current table.</li><li id="dli_01_0448__li6653194011481"><strong id="dli_01_0448__b1868414236293">Add Column</strong>: This permission allows you to add columns to the current table.</li><li id="dli_01_0448__li186531240124810"><strong id="dli_01_0448__b11229173814372">Grant Permission</strong>: This permission allows you to grant table permissions to other users or projects.</li><li id="dli_01_0448__li66531440174811"><strong id="dli_01_0448__b1752995116373">Revoke Permission</strong>: This permission allows you to revoke the table's permissions that other users or projects have but cannot revoke the table owner's permissions.</li><li id="dli_01_0448__li8654194024817"><strong id="dli_01_0448__b814732953819">View Other Users' Permissions</strong>: This permission allows you to query other users' permission on the current table.</li></ul>
<div class="p" id="dli_01_0448__p18654640154814">The partition table also has the following permissions:<ul id="dli_01_0448__ul76544409484"><li id="dli_01_0448__li1865415406488"><strong id="dli_01_0448__b46593558555">Add Partition</strong>: This permission allows you to add a partition to a partition table.</li><li id="dli_01_0448__li126548407483"><strong id="dli_01_0448__b126796584556">Delete Partition</strong>: This permission allows you to delete existing partitions from a partition table.</li><li id="dli_01_0448__li965474084819"><strong id="dli_01_0448__b523170183913">Configure Path for Partition</strong>: This permission allows you to set the path of a partition in a partition table to a specified OBS path.</li><li id="dli_01_0448__li1065424019489"><strong id="dli_01_0448__b69983116565">Rename Table Partition</strong>: This permission allows you to rename partitions in a partition table.</li><li id="dli_01_0448__li126547402487"><strong id="dli_01_0448__b088993155619">Restore Table Partition</strong>: This permission allows you to export partition information from the file system and save the information to metadata.</li><li id="dli_01_0448__li86541140134815"><strong id="dli_01_0448__b323918620561">View All Partitions</strong>: This permission allows you to view all partitions in a partition table.</li></ul>
</div>
</li></ul>
@ -92,7 +92,7 @@
</thead>
<tbody><tr id="dli_01_0448__row20659134011489"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.4.2.1.2.2.3.3.2.3.1.1 "><p id="dli_01_0448__p1265724017483">Authorization Object</p>
</td>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.4.2.1.2.2.3.3.2.3.1.2 "><p id="dli_01_0448__p565864034813">Select <strong id="dli_01_0448__b2001186953">User</strong> or <strong id="dli_01_0448__b2001744462">Project</strong>.</p>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.4.2.1.2.2.3.3.2.3.1.2 "><p id="dli_01_0448__p565864034813">Select <strong id="dli_01_0448__b795663969">User</strong> or <strong id="dli_01_0448__b576928764">Project</strong>.</p>
</td>
</tr>
<tr id="dli_01_0448__row19659140144815"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.4.2.1.2.2.3.3.2.3.1.1 "><p id="dli_01_0448__p1865915409485">Username/Project</p>

File diff suppressed because it is too large Load Diff

View File

@ -1,6 +1,6 @@
<a name="dli_01_0455"></a><a name="dli_01_0455"></a>
<h1 class="topictitle1">Creating a Flink SQL job</h1>
<h1 class="topictitle1">Creating a Flink SQL Job</h1>
<div id="body1574761276020"><p id="dli_01_0455__p1426222391410">This section describes how to create a Flink SQL job. You can use Flink SQLs to develop jobs to meet your service requirements. Using SQL statements simplifies logic implementation. You can edit Flink SQL statements for your job in the DLI SQL editor. This section describes how to use the SQL editor to write Flink SQL statements.</p>
<p id="dli_01_0455__p0792237205215"><a href="dli_01_0498.html">DLI Flink OpenSource SQL jobs</a> are fully compatible with the syntax of Flink 1.10 and 1.12 provided by the community. In addition, Redis, DWS(GaussDB), and DIS data source types are added based on the community connector.</p>
<div class="section" id="dli_01_0455__section9243114805018"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_01_0455__ul32010161007"><li id="dli_01_0455__li142013162002">You have prepared the data input and data output channels. For details, see <a href="dli_01_0454.html">Preparing Flink Job Data</a>.</li><li id="dli_01_0455__li7520181309">When you use a Flink SQL job to access other external data sources, such as OpenTSDB, HBase, Kafka, DWS, RDS, CSS, CloudTable, DCS Redis, and DDS Mongos, you need to create a cross-source connection to connect the job running queue to the external data source.<ul id="dli_01_0455__ul3477133419715"><li id="dli_01_0455__li18451528175">For details about the external data sources that can be accessed by Flink jobs, see <a href="dli_01_0410.html">Datasource Connection and Cross-Source Analysis</a>.</li><li id="dli_01_0455__li28469131945">For details about how to create a datasource connection, see <a href="dli_01_0426.html">Enhanced Datasource Connections</a>. After a datasource connection is created, you can choose <strong id="dli_01_0455__b136411947171620">More</strong> &gt; <strong id="dli_01_0455__b8991851171618">Test Address Connectivity</strong> in the <strong id="dli_01_0455__b2161654131616">Operation</strong> column on the <strong id="dli_01_0455__b65150213175">Queue Management</strong> page to check whether the network connection between the queue and the external data source is normal. For details, see <a href="dli_01_0489.html">Testing Address Connectivity</a>.</li></ul>
@ -32,7 +32,7 @@
</tr>
<tr id="dli_01_0455__row53646587415"><td class="cellrowborder" valign="top" width="20%" headers="mcps1.3.4.2.3.2.1.2.3.1.1 "><p id="dli_01_0455__p2364958104115">Template Name</p>
</td>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.4.2.3.2.1.2.3.1.2 "><p id="dli_01_0455__p15364458144115">You can select a sample template or a custom job template. For details about templates, see <a href="dli_01_0464.html">Flink Template Management</a>.</p>
<td class="cellrowborder" valign="top" width="80%" headers="mcps1.3.4.2.3.2.1.2.3.1.2 "><p id="dli_01_0455__p15364458144115">You can select a sample template or a custom job template. For details about templates, see <a href="dli_01_0464.html">Managing Flink Templates</a>.</p>
</td>
</tr>
</tbody>
@ -63,7 +63,7 @@
</tr>
<tr id="dli_01_0455__row16158184717215"><td class="cellrowborder" valign="top" width="22.189999999999998%" headers="mcps1.3.4.2.7.2.1.2.3.1.1 "><p id="dli_01_0455__p1015810474212">CUs</p>
</td>
<td class="cellrowborder" valign="top" width="77.81%" headers="mcps1.3.4.2.7.2.1.2.3.1.2 "><p id="dli_01_0455__p71588471329">Sum of the number of compute units and job manager CUs of DLI. One CU equals one vCPU and 4 GB.</p>
<td class="cellrowborder" valign="top" width="77.81%" headers="mcps1.3.4.2.7.2.1.2.3.1.2 "><p id="dli_01_0455__p71588471329">Sum of the number of compute units and job manager CUs of DLI. One CU equals 1 vCPU and 4 GB.</p>
<p id="dli_01_0455__p315820478211">The configured number of CUs is the number of CUs required for job running and cannot exceed the number of CUs in the bound queue.</p>
</td>
</tr>
@ -99,7 +99,7 @@
<div class="caution" id="dli_01_0455__note01592479215"><span class="cautiontitle"> CAUTION: </span><div class="cautionbody"><p id="dli_01_0455__p9159947824">You are advised to select this parameter. Otherwise, no run log is generated after the job is executed. If the job is abnormal, the run log cannot be obtained for fault locating.</p>
</div></div>
<p id="dli_01_0455__p141596472219">If this option is selected, you need to set the following parameters:</p>
<div class="p" id="dli_01_0455__p10159647525"><span class="parmname" id="dli_01_0455__parmname2095574317192"><b>OBS Bucket</b></span>: Select an OBS bucket to store user job logs. If the selected OBS bucket is not authorized, click <span class="uicontrol" id="dli_01_0455__uicontrol17351463209"><b>Authorize</b></span>.<div class="note" id="dli_01_0455__note515914471629"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0455__p81591547627">If both <span class="uicontrol" id="dli_01_0455__uicontrol665694969"><b>Enable Checkpointing</b></span> and <span class="uicontrol" id="dli_01_0455__uicontrol845751304"><b>Save Job Log</b></span> are selected, you only need to authorize OBS once.</p>
<div class="p" id="dli_01_0455__p10159647525"><span class="parmname" id="dli_01_0455__parmname2095574317192"><b>OBS Bucket</b></span>: Select an OBS bucket to store user job logs. If the selected OBS bucket is not authorized, click <span class="uicontrol" id="dli_01_0455__uicontrol17351463209"><b>Authorize</b></span>.<div class="note" id="dli_01_0455__note515914471629"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0455__p81591547627">If both <span class="uicontrol" id="dli_01_0455__uicontrol2030436459"><b>Enable Checkpointing</b></span> and <span class="uicontrol" id="dli_01_0455__uicontrol22834079"><b>Save Job Log</b></span> are selected, you only need to authorize OBS once.</p>
</div></div>
</div>
</td>

File diff suppressed because it is too large Load Diff

View File

@ -5,11 +5,12 @@
<div class="note" id="dli_01_0458__note117456916519"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0458__ul11529173814143"><li id="dli_01_0458__li753012384147">Currently, only Flink SQL jobs support this function.</li><li id="dli_01_0458__li1753093891415">The job debugging function is used only to verify the SQL logic and does not involve data write operations.</li></ul>
</div></div>
<div class="section" id="dli_01_0458__section610115011434"><h4 class="sectiontitle">Procedures</h4><ol id="dli_01_0458__ol077443514394"><li id="dli_01_0458__li561395535111"><span>In the left navigation pane of the DLI management console, choose <span class="menucascade" id="dli_01_0458__menucascade1078623412516"><b><span class="uicontrol" id="dli_01_0458__uicontrol4780133413257">Job Management</span></b></span> &gt; <span class="menucascade" id="dli_01_0458__menucascade778673418253"><b><span class="uicontrol" id="dli_01_0458__uicontrol15786123452515">Flink Jobs</span></b></span>. The <span class="wintitle" id="dli_01_0458__wintitle378733432519"><b>Flink Jobs</b></span> page is displayed.</span></li><li id="dli_01_0458__li161312138523"><span>In the <span class="menucascade" id="dli_01_0458__menucascade4828124195311"><b><span class="uicontrol" id="dli_01_0458__uicontrol1882819241538">Operation</span></b></span> column of the created Flink SQL job, click <span class="uicontrol" id="dli_01_0458__uicontrol1988815318542"><b>Edit</b></span>. The page for editing the Flink SQL job is displayed.</span><p><p id="dli_01_0458__p78341740175220">For a job that is being created, you can debug the job on the editing page.</p>
</p></li><li id="dli_01_0458__li2280172114275"><span>Click <span class="uicontrol" id="dli_01_0458__uicontrol7981124473618"><b>Debug</b></span> above the SQL editing box to parse the edited SQL statements. The <span class="menucascade" id="dli_01_0458__menucascade938109471145557"><b><span class="uicontrol" id="dli_01_0458__uicontrol1350739082144221">Debugging Parameters</span></b></span> tab is displayed on the right of the page.</span><p><p id="dli_01_0458__p3589113213413"></p>
<ul id="dli_01_0458__ul9991625164015"><li id="dli_01_0458__li1499012258408"><span class="parmname" id="dli_01_0458__parmname1723183812345"><b>Dump Bucket</b></span>: Select an OBS bucket to save debugging logs. If you select an unauthorized OBS bucket, click <span class="uicontrol" id="dli_01_0458__uicontrol6231938133413"><b>Authorize</b></span>.</li><li id="dli_01_0458__li6991172524017"><span class="parmname" id="dli_01_0458__parmname93421511205614"><b>Data Input Mode</b></span>: You can select CSV data stored in the OBS bucket or manually enter the data.<ul id="dli_01_0458__ul799192584014"><li id="dli_01_0458__li0991425154011"><span class="parmvalue" id="dli_01_0458__parmvalue2165183911566"><b>OBS (CSV)</b></span><p id="dli_01_0458__p17991142519401">If you select this value, prepare OBS data first before using DLI. For details, see <a href="dli_01_0454.html">Preparing Flink Job Data</a>. OBS data is stored in CSV format, where multiple records are separated by line breaks and different fields in a single record are separated by commas (,). In addition, you need to select a specific object in OBS as the input source data.</p>
<p id="dli_01_0458__p11991325134014"></p>
</li><li id="dli_01_0458__li1991192514401"><span class="parmvalue" id="dli_01_0458__parmvalue205262292463"><b>Manual typing</b></span><p id="dli_01_0458__p3991152517404">If you select this value, compile SQL statements as data sources. In this mode, you need to enter the value of each field in a single record.</p>
<p id="dli_01_0458__p8991142584014"></p>
</p></li><li id="dli_01_0458__li2280172114275"><span>Click <span class="uicontrol" id="dli_01_0458__uicontrol7981124473618"><b>Debug</b></span> above the SQL editing box to parse the edited SQL statements. The <span class="menucascade" id="dli_01_0458__menucascade938109471145557"><b><span class="uicontrol" id="dli_01_0458__uicontrol1350739082144221">Debugging Parameters</span></b></span> tab is displayed on the right of the page.</span><p><p id="dli_01_0458__p1521749101214"></p>
<p id="dli_01_0458__p052213492120"></p>
<ul id="dli_01_0458__ul2052214931214"><li id="dli_01_0458__li1452254991219"><span class="parmname" id="dli_01_0458__parmname1723183812345"><b>Dump Bucket</b></span>: Select an OBS bucket to save debugging logs. If you select an unauthorized OBS bucket, click <span class="uicontrol" id="dli_01_0458__uicontrol6231938133413"><b>Authorize</b></span>.</li><li id="dli_01_0458__li125229499124"><span class="parmname" id="dli_01_0458__parmname93421511205614"><b>Data Input Mode</b></span>: You can select CSV data stored in the OBS bucket or manually enter the data.<ul id="dli_01_0458__ul3522104912122"><li id="dli_01_0458__li25221949171211"><span class="parmvalue" id="dli_01_0458__parmvalue2165183911566"><b>OBS (CSV)</b></span><p id="dli_01_0458__p1552264991217">If you select this value, prepare OBS data first before using DLI. For details, see <a href="dli_01_0454.html">Preparing Flink Job Data</a>. OBS data is stored in CSV format, where multiple records are separated by line breaks and different fields in a single record are separated by commas (,). In addition, you need to select a specific object in OBS as the input source data.</p>
<p id="dli_01_0458__p17522134961216"></p>
</li><li id="dli_01_0458__li552254921212"><span class="parmvalue" id="dli_01_0458__parmvalue109470199816"><b>Manual typing</b></span><p id="dli_01_0458__p125221249181216">If you select this value, compile SQL statements as data sources. In this mode, you need to enter the value of each field in a single record.</p>
<p id="dli_01_0458__p7522174916121"></p>
</li></ul>
</li></ul>
</p></li><li id="dli_01_0458__li15234164712911"><span>Click <span class="uicontrol" id="dli_01_0458__uicontrol891873687145739"><b>Start Debugging</b></span>. Once debugging is complete, the <span class="menucascade" id="dli_01_0458__menucascade13235934814584"><b><span class="uicontrol" id="dli_01_0458__uicontrol209383070414584">Debugging Result</span></b></span> page appears.</span><p><ul id="dli_01_0458__ul293616433915"><li id="dli_01_0458__li18935143391">If the debugging result meets the expectation, the job is running properly.</li><li id="dli_01_0458__li793614438920">If the debugging result does not meet the expectation, business logic errors may have occurred. In this case, modify SQL statements and conduct debugging again.</li></ul>

View File

@ -19,7 +19,7 @@
</div>
<div class="section" id="dli_01_0461__section8678193324114"><a name="dli_01_0461__section8678193324114"></a><a name="section8678193324114"></a><h4 class="sectiontitle">Stopping a Job</h4><p id="dli_01_0461__p10413556134214">You can stop a job in the <strong id="dli_01_0461__b199511322168">Running</strong> or <span class="parmvalue" id="dli_01_0461__parmvalue16170152024918"><b>Submitting</b></span> state.</p>
<ol id="dli_01_0461__ol999765714431"><li id="dli_01_0461__li1537162515919"><span>In the left navigation pane of the DLI management console, choose <span class="menucascade" id="dli_01_0461__menucascade4839243185810"><b><span class="uicontrol" id="dli_01_0461__uicontrol7834114315813">Job Management</span></b></span> &gt; <span class="menucascade" id="dli_01_0461__menucascade3839143155817"><b><span class="uicontrol" id="dli_01_0461__uicontrol17839124317581">Flink Jobs</span></b></span>. The <span class="wintitle" id="dli_01_0461__wintitle984019439581"><b>Flink Jobs</b></span> page is displayed.</span></li><li id="dli_01_0461__li97556283351"><span>Stop a job using either of the following methods:</span><p><ul id="dli_01_0461__ul1663022133517"><li id="dli_01_0461__li1684613360294">Stopping a job<p id="dli_01_0461__p539375319543"><a name="dli_01_0461__li1684613360294"></a><a name="li1684613360294"></a>Locate the row that contains the job to be stopped, click <span class="uicontrol" id="dli_01_0461__uicontrol14303162712597"><b>More</b></span> in the <span class="parmname" id="dli_01_0461__parmname5957193318596"><b>Operation</b></span> column, and select <span class="uicontrol" id="dli_01_0461__uicontrol34263915919"><b>Stop</b></span>.</p>
<p id="dli_01_0461__p1224864211298">Alternatively, you can select the row where the job you want to stop locates and click <span class="uicontrol" id="dli_01_0461__uicontrol2034148533"><b>Stop</b></span> in the upper left of the job list.</p>
<p id="dli_01_0461__p1224864211298">Alternatively, you can select the row where the job you want to stop locates and click <span class="uicontrol" id="dli_01_0461__uicontrol139240364"><b>Stop</b></span> in the upper left of the job list.</p>
</li><li id="dli_01_0461__li103984412300">Batch stopping jobs<p id="dli_01_0461__p1176518147308"><a name="dli_01_0461__li103984412300"></a><a name="li103984412300"></a>Locate the rows containing the jobs you want to stop and click <span class="uicontrol" id="dli_01_0461__uicontrol1927821713351"><b>Stop</b></span> in the upper left of the job list.</p>
</li></ul>
</p></li><li id="dli_01_0461__li14333204535219"><span>In the displayed <span class="wintitle" id="dli_01_0461__wintitle1796120483017"><b>Stop Job</b></span> dialog box, click <span class="uicontrol" id="dli_01_0461__uicontrol1453612541409"><b>OK</b></span> to stop the job.</span><p><div class="p" id="dli_01_0461__p167175381238"><div class="note" id="dli_01_0461__note16893119216"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0461__ul186891211102111"><li id="dli_01_0461__li768881110218">Before stopping a job, you can trigger a savepoint to save the job status information. When you start the job again, you can choose whether to restore the job from the savepoint.</li><li id="dli_01_0461__li5688161116219">If you select <strong id="dli_01_0461__b1533414174116">Trigger savepoint</strong>, a savepoint is created. If <strong id="dli_01_0461__b108781513154118">Trigger savepoint</strong> is not selected, no savepoint is created. By default, the savepoint function is disabled.</li><li id="dli_01_0461__li106891811102110">The lifecycle of a savepoint starts when the savepoint is triggered and stops the job, and ends when the job is restarted. The savepoint is automatically deleted after the job is restarted.</li></ul>
@ -47,7 +47,7 @@
</div>
<div class="section" id="dli_01_0461__section75781665389"><a name="dli_01_0461__section75781665389"></a><a name="section75781665389"></a><h4 class="sectiontitle">Importing a Job</h4><p id="dli_01_0461__p195783616382">You can import the Flink job configuration file stored in the OBS bucket to the <strong id="dli_01_0461__b462516574455">Flink Jobs</strong> page of DLI.</p>
<p id="dli_01_0461__p1629324710420">This mode is applicable to the scenario where a large number of jobs need to be created when you switch to another region, project, or user. In this case, you do not need to create a job. You only need to export the original job, log in to the system in a new region or project, or use a new user to import the job.</p>
<p id="dli_01_0461__p474110307448">If you need to import a self-created job, you are advised to use the job creation function. For details, see <a href="dli_01_0455.html">Creating a Flink SQL job</a>, <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</p>
<p id="dli_01_0461__p474110307448">If you need to import a self-created job, you are advised to use the job creation function. For details, see <a href="dli_01_0455.html">Creating a Flink SQL Job</a>, <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</p>
<div class="note" id="dli_01_0461__note10293547104214"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0461__ul152587411449"><li id="dli_01_0461__li1425816414446">When switching to another project or user, you need to grant permissions to the new project or user. For details, see <a href="dli_01_0479.html">Managing Flink Job Permissions</a>.</li><li id="dli_01_0461__li125913474417">Only jobs whose data format is the same as that of Flink jobs exported from DLI can be imported.</li></ul>
</div></div>
<ol id="dli_01_0461__ol55781266387"><li id="dli_01_0461__li1757816618386"><span>In the left navigation pane of the DLI management console, choose <span class="menucascade" id="dli_01_0461__menucascade113281655173817"><b><span class="uicontrol" id="dli_01_0461__uicontrol183274559388">Job Management</span></b></span> &gt; <span class="menucascade" id="dli_01_0461__menucascade432875583817"><b><span class="uicontrol" id="dli_01_0461__uicontrol1932814554388">Flink Jobs</span></b></span>. The <span class="wintitle" id="dli_01_0461__wintitle1432819554381"><b>Flink Jobs</b></span> page is displayed.</span></li></ol><ol start="2" id="dli_01_0461__ol19578136113820"><li id="dli_01_0461__li12976193720419"><span>Click <span class="uicontrol" id="dli_01_0461__uicontrol78907524411"><b>Import Job</b></span> in the upper right corner. The <span class="wintitle" id="dli_01_0461__wintitle10552525174113"><b>Import Job</b></span> dialog box is displayed.</span><p><p id="dli_01_0461__p778004722517"></p>
@ -58,7 +58,9 @@
<ol id="dli_01_0461__ol313811191119"><li id="dli_01_0461__li613891119111"><span>In the left navigation pane of the DLI management console, choose <span class="menucascade" id="dli_01_0461__menucascade13641126848"><b><span class="uicontrol" id="dli_01_0461__uicontrol19641326544">Job Management</span></b></span> &gt; <span class="menucascade" id="dli_01_0461__menucascade36411626344"><b><span class="uicontrol" id="dli_01_0461__uicontrol6641826045">Flink Jobs</span></b></span>. The <span class="wintitle" id="dli_01_0461__wintitle064217268412"><b>Flink Jobs</b></span> page is displayed.</span></li><li id="dli_01_0461__li1616613121426"><span>In the <span class="parmname" id="dli_01_0461__parmname6935035142"><b>Operation</b></span> column of the job whose name and description need to be modified, choose <span class="menucascade" id="dli_01_0461__menucascade569213064314"><b><span class="uicontrol" id="dli_01_0461__uicontrol1569210011431">More &gt; Modify Name and Description</span></b></span>. The <span class="wintitle" id="dli_01_0461__wintitle174868211557"><b>Modify Name and Description</b></span> dialog box is displayed. Change the name or modify the description of a job.</span></li><li id="dli_01_0461__li1365245111415"><span>Click <span class="uicontrol" id="dli_01_0461__uicontrol1555710432511"><b>OK</b></span>.</span></li></ol>
</div>
<p id="dli_01_0461__p1487985091717"></p>
<div class="section" id="dli_01_0461__section11401152191015"><a name="dli_01_0461__section11401152191015"></a><a name="section11401152191015"></a><h4 class="sectiontitle">Triggering a Savepoint</h4><p id="dli_01_0461__p164071198341">You can click <strong id="dli_01_0461__b9543154513574">Trigger Savepoint</strong> for jobs in the <strong id="dli_01_0461__b142701313135713">Running</strong> status to save the job status.</p>
<div class="section" id="dli_01_0461__section11401152191015"><a name="dli_01_0461__section11401152191015"></a><a name="section11401152191015"></a><h4 class="sectiontitle">Triggering a Savepoint</h4><p id="dli_01_0461__p2035416015116">When you need to stop a job, you can create a savepoint to save the job status information. In this case, when you restart the job, you can choose to restore the job from the latest savepoint.</p>
<div class="note" id="dli_01_0461__note15163351205819"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0461__ul1163145111589"><li id="dli_01_0461__li1918412201318">You can click <strong id="dli_01_0461__b9956103515541">Trigger Savepoint</strong> for jobs in the <strong id="dli_01_0461__b7956173595411">Running</strong> status to save the job status.</li><li id="dli_01_0461__li171632515583">The lifecycle of a savepoint starts when the savepoint is triggered and stops the job, and ends when the job is restarted. The savepoint is automatically deleted after the job is restarted.</li></ul>
</div></div>
</div>
<div class="section" id="dli_01_0461__section83412445175"><a name="dli_01_0461__section83412445175"></a><a name="section83412445175"></a><h4 class="sectiontitle">Importing to a Savepoint</h4><p id="dli_01_0461__p16330125199">You can import a savepoint to restore the job status. For details about the savepoint, see <a href="https://nightlies.apache.org/flink/flink-docs-release-1.10/dev/stream/state/checkpointing.html" target="_blank" rel="noopener noreferrer">Checkpointing</a> at the official website of Flink.</p>
<p id="dli_01_0461__p559192761014">You need to select the OBS path of the save point.</p>

File diff suppressed because it is too large Load Diff

View File

@ -4,56 +4,70 @@
<div id="body1579424003555"><p id="dli_01_0475__p637933145820">Request conditions are useful in determining when a custom policy takes effect. A request condition consists of a condition key and operator. Condition keys are either global or service-level and are used in the Condition element of a policy statement. Global condition keys (starting with <strong id="dli_01_0475__b62171450588">g:</strong>) are available for operations of all services, while service-level condition keys (starting with a service name such as <strong id="dli_01_0475__b22183505811">dli</strong>) are available only for operations of a specific service. An operator is used together with a condition key to form a complete condition statement.</p>
<p id="dli_01_0475__p1838123313587">IAM provides a set of DLI predefined condition keys. The following table lists the predefined condition keys of <em id="dli_01_0475__i25891446131220">DLI</em>.</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0475__table198664321589" frame="border" border="1" rules="all"><caption><b>Table 1 </b>DLI request conditions</caption><thead align="left"><tr id="dli_01_0475__dli_01_0451_row1538733145813"><th align="left" class="cellrowborder" valign="top" width="25.509999999999998%" id="mcps1.3.3.2.4.1.1"><p id="dli_01_0475__dli_01_0451_p338173325815">Condition Key</p>
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0475__table198664321589" frame="border" border="1" rules="all"><caption><b>Table 1 </b>DLI request conditions</caption><thead align="left"><tr id="dli_01_0475__dli_01_0451_row1538733145813"><th align="left" class="cellrowborder" valign="top" width="23.59%" id="mcps1.3.3.2.5.1.1"><p id="dli_01_0475__dli_01_0451_p338173325815">Condition Key</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="25.28%" id="mcps1.3.3.2.4.1.2"><p id="dli_01_0475__dli_01_0451_p738933175812">Operator</p>
<th align="left" class="cellrowborder" valign="top" width="7.5200000000000005%" id="mcps1.3.3.2.5.1.2"><p id="dli_01_0475__dli_01_0451_p15873173953011">Type</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="49.21%" id="mcps1.3.3.2.4.1.3"><p id="dli_01_0475__dli_01_0451_p17398335586">Description</p>
<th align="left" class="cellrowborder" valign="top" width="23.380000000000003%" id="mcps1.3.3.2.5.1.3"><p id="dli_01_0475__dli_01_0451_p738933175812">Operator</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="45.51%" id="mcps1.3.3.2.5.1.4"><p id="dli_01_0475__dli_01_0451_p17398335586">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0475__dli_01_0451_row143919330583"><td class="cellrowborder" valign="top" width="25.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dli_01_0475__dli_01_0451_p19919142173917">g:CurrentTime</p>
<tbody><tr id="dli_01_0475__dli_01_0451_row143919330583"><td class="cellrowborder" valign="top" width="23.59%" headers="mcps1.3.3.2.5.1.1 "><p id="dli_01_0475__dli_01_0451_p19919142173917">g:CurrentTime</p>
</td>
<td class="cellrowborder" valign="top" width="25.28%" headers="mcps1.3.3.2.4.1.2 "><p id="dli_01_0475__dli_01_0451_p155913814012">Date and time</p>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.2.5.1.2 "><p id="dli_01_0475__dli_01_0451_p1487473917301">Global</p>
</td>
<td class="cellrowborder" valign="top" width="49.21%" headers="mcps1.3.3.2.4.1.3 "><p id="dli_01_0475__dli_01_0451_p17727152964018">Time when an authentication request is received</p>
<td class="cellrowborder" valign="top" width="23.380000000000003%" headers="mcps1.3.3.2.5.1.3 "><p id="dli_01_0475__dli_01_0451_p155913814012">Date and time</p>
</td>
<td class="cellrowborder" valign="top" width="45.51%" headers="mcps1.3.3.2.5.1.4 "><p id="dli_01_0475__dli_01_0451_p17727152964018">Time when an authentication request is received</p>
<div class="note" id="dli_01_0475__dli_01_0451_note1351115310408"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0475__dli_01_0451_p1261205354015">The time is expressed in the format defined by <span class="parmname" id="dli_01_0475__dli_01_0451_parmname96946368545"><b>ISO 8601</b></span>, for example, <strong id="dli_01_0475__dli_01_0451_b186941936185411">2012-11-11T23:59:59Z</strong>.</p>
</div></div>
</td>
</tr>
<tr id="dli_01_0475__dli_01_0451_row3745155113913"><td class="cellrowborder" valign="top" width="25.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dli_01_0475__dli_01_0451_p16919152112397">g:MFAPresent</p>
<tr id="dli_01_0475__dli_01_0451_row3745155113913"><td class="cellrowborder" valign="top" width="23.59%" headers="mcps1.3.3.2.5.1.1 "><p id="dli_01_0475__dli_01_0451_p16919152112397">g:MFAPresent</p>
</td>
<td class="cellrowborder" valign="top" width="25.28%" headers="mcps1.3.3.2.4.1.2 "><p id="dli_01_0475__dli_01_0451_p056013824016">Boolean</p>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.2.5.1.2 "><p id="dli_01_0475__dli_01_0451_p387412394302">Global</p>
</td>
<td class="cellrowborder" valign="top" width="49.21%" headers="mcps1.3.3.2.4.1.3 "><p id="dli_01_0475__dli_01_0451_p672712912402">Whether multi-factor authentication is used during user login</p>
<td class="cellrowborder" valign="top" width="23.380000000000003%" headers="mcps1.3.3.2.5.1.3 "><p id="dli_01_0475__dli_01_0451_p056013824016">Boolean</p>
</td>
<td class="cellrowborder" valign="top" width="45.51%" headers="mcps1.3.3.2.5.1.4 "><p id="dli_01_0475__dli_01_0451_p672712912402">Whether multi-factor authentication is used during user login</p>
</td>
</tr>
<tr id="dli_01_0475__dli_01_0451_row14691705398"><td class="cellrowborder" valign="top" width="25.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dli_01_0475__dli_01_0451_p12919321193915">g:UserId</p>
<tr id="dli_01_0475__dli_01_0451_row14691705398"><td class="cellrowborder" valign="top" width="23.59%" headers="mcps1.3.3.2.5.1.1 "><p id="dli_01_0475__dli_01_0451_p12919321193915">g:UserId</p>
</td>
<td class="cellrowborder" valign="top" width="25.28%" headers="mcps1.3.3.2.4.1.2 "><p id="dli_01_0475__dli_01_0451_p456015854016">String</p>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.2.5.1.2 "><p id="dli_01_0475__dli_01_0451_p387473910308">Global</p>
</td>
<td class="cellrowborder" valign="top" width="49.21%" headers="mcps1.3.3.2.4.1.3 "><p id="dli_01_0475__dli_01_0451_p172782916402">ID of the current login user</p>
<td class="cellrowborder" valign="top" width="23.380000000000003%" headers="mcps1.3.3.2.5.1.3 "><p id="dli_01_0475__dli_01_0451_p456015854016">String</p>
</td>
<td class="cellrowborder" valign="top" width="45.51%" headers="mcps1.3.3.2.5.1.4 "><p id="dli_01_0475__dli_01_0451_p172782916402">ID of the current login user</p>
</td>
</tr>
<tr id="dli_01_0475__dli_01_0451_row836712155397"><td class="cellrowborder" valign="top" width="25.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dli_01_0475__dli_01_0451_p5919162113919">g:UserName</p>
<tr id="dli_01_0475__dli_01_0451_row836712155397"><td class="cellrowborder" valign="top" width="23.59%" headers="mcps1.3.3.2.5.1.1 "><p id="dli_01_0475__dli_01_0451_p5919162113919">g:UserName</p>
</td>
<td class="cellrowborder" valign="top" width="25.28%" headers="mcps1.3.3.2.4.1.2 "><p id="dli_01_0475__dli_01_0451_p3560488402">String</p>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.2.5.1.2 "><p id="dli_01_0475__dli_01_0451_p087410397303">Global</p>
</td>
<td class="cellrowborder" valign="top" width="49.21%" headers="mcps1.3.3.2.4.1.3 "><p id="dli_01_0475__dli_01_0451_p13727129144018">Current login user</p>
<td class="cellrowborder" valign="top" width="23.380000000000003%" headers="mcps1.3.3.2.5.1.3 "><p id="dli_01_0475__dli_01_0451_p3560488402">String</p>
</td>
<td class="cellrowborder" valign="top" width="45.51%" headers="mcps1.3.3.2.5.1.4 "><p id="dli_01_0475__dli_01_0451_p13727129144018">Current login user</p>
</td>
</tr>
<tr id="dli_01_0475__dli_01_0451_row15368315143920"><td class="cellrowborder" valign="top" width="25.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dli_01_0475__dli_01_0451_p991912214397">g:ProjectName</p>
<tr id="dli_01_0475__dli_01_0451_row15368315143920"><td class="cellrowborder" valign="top" width="23.59%" headers="mcps1.3.3.2.5.1.1 "><p id="dli_01_0475__dli_01_0451_p991912214397">g:ProjectName</p>
</td>
<td class="cellrowborder" valign="top" width="25.28%" headers="mcps1.3.3.2.4.1.2 "><p id="dli_01_0475__dli_01_0451_p4560178124011">String</p>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.2.5.1.2 "><p id="dli_01_0475__dli_01_0451_p138741639123019">Global</p>
</td>
<td class="cellrowborder" valign="top" width="49.21%" headers="mcps1.3.3.2.4.1.3 "><p id="dli_01_0475__dli_01_0451_p9727729194010">Project that you have logged in to</p>
<td class="cellrowborder" valign="top" width="23.380000000000003%" headers="mcps1.3.3.2.5.1.3 "><p id="dli_01_0475__dli_01_0451_p4560178124011">String</p>
</td>
<td class="cellrowborder" valign="top" width="45.51%" headers="mcps1.3.3.2.5.1.4 "><p id="dli_01_0475__dli_01_0451_p9727729194010">Project that you have logged in to</p>
</td>
</tr>
<tr id="dli_01_0475__dli_01_0451_row136916154397"><td class="cellrowborder" valign="top" width="25.509999999999998%" headers="mcps1.3.3.2.4.1.1 "><p id="dli_01_0475__dli_01_0451_p159190217393">g:DomainName</p>
<tr id="dli_01_0475__dli_01_0451_row136916154397"><td class="cellrowborder" valign="top" width="23.59%" headers="mcps1.3.3.2.5.1.1 "><p id="dli_01_0475__dli_01_0451_p159190217393">g:DomainName</p>
</td>
<td class="cellrowborder" valign="top" width="25.28%" headers="mcps1.3.3.2.4.1.2 "><p id="dli_01_0475__dli_01_0451_p15560889409">String</p>
<td class="cellrowborder" valign="top" width="7.5200000000000005%" headers="mcps1.3.3.2.5.1.2 "><p id="dli_01_0475__dli_01_0451_p787412395302">Global</p>
</td>
<td class="cellrowborder" valign="top" width="49.21%" headers="mcps1.3.3.2.4.1.3 "><p id="dli_01_0475__dli_01_0451_p07271829124010">Domain that you have logged in to</p>
<td class="cellrowborder" valign="top" width="23.380000000000003%" headers="mcps1.3.3.2.5.1.3 "><p id="dli_01_0475__dli_01_0451_p15560889409">String</p>
</td>
<td class="cellrowborder" valign="top" width="45.51%" headers="mcps1.3.3.2.5.1.4 "><p id="dli_01_0475__dli_01_0451_p07271829124010">Domain that you have logged in to</p>
</td>
</tr>
</tbody>

View File

@ -39,8 +39,8 @@
<div class="section" id="dli_01_0479__section16691939193512"><h4 class="sectiontitle">Flink Job Permissions</h4><ul id="dli_01_0479__ul576454263610"><li id="dli_01_0479__li776474216361"><strong id="dli_01_0479__b182961149155015">View Job Details</strong><ul id="dli_01_0479__ul655191712377"><li id="dli_01_0479__li99239145374">Tenants and the admin user can view and operate all jobs.</li><li id="dli_01_0479__li119431414193719">Subusers and users with the read-only permission can only view their own jobs.<div class="note" id="dli_01_0479__note531183553419"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0479__p13263573410">If another user grants any permission other than the job viewing permission to a subuser, the job is displayed in the job list, but the details cannot be viewed by the subuser.</p>
</div></div>
</li></ul>
</li><li id="dli_01_0479__li814019268371"><strong id="dli_01_0479__b198247566537">Start Job</strong><ul id="dli_01_0479__ul752716568373"><li id="dli_01_0479__li75271356103713">To use an exclusive queue, you must have the permission to submit and start jobs.</li><li id="dli_01_0479__li175271356203711">To use a shared queue, you only need to have the permission to start jobs.</li></ul>
</li><li id="dli_01_0479__li5290184073819"><a name="dli_01_0479__li5290184073819"></a><a name="li5290184073819"></a><strong id="dli_01_0479__b198491813547">Stop Job</strong><ul id="dli_01_0479__ul18152141015381"><li id="dli_01_0479__li415241020381">To use an exclusive queue, you must have the permission to stop jobs and queues.</li><li id="dli_01_0479__li41523105385">To use a shared queue, you only need to have the permission to stop jobs.</li></ul>
</li><li id="dli_01_0479__li814019268371"><strong id="dli_01_0479__b198247566537">Start Job</strong><ul id="dli_01_0479__ul752716568373"><li id="dli_01_0479__li75271356103713">To use a dedicated queue, you must have the permission to submit and start jobs.</li><li id="dli_01_0479__li175271356203711">To use a shared queue, you only need to have the permission to start jobs.</li></ul>
</li><li id="dli_01_0479__li5290184073819"><a name="dli_01_0479__li5290184073819"></a><a name="li5290184073819"></a><strong id="dli_01_0479__b198491813547">Stop Job</strong><ul id="dli_01_0479__ul18152141015381"><li id="dli_01_0479__li415241020381">To use a dedicated queue, you must have the permission to stop jobs and queues.</li><li id="dli_01_0479__li41523105385">To use a shared queue, you only need to have the permission to stop jobs.</li></ul>
</li><li id="dli_01_0479__li03585612391"><strong id="dli_01_0479__b117601129195520">Delete Job</strong><ul id="dli_01_0479__ul10264185221011"><li id="dli_01_0479__li11320175017109">If a job can be deleted, you can delete the job if you were granted this permission.</li><li id="dli_01_0479__li9321850161014">If a job cannot be deleted, the system stops the job before you delete it. For details about how to stop a job, see <a href="#dli_01_0479__li5290184073819">Stop Job</a>. In addition, you must have the permission to delete the job.</li></ul>
</li><li id="dli_01_0479__li1444524074015"><strong id="dli_01_0479__b162251453191412">Create Job</strong><ul id="dli_01_0479__ul4827713144117"><li id="dli_01_0479__li11776151117416">By default, sub-users cannot create jobs.</li><li id="dli_01_0479__li1477611112411">To create a job, you must have this permission. Currently, only the admin user has the permission to create jobs. In addition, the user must have the permission of the related package group or package used by the job.</li></ul>
</li><li id="dli_01_0479__li176046219413"><strong id="dli_01_0479__b15367161915169">Modify Job</strong><p id="dli_01_0479__p134784383365">When modifying a job, you need to have the permission to update the job and the permission to the package group or package used by the job belongs.</p>

View File

@ -6,70 +6,72 @@
</div>
<div class="section" id="dli_01_0488__section66721076259"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0488__ul195466441251"><li id="dli_01_0488__li129878419390">Periodic scaling can be performed for a newly created queue only when there were jobs running in this queue.</li><li id="dli_01_0488__li89401050122211">Scheduled scaling tasks are available only for a queue with more than 64 CUs. That is, the minimum specifications of a queue are 64 CUs.</li><li id="dli_01_0488__li1854764410251">A maximum of 12 scheduled tasks can be created for each queue.</li><li id="dli_01_0488__li11546102612918">When each scheduled task starts, the actual start time of the specification change has a deviation of 5 minutes. It is recommended that the task start time be at least 20 minutes earlier than the time when the queue is actually used.</li><li id="dli_01_0488__li15547144132520">The interval between two scheduled tasks must be at least 2 hours.</li><li id="dli_01_0488__li750511409474">Changing the specifications of a queue is time-consuming. The time required for changing the specifications depends on the difference between the target specifications and the current specifications. You can view the specifications of the current queue on the <span class="wintitle" id="dli_01_0488__wintitle098717147548"><b>Queue Management</b></span> page.</li><li id="dli_01_0488__li554715448252">If a job is running in the current queue, the queue may fail to be scaled in to the target CU amount value. Instead, it will be scaled in to a value between the current queue specifications and the target specifications. The system will try to scale in again 1 hour later until the next scheduled task starts.</li><li id="dli_01_0488__li1054711448256">If a scheduled task does not scale out or scale in to the target CU amount value, the system triggers the scaling plan again 15 minutes later until the next scheduled task starts.</li></ul>
</div>
<div class="section" id="dli_01_0488__section05718426377"><h4 class="sectiontitle">Creating Periodic Task</h4><ul id="dli_01_0488__ul761624161310"><li id="dli_01_0488__li12618244134">If only scale-out or scale-in is required, you need to create only one task for changing specifications. Set the <span class="parmname" id="dli_01_0488__parmname1955243416549"><b>Task Name</b></span>, <span class="parmname" id="dli_01_0488__parmname1555223465417"><b>Final CU Count</b></span>, and <span class="parmname" id="dli_01_0488__parmname255333413548"><b>Executed</b></span> parameters. For details, see <a href="#dli_01_0488__table15710625151416">Table 1</a>.</li><li id="dli_01_0488__li196113244133">To set both scale-out and scale-in parameters, you need to create two periodic tasks, and set the <span class="parmname" id="dli_01_0488__parmname5651736165416"><b>Task Name</b></span>, <span class="parmname" id="dli_01_0488__parmname106511336155410"><b>Final CU Count</b></span>, and <span class="parmname" id="dli_01_0488__parmname165293610549"><b>Executed</b></span> parameters. For details, see <a href="#dli_01_0488__table15710625151416">Table 1</a>.</li></ul>
<div class="section" id="dli_01_0488__section05718426377"><h4 class="sectiontitle">Creating Periodic Task</h4><ul id="dli_01_0488__ul761624161310"><li id="dli_01_0488__li12618244134">If only scale-out or scale-in is required, you need to create only one task for changing specifications. Set the <span class="parmname" id="dli_01_0488__parmname1955243416549"><b>Task Name</b></span>, <span class="parmname" id="dli_01_0488__parmname1555223465417"><b>Final CU Count</b></span>, and <span class="parmname" id="dli_01_0488__parmname255333413548"><b>Executed</b></span> parameters. For details, see <a href="#dli_01_0488__table828194717819">Table 1</a>.</li><li id="dli_01_0488__li196113244133">To set both scale-out and scale-in parameters, you need to create two periodic tasks, and set the <span class="parmname" id="dli_01_0488__parmname5651736165416"><b>Task Name</b></span>, <span class="parmname" id="dli_01_0488__parmname106511336155410"><b>Final CU Count</b></span>, and <span class="parmname" id="dli_01_0488__parmname165293610549"><b>Executed</b></span> parameters. For details, see <a href="#dli_01_0488__table828194717819">Table 1</a>.</li></ul>
<p id="dli_01_0488__p2089341118294">The procedure is as follows:</p>
<ol id="dli_01_0488__ol2893111112910"><li id="dli_01_0488__li15893151113292">On the left of the DLI management console, click <strong id="dli_01_0488__b171171545104411">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0488__uicontrol511854518447"><b>Queue Management</b></span>.</li><li id="dli_01_0488__li289311117290">Locate the queue for which you want to schedule a periodic task for elastic scaling, and choose <strong id="dli_01_0488__b59241229112812">More</strong> &gt; <strong id="dli_01_0488__b13868174142811">Schedule CU Changes</strong> in the <strong id="dli_01_0488__b119534416289">Operation</strong> column.</li><li id="dli_01_0488__li980594010152">On the displayed page, click <span class="uicontrol" id="dli_01_0488__uicontrol1270356132810"><b>Create Periodic Task</b></span> in the upper right corner.</li><li id="dli_01_0488__li208949116292">On the <span class="wintitle" id="dli_01_0488__wintitle108671649105420"><b>Create Periodic Task</b></span> page, set the required parameters. Click <strong id="dli_01_0488__b0941135055413">OK</strong>.
<div class="tablenoborder"><a name="dli_01_0488__table15710625151416"></a><a name="table15710625151416"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0488__table15710625151416" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0488__row1771212551410"><th align="left" class="cellrowborder" valign="top" width="11.93%" id="mcps1.3.3.4.4.3.2.3.1.1"><p id="dli_01_0488__p4713132520147">Parameter</p>
<div class="tablenoborder"><a name="dli_01_0488__table828194717819"></a><a name="table828194717819"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0488__table828194717819" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="dli_01_0488__row122792047880"><th align="left" class="cellrowborder" valign="top" width="11.93%" id="mcps1.3.3.4.4.3.2.3.1.1"><p id="dli_01_0488__p162795471184">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="88.07000000000001%" id="mcps1.3.3.4.4.3.2.3.1.2"><p id="dli_01_0488__p1971410256146">Description</p>
<th align="left" class="cellrowborder" valign="top" width="88.07000000000001%" id="mcps1.3.3.4.4.3.2.3.1.2"><p id="dli_01_0488__p152796471288">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0488__row207191525121418"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p5721152541411">Task Name</p>
<tbody><tr id="dli_01_0488__row527912475814"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p02797474810">Task Name</p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p77232254145">Enter the name of the periodic task.</p>
<ul id="dli_01_0488__ul6086155113624"><li id="dli_01_0488__li18128679113624">The task name can contain only digits, letters, and underscores (_), but cannot contain only digits or start with an underscore (_) or be left unspecified.</li><li id="dli_01_0488__li4272383113630">The name can contain a maximum of 128 characters.</li></ul>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p1227924713818">Enter the name of the periodic task.</p>
<ul id="dli_01_0488__ul527994719816"><li id="dli_01_0488__li927924720812">The task name can contain only digits, letters, and underscores (_), but cannot contain only digits or start with an underscore (_) or be left unspecified.</li><li id="dli_01_0488__li32791471985">The name can contain a maximum of 128 characters.</li></ul>
</td>
</tr>
<tr id="dli_01_0488__row1273232517141"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p1873352541412">Enable Task</p>
<tr id="dli_01_0488__row1827974717817"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p1327974718820">Enable Task</p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p121195661116">Whether to enable periodic elastic scaling. The task is enabled by default. If disabled, the task will not be triggered on time.</p>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p1727915475811">Whether to enable periodic elastic scaling. The task is enabled by default. If disabled, the task will not be triggered on time.</p>
</td>
</tr>
<tr id="dli_01_0488__row448155491219"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p949145419126">Validity Period</p>
<tr id="dli_01_0488__row8280154720817"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p8280144715812">Validity Period</p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p64995471212">Set the time segment for executing the periodic task, including <strong id="dli_01_0488__b208488166559">Date</strong> and <strong id="dli_01_0488__b14849111612551">Time</strong>. If there is no time segment restriction, leave this parameter empty, indicating that the task takes effect permanently. If you need to specify the time segment for the task to take effect, set this parameter based on the service requirements.</p>
<div class="note" id="dli_01_0488__note3736244181711"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0488__ul821572214316"><li id="dli_01_0488__li1215192253115">The start time of the <span class="parmname" id="dli_01_0488__parmname11183520155513"><b>Validity Period</b></span> must be later than the current system time.</li><li id="dli_01_0488__li127241629103116">If only scale-out is configured, the system does not automatically scale in after the <strong id="dli_01_0488__b9955928115516">Validity Period</strong> expires. You need to manually modify or configure a periodic scale-in task. Similarly, if only scale-in is configured, you need to manually modify or configure a periodic scale-out task. That is, a scheduled scaling task is executed at a time.</li><li id="dli_01_0488__li1055533317383">If both scale-out and scale-in are configured, the system scales in or out resources based on the configured queue specifications within the validity period. After the validity period expires, the system retains the last configured queue specifications.</li></ul>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p192801047681">Time segment for executing the periodic task. The options include <strong id="dli_01_0488__b1291514141294">Date</strong> and <strong id="dli_01_0488__b091513141494">Time</strong>. If there is no time segment restriction, leave this parameter empty, indicating that the task takes effect permanently. If you need to specify the time segment for the task to take effect, set this parameter based on the service requirements.</p>
<div class="note" id="dli_01_0488__note92800473818"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0488__ul1928014715815"><li id="dli_01_0488__li528011476810">The start time of the <span class="parmname" id="dli_01_0488__parmname11183520155513"><b>Validity Period</b></span> must be later than the current system time.</li><li id="dli_01_0488__li62809471684">If only scale-out is configured, the system does not automatically scale in after the <strong id="dli_01_0488__b9955928115516">Validity Period</strong> expires. You need to manually modify or configure a periodic scale-in task. Similarly, if only scale-in is configured, you need to manually modify or configure a periodic scale-out task. That is, a scheduled scaling task is executed at a time.</li><li id="dli_01_0488__li102805472820">If both scale-out and scale-in are configured, the system scales in or out resources based on the configured queue specifications within the validity period. After the validity period expires, the system retains the last configured queue specifications.</li></ul>
</div></div>
</td>
</tr>
<tr id="dli_01_0488__row104915420127"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p145015412129">Actual CUs</p>
<tr id="dli_01_0488__row1128044714810"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p2028014715813">Actual CUs</p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p1450854151217">Queue specifications before scale-in or scale-out.</p>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p12280174712811">Queue specifications before scale-in or scale-out.</p>
</td>
</tr>
<tr id="dli_01_0488__row196301113132119"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p1563091362113">Final CUs</p>
<tr id="dli_01_0488__row928017472087"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p92801447389">Final CUs</p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p1663001313215">Specifications after the queue is scaled in or out.</p>
<div class="note" id="dli_01_0488__note95291031112415"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0488__ul035715239466"><li id="dli_01_0488__li684402510463">By default, the maximum specifications of a queue are <strong id="dli_01_0488__b41630410554">512 CUs</strong>.</li><li id="dli_01_0488__li83571234461">The minimum queue specifications for scheduled scaling are 64 CUs. That is, only when <strong id="dli_01_0488__b122503469916">Actual CUs</strong> are more than 64 CUs, the scheduled scaling can be performed.</li><li id="dli_01_0488__li1065012416595">The value of <strong id="dli_01_0488__b151818411219">Actual CUs</strong> must be a multiple of 16.</li></ul>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p7280134716814">Specifications after the queue is scaled in or out.</p>
<div class="note" id="dli_01_0488__note1928014471385"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0488__ul42804471482"><li id="dli_01_0488__li12802471580">By default, the maximum specifications of a queue are <strong id="dli_01_0488__b41630410554">512 CUs</strong>.</li><li id="dli_01_0488__li12802478813">The minimum queue specifications for scheduled scaling are 64 CUs. That is, only when <strong id="dli_01_0488__b122503469916">Actual CUs</strong> are more than 64 CUs, the scheduled scaling can be performed.</li><li id="dli_01_0488__li42801471589">The value of <strong id="dli_01_0488__b151818411219">Actual CUs</strong> must be a multiple of 16.</li></ul>
</div></div>
</td>
</tr>
<tr id="dli_01_0488__row3630121319213"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p14631161311219">Repeat</p>
<tr id="dli_01_0488__row72811471985"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p1228014474811">Repeat</p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><div class="p" id="dli_01_0488__p963171317218">Time when scheduled scale-out or scale-in is repeat. Scheduled tasks can be scheduled by week in <span class="parmname" id="dli_01_0488__parmname1043784552615"><b>Repeat</b></span>.<ul id="dli_01_0488__ul8668175312013"><li id="dli_01_0488__li196687532010">By default, this parameter is not configured, indicating that the task is executed only once at the time specified by <strong id="dli_01_0488__b1427182611549">Executed</strong>.</li><li id="dli_01_0488__li16668453605">If you select all, the plan is executed every day.</li><li id="dli_01_0488__li18668155319014">If you select some options of <strong id="dli_01_0488__b941215556264">Repeat</strong>, the plan is executed once a week at all specified days.</li></ul>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><div class="p" id="dli_01_0488__p1128011471984">Time when scheduled scale-out or scale-in is repeat. Scheduled tasks can be scheduled by week in <span class="parmname" id="dli_01_0488__parmname1043784552615"><b>Repeat</b></span>.<ul id="dli_01_0488__ul628054710812"><li id="dli_01_0488__li3280204719814">By default, this parameter is not configured, indicating that the task is executed only once at the time specified by <strong id="dli_01_0488__b1427182611549">Executed</strong>.</li><li id="dli_01_0488__li1828012475811">If you select all, the plan is executed every day.</li><li id="dli_01_0488__li0280164714820">If you select some options of <strong id="dli_01_0488__b941215556264">Repeat</strong>, the plan is executed once a week at all specified days.</li></ul>
</div>
<div class="note" id="dli_01_0488__note192566252713"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0488__ul10455141717461"><li id="dli_01_0488__li184553170468">You do not need to set this parameter if you only need to perform scale-in or scale-out once.</li><li id="dli_01_0488__li345514173464">If you have set scaling, you can set <span class="parmname" id="dli_01_0488__parmname9347251115510"><b>Repeat</b></span> as required. You can also set the repeat period together with the validity period.</li></ul>
<div class="note" id="dli_01_0488__note1028119472816"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="dli_01_0488__ul72818476810"><li id="dli_01_0488__li15281124718817">You do not need to set this parameter if you only need to perform scale-in or scale-out once.</li><li id="dli_01_0488__li82810471813">If you have set scaling, you can set <span class="parmname" id="dli_01_0488__parmname9347251115510"><b>Repeat</b></span> as required. You can also set the repeat period together with the validity period.</li></ul>
</div></div>
</td>
</tr>
<tr id="dli_01_0488__row663115138214"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p563111312218"><strong id="dli_01_0488__b14285952105514">Executed</strong></p>
<tr id="dli_01_0488__row1281947981"><td class="cellrowborder" valign="top" width="11.93%" headers="mcps1.3.3.4.4.3.2.3.1.1 "><p id="dli_01_0488__p11281204716813"><strong id="dli_01_0488__b14285952105514">Executed</strong></p>
</td>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p1386845834214">Time when scheduled scale-out or scale-in is performed</p>
<ul id="dli_01_0488__ul763810437"><li id="dli_01_0488__li16304179930">When each scheduled task starts, the actual start time of the specification change has a deviation of 5 minutes. It is recommended that the task start time be at least 20 minutes earlier than the time when the queue is actually used.</li><li id="dli_01_0488__li163051591131">The interval between two scheduled tasks must be at least 2 hours.</li></ul>
<td class="cellrowborder" valign="top" width="88.07000000000001%" headers="mcps1.3.3.4.4.3.2.3.1.2 "><p id="dli_01_0488__p4281147380">Time when scheduled scale-out or scale-in is performed</p>
<ul id="dli_01_0488__ul728164716816"><li id="dli_01_0488__li1928116471183">When each scheduled task starts, the actual start time of the specification change has a deviation of 5 minutes. It is recommended that the task start time be at least 20 minutes earlier than the time when the queue is actually used.</li><li id="dli_01_0488__li328111479816">The interval between two scheduled tasks must be at least 2 hours.</li></ul>
</td>
</tr>
</tbody>
</table>
</div>
<p id="dli_01_0488__p19798124017010">After a periodic task is created, you can view the specification change of the current queue and the latest execution time on the page for scheduling CU changes.</p>
<p id="dli_01_0488__p15557854505">Alternatively, on the <strong id="dli_01_0488__b11882134982418">Queue Management</strong> page, check whether the <span class="parmname" id="dli_01_0488__parmname1888394913243"><b>Specifications</b></span> change to determine whether the scaling is successful.</p>
<p id="dli_01_0488__p2423181675918">You can also go to the <strong id="dli_01_0488__b33161459145517">Job Management</strong> page and check the status of the <strong id="dli_01_0488__b163177595555">SCALE_QUEUE</strong> job. If the job status is <strong id="dli_01_0488__b9317155919556">Scaling</strong>, the queue is being scaled in or out.</p>
<p id="dli_01_0488__p32812470820">After a periodic task is created, you can view the specification change of the current queue and the latest execution time on the page for scheduling CU changes.</p>
<p id="dli_01_0488__p528110471985">Alternatively, on the <strong id="dli_01_0488__b11882134982418">Queue Management</strong> page, check whether the <span class="parmname" id="dli_01_0488__parmname1888394913243"><b>Specifications</b></span> change to determine whether the scaling is successful.</p>
<p id="dli_01_0488__p1828119477813">You can also go to the <strong id="dli_01_0488__b33161459145517">Job Management</strong> page and check the status of the <strong id="dli_01_0488__b163177595555">SCALE_QUEUE</strong> job. If the job status is <strong id="dli_01_0488__b9317155919556">Scaling</strong>, the queue is being scaled in or out.</p>
</li></ol>
</div>
<div class="section" id="dli_01_0488__section9221637416"><h4 class="sectiontitle">Modifying a Scheduled Task</h4><p id="dli_01_0488__p429311109214">If the configured scheduled task does not meet service requirements, you can click <strong id="dli_01_0488__b135631275618">Modify</strong> in the <span class="wintitle" id="dli_01_0488__wintitle155633235618"><b>Operation</b></span> column of the task on the <strong id="dli_01_0488__b1355205212271">Schedule CU Changes</strong> page to modify the task configuration.</p>
<div class="section" id="dli_01_0488__section9221637416"><h4 class="sectiontitle">Modifying a Scheduled Task</h4><p id="dli_01_0488__p429311109214">If a periodic task cannot meet service requirements anymore, you can modify it on the <span class="wintitle" id="dli_01_0488__wintitle1246021782012"><b>Schedule CU Changes</b></span> page.</p>
<ol id="dli_01_0488__ol357711139810"><li id="dli_01_0488__li4577141311814">In the navigation pane of the DLI management console, choose <strong id="dli_01_0488__b527634671015">Resources</strong> &gt;<span class="uicontrol" id="dli_01_0488__uicontrol11276144681013"><b>Queue Management</b></span>.</li><li id="dli_01_0488__li105776135817">Locate the queue for which you want to schedule a periodic task for elastic scaling, and choose <strong id="dli_01_0488__b1180360131119">More</strong> &gt; <strong id="dli_01_0488__b780313011119">Schedule CU Changes</strong> in the <strong id="dli_01_0488__b28041309112">Operation</strong> column.</li><li id="dli_01_0488__li1557891318811">On the displayed page, click <strong id="dli_01_0488__b1919812167173">Modify</strong> in the <strong id="dli_01_0488__b65081818121719">Operation</strong> column. In the displayed dialog box, modify the task parameters as needed.</li></ol>
</div>
<div class="section" id="dli_01_0488__section289714226"><h4 class="sectiontitle">Deleting a Scheduled Task</h4><p id="dli_01_0488__p12901834122213">If you do not need a scheduled task for queue scaling, click <strong id="dli_01_0488__b2080705795419">Delete</strong> in the <span class="wintitle" id="dli_01_0488__wintitle38131957205420"><b>Operation</b></span> column of the task on the <span class="wintitle" id="dli_01_0488__wintitle98131257165413"><b>Schedule CU Changes</b></span> page. In the displayed <span class="wintitle" id="dli_01_0488__wintitle881318574546"><b>Delete Periodic Task</b></span> dialog box, click <strong id="dli_01_0488__b98142579548">Yes</strong>.</p>
<div class="section" id="dli_01_0488__section289714226"><h4 class="sectiontitle">Deleting a Scheduled Task</h4><p id="dli_01_0488__p105855371214">If you do not need the task anymore, delete the task on the <strong id="dli_01_0488__b186029341916">Schedule CU Changes</strong> page.</p>
<ol id="dli_01_0488__ol1531810254121"><li id="dli_01_0488__li1576235581216">In the navigation pane of the DLI management console, choose <strong id="dli_01_0488__b182618195198">Resources</strong> &gt;<span class="uicontrol" id="dli_01_0488__uicontrol14261819101915"><b>Queue Management</b></span>.</li><li id="dli_01_0488__li2318202591215">Locate the queue for which you want to schedule a periodic task for elastic scaling, and choose <strong id="dli_01_0488__b16336132813199">More</strong> &gt; <strong id="dli_01_0488__b14336192812198">Schedule CU Changes</strong> in the <strong id="dli_01_0488__b533615289198">Operation</strong> column.</li><li id="dli_01_0488__li1131817257124">On the displayed page, click <strong id="dli_01_0488__b893216373190">Delete</strong> in the <strong id="dli_01_0488__b11932103731914">Operation</strong> column. In the displayed dialog box, click <strong id="dli_01_0488__b1988526132013">OK</strong>.</li></ol>
</div>
</div>
<div>

File diff suppressed because it is too large Load Diff

View File

@ -19,7 +19,7 @@
</div>
<div class="section" id="dli_01_05111__section123755912574"><h4 class="sectiontitle">Precautions</h4><p id="dli_01_05111__p8691415573">When a sub-account uses the TPC-H test suite, the main account needs to grant the sub-account the OBS access permission and the permission to view the main account table. If the master account has not logged in to DLI, the sub-account needs to have the permissions to create databases and tables in addition to the preceding permissions.</p>
</div>
<div class="section" id="dli_01_05111__section1814532318216"><h4 class="sectiontitle">Operation Description</h4><p id="dli_01_05111__p75704242217">For details, see <a href="dli_01_0021.html">SQL Template Management</a>.</p>
<div class="section" id="dli_01_05111__section1814532318216"><h4 class="sectiontitle">Operation Description</h4><p id="dli_01_05111__p75704242217">For details, see <a href="dli_01_0021.html">Managing SQL Templates</a>.</p>
</div>
</div>
<div>

View File

@ -3,22 +3,19 @@
<h1 class="topictitle1">Developing and Submitting a Spark SQL Job Using the TPC-H Sample Template</h1>
<div id="body0000001220042371"><p id="dli_01_0512__p3438152482217">To facilitate SQL operations, DLI allows you to customize query templates or save the SQL statements in use as templates. After templates are saved, you do not need to write SQL statements. You can directly perform the SQL operations using the templates.</p>
<p id="dli_01_0512__p27712208201">The current system provides various standard TPC-H query statement templates. You can select a template as needed. This example shows how to use a TPC-H template to develop and submit a Spark SQL job.</p>
<p id="dli_01_0512__p9606134314412"><a href="#dli_01_0512__section3751181910618">Step 1: Logging in to the Cloud</a></p>
<p id="dli_01_0512__p292291825719"><a href="#dli_01_0512__section19012773105034">Step 2: Logging In to the DLI Management Console</a></p>
<p id="dli_01_0512__p11591724205712"><a href="#dli_01_0512__section21590507141153">Step 3: Execute the TPC-H Sample Template and View the Result</a></p>
<p id="dli_01_0512__p292291825719"><a href="#dli_01_0512__section19012773105034">Step 1: Log In to the Management Console</a></p>
<p id="dli_01_0512__p11591724205712"><a href="#dli_01_0512__section21590507141153">Step 2: Execute the TPC-H Sample Template and View the Result</a></p>
<p id="dli_01_0512__p168551319192615">For details about the templates, see SQL Template Management.</p>
<div class="section" id="dli_01_0512__section3751181910618"><a name="dli_01_0512__section3751181910618"></a><a name="section3751181910618"></a><h4 class="sectiontitle">Step 1: Logging in to the Cloud</h4><ol id="dli_01_0512__ol77610191561"><li id="dli_01_0512__li1968871183717">Open the DLI homepage.</li><li id="dli_01_0512__li1597319241563">On the login page, enter the username and password, and click <strong id="dli_01_0512__b12664025104912">Log In</strong>.</li></ol>
</div>
<div class="section" id="dli_01_0512__section19012773105034"><a name="dli_01_0512__section19012773105034"></a><a name="section19012773105034"></a><h4 class="sectiontitle">Step 2: Logging In to the DLI Management Console</h4><ol id="dli_01_0512__ol33542655111116"><li id="dli_01_0512__li2096805415940">.</li><li id="dli_01_0512__li38964926151831">The DLI management console page is displayed.<div class="note" id="dli_01_0512__note231512513316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0512__p231925193114">You need to perform authorization when accessing the DLI management console for the first time. For details, see Service Authorization.</p>
<div class="section" id="dli_01_0512__section19012773105034"><a name="dli_01_0512__section19012773105034"></a><a name="section19012773105034"></a><h4 class="sectiontitle">Step 1: Log In to the Management Console</h4><ol id="dli_01_0512__ol33542655111116"><li id="dli_01_0512__li26251138164819">Go to the DLI console.</li><li id="dli_01_0512__li285072071584">From the menu on top of the homepage, move your cursor on <span class="uicontrol" id="dli_01_0512__uicontrol7290531195714"><b>Products</b></span>.</li><li id="dli_01_0512__li2096805415940">.</li><li id="dli_01_0512__li38964926151831">The DLI management console page is displayed.<div class="note" id="dli_01_0512__note231512513316"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0512__p231925193114">You need to perform authorization when accessing the DLI management console for the first time. For details, see Service Authorization.</p>
</div></div>
</li></ol>
</div>
<div class="section" id="dli_01_0512__section21590507141153"><a name="dli_01_0512__section21590507141153"></a><a name="section21590507141153"></a><h4 class="sectiontitle">Step 3: Execute the TPC-H Sample Template and View the Result</h4><ol id="dli_01_0512__ol39318463153849"><li id="dli_01_0512__li237731311316">On the DLI management console, choose <strong id="dli_01_0512__b122621281817">Job Templates</strong> &gt; <strong id="dli_01_0512__b12449158183">SQL Templates</strong>, and click the <strong id="dli_01_0512__b25811917191816">Sample Templates</strong> tab. Locate the <strong id="dli_01_0512__b11586420111811">Q1_Price_summary_report_query</strong> template under <strong id="dli_01_0512__b1755062791819">tpchQuery</strong>, and click <strong id="dli_01_0512__b3909152961818">Execute</strong> in the <strong id="dli_01_0512__b8156932131816">Operation</strong> column. The <strong id="dli_01_0512__b198031535131810">SQL Editor</strong> page is displayed.<p id="dli_01_0512__p1290842593220"></p>
<div class="section" id="dli_01_0512__section21590507141153"><a name="dli_01_0512__section21590507141153"></a><a name="section21590507141153"></a><h4 class="sectiontitle">Step 2: Execute the TPC-H Sample Template and View the Result</h4><ol id="dli_01_0512__ol39318463153849"><li id="dli_01_0512__li237731311316">On the DLI management console, choose <strong id="dli_01_0512__b122621281817">Job Templates</strong> &gt; <strong id="dli_01_0512__b12449158183">SQL Templates</strong>, and click the <strong id="dli_01_0512__b25811917191816">Sample Templates</strong> tab. Locate the <strong id="dli_01_0512__b11586420111811">Q1_Price_summary_report_query</strong> template under <strong id="dli_01_0512__b1755062791819">tpchQuery</strong>, and click <strong id="dli_01_0512__b3909152961818">Execute</strong> in the <strong id="dli_01_0512__b8156932131816">Operation</strong> column. The <strong id="dli_01_0512__b198031535131810">SQL Editor</strong> page is displayed.<p id="dli_01_0512__p1290842593220"></p>
</li><li id="dli_01_0512__li23198164153933">In the upper part of the editing window, set <strong id="dli_01_0512__b18464102717327">Engine</strong> to <strong id="dli_01_0512__b922973312320">spark</strong>, <strong id="dli_01_0512__b15155355327">Queues</strong> to <strong id="dli_01_0512__b19307193712320">default</strong>, and <strong id="dli_01_0512__b1593184253213">Databases</strong> to <strong id="dli_01_0512__b1562965219321">default</strong>, and click <strong id="dli_01_0512__b139531113336">Execute</strong>.<p id="dli_01_0512__p016617263419"></p>
</li><li id="dli_01_0512__li1588111441197">View the query result in the <strong id="dli_01_0512__b8520192512337">View Result</strong> tab in the lower part of the SQL Editor page.<p id="dli_01_0512__p185671909495"></p>
</li></ol>
<p id="dli_01_0512__p82041810111510">This example uses the <strong id="dli_01_0512__b1448173217340">default</strong> queue and database preset in the system as an example. You can also run query statements on a self-created queue and database.</p>
<p id="dli_01_0512__p5151144614147">For details about how to create a queue, see Creating a Queue. For details about how to create a database, see Creating a Database.</p>
<p id="dli_01_0512__p5151144614147">For details about how to create a queue, see "Creating a Queue" in <em id="dli_01_0512__i1560915512236">Data Lake Insight User Guide</em>. For details about how to create a database, see Creating a Database.</p>
</div>
</div>
<div>

View File

@ -0,0 +1,21 @@
<a name="dli_01_0513"></a><a name="dli_01_0513"></a>
<h1 class="topictitle1">Other Common Operations</h1>
<div id="body0000001174986330"><p id="dli_01_0513__p8060118"></p>
</div>
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0420.html">Importing Data to a DLI Table</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0445.html">Viewing Monitoring Metrics</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0318.html">DLI Operations That Can Be Recorded by CTS</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0550.html">Quotas</a></strong><br>
</li>
</ul>
</div>

File diff suppressed because it is too large Load Diff

View File

@ -8,13 +8,13 @@
<div>
<ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_01_0378.html">DLI Introduction</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_01_0378.html">What Is Data Lake Insight?</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_07_0007.html">Advantages</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_07_0002.html">Application Scenarios</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_07_0005.html">Constraints and Limitations on Using DLI</a></strong><br>
<li class="ulchildlink"><strong><a href="dli_07_0005.html">Prerequisites</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_07_0006.html">Permissions Management</a></strong><br>
</li>

View File

@ -1,7 +1,7 @@
<a name="dli_01_0539"></a><a name="dli_01_0539"></a>
<h1 class="topictitle1">FAQs</h1>
<h1 class="topictitle1">FAQ</h1>
<div id="body0000001462093757"><p id="dli_01_0539__p8060118"></p>
</div>
@ -24,6 +24,10 @@
</li>
<li class="ulchildlink"><strong><a href="dli_03_0022.html">Datasource Connections</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_03_0056.html">APIs</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_03_0058.html">SDKs</a></strong><br>
</li>
</ul>
</div>

View File

@ -0,0 +1,24 @@
<a name="dli_01_0550"></a><a name="dli_01_0550"></a>
<h1 class="topictitle1">Quotas</h1>
<div id="body8662426"><div class="section" id="dli_01_0550__en-us_topic_0206791776_section15918030161717"><h4 class="sectiontitle">What Is a Quota?</h4><p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p1415804616467">A quota limits the quantity of a resource available to users, thereby preventing spikes in the usage of the resource.</p>
<p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p547171214515">You can also request for an increased quota if your existing quota cannot meet your service requirements.</p>
</div>
<div class="section" id="dli_01_0550__en-us_topic_0206791776_section1855113150458"><h4 class="sectiontitle">How Do I View My Quotas?</h4><ol id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_ol1222775692215"><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li5648642910048">Log in to the management console.</li><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li464234814566">Click <span><img id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_en-us_topic_0014250631_en-us_topic_0014250631_image1733511156361" src="en-us_image_0000001373007370.png"></span> in the upper left corner and select <strong id="dli_01_0550__en-us_topic_0206791776_b7257115092114">Region</strong> and <strong id="dli_01_0550__en-us_topic_0206791776_b3258350102110">Project</strong>.</li><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li3861581410048">Click <span><img id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_image99377514309" src="en-us_image_0000001372847466.png"></span> (the <strong id="dli_01_0550__en-us_topic_0206791776_b143149912215">My Quotas</strong> icon) in the upper right corner.<p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p1748884433119">The <strong id="dli_01_0550__en-us_topic_0206791776_b6138117162210">Service Quota</strong> page is displayed.</p>
</li><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li1579070183211">View the used and total quota of each type of resources on the displayed page.<p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p97891720193417"><a name="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li1579070183211"></a><a name="en-us_topic_0206791776_en-us_topic_0040259342_li1579070183211"></a>If a quota cannot meet service requirements, increase a quota.</p>
</li></ol>
</div>
<div class="section" id="dli_01_0550__en-us_topic_0206791776_section44761618172016"><h4 class="sectiontitle">How Do I Apply for a Higher Quota?</h4><p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p133802054172617">The system does not support online quota adjustment. To increase a resource quota, dial the hotline or send an email to the customer service. We will process your application and inform you of the progress by phone call or email.</p>
<p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p13745144852611">Before dialing the hotline number or sending an email, ensure that the following information has been obtained:</p>
<ul id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_ul37351029162712"><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li12735132932714">Domain name, project name, and project ID<p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p1973542918275"><a name="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li12735132932714"></a><a name="en-us_topic_0206791776_en-us_topic_0040259342_li12735132932714"></a>Log in to the management console using the cloud account, click the username in the upper right corner, select <strong id="dli_01_0550__en-us_topic_0206791776_b18872194484610">My Credentials</strong> from the drop-down list, and obtain the domain name, project name, and project ID on the <strong id="dli_01_0550__en-us_topic_0206791776_b687374413466">My Credentials</strong> page.</p>
</li><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li13735182917270">Quota information, including:<ul id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_ul6735102912273"><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li1073516296277">Service name</li><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li6735152972712">Quota type</li><li id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_li773592992718">Required quota</li></ul>
</li></ul>
<p id="dli_01_0550__en-us_topic_0206791776_en-us_topic_0040259342_p126761744182712"><a href="https://docs.otc.t-systems.com/en-us/public/learnmore.html" target="_blank" rel="noopener noreferrer">Learn how to obtain the service hotline and email address.</a></p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0513.html">Other Common Operations</a></div>
</div>
</div>

Some files were not shown because too many files have changed in this diff Show More