lboka/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity

dli_umn_20240808

Browse Source 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
Co-committed-by: Su, Xiaomeng <suxiaomeng1@huawei.com>
This commit is contained in:
Su, Xiaomeng 2024-08-09 11:00:57 +00:00 committed by zuul
parent 0e5fae25ff
commit fdd43c552e
53 changed files with 1223 additions and 1744 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1388
docs/dli/umn/ALL_META.TXT.json
View File

File diff suppressed because it is too large Load Diff

527
docs/dli/umn/CLASS.TXT.json
View File

File diff suppressed because it is too large Load Diff

12
docs/dli/umn/dli_01_00006.html
View File

@ -8,7 +8,13 @@
</th> </th>
</tr> </tr>
</thead> </thead>
<tbody><tr id="dli_01_00006__row1340918386143"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p10168134117141">2024-04-28</p> <tbody><tr id="dli_01_00006__row15952172464713"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p738312864717">2024-07-11</p>
</td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p15383828204713">Modified the following section:</p>
<ul id="dli_01_00006__ul13383162834712"><li id="dli_01_00006__li18383142874716">Added the description on how to import OBS data to DLI to <a href="dli_01_0420.html">Importing Data to a DLI Table</a>.</li></ul>
</td>
</tr>
<tr id="dli_01_00006__row1340918386143"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p10168134117141">2024-04-28</p>
</td> </td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p181681241101413">Added the following section:</p> <td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p181681241101413">Added the following section:</p>
<ul id="dli_01_00006__ul416812413143"><li id="dli_01_00006__li11168134151414"><a href="dli_01_0508.html">Elastic Resource Pool</a></li></ul> <ul id="dli_01_00006__ul416812413143"><li id="dli_01_00006__li11168134151414"><a href="dli_01_0508.html">Elastic Resource Pool</a></li></ul>
@ -25,13 +31,13 @@
<tr id="dli_01_00006__row6758122510313"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p462652810312">2023-11-01</p> <tr id="dli_01_00006__row6758122510313"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p462652810312">2023-11-01</p>
</td> </td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p462614287310">Modified the following content:</p> <td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p462614287310">Modified the following content:</p>
<ul id="dli_01_00006__ul13626728339"><li id="dli_01_00006__li186261028531">Modified the link for obtaining service support during quota application in <a href="dli_07_0009.html">Quotas</a>.</li><li id="dli_01_00006__li26261928732">Added the link to <em id="dli_01_00006__i1661311132615">Data Lake Insight API Reference</em> to <a href="dli_01_0378.html">What Is Data Lake Insight?</a>.</li><li id="dli_01_00006__li14959163416518">Modified the method of obtaining host information in <a href="dli_01_0013.html">Modifying Host Information</a>.</li></ul> <ul id="dli_01_00006__ul13626728339"><li id="dli_01_00006__li186261028531">Modified the link for obtaining service support during quota application in <a href="dli_07_0009.html">Quotas</a>.</li><li id="dli_01_00006__li26261928732">Added the link to <em id="dli_01_00006__i1661311132615">Data Lake Insight API Reference</em> to <a href="dli_01_0378.html">What Is Data Lake Insight?</a>.</li><li id="dli_01_00006__li14959163416518">Modified the method of obtaining host information in <a href="dli_01_0013.html">Modifying Host Information in an Elastic Resource Pool</a>.</li></ul>
</td> </td>
</tr> </tr>
<tr id="dli_01_00006__row1569106191216"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p36566919122">2023-10-08</p> <tr id="dli_01_00006__row1569106191216"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p36566919122">2023-10-08</p>
</td> </td>
<td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p465618951211">Modified the following content:</p> <td class="cellrowborder" valign="top" width="66.33%" headers="mcps1.3.1.1.3.1.2 "><p id="dli_01_00006__p465618951211">Modified the following content:</p>
<ul id="dli_01_00006__ul82951936181419"><li id="dli_01_00006__li129511362149">Modified the example format in <a href="dli_01_0384.html#dli_01_0384__table17326112613420">Table 2</a>.</li><li id="dli_01_00006__li3295163691419">Added constraints and limitations on tag keys and tag values to <a href="dli_01_0455.html">Creating a Flink SQL Job</a> and <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</li></ul> <ul id="dli_01_00006__ul82951936181419"><li id="dli_01_00006__li129511362149">Modified the example format in <a href="dli_01_0384.html#dli_01_0384__table17326112613420">Table 2</a>.</li><li id="dli_01_00006__li3295163691419">Added notes and constraints on tag keys and tag values to <a href="dli_01_0455.html">Creating a Flink SQL Job</a> and <a href="dli_01_0457.html">Creating a Flink Jar Job</a>.</li></ul>
</td> </td>
</tr> </tr>
<tr id="dli_01_00006__row12145240104716"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p193441641970">2023-09-07</p> <tr id="dli_01_00006__row12145240104716"><td class="cellrowborder" valign="top" width="33.67%" headers="mcps1.3.1.1.3.1.1 "><p id="dli_01_00006__p193441641970">2023-09-07</p>

4
docs/dli/umn/dli_01_0003.html
View File

@ -8,9 +8,9 @@
<div class="caution" id="dli_01_0003__note132261042151315"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dli_01_0003__p1822614217130">In cross-source development scenarios, there is a risk of password leakage if datasource authentication information is directly configured. You are advised to use the datasource authentication provided by DLI. For details, see <a href="dli_01_0561.html">Overview</a>.</p> <div class="caution" id="dli_01_0003__note132261042151315"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dli_01_0003__p1822614217130">In cross-source development scenarios, there is a risk of password leakage if datasource authentication information is directly configured. You are advised to use the datasource authentication provided by DLI. For details, see <a href="dli_01_0561.html">Overview</a>.</p>
</div></div> </div></div>
</div> </div>
<div class="section" id="dli_01_0003__section12469945134416"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0003__ul920917130139"><li id="dli_01_0003__dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0003__dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0003__dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0003__dli_07_0005_li3320184019107"><strong id="dli_01_0003__dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li><li id="dli_01_0003__dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0003__dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0003__dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li><li id="dli_01_0003__dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the constraints on IP addresses are as follows:<ul id="dli_01_0003__dli_07_0005_ul1254904218358"><li id="dli_01_0003__dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0003__dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p26801956133713"><a name="dli_01_0003__dli_07_0005_li151317527373"></a><a name="dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0003__dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0003__dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0003__dli_07_0005_b37501722194120">.</strong><em id="dli_01_0003__dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0003__dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0003__dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0003__dli_07_0005_b11270124004111">.</strong><em id="dli_01_0003__dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0003__dli_07_0005_b14999194774118">:8181</strong>.</p> <div class="section" id="dli_01_0003__section12469945134416"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0003__ul920917130139"><li id="dli_01_0003__dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0003__dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0003__dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0003__dli_07_0005_li3320184019107"><strong id="dli_01_0003__dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li><li id="dli_01_0003__dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0003__dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0003__dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li><li id="dli_01_0003__dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the notes and constraints on IP addresses are:<ul id="dli_01_0003__dli_07_0005_ul1254904218358"><li id="dli_01_0003__dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0003__dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p26801956133713"><a name="dli_01_0003__dli_07_0005_li151317527373"></a><a name="dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0003__dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0003__dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0003__dli_07_0005_b37501722194120">.</strong><em id="dli_01_0003__dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0003__dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0003__dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0003__dli_07_0005_b11270124004111">.</strong><em id="dli_01_0003__dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0003__dli_07_0005_b14999194774118">:8181</strong>.</p>
</li></ul> </li></ul>
</li><li id="dli_01_0003__dli_07_0005_li1152733115719">When checking the connectivity of datasource connections, the constraints on domain names are as follows:<ul id="dli_01_0003__dli_07_0005_ul166789287574"><li id="dli_01_0003__dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0003__dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0003__dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0003__dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0003__dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0003__dli_07_0005_li1867852835715">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p1667882814575"><a name="dli_01_0003__dli_07_0005_li1867852835715"></a><a name="dli_07_0005_li1867852835715"></a>For example, <strong id="dli_01_0003__dli_07_0005_b177998236451">example.com:8080</strong>.</p> </li><li id="dli_01_0003__dli_07_0005_li1152733115719">When checking the connectivity of datasource connections, the notes and constraints on domain names are:<ul id="dli_01_0003__dli_07_0005_ul166789287574"><li id="dli_01_0003__dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0003__dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0003__dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0003__dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0003__dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0003__dli_07_0005_li1867852835715">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0003__dli_07_0005_p1667882814575"><a name="dli_01_0003__dli_07_0005_li1867852835715"></a><a name="dli_07_0005_li1867852835715"></a>For example, <strong id="dli_01_0003__dli_07_0005_b177998236451">example.com:8080</strong>.</p>
</li></ul> </li></ul>
</li></ul> </li></ul>
</div> </div>

12
docs/dli/umn/dli_01_0005.html
View File

@ -52,9 +52,9 @@
</p></li></ol> </p></li></ol>
</div> </div>
<div class="section" id="dli_01_0005__section46221651175510"><h4 class="sectiontitle">Creating a Table</h4><p id="dli_01_0005__p56362480175534">Before creating a table, ensure that a database has been created.</p> <div class="section" id="dli_01_0005__section46221651175510"><h4 class="sectiontitle">Creating a Table</h4><p id="dli_01_0005__p56362480175534">Before creating a table, ensure that a database has been created.</p>
<ol id="dli_01_0005__ol12637600103738"><li id="dli_01_0005__li59972234153426"><span>You can create a table on either the <span class="wintitle" id="dli_01_0005__wintitle1376768905"><b>Databases and Tables</b></span> page or the <span class="wintitle" id="dli_01_0005__wintitle1088857340"><b>SQL Editor</b></span> page.</span><p><div class="note" id="dli_01_0005__note159813598457"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0005__p4599175984510">Datasource connection tables, such as View tables, HBase (MRS) tables, OpenTSDB (MRS) tables, GaussDB(DWS) tables, RDS tables, and CSS tables, cannot be created. You can use SQL to create views and datasource connection tables. For details, see sections <strong id="dli_01_0005__b168941558103516">Creating a View</strong> and <strong id="dli_01_0005__b1989435873520">Creating a Datasource Connection Table</strong> in the <em id="dli_01_0005__i12894358133518">Data Lake Insight SQL Syntax Reference</em>.</p> <ol id="dli_01_0005__ol12637600103738"><li id="dli_01_0005__li59972234153426"><span>You can create a table on either the <span class="wintitle" id="dli_01_0005__wintitle1675474472"><b>Databases and Tables</b></span> page or the <span class="wintitle" id="dli_01_0005__wintitle1449176807"><b>SQL Editor</b></span> page.</span><p><div class="note" id="dli_01_0005__note159813598457"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0005__p4599175984510">Datasource connection tables, such as View tables, HBase (MRS) tables, OpenTSDB (MRS) tables, GaussDB(DWS) tables, RDS tables, and CSS tables, cannot be created. You can use SQL to create views and datasource connection tables. For details, see sections <strong id="dli_01_0005__b168941558103516">Creating a View</strong> and <strong id="dli_01_0005__b1989435873520">Creating a Datasource Connection Table</strong> in the <em id="dli_01_0005__i12894358133518">Data Lake Insight SQL Syntax Reference</em>.</p>
</div></div> </div></div>
<ul id="dli_01_0005__ul22405571153537"><li id="dli_01_0005__li323555153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle1461587218"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0005__ol2911995153537"><li id="dli_01_0005__li26207957153537">On the left of the management console, choose <span class="uicontrol" id="dli_01_0005__uicontrol1559686145"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0005__uicontrol8597462415"><b>Databases and Tables</b></span>.</li><li id="dli_01_0005__li17500832153537">On the <strong id="dli_01_0005__b1211154675314">Databases and Tables</strong> page, select the database for which you want to create a table. In the <strong id="dli_01_0005__b10345811102717">Operation</strong> column, click <strong id="dli_01_0005__b12918326165019">More</strong> &gt; <strong id="dli_01_0005__b63873822719">Create Table</strong> to create a table in the current database.</li></ol> <ul id="dli_01_0005__ul22405571153537"><li id="dli_01_0005__li323555153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle287164588"><b>Data Management</b></span> page:<ol type="a" id="dli_01_0005__ol2911995153537"><li id="dli_01_0005__li26207957153537">On the left of the management console, choose <span class="uicontrol" id="dli_01_0005__uicontrol1559686145"><b>Data Management</b></span> &gt; <span class="uicontrol" id="dli_01_0005__uicontrol8597462415"><b>Databases and Tables</b></span>.</li><li id="dli_01_0005__li17500832153537">On the <strong id="dli_01_0005__b1211154675314">Databases and Tables</strong> page, select the database for which you want to create a table. In the <strong id="dli_01_0005__b10345811102717">Operation</strong> column, click <strong id="dli_01_0005__b12918326165019">More</strong> &gt; <strong id="dli_01_0005__b63873822719">Create Table</strong> to create a table in the current database.</li></ol>
</li><li id="dli_01_0005__li8281288153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle8700317181735"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0005__ol7422733153537"><li id="dli_01_0005__li66804605153537">On the left of the management console, click <span class="uicontrol" id="dli_01_0005__uicontrol11208161745"><b>SQL Editor</b></span>.</li><li id="dli_01_0005__li17512154252514">In the navigation pane of the displayed <span class="wintitle" id="dli_01_0005__wintitle13614162313185"><b>SQL Editor</b></span> page, click <strong id="dli_01_0005__b4615162311183">Databases</strong>. You can create a table in either of the following ways:<ul id="dli_01_0005__ul782474512255"><li id="dli_01_0005__li1893119202619">Click a database name. In the <strong id="dli_01_0005__b5326151882218">Tables</strong> area, click <span><img id="dli_01_0005__image8327018142210" src="en-us_image_0237539075.png"></span> on the right to create a table in the current database.</li><li id="dli_01_0005__li16824154518252">Click <span><img id="dli_01_0005__image33241729102220" src="en-us_image_0237532018.png"></span> on the right of the database and choose <strong id="dli_01_0005__b1325182917220">Create Table</strong> from the shortcut menu to create a table in the current database.</li></ul> </li><li id="dli_01_0005__li8281288153537">To create a table on the <span class="wintitle" id="dli_01_0005__wintitle8700317181735"><b>SQL Editor</b></span> page:<ol type="a" id="dli_01_0005__ol7422733153537"><li id="dli_01_0005__li66804605153537">On the left of the management console, click <span class="uicontrol" id="dli_01_0005__uicontrol11208161745"><b>SQL Editor</b></span>.</li><li id="dli_01_0005__li17512154252514">In the navigation pane of the displayed <span class="wintitle" id="dli_01_0005__wintitle13614162313185"><b>SQL Editor</b></span> page, click <strong id="dli_01_0005__b4615162311183">Databases</strong>. You can create a table in either of the following ways:<ul id="dli_01_0005__ul782474512255"><li id="dli_01_0005__li1893119202619">Click a database name. In the <strong id="dli_01_0005__b5326151882218">Tables</strong> area, click <span><img id="dli_01_0005__image8327018142210" src="en-us_image_0237539075.png"></span> on the right to create a table in the current database.</li><li id="dli_01_0005__li16824154518252">Click <span><img id="dli_01_0005__image33241729102220" src="en-us_image_0237532018.png"></span> on the right of the database and choose <strong id="dli_01_0005__b1325182917220">Create Table</strong> from the shortcut menu to create a table in the current database.</li></ul>
</li></ol> </li></ol>
</li></ul> </li></ul>
@ -108,7 +108,7 @@
<tr id="dli_01_0005__row16943758105944"><td class="cellrowborder" valign="top" width="14.38%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.1 "><p id="dli_01_0005__p30267119105944">Type</p> <tr id="dli_01_0005__row16943758105944"><td class="cellrowborder" valign="top" width="14.38%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.1 "><p id="dli_01_0005__p30267119105944">Type</p>
</td> </td>
<td class="cellrowborder" valign="top" width="71.45%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.2 "><p id="dli_01_0005__p35717569105944">Data type of a column. This parameter corresponds to <span class="parmname" id="dli_01_0005__parmname4155795592938"><b>Column Name</b></span>.</p> <td class="cellrowborder" valign="top" width="71.45%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.2 "><p id="dli_01_0005__p35717569105944">Data type of a column. This parameter corresponds to <span class="parmname" id="dli_01_0005__parmname4155795592938"><b>Column Name</b></span>.</p>
<ul id="dli_01_0005__ul2957726293032"><li id="dli_01_0005__li159154616209"><strong id="dli_01_0005__b12861523131117">string</strong>: The data is of the string type.</li><li id="dli_01_0005__li5035776893042"><strong id="dli_01_0005__b84235270617190">int</strong>: Each integer is stored on four bytes.</li><li id="dli_01_0005__li197172662015"><strong id="dli_01_0005__b71123713114">date</strong>: The value ranges from 0000-01-01 to 9999-12-31.</li><li id="dli_01_0005__li15276123414204"><strong id="dli_01_0005__b2073283811110">double</strong>: Each number is stored on eight bytes.</li><li id="dli_01_0005__li122191918102110"><strong id="dli_01_0005__b87406418113">boolean</strong>: Each value is stored on one byte.</li><li id="dli_01_0005__li616884422117"><strong id="dli_01_0005__b842352706172140">decimal</strong>: The valid bits are positive integers between 1 to 38, including 1 and 38. The decimal digits are integers less than 10.</li><li id="dli_01_0005__li6039945193127"><strong id="dli_01_0005__b657553069">smallint/short</strong>: The number is stored on two bytes.</li><li id="dli_01_0005__li157282298387"><strong id="dli_01_0005__b1143915494">bigint/long</strong>: The number is stored on eight bytes.</li><li id="dli_01_0005__li4930135112217"><strong id="dli_01_0005__b1251694319111">timestamp</strong>: The data indicates a date and time. The value can be accurate to six decimal points.</li><li id="dli_01_0005__li482517952312"><strong id="dli_01_0005__b161442457112">float</strong>: Each number is stored on four bytes.</li><li id="dli_01_0005__li246310417404"><strong id="dli_01_0005__b0368141417399">tinyint</strong>: Each number is stored on one byte. Only OBS tables support this data type.</li></ul> <ul id="dli_01_0005__ul2957726293032"><li id="dli_01_0005__li159154616209"><strong id="dli_01_0005__b12861523131117">string</strong>: The data is of the string type.</li><li id="dli_01_0005__li5035776893042"><strong id="dli_01_0005__b84235270617190">int</strong>: Each integer is stored on four bytes.</li><li id="dli_01_0005__li197172662015"><strong id="dli_01_0005__b71123713114">date</strong>: The value ranges from 0000-01-01 to 9999-12-31.</li><li id="dli_01_0005__li15276123414204"><strong id="dli_01_0005__b2073283811110">double</strong>: Each number is stored on eight bytes.</li><li id="dli_01_0005__li122191918102110"><strong id="dli_01_0005__b87406418113">boolean</strong>: Each value is stored on one byte.</li><li id="dli_01_0005__li616884422117"><strong id="dli_01_0005__b842352706172140">decimal</strong>: The valid bits are positive integers between 1 to 38, including 1 and 38. The decimal digits are integers less than 10.</li><li id="dli_01_0005__li6039945193127"><strong id="dli_01_0005__b1518923443">smallint/short</strong>: The number is stored on two bytes.</li><li id="dli_01_0005__li157282298387"><strong id="dli_01_0005__b612962805">bigint/long</strong>: The number is stored on eight bytes.</li><li id="dli_01_0005__li4930135112217"><strong id="dli_01_0005__b1251694319111">timestamp</strong>: The data indicates a date and time. The value can be accurate to six decimal points.</li><li id="dli_01_0005__li482517952312"><strong id="dli_01_0005__b161442457112">float</strong>: Each number is stored on four bytes.</li><li id="dli_01_0005__li246310417404"><strong id="dli_01_0005__b0368141417399">tinyint</strong>: Each number is stored on one byte. Only OBS tables support this data type.</li></ul>
</td> </td>
<td class="cellrowborder" valign="top" width="14.17%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.3 "><p id="dli_01_0005__p7441963105944">string</p> <td class="cellrowborder" valign="top" width="14.17%" headers="mcps1.3.4.3.2.2.1.2.5.2.4.1.3 "><p id="dli_01_0005__p7441963105944">string</p>
</td> </td>
@ -144,7 +144,7 @@
<tbody><tr id="dli_01_0005__row3015013618314"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p3709008118455">Data Format</p> <tbody><tr id="dli_01_0005__row3015013618314"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p3709008118455">Data Format</p>
</td> </td>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p5150660918455">DLI supports the following data formats:</p> <td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p5150660918455">DLI supports the following data formats:</p>
<ul id="dli_01_0005__ul4173603518455"><li id="dli_01_0005__li4007999518455"><strong id="dli_01_0005__b84235270614338">Parquet</strong>: DLI can read non-compressed data or data that is compressed using Snappy and gzip.</li><li id="dli_01_0005__li2517563618455"><strong id="dli_01_0005__b1357964886">CSV</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li2525413518455"><strong id="dli_01_0005__b1998844488">ORC</strong>: DLI can read non-compressed data or data that is compressed using Snappy.</li><li id="dli_01_0005__li2596063018455"><strong id="dli_01_0005__b1305079628">JSON</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li20203144172320"><strong id="dli_01_0005__b14575133533814">Avro</strong>: DLI can read uncompressed Avro data.</li></ul> <ul id="dli_01_0005__ul4173603518455"><li id="dli_01_0005__li4007999518455"><strong id="dli_01_0005__b84235270614338">Parquet</strong>: DLI can read non-compressed data or data that is compressed using Snappy and gzip.</li><li id="dli_01_0005__li2517563618455"><strong id="dli_01_0005__b1650277512">CSV</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li2525413518455"><strong id="dli_01_0005__b1415457777">ORC</strong>: DLI can read non-compressed data or data that is compressed using Snappy.</li><li id="dli_01_0005__li2596063018455"><strong id="dli_01_0005__b2019437101">JSON</strong>: DLI can read non-compressed data or data that is compressed using gzip.</li><li id="dli_01_0005__li20203144172320"><strong id="dli_01_0005__b14575133533814">Avro</strong>: DLI can read uncompressed Avro data.</li></ul>
</td> </td>
<td class="cellrowborder" valign="top" width="30.330000000000002%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.3 "><p id="dli_01_0005__p2243627218455">CSV</p> <td class="cellrowborder" valign="top" width="30.330000000000002%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.3 "><p id="dli_01_0005__p2243627218455">CSV</p>
</td> </td>
@ -177,7 +177,7 @@
</tr> </tr>
<tr id="dli_01_0005__row21739275111826"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p39917803111836">User-defined Quotation Character</p> <tr id="dli_01_0005__row21739275111826"><td class="cellrowborder" valign="top" width="16.6%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.1 "><p id="dli_01_0005__p39917803111836">User-defined Quotation Character</p>
</td> </td>
<td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p056181715520">This parameter is valid only when <span class="parmname" id="dli_01_0005__parmname570681823"><b>Data Format</b></span> is set to <span class="parmvalue" id="dli_01_0005__parmvalue141904967"><b>CSV</b></span> and you select <strong id="dli_01_0005__b255429859">User-defined Quotation Character</strong>.</p> <td class="cellrowborder" valign="top" width="53.06999999999999%" headers="mcps1.3.4.3.2.2.1.2.6.2.4.1.2 "><p id="dli_01_0005__p056181715520">This parameter is valid only when <span class="parmname" id="dli_01_0005__parmname1092179271"><b>Data Format</b></span> is set to <span class="parmvalue" id="dli_01_0005__parmvalue1454536889"><b>CSV</b></span> and you select <strong id="dli_01_0005__b1778514312">User-defined Quotation Character</strong>.</p>
<p id="dli_01_0005__p57404199154711">The following quotation characters are supported:</p> <p id="dli_01_0005__p57404199154711">The following quotation characters are supported:</p>
<ul id="dli_01_0005__ul41920423111836"><li id="dli_01_0005__li41739488111836">Single quotation mark (')</li><li id="dli_01_0005__li40111075111836">Double quotation marks (")</li><li id="dli_01_0005__li25455362111836">Others: Enter a user-defined quotation character.</li></ul> <ul id="dli_01_0005__ul41920423111836"><li id="dli_01_0005__li41739488111836">Single quotation mark (')</li><li id="dli_01_0005__li40111075111836">Double quotation marks (")</li><li id="dli_01_0005__li25455362111836">Others: Enter a user-defined quotation character.</li></ul>
</td> </td>
@ -213,7 +213,7 @@
</table> </table>
</div> </div>
</li></ul> </li></ul>
</p></li><li id="dli_01_0005__li6436679692218"><span>Click <span class="uicontrol" id="dli_01_0005__uicontrol1346028235151847"><b>OK</b></span>.</span><p><p id="dli_01_0005__p2255887315231">After a table is created, you can view and select the table for use on the <span class="wintitle" id="dli_01_0005__wintitle363616988"><b>Data Management</b></span> page or <strong id="dli_01_0005__b1856357937">SQL Editor</strong> page.</p> </p></li><li id="dli_01_0005__li6436679692218"><span>Click <span class="uicontrol" id="dli_01_0005__uicontrol1346028235151847"><b>OK</b></span>.</span><p><p id="dli_01_0005__p2255887315231">After a table is created, you can view and select the table for use on the <span class="wintitle" id="dli_01_0005__wintitle1248649571"><b>Data Management</b></span> page or <strong id="dli_01_0005__b1826378134">SQL Editor</strong> page.</p>
</p></li><li id="dli_01_0005__li66082750104919"><span>(Optional) After a DLI table is created, you can decide whether to directly import data to the table.</span></li></ol> </p></li><li id="dli_01_0005__li66082750104919"><span>(Optional) After a DLI table is created, you can decide whether to directly import data to the table.</span></li></ol>
</div> </div>
</div> </div>

5
docs/dli/umn/dli_01_0006.html
View File

@ -5,9 +5,9 @@
<p id="dli_01_0006__p25621621205213">For example, to connect DLI to the MRS, RDS, CSS, Kafka, or GaussDB(DWS) data source, you need to enable the network between DLI and the VPC of the data source.</p> <p id="dli_01_0006__p25621621205213">For example, to connect DLI to the MRS, RDS, CSS, Kafka, or GaussDB(DWS) data source, you need to enable the network between DLI and the VPC of the data source.</p>
<p id="dli_01_0006__p163611244144318">Create an enhanced datasource connection on the console.</p> <p id="dli_01_0006__p163611244144318">Create an enhanced datasource connection on the console.</p>
</div> </div>
<div class="section" id="dli_01_0006__section1497781674410"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0006__dli_01_0003_ul920917130139"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3320184019107"><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the constraints on IP addresses are as follows:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul1254904218358"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p26801956133713"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373"></a><a name="dli_01_0003_dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b37501722194120">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11270124004111">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b14999194774118">:8181</strong>.</p> <div class="section" id="dli_01_0006__section1497781674410"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0006__dli_01_0003_ul920917130139"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li82094136131">Datasource connections cannot be created for the <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b275994812408">default</strong> queue.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li2599248131320">Flink jobs can directly access DIS, OBS, and SMN data sources without using datasource connections.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3320184019107"><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b15858611104116">VPC Administrator</strong> permissions are required for enhanced connections to use VPCs, subnets, routes, VPC peering connections.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li196339467176">If you use an enhanced datasource connection, the CIDR block of the elastic resource pool or queue cannot overlap with that of the data source.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li3209111310134">Only queues bound with datasource connections can access datasource tables.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1120931310136">Datasource tables do not support the preview function.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li11700174614348">When checking the connectivity of datasource connections, the notes and constraints on IP addresses are:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul1254904218358"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li15478548367">The IP address must be valid, which consists of four decimal numbers separated by periods (.). The value ranges from 0 to 255.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373">During the test, you can add a port after the IP address and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p26801956133713"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li151317527373"></a><a name="dli_01_0003_dli_07_0005_li151317527373"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11869171511417">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i1917651964115">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b37501722194120">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i151751326134116">xx</em> or <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b4220103184115">192.168.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i7940153620413">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11270124004111">.</strong><em id="dli_01_0006__dli_01_0003_dli_07_0005_i197261843124114">xx</em><strong id="dli_01_0006__dli_01_0003_dli_07_0005_b14999194774118">:8181</strong>.</p>
</li></ul> </li></ul>
</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1152733115719">When checking the connectivity of datasource connections, the constraints on domain names are as follows:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul166789287574"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1867852835715">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p1667882814575"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li1867852835715"></a><a name="dli_01_0003_dli_07_0005_li1867852835715"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b177998236451">example.com:8080</strong>.</p> </li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1152733115719">When checking the connectivity of datasource connections, the notes and constraints on domain names are:<ul id="dli_01_0006__dli_01_0003_dli_07_0005_ul166789287574"><li id="dli_01_0006__dli_01_0003_dli_07_0005_li46789287572">The domain name can contain 1 to 255 characters. Only letters, digits, underscores (_), and hyphens (-) are allowed.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li667882815577">The top-level domain name must contain at least two letters, for example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b11901337154319">.com</strong>, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b1028814124319">.net</strong>, and <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b19839144614436">.cn</strong>.</li><li id="dli_01_0006__dli_01_0003_dli_07_0005_li1867852835715">During the test, you can add a port after the domain name and separate them with colons (:). The port can contain a maximum of five digits. The value ranges from 0 to 65535.<p id="dli_01_0006__dli_01_0003_dli_07_0005_p1667882814575"><a name="dli_01_0006__dli_01_0003_dli_07_0005_li1867852835715"></a><a name="dli_01_0003_dli_07_0005_li1867852835715"></a>For example, <strong id="dli_01_0006__dli_01_0003_dli_07_0005_b177998236451">example.com:8080</strong>.</p>
</li></ul> </li></ul>
</li></ul> </li></ul>
</div> </div>
@ -137,7 +137,6 @@
</li><li id="dli_01_0006__li1023215315019">In the navigation pane of the DLI management console, choose <strong id="dli_01_0006__b483294518252">Resources</strong> &gt; <strong id="dli_01_0006__b61881242152610">Queue Management</strong>.</li><li id="dli_01_0006__li19115123085812">Locate the queue bound with the enhanced datasource connection, click <strong id="dli_01_0006__b10722162617297">More</strong> in the <strong id="dli_01_0006__b414583012918">Operation</strong> column, and select <strong id="dli_01_0006__b138134122913">Test Address Connectivity</strong>.</li><li id="dli_01_0006__li1511513013586">Enter the data source connection address and port number to test the network connectivity.<p id="dli_01_0006__p084013813116"><a name="dli_01_0006__li1511513013586"></a><a name="li1511513013586"></a>Format: <em id="dli_01_0006__i16130144143715">IP address</em><strong id="dli_01_0006__b138521449193715">:</strong><em id="dli_01_0006__i094064620377">Port number</em></p> </li><li id="dli_01_0006__li1023215315019">In the navigation pane of the DLI management console, choose <strong id="dli_01_0006__b483294518252">Resources</strong> &gt; <strong id="dli_01_0006__b61881242152610">Queue Management</strong>.</li><li id="dli_01_0006__li19115123085812">Locate the queue bound with the enhanced datasource connection, click <strong id="dli_01_0006__b10722162617297">More</strong> in the <strong id="dli_01_0006__b414583012918">Operation</strong> column, and select <strong id="dli_01_0006__b138134122913">Test Address Connectivity</strong>.</li><li id="dli_01_0006__li1511513013586">Enter the data source connection address and port number to test the network connectivity.<p id="dli_01_0006__p084013813116"><a name="dli_01_0006__li1511513013586"></a><a name="li1511513013586"></a>Format: <em id="dli_01_0006__i16130144143715">IP address</em><strong id="dli_01_0006__b138521449193715">:</strong><em id="dli_01_0006__i094064620377">Port number</em></p>
<div class="caution" id="dli_01_0006__note201151126864"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dli_01_0006__p7115112614615">Before testing the connection, ensure that the security group of the external data source has allowed access from the CIDR block of the queue.</p> <div class="caution" id="dli_01_0006__note201151126864"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="dli_01_0006__p7115112614615">Before testing the connection, ensure that the security group of the external data source has allowed access from the CIDR block of the queue.</p>
</div></div> </div></div>
<p id="dli_01_0006__p76831391819"></p>
</li></ol> </li></ol>
</p></li></ol> </p></li></ol>
</div> </div>

2
docs/dli/umn/dli_01_0012.html
View File

@ -16,7 +16,7 @@
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0443.html">Modifying the CIDR Block</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0443.html">Modifying the CIDR Block</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0487.html">Elastic Queue Scaling</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0487.html">Elastic Scaling of Queues</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0488.html">Scheduling CU Changes</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0488.html">Scheduling CU Changes</a></strong><br>
</li> </li>

2
docs/dli/umn/dli_01_0013.html
View File

@ -1,6 +1,6 @@
<a name="dli_01_0013"></a><a name="dli_01_0013"></a> <a name="dli_01_0013"></a><a name="dli_01_0013"></a>
<h1 class="topictitle1">Modifying Host Information</h1> <h1 class="topictitle1">Modifying Host Information in an Elastic Resource Pool</h1>
<div id="body0000001152155459"><div class="section" id="dli_01_0013__section17615192019582"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0013__p171819419301">Host information is the mapping between host IP addresses and domain names. After you configure host information, jobs can only use the configured domain names to access corresponding hosts. After a datasource connection is created, you can modify the host information.</p> <div id="body0000001152155459"><div class="section" id="dli_01_0013__section17615192019582"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0013__p171819419301">Host information is the mapping between host IP addresses and domain names. After you configure host information, jobs can only use the configured domain names to access corresponding hosts. After a datasource connection is created, you can modify the host information.</p>
<p id="dli_01_0013__p10340194113302">When accessing the HBase cluster of MRS, you need to configure the host name (domain name) and IP address of the instance.</p> <p id="dli_01_0013__p10340194113302">When accessing the HBase cluster of MRS, you need to configure the host name (domain name) and IP address of the instance.</p>
</div> </div>

3
docs/dli/umn/dli_01_0228.html
View File

@ -5,7 +5,7 @@
<ul id="dli_01_0228__ul32249213483"><li id="dli_01_0228__li64311610480"><a href="dli_01_0447.html">Database Permission Management</a></li><li id="dli_01_0228__li17850104752718"><a href="dli_01_0448.html">Table Permission Management</a></li><li id="dli_01_0228__li1522410210481"><a href="dli_01_0005.html">Creating a Database or a Table</a></li><li id="dli_01_0228__li8722114154811"><a href="dli_01_0011.html">Deleting a Database or a Table</a></li><li id="dli_01_0228__li12992135510143"><a href="dli_01_0376.html">Changing the Owners of Databases and Tables</a></li><li id="dli_01_0228__li1113620265596"><a href="dli_01_0253.html">Importing Data</a></li><li id="dli_01_0228__li14584132917710"><a href="dli_01_0010.html">Exporting Data</a></li><li id="dli_01_0228__li942943801"><a href="dli_01_0008.html">Viewing Metadata</a></li><li id="dli_01_0228__li18775184515263"><a href="dli_01_0007.html">Previewing Data</a></li></ul> <ul id="dli_01_0228__ul32249213483"><li id="dli_01_0228__li64311610480"><a href="dli_01_0447.html">Database Permission Management</a></li><li id="dli_01_0228__li17850104752718"><a href="dli_01_0448.html">Table Permission Management</a></li><li id="dli_01_0228__li1522410210481"><a href="dli_01_0005.html">Creating a Database or a Table</a></li><li id="dli_01_0228__li8722114154811"><a href="dli_01_0011.html">Deleting a Database or a Table</a></li><li id="dli_01_0228__li12992135510143"><a href="dli_01_0376.html">Changing the Owners of Databases and Tables</a></li><li id="dli_01_0228__li1113620265596"><a href="dli_01_0253.html">Importing Data</a></li><li id="dli_01_0228__li14584132917710"><a href="dli_01_0010.html">Exporting Data</a></li><li id="dli_01_0228__li942943801"><a href="dli_01_0008.html">Viewing Metadata</a></li><li id="dli_01_0228__li18775184515263"><a href="dli_01_0007.html">Previewing Data</a></li></ul>
<div class="section" id="dli_01_0228__section16492487104553"><h4 class="sectiontitle">Difference Between DLI Tables and OBS Tables</h4><ul id="dli_01_0228__ul6989252182716"><li id="dli_01_0228__li10989352182718">Data stored in DLI tables is applicable to delay-sensitive services, such as interactive queries.</li><li id="dli_01_0228__li119895525278">Data stored in OBS tables is applicable to delay-insensitive services, such as historical data statistics and analysis.</li></ul> <div class="section" id="dli_01_0228__section16492487104553"><h4 class="sectiontitle">Difference Between DLI Tables and OBS Tables</h4><ul id="dli_01_0228__ul6989252182716"><li id="dli_01_0228__li10989352182718">Data stored in DLI tables is applicable to delay-sensitive services, such as interactive queries.</li><li id="dli_01_0228__li119895525278">Data stored in OBS tables is applicable to delay-insensitive services, such as historical data statistics and analysis.</li></ul>
</div> </div>
<div class="section" id="dli_01_0228__section8360181926"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0228__ul71458485820"><li id="dli_01_0228__li6145847588"><strong id="dli_01_0228__b1276146173415">Database</strong><ul id="dli_01_0228__ul1633525163914"><li id="dli_01_0228__dli_07_0005_li1479161816292"><strong id="dli_01_0228__dli_07_0005_b13239173891316">default</strong> is the database built in DLI. You cannot create a database named <strong id="dli_01_0228__dli_07_0005_b112224411159">default</strong>.</li><li id="dli_01_0228__dli_07_0005_li44809180291">DLI supports a maximum of 50 databases.</li></ul> <div class="section" id="dli_01_0228__section8360181926"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_01_0228__ul71458485820"><li id="dli_01_0228__li6145847588"><strong id="dli_01_0228__b1276146173415">Database</strong><ul id="dli_01_0228__ul1633525163914"><li id="dli_01_0228__dli_07_0005_li1479161816292"><strong id="dli_01_0228__dli_07_0005_b13239173891316">default</strong> is the database built in DLI. You cannot create a database named <strong id="dli_01_0228__dli_07_0005_b112224411159">default</strong>.</li><li id="dli_01_0228__dli_07_0005_li44809180291">DLI supports a maximum of 50 databases.</li></ul>
</li><li id="dli_01_0228__li12825137105816"><strong id="dli_01_0228__b1557811476344">Table</strong><ul id="dli_01_0228__ul618418211249"><li id="dli_01_0228__dli_07_0005_li73208126302">DLI supports a maximum of 5,000 tables.</li><li id="dli_01_0228__dli_07_0005_li693410241413">DLI supports the following table types:<ul id="dli_01_0228__dli_07_0005_ul165901694404"><li id="dli_01_0228__dli_07_0005_li18590159184015"><strong id="dli_01_0228__dli_07_0005_b36524201812">MANAGED</strong>: Data is stored in a DLI table.</li><li id="dli_01_0228__dli_07_0005_li45903944018"><strong id="dli_01_0228__dli_07_0005_b9982521141813">EXTERNAL</strong>: Data is stored in an OBS table.</li><li id="dli_01_0228__dli_07_0005_li15901990404"><strong id="dli_01_0228__dli_07_0005_b58918578193">View</strong>: A view can only be created using SQL statements.</li><li id="dli_01_0228__dli_07_0005_li18935939173511">Datasource table: The table type is also <strong id="dli_01_0228__dli_07_0005_b96694314229">EXTERNAL</strong>.</li></ul> </li><li id="dli_01_0228__li12825137105816"><strong id="dli_01_0228__b1557811476344">Table</strong><ul id="dli_01_0228__ul618418211249"><li id="dli_01_0228__dli_07_0005_li73208126302">DLI supports a maximum of 5,000 tables.</li><li id="dli_01_0228__dli_07_0005_li693410241413">DLI supports the following table types:<ul id="dli_01_0228__dli_07_0005_ul165901694404"><li id="dli_01_0228__dli_07_0005_li18590159184015"><strong id="dli_01_0228__dli_07_0005_b36524201812">MANAGED</strong>: Data is stored in a DLI table.</li><li id="dli_01_0228__dli_07_0005_li45903944018"><strong id="dli_01_0228__dli_07_0005_b9982521141813">EXTERNAL</strong>: Data is stored in an OBS table.</li><li id="dli_01_0228__dli_07_0005_li15901990404"><strong id="dli_01_0228__dli_07_0005_b58918578193">View</strong>: A view can only be created using SQL statements.</li><li id="dli_01_0228__dli_07_0005_li18935939173511">Datasource table: The table type is also <strong id="dli_01_0228__dli_07_0005_b96694314229">EXTERNAL</strong>.</li></ul>
</li><li id="dli_01_0228__dli_07_0005_li57353313451">You cannot specify a storage path when creating a DLI table.</li></ul> </li><li id="dli_01_0228__dli_07_0005_li57353313451">You cannot specify a storage path when creating a DLI table.</li></ul>
</li><li id="dli_01_0228__li1179253611016"><strong id="dli_01_0228__b1277364893412">Data import</strong><ul id="dli_01_0228__ul124061017728"><li id="dli_01_0228__dli_07_0005_li63275321982">Only OBS data can be imported to DLI or OBS.</li><li id="dli_01_0228__dli_07_0005_li61329131626">You can import data in CSV, Parquet, ORC, JSON, or Avro format from OBS to tables created on DLI.</li><li id="dli_01_0228__dli_07_0005_li10140613526">To import data in CSV format to a partitioned table, place the partition column in the last column of the data source.</li><li id="dli_01_0228__dli_07_0005_li36678261928">The encoding format of imported data can only be UTF-8.</li></ul> </li><li id="dli_01_0228__li1179253611016"><strong id="dli_01_0228__b1277364893412">Data import</strong><ul id="dli_01_0228__ul124061017728"><li id="dli_01_0228__dli_07_0005_li63275321982">Only OBS data can be imported to DLI or OBS.</li><li id="dli_01_0228__dli_07_0005_li61329131626">You can import data in CSV, Parquet, ORC, JSON, or Avro format from OBS to tables created on DLI.</li><li id="dli_01_0228__dli_07_0005_li10140613526">To import data in CSV format to a partitioned table, place the partition column in the last column of the data source.</li><li id="dli_01_0228__dli_07_0005_li36678261928">The encoding format of imported data can only be UTF-8.</li></ul>
@ -104,6 +104,7 @@
<tr id="dli_01_0228__row1808175125310"><td class="cellrowborder" valign="top" width="19.35%" headers="mcps1.3.6.4.2.3.1.1 "><p id="dli_01_0228__p108087545313">Last Accessed</p> <tr id="dli_01_0228__row1808175125310"><td class="cellrowborder" valign="top" width="19.35%" headers="mcps1.3.6.4.2.3.1.1 "><p id="dli_01_0228__p108087545313">Last Accessed</p>
</td> </td>
<td class="cellrowborder" valign="top" width="80.65%" headers="mcps1.3.6.4.2.3.1.2 "><p id="dli_01_0228__p4808551537">Last time when an operation was performed on the table.</p> <td class="cellrowborder" valign="top" width="80.65%" headers="mcps1.3.6.4.2.3.1.2 "><p id="dli_01_0228__p4808551537">Last time when an operation was performed on the table.</p>
<p id="dli_01_0228__p1037815446506">The last access time of a table refers only to the last time it was updated, not the time it was read (SELECT operation).</p>
</td> </td>
</tr> </tr>
<tr id="dli_01_0228__row1695151133915"><td class="cellrowborder" valign="top" width="19.35%" headers="mcps1.3.6.4.2.3.1.1 "><p id="dli_01_0228__p369521113915">Operation</p> <tr id="dli_01_0228__row1695151133915"><td class="cellrowborder" valign="top" width="19.35%" headers="mcps1.3.6.4.2.3.1.1 "><p id="dli_01_0228__p369521113915">Operation</p>

336
docs/dli/umn/dli_01_0318.html
View File

File diff suppressed because it is too large Load Diff

1
docs/dli/umn/dli_01_0320.html
View File

@ -218,7 +218,6 @@
</div></div> </div></div>
</li><li id="dli_01_0320__li11718136711">Select a queue from the queue list in the upper left corner of the SQL job editing window. For details about how to create a queue, see <a href="dli_01_0363.html">Creating a Queue</a>.</li><li id="dli_01_0320__li813817834515">In the upper right corner of the SQL job editing window, select a database, for example, <span class="parmvalue" id="dli_01_0320__parmvalue15915351115713"><b>qw</b></span>, from the <strong id="dli_01_0320__b237816275810">Databases</strong> drop-down list.</li><li id="dli_01_0320__li38121318715">Create a table, for example, <strong id="dli_01_0320__b104711617155817">qw</strong>. For details about how to create a database and table, see <a href="dli_01_0005.html">Creating a Database or a Table</a>.</li><li id="dli_01_0320__li11683162919577">In the SQL job editing window, enter the following SQL statement:<div class="codecoloring" codetype="Sql" id="dli_01_0320__screen109547551261"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">qw</span><span class="p">.</span><span class="n">qw</span><span class="w"> </span><span class="k">LIMIT</span><span class="w"> </span><span class="mi">10</span><span class="p">;</span> </li><li id="dli_01_0320__li11718136711">Select a queue from the queue list in the upper left corner of the SQL job editing window. For details about how to create a queue, see <a href="dli_01_0363.html">Creating a Queue</a>.</li><li id="dli_01_0320__li813817834515">In the upper right corner of the SQL job editing window, select a database, for example, <span class="parmvalue" id="dli_01_0320__parmvalue15915351115713"><b>qw</b></span>, from the <strong id="dli_01_0320__b237816275810">Databases</strong> drop-down list.</li><li id="dli_01_0320__li38121318715">Create a table, for example, <strong id="dli_01_0320__b104711617155817">qw</strong>. For details about how to create a database and table, see <a href="dli_01_0005.html">Creating a Database or a Table</a>.</li><li id="dli_01_0320__li11683162919577">In the SQL job editing window, enter the following SQL statement:<div class="codecoloring" codetype="Sql" id="dli_01_0320__screen109547551261"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">qw</span><span class="p">.</span><span class="n">qw</span><span class="w"> </span><span class="k">LIMIT</span><span class="w"> </span><span class="mi">10</span><span class="p">;</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
<p id="dli_01_0320__p96831529105716">Alternatively, you can double-click the table name <strong id="dli_01_0320__b17891859173716">qw</strong>. The query statement is automatically entered in the SQL job editing window.</p> <p id="dli_01_0320__p96831529105716">Alternatively, you can double-click the table name <strong id="dli_01_0320__b17891859173716">qw</strong>. The query statement is automatically entered in the SQL job editing window.</p>
</li><li id="dli_01_0320__li492135719">On top of the editing window, click <strong id="dli_01_0320__b26243389451">More</strong> &gt; <span class="uicontrol" id="dli_01_0320__uicontrol58893394529"><b>Verify Syntax</b></span> to check whether the SQL statement is correct.<ol type="a" id="dli_01_0320__ol1395130715"><li id="dli_01_0320__li153171751102318">If the verification fails, check the SQL statement syntax by referring to <em id="dli_01_0320__i5105915141610">Data Lake Insight SQL Syntax Reference</em>.</li><li id="dli_01_0320__li09121319713">If the syntax verification is successful, click <strong id="dli_01_0320__b238821051610">Execute</strong>. Read and agree to the privacy agreement. Click <strong id="dli_01_0320__b386161413163">OK</strong> to execute the SQL statement.</li><li id="dli_01_0320__li4991320712">After the execution is complete, you can view the execution result in the area under the SQL job editing window.</li></ol> </li><li id="dli_01_0320__li492135719">On top of the editing window, click <strong id="dli_01_0320__b26243389451">More</strong> &gt; <span class="uicontrol" id="dli_01_0320__uicontrol58893394529"><b>Verify Syntax</b></span> to check whether the SQL statement is correct.<ol type="a" id="dli_01_0320__ol1395130715"><li id="dli_01_0320__li153171751102318">If the verification fails, check the SQL statement syntax by referring to <em id="dli_01_0320__i5105915141610">Data Lake Insight SQL Syntax Reference</em>.</li><li id="dli_01_0320__li09121319713">If the syntax verification is successful, click <strong id="dli_01_0320__b238821051610">Execute</strong>. Read and agree to the privacy agreement. Click <strong id="dli_01_0320__b386161413163">OK</strong> to execute the SQL statement.</li><li id="dli_01_0320__li4991320712">After the execution is complete, you can view the execution result in the area under the SQL job editing window.</li></ol>

2
docs/dli/umn/dli_01_0384.html
View File

@ -5,7 +5,7 @@
<p id="dli_01_0384__p55886519497">On the <strong id="dli_01_0384__b943884293511">Overview</strong> page, click <strong id="dli_01_0384__b202117331159">Create Job</strong> in the upper right corner of the <strong id="dli_01_0384__b1696101133814">Spark Jobs</strong> tab or click <strong id="dli_01_0384__b56361152153720">Create Job</strong> in the upper right corner of the <strong id="dli_01_0384__b993313572052">Spark Jobs</strong> page. The Spark job editing page is displayed.</p> <p id="dli_01_0384__p55886519497">On the <strong id="dli_01_0384__b943884293511">Overview</strong> page, click <strong id="dli_01_0384__b202117331159">Create Job</strong> in the upper right corner of the <strong id="dli_01_0384__b1696101133814">Spark Jobs</strong> tab or click <strong id="dli_01_0384__b56361152153720">Create Job</strong> in the upper right corner of the <strong id="dli_01_0384__b993313572052">Spark Jobs</strong> page. The Spark job editing page is displayed.</p>
<p id="dli_01_0384__p9427817122315">On the Spark job editing page, a message is displayed, indicating that a temporary DLI data bucket will be created. The created bucket is used to store temporary data generated by DLI, such as job logs and job results. You cannot view job logs if you choose not to create it. The bucket will be created and the default bucket name is used.</p> <p id="dli_01_0384__p9427817122315">On the Spark job editing page, a message is displayed, indicating that a temporary DLI data bucket will be created. The created bucket is used to store temporary data generated by DLI, such as job logs and job results. You cannot view job logs if you choose not to create it. The bucket will be created and the default bucket name is used.</p>
<p id="dli_01_0384__p133447114312">If you do not need to create a DLI temporary data bucket and do not want to receive this message, select <strong id="dli_01_0384__b142701146121612">Do not show again</strong> and click <span class="uicontrol" id="dli_01_0384__uicontrol227014464167"><b>Cancel</b></span>.</p> <p id="dli_01_0384__p133447114312">If you do not need to create a DLI temporary data bucket and do not want to receive this message, select <strong id="dli_01_0384__b142701146121612">Do not show again</strong> and click <span class="uicontrol" id="dli_01_0384__uicontrol227014464167"><b>Cancel</b></span>.</p>
<div class="section" id="dli_01_0384__section1661873674317"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_01_0384__ul9494144631313"><li id="dli_01_0384__li749524681310">You have uploaded the dependencies to the corresponding OBS bucket on the <span class="wintitle" id="dli_01_0384__wintitle1216872122520"><b>Data Management &gt; Package Management</b></span> page. For details, see <a href="dli_01_0367.html">Creating a Package</a>.</li><li id="dli_01_0384__li7520181309">Before creating a Spark job to access other external data sources, such as OpenTSDB, HBase, Kafka, GaussDB(DWS), RDS, CSS, CloudTable, DCS Redis, and DDS, you need to create a datasource connection to enable the network between the job running queue and external data sources.<ul id="dli_01_0384__ul850415199816"><li id="dli_01_0384__li18451528175">For details about the external data sources that can be accessed by Spark jobs, see <a href="dli_01_0410.html">Cross-Source Analysis Development Methods</a>.</li><li id="dli_01_0384__li28469131945">For details about how to create a datasource connection, see <a href="dli_01_0426.html">Enhanced Datasource Connections</a>.<p id="dli_01_0384__p881561411413">On the <strong id="dli_01_0384__b122201520155014">Resources</strong> &gt; <strong id="dli_01_0384__b1022002035016">Queue Management</strong> page, locate the queue you have created, and choose <strong id="dli_01_0384__b112219202504">More</strong> &gt; <strong id="dli_01_0384__b0221182013507">Test Address Connectivity</strong> in the <strong id="dli_01_0384__b422192015507">Operation</strong> column to check whether the network connection between the queue and the data source is normal. For details, see <a href="dli_01_0489.html">Testing Address Connectivity</a>.</p> <div class="section" id="dli_01_0384__section1661873674317"><h4 class="sectiontitle">Prerequisites</h4><ul id="dli_01_0384__ul9494144631313"><li id="dli_01_0384__li749524681310">You have uploaded the dependencies to the corresponding OBS bucket on the <span class="wintitle" id="dli_01_0384__wintitle1216872122520"><b>Data Management &gt; Package Management</b></span> page. For details, see <a href="dli_01_0367.html">Creating a Package</a>.</li><li id="dli_01_0384__li7520181309">Before creating a Spark job to access other external data sources, such as OpenTSDB, HBase, Kafka, GaussDB(DWS), RDS, CSS, CloudTable, DCS Redis, and DDS, you need to create a datasource connection to enable the network between the job running queue and external data sources.<ul id="dli_01_0384__ul850415199816"><li id="dli_01_0384__li18451528175">For details about the external data sources that can be accessed by Spark jobs, see <a href="dli_01_0410.html">Cross-Source Analysis Development Methods</a>.</li><li id="dli_01_0384__li28469131945">For details about how to create a datasource connection, see <a href="dli_01_0426.html">Enhanced Datasource Connections</a>.<p id="dli_01_0384__p881561411413">On the <strong id="dli_01_0384__b122201520155014">Resources</strong> &gt; <strong id="dli_01_0384__b1022002035016">Queue Management</strong> page, locate the queue you have created, click <strong id="dli_01_0384__b112219202504">More</strong> in the <strong id="dli_01_0384__b422192015507">Operation</strong> column, and select <strong id="dli_01_0384__b0221182013507">Test Address Connectivity</strong> to check if the network connection between the queue and the data source is normal. For details, see <a href="dli_01_0489.html">Testing Address Connectivity</a>.</p>
</li></ul> </li></ul>
</li></ul> </li></ul>
</div> </div>

329
docs/dli/umn/dli_01_0397.html
View File

File diff suppressed because it is too large Load Diff

2
docs/dli/umn/dli_01_0407.html
View File

@ -5,7 +5,7 @@
<ul id="dli_01_0407__ul8657743184112"><li id="dli_01_0407__li14139145132717"><a href="dli_01_0477.html">Managing Package Permissions</a></li><li id="dli_01_0407__li19657124354118"><a href="dli_01_0367.html">Creating a Package</a></li><li id="dli_01_0407__li9471154134119"><a href="dli_01_0369.html">Deleting a Package</a><div class="note" id="dli_01_0407__note1393218181435"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0407__p79328181837">You can delete program packages in batches.</p> <ul id="dli_01_0407__ul8657743184112"><li id="dli_01_0407__li14139145132717"><a href="dli_01_0477.html">Managing Package Permissions</a></li><li id="dli_01_0407__li19657124354118"><a href="dli_01_0367.html">Creating a Package</a></li><li id="dli_01_0407__li9471154134119"><a href="dli_01_0369.html">Deleting a Package</a><div class="note" id="dli_01_0407__note1393218181435"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0407__p79328181837">You can delete program packages in batches.</p>
</div></div> </div></div>
</li><li id="dli_01_0407__li624821620217"><a href="dli_01_0478.html">Modifying the Owner</a></li></ul> </li><li id="dli_01_0407__li624821620217"><a href="dli_01_0478.html">Modifying the Owner</a></li></ul>
<div class="section" id="dli_01_0407__section194110185420"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0407__ul1261192724211"><li id="dli_01_0407__dli_07_0005_li349215401873">A package can be deleted, but a package group cannot be deleted.</li><li id="dli_01_0407__dli_07_0005_li19798321755">The following types of packages can be uploaded:<ul id="dli_01_0407__dli_07_0005_ul1349114421957"><li id="dli_01_0407__dli_07_0005_li359082764320"><strong id="dli_01_0407__dli_07_0005_b13479075378">JAR</strong>: JAR file</li><li id="dli_01_0407__dli_07_0005_li125901627144314"><strong id="dli_01_0407__dli_07_0005_b494901012379">PyFile</strong>: User Python file</li><li id="dli_01_0407__dli_07_0005_li165901027104315"><strong id="dli_01_0407__dli_07_0005_b2012941493715">File</strong>: User file</li><li id="dli_01_0407__dli_07_0005_li259002734313"><strong id="dli_01_0407__dli_07_0005_b17210171817377">ModelFile</strong>: User AI model file</li></ul> <div class="section" id="dli_01_0407__section194110185420"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_01_0407__ul1261192724211"><li id="dli_01_0407__dli_07_0005_li349215401873">A package can be deleted, but a package group cannot be deleted.</li><li id="dli_01_0407__dli_07_0005_li19798321755">The following types of packages can be uploaded:<ul id="dli_01_0407__dli_07_0005_ul1349114421957"><li id="dli_01_0407__dli_07_0005_li359082764320"><strong id="dli_01_0407__dli_07_0005_b13479075378">JAR</strong>: JAR file</li><li id="dli_01_0407__dli_07_0005_li125901627144314"><strong id="dli_01_0407__dli_07_0005_b494901012379">PyFile</strong>: User Python file</li><li id="dli_01_0407__dli_07_0005_li165901027104315"><strong id="dli_01_0407__dli_07_0005_b2012941493715">File</strong>: User file</li><li id="dli_01_0407__dli_07_0005_li259002734313"><strong id="dli_01_0407__dli_07_0005_b17210171817377">ModelFile</strong>: User AI model file</li></ul>
</li></ul> </li></ul>
</div> </div>
<div class="section" id="dli_01_0407__section153923398422"><h4 class="sectiontitle">Package Management Page</h4> <div class="section" id="dli_01_0407__section153923398422"><h4 class="sectiontitle">Package Management Page</h4>

10
docs/dli/umn/dli_01_0420.html
View File

@ -1,13 +1,9 @@
<a name="dli_01_0420"></a><a name="dli_01_0420"></a> <a name="dli_01_0420"></a><a name="dli_01_0420"></a>
<h1 class="topictitle1">Importing Data to a DLI Table</h1> <h1 class="topictitle1">Importing Data to a DLI Table</h1>
<div id="body1562569148234"><div class="section" id="dli_01_0420__section133251856125910"><h4 class="sectiontitle">Importing Data Using OBS</h4><p id="dli_01_0420__p1323701709">On the DLI management console, you can import data stored on OBS to DLI tables from <span class="wintitle" id="dli_01_0420__wintitle89211315417"><b>Data Management &gt; Databases and Tables &gt; Table Management</b></span> and <span class="wintitle" id="dli_01_0420__wintitle892143164111"><b>SQL Editor</b></span> pages. For details, see <a href="dli_01_0253.html">Importing Data to the Table</a>.</p> <div id="body1562569148234"><div class="section" id="dli_01_0420__section133251856125910"><h4 class="sectiontitle">Importing Data Using OBS</h4><p id="dli_01_0420__p749714530489">On the DLI management console, you can import data stored in OBS into DLI tables.</p>
</div> <p id="dli_01_0420__p162143187495">To import OBS data to a DLI table, either choose <strong id="dli_01_0420__b136133052112">Data Management</strong> &gt; <strong id="dli_01_0420__b38655463212">Databases and Tables</strong> in the navigation pane on the left, locate a desired database, and click <strong id="dli_01_0420__b1322318862413">Tables</strong> in the <strong id="dli_01_0420__b96261172419">Operation</strong> column, or choose <strong id="dli_01_0420__b11242245152419">SQL Editor</strong> in the navigation pane on the left.</p>
<div class="section" id="dli_01_0420__section28435121978"><h4 class="sectiontitle">Importing Data Using CDM</h4><p id="dli_01_0420__p194281108614">Use the Cloud Data Migration (CDM) service to import data from OBS to DLI. You need to create a CDM queue first.</p> <p id="dli_01_0420__p1323701709">For details, see <a href="dli_01_0253.html">Importing Data to the Table</a>.</p>
<p id="dli_01_0420__p194387111764"></p>
<p id="dli_01_0420__p432410177">For details about how to create the queue, see "Migrating Data from OBS to DLI" in the <em id="dli_01_0420__i176611119191112">Cloud Data Migration User Guide</em>.</p>
<p id="dli_01_0420__p64282955162347">Pay attention to the following configurations:</p>
<ul id="dli_01_0420__ul64528752175931"><li id="dli_01_0420__li40214787175931">The VPC to which the DLI account belongs is the same as the VPC of the CDM queue.</li><li id="dli_01_0420__li2079024818231">You need to create two links, including a DLI link and an OBS link.</li><li id="dli_01_0420__li45910066182646">The format of the file to be transmitted can be <span class="parmvalue" id="dli_01_0420__parmvalue183613429422"><b>CSV</b></span> or <span class="parmvalue" id="dli_01_0420__parmvalue188361842134219"><b>JSON</b></span>.</li></ul>
</div> </div>
<div class="section" id="dli_01_0420__section85731517984"><h4 class="sectiontitle">Importing Data Using DIS</h4><p id="dli_01_0420__p0948926193816">Use the Data Ingestion Service (DIS) service to import data to DLI. You need to create a DIS stream.</p> <div class="section" id="dli_01_0420__section85731517984"><h4 class="sectiontitle">Importing Data Using DIS</h4><p id="dli_01_0420__p0948926193816">Use the Data Ingestion Service (DIS) service to import data to DLI. You need to create a DIS stream.</p>
<p id="dli_01_0420__p996411571081"></p> <p id="dli_01_0420__p996411571081"></p>

4
docs/dli/umn/dli_01_0426.html
View File

@ -10,9 +10,11 @@
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0006.html">Creating an Enhanced Datasource Connection</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0006.html">Creating an Enhanced Datasource Connection</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0624.html">Establishing a Network Connection Between DLI and Resources in a Shared VPC</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_01_0553.html">Deleting an Enhanced Datasource Connection</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0553.html">Deleting an Enhanced Datasource Connection</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0013.html">Modifying Host Information</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0013.html">Modifying Host Information in an Elastic Resource Pool</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_01_0514.html">Binding and Unbinding a Queue</a></strong><br> <li class="ulchildlink"><strong><a href="dli_01_0514.html">Binding and Unbinding a Queue</a></strong><br>
</li> </li>

2
docs/dli/umn/dli_01_0454.html
View File

@ -17,7 +17,7 @@
<p id="dli_01_0454__p1621014250457">If the port of the Kafka server is listened on by the host name, you need to add the mapping between the host name and IP address of the Kafka Broker node to the datasource connection.</p> <p id="dli_01_0454__p1621014250457">If the port of the Kafka server is listened on by the host name, you need to add the mapping between the host name and IP address of the Kafka Broker node to the datasource connection.</p>
</li><li id="dli_01_0454__li178971556151812">CloudTable as the data input and output channel<p id="dli_01_0454__p8226634192020"><a name="dli_01_0454__li178971556151812"></a><a name="li178971556151812"></a>To use CloudTable as the data input and output channel, create a cluster in CloudTable and obtain the cluster ID. </p> </li><li id="dli_01_0454__li178971556151812">CloudTable as the data input and output channel<p id="dli_01_0454__p8226634192020"><a name="dli_01_0454__li178971556151812"></a><a name="li178971556151812"></a>To use CloudTable as the data input and output channel, create a cluster in CloudTable and obtain the cluster ID. </p>
</li><li id="dli_01_0454__li863414234458">CSS as the output channel<p id="dli_01_0454__p17335201910140"><a name="dli_01_0454__li863414234458"></a><a name="li863414234458"></a>To use CSS as the data output channel, create a cluster in CSS and obtain the cluster's private network address. For details, see <span class="menucascade" id="dli_01_0454__en-us_topic_0077514348_menucascade11750352152811"><b><span class="uicontrol" id="dli_01_0454__uicontrol1675035262818">Getting Started</span></b></span> in the <em id="dli_01_0454__en-us_topic_0077514348_i26427372913">Cloud Search Service User Guide</em>.</p> </li><li id="dli_01_0454__li863414234458">CSS as the output channel<p id="dli_01_0454__p17335201910140"><a name="dli_01_0454__li863414234458"></a><a name="li863414234458"></a>To use CSS as the data output channel, create a cluster in CSS and obtain the cluster's private network address. For details, see <span class="menucascade" id="dli_01_0454__en-us_topic_0077514348_menucascade11750352152811"><b><span class="uicontrol" id="dli_01_0454__uicontrol1675035262818">Getting Started</span></b></span> in the <em id="dli_01_0454__en-us_topic_0077514348_i26427372913">Cloud Search Service User Guide</em>.</p>
</li><li id="dli_01_0454__li758418951518">DCS as the output channel<p id="dli_01_0454__p1910175514211"><a name="dli_01_0454__li758418951518"></a><a name="li758418951518"></a>To use DCS as the output channel, create a Redis cache instance in DCS and obtain the address used for Flink jobs to connect to the Redis instance. For detailed operations, see <span class="menucascade" id="dli_01_0454__en-us_topic_0077514348_menucascade42497134134449"><b><span class="uicontrol" id="dli_01_0454__uicontrol46929886134449">Getting Started</span></b></span> in the <em id="dli_01_0454__en-us_topic_0077514348_i84235269795324">Distributed Cache Service User Guide</em>.</p> </li><li id="dli_01_0454__li758418951518">DCS as the output channel<p id="dli_01_0454__p1910175514211"><a name="dli_01_0454__li758418951518"></a><a name="li758418951518"></a>To use DCS as the output channel, create a Redis cache instance in DCS and obtain the address used for Flink jobs to connect to the Redis instance. For details, see "Buying a DCS Redis Instance" in <em id="dli_01_0454__en-us_topic_0077514348_i84235269795324">Distributed Cache Service User Guide</em>.</p>
</li></ul> </li></ul>
</div> </div>
<div> <div>

10
docs/dli/umn/dli_01_0487.html
View File

@ -1,18 +1,20 @@
<a name="dli_01_0487"></a><a name="dli_01_0487"></a> <a name="dli_01_0487"></a><a name="dli_01_0487"></a>
<h1 class="topictitle1">Elastic Queue Scaling</h1> <h1 class="topictitle1">Elastic Scaling of Queues</h1>
<div id="body0000001132627173"><div class="section" id="dli_01_0487__section93461821194314"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_01_0487__p685412834314">Elastic scaling can be performed for a newly created queue only when there were jobs running in this queue.</p> <div id="body0000001132627173"><div class="section" id="dli_01_0487__section93461821194314"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_01_0487__p685412834314">Elastic scaling can be performed for a newly created queue only when there were jobs running in this queue.</p>
</div> </div>
<div class="section" id="dli_01_0487__section124281747104415"><h4 class="sectiontitle">Precautions</h4><ul id="dli_01_0487__ul799312234711"><li id="dli_01_0487__li253742810428">Queues with 16 CUs do not support scale-out or scale-in.</li><li id="dli_01_0487__li6119202221710">Queues with 64 CUs do not support scale-in.</li><li id="dli_01_0487__li1399332124718">If <strong id="dli_01_0487__b13484143125312">Status of queue xxx is assigning, which is not available</strong> is displayed on the <strong id="dli_01_0487__b7484183115319">Elastic Scaling</strong> page, the queue can be scaled only after the queue resources are allocated.</li></ul> <div class="section" id="dli_01_0487__section124281747104415"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_01_0487__ul799312234711"><li id="dli_01_0487__li253742810428">Queues with 16 CUs do not support scale-out or scale-in.</li><li id="dli_01_0487__li6119202221710">Queues with 64 CUs do not support scale-in.</li><li id="dli_01_0487__li1399332124718">If <strong id="dli_01_0487__b13484143125312">Status of queue xxx is assigning, which is not available</strong> is displayed on the <strong id="dli_01_0487__b7484183115319">Elastic Scaling</strong> page, the queue can be scaled only after the queue resources are allocated.</li><li id="dli_01_0487__li1694171711483">If there are not enough physical resources, a queue may not be able to scale out to the desired target size.</li><li id="dli_01_0487__li1268615126412">The system does not guarantee that a queue will be scaled in to the desired target size. Typically, the system checks the resource usage before scaling in the queue to determine if there is enough space for scaling in. If the existing resources cannot be scaled in according to the minimum scaling step, the queue may not be scaled in successfully or only partially.<p id="dli_01_0487__p7333183017211"><a name="dli_01_0487__li1268615126412"></a><a name="li1268615126412"></a>The scaling step may vary depending on the resource specifications, usually 16 CUs, 32 CUs, 48 CUs, 64 CUs, etc.</p>
<p id="dli_01_0487__p1724813171145">For example, if the queue size is 48 CUs and job execution uses 18 CUs, the remaining 30 CUs do not meet the requirement for scaling in by the minimum step of 32 CUs. If a scaling in task is executed, it will fail.</p>
</li></ul>
</div> </div>
<div class="section" id="dli_01_0487__section1889631562717"><h4 class="sectiontitle">Scaling Out</h4><p id="dli_01_0487__p1097183618266">If the current queue specifications do not meet service requirements, you can add the number of CUs to scale out the queue.</p> <div class="section" id="dli_01_0487__section1889631562717"><h4 class="sectiontitle">Scaling Out</h4><p id="dli_01_0487__p1097183618266">If the current queue specifications do not meet service requirements, you can add the number of CUs to scale out the queue.</p>
<div class="note" id="dli_01_0487__note2898825113314"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0487__p7305155319433">Scale-out is time-consuming. After you perform scale-out on the <span class="wintitle" id="dli_01_0487__wintitle1683985816522"><b>Elastic Scaling</b></span> page of DLI, wait for about 10 minutes. The duration is related to the CU amount to add. After a period of time, refresh the <strong id="dli_01_0487__b158461158105216">Queue Management</strong> page and check whether values of <strong id="dli_01_0487__b58478587522">Specifications</strong> and <strong id="dli_01_0487__b2848658185216">Actual CUs</strong> are the same to determine whether the scale-out is successful. Alternatively, on the <strong id="dli_01_0487__b1091191817541">Job Management</strong> page, check the status of the <strong id="dli_01_0487__b179184185545">SCALE_QUEUE</strong> SQL job. If the job status is <strong id="dli_01_0487__b12918161819545">Scaling</strong>, the queue is being scaled out.</p> <div class="note" id="dli_01_0487__note2898825113314"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0487__p225072184914">Scale-out is time-consuming. After you perform scale-out on the <span class="wintitle" id="dli_01_0487__wintitle1683985816522"><b>Elastic Scaling</b></span> page of DLI, wait for about 10 minutes. The duration is related to the CU amount to add. After a period of time, refresh the <strong id="dli_01_0487__b158461158105216">Queue Management</strong> page and check whether values of <strong id="dli_01_0487__b58478587522">Specifications</strong> and <strong id="dli_01_0487__b2848658185216">Actual CUs</strong> are the same to determine whether the scale-out is successful. Alternatively, on the <strong id="dli_01_0487__b1091191817541">Job Management</strong> page, check the status of the <strong id="dli_01_0487__b179184185545">SCALE_QUEUE</strong> SQL job. If the job status is <strong id="dli_01_0487__b12918161819545">Scaling</strong>, the queue is being scaled out.</p>
</div></div> </div></div>
<p id="dli_01_0487__p31364408338">The procedure is as follows:</p> <p id="dli_01_0487__p31364408338">The procedure is as follows:</p>
<ol id="dli_01_0487__ol2891331122818"><li id="dli_01_0487__li1891031122813">On the left of the DLI management console, click <strong id="dli_01_0487__b11319140174410">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0487__uicontrol6325114011449"><b>Queue Management</b></span>.</li><li id="dli_01_0487__li1711818752915">Select the queue to be scaled out and choose <strong id="dli_01_0487__b18719134811543">More &gt; Elastic Scaling</strong> in the <span class="parmname" id="dli_01_0487__parmname1272514818541"><b>Operation</b></span> column.</li><li id="dli_01_0487__li2021124816302">On the displayed page, select <strong id="dli_01_0487__b16568834135520">Scale-out</strong> for <span class="parmname" id="dli_01_0487__parmname1257493455519"><b>Operation</b></span> and set the scale-out amount.</li><li id="dli_01_0487__li588911107595">Click .</li></ol> <ol id="dli_01_0487__ol2891331122818"><li id="dli_01_0487__li1891031122813">On the left of the DLI management console, click <strong id="dli_01_0487__b11319140174410">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0487__uicontrol6325114011449"><b>Queue Management</b></span>.</li><li id="dli_01_0487__li1711818752915">Select the queue to be scaled out and choose <strong id="dli_01_0487__b18719134811543">More &gt; Elastic Scaling</strong> in the <span class="parmname" id="dli_01_0487__parmname1272514818541"><b>Operation</b></span> column.</li><li id="dli_01_0487__li2021124816302">On the displayed page, select <strong id="dli_01_0487__b16568834135520">Scale-out</strong> for <span class="parmname" id="dli_01_0487__parmname1257493455519"><b>Operation</b></span> and set the scale-out amount.</li><li id="dli_01_0487__li588911107595">Click .</li></ol>
</div> </div>
<div class="section" id="dli_01_0487__section12713340163517"><h4 class="sectiontitle">Scaling In</h4><p id="dli_01_0487__p588782592917">If the current queue specifications are too much for your computing service, you can reduce the number of CUs to scale in the queue.</p> <div class="section" id="dli_01_0487__section12713340163517"><h4 class="sectiontitle">Scaling In</h4><p id="dli_01_0487__p588782592917">If the current queue specifications are too much for your computing service, you can reduce the number of CUs to scale in the queue.</p>
<div class="note" id="dli_01_0487__note17892151111290"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0487__ul1571641564613"><li id="dli_01_0487__li4716215184613">Scale-in is time-consuming. After you perform scale-in on the <span class="wintitle" id="dli_01_0487__wintitle129516254574"><b>Elastic Scaling</b></span> page of DLI, wait for about 10 minutes. The duration is related to the CU amount to reduce. After a period of time, refresh the <span class="wintitle" id="dli_01_0487__wintitle1010122595711"><b>Queue Management</b></span> page and check whether values of <strong id="dli_01_0487__b11102192515712">Specifications</strong> and <strong id="dli_01_0487__b410214257571">Actual CUs</strong> are the same to determine whether the scale-in is successful. Alternatively, on the <strong id="dli_01_0487__b185682573577">Job Management</strong> page, check the status of the <strong id="dli_01_0487__b13575195735711">SCALE_QUEUE</strong> SQL job. If the job status is <strong id="dli_01_0487__b165762057155711">Scaling</strong>, the queue is being scaled in.</li><li id="dli_01_0487__li56151579476">The system may not fully scale in the queue to the target size. If the current queue is in use or the service volume of the queue is large, the scale-in may fail or only partial specifications may be reduced.</li><li id="dli_01_0487__li4286153619115">By default, the minimum number of CUs is <strong id="dli_01_0487__b193801953185816">16</strong>. That is, when the queue specifications are <strong id="dli_01_0487__b12387453155820">16 CUs</strong>, you cannot scale in the queue.</li></ul> <div class="note" id="dli_01_0487__note17892151111290"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0487__ul1571641564613"><li id="dli_01_0487__li4716215184613">Scale-in is time-consuming. After you perform scale-in on the <span class="wintitle" id="dli_01_0487__wintitle129516254574"><b>Elastic Scaling</b></span> page of DLI, wait for about 10 minutes. The duration is related to the CU amount to reduce. After a period of time, refresh the <span class="wintitle" id="dli_01_0487__wintitle1010122595711"><b>Queue Management</b></span> page and check whether values of <strong id="dli_01_0487__b11102192515712">Specifications</strong> and <strong id="dli_01_0487__b410214257571">Actual CUs</strong> are the same to determine whether the scale-in is successful. Alternatively, on the <strong id="dli_01_0487__b185682573577">Job Management</strong> page, check the status of the <strong id="dli_01_0487__b13575195735711">SCALE_QUEUE</strong> SQL job. If the job status is <strong id="dli_01_0487__b165762057155711">Scaling</strong>, the queue is being scaled in.</li><li id="dli_01_0487__li4286153619115">By default, the minimum number of CUs is <strong id="dli_01_0487__b193801953185816">16</strong>. That is, when the queue specifications are <strong id="dli_01_0487__b12387453155820">16 CUs</strong>, you cannot scale in the queue.</li></ul>
</div></div> </div></div>
<p id="dli_01_0487__p2089341118294">The procedure is as follows:</p> <p id="dli_01_0487__p2089341118294">The procedure is as follows:</p>
<ol id="dli_01_0487__ol2893111112910"><li id="dli_01_0487__li15893151113292">On the left of the DLI management console, click <strong id="dli_01_0487__b17573419442">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0487__uicontrol11757241134416"><b>Queue Management</b></span>.</li><li id="dli_01_0487__li289311117290">Select the queue to be scaled out, click <strong id="dli_01_0487__b1668918287712">More</strong> in the <span class="parmname" id="dli_01_0487__parmname67283139596"><b>Operation</b></span> column, and select <strong id="dli_01_0487__b94468423717">Elastic Scaling</strong>.</li><li id="dli_01_0487__li208949116292">On the displayed page, select <strong id="dli_01_0487__b116409262599">Scale-in</strong> for <span class="parmname" id="dli_01_0487__parmname164752655915"><b>Operation</b></span> and set the scale-in amount.</li><li id="dli_01_0487__li106997587918">Click .</li></ol> <ol id="dli_01_0487__ol2893111112910"><li id="dli_01_0487__li15893151113292">On the left of the DLI management console, click <strong id="dli_01_0487__b17573419442">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0487__uicontrol11757241134416"><b>Queue Management</b></span>.</li><li id="dli_01_0487__li289311117290">Select the queue to be scaled out, click <strong id="dli_01_0487__b1668918287712">More</strong> in the <span class="parmname" id="dli_01_0487__parmname67283139596"><b>Operation</b></span> column, and select <strong id="dli_01_0487__b94468423717">Elastic Scaling</strong>.</li><li id="dli_01_0487__li208949116292">On the displayed page, select <strong id="dli_01_0487__b116409262599">Scale-in</strong> for <span class="parmname" id="dli_01_0487__parmname164752655915"><b>Operation</b></span> and set the scale-in amount.</li><li id="dli_01_0487__li106997587918">Click .</li></ol>

4
docs/dli/umn/dli_01_0489.html
View File

@ -2,8 +2,8 @@
<h1 class="topictitle1">Testing Address Connectivity</h1> <h1 class="topictitle1">Testing Address Connectivity</h1>
<div id="body1594301082896"><p id="dli_01_0489__p8060118">It can be used to test the connectivity between the DLI queue and the peer IP address specified by the user in common scenarios, or the connectivity between the DLI queue and the peer IP address bound to the datasource connection in datasource connection scenarios. The operation is as follows:</p> <div id="body1594301082896"><p id="dli_01_0489__p8060118">It can be used to test the connectivity between the DLI queue and the peer IP address specified by the user in common scenarios, or the connectivity between the DLI queue and the peer IP address bound to the datasource connection in datasource connection scenarios. The operation is as follows:</p>
<ol id="dli_01_0489__ol1244412250287"><li id="dli_01_0489__li1771354513304">On the <span class="wintitle" id="dli_01_0489__wintitle14507737191011"><b>Queue Management</b></span> page, locate the row containing the target queue, click <strong id="dli_01_0489__b6992221920">More</strong> in the <span class="parmname" id="dli_01_0489__parmname12662123131512"><b>Operation</b></span> column, and select <strong id="dli_01_0489__b156808121698">Test Address Connectivity</strong>.</li><li id="dli_01_0489__li9592558175016">On the <strong id="dli_01_0489__b743522610911">Test Address Connectivity</strong> page, enter the address to be tested. The domain name and IP address are supported, and the port number can be specified.</li><li id="dli_01_0489__li915918121711">Click <strong id="dli_01_0489__b84235270614344">Test</strong>.<p id="dli_01_0489__p1461846070">If the test address is reachable, a message is displayed on the page, indicating that the address is reachable.</p> <ol id="dli_01_0489__ol1244412250287"><li id="dli_01_0489__li1771354513304">On the <strong id="dli_01_0489__b1215123412711">Queue Management</strong> page, locate the row containing the target queue, click <strong id="dli_01_0489__b6992221920">More</strong> in the <strong id="dli_01_0489__b185817410278">Operation</strong> column, and select <strong id="dli_01_0489__b156808121698">Test Address Connectivity</strong>.</li><li id="dli_01_0489__li9592558175016">On the <strong id="dli_01_0489__b118561248152715">Test Address Connectivity</strong> page, enter the address to be tested. The domain name and IP address are supported, and the port number can be specified.</li><li id="dli_01_0489__li915918121711">Click <strong id="dli_01_0489__b84235270614344">Test</strong>.<p id="dli_01_0489__p1461846070">If the test address is reachable, a message is displayed on the page, indicating that the address is reachable.</p>
<p id="dli_01_0489__p11009167716">If the test address is unreachable, the system displays a message indicating that the address is unreachable. Check the network configurations and try again. Network configurations include the VPC peering and the datasource connection. Check whether they have been activated.</p> <p id="dli_01_0489__p11009167716">If the test address is unreachable, the system displays a message indicating that the address is unreachable. Check the network configurations and retry. Network configurations include the VPC peering and the datasource connection. Check whether they have been activated.</p>
</li></ol> </li></ol>
</div> </div>
<div> <div>

17
docs/dli/umn/dli_01_0504.html
View File

@ -21,7 +21,7 @@
<td class="cellrowborder" valign="top" width="15.001500150015001%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dli_01_0504__dli_07_0027_p12915849201115">16–64 CUs</p> <td class="cellrowborder" valign="top" width="15.001500150015001%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dli_01_0504__dli_07_0027_p12915849201115">16–64 CUs</p>
</td> </td>
<td class="cellrowborder" valign="top" width="36.16361636163616%" headers="mcps1.3.2.3.2.5.1.3 "><ul id="dli_01_0504__dli_07_0027_ul17151840141712"><li id="dli_01_0504__dli_07_0027_li4716114031710">High reliability and availability are not supported.</li><li id="dli_01_0504__dli_07_0027_li671612406179">Queue properties and job priorities cannot be set.</li><li id="dli_01_0504__dli_07_0027_li819982791611">Notebook instances cannot be interconnected with.</li></ul> <td class="cellrowborder" valign="top" width="36.16361636163616%" headers="mcps1.3.2.3.2.5.1.3 "><ul id="dli_01_0504__dli_07_0027_ul17151840141712"><li id="dli_01_0504__dli_07_0027_li4716114031710">High reliability and availability are not supported.</li><li id="dli_01_0504__dli_07_0027_li671612406179">Queue properties and job priorities cannot be set.</li><li id="dli_01_0504__dli_07_0027_li819982791611">Notebook instances cannot be interconnected with.</li></ul>
<p id="dli_01_0504__p75015239367">For more constraints and limitations on elastic resource pools, see <a href="#dli_01_0504__section1929612253596">Constraints</a>.</p> <p id="dli_01_0504__p75015239367">For more notes and constraints on elastic resource pools, see <a href="#dli_01_0504__section1929612253596">Notes and Constraints</a>.</p>
</td> </td>
<td class="cellrowborder" valign="top" width="33.1033103310331%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dli_01_0504__dli_07_0027_p39154491111">This edition is suitable for testing scenarios with low resource consumption and low requirements for resource reliability and availability.</p> <td class="cellrowborder" valign="top" width="33.1033103310331%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dli_01_0504__dli_07_0027_p39154491111">This edition is suitable for testing scenarios with low resource consumption and low requirements for resource reliability and availability.</p>
</td> </td>
@ -30,7 +30,7 @@
</td> </td>
<td class="cellrowborder" valign="top" width="15.001500150015001%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dli_01_0504__dli_07_0027_p2915194931116">64 CUs or higher</p> <td class="cellrowborder" valign="top" width="15.001500150015001%" headers="mcps1.3.2.3.2.5.1.2 "><p id="dli_01_0504__dli_07_0027_p2915194931116">64 CUs or higher</p>
</td> </td>
<td class="cellrowborder" valign="top" width="36.16361636163616%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dli_01_0504__p0358194918357">For more constraints and limitations on elastic resource pools, see <a href="#dli_01_0504__section1929612253596">Constraints</a>.</p> <td class="cellrowborder" valign="top" width="36.16361636163616%" headers="mcps1.3.2.3.2.5.1.3 "><p id="dli_01_0504__p0358194918357">For more notes and constraints on elastic resource pools, see <a href="#dli_01_0504__section1929612253596">Notes and Constraints</a>.</p>
</td> </td>
<td class="cellrowborder" valign="top" width="33.1033103310331%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dli_01_0504__dli_07_0027_p6915124931114">This edition offers powerful computing capabilities, high availability, and flexible resource management. It is suitable for large-scale computing tasks and business scenarios with long-term resource planning needs.</p> <td class="cellrowborder" valign="top" width="33.1033103310331%" headers="mcps1.3.2.3.2.5.1.4 "><p id="dli_01_0504__dli_07_0027_p6915124931114">This edition offers powerful computing capabilities, high availability, and flexible resource management. It is suitable for large-scale computing tasks and business scenarios with long-term resource planning needs.</p>
</td> </td>
@ -39,8 +39,13 @@
</table> </table>
</div> </div>
</div> </div>
<div class="section" id="dli_01_0504__section1929612253596"><a name="dli_01_0504__section1929612253596"></a><a name="section1929612253596"></a><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0504__ul1797121511815"><li id="dli_01_0504__dli_07_0005_li3628552124310">The region of an elastic resource pool cannot be changed.</li><li id="dli_01_0504__dli_07_0005_li209401414121714">Jobs of Flink 1.10 or later can run in elastic resource pools.</li><li id="dli_01_0504__dli_07_0005_li197515151815">The network segment of an elastic resource pool cannot be changed after being set.</li><li id="dli_01_0504__dli_07_0005_li480622642113">You can only view the scaling history of resource pools in the last 30 days.</li><li id="dli_01_0504__dli_07_0005_li77421446307">Elastic resource pools cannot access the Internet.<p id="dli_01_0504__dli_07_0005_p7664134817018"><a name="dli_01_0504__dli_07_0005_li77421446307"></a><a name="dli_07_0005_li77421446307"></a></p> <div class="section" id="dli_01_0504__section1929612253596"><a name="dli_01_0504__section1929612253596"></a><a name="section1929612253596"></a><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_01_0504__ul1797121511815"><li id="dli_01_0504__dli_07_0005_li3628552124310">The region of an elastic resource pool cannot be changed.</li><li id="dli_01_0504__dli_07_0005_li209401414121714">Jobs of Flink 1.10 or later can run in elastic resource pools.</li><li id="dli_01_0504__dli_07_0005_li197515151815">The network segment of an elastic resource pool cannot be changed after being set.</li><li id="dli_01_0504__dli_07_0005_li480622642113">You can only view the scaling history of resource pools in the last 30 days.</li><li id="dli_01_0504__dli_07_0005_li77421446307">Elastic resource pools cannot access the Internet.<p id="dli_01_0504__dli_07_0005_p7664134817018"><a name="dli_01_0504__dli_07_0005_li77421446307"></a><a name="dli_07_0005_li77421446307"></a></p>
<p id="dli_01_0504__dli_07_0005_p26641483016"></p> <p id="dli_01_0504__dli_07_0005_p26641483016"></p>
</li><li id="dli_01_0504__dli_07_0005_li06624212125">Changes to elastic resource pool CUs can occur when setting the CU, adding or deleting queues in an elastic resource pool, or modifying the scaling policies of queues in an elastic resource pool, or when the system automatically triggers elastic resource pool scaling. However, in some cases, the system cannot guarantee that the scaling will reach the target CUs as planned.<ul id="dli_01_0504__dli_07_0005_ul463510104131"><li id="dli_01_0504__dli_07_0005_li198241284132">If there are not enough physical resources, an elastic resource pool may not be able to scale out to the desired target size.</li><li id="dli_01_0504__dli_07_0005_li1137572261420">The system does not guarantee that an elastic resource pool will be scaled in to the desired target size.<p id="dli_01_0504__dli_07_0005_p79536261513"><a name="dli_01_0504__dli_07_0005_li1137572261420"></a><a name="dli_07_0005_li1137572261420"></a>The system checks the resource usage before scaling in the elastic resource pool to determine if there is enough space for scaling in. If the existing resources cannot be scaled in according to the minimum scaling step, the pool may not be scaled in successfully or only partially.</p>
<p id="dli_01_0504__dli_07_0005_p755525181519">The scaling step may vary depending on the resource specifications, usually 16 CUs, 32 CUs, 48 CUs, 64 CUs, etc.</p>
<p id="dli_01_0504__dli_07_0005_p148652742019">For example, if the elastic resource pool has a capacity of 192 CUs and the queues in the pool are using 68 CUs due to running jobs, the plan is to scale in to 64 CUs.</p>
<p id="dli_01_0504__dli_07_0005_p515442321417">When executing a scaling in task, the system determines that there are 124 CUs remaining and scales in by the minimum step of 64 CUs. However, the remaining 60 CUs cannot be scaled in any further. Therefore, after the elastic resource pool executes the scaling in task, its capacity is reduced to 128 CUs.</p>
</li></ul>
</li></ul> </li></ul>
</div> </div>
<div class="section" id="dli_01_0504__section1410181163918"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0504__p86721558536"><strong id="dli_01_0504__b19573880544">Resources too fixed to meet a range of requirements.</strong></p> <div class="section" id="dli_01_0504__section1410181163918"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0504__p86721558536"><strong id="dli_01_0504__b19573880544">Resources too fixed to meet a range of requirements.</strong></p>
@ -71,11 +76,11 @@
</th> </th>
</tr> </tr>
</thead> </thead>
<tbody><tr id="dli_01_0504__row546773765617"><td class="cellrowborder" valign="top" width="8.89%" headers="mcps1.3.5.6.1.4.1.1 "><p id="dli_01_0504__p931417598126">Efficiency</p> <tbody><tr id="dli_01_0504__row546773765617"><td class="cellrowborder" valign="top" width="8.89%" headers="mcps1.3.5.6.1.4.1.1 "><p id="dli_01_0504__p931417598126">Scale-out duration</p>
</td> </td>
<td class="cellrowborder" valign="top" width="40.29%" headers="mcps1.3.5.6.1.4.1.2 "><p id="dli_01_0504__p14467193710561">You need to set scaling tasks repeatedly to improve the resource utilization.</p> <td class="cellrowborder" valign="top" width="40.29%" headers="mcps1.3.5.6.1.4.1.2 "><p id="dli_01_0504__p14467193710561">You will need to spend several minutes manually scaling out.</p>
</td> </td>
<td class="cellrowborder" valign="top" width="50.82%" headers="mcps1.3.5.6.1.4.1.3 "><p id="dli_01_0504__p74679377564">Dynamic scaling can be done in seconds.</p> <td class="cellrowborder" valign="top" width="50.82%" headers="mcps1.3.5.6.1.4.1.3 "><p id="dli_01_0504__p74679377564">No manual intervention is required, as dynamic scale out can be done in seconds.</p>
</td> </td>
</tr> </tr>
<tr id="dli_01_0504__row10467637105611"><td class="cellrowborder" rowspan="2" valign="top" width="8.89%" headers="mcps1.3.5.6.1.4.1.1 "><p id="dli_01_0504__p631415913126">Resource utilization</p> <tr id="dli_01_0504__row10467637105611"><td class="cellrowborder" rowspan="2" valign="top" width="8.89%" headers="mcps1.3.5.6.1.4.1.1 "><p id="dli_01_0504__p631415913126">Resource utilization</p>

5
docs/dli/umn/dli_01_0524.html
View File

@ -1,10 +1,7 @@
<a name="dli_01_0524"></a><a name="dli_01_0524"></a> <a name="dli_01_0524"></a><a name="dli_01_0524"></a>
<h1 class="topictitle1">Modifying Specifications</h1> <h1 class="topictitle1">Modifying Specifications</h1>
<div id="body0000001261847604"><div class="section" id="dli_01_0524__section196121847151611"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0524__p49521211151713">If CUs of a yearly/monthly elastic resource pool cannot meet your service requirements, you can modify the CUs. In this case, you will be charged based on the number of CUs exceeding that of the yearly/monthly elastic resource pool.</p> <div id="body0000001261847604"><div class="section" id="dli_01_0524__section196121847151611"><h4 class="sectiontitle">Scenario</h4><p id="dli_01_0524__p391142161516">If the current specifications of your elastic resource pool do not meet your service needs, you can modify them using the change specifications function.</p>
<p id="dli_01_0524__p13221171882317">For example, you have purchased an elastic resource pool with 64 CUs, and you find that most time data processing needs 128 CUs. You can add 64 CUs to the elastic resource pool and be billed based on a CU/hour basis. To save more, you can scale up your elastic resource pool to 128 CUs and be billed on a yearly/monthly basis for the 128-CU package.</p>
</div>
<div class="section" id="dli_01_0524__section124281747104415"><h4 class="sectiontitle">Precautions</h4><p id="dli_01_0524__p14189112213380">Currently, only yearly/monthly elastic resource pools can be scaled.</p>
</div> </div>
<div class="section" id="dli_01_0524__section1889631562717"><h4 class="sectiontitle">Scaling Up</h4><ol id="dli_01_0524__ol2891331122818"><li id="dli_01_0524__li1891031122813">In the navigation pane on the left of the console, choose <strong id="dli_01_0524__b1234482712510">Resources</strong> &gt; <strong id="dli_01_0524__b534532717259">Resource Pool</strong>.</li><li id="dli_01_0524__li1711818752915">Select the elastic resource pool you want and choose <strong id="dli_01_0524__b84851766201">More</strong> &gt; <strong id="dli_01_0524__b04851766203">Modify Specifications</strong> in the <span class="parmname" id="dli_01_0524__parmname12569421191"><b>Operation</b></span> column.</li><li id="dli_01_0524__li2021124816302">In the <span class="wintitle" id="dli_01_0524__wintitle397793417340"><b>Modify Specifications</b></span> dialog page, set <span class="parmname" id="dli_01_0524__parmname1857164912311"><b>Operation</b></span> to <strong id="dli_01_0524__b541719202415">Scale-out</strong> and specify the number of CUs you want to add.</li><li id="dli_01_0524__li9522173719279">Confirm the changes and click <strong id="dli_01_0524__b71971933152818">OK</strong>.</li><li id="dli_01_0524__li136261555204917">Choose <strong id="dli_01_0524__b1059915416271">Job Management</strong> &gt; <strong id="dli_01_0524__b1275920917275">SQL Jobs</strong> to view the status of the SCALE_POOL SQL job.<p id="dli_01_0524__p923895617495">If the job status is <strong id="dli_01_0524__b14981917192816">Scaling</strong>, the elastic resource pool is scaling up. Wait until the job status changes to <strong id="dli_01_0524__b19231750182817">Finished</strong>.</p> <div class="section" id="dli_01_0524__section1889631562717"><h4 class="sectiontitle">Scaling Up</h4><ol id="dli_01_0524__ol2891331122818"><li id="dli_01_0524__li1891031122813">In the navigation pane on the left of the console, choose <strong id="dli_01_0524__b1234482712510">Resources</strong> &gt; <strong id="dli_01_0524__b534532717259">Resource Pool</strong>.</li><li id="dli_01_0524__li1711818752915">Select the elastic resource pool you want and choose <strong id="dli_01_0524__b84851766201">More</strong> &gt; <strong id="dli_01_0524__b04851766203">Modify Specifications</strong> in the <span class="parmname" id="dli_01_0524__parmname12569421191"><b>Operation</b></span> column.</li><li id="dli_01_0524__li2021124816302">In the <span class="wintitle" id="dli_01_0524__wintitle397793417340"><b>Modify Specifications</b></span> dialog page, set <span class="parmname" id="dli_01_0524__parmname1857164912311"><b>Operation</b></span> to <strong id="dli_01_0524__b541719202415">Scale-out</strong> and specify the number of CUs you want to add.</li><li id="dli_01_0524__li9522173719279">Confirm the changes and click <strong id="dli_01_0524__b71971933152818">OK</strong>.</li><li id="dli_01_0524__li136261555204917">Choose <strong id="dli_01_0524__b1059915416271">Job Management</strong> &gt; <strong id="dli_01_0524__b1275920917275">SQL Jobs</strong> to view the status of the SCALE_POOL SQL job.<p id="dli_01_0524__p923895617495">If the job status is <strong id="dli_01_0524__b14981917192816">Scaling</strong>, the elastic resource pool is scaling up. Wait until the job status changes to <strong id="dli_01_0524__b19231750182817">Finished</strong>.</p>
</li></ol> </li></ol>

6
docs/dli/umn/dli_01_0531.html
View File

@ -14,7 +14,7 @@
</div> </div>
<div class="section" id="dli_01_0531__en-us_topic_0000001354966081_section1133722963618"><a name="dli_01_0531__en-us_topic_0000001354966081_section1133722963618"></a><a name="en-us_topic_0000001354966081_section1133722963618"></a><h4 class="sectiontitle">Step 1: Prepare a Data Source</h4><p id="dli_01_0531__p186811205118">In this example, Kafka is the data source.</p> <div class="section" id="dli_01_0531__en-us_topic_0000001354966081_section1133722963618"><a name="dli_01_0531__en-us_topic_0000001354966081_section1133722963618"></a><a name="en-us_topic_0000001354966081_section1133722963618"></a><h4 class="sectiontitle">Step 1: Prepare a Data Source</h4><p id="dli_01_0531__p186811205118">In this example, Kafka is the data source.</p>
<p id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_p139881257176">For more information about Flink job data, see <a href="dli_01_0454.html">Preparing Flink Job Data</a>.</p> <p id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_p139881257176">For more information about Flink job data, see <a href="dli_01_0454.html">Preparing Flink Job Data</a>.</p>
<p id="dli_01_0531__p1398817179114">Enable DIS to import Kafka data to DLI. For details, see "Buying a Kafka Instance" in the <em id="dli_01_0531__i1843863474017">Distributed Message Service Kafka User Guide</em>.</p> <p id="dli_01_0531__p1398817179114">Enable DIS to import Kafka data to DLI. For details, see "Buying a Kafka Instance" in <em id="dli_01_0531__i17891192394316">Distributed Message Service Kafka User Guide</em>.</p>
<ol id="dli_01_0531__en-us_topic_0000001354966081_ol1463517113371"><li id="dli_01_0531__en-us_topic_0000001354966081_li485218325375"><a name="dli_01_0531__en-us_topic_0000001354966081_li485218325375"></a><a name="en-us_topic_0000001354966081_li485218325375"></a>Create the dependent Kafka resources.<div class="p" id="dli_01_0531__en-us_topic_0000001354966081_p16367638193718"><a name="dli_01_0531__en-us_topic_0000001354966081_li485218325375"></a><a name="en-us_topic_0000001354966081_li485218325375"></a>Before creating a Kafka instance, ensure the availability of resources, including a virtual private cloud (VPC), subnet, security group, and security group rules.<ul id="dli_01_0531__en-us_topic_0000001354966081_ul13635171183719"><li id="dli_01_0531__en-us_topic_0000001354966081_li863518116373">For details about how to create a VPC and subnet, see "Creating a VPC and Subnet" in <em id="dli_01_0531__i76131515133212">Virtual Private Cloud User Guide</em>. For details about how to create and use a subnet in an existing VPC, see "Create a Subnet for the VPC" in <em id="dli_01_0531__i156171215103218">Virtual Private Cloud User Guide</em>.<div class="note" id="dli_01_0531__en-us_topic_0000001354966081_note18635411193717"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0531__en-us_topic_0000001354966081_ul18635211103710"><li id="dli_01_0531__en-us_topic_0000001354966081_li863561118371">The created VPC and the Kafka instance you will create must be in the same region.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li1635811133713">Retain the default settings unless otherwise specified.</li></ul> <ol id="dli_01_0531__en-us_topic_0000001354966081_ol1463517113371"><li id="dli_01_0531__en-us_topic_0000001354966081_li485218325375"><a name="dli_01_0531__en-us_topic_0000001354966081_li485218325375"></a><a name="en-us_topic_0000001354966081_li485218325375"></a>Create the dependent Kafka resources.<div class="p" id="dli_01_0531__en-us_topic_0000001354966081_p16367638193718"><a name="dli_01_0531__en-us_topic_0000001354966081_li485218325375"></a><a name="en-us_topic_0000001354966081_li485218325375"></a>Before creating a Kafka instance, ensure the availability of resources, including a virtual private cloud (VPC), subnet, security group, and security group rules.<ul id="dli_01_0531__en-us_topic_0000001354966081_ul13635171183719"><li id="dli_01_0531__en-us_topic_0000001354966081_li863518116373">For details about how to create a VPC and subnet, see "Creating a VPC and Subnet" in <em id="dli_01_0531__i76131515133212">Virtual Private Cloud User Guide</em>. For details about how to create and use a subnet in an existing VPC, see "Create a Subnet for the VPC" in <em id="dli_01_0531__i156171215103218">Virtual Private Cloud User Guide</em>.<div class="note" id="dli_01_0531__en-us_topic_0000001354966081_note18635411193717"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_01_0531__en-us_topic_0000001354966081_ul18635211103710"><li id="dli_01_0531__en-us_topic_0000001354966081_li863561118371">The created VPC and the Kafka instance you will create must be in the same region.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li1635811133713">Retain the default settings unless otherwise specified.</li></ul>
</div></div> </div></div>
</li><li id="dli_01_0531__en-us_topic_0000001354966081_li26353112371">For details about how to create a security group, see "Creating a Security Group" in the <em id="dli_01_0531__i1846865413317">Virtual Private Cloud User Guide</em>. For details about how to add rules to a security group, see "Creating a Subnet for the VPC" in the <em id="dli_01_0531__i1646913547335">Virtual Private Cloud User Guide</em>.</li></ul> </li><li id="dli_01_0531__en-us_topic_0000001354966081_li26353112371">For details about how to create a security group, see "Creating a Security Group" in the <em id="dli_01_0531__i1846865413317">Virtual Private Cloud User Guide</em>. For details about how to add rules to a security group, see "Creating a Subnet for the VPC" in the <em id="dli_01_0531__i1646913547335">Virtual Private Cloud User Guide</em>.</li></ul>
@ -29,7 +29,7 @@
</div> </div>
<div class="section" id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_section10891114913473"><a name="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_section10891114913473"></a><a name="en-us_topic_0000001354966081_dli_01_0481_section10891114913473"></a><h4 class="sectiontitle">Step 2: Prepare a Data Output Channel</h4><p id="dli_01_0531__en-us_topic_0000001354966081_p17169195217173">To use RDS as the data output channel, create an RDS MySQL instance. For details, see "Getting Started with RDS for MySQL" in <em id="dli_01_0531__i1625782415572">Getting Started with Relational Database Service</em>.</p> <div class="section" id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_section10891114913473"><a name="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_section10891114913473"></a><a name="en-us_topic_0000001354966081_dli_01_0481_section10891114913473"></a><h4 class="sectiontitle">Step 2: Prepare a Data Output Channel</h4><p id="dli_01_0531__en-us_topic_0000001354966081_p17169195217173">To use RDS as the data output channel, create an RDS MySQL instance. For details, see "Getting Started with RDS for MySQL" in <em id="dli_01_0531__i1625782415572">Getting Started with Relational Database Service</em>.</p>
<ol id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_ol5293104072113"><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li15294154032113">Log in to the RDS management console.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li3294104014219">Select a region in the upper left corner.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li0294104042117">Click <strong id="dli_01_0531__b81987177166">Buy DB Instance</strong> in the upper right corner of the page and set related parameters. Retain the default values for other parameters.<ul id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_ul179404349231"><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li3940143492310"><strong id="dli_01_0531__b476475041105136">Region</strong>: Select the region where DLI is located.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li1940034172317"><strong id="dli_01_0531__b467917245818">DB Instance Name</strong>: Enter <strong id="dli_01_0531__b187988281783">rds-dliflink</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li15941193412237"><strong id="dli_01_0531__b303305023105136">DB Engine</strong>: Select <strong id="dli_01_0531__b17505175010816">MySQL</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li8941203418239"><strong id="dli_01_0531__b367749497105136">DB Engine Version</strong>: Select <strong id="dli_01_0531__b18238681396">8.0</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li1994143442313"><strong id="dli_01_0531__b1927277961105136">DB Instance Type</strong>: Select <strong id="dli_01_0531__b101808426915">Primary/Standby</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li11941193417239"><strong id="dli_01_0531__b1334982692105136">Storage Type</strong>: Cloud SSD may be selected by default.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li179411434142311"><strong id="dli_01_0531__b20589112841111">Primary AZ</strong>: Select a custom AZ.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li2751547193914"><strong id="dli_01_0531__b0751112411124">Standby AZ</strong>: Select a custom AZ.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li14423141419338"><strong id="dli_01_0531__b15537175420244">Instance Class</strong>: Select a class as needed and choose <strong id="dli_01_0531__b126514431247">2 vCPUs | 8 GB</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li1697112162336"><strong id="dli_01_0531__b94450916254">Storage Space (GB)</strong>: Set it to <strong id="dli_01_0531__b623593117257">40</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li5835826103312"><strong id="dli_01_0531__b92193718266">VPC</strong>: Select the VPC and subnet created in <a href="#dli_01_0531__en-us_topic_0000001354966081_li485218325375">1</a>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li658213312336"><strong id="dli_01_0531__b48276287267">Database Port</strong>: Enter <strong id="dli_01_0531__b11843103719266">3306</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li175371539103311"><strong id="dli_01_0531__b1035302503105136">Security Group</strong>: Select the security group created in <a href="#dli_01_0531__en-us_topic_0000001354966081_li485218325375">1</a>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li68642415331"><strong id="dli_01_0531__b6293317192716">Administrator Password</strong>: **** (Keep the password secure. The system cannot retrieve your password.)</li><li id="dli_01_0531__en-us_topic_0000001354966081_li272097184320"><strong id="dli_01_0531__b351057194105136">Confirm Password</strong>: ****</li><li id="dli_01_0531__en-us_topic_0000001354966081_li147665053320"><strong id="dli_01_0531__b064491112812">Parameter Template</strong>: Choose <strong id="dli_01_0531__b15420111010282">Default-MySQL-8.0</strong>.</li></ul> <ol id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_ol5293104072113"><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li15294154032113">Log in to the RDS management console.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li3294104014219">Select a region in the upper left corner.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li0294104042117">Click <strong id="dli_01_0531__b81987177166">Buy DB Instance</strong> in the upper right corner of the page and set related parameters. Retain the default values for other parameters.<ul id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_ul179404349231"><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li3940143492310"><strong id="dli_01_0531__b476475041105136">Region</strong>: Select the region where DLI is located.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li1940034172317"><strong id="dli_01_0531__b467917245818">DB Instance Name</strong>: Enter <strong id="dli_01_0531__b187988281783">rds-dliflink</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li15941193412237"><strong id="dli_01_0531__b303305023105136">DB Engine</strong>: Select <strong id="dli_01_0531__b17505175010816">MySQL</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li8941203418239"><strong id="dli_01_0531__b367749497105136">DB Engine Version</strong>: Select <strong id="dli_01_0531__b18238681396">8.0</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li1994143442313"><strong id="dli_01_0531__b1927277961105136">DB Instance Type</strong>: Select <strong id="dli_01_0531__b101808426915">Primary/Standby</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li11941193417239"><strong id="dli_01_0531__b1334982692105136">Storage Type</strong>: Cloud SSD may be selected by default.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li179411434142311"><strong id="dli_01_0531__b20589112841111">Primary AZ</strong>: Select a custom AZ.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li2751547193914"><strong id="dli_01_0531__b0751112411124">Standby AZ</strong>: Select a custom AZ.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li14423141419338"><strong id="dli_01_0531__b15537175420244">Instance Class</strong>: Select a class as needed and choose <strong id="dli_01_0531__b126514431247">2 vCPUs | 8 GB</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li1697112162336"><strong id="dli_01_0531__b94450916254">Storage Space (GB)</strong>: Set it to <strong id="dli_01_0531__b623593117257">40</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li5835826103312"><strong id="dli_01_0531__b92193718266">VPC</strong>: Select the VPC and subnet created in <a href="#dli_01_0531__en-us_topic_0000001354966081_li485218325375">1</a>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li658213312336"><strong id="dli_01_0531__b48276287267">Database Port</strong>: Enter <strong id="dli_01_0531__b11843103719266">3306</strong>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li175371539103311"><strong id="dli_01_0531__b1035302503105136">Security Group</strong>: Select the security group created in <a href="#dli_01_0531__en-us_topic_0000001354966081_li485218325375">1</a>.</li><li id="dli_01_0531__en-us_topic_0000001354966081_li68642415331"><strong id="dli_01_0531__b6293317192716">Administrator Password</strong>: **** (Keep the password secure. The system cannot retrieve your password.)</li><li id="dli_01_0531__en-us_topic_0000001354966081_li272097184320"><strong id="dli_01_0531__b351057194105136">Confirm Password</strong>: ****</li><li id="dli_01_0531__en-us_topic_0000001354966081_li147665053320"><strong id="dli_01_0531__b064491112812">Parameter Template</strong>: Choose <strong id="dli_01_0531__b15420111010282">Default-MySQL-8.0</strong>.</li></ul>
</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li1594832472410">Click <strong id="dli_01_0531__b112134191295">Next</strong> and confirm the specifications.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li094892411245">Click <strong id="dli_01_0531__b6265172510312">Submit</strong>. The RDS DB instance is created.</li><li id="dli_01_0531__li34158359532">Log in to the MySQL database and create table <strong id="dli_01_0531__b1729043326105136">orders</strong> in database <strong id="dli_01_0531__b1988609275105136">flink</strong>.<div class="p" id="dli_01_0531__p1989914125615">Log in to the MySQL instance, click the <strong id="dli_01_0531__b1538985311130">flink</strong> database. On the displayed page, click <strong id="dli_01_0531__b11741965713">SQL Window</strong>. Enter the following table creation statement in the SQL editing pane to create a table.<pre class="screen" id="dli_01_0531__screen135671111611">CREATE TABLE `flink`.`orders` ( </li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li1594832472410">Click <strong id="dli_01_0531__b112134191295">Next</strong> and confirm the specifications.</li><li id="dli_01_0531__en-us_topic_0000001354966081_dli_01_0481_li094892411245">Click <strong id="dli_01_0531__b6265172510312">Submit</strong>. The RDS DB instance is created.</li><li id="dli_01_0531__li34158359532">Log in to the MySQL database and create table <strong id="dli_01_0531__b1636061443318">orders</strong> in database <strong id="dli_01_0531__b17360181453313">flink</strong>.<div class="p" id="dli_01_0531__p1989914125615">Log in to the MySQL instance, click the <strong id="dli_01_0531__b1538985311130">flink</strong> database. On the displayed page, click <strong id="dli_01_0531__b11741965713">SQL Window</strong>. Enter the following table creation statement in the SQL editing pane to create a table.<pre class="screen" id="dli_01_0531__screen135671111611">CREATE TABLE `flink`.`orders` (
`order_id` VARCHAR(32) NOT NULL, `order_id` VARCHAR(32) NOT NULL,
`order_channel` VARCHAR(32) NULL, `order_channel` VARCHAR(32) NULL,
`order_time` VARCHAR(32) NULL, `order_time` VARCHAR(32) NULL,
@ -124,7 +124,7 @@ insert into jdbcSink select * from kafkaSource;</pre>
</li><li id="dli_01_0531__li04161435115319">Connect to the Kafka cluster and send the following test data to the Kafka topics:<pre class="screen" id="dli_01_0531__screen7221125915147">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"} </li><li id="dli_01_0531__li04161435115319">Connect to the Kafka cluster and send the following test data to the Kafka topics:<pre class="screen" id="dli_01_0531__screen7221125915147">{"order_id":"202103241000000001", "order_channel":"webShop", "order_time":"2021-03-24 10:00:00", "pay_amount":"100.00", "real_pay":"100.00", "pay_time":"2021-03-24 10:02:03", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}
{"order_id":"202103241606060001", "order_channel":"appShop", "order_time":"2021-03-24 16:06:06", "pay_amount":"200.00", "real_pay":"180.00", "pay_time":"2021-03-24 16:10:06", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre> {"order_id":"202103241606060001", "order_channel":"appShop", "order_time":"2021-03-24 16:06:06", "pay_amount":"200.00", "real_pay":"180.00", "pay_time":"2021-03-24 16:10:06", "user_id":"0001", "user_name":"Alice", "area_id":"330106"}</pre>
</li><li id="dli_01_0531__li17537162012111">Run the following SQL statement in the MySQL database to view data in the table:<pre class="screen" id="dli_01_0531__screen3359228654">select * from order;</pre> </li><li id="dli_01_0531__li17537162012111">Run the following SQL statement in the MySQL database to view data in the table:<pre class="screen" id="dli_01_0531__screen3359228654">select * from orders;</pre>
<div class="p" id="dli_01_0531__p1533922214118">The following is an example of the execution result copied from the MySQL database:<pre class="screen" id="dli_01_0531__screen1612710234445">202103241000000001,webShop,2021-03-24 10:00:00,100.0,100.0,2021-03-24 10:02:03,0001,Alice,330106 <div class="p" id="dli_01_0531__p1533922214118">The following is an example of the execution result copied from the MySQL database:<pre class="screen" id="dli_01_0531__screen1612710234445">202103241000000001,webShop,2021-03-24 10:00:00,100.0,100.0,2021-03-24 10:02:03,0001,Alice,330106
202103241606060001,appShop,2021-03-24 16:06:06,200.0,180.0,2021-03-24 16:10:06,0001,Alice,330106</pre> 202103241606060001,appShop,2021-03-24 16:06:06,200.0,180.0,2021-03-24 16:10:06,0001,Alice,330106</pre>
</div> </div>

2
docs/dli/umn/dli_01_0538.html
View File

@ -14,7 +14,7 @@
</li> </li>
<li class="ulchildlink"><strong><a href="dli_07_0002.html">Application Scenarios</a></strong><br> <li class="ulchildlink"><strong><a href="dli_07_0002.html">Application Scenarios</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_07_0005.html">Constraints and Limitations</a></strong><br> <li class="ulchildlink"><strong><a href="dli_07_0005.html">Notes and Constraints</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_07_0006.html">Permissions Management</a></strong><br> <li class="ulchildlink"><strong><a href="dli_07_0006.html">Permissions Management</a></strong><br>
</li> </li>

2
docs/dli/umn/dli_01_0552.html
View File

@ -53,7 +53,7 @@
<tbody><tr id="dli_01_0552__row153618208396"><td class="cellrowborder" valign="top" width="16.35%" headers="mcps1.3.3.2.5.2.1.2.2.3.1.1 "><p id="dli_01_0552__p1379843245517">Tag key</p> <tbody><tr id="dli_01_0552__row153618208396"><td class="cellrowborder" valign="top" width="16.35%" headers="mcps1.3.3.2.5.2.1.2.2.3.1.1 "><p id="dli_01_0552__p1379843245517">Tag key</p>
</td> </td>
<td class="cellrowborder" valign="top" width="83.65%" headers="mcps1.3.3.2.5.2.1.2.2.3.1.2 "><p id="dli_01_0552__p579853215514">You can specify the tag key in either of the following ways:</p> <td class="cellrowborder" valign="top" width="83.65%" headers="mcps1.3.3.2.5.2.1.2.2.3.1.2 "><p id="dli_01_0552__p579853215514">You can specify the tag key in either of the following ways:</p>
<ul id="dli_01_0552__ul167988328555"><li id="dli_01_0552__li177982322556">Click the text box for tag key and select a predefined tag key from the drop-down list.<p id="dli_01_0552__p1079818321559"><a name="dli_01_0552__li177982322556"></a><a name="li177982322556"></a>To add a predefined tag, you need to create one on TMS and then select it from the <span class="parmname" id="dli_01_0552__parmname851030852"><b>Tag key</b></span> drop-down list. You can click <strong id="dli_01_0552__b4491062884179">View predefined tags</strong> to go to the <strong id="dli_01_0552__b20231504734179">Predefined Tags</strong> page of the TMS console. Then, click <strong id="dli_01_0552__b19884236014179">Create Tag</strong> in the upper corner of the page to create a predefined tag.</p> <ul id="dli_01_0552__ul167988328555"><li id="dli_01_0552__li177982322556">Click the text box for tag key and select a predefined tag key from the drop-down list.<p id="dli_01_0552__p1079818321559"><a name="dli_01_0552__li177982322556"></a><a name="li177982322556"></a>To add a predefined tag, you need to create one on TMS and then select it from the <span class="parmname" id="dli_01_0552__parmname1686446939"><b>Tag key</b></span> drop-down list. You can click <strong id="dli_01_0552__b4491062884179">View predefined tags</strong> to go to the <strong id="dli_01_0552__b20231504734179">Predefined Tags</strong> page of the TMS console. Then, click <strong id="dli_01_0552__b19884236014179">Create Tag</strong> in the upper corner of the page to create a predefined tag.</p>
<p id="dli_01_0552__p17798632185518"></p> <p id="dli_01_0552__p17798632185518"></p>
</li></ul> </li></ul>
<ul id="dli_01_0552__ul19798133225511"><li id="dli_01_0552__li979816327556">Enter a tag key in the text box.<div class="note" id="dli_01_0552__note137981432165512"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0552__dli_01_0363_p1448010306508_1">A tag key can contain a maximum of 128 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0552__dli_01_0363_b544594615499_1">_sys_</strong>.</p> <ul id="dli_01_0552__ul19798133225511"><li id="dli_01_0552__li979816327556">Enter a tag key in the text box.<div class="note" id="dli_01_0552__note137981432165512"><span class="notetitle"> NOTE: </span><div class="notebody"><p id="dli_01_0552__dli_01_0363_p1448010306508_1">A tag key can contain a maximum of 128 characters. Only letters, digits, spaces, and special characters (_.:=+-@) are allowed, but the value cannot start or end with a space or start with <strong id="dli_01_0552__dli_01_0363_b544594615499_1">_sys_</strong>.</p>

6
docs/dli/umn/dli_01_0561.html
View File

@ -5,7 +5,7 @@
<ul id="dli_01_0561__ul156239498237"><li id="dli_01_0561__li126055019296">DEW is a comprehensive cloud-based encryption service that addresses data security, key security, and complex key management issues. You are advised to use DEW to store authentication information for data sources. </li><li id="dli_01_0561__li10624114917236">Datasource authentication is used to manage authentication information for accessing specified data sources. After datasource authentication is configured, you do not need to repeatedly configure data source authentication information in jobs, improving data source authentication security while enabling DLI to securely access data sources.</li></ul> <ul id="dli_01_0561__ul156239498237"><li id="dli_01_0561__li126055019296">DEW is a comprehensive cloud-based encryption service that addresses data security, key security, and complex key management issues. You are advised to use DEW to store authentication information for data sources. </li><li id="dli_01_0561__li10624114917236">Datasource authentication is used to manage authentication information for accessing specified data sources. After datasource authentication is configured, you do not need to repeatedly configure data source authentication information in jobs, improving data source authentication security while enabling DLI to securely access data sources.</li></ul>
<p id="dli_01_0561__p57041040182319">This section describes how to use datasource authentication provided by DLI.</p> <p id="dli_01_0561__p57041040182319">This section describes how to use datasource authentication provided by DLI.</p>
</div> </div>
<div class="section" id="dli_01_0561__section517414134610"><h4 class="sectiontitle">Constraints</h4><ul id="dli_01_0561__ul5202111174615"><li id="dli_01_0561__dli_07_0005_li5740129958">Only Spark SQL and Flink OpenSource SQL 1.12 jobs support datasource authentication.</li><li id="dli_01_0561__dli_07_0005_li154891441163">DLI supports four types of datasource authentication. Select an authentication type specific to each data source.<ul id="dli_01_0561__dli_07_0005_ul0871132418539"><li id="dli_01_0561__dli_07_0005_li620019356202">CSS: applies to 6.5.4 or later CSS clusters with the security mode enabled.</li><li id="dli_01_0561__dli_07_0005_li1048719385204">Kerberos: applies to MRS security clusters with Kerberos authentication enabled.</li><li id="dli_01_0561__dli_07_0005_li1738715443207">Kafka_SSL: applies to Kafka with SSL enabled.</li><li id="dli_01_0561__dli_07_0005_li11175781668">Password: applies to GaussDB(DWS), RDS, DDS, and DCS.</li></ul> <div class="section" id="dli_01_0561__section517414134610"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_01_0561__ul5202111174615"><li id="dli_01_0561__dli_07_0005_li5740129958">Only Spark SQL and Flink OpenSource SQL 1.12 jobs support datasource authentication.</li><li id="dli_01_0561__dli_07_0005_li154891441163">DLI supports four types of datasource authentication. Select an authentication type specific to each data source.<ul id="dli_01_0561__dli_07_0005_ul0871132418539"><li id="dli_01_0561__dli_07_0005_li620019356202">CSS: applies to 6.5.4 or later CSS clusters with the security mode enabled.</li><li id="dli_01_0561__dli_07_0005_li1048719385204">Kerberos: applies to MRS security clusters with Kerberos authentication enabled.</li><li id="dli_01_0561__dli_07_0005_li1738715443207">Kafka_SSL: applies to Kafka with SSL enabled.</li><li id="dli_01_0561__dli_07_0005_li11175781668">Password: applies to GaussDB(DWS), RDS, DDS, and DCS.</li></ul>
</li></ul> </li></ul>
</div> </div>
<div class="section" id="dli_01_0561__section654944414569"><h4 class="sectiontitle">Datasource Authentication Types</h4><p id="dli_01_0561__p2244194574619">DLI supports four types of datasource authentication. Select an authentication type specific to each data source.</p> <div class="section" id="dli_01_0561__section654944414569"><h4 class="sectiontitle">Datasource Authentication Types</h4><p id="dli_01_0561__p2244194574619">DLI supports four types of datasource authentication. Select an authentication type specific to each data source.</p>
@ -18,7 +18,7 @@
</th> </th>
<th align="left" class="cellrowborder" valign="top" width="32.08%" id="mcps1.3.4.4.2.4.1.2"><p id="dli_01_0561__p88116615010">Data Source</p> <th align="left" class="cellrowborder" valign="top" width="32.08%" id="mcps1.3.4.4.2.4.1.2"><p id="dli_01_0561__p88116615010">Data Source</p>
</th> </th>
<th align="left" class="cellrowborder" valign="top" width="45.739999999999995%" id="mcps1.3.4.4.2.4.1.3"><p id="dli_01_0561__p1629012551596">Constraints</p> <th align="left" class="cellrowborder" valign="top" width="45.739999999999995%" id="mcps1.3.4.4.2.4.1.3"><p id="dli_01_0561__p1629012551596">Notes and Constraints</p>
</th> </th>
</tr> </tr>
</thead> </thead>
@ -47,7 +47,7 @@
</th> </th>
<th align="left" class="cellrowborder" valign="top" width="22.220000000000002%" id="mcps1.3.4.5.2.5.1.3"><p id="dli_01_0561__p17800204533717">Data Source</p> <th align="left" class="cellrowborder" valign="top" width="22.220000000000002%" id="mcps1.3.4.5.2.5.1.3"><p id="dli_01_0561__p17800204533717">Data Source</p>
</th> </th>
<th align="left" class="cellrowborder" valign="top" width="47.38%" id="mcps1.3.4.5.2.5.1.4"><p id="dli_01_0561__p2800104553714">Constraints</p> <th align="left" class="cellrowborder" valign="top" width="47.38%" id="mcps1.3.4.5.2.5.1.4"><p id="dli_01_0561__p2800104553714">Notes and Constraints</p>
</th> </th>
</tr> </tr>
</thead> </thead>

2
docs/dli/umn/dli_01_0563.html
View File

@ -5,7 +5,7 @@
<p id="dli_01_0563__p1434334615514">You can set Spark driver parameters to improve the scheduling efficiency of queues.</p> <p id="dli_01_0563__p1434334615514">You can set Spark driver parameters to improve the scheduling efficiency of queues.</p>
<p id="dli_01_0563__p1692213243586">This section describes how to set queue properties on the management console.</p> <p id="dli_01_0563__p1692213243586">This section describes how to set queue properties on the management console.</p>
</div> </div>
<div class="section" id="dli_01_0563__section490110401914"><h4 class="sectiontitle">Constraints and Limitations</h4><ul id="dli_01_0563__ul37061018141013"><li id="dli_01_0563__li170616188108">Only SQL queues of the Spark engine support configuring queue properties.</li><li id="dli_01_0563__li670615186105">Setting queue properties is only supported after the queue has been created.</li><li id="dli_01_0563__li182034612484">Currently, only queue properties related to the Spark driver can be set.</li><li id="dli_01_0563__li137065182105">Queue properties cannot be set in batches.</li><li id="dli_01_0563__li692083533015">For a queue in an elastic resource pool, if the minimum CUs of the queue is less than 16 CUs, both <strong id="dli_01_0563__b2068144823111">Max. Spark Driver Instances</strong> and <strong id="dli_01_0563__b1011625353113">Max. Prestart Spark Driver Instances</strong> set in the queue properties do not apply.</li></ul> <div class="section" id="dli_01_0563__section490110401914"><h4 class="sectiontitle">Notes and Constraints</h4><ul id="dli_01_0563__ul37061018141013"><li id="dli_01_0563__li170616188108">Only SQL queues of the Spark engine support configuring queue properties.</li><li id="dli_01_0563__li670615186105">Setting queue properties is only supported after the queue has been created.</li><li id="dli_01_0563__li182034612484">Currently, only queue properties related to the Spark driver can be set.</li><li id="dli_01_0563__li137065182105">Queue properties cannot be set in batches.</li><li id="dli_01_0563__li692083533015">For a queue in an elastic resource pool, if the minimum CUs of the queue is less than 16 CUs, both <strong id="dli_01_0563__b2068144823111">Max. Spark Driver Instances</strong> and <strong id="dli_01_0563__b1011625353113">Max. Prestart Spark Driver Instances</strong> set in the queue properties do not apply.</li></ul>
</div> </div>
<div class="section" id="dli_01_0563__section6449141015018"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0563__ol538002619612"><li id="dli_01_0563__li11381226963">In the navigation pane of the DLI management console, choose <strong id="dli_01_0563__b122018814610">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0563__uicontrol22029813615"><b>Queue Management</b></span>.</li><li id="dli_01_0563__li1838119261568">In the <strong id="dli_01_0563__b1694313163614">Operation</strong> column of the queue, choose <strong id="dli_01_0563__b1994420161367">More</strong> &gt; <strong id="dli_01_0563__b2944116662">Set Property</strong>.</li><li id="dli_01_0563__li123811269617">Go to the queue property setting page and set property parameters. For details about the property parameters, see <a href="#dli_01_0563__table206971632142710">Table 1</a>. <div class="section" id="dli_01_0563__section6449141015018"><h4 class="sectiontitle">Procedure</h4><ol id="dli_01_0563__ol538002619612"><li id="dli_01_0563__li11381226963">In the navigation pane of the DLI management console, choose <strong id="dli_01_0563__b122018814610">Resources</strong> &gt; <span class="uicontrol" id="dli_01_0563__uicontrol22029813615"><b>Queue Management</b></span>.</li><li id="dli_01_0563__li1838119261568">In the <strong id="dli_01_0563__b1694313163614">Operation</strong> column of the queue, choose <strong id="dli_01_0563__b1994420161367">More</strong> &gt; <strong id="dli_01_0563__b2944116662">Set Property</strong>.</li><li id="dli_01_0563__li123811269617">Go to the queue property setting page and set property parameters. For details about the property parameters, see <a href="#dli_01_0563__table206971632142710">Table 1</a>.
<div class="tablenoborder"><a name="dli_01_0563__table206971632142710"></a><a name="table206971632142710"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0563__table206971632142710" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Queue properties</caption><thead align="left"><tr id="dli_01_0563__row9697123212276"><th align="left" class="cellrowborder" valign="top" width="27.312731273127312%" id="mcps1.3.3.2.3.2.2.4.1.1"><p id="dli_01_0563__p269713324274">Property</p> <div class="tablenoborder"><a name="dli_01_0563__table206971632142710"></a><a name="table206971632142710"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0563__table206971632142710" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Queue properties</caption><thead align="left"><tr id="dli_01_0563__row9697123212276"><th align="left" class="cellrowborder" valign="top" width="27.312731273127312%" id="mcps1.3.3.2.3.2.2.4.1.1"><p id="dli_01_0563__p269713324274">Property</p>

71
docs/dli/umn/dli_01_0624.html Normal file
View File

@ -0,0 +1,71 @@
<a name="dli_01_0624"></a><a name="dli_01_0624"></a>
<h1 class="topictitle1">Establishing a Network Connection Between DLI and Resources in a Shared VPC</h1>
<div id="body0000001891589656"><div class="section" id="dli_01_0624__section37181934438"><h4 class="sectiontitle">VPC Sharing Overview</h4><p id="dli_01_0624__p35220117436">VPC sharing allows sharing VPC resources created in one account with other accounts using Resource Access Manager (RAM). For example, account A can share its VPC and subnets with account B. After accepting the share, account B can view the shared VPC and subnets and use them to create resources.</p>
</div>
<div class="section" id="dli_01_0624__section116964284445"><h4 class="sectiontitle">DLI Use Cases</h4><p id="dli_01_0624__p71701315144619">An enterprise IT management account creates a VPC and subnets and shares them with other service accounts to facilitate centralized configuration of VPC security policies and orderly resource management.</p>
<p id="dli_01_0624__p11132824115416">Service accounts use the shared VPC and subnets to create resources and want to use DLI to submit jobs and access resources in the shared VPC. To do this, they need to establish a network connection between DLI and the resources in the shared VPC.</p>
<p id="dli_01_0624__p15128148195419">For example, account A is the enterprise IT management account and the owner of VPC resources. It creates the VPC and subnets and shares them with service account B.</p>
<p id="dli_01_0624__p933295645517">Account B is a service account that uses the shared VPC and subnets to create resources and uses DLI to access them.</p>
</div>
<div class="section" id="dli_01_0624__section11635139535"><h4 class="sectiontitle">Prerequisites</h4><p id="dli_01_0624__p104859375561">Account A, as the resource owner, has created a VPC and subnets and designated account B as the principal.</p>
</div>
<div class="section" id="dli_01_0624__section68241035155714"><h4 class="sectiontitle">Establishing a Network Connection Between DLI and Resources in a Shared VPC</h4><ol id="dli_01_0624__ol14756101414"><li id="dli_01_0624__li18199101878"><a name="dli_01_0624__li18199101878"></a><a name="li18199101878"></a><span>Account A creates an enhanced datasource connection.</span><p><ol type="a" id="dli_01_0624__ol1640216143711"><li id="dli_01_0624__li14475171018418">Log in to the DLI management console using account A.</li><li id="dli_01_0624__li16495421185119">In the navigation pane on the left, choose <strong id="dli_01_0624__b161022458319">Datasource Connections</strong>.</li><li id="dli_01_0624__li2495142110511">On the displayed <strong id="dli_01_0624__b1098545823112">Enhanced</strong> tab, click <strong id="dli_01_0624__b6985125853115">Create</strong>.<p id="dli_01_0624__p948710436532">Set parameters based on <a href="#dli_01_0624__table9495521165111">Table 1</a>.</p>
<div class="tablenoborder"><a name="dli_01_0624__table9495521165111"></a><a name="table9495521165111"></a><table cellpadding="4" cellspacing="0" summary="" id="dli_01_0624__table9495521165111" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameters for creating an enhanced datasource connection</caption><thead align="left"><tr id="dli_01_0624__row649562118511"><th align="left" class="cellrowborder" valign="top" width="21.22%" id="mcps1.3.4.2.1.2.1.3.4.2.3.1.1"><p id="dli_01_0624__p249592118517">Parameter</p>
</th>
<th align="left" class="cellrowborder" valign="top" width="78.78%" id="mcps1.3.4.2.1.2.1.3.4.2.3.1.2"><p id="dli_01_0624__p2495122112514">Description</p>
</th>
</tr>
</thead>
<tbody><tr id="dli_01_0624__row14496152114519"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0624__p3496102175119">Connection Name</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0624__p1249620215510">Name of the datasource connection to be created</p>
</td>
</tr>
<tr id="dli_01_0624__row24721288912"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0624__p94727281099">Resource Pool</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0624__p5472928792">You do not need to set this parameter in this scenario.</p>
</td>
</tr>
<tr id="dli_01_0624__row18496121115111"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0624__p1849632116516">VPC</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0624__p8496921145118">VPC shared by account A to account B</p>
</td>
</tr>
<tr id="dli_01_0624__row34961721195112"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0624__p184961221205112">Subnet</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0624__p3496321195112">Subnet shared by account A to account B</p>
</td>
</tr>
<tr id="dli_01_0624__row64971421195112"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0624__p8497132119511">Host Information</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0624__p38613346124">You do not need to set this parameter in this scenario.</p>
</td>
</tr>
<tr id="dli_01_0624__row949742175117"><td class="cellrowborder" valign="top" width="21.22%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.1 "><p id="dli_01_0624__p26171435204818">Tags</p>
</td>
<td class="cellrowborder" valign="top" width="78.78%" headers="mcps1.3.4.2.1.2.1.3.4.2.3.1.2 "><p id="dli_01_0624__p52841736202320">Tags used to identify cloud resources. A tag includes the tag key and tag value.</p>
</td>
</tr>
</tbody>
</table>
</div>
</li><li id="dli_01_0624__li849810219511">Click <strong id="dli_01_0624__b1409161112399">OK</strong>.</li></ol>
</p></li><li id="dli_01_0624__li691191021719"><span>Account A grants account B access to the enhanced datasource connection created in <a href="#dli_01_0624__li18199101878">1</a>.</span><p><ol type="a" id="dli_01_0624__ol197019321189"><li id="dli_01_0624__li47011932151811">In the enhanced datasource connection list, locate the row containing the newly created one, click <strong id="dli_01_0624__b427983914146">More</strong> in the <strong id="dli_01_0624__b878212320159">Operation</strong> column, and select <strong id="dli_01_0624__b1098711343152">Manage Permission</strong> from the drop-down list.</li><li id="dli_01_0624__li8701193291810">In the displayed <strong id="dli_01_0624__b125812036161617">Permissions</strong> dialog box, select <strong id="dli_01_0624__b73781427141618">Grant Permission</strong> for <strong id="dli_01_0624__b1296412167163">Set Permission</strong>, enter the ID of the project account B belongs to in <strong id="dli_01_0624__b191642916189">Project ID</strong>, and click <strong id="dli_01_0624__b891445351816">OK</strong>.</li></ol>
</p></li><li id="dli_01_0624__li4293102016196"><span>Account B binds a DLI elastic resource pool to the shared enhanced datasource connection.</span><p><ol type="a" id="dli_01_0624__ol910513308218"><li id="dli_01_0624__li17941558132014">Log in to the DLI management console using account B.</li><li id="dli_01_0624__li13981826102113">In the navigation pane on the left, choose <strong id="dli_01_0624__b1360245643111">Datasource Connections</strong>.</li><li id="dli_01_0624__li398172615216">On the displayed <strong id="dli_01_0624__b1633452619248">Enhanced</strong> tab, locate the row containing the enhanced datasource connection shared by account A, click <strong id="dli_01_0624__b5410103352310">More</strong> in the <strong id="dli_01_0624__b2410163332313">Operation</strong> column, and select <strong id="dli_01_0624__b1941183332312">Bind Resource Pool</strong> from the drop-down list.</li><li id="dli_01_0624__li16221113212239">In the displayed <strong id="dli_01_0624__b1050213163264">Bind Resource Pool</strong> dialog box, select the created elastic resource pool for <strong id="dli_01_0624__b1918419590269">Resource Pool</strong> and click <strong id="dli_01_0624__b1412819510273">OK</strong>.<p id="dli_01_0624__p19903529113515">If there is no elastic resource pool available, create one by referring to <a href="dli_01_0505.html">Creating an Elastic Resource Pool</a>.</p>
</li></ol>
</p></li><li id="dli_01_0624__li1560644944113"><span>Account B tests the network connectivity between the elastic resource pool and resources in the VPC.</span><p><div class="note" id="dli_01_0624__note194071044420"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_01_0624__p9940171054413">If there are resources in the shared VPC, ensure that the security group the resources belong to has allowed access to the elastic resource pool's CIDR block.</p>
</div></div>
<ol type="a" id="dli_01_0624__ol1811520304587"><li id="dli_01_0624__li276192795918">Obtain the private IP address and port number of the data source in the shared VPC.<p id="dli_01_0624__p1877211582594"><a name="dli_01_0624__li276192795918"></a><a name="li276192795918"></a>Take the RDS data source as an example. On the <strong id="dli_01_0624__b1782573293011">Instances</strong> page, click the target DB instance. On the displayed page, locate the <strong id="dli_01_0624__b188263328300">Connection Information</strong> pane and view the private IP address. In the <strong id="dli_01_0624__b128125533012">Connection Information</strong> pane, locate the <strong id="dli_01_0624__b1181195511300">Database Port</strong> to view the port number of the RDS DB instance.</p>
</li><li id="dli_01_0624__li1023215315019">In the navigation pane of the DLI management console, choose <strong id="dli_01_0624__b6168158310">Resources</strong> &gt; <strong id="dli_01_0624__b101681511318">Queue Management</strong>.</li><li id="dli_01_0624__li19115123085812">Locate the queue under the elastic resource pool bound with the enhanced datasource connection, click <strong id="dli_01_0624__b46962012183119">More</strong> in the <strong id="dli_01_0624__b8697112103112">Operation</strong> column, and select <strong id="dli_01_0624__b069701203118">Test Address Connectivity</strong>.</li><li id="dli_01_0624__li7983151711458">Enter the data source connection address and port number to test the network connectivity.<p id="dli_01_0624__p341321844514"><a name="dli_01_0624__li7983151711458"></a><a name="li7983151711458"></a>If the address is reachable, it means that account B has established a network connection between the DLI resource and the resources in the shared VPC. Account B can then submit jobs to the elastic resource pool's queue and access the resources in the shared VPC.</p>
</li></ol>
</p></li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_01_0426.html">Enhanced Datasource Connections</a></div>
</div>
</div>

11
docs/dli/umn/dli_03_0002.html
View File

@ -1,11 +0,0 @@
<a name="dli_03_0002"></a><a name="dli_03_0002"></a>
<h1 class="topictitle1">What Is DLI?</h1>
<div id="body8662426"><p id="dli_03_0002__en-us_topic_0206809164_p555476199400">Data Lake Insight (DLI) is a serverless data processing and analysis service fully compatible with Apache Spark and Apache Flink ecosystems. It frees you from managing any server. DLI supports standard SQL and is compatible with Spark and Flink SQL. It also supports multiple access modes, and is compatible with mainstream data formats. DLI supports SQL statements and Spark applications for heterogeneous data sources, including CloudTable, RDS, GaussDB(DWS), CSS, OBS, custom databases on ECSs, and offline databases.</p>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_03_0221.html">Usage</a></div>
</div>
</div>

11
docs/dli/umn/dli_03_0017.html
View File

@ -1,23 +1,21 @@
<a name="dli_03_0017"></a><a name="dli_03_0017"></a> <a name="dli_03_0017"></a><a name="dli_03_0017"></a>
<h1 class="topictitle1">How Do I Set the AK/SK for a Queue to Operate an OBS Table?</h1> <h1 class="topictitle1">How Do I Set the AK/SK for a Queue to Operate an OBS Table?</h1>
<div id="body8662426"><div class="note" id="dli_03_0017__en-us_topic_0206809306_note84662050195919"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_03_0017__en-us_topic_0206809306_p1140569541">Hard-coded or plaintext AK and SK pose significant security risks. To ensure security, encrypt your AK and SK, store them in configuration files or environment variables, and decrypt them when needed.</p> <div id="body8662426"><div class="section" id="dli_03_0017__en-us_topic_0206809306_section14374189123112"><h4 class="sectiontitle">Setting Up a Spark Jar Job to Obtain the AK/SK</h4><ul id="dli_03_0017__en-us_topic_0206809306_ul0763234018"><li id="dli_03_0017__en-us_topic_0206809306_li4988101981814"><strong id="dli_03_0017__en-us_topic_0206809306_b51747113263">To obtain the AK/SK, set the parameters as follows:</strong><ul id="dli_03_0017__en-us_topic_0206809306_ul1681421101910"><li id="dli_03_0017__en-us_topic_0206809306_li16773264014">Create a SparkContext using code.<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen445105195">val sc: SparkContext = new SparkContext()
</div></div>
<ul id="dli_03_0017__en-us_topic_0206809306_ul0763234018"><li id="dli_03_0017__en-us_topic_0206809306_li4988101981814">If the AK and SK are obtained, set the parameters as follows:<ul id="dli_03_0017__en-us_topic_0206809306_ul1681421101910"><li id="dli_03_0017__en-us_topic_0206809306_li16773264014">Create SparkContext using code<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen445105195">val sc: SparkContext = new SparkContext()
sc.hadoopConfiguration.set("fs.obs.access.key", ak) sc.hadoopConfiguration.set("fs.obs.access.key", ak)
sc.hadoopConfiguration.set("fs.obs.secret.key", sk)</pre> sc.hadoopConfiguration.set("fs.obs.secret.key", sk)</pre>
</li><li id="dli_03_0017__en-us_topic_0206809306_li15659814124010">Create SparkSession using code<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen1327819344407">val sparkSession: SparkSession = SparkSession </li><li id="dli_03_0017__en-us_topic_0206809306_li15659814124010">Create a SparkSession using code.<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen1327819344407">val sparkSession: SparkSession = SparkSession
.builder() .builder()
.config("spark.hadoop.fs.obs.access.key", ak) .config("spark.hadoop.fs.obs.access.key", ak)
.config("spark.hadoop.fs.obs.secret.key", sk) .config("spark.hadoop.fs.obs.secret.key", sk)
.enableHiveSupport() .enableHiveSupport()
.getOrCreate()</pre> .getOrCreate()</pre>
</li></ul> </li></ul>
</li><li id="dli_03_0017__en-us_topic_0206809306_li1527912306196">If <strong id="dli_03_0017__en-us_topic_0206809306_b20177194413115">ak</strong>, <strong id="dli_03_0017__en-us_topic_0206809306_b10889204518118">sk</strong>, and <strong id="dli_03_0017__en-us_topic_0206809306_b1312611511714">securitytoken</strong> are obtained, the temporary AK/SK and security token must be used at the same time during authentication. The setting is as follows:<ul id="dli_03_0017__en-us_topic_0206809306_ul10494125312204"><li id="dli_03_0017__en-us_topic_0206809306_li549211545202">Create SparkContext using code<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen134921254152020">val sc: SparkContext = new SparkContext() </li><li id="dli_03_0017__en-us_topic_0206809306_li1527912306196"><strong id="dli_03_0017__en-us_topic_0206809306_b112016367337">To obtain the AK/SK and security token and use them together for authentication, set the parameters as follows:</strong><ul id="dli_03_0017__en-us_topic_0206809306_ul10494125312204"><li id="dli_03_0017__en-us_topic_0206809306_li549211545202">Create a SparkContext using code.<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen134921254152020">val sc: SparkContext = new SparkContext()
sc.hadoopConfiguration.set("fs.obs.access.key", ak) sc.hadoopConfiguration.set("fs.obs.access.key", ak)
sc.hadoopConfiguration.set("fs.obs.secret.key", sk) sc.hadoopConfiguration.set("fs.obs.secret.key", sk)
sc.hadoopConfiguration.set("fs.obs.session.token", sts)</pre> sc.hadoopConfiguration.set("fs.obs.session.token", sts)</pre>
</li><li id="dli_03_0017__en-us_topic_0206809306_li4492175482019">Create SparkSession using code<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen16492195411204">val sparkSession: SparkSession = SparkSession </li><li id="dli_03_0017__en-us_topic_0206809306_li4492175482019">Create a SparkSession using code.<pre class="screen" id="dli_03_0017__en-us_topic_0206809306_screen16492195411204">val sparkSession: SparkSession = SparkSession
.builder() .builder()
.config("spark.hadoop.fs.obs.access.key", ak) .config("spark.hadoop.fs.obs.access.key", ak)
.config("spark.hadoop.fs.obs.secret.key", sk) .config("spark.hadoop.fs.obs.secret.key", sk)
@ -27,6 +25,7 @@ sc.hadoopConfiguration.set("fs.obs.session.token", sts)</pre>
</li></ul> </li></ul>
</li></ul> </li></ul>
</div> </div>
</div>
<div> <div>
<div class="familylinks"> <div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_03_0217.html">Job Development</a></div> <div class="parentlink"><strong>Parent topic:</strong> <a href="dli_03_0217.html">Job Development</a></div>

2
docs/dli/umn/dli_03_0032.html
View File

@ -3,7 +3,7 @@
<h1 class="topictitle1">How Do I Increase a Quota?</h1> <h1 class="topictitle1">How Do I Increase a Quota?</h1>
<div id="body8662426"><div class="section" id="dli_03_0032__en-us_topic_0215567055_section44761618172016"><h4 class="sectiontitle">Increasing a Quota</h4><p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p133802054172617">The system does not support online quota adjustment. To increase a resource quota, dial the hotline or send an email to the customer service. We will process your application and inform you of the progress by phone call or email.</p> <div id="body8662426"><div class="section" id="dli_03_0032__en-us_topic_0215567055_section44761618172016"><h4 class="sectiontitle">Increasing a Quota</h4><p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p133802054172617">The system does not support online quota adjustment. To increase a resource quota, dial the hotline or send an email to the customer service. We will process your application and inform you of the progress by phone call or email.</p>
<p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p13745144852611">Before dialing the hotline number or sending an email, ensure that the following information has been obtained:</p> <p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p13745144852611">Before dialing the hotline number or sending an email, ensure that the following information has been obtained:</p>
<ul id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_ul37351029162712"><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li12735132932714">Domain name, project name, and project ID<p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p1973542918275"><a name="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li12735132932714"></a><a name="en-us_topic_0215567055_en-us_topic_0040259342_li12735132932714"></a>Log in to the management console, click the username in the upper-right corner, choose <strong id="dli_03_0032__en-us_topic_0215567055_b1830113104919">My Credentials</strong>, and obtain the domain name, project name, and project ID.</p> <ul id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_ul37351029162712"><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li12735132932714">Domain name, project name, and project ID<p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p1973542918275"><a name="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li12735132932714"></a><a name="en-us_topic_0215567055_en-us_topic_0040259342_li12735132932714"></a>To obtain the preceding information, log in to the management console, click the username in the upper right corner, and choose <strong id="dli_03_0032__en-us_topic_0215567055_b19902152015211">My Credentials</strong> from the drop-down list.</p>
</li><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li13735182917270">Quota information, including:<ul id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_ul6735102912273"><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li1073516296277">Service name</li><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li6735152972712">Quota type</li><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li773592992718">Required quota</li></ul> </li><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li13735182917270">Quota information, including:<ul id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_ul6735102912273"><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li1073516296277">Service name</li><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li6735152972712">Quota type</li><li id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_li773592992718">Required quota</li></ul>
</li></ul> </li></ul>
<p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p126761744182712"><a href="https://open-telekom-cloud.com/en/contact" target="_blank" rel="noopener noreferrer">Learn how to obtain the service hotline and email address.</a></p> <p id="dli_03_0032__en-us_topic_0215567055_en-us_topic_0040259342_p126761744182712"><a href="https://open-telekom-cloud.com/en/contact" target="_blank" rel="noopener noreferrer">Learn how to obtain the service hotline and email address.</a></p>

53
docs/dli/umn/dli_03_0038.html
View File

@ -1,53 +0,0 @@
<a name="dli_03_0038"></a><a name="dli_03_0038"></a>
<h1 class="topictitle1">How Do I Configure Checkpoints for Flink Jar Jobs and Save the Checkpoints to OBS?</h1>
<div id="body8662426"><p id="dli_03_0038__en-us_topic_0215567061_p56251150205517">The procedure is as follows:</p>
<ol id="dli_03_0038__en-us_topic_0215567061_ol132554175512"><li id="dli_03_0038__en-us_topic_0215567061_li11325104175516">Add the following code to the JAR file code of the Flink Jar job:<pre class="screen" id="dli_03_0038__en-us_topic_0215567061_screen72861155194718">// Configure the pom file on which the StreamExecutionEnvironment depends.
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
env.getCheckpointConfig().setCheckpointInterval(40000);
env.getCheckpointConfig().enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
RocksDBStateBackend rocksDbBackend = new RocksDBStateBackend(new FsStateBackend("obs://${bucket}/jobs/checkpoint/my_jar"), false);
rocksDbBackend.setOptions(new OptionsFactory() {
@Override
public DBOptions createDBOptions(DBOptions currentOptions) {
return currentOptions
.setMaxLogFileSize(64 * 1024 * 1024)
.setKeepLogFileNum(3);
}
@Override
public ColumnFamilyOptions createColumnOptions(ColumnFamilyOptions currentOptions) {
return currentOptions;
}
});
env.setStateBackend(rocksDbBackend);</pre>
<div class="note" id="dli_03_0038__en-us_topic_0215567061_note1550723211552"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_03_0038__en-us_topic_0215567061_p9861112216526">The preceding code saves the checkpoint to the <strong id="dli_03_0038__en-us_topic_0215567061_b1280372317545">${bucket}</strong> bucket in <strong id="dli_03_0038__en-us_topic_0215567061_b1028520345412">jobs/checkpoint/my_jar</strong> path every 40 seconds in <strong id="dli_03_0038__en-us_topic_0215567061_b1843412334543">EXACTLY_ONCE</strong> mode.</p>
<p id="dli_03_0038__en-us_topic_0215567061_p3607113119538">Pay attention to the checkpoint storage path. Generally, the checkpoint is stored in the OBS bucket. The path format is as follows:</p>
<ul id="dli_03_0038__en-us_topic_0215567061_ul207281741155519"><li id="dli_03_0038__en-us_topic_0215567061_li2656195310532">Path format: <strong id="dli_03_0038__en-us_topic_0215567061_b1314151203018">obs://$</strong><em id="dli_03_0038__en-us_topic_0215567061_i102861743016">{bucket}</em><strong id="dli_03_0038__en-us_topic_0215567061_b138191203302">/</strong><em id="dli_03_0038__en-us_topic_0215567061_i3683123123016">xxx</em><strong id="dli_03_0038__en-us_topic_0215567061_b270512633017">/</strong><em id="dli_03_0038__en-us_topic_0215567061_i184366297305">xxx</em><strong id="dli_03_0038__en-us_topic_0215567061_b1391313553016">/</strong><em id="dli_03_0038__en-us_topic_0215567061_i02161433113013">xxx</em></li><li id="dli_03_0038__en-us_topic_0215567061_li177099373193">Add the following configuration to the POM file for the packages on which the StreamExecutionEnvironment depends: <pre class="screen" id="dli_03_0038__en-us_topic_0215567061_screen125777567191">&lt;dependency&gt;
&lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
&lt;artifactId&gt;flink-streaming-java_${scala.binary.version}&lt;/artifactId&gt;
&lt;version&gt;${flink.version}&lt;/version&gt;
&lt;scope&gt;provided&lt;/scope&gt;
&lt;/dependency&gt;</pre>
</li></ul>
</div></div>
</li><li id="dli_03_0038__en-us_topic_0215567061_li16530131375812">Configure <strong id="dli_03_0038__en-us_topic_0215567061_b382903363917">Runtime Configuration</strong> and <strong id="dli_03_0038__en-us_topic_0215567061_b813082017112">Restore Job from Checkpoint</strong> for a DLI Flink Jar job.<ul id="dli_03_0038__en-us_topic_0215567061_ul18371837182512"><li id="dli_03_0038__en-us_topic_0215567061_li173711337182517">Constraints on parameter optimization<ol type="a" id="dli_03_0038__en-us_topic_0215567061_ol20219122512611"><li id="dli_03_0038__en-us_topic_0215567061_li13219152519262">In the left navigation pane of the DLI console, choose <strong id="dli_03_0038__en-us_topic_0215567061_b1316611115416">Global Configuration</strong> &gt; <strong id="dli_03_0038__en-us_topic_0215567061_b1379164219554">Service Authorization</strong>. On the page displayed, select <strong id="dli_03_0038__en-us_topic_0215567061_b10426257105611">Tenant Administrator(Global service)</strong> and click <strong id="dli_03_0038__en-us_topic_0215567061_b1315417113575">Update</strong>.</li><li id="dli_03_0038__en-us_topic_0215567061_li196306357262">The bucket to which data is written must be an OBS bucket created by a main account.</li></ol>
</li></ul>
<ul id="dli_03_0038__en-us_topic_0215567061_ul2409151817151"><li id="dli_03_0038__en-us_topic_0215567061_li1741251831516">Configuring <strong id="dli_03_0038__en-us_topic_0215567061_b19382414163720">Restore Job from Checkpoint</strong><ol type="a" id="dli_03_0038__en-us_topic_0215567061_ol104471320125811"><li id="dli_03_0038__en-us_topic_0215567061_li17758171244">Select <strong id="dli_03_0038__en-us_topic_0215567061_b1676611115100">Auto Restart upon Exception</strong>.</li><li id="dli_03_0038__en-us_topic_0215567061_li1647715617413">Select <strong id="dli_03_0038__en-us_topic_0215567061_b8201636101416">Restore Job from Checkpoint</strong> and set the <strong id="dli_03_0038__en-us_topic_0215567061_b176918334144">Checkpoint Path</strong>.<p id="dli_03_0038__en-us_topic_0215567061_p4605181712511">The checkpoint path is the same as that you set in JAR file code. The format is as follows:</p>
<ul id="dli_03_0038__en-us_topic_0215567061_ul1092113535519"><li id="dli_03_0038__en-us_topic_0215567061_li7921553954">${bucket}/xxx/xxx/xxx</li><li id="dli_03_0038__en-us_topic_0215567061_li6374320619">Example<p id="dli_03_0038__en-us_topic_0215567061_p76786211761"><a name="dli_03_0038__en-us_topic_0215567061_li6374320619"></a><a name="en-us_topic_0215567061_li6374320619"></a>If the path in the JAR file is <strong id="dli_03_0038__en-us_topic_0215567061_b72404520198">obs://mybucket/jobs/checkpoint/jar-3</strong>,</p>
<p id="dli_03_0038__en-us_topic_0215567061_p1358474961">Set <strong id="dli_03_0038__en-us_topic_0215567061_b11181818131917">Checkpoint Path</strong> to <strong id="dli_03_0038__en-us_topic_0215567061_b1115882111919">mybucket/jobs/checkpoint/jar-3</strong>.</p>
</li></ul>
</li></ol>
<div class="note" id="dli_03_0038__en-us_topic_0215567061_note813012121076"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="dli_03_0038__en-us_topic_0215567061_ul1342414571176"><li id="dli_03_0038__en-us_topic_0215567061_li1716613491787">The checkpoint path for each Flink Jar job must be unique. Otherwise, data cannot be restored.</li><li id="dli_03_0038__en-us_topic_0215567061_li1442418573715">DLI can access files in the checkpoint path only after DLI is authorized to access the OBS bucket.</li></ul>
</div></div>
</li></ul>
</li><li id="dli_03_0038__en-us_topic_0215567061_li166898359101">Check whether the job is restored from the checkpoint.</li></ol>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dli_03_0132.html">Flink Jar Jobs</a></div>
</div>
</div>

6
docs/dli/umn/dli_03_0044.html
View File

@ -1,8 +1,8 @@
<a name="dli_03_0044"></a><a name="dli_03_0044"></a> <a name="dli_03_0044"></a><a name="dli_03_0044"></a>
<h1 class="topictitle1">Does a Flink JAR Job Support Configuration File Upload? How Do I Upload a Configuration File?</h1> <h1 class="topictitle1">Can I Upload Configuration Files for Flink Jar Jobs?</h1>
<div id="body8662426"><div class="section" id="dli_03_0044__en-us_topic_0261373736_section1812017404262"><h4 class="sectiontitle">Uploading a Configuration File for a Flink JAR Job</h4><p id="dli_03_0044__en-us_topic_0261373736_p436703917344">Configuration files can be uploaded for user-defined jobs (JAR).</p> <div id="body8662426"><div class="section" id="dli_03_0044__en-us_topic_0261373736_section1812017404262"><h4 class="sectiontitle">Uploading a Configuration File for a Flink Jar Job</h4><p id="dli_03_0044__en-us_topic_0261373736_p436703917344">You can upload configuration files for custom jobs (Jar).</p>
<ol id="dli_03_0044__en-us_topic_0261373736_ol1626774516343"><li id="dli_03_0044__en-us_topic_0261373736_li19615165223414">Upload the configuration file to DLI through <strong id="dli_03_0044__en-us_topic_0261373736_b1283912540351">Package Management</strong>.</li><li id="dli_03_0044__en-us_topic_0261373736_li951010414351">In the <strong id="dli_03_0044__en-us_topic_0261373736_b36456499366">Other Dependencies</strong> area of the Flink JAR job, select the created DLI package.</li><li id="dli_03_0044__en-us_topic_0261373736_li6268204593415">Load the file through <strong id="dli_03_0044__en-us_topic_0261373736_b3618202417370">ClassName.class.getClassLoader().getResource("userData/fileName")</strong> in the code. In the file name, <strong id="dli_03_0044__en-us_topic_0261373736_b17920430173719">fileName</strong> indicates the name of the file to be accessed, and <strong id="dli_03_0044__en-us_topic_0261373736_b82075350378">ClassName</strong> indicates the name of the class that needs to access the file.</li></ol> <ol id="dli_03_0044__en-us_topic_0261373736_ol1626774516343"><li id="dli_03_0044__en-us_topic_0261373736_li19615165223414">Upload the configuration file to DLI through <strong id="dli_03_0044__en-us_topic_0261373736_b1283912540351">Package Management</strong>.</li><li id="dli_03_0044__en-us_topic_0261373736_li951010414351">In the <strong id="dli_03_0044__en-us_topic_0261373736_b36456499366">Other Dependencies</strong> area of the Flink Jar job, select the created DLI package.</li><li id="dli_03_0044__en-us_topic_0261373736_li6268204593415">Load the file through <strong id="dli_03_0044__en-us_topic_0261373736_b3618202417370">ClassName.class.getClassLoader().getResource("userData/fileName")</strong> in the code. In the file name, <strong id="dli_03_0044__en-us_topic_0261373736_b17920430173719">fileName</strong> indicates the name of the file to be accessed, and <strong id="dli_03_0044__en-us_topic_0261373736_b82075350378">ClassName</strong> indicates the name of the class that needs to access the file.</li></ol>
</div> </div>
<div class="section" id="dli_03_0044__en-us_topic_0261373736_section4291385272"><h4 class="sectiontitle">Using a Configuration File</h4><ul id="dli_03_0044__en-us_topic_0261373736_ul11302205172912"><li id="dli_03_0044__en-us_topic_0261373736_li0302125113292">Solution 1: Load the file content to the memory in the <strong id="dli_03_0044__en-us_topic_0261373736_b15747173954310">main</strong> function and broadcast the content to each taskmanager. This method is applicable to the scenario where a small number of variables need to be loaded in advance.</li><li id="dli_03_0044__en-us_topic_0261373736_li63027512297">Solution 2: Load the file when initializing the operator in <strong id="dli_03_0044__en-us_topic_0261373736_b1431862812572">open</strong>. A relative or absolute path can be used.<p id="dli_03_0044__en-us_topic_0261373736_p5620103717292">Take Kafka sink as an example. Two files (<strong id="dli_03_0044__en-us_topic_0261373736_b268353912473">userData/kafka-sink.conf</strong> and <strong id="dli_03_0044__en-us_topic_0261373736_b989964411478">userData/client.truststore.jks</strong>) need to be loaded.</p> <div class="section" id="dli_03_0044__en-us_topic_0261373736_section4291385272"><h4 class="sectiontitle">Using a Configuration File</h4><ul id="dli_03_0044__en-us_topic_0261373736_ul11302205172912"><li id="dli_03_0044__en-us_topic_0261373736_li0302125113292">Solution 1: Load the file content to the memory in the <strong id="dli_03_0044__en-us_topic_0261373736_b15747173954310">main</strong> function and broadcast the content to each taskmanager. This method is applicable to the scenario where a small number of variables need to be loaded in advance.</li><li id="dli_03_0044__en-us_topic_0261373736_li63027512297">Solution 2: Load the file when initializing the operator in <strong id="dli_03_0044__en-us_topic_0261373736_b1431862812572">open</strong>. A relative or absolute path can be used.<p id="dli_03_0044__en-us_topic_0261373736_p5620103717292">Take Kafka sink as an example. Two files (<strong id="dli_03_0044__en-us_topic_0261373736_b268353912473">userData/kafka-sink.conf</strong> and <strong id="dli_03_0044__en-us_topic_0261373736_b989964411478">userData/client.truststore.jks</strong>) need to be loaded.</p>
<ul id="dli_03_0044__en-us_topic_0261373736_ul2793324324"><li id="dli_03_0044__en-us_topic_0261373736_li169441528113212"><strong id="dli_03_0044__en-us_topic_0261373736_b963173510326">Example of using a relative path:</strong><pre class="screen" id="dli_03_0044__en-us_topic_0261373736_screen1075255119304"><strong id="dli_03_0044__en-us_topic_0261373736_b1460554803110">Relative path: confPath = userData/kafka-sink.conf</strong> <ul id="dli_03_0044__en-us_topic_0261373736_ul2793324324"><li id="dli_03_0044__en-us_topic_0261373736_li169441528113212"><strong id="dli_03_0044__en-us_topic_0261373736_b963173510326">Example of using a relative path:</strong><pre class="screen" id="dli_03_0044__en-us_topic_0261373736_screen1075255119304"><strong id="dli_03_0044__en-us_topic_0261373736_b1460554803110">Relative path: confPath = userData/kafka-sink.conf</strong>

3
docs/dli/umn/dli_03_0075.html
View File

@ -28,7 +28,6 @@
<span class="w"> </span><span class="n">sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;{{mySk}}&quot;</span> <span class="w"> </span><span class="n">sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;{{mySk}}&quot;</span>
<span class="p">);</span> <span class="p">);</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
<p id="dli_03_0075__en-us_topic_0278811726_p11128121811318">Structure of the data storage directory in OBS: <strong id="dli_03_0075__en-us_topic_0278811726_b6913454145018">obs://obs-sink/car_infos/day=xx/part-x-x</strong>.</p> <p id="dli_03_0075__en-us_topic_0278811726_p11128121811318">Structure of the data storage directory in OBS: <strong id="dli_03_0075__en-us_topic_0278811726_b6913454145018">obs://obs-sink/car_infos/day=xx/part-x-x</strong>.</p>
<p id="dli_03_0075__en-us_topic_0278811726_p61281918111319">After the data is generated, the OBS partition table can be established for subsequent batch processing through the following SQL statements:</p> <p id="dli_03_0075__en-us_topic_0278811726_p61281918111319">After the data is generated, the OBS partition table can be established for subsequent batch processing through the following SQL statements:</p>
@ -48,11 +47,9 @@
<span class="w"> </span><span class="n">stored</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">parquet</span> <span class="w"> </span><span class="n">stored</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">parquet</span>
<span class="w"> </span><span class="k">location</span><span class="w"> </span><span class="s1">'obs://obs-sink/car-infos'</span><span class="p">;</span> <span class="w"> </span><span class="k">location</span><span class="w"> </span><span class="s1">'obs://obs-sink/car-infos'</span><span class="p">;</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
</li><li id="dli_03_0075__en-us_topic_0278811726_li1185684751719">Restore partition information from the associated OBS path.<div class="codecoloring" codetype="Sql" id="dli_03_0075__en-us_topic_0278811726_screen363017182184"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">alter</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="n">recover</span><span class="w"> </span><span class="n">partitions</span><span class="p">;</span> </li><li id="dli_03_0075__en-us_topic_0278811726_li1185684751719">Restore partition information from the associated OBS path.<div class="codecoloring" codetype="Sql" id="dli_03_0075__en-us_topic_0278811726_screen363017182184"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">alter</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="n">recover</span><span class="w"> </span><span class="n">partitions</span><span class="p">;</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
</li></ol> </li></ol>
</div> </div>

3
docs/dli/umn/dli_03_0089.html
View File

@ -30,7 +30,6 @@
<span class="w"> </span><span class="n">sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;{{mySk}}&quot;</span> <span class="w"> </span><span class="n">sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="ss">&quot;{{mySk}}&quot;</span>
<span class="p">);</span> <span class="p">);</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
<p id="dli_03_0089__en-us_topic_0000001200929444_p11128121811318">Structure of the data storage directory in OBS: <strong id="dli_03_0089__en-us_topic_0000001200929444_b178474293304">obs://obs-sink/car_infos/day=xx/part-x-x</strong>.</p> <p id="dli_03_0089__en-us_topic_0000001200929444_p11128121811318">Structure of the data storage directory in OBS: <strong id="dli_03_0089__en-us_topic_0000001200929444_b178474293304">obs://obs-sink/car_infos/day=xx/part-x-x</strong>.</p>
<p id="dli_03_0089__en-us_topic_0000001200929444_p61281918111319">After the data is generated, the OBS partition table can be established for subsequent batch processing through the following SQL statements:</p> <p id="dli_03_0089__en-us_topic_0000001200929444_p61281918111319">After the data is generated, the OBS partition table can be established for subsequent batch processing through the following SQL statements:</p>
@ -50,11 +49,9 @@
<span class="w"> </span><span class="n">stored</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">parquet</span> <span class="w"> </span><span class="n">stored</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="n">parquet</span>
<span class="w"> </span><span class="k">location</span><span class="w"> </span><span class="s1">'obs://obs-sink/car-infos'</span><span class="p">;</span> <span class="w"> </span><span class="k">location</span><span class="w"> </span><span class="s1">'obs://obs-sink/car-infos'</span><span class="p">;</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
</li><li id="dli_03_0089__en-us_topic_0000001200929444_li1185684751719">Restore partition information from the associated OBS path.<div class="codecoloring" codetype="Sql" id="dli_03_0089__en-us_topic_0000001200929444_screen363017182184"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">alter</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="n">recover</span><span class="w"> </span><span class="n">partitions</span><span class="p">;</span> </li><li id="dli_03_0089__en-us_topic_0000001200929444_li1185684751719">Restore partition information from the associated OBS path.<div class="codecoloring" codetype="Sql" id="dli_03_0089__en-us_topic_0000001200929444_screen363017182184"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">alter</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="n">car_infos</span><span class="w"> </span><span class="n">recover</span><span class="w"> </span><span class="n">partitions</span><span class="p">;</span>
</pre></div></td></tr></table></div> </pre></div></td></tr></table></div>
</div> </div>
</li></ol> </li></ol>
</div> </div>

37
docs/dli/umn/dli_03_0093.html
View File

@ -1,11 +1,38 @@
<a name="dli_03_0093"></a><a name="dli_03_0093"></a> <a name="dli_03_0093"></a><a name="dli_03_0093"></a>
<h1 class="topictitle1">How Do I Eliminate Data Skew by Configuring AE Parameters?</h1> <h1 class="topictitle1">How Do I Do When Data Skew Occurs During the Execution of a SQL Job?</h1>
<div id="body8662426"><div class="section" id="dli_03_0093__en-us_topic_0000001245928943_section3741052181318"><h4 class="sectiontitle">Scenario</h4><p id="dli_03_0093__en-us_topic_0000001245928943_p2044116201514">If the execution of an SQL statement takes a long time, you need to access the Spark UI to check the execution status.</p> <div id="body8662426"><div class="section" id="dli_03_0093__en-us_topic_0000001245928943_section3741052181318"><h4 class="sectiontitle">What Is Data Skew?</h4><p id="dli_03_0093__en-us_topic_0000001245928943_p147047145313">Data skew is a common issue during the execution of SQL jobs. When data is unevenly distributed, some compute nodes process significantly more data than others, which can impact the efficiency of the entire computation process.</p>
<p id="dli_03_0093__en-us_topic_0000001245928943_p1831844914150">If data skew occurs, the running time of a stage exceeds 20 minutes and only one task is running.</p> <p id="dli_03_0093__en-us_topic_0000001245928943_p2044116201514">For example, if you notice that a SQL query is taking a long time to execute, you can check its status in SparkUI. See <a href="#dli_03_0093__en-us_topic_0000001245928943_fig1563419919123">Figure 1</a>. If you see a stage that has been running for over 20 minutes with only one task remaining, it is likely due to data skew.</p>
<div class="fignone" id="dli_03_0093__en-us_topic_0000001245928943_fig1563419919123"><span class="figcap"><b>Figure 1 </b>Data skew example</span><br><span><img id="dli_03_0093__en-us_topic_0000001245928943_image20634397125" src="en-us_image_0000001200929158.png"></span></div> <div class="fignone" id="dli_03_0093__en-us_topic_0000001245928943_fig1563419919123"><a name="dli_03_0093__en-us_topic_0000001245928943_fig1563419919123"></a><a name="en-us_topic_0000001245928943_fig1563419919123"></a><span class="figcap"><b>Figure 1 </b>Data skew example</span><br><span><img id="dli_03_0093__en-us_topic_0000001245928943_image20634397125" src="en-us_image_0000001200929158.png"></span></div>
</div> </div>
<div class="section" id="dli_03_0093__en-us_topic_0000001245928943_section103871422101613"><h4 class="sectiontitle">Procedure</h4><ol id="dli_03_0093__en-us_topic_0000001245928943_ol0886142111382"><li id="dli_03_0093__en-us_topic_0000001245928943_li2070131479">Log in to the DLI management console. Choose <strong id="dli_03_0093__en-us_topic_0000001245928943_b12393181651513">Job Management</strong> &gt; <strong id="dli_03_0093__en-us_topic_0000001245928943_b17732111851513">SQL Jobs</strong> in the navigation pane. On the displayed page, locate the job you want to modify and click <strong id="dli_03_0093__en-us_topic_0000001245928943_b896812519154">Edit</strong> in the <strong id="dli_03_0093__en-us_topic_0000001245928943_b6115983153">Operation</strong> column to switch to the <strong id="dli_03_0093__en-us_topic_0000001245928943_b14811012171514">SQL Editor</strong> page.</li><li id="dli_03_0093__en-us_topic_0000001245928943_li08446247383">On the <strong id="dli_03_0093__en-us_topic_0000001245928943_b78691034181518">SQL editor</strong> page, click <strong id="dli_03_0093__en-us_topic_0000001245928943_b179641537181519">Set Property</strong> and add the following Spark parameters through the <strong id="dli_03_0093__en-us_topic_0000001245928943_b4121219171919">Settings</strong> pane:<p id="dli_03_0093__en-us_topic_0000001245928943_p1261618162473"></p> <div class="section" id="dli_03_0093__en-us_topic_0000001245928943_section101101434103017"><h4 class="sectiontitle">Common Data Skew Scenarios</h4><ul id="dli_03_0093__en-us_topic_0000001245928943_ul1834235903214"><li id="dli_03_0093__en-us_topic_0000001245928943_li2342859123214">Group By aggregation skew<p id="dli_03_0093__en-us_topic_0000001245928943_p171381321163310"><a name="dli_03_0093__en-us_topic_0000001245928943_li2342859123214"></a><a name="en-us_topic_0000001245928943_li2342859123214"></a>During the execution of Group By aggregation, if some grouping keys have significantly more data than others, the larger groups will consume more compute resources and time during the aggregation process, resulting in slower processing speeds and data skew.</p>
</li><li id="dli_03_0093__en-us_topic_0000001245928943_li173421359183214">JOIN operation skew<p id="dli_03_0093__en-us_topic_0000001245928943_p48908265331"><a name="dli_03_0093__en-us_topic_0000001245928943_li173421359183214"></a><a name="en-us_topic_0000001245928943_li173421359183214"></a>During table JOIN operations, if the keys involved in the JOIN are unevenly distributed in one of the tables, a large amount of data will be concentrated in a few tasks while others have already completed, causing data skew.</p>
</li></ul>
</div>
<div class="section" id="dli_03_0093__en-us_topic_0000001245928943_section19311210103514"><h4 class="sectiontitle">Solution for Group By Data Skew</h4><p id="dli_03_0093__en-us_topic_0000001245928943_p1781314135199">Select a subset of data and run <strong id="dli_03_0093__en-us_topic_0000001245928943_b49485217101">select count(*) as sum,Key from tbl group by Key order by sum desc</strong> to identify which keys are causing data skew.</p>
<p id="dli_03_0093__en-us_topic_0000001245928943_p1181311313199">Then, for the skewed keys, you can handle them separately by adding a salt to split them into multiple tasks for individual statistics, and finally combine the results of the separate statistics.</p>
<p id="dli_03_0093__en-us_topic_0000001245928943_p581317138195">For example, consider the following SQL query where <strong id="dli_03_0093__en-us_topic_0000001245928943_b203721041212">Key01</strong> is identified as the skewed key causing a single task to process a large amount of data. The following steps can be taken to handle it:</p>
<pre class="screen" id="dli_03_0093__en-us_topic_0000001245928943_screen12282102119198">SELECT
a.Key,
SUM(a.sum) AS Cnt
FROM
(
SELECT
Key,
count(*) AS sum
FROM
tbl
GROUP BY
Key,
CASE
WHEN KEY = 'Key01' THEN floor(random () * 200)
ELSE 0
END
) a
GROUP BY
a.Key;</pre>
</div>
<div class="section" id="dli_03_0093__en-us_topic_0000001245928943_section103871422101613"><h4 class="sectiontitle">Solution for JOIN Data Skew</h4><ol id="dli_03_0093__en-us_topic_0000001245928943_ol0886142111382"><li id="dli_03_0093__en-us_topic_0000001245928943_li2070131479">Log in to the DLI management console. Choose <strong id="dli_03_0093__en-us_topic_0000001245928943_b12393181651513">Job Management</strong> &gt; <strong id="dli_03_0093__en-us_topic_0000001245928943_b17732111851513">SQL Jobs</strong> in the navigation pane. On the displayed page, locate the job you want to modify and click <strong id="dli_03_0093__en-us_topic_0000001245928943_b896812519154">Edit</strong> in the <strong id="dli_03_0093__en-us_topic_0000001245928943_b6115983153">Operation</strong> column to switch to the <strong id="dli_03_0093__en-us_topic_0000001245928943_b14811012171514">SQL Editor</strong> page.</li><li id="dli_03_0093__en-us_topic_0000001245928943_li08446247383">On the <strong id="dli_03_0093__en-us_topic_0000001245928943_b78691034181518">SQL editor</strong> page, click <strong id="dli_03_0093__en-us_topic_0000001245928943_b179641537181519">Set Property</strong> and add the following Spark parameters through the <strong id="dli_03_0093__en-us_topic_0000001245928943_b4121219171919">Settings</strong> pane:<p id="dli_03_0093__en-us_topic_0000001245928943_p1261618162473"></p>
<p id="dli_03_0093__en-us_topic_0000001245928943_p17791144594710">The string followed by the colons (:) are the configuration parameters, and the strings following the colons are the values.</p> <p id="dli_03_0093__en-us_topic_0000001245928943_p17791144594710">The string followed by the colons (:) are the configuration parameters, and the strings following the colons are the values.</p>
<pre class="screen" id="dli_03_0093__en-us_topic_0000001245928943_screen225010144210">spark.sql.enableToString:false <pre class="screen" id="dli_03_0093__en-us_topic_0000001245928943_screen225010144210">spark.sql.enableToString:false
spark.sql.adaptive.join.enabled:true spark.sql.adaptive.join.enabled:true

2
docs/dli/umn/dli_03_0100.html
View File

@ -2,7 +2,7 @@
<h1 class="topictitle1">How Do I Manage Fine-Grained DLI Permissions?</h1> <h1 class="topictitle1">How Do I Manage Fine-Grained DLI Permissions?</h1>
<div id="body8662426"><p id="dli_03_0100__en-us_topic_0000001103929830_p8060118">DLI has a comprehensive permission control mechanism and supports fine-grained authentication through Identity and Access Management (IAM). You can create policies in IAM to manage DLI permissions.</p> <div id="body8662426"><p id="dli_03_0100__en-us_topic_0000001103929830_p8060118">DLI has a comprehensive permission control mechanism and supports fine-grained authentication through Identity and Access Management (IAM). You can create policies in IAM to manage DLI permissions.</p>
<p id="dli_03_0100__en-us_topic_0000001103929830_p188991750134717">IAM allows you to create IAM users for your employees using your cloud account, and assign permissions to control their access to specific types of resources. For example, some software developers in your enterprise need to use DLI resources but must not delete them or perform any high-risk operations. To achieve this result, you can create IAM users for the software developers and grant them only the permissions required for using DLI resources.</p> <p id="dli_03_0100__en-us_topic_0000001103929830_p188991750134717">With IAM, you can use your account to create IAM users for your employees, and assign permissions to the users to control their access to specific resource types. For example, some software developers in your enterprise need to use DLI resources but must not delete them or perform any high-risk operations. To achieve this result, you can create IAM users for the software developers and grant them only the permissions required for using DLI resources.</p>
<div class="note" id="dli_03_0100__en-us_topic_0000001103929830_note17965103310292"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_03_0100__en-us_topic_0000001103929830_p2966183352919">For a new user, you need to log in for the system to record the metadata before using DLI.</p> <div class="note" id="dli_03_0100__en-us_topic_0000001103929830_note17965103310292"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="dli_03_0100__en-us_topic_0000001103929830_p2966183352919">For a new user, you need to log in for the system to record the metadata before using DLI.</p>
</div></div> </div></div>
<p id="dli_03_0100__en-us_topic_0000001103929830_p446382154810">IAM is free to use, and you only need to pay for the resources in your account. </p> <p id="dli_03_0100__en-us_topic_0000001103929830_p446382154810">IAM is free to use, and you only need to pay for the resources in your account. </p>

2
docs/dli/umn/dli_03_0119.html
View File

@ -1,6 +1,6 @@
<a name="dli_03_0119"></a><a name="dli_03_0119"></a> <a name="dli_03_0119"></a><a name="dli_03_0119"></a>
<h1 class="topictitle1">Why Does the Submission Fail Due to Flink JAR File Conflict?</h1> <h1 class="topictitle1">Why Does a Flink Jar Package Conflict Result in Submission Failure?</h1>
<div id="body8662426"><div class="section" id="dli_03_0119__en-us_topic_0000001151736079_section069106105820"><h4 class="sectiontitle">Symptom</h4><p id="dli_03_0119__en-us_topic_0000001151736079_p291919410372">The dependency of your Flink job conflicts with a built-in dependency of the DLI Flink platform. As a result, the job submission fails.</p> <div id="body8662426"><div class="section" id="dli_03_0119__en-us_topic_0000001151736079_section069106105820"><h4 class="sectiontitle">Symptom</h4><p id="dli_03_0119__en-us_topic_0000001151736079_p291919410372">The dependency of your Flink job conflicts with a built-in dependency of the DLI Flink platform. As a result, the job submission fails.</p>
</div> </div>
<div class="section" id="dli_03_0119__en-us_topic_0000001151736079_section145021430105819"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0119__en-us_topic_0000001151736079_p18267171317404">Delete your JAR file that is the same as an existing one of the DLI Flink platform.</p> <div class="section" id="dli_03_0119__en-us_topic_0000001151736079_section145021430105819"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0119__en-us_topic_0000001151736079_p18267171317404">Delete your JAR file that is the same as an existing one of the DLI Flink platform.</p>

6
docs/dli/umn/dli_03_0132.html
View File

@ -4,11 +4,9 @@
<div id="body8662426"></div> <div id="body8662426"></div>
<div> <div>
<ul class="ullinks"> <ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_03_0038.html">How Do I Configure Checkpoints for Flink Jar Jobs and Save the Checkpoints to OBS?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0044.html">Can I Upload Configuration Files for Flink Jar Jobs?</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_03_0044.html">Does a Flink JAR Job Support Configuration File Upload? How Do I Upload a Configuration File?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0119.html">Why Does a Flink Jar Package Conflict Result in Submission Failure?</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_03_0119.html">Why Does the Submission Fail Due to Flink JAR File Conflict?</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_03_0161.html">Why Does a Flink Jar Job Fail to Access GaussDB(DWS) and a Message Is Displayed Indicating Too Many Client Connections?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0161.html">Why Does a Flink Jar Job Fail to Access GaussDB(DWS) and a Message Is Displayed Indicating Too Many Client Connections?</a></strong><br>
</li> </li>

9
docs/dli/umn/dli_03_0138.html
View File

@ -1,11 +1,12 @@
<a name="dli_03_0138"></a><a name="dli_03_0138"></a> <a name="dli_03_0138"></a><a name="dli_03_0138"></a>
<h1 class="topictitle1">Why Is a Message Displayed Indicating That the OBS Bucket Is Not Authorized When I Select an OBS Bucket for a Flink SQL Job?</h1> <h1 class="topictitle1">Why Is a Message Displayed Indicating That the OBS Bucket Is Not Authorized When I Select an OBS Bucket for a Flink SQL Job?</h1>
<div id="body8662426"><ul id="dli_03_0138__en-us_topic_0000001163672215_ul5166171204214"><li id="dli_03_0138__en-us_topic_0000001163672215_li16166151214212">Symptom<p id="dli_03_0138__en-us_topic_0000001163672215_p3885920184218"><a name="dli_03_0138__en-us_topic_0000001163672215_li16166151214212"></a><a name="en-us_topic_0000001163672215_li16166151214212"></a>When you create a Flink SQL job and configure the parameters, you select an OBS bucket you have created. The system displays a message indicating that the OBS bucket is not authorized. After you click <strong id="dli_03_0138__en-us_topic_0000001163672215_b510183683711">Authorize</strong>, the system displays a message indicating that an internal error occurred on the server and you need to contact customer service or try again later.</p> <div id="body8662426"><div class="section" id="dli_03_0138__en-us_topic_0000001163672215_section982017354814"><h4 class="sectiontitle">Symptom</h4><p id="dli_03_0138__en-us_topic_0000001163672215_p3885920184218">When you create a Flink SQL job and configure the parameters, you select an OBS bucket you have created. The system displays a message indicating that the OBS bucket is not authorized. After you click <strong id="dli_03_0138__en-us_topic_0000001163672215_b510183683711">Authorize</strong>, the system displays a message indicating that an internal error occurred on the server and you need to contact customer service or try again later.</p>
</li><li id="dli_03_0138__en-us_topic_0000001163672215_li863911444720">Solution<p id="dli_03_0138__en-us_topic_0000001163672215_p1625225144718"><a name="dli_03_0138__en-us_topic_0000001163672215_li863911444720"></a><a name="en-us_topic_0000001163672215_li863911444720"></a>On the settings page, press F12 to view the error details. The following is an example:</p> </div>
<div class="section" id="dli_03_0138__en-us_topic_0000001163672215_section57467123486"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0138__en-us_topic_0000001163672215_p1625225144718">On the settings page, press F12 to view the error details. The following is an example:</p>
<pre class="screen" id="dli_03_0138__en-us_topic_0000001163672215_screen6314327104810">{"error_msg":"An internal error occurred. {0} Contact customer services or try again later ","error_json_opt":{"error": "Unexpected exception[NoSuchElementException: None.get]"},"error_code":"DLI.10001"}</pre> <pre class="screen" id="dli_03_0138__en-us_topic_0000001163672215_screen6314327104810">{"error_msg":"An internal error occurred. {0} Contact customer services or try again later ","error_json_opt":{"error": "Unexpected exception[NoSuchElementException: None.get]"},"error_code":"DLI.10001"}</pre>
<p id="dli_03_0138__en-us_topic_0000001163672215_p1294312054915">Check whether a DLI agency has been created. If you do not have the permission to create an agency. On the DLI console, choose <strong id="dli_03_0138__en-us_topic_0000001163672215_b1226251312438">Global Configuration</strong> &gt; <strong id="dli_03_0138__en-us_topic_0000001163672215_b51921434114319">Service Authorization</strong>, select <strong id="dli_03_0138__en-us_topic_0000001163672215_b141912392433">Tenant Administrator (Global service)</strong>, and click <strong id="dli_03_0138__en-us_topic_0000001163672215_b19811135194412">Update</strong>.</p> <p id="dli_03_0138__en-us_topic_0000001163672215_p414442113481">Check whether a DLI agency has been created. If you do not have the permission to create an agency. On the DLI console, choose <strong id="dli_03_0138__en-us_topic_0000001163672215_b1226251312438">Global Configuration</strong> &gt; <strong id="dli_03_0138__en-us_topic_0000001163672215_b51921434114319">Service Authorization</strong>, select <strong id="dli_03_0138__en-us_topic_0000001163672215_b141912392433">Tenant Administrator (Global service)</strong>, and click <strong id="dli_03_0138__en-us_topic_0000001163672215_b19811135194412">Update</strong>.</p>
</li></ul> </div>
</div> </div>
<div> <div>
<div class="familylinks"> <div class="familylinks">

2
docs/dli/umn/dli_03_0139.html
View File

@ -2,7 +2,7 @@
<h1 class="topictitle1">How Do I Authorize a Subuser to View Flink Jobs?</h1> <h1 class="topictitle1">How Do I Authorize a Subuser to View Flink Jobs?</h1>
<div id="body8662426"><p id="dli_03_0139__en-us_topic_0000001117353058_p8060118">A sub-user can view queues but cannot view Flink jobs. You can authorize the sub-user using DLI or IAM.</p> <div id="body8662426"><p id="dli_03_0139__en-us_topic_0000001117353058_p8060118">A sub-user can view queues but cannot view Flink jobs. You can authorize the sub-user using DLI or IAM.</p>
<ul id="dli_03_0139__en-us_topic_0000001117353058_ul17620104817164"><li id="dli_03_0139__en-us_topic_0000001117353058_li166211948121616">Authorization on DLI<ol id="dli_03_0139__en-us_topic_0000001117353058_ol4536121625818"><li id="dli_03_0139__en-us_topic_0000001117353058_li1053611166582">Log in to the DLI console using a tenant account, a job owner account, or an account with the <strong id="dli_03_0139__en-us_topic_0000001117353058_b96121014185511">DLI Service Administrator</strong> permission.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li189911684597">Choose <strong id="dli_03_0139__en-us_topic_0000001117353058_b6142112214556">Job Management</strong> &gt; <strong id="dli_03_0139__en-us_topic_0000001117353058_b125811224145516">Flink Jobs</strong>. On the displayed page, locate the target job.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li95361716115813">In the <strong id="dli_03_0139__en-us_topic_0000001117353058_b159621232175515">Operation</strong> column of the target job, choose <strong id="dli_03_0139__en-us_topic_0000001117353058_b17482938115519">More</strong> &gt; <strong id="dli_03_0139__en-us_topic_0000001117353058_b17521144219550">Permissions</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li7170108316">On the displayed page, click <strong id="dli_03_0139__en-us_topic_0000001117353058_b16298121185719">Grant Permission</strong>. Enter the name of the user to be authorized and select the required permissions. Click <strong id="dli_03_0139__en-us_topic_0000001117353058_b18687182014589">OK</strong>. The authorized user can view the job and perform related operations.</li></ol> <ul id="dli_03_0139__en-us_topic_0000001117353058_ul17620104817164"><li id="dli_03_0139__en-us_topic_0000001117353058_li166211948121616">Authorization on DLI<ol id="dli_03_0139__en-us_topic_0000001117353058_ol4536121625818"><li id="dli_03_0139__en-us_topic_0000001117353058_li1053611166582">Log in to the DLI console using a tenant account, a job owner account, or an account with the <strong id="dli_03_0139__en-us_topic_0000001117353058_b1512422361213">DLI Service Administrator</strong> permission.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li189911684597">Choose <strong id="dli_03_0139__en-us_topic_0000001117353058_b6142112214556">Job Management</strong> &gt; <strong id="dli_03_0139__en-us_topic_0000001117353058_b125811224145516">Flink Jobs</strong>. On the displayed page, locate the target job.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li95361716115813">In the <strong id="dli_03_0139__en-us_topic_0000001117353058_b159621232175515">Operation</strong> column of the target job, choose <strong id="dli_03_0139__en-us_topic_0000001117353058_b17482938115519">More</strong> &gt; <strong id="dli_03_0139__en-us_topic_0000001117353058_b17521144219550">Permissions</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li7170108316">On the displayed page, click <strong id="dli_03_0139__en-us_topic_0000001117353058_b16298121185719">Grant Permission</strong>. Enter the name of the user to be authorized and select the required permissions. Click <strong id="dli_03_0139__en-us_topic_0000001117353058_b18687182014589">OK</strong>. The authorized user can view the job and perform related operations.</li></ol>
</li><li id="dli_03_0139__en-us_topic_0000001117353058_li9465195841615">Authorization on IAM<ol id="dli_03_0139__en-us_topic_0000001117353058_ol7448520111711"><li id="dli_03_0139__en-us_topic_0000001117353058_li10449102051716">Log in to the IAM console. In the navigation pane, choose <strong id="dli_03_0139__en-us_topic_0000001117353058_b4984105619110">Permissions</strong> &gt; <strong id="dli_03_0139__en-us_topic_0000001117353058_b187938522117">Policies/Roles</strong>. On the displayed page, click <strong id="dli_03_0139__en-us_topic_0000001117353058_b19912101617319">Create Custom Policy</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li598911295292"><a name="dli_03_0139__en-us_topic_0000001117353058_li598911295292"></a><a name="en-us_topic_0000001117353058_li598911295292"></a>Create a permission policy for the subuser to view DLI Flink jobs.<ul id="dli_03_0139__en-us_topic_0000001117353058_ul5526339297"><li id="dli_03_0139__en-us_topic_0000001117353058_li14733468297"><strong id="dli_03_0139__en-us_topic_0000001117353058_b391425312311">Policy Name</strong>: Use the default name or customize a name.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li1967414214309"><strong id="dli_03_0139__en-us_topic_0000001117353058_b1419718571731">Scope</strong>: Select <strong id="dli_03_0139__en-us_topic_0000001117353058_b20198757631">Project-level services</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li1852633202912"><strong id="dli_03_0139__en-us_topic_0000001117353058_b24773119418">Policy View</strong>: Select <span class="parmvalue" id="dli_03_0139__en-us_topic_0000001117353058_parmvalue13478171119412"><b>Visual editor</b></span>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li13979374304"><strong id="dli_03_0139__en-us_topic_0000001117353058_b1012520242413">Policy Content</strong>: Select <strong id="dli_03_0139__en-us_topic_0000001117353058_b177922319411">Allow</strong>, <strong id="dli_03_0139__en-us_topic_0000001117353058_b49716409413">Data Lake Insight</strong>, and <strong id="dli_03_0139__en-us_topic_0000001117353058_b12211720515">dli:jobs:list_all</strong> in sequence.</li></ul> </li><li id="dli_03_0139__en-us_topic_0000001117353058_li9465195841615">Authorization on IAM<ol id="dli_03_0139__en-us_topic_0000001117353058_ol7448520111711"><li id="dli_03_0139__en-us_topic_0000001117353058_li10449102051716">Log in to the IAM console. In the navigation pane, choose <strong id="dli_03_0139__en-us_topic_0000001117353058_b4984105619110">Permissions</strong> &gt; <strong id="dli_03_0139__en-us_topic_0000001117353058_b187938522117">Policies/Roles</strong>. On the displayed page, click <strong id="dli_03_0139__en-us_topic_0000001117353058_b19912101617319">Create Custom Policy</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li598911295292"><a name="dli_03_0139__en-us_topic_0000001117353058_li598911295292"></a><a name="en-us_topic_0000001117353058_li598911295292"></a>Create a permission policy for the subuser to view DLI Flink jobs.<ul id="dli_03_0139__en-us_topic_0000001117353058_ul5526339297"><li id="dli_03_0139__en-us_topic_0000001117353058_li14733468297"><strong id="dli_03_0139__en-us_topic_0000001117353058_b391425312311">Policy Name</strong>: Use the default name or customize a name.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li1967414214309"><strong id="dli_03_0139__en-us_topic_0000001117353058_b1419718571731">Scope</strong>: Select <strong id="dli_03_0139__en-us_topic_0000001117353058_b20198757631">Project-level services</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li1852633202912"><strong id="dli_03_0139__en-us_topic_0000001117353058_b24773119418">Policy View</strong>: Select <span class="parmvalue" id="dli_03_0139__en-us_topic_0000001117353058_parmvalue13478171119412"><b>Visual editor</b></span>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li13979374304"><strong id="dli_03_0139__en-us_topic_0000001117353058_b1012520242413">Policy Content</strong>: Select <strong id="dli_03_0139__en-us_topic_0000001117353058_b177922319411">Allow</strong>, <strong id="dli_03_0139__en-us_topic_0000001117353058_b49716409413">Data Lake Insight</strong>, and <strong id="dli_03_0139__en-us_topic_0000001117353058_b12211720515">dli:jobs:list_all</strong> in sequence.</li></ul>
<p id="dli_03_0139__en-us_topic_0000001117353058_p136821748173219">Click <strong id="dli_03_0139__en-us_topic_0000001117353058_b10680111412517">OK</strong> to create the policy.</p> <p id="dli_03_0139__en-us_topic_0000001117353058_p136821748173219">Click <strong id="dli_03_0139__en-us_topic_0000001117353058_b10680111412517">OK</strong> to create the policy.</p>
</li><li id="dli_03_0139__en-us_topic_0000001117353058_li382262153413">Go to the <strong id="dli_03_0139__en-us_topic_0000001117353058_b44949864">User Group</strong> page, locate the user group to which the subuser to be authorized belongs and click the user group name. On the displayed page, click <strong id="dli_03_0139__en-us_topic_0000001117353058_b17188140978">Assign</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li98837361">Grant permissions to the user group.<ul id="dli_03_0139__en-us_topic_0000001117353058_ul53118173611"><li id="dli_03_0139__en-us_topic_0000001117353058_li1860192410372">Select <strong id="dli_03_0139__en-us_topic_0000001117353058_b1630810491516">Region-specific projects</strong> for <strong id="dli_03_0139__en-us_topic_0000001117353058_b187253401174">Scope</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li1158216916394">Select the permission policy created in <a href="#dli_03_0139__en-us_topic_0000001117353058_li598911295292">2</a> for <strong id="dli_03_0139__en-us_topic_0000001117353058_b155236188813">Permissions</strong>.<p id="dli_03_0139__en-us_topic_0000001117353058_p1218014144396">You can also select <strong id="dli_03_0139__en-us_topic_0000001117353058_b19286446989">DLI Service Administrator</strong> (with all DLI permissions) for the subuser to view Flink jobs.</p> </li><li id="dli_03_0139__en-us_topic_0000001117353058_li382262153413">Go to the <strong id="dli_03_0139__en-us_topic_0000001117353058_b44949864">User Group</strong> page, locate the user group to which the subuser to be authorized belongs and click the user group name. On the displayed page, click <strong id="dli_03_0139__en-us_topic_0000001117353058_b17188140978">Assign</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li98837361">Grant permissions to the user group.<ul id="dli_03_0139__en-us_topic_0000001117353058_ul53118173611"><li id="dli_03_0139__en-us_topic_0000001117353058_li1860192410372">Select <strong id="dli_03_0139__en-us_topic_0000001117353058_b1630810491516">Region-specific projects</strong> for <strong id="dli_03_0139__en-us_topic_0000001117353058_b187253401174">Scope</strong>.</li><li id="dli_03_0139__en-us_topic_0000001117353058_li1158216916394">Select the permission policy created in <a href="#dli_03_0139__en-us_topic_0000001117353058_li598911295292">2</a> for <strong id="dli_03_0139__en-us_topic_0000001117353058_b155236188813">Permissions</strong>.<p id="dli_03_0139__en-us_topic_0000001117353058_p1218014144396">You can also select <strong id="dli_03_0139__en-us_topic_0000001117353058_b19286446989">DLI Service Administrator</strong> (with all DLI permissions) for the subuser to view Flink jobs.</p>

2
docs/dli/umn/dli_03_0140.html
View File

@ -2,7 +2,7 @@
<h1 class="topictitle1">Why Does My Account Have Insufficient Permissions Due to Arrears?</h1> <h1 class="topictitle1">Why Does My Account Have Insufficient Permissions Due to Arrears?</h1>
<div id="body8662426"><p id="dli_03_0140__en-us_topic_0000001163757385_p1929318813177">When you submit a job, a message is displayed indicating that the job fails to be submitted due to insufficient permission caused by arrears. In this case, you need to check the roles in your token:</p> <div id="body8662426"><p id="dli_03_0140__en-us_topic_0000001163757385_p1929318813177">When you submit a job, a message is displayed indicating that the job fails to be submitted due to insufficient permission caused by arrears. In this case, you need to check the roles in your token:</p>
<ul id="dli_03_0140__en-us_topic_0000001163757385_ul281916395198"><li id="dli_03_0140__en-us_topic_0000001163757385_li138225388207"><strong id="dli_03_0140__en-us_topic_0000001163757385_b1296704434013">op_restrict</strong>: The account permission is restricted due to insufficient balance. If the current account balance is insufficient, all online user tokens under that account will be revoked. If they log in again, the <strong id="dli_03_0140__en-us_topic_0000001163757385_b51781581734">op_restrict</strong> permission is added to the obtained tokens, limiting their operations.</li><li id="dli_03_0140__en-us_topic_0000001163757385_li108191239161917"><strong id="dli_03_0140__en-us_topic_0000001163757385_b975781034512">op_suspended</strong>: Your account is suspended due to arrears or other reasons. If the current account is in arrears, all online user tokens under that account will be revoked. If they log in again, the <strong id="dli_03_0140__en-us_topic_0000001163757385_b016812443316">op_suspended</strong> permission is added to the obtained tokens, limiting their operations (excluding cloud service users).</li></ul> <ul id="dli_03_0140__en-us_topic_0000001163757385_ul281916395198"><li id="dli_03_0140__en-us_topic_0000001163757385_li138225388207"><strong id="dli_03_0140__en-us_topic_0000001163757385_b1296704434013">op_restrict</strong>: The account permission is restricted due to insufficient balance. If the current account balance is insufficient, all online user tokens under that account will be revoked. If they log in again, the <strong id="dli_03_0140__en-us_topic_0000001163757385_b16691447112010">op_restrict</strong> permission is added to the obtained tokens, limiting their operations.</li><li id="dli_03_0140__en-us_topic_0000001163757385_li108191239161917"><strong id="dli_03_0140__en-us_topic_0000001163757385_b975781034512">op_suspended</strong>: Your account is suspended due to arrears or other reasons. If the current account is in arrears, all online user tokens under that account will be revoked. If they log in again, the <strong id="dli_03_0140__en-us_topic_0000001163757385_b15808951122015">op_suspended</strong> permission is added to the obtained tokens, limiting their operations (excluding cloud service users).</li></ul>
<p id="dli_03_0140__en-us_topic_0000001163757385_p27157381229">If the two roles described about are in your token, user operations are restricted.</p> <p id="dli_03_0140__en-us_topic_0000001163757385_p27157381229">If the two roles described about are in your token, user operations are restricted.</p>
</div> </div>
<div> <div>

2
docs/dli/umn/dli_03_0160.html
View File

@ -3,7 +3,7 @@
<h1 class="topictitle1">Why Is Error "No such user. userName:xxxx." Reported on the Flink Job Management Page When I Grant Permission to a User?</h1> <h1 class="topictitle1">Why Is Error "No such user. userName:xxxx." Reported on the Flink Job Management Page When I Grant Permission to a User?</h1>
<div id="body8662426"><div class="section" id="dli_03_0160__en-us_topic_0000001171241720_section11358622151914"><h4 class="sectiontitle">Symptom</h4><p id="dli_03_0160__en-us_topic_0000001171241720_p196708262194">Choose <strong id="dli_03_0160__en-us_topic_0000001171241720_b0914182822313">Job Management</strong> &gt; <strong id="dli_03_0160__en-us_topic_0000001171241720_b4942630162317">Flink Jobs</strong>. In the <strong id="dli_03_0160__en-us_topic_0000001171241720_b29021834162316">Operation</strong> column of the target job, choose <strong id="dli_03_0160__en-us_topic_0000001171241720_b69521341122319">More</strong> &gt; <strong id="dli_03_0160__en-us_topic_0000001171241720_b1880215438238">Permissions</strong>. When a new user is authorized, <strong id="dli_03_0160__en-us_topic_0000001171241720_b414015942414">No such user. userName:xxxx.</strong> is displayed.</p> <div id="body8662426"><div class="section" id="dli_03_0160__en-us_topic_0000001171241720_section11358622151914"><h4 class="sectiontitle">Symptom</h4><p id="dli_03_0160__en-us_topic_0000001171241720_p196708262194">Choose <strong id="dli_03_0160__en-us_topic_0000001171241720_b0914182822313">Job Management</strong> &gt; <strong id="dli_03_0160__en-us_topic_0000001171241720_b4942630162317">Flink Jobs</strong>. In the <strong id="dli_03_0160__en-us_topic_0000001171241720_b29021834162316">Operation</strong> column of the target job, choose <strong id="dli_03_0160__en-us_topic_0000001171241720_b69521341122319">More</strong> &gt; <strong id="dli_03_0160__en-us_topic_0000001171241720_b1880215438238">Permissions</strong>. When a new user is authorized, <strong id="dli_03_0160__en-us_topic_0000001171241720_b414015942414">No such user. userName:xxxx.</strong> is displayed.</p>
</div> </div>
<div class="section" id="dli_03_0160__en-us_topic_0000001171241720_section1823248152219"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0160__en-us_topic_0000001171241720_p1837141072211">Check whether the username exists and whether the user has logged in to the system before authorization.</p> <div class="section" id="dli_03_0160__en-us_topic_0000001171241720_section1823248152219"><h4 class="sectiontitle">Solution</h4><ul id="dli_03_0160__en-us_topic_0000001171241720_ul12596161944514"><li id="dli_03_0160__en-us_topic_0000001171241720_li359715191453">Check whether the username exists.</li><li id="dli_03_0160__en-us_topic_0000001171241720_li17597171914452">If the username exists, relog in to the management console to grant permissions.</li></ul>
</div> </div>
</div> </div>
<div> <div>

2
docs/dli/umn/dli_03_0165.html
View File

@ -4,7 +4,7 @@
<div id="body8662426"><div class="section" id="dli_03_0165__en-us_topic_0000001240589031_section1095918124211"><h4 class="sectiontitle">Symptom</h4><div class="p" id="dli_03_0165__en-us_topic_0000001240589031_p19814151134218">An exception occurred when a Flink Jar job is running. The following error information is displayed in the job log:<pre class="screen" id="dli_03_0165__en-us_topic_0000001240589031_screen9529142034313">org.apache.flink.shaded.curator.org.apache.curator.ConnectionState - Authentication failed</pre> <div id="body8662426"><div class="section" id="dli_03_0165__en-us_topic_0000001240589031_section1095918124211"><h4 class="sectiontitle">Symptom</h4><div class="p" id="dli_03_0165__en-us_topic_0000001240589031_p19814151134218">An exception occurred when a Flink Jar job is running. The following error information is displayed in the job log:<pre class="screen" id="dli_03_0165__en-us_topic_0000001240589031_screen9529142034313">org.apache.flink.shaded.curator.org.apache.curator.ConnectionState - Authentication failed</pre>
</div> </div>
</div> </div>
<div class="section" id="dli_03_0165__en-us_topic_0000001240589031_section337072094918"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0165__en-us_topic_0000001240589031_p11556022194918">Service authorization is not configured for the account on the <strong id="dli_03_0165__en-us_topic_0000001240589031_b978444010">Global Configuration</strong> page. When the account is used to create a datasource connection to access external data, the access fails.</p> <div class="section" id="dli_03_0165__en-us_topic_0000001240589031_section337072094918"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0165__en-us_topic_0000001240589031_p11556022194918">Service authorization is not configured for the account on the <strong id="dli_03_0165__en-us_topic_0000001240589031_b08181125181415">Global Configuration</strong> page. When the account is used to create a datasource connection to access external data, the access fails.</p>
</div> </div>
<div class="section" id="dli_03_0165__en-us_topic_0000001240589031_section167501644164320"><h4 class="sectiontitle">Solution</h4><ol id="dli_03_0165__en-us_topic_0000001240589031_ol1746219303519"><li id="dli_03_0165__en-us_topic_0000001240589031_li7462143095117"><span>Log in to the DLI management console. Choose <strong id="dli_03_0165__en-us_topic_0000001240589031_b395814133431">Global Configuration</strong> &gt; <strong id="dli_03_0165__en-us_topic_0000001240589031_b119341520114316">Service Authorization</strong> in the navigation pane.</span></li><li id="dli_03_0165__en-us_topic_0000001240589031_li122101855165510"><span>On the <strong id="dli_03_0165__en-us_topic_0000001240589031_b11041249124312">Service Authorization</strong> page, select all agency permissions.</span></li><li id="dli_03_0165__en-us_topic_0000001240589031_li13948347125920"><span>Click <strong id="dli_03_0165__en-us_topic_0000001240589031_b181441397448">Update</strong>. If the message "Agency permissions updated successfully" is displayed, the modification is successful.</span></li><li id="dli_03_0165__en-us_topic_0000001240589031_li1023214471407"><span>After the authorization is complete, create a datasource connection and run the job again.</span></li></ol> <div class="section" id="dli_03_0165__en-us_topic_0000001240589031_section167501644164320"><h4 class="sectiontitle">Solution</h4><ol id="dli_03_0165__en-us_topic_0000001240589031_ol1746219303519"><li id="dli_03_0165__en-us_topic_0000001240589031_li7462143095117"><span>Log in to the DLI management console. Choose <strong id="dli_03_0165__en-us_topic_0000001240589031_b395814133431">Global Configuration</strong> &gt; <strong id="dli_03_0165__en-us_topic_0000001240589031_b119341520114316">Service Authorization</strong> in the navigation pane.</span></li><li id="dli_03_0165__en-us_topic_0000001240589031_li122101855165510"><span>On the <strong id="dli_03_0165__en-us_topic_0000001240589031_b11041249124312">Service Authorization</strong> page, select all agency permissions.</span></li><li id="dli_03_0165__en-us_topic_0000001240589031_li13948347125920"><span>Click <strong id="dli_03_0165__en-us_topic_0000001240589031_b181441397448">Update</strong>. If the message "Agency permissions updated successfully" is displayed, the modification is successful.</span></li><li id="dli_03_0165__en-us_topic_0000001240589031_li1023214471407"><span>After the authorization is complete, create a datasource connection and run the job again.</span></li></ol>
</div> </div>

3
docs/dli/umn/dli_03_0171.html
View File

@ -7,8 +7,7 @@
</div> </div>
<div class="section" id="dli_03_0171__en-us_topic_0000001203378546_section97641519173020"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0171__en-us_topic_0000001203378546_p119167231301">The default queue is a public preset queue in the system for function trials. When multiple users submit jobs to this queue, traffic control might be triggered. As a result, the jobs fail to be submitted.</p> <div class="section" id="dli_03_0171__en-us_topic_0000001203378546_section97641519173020"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0171__en-us_topic_0000001203378546_p119167231301">The default queue is a public preset queue in the system for function trials. When multiple users submit jobs to this queue, traffic control might be triggered. As a result, the jobs fail to be submitted.</p>
</div> </div>
<div class="section" id="dli_03_0171__en-us_topic_0000001203378546_section1038513515321"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0171__en-us_topic_0000001203378546_p1146581014322">Buy a custom queue for your jobs. The procedure is as follows:</p> <div class="section" id="dli_03_0171__en-us_topic_0000001203378546_section1038513515321"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0171__en-us_topic_0000001203378546_p1146581014322">Buy a custom queue for your jobs.</p>
<ol id="dli_03_0171__en-us_topic_0000001203378546_ol14991540153315"><li id="dli_03_0171__en-us_topic_0000001203378546_li1059116511492">In the navigation pane of the DLI management console, choose <span class="uicontrol" id="dli_03_0171__en-us_topic_0000001203378546_uicontrol17525333112913"><b>Queue Management</b></span>.</li><li id="dli_03_0171__en-us_topic_0000001203378546_li759275184915">In the upper right corner of the <span class="wintitle" id="dli_03_0171__en-us_topic_0000001203378546_wintitle177728465299"><b>Queue Management</b></span> page, click <strong id="dli_03_0171__en-us_topic_0000001203378546_b1477394682910">Buy Queue</strong> to create a queue.</li><li id="dli_03_0171__en-us_topic_0000001203378546_li19510749133518">On the <strong id="dli_03_0171__en-us_topic_0000001203378546_b26395547302">Buy Queue</strong> page, set the required parameters as you need. Especially, set <strong id="dli_03_0171__en-us_topic_0000001203378546_b41113224312">Type</strong> to <strong id="dli_03_0171__en-us_topic_0000001203378546_b18165183343120">For SQL</strong>.</li><li id="dli_03_0171__en-us_topic_0000001203378546_li1971018493717">Submit your SQL jobs to the newly created queue.</li></ol>
</div> </div>
</div> </div>
<div> <div>

2
docs/dli/umn/dli_03_0172.html
View File

@ -14,7 +14,7 @@ at java.lang.Thread.run(Thread.java:748)</pre>
<div class="section" id="dli_03_0172__en-us_topic_0000001204269300_section175074137553"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0172__en-us_topic_0000001204269300_p12322191805519">When you create a migration job to DLI on the CDM console, you set <strong id="dli_03_0172__en-us_topic_0000001204269300_b288517122104">Resource Queue</strong> to a DLI queue for general purpose. It should be a queue for SQL.</p> <div class="section" id="dli_03_0172__en-us_topic_0000001204269300_section175074137553"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0172__en-us_topic_0000001204269300_p12322191805519">When you create a migration job to DLI on the CDM console, you set <strong id="dli_03_0172__en-us_topic_0000001204269300_b288517122104">Resource Queue</strong> to a DLI queue for general purpose. It should be a queue for SQL.</p>
</div> </div>
<div class="section" id="dli_03_0172__en-us_topic_0000001204269300_section14516191715918"><h4 class="sectiontitle">Solution</h4><ol id="dli_03_0172__en-us_topic_0000001204269300_ol266423125913"><li id="dli_03_0172__en-us_topic_0000001204269300_li153902141886">On the DLI management console and click <strong id="dli_03_0172__en-us_topic_0000001204269300_b1480216921215">Queue Management</strong> in the navigation pane on the left. On the <strong id="dli_03_0172__en-us_topic_0000001204269300_b19375535121212">Queue Management</strong> page, check whether there are SQL queues.<ul id="dli_03_0172__en-us_topic_0000001204269300_ul1227419209811"><li id="dli_03_0172__en-us_topic_0000001204269300_li12741620189">If there are, go to <a href="#dli_03_0172__en-us_topic_0000001204269300_li15230171718316">3</a>.</li><li id="dli_03_0172__en-us_topic_0000001204269300_li439892210819">If there are no SQL queues, go to <a href="#dli_03_0172__en-us_topic_0000001204269300_li147241748216">2</a> to buy an SQL queue.</li></ul> <div class="section" id="dli_03_0172__en-us_topic_0000001204269300_section14516191715918"><h4 class="sectiontitle">Solution</h4><ol id="dli_03_0172__en-us_topic_0000001204269300_ol266423125913"><li id="dli_03_0172__en-us_topic_0000001204269300_li153902141886">On the DLI management console and click <strong id="dli_03_0172__en-us_topic_0000001204269300_b1480216921215">Queue Management</strong> in the navigation pane on the left. On the <strong id="dli_03_0172__en-us_topic_0000001204269300_b19375535121212">Queue Management</strong> page, check whether there are SQL queues.<ul id="dli_03_0172__en-us_topic_0000001204269300_ul1227419209811"><li id="dli_03_0172__en-us_topic_0000001204269300_li12741620189">If there are, go to <a href="#dli_03_0172__en-us_topic_0000001204269300_li15230171718316">3</a>.</li><li id="dli_03_0172__en-us_topic_0000001204269300_li439892210819">If there are no SQL queues, go to <a href="#dli_03_0172__en-us_topic_0000001204269300_li147241748216">2</a> to buy an SQL queue.</li></ul>
</li><li id="dli_03_0172__en-us_topic_0000001204269300_li147241748216"><a name="dli_03_0172__en-us_topic_0000001204269300_li147241748216"></a><a name="en-us_topic_0000001204269300_li147241748216"></a>Click <strong id="dli_03_0172__en-us_topic_0000001204269300_b1030012319167">Buy Queue</strong> to create a queue. Set <strong id="dli_03_0172__en-us_topic_0000001204269300_b7681921613">Type</strong> to <strong id="dli_03_0172__en-us_topic_0000001204269300_b1179816125169">For SQL</strong>, set other parameters required, and click <strong id="dli_03_0172__en-us_topic_0000001204269300_b255619251169">Buy</strong>.</li><li id="dli_03_0172__en-us_topic_0000001204269300_li15230171718316"><a name="dli_03_0172__en-us_topic_0000001204269300_li15230171718316"></a><a name="en-us_topic_0000001204269300_li15230171718316"></a>Go back to the CDM console and create a data migration job. Set <strong id="dli_03_0172__en-us_topic_0000001204269300_b1016784851720">Resource Queue</strong> to the created DLI SQL queue.</li><li id="dli_03_0172__en-us_topic_0000001204269300_li13061022645">Submit the migration job and view the job execution logs.</li></ol> </li><li id="dli_03_0172__en-us_topic_0000001204269300_li147241748216"><a name="dli_03_0172__en-us_topic_0000001204269300_li147241748216"></a><a name="en-us_topic_0000001204269300_li147241748216"></a>Choose <strong id="dli_03_0172__en-us_topic_0000001204269300_b4969152125313">Resources</strong> &gt; <strong id="dli_03_0172__en-us_topic_0000001204269300_b057931110535">Resource Pool</strong>. On the displayed page, locate the purchased elastic resource pool, and click <strong id="dli_03_0172__en-us_topic_0000001204269300_b740754912531">Add Queue</strong> in the <strong id="dli_03_0172__en-us_topic_0000001204269300_b835518547531">Operation</strong> column. Set <strong id="dli_03_0172__en-us_topic_0000001204269300_b886816320546">Type</strong> to <strong id="dli_03_0172__en-us_topic_0000001204269300_b1736516445545">For SQL</strong>, set other parameters, and submit the creation request.</li><li id="dli_03_0172__en-us_topic_0000001204269300_li15230171718316"><a name="dli_03_0172__en-us_topic_0000001204269300_li15230171718316"></a><a name="en-us_topic_0000001204269300_li15230171718316"></a>Go back to the CDM console and create a data migration job. Set <strong id="dli_03_0172__en-us_topic_0000001204269300_b1016784851720">Resource Queue</strong> to the created DLI SQL queue.</li><li id="dli_03_0172__en-us_topic_0000001204269300_li13061022645">Submit the migration job and view the job execution logs.</li></ol>
</div> </div>
</div> </div>
<div> <div>

6
docs/dli/umn/dli_03_0179.html
View File

@ -6,7 +6,7 @@
</div> </div>
<div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section632915396218"><h4 class="sectiontitle">Fault Locating</h4><p id="dli_03_0179__en-us_topic_0000001250602862_p3253105712168">The issues here are described in order of how likely they are to occur.</p> <div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section632915396218"><h4 class="sectiontitle">Fault Locating</h4><p id="dli_03_0179__en-us_topic_0000001250602862_p3253105712168">The issues here are described in order of how likely they are to occur.</p>
<p id="dli_03_0179__en-us_topic_0000001250602862_p10253195781620">Troubleshoot the issue by ruling out the causes described here, one by one.</p> <p id="dli_03_0179__en-us_topic_0000001250602862_p10253195781620">Troubleshoot the issue by ruling out the causes described here, one by one.</p>
<ul id="dli_03_0179__en-us_topic_0000001250602862_ul35131759141614"><li id="dli_03_0179__en-us_topic_0000001250602862_li951310596166"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1598718651714">Check Whether a Port Number Is Added to the End of the Domain Name or IP Address</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li5863713181712"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1248131717217">Check Whether the Information of the Peer VPC and Subnet Are Correct.</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li121541721161712"><a href="#dli_03_0179__en-us_topic_0000001250602862_section11573112152719">Check Whether the CIDR Block of the Queue Overlaps with That of the Data Source</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li9879124141717"><a href="#dli_03_0179__en-us_topic_0000001250602862_section10740107124612">Check Whether the VPC Administrator Permission Is Granted to DLI</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li2841248101715"><a href="#dli_03_0179__en-us_topic_0000001250602862_section96661220175613">Check Whether the Destination Security Group Allows Access from the CIDR Block of the Queue</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li983385313173"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1510101713111">Check the Route Information of the VPC Peering Connection Corresponding to an Enhanced Datasource Connection</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li5985121815"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1789710470256">Check Whether VPC Network ACL Rules Are Configured to Restrict Network Access</a></li></ul> <ul id="dli_03_0179__en-us_topic_0000001250602862_ul35131759141614"><li id="dli_03_0179__en-us_topic_0000001250602862_li951310596166"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1598718651714">Check Whether a Port Number Is Added to the End of the Domain Name or IP Address</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li5863713181712"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1248131717217">Check Whether the Information of the Peer VPC and Subnet Are Correct.</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li121541721161712"><a href="#dli_03_0179__en-us_topic_0000001250602862_section11573112152719">Check Whether the CIDR Block of the Queue Overlaps with That of the Data Source</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li9879124141717"><a href="#dli_03_0179__en-us_topic_0000001250602862_section10740107124612">Check Whether the DLI Datasource Connections Agency Access Permission Is Granted to DLI</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li2841248101715"><a href="#dli_03_0179__en-us_topic_0000001250602862_section96661220175613">Check Whether the Destination Security Group Allows Access from the CIDR Block of the Queue</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li983385313173"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1510101713111">Check the Route Information of the VPC Peering Connection Corresponding to an Enhanced Datasource Connection</a></li><li id="dli_03_0179__en-us_topic_0000001250602862_li5985121815"><a href="#dli_03_0179__en-us_topic_0000001250602862_section1789710470256">Check Whether VPC Network ACL Rules Are Configured to Restrict Network Access</a></li></ul>
</div> </div>
<div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section1598718651714"><a name="dli_03_0179__en-us_topic_0000001250602862_section1598718651714"></a><a name="en-us_topic_0000001250602862_section1598718651714"></a><h4 class="sectiontitle">Check Whether a Port Number Is Added to the End of the Domain Name or IP Address</h4><p id="dli_03_0179__en-us_topic_0000001250602862_p318024811715">The port number is required for the connectivity test.</p> <div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section1598718651714"><a name="dli_03_0179__en-us_topic_0000001250602862_section1598718651714"></a><a name="en-us_topic_0000001250602862_section1598718651714"></a><h4 class="sectiontitle">Check Whether a Port Number Is Added to the End of the Domain Name or IP Address</h4><p id="dli_03_0179__en-us_topic_0000001250602862_p318024811715">The port number is required for the connectivity test.</p>
<p id="dli_03_0179__en-us_topic_0000001250602862_p115592911819">The following example tests the connectivity between a queue and a specified RDS DB instance. The RDS DB instance uses port 3306.</p> <p id="dli_03_0179__en-us_topic_0000001250602862_p115592911819">The following example tests the connectivity between a queue and a specified RDS DB instance. The RDS DB instance uses port 3306.</p>
@ -21,8 +21,8 @@
<p id="dli_03_0179__en-us_topic_0000001250602862_p183311012195218"><strong id="dli_03_0179__en-us_topic_0000001250602862_b2552714155610">Solution</strong>: Modify the CIDR block of the queue or create another queue.</p> <p id="dli_03_0179__en-us_topic_0000001250602862_p183311012195218"><strong id="dli_03_0179__en-us_topic_0000001250602862_b2552714155610">Solution</strong>: Modify the CIDR block of the queue or create another queue.</p>
<p id="dli_03_0179__en-us_topic_0000001250602862_p81371249195210">Planing the CIDR blocks for your queues helps you to avoid this problem.</p> <p id="dli_03_0179__en-us_topic_0000001250602862_p81371249195210">Planing the CIDR blocks for your queues helps you to avoid this problem.</p>
</div> </div>
<div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section10740107124612"><a name="dli_03_0179__en-us_topic_0000001250602862_section10740107124612"></a><a name="en-us_topic_0000001250602862_section10740107124612"></a><h4 class="sectiontitle">Check Whether the VPC Administrator Permission Is Granted to DLI</h4><p id="dli_03_0179__en-us_topic_0000001250602862_p11740107154611">View the connection logs to check whether there is the required permission.</p> <div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section10740107124612"><a name="dli_03_0179__en-us_topic_0000001250602862_section10740107124612"></a><a name="en-us_topic_0000001250602862_section10740107124612"></a><h4 class="sectiontitle">Check Whether the DLI Datasource Connections Agency Access Permission Is Granted to DLI</h4><p id="dli_03_0179__en-us_topic_0000001250602862_p11740107154611">You can determine if a connection failure is due to insufficient permissions by checking the connection logs.</p>
<p id="dli_03_0179__en-us_topic_0000001250602862_p1541877125312"><strong id="dli_03_0179__en-us_topic_0000001250602862_b11618836067">Solution</strong>: Grant DLI the VPC Administrator permission and cancel the IAM ReadOnlyAccess authorization.</p> <p id="dli_03_0179__en-us_topic_0000001250602862_p1541877125312">Solution: Add the <strong id="dli_03_0179__en-us_topic_0000001250602862_b1515644901916">DLI Datasource Connections Agency Access</strong> authorization on the <strong id="dli_03_0179__en-us_topic_0000001250602862_b77421037122012">Global Configuration</strong> &gt; <strong id="dli_03_0179__en-us_topic_0000001250602862_b186335441201">Service Authorization</strong> page.</p>
</div> </div>
<div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section96661220175613"><a name="dli_03_0179__en-us_topic_0000001250602862_section96661220175613"></a><a name="en-us_topic_0000001250602862_section96661220175613"></a><h4 class="sectiontitle">Check Whether the Destination Security Group Allows Access from the CIDR Block of the Queue</h4><div class="p" id="dli_03_0179__en-us_topic_0000001250602862_p3585134125816">To connect to Kafka, GaussDB(DWS), and RDS instances, add security group rules for the DLI CIDR block to the security group where the instances belong. For example, to connect a queue to RDS, perform the following operations:<ol id="dli_03_0179__en-us_topic_0000001250602862_ol89018481616"><li id="dli_03_0179__en-us_topic_0000001250602862_li9887123885818">Log in to the DLI console, choose <strong id="dli_03_0179__en-us_topic_0000001250602862_b207721648161110">Resources</strong> &gt; <strong id="dli_03_0179__en-us_topic_0000001250602862_b97922331393">Queue Management</strong> in the navigation pane on the left. On the displayed page, select the target queue, and click <span><img id="dli_03_0179__en-us_topic_0000001250602862_image105154539180" src="en-us_image_0000001428187933.png"></span> to expand the row containing the target queue to view its CIDR block.</li><li id="dli_03_0179__en-us_topic_0000001250602862_li588753820589">On the <strong id="dli_03_0179__en-us_topic_0000001250602862_b145942516124">Instance Management </strong>page of the RDS console, click the instance name. In the <strong id="dli_03_0179__en-us_topic_0000001250602862_b059172571219">Connection Information</strong> area, locate <strong id="dli_03_0179__en-us_topic_0000001250602862_b15591125111211">Database Port</strong> to obtain the port number of the RDS DB instance.</li><li id="dli_03_0179__en-us_topic_0000001250602862_li108877382588">In the <strong id="dli_03_0179__en-us_topic_0000001250602862_b14874172881220">Connection Information</strong> area locate the<strong id="dli_03_0179__en-us_topic_0000001250602862_b78741928101211"> Security Group</strong> and click the group name to switch to the security group management page. Select the <strong id="dli_03_0179__en-us_topic_0000001250602862_b14251731161210">Inbound Rules</strong> tab and click <strong id="dli_03_0179__en-us_topic_0000001250602862_b84261031131210">Add Rule</strong>. Set the priority to 1, protocol to TCP, port to the database port number, and source to the CIDR block of the DLI queue. Click <strong id="dli_03_0179__en-us_topic_0000001250602862_b516614403125">OK</strong>.</li></ol> <div class="section" id="dli_03_0179__en-us_topic_0000001250602862_section96661220175613"><a name="dli_03_0179__en-us_topic_0000001250602862_section96661220175613"></a><a name="en-us_topic_0000001250602862_section96661220175613"></a><h4 class="sectiontitle">Check Whether the Destination Security Group Allows Access from the CIDR Block of the Queue</h4><div class="p" id="dli_03_0179__en-us_topic_0000001250602862_p3585134125816">To connect to Kafka, GaussDB(DWS), and RDS instances, add security group rules for the DLI CIDR block to the security group where the instances belong. For example, to connect a queue to RDS, perform the following operations:<ol id="dli_03_0179__en-us_topic_0000001250602862_ol89018481616"><li id="dli_03_0179__en-us_topic_0000001250602862_li9887123885818">Log in to the DLI console, choose <strong id="dli_03_0179__en-us_topic_0000001250602862_b207721648161110">Resources</strong> &gt; <strong id="dli_03_0179__en-us_topic_0000001250602862_b97922331393">Queue Management</strong> in the navigation pane on the left. On the displayed page, select the target queue, and click <span><img id="dli_03_0179__en-us_topic_0000001250602862_image105154539180" src="en-us_image_0000001428187933.png"></span> to expand the row containing the target queue to view its CIDR block.</li><li id="dli_03_0179__en-us_topic_0000001250602862_li588753820589">On the <strong id="dli_03_0179__en-us_topic_0000001250602862_b145942516124">Instance Management </strong>page of the RDS console, click the instance name. In the <strong id="dli_03_0179__en-us_topic_0000001250602862_b059172571219">Connection Information</strong> area, locate <strong id="dli_03_0179__en-us_topic_0000001250602862_b15591125111211">Database Port</strong> to obtain the port number of the RDS DB instance.</li><li id="dli_03_0179__en-us_topic_0000001250602862_li108877382588">In the <strong id="dli_03_0179__en-us_topic_0000001250602862_b14874172881220">Connection Information</strong> area locate the<strong id="dli_03_0179__en-us_topic_0000001250602862_b78741928101211"> Security Group</strong> and click the group name to switch to the security group management page. Select the <strong id="dli_03_0179__en-us_topic_0000001250602862_b14251731161210">Inbound Rules</strong> tab and click <strong id="dli_03_0179__en-us_topic_0000001250602862_b84261031131210">Add Rule</strong>. Set the priority to 1, protocol to TCP, port to the database port number, and source to the CIDR block of the DLI queue. Click <strong id="dli_03_0179__en-us_topic_0000001250602862_b516614403125">OK</strong>.</li></ol>
</div> </div>

2
docs/dli/umn/dli_03_0184.html
View File

@ -6,7 +6,7 @@
<div class="section" id="dli_03_0184__en-us_topic_0000001266163792_section14394625104614"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0184__en-us_topic_0000001266163792_p10211727144615">If a table exists but cannot be queried, there is a high probability that the current user does not have the permission to query or operate the table.</p> <div class="section" id="dli_03_0184__en-us_topic_0000001266163792_section14394625104614"><h4 class="sectiontitle">Possible Causes</h4><p id="dli_03_0184__en-us_topic_0000001266163792_p10211727144615">If a table exists but cannot be queried, there is a high probability that the current user does not have the permission to query or operate the table.</p>
</div> </div>
<div class="section" id="dli_03_0184__en-us_topic_0000001266163792_section106016561456"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0184__en-us_topic_0000001266163792_p86996582452">Contact the user who creates the table and obtain the required permissions. To assign permissions, perform the following steps:</p> <div class="section" id="dli_03_0184__en-us_topic_0000001266163792_section106016561456"><h4 class="sectiontitle">Solution</h4><p id="dli_03_0184__en-us_topic_0000001266163792_p86996582452">Contact the user who creates the table and obtain the required permissions. To assign permissions, perform the following steps:</p>
<ol id="dli_03_0184__en-us_topic_0000001266163792_ol1086392218489"><li id="dli_03_0184__en-us_topic_0000001266163792_li486314227483">Log in to the DLI management console as the user who creates the table. Choose <strong id="dli_03_0184__en-us_topic_0000001266163792_b1682785994320">Data Management</strong> &gt; <strong id="dli_03_0184__en-us_topic_0000001266163792_b365091912456">Databases and Tables</strong> form the navigation pane on the left.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li15360122910507">Click the database name. The table management page is displayed. In the <strong id="dli_03_0184__en-us_topic_0000001266163792_b10763185714455">Operation</strong> column of the target table, click <strong id="dli_03_0184__en-us_topic_0000001266163792_b1565213153466">Permissions</strong>. The table permission management page is displayed.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li6107624195115">Click <strong id="dli_03_0184__en-us_topic_0000001266163792_b199931315465">Set Permission</strong>. In the displayed dialog box, set <strong id="dli_03_0184__en-us_topic_0000001266163792_b10746175694620">Authorization Object</strong> to <strong id="dli_03_0184__en-us_topic_0000001266163792_b1762010154718">User</strong>, set <strong id="dli_03_0184__en-us_topic_0000001266163792_b10428610144711">Username</strong> to the name of the user that requires the permission, and select the required permissions. For example, <strong id="dli_03_0184__en-us_topic_0000001266163792_b84521433104814">Select Table</strong> and <strong id="dli_03_0184__en-us_topic_0000001266163792_b8262164624810">Insert</strong> permissions.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li2056373615528">Click <strong id="dli_03_0184__en-us_topic_0000001266163792_b2030812546486">OK</strong>.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li892514486525">Log in to the DLI console as the user that has been granted permission and check whether the table can be queried.</li></ol> <ol id="dli_03_0184__en-us_topic_0000001266163792_ol1086392218489"><li id="dli_03_0184__en-us_topic_0000001266163792_li486314227483">Log in to the DLI management console as the user who creates the table. Choose <strong id="dli_03_0184__en-us_topic_0000001266163792_b17980191111152">Data Management</strong> &gt; <strong id="dli_03_0184__en-us_topic_0000001266163792_b098113113157">Databases and Tables</strong> form the navigation pane on the left.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li15360122910507">Click the database name. The table management page is displayed. In the <strong id="dli_03_0184__en-us_topic_0000001266163792_b10763185714455">Operation</strong> column of the target table, click <strong id="dli_03_0184__en-us_topic_0000001266163792_b1565213153466">Permissions</strong>. The table permission management page is displayed.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li6107624195115">Click <strong id="dli_03_0184__en-us_topic_0000001266163792_b199931315465">Set Permission</strong>. In the displayed dialog box, set <strong id="dli_03_0184__en-us_topic_0000001266163792_b10746175694620">Authorization Object</strong> to <strong id="dli_03_0184__en-us_topic_0000001266163792_b1762010154718">User</strong>, set <strong id="dli_03_0184__en-us_topic_0000001266163792_b10428610144711">Username</strong> to the name of the user that requires the permission, and select the required permissions. For example, <strong id="dli_03_0184__en-us_topic_0000001266163792_b84521433104814">Select Table</strong> and <strong id="dli_03_0184__en-us_topic_0000001266163792_b8262164624810">Insert</strong> permissions.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li2056373615528">Click <strong id="dli_03_0184__en-us_topic_0000001266163792_b2030812546486">OK</strong>.</li><li id="dli_03_0184__en-us_topic_0000001266163792_li892514486525">Log in to the DLI console as the user that has been granted permission and check whether the table can be queried.</li></ol>
</div> </div>
</div> </div>
<div> <div>

2
docs/dli/umn/dli_03_0196.html
View File

@ -28,7 +28,7 @@
<div class="p" id="dli_03_0196__en-us_topic_0000001299622958_p0727143963920">Shuffle data skew is caused by unbalanced number of key values in join.<ol id="dli_03_0196__en-us_topic_0000001299622958_ol14564192912014"><li id="dli_03_0196__en-us_topic_0000001299622958_li1640316263209">Perform <strong id="dli_03_0196__en-us_topic_0000001299622958_b13316106275">group by</strong> and <strong id="dli_03_0196__en-us_topic_0000001299622958_b64301412202715">count</strong> on a join to collect statistics on the number of key values of each join. The following is an example:<p id="dli_03_0196__en-us_topic_0000001299622958_p12194155618229">Join table <strong id="dli_03_0196__en-us_topic_0000001299622958_b699817249284">lefttbl</strong> and table <strong id="dli_03_0196__en-us_topic_0000001299622958_b916702852810">righttbl</strong>. <strong id="dli_03_0196__en-us_topic_0000001299622958_b48723852813">num</strong> in the <strong id="dli_03_0196__en-us_topic_0000001299622958_b10979124310284">lefttbl</strong> table is the key value of the join. You can perform <strong id="dli_03_0196__en-us_topic_0000001299622958_b2598550133012">group by</strong> and <strong id="dli_03_0196__en-us_topic_0000001299622958_b1174125663016">count</strong> on <strong id="dli_03_0196__en-us_topic_0000001299622958_b1579237103116">lefttbl.num</strong>.</p> <div class="p" id="dli_03_0196__en-us_topic_0000001299622958_p0727143963920">Shuffle data skew is caused by unbalanced number of key values in join.<ol id="dli_03_0196__en-us_topic_0000001299622958_ol14564192912014"><li id="dli_03_0196__en-us_topic_0000001299622958_li1640316263209">Perform <strong id="dli_03_0196__en-us_topic_0000001299622958_b13316106275">group by</strong> and <strong id="dli_03_0196__en-us_topic_0000001299622958_b64301412202715">count</strong> on a join to collect statistics on the number of key values of each join. The following is an example:<p id="dli_03_0196__en-us_topic_0000001299622958_p12194155618229">Join table <strong id="dli_03_0196__en-us_topic_0000001299622958_b699817249284">lefttbl</strong> and table <strong id="dli_03_0196__en-us_topic_0000001299622958_b916702852810">righttbl</strong>. <strong id="dli_03_0196__en-us_topic_0000001299622958_b48723852813">num</strong> in the <strong id="dli_03_0196__en-us_topic_0000001299622958_b10979124310284">lefttbl</strong> table is the key value of the join. You can perform <strong id="dli_03_0196__en-us_topic_0000001299622958_b2598550133012">group by</strong> and <strong id="dli_03_0196__en-us_topic_0000001299622958_b1174125663016">count</strong> on <strong id="dli_03_0196__en-us_topic_0000001299622958_b1579237103116">lefttbl.num</strong>.</p>
<pre class="screen" id="dli_03_0196__en-us_topic_0000001299622958_screen61941556102213">SELECT * FROM lefttbl a LEFT join righttbl b on a.num = b.int2; <pre class="screen" id="dli_03_0196__en-us_topic_0000001299622958_screen61941556102213">SELECT * FROM lefttbl a LEFT join righttbl b on a.num = b.int2;
SELECT count(1) as count,num from lefttbl group by lefttbl.num ORDER BY count desc;</pre> SELECT count(1) as count,num from lefttbl group by lefttbl.num ORDER BY count desc;</pre>
</li><li id="dli_03_0196__en-us_topic_0000001299622958_li1719485619227">Use <strong id="dli_03_0196__en-us_topic_0000001299622958_b1432910173515">concat(cast(round(rand() * 999999999) as string)</strong> to generate a random number for each key value.</li><li id="dli_03_0196__en-us_topic_0000001299622958_li6869616102618">If the skew is serious and random numbers cannot be generated, see <a href="dli_03_0093.html#dli_03_0093">How Do I Eliminate Data Skew by Configuring AE Parameters?</a></li></ol> </li><li id="dli_03_0196__en-us_topic_0000001299622958_li1719485619227">Use <strong id="dli_03_0196__en-us_topic_0000001299622958_b1432910173515">concat(cast(round(rand() * 999999999) as string)</strong> to generate a random number for each key value.</li><li id="dli_03_0196__en-us_topic_0000001299622958_li6869616102618">If the skew is serious and random numbers cannot be generated, see <a href="dli_03_0093.html#dli_03_0093">How Do I Do When Data Skew Occurs During the Execution of a SQL Job?</a></li></ol>
</div> </div>
</div> </div>
</div> </div>

2
docs/dli/umn/dli_03_0211.html
View File

@ -11,7 +11,7 @@
</li> </li>
<li class="ulchildlink"><strong><a href="dli_03_0116.html">How Do I View SQL Execution Records?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0116.html">How Do I View SQL Execution Records?</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_03_0093.html">How Do I Eliminate Data Skew by Configuring AE Parameters?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0093.html">How Do I Do When Data Skew Occurs During the Execution of a SQL Job?</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_03_0184.html">What Can I Do If a Table Cannot Be Queried on the DLI Console?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0184.html">What Can I Do If a Table Cannot Be Queried on the DLI Console?</a></strong><br>
</li> </li>

2
docs/dli/umn/dli_03_0221.html
View File

@ -5,8 +5,6 @@
</div> </div>
<div> <div>
<ul class="ullinks"> <ul class="ullinks">
<li class="ulchildlink"><strong><a href="dli_03_0002.html">What Is DLI?</a></strong><br>
</li>
<li class="ulchildlink"><strong><a href="dli_03_0025.html">Which Data Formats Does DLI Support?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0025.html">Which Data Formats Does DLI Support?</a></strong><br>
</li> </li>
<li class="ulchildlink"><strong><a href="dli_03_0115.html">What Are the Differences Between MRS Spark and DLI Spark?</a></strong><br> <li class="ulchildlink"><strong><a href="dli_03_0115.html">What Are the Differences Between MRS Spark and DLI Spark?</a></strong><br>

Some files were not shown because too many files have changed in this diff Show More