vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0446.html
Lu, Huayi a24ca60074 DWS DEVELOPER 811 version 
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2023-01-19 13:37:49 +00:00

72 lines
16 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001098814594"></a><a name="EN-US_TOPIC_0000001098814594"></a>
<h1 class="topictitle1">SQL Self-Diagnosis</h1>
<div id="body1530499427336"><p id="EN-US_TOPIC_0000001098814594__p11285145044511">Performance issues may occur when you query data or run the <strong id="EN-US_TOPIC_0000001098814594__b163851946164913">INSERT</strong>, <strong id="EN-US_TOPIC_0000001098814594__b15847182015020">DELETE</strong>, <strong id="EN-US_TOPIC_0000001098814594__b1782917243502">UPDATE</strong>, or <strong id="EN-US_TOPIC_0000001098814594__b9143102975016">CREATE TABLE AS</strong> statement. You can query the <strong id="EN-US_TOPIC_0000001098814594__b3540146113318">warning</strong> column in the <a href="dws_04_0706.html">GS_WLM_SESSION_STATISTICS</a>, <a href="dws_04_0705.html">GS_WLM_SESSION_HISTORY</a>, and <a href="dws_04_0566.html">GS_WLM_SESSION_INFO</a> views to obtain performance diagnosis information for tuning.</p>
<p id="EN-US_TOPIC_0000001098814594__p3467204017162">Alarms that can trigger SQL self-diagnosis depend on the settings of <a href="dws_04_0922.html#EN-US_TOPIC_0000001145694507__section153571329142612">resource_track_level</a>. If <strong id="EN-US_TOPIC_0000001098814594__b137101125193818">resource_track_level</strong> is set to <strong id="EN-US_TOPIC_0000001098814594__b1826118295386">query</strong> or <strong id="EN-US_TOPIC_0000001098814594__b2115183243810">perf</strong>, you can diagnose alarms indicating that statistics of multiple columns or a single column are not collected or SQL statements are not pushed down. If <strong id="EN-US_TOPIC_0000001098814594__b21836753915">resource_track_level</strong> is set to <strong id="EN-US_TOPIC_0000001098814594__b89217933911">operator</strong>, all alarm scenarios can be diagnosed.</p>
<p id="EN-US_TOPIC_0000001098814594__p17329109171812">Whether a SQL plan will be diagnosed depends on the settings of <a href="dws_04_0922.html#EN-US_TOPIC_0000001145694507__section1089022732713">resource_track_cost</a>. A SQL plan will be diagnosed only if its execution cost is greater than <strong id="EN-US_TOPIC_0000001098814594__b1889174383417">resource_track_cost</strong>. You can use the <strong id="EN-US_TOPIC_0000001098814594__b1858105916359">EXPLAIN</strong> keyword to check the plan execution cost.</p>
<div class="section" id="EN-US_TOPIC_0000001098814594__section1451592315913"><h4 class="sectiontitle">Alarms</h4><p id="EN-US_TOPIC_0000001098814594__p18285150154514">Currently, the following performance alarms will be reported:</p>
<ul id="EN-US_TOPIC_0000001098814594__ul17921548181010"><li id="EN-US_TOPIC_0000001098814594__li8793648191014">Some column statistics are not collected.<p id="EN-US_TOPIC_0000001098814594__p18528555185111"><a name="EN-US_TOPIC_0000001098814594__li8793648191014"></a><a name="li8793648191014"></a>If statistics of a single column or multiple columns are not collected, an alarm is reported. For details about the optimization, see <a href="dws_04_0436.html">Updating Statistics</a> and <a href="dws_04_0449.html">Optimizing Statistics</a>.</p>
<p id="EN-US_TOPIC_0000001098814594__p3793144801018">If no statistics are collected for the OBS foreign table and HDFS foreign table in the query statement, an alarm indicating that statistics are not collected will be reported. Because the <strong id="EN-US_TOPIC_0000001098814594__b16881645153515">ANALYZE</strong> performance of the OBS foreign table and HDFS foreign table is poor, you are not advised to perform <strong id="EN-US_TOPIC_0000001098814594__b9881184593510">ANALYZE</strong> on these tables. Instead, you are advised to use the <strong id="EN-US_TOPIC_0000001098814594__b118811445173518">ALTER FOREIGN TABLE</strong> syntax to modify the <strong id="EN-US_TOPIC_0000001098814594__b58825450352">totalrows</strong> attribute of the foreign table to correct the estimated number of rows.</p>
<p id="EN-US_TOPIC_0000001098814594__p117951648201020">Example alarms:</p>
<p id="EN-US_TOPIC_0000001098814594__p4795194821018">No statistics about a table are not collected.</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen207961348111019">Statistic Not Collect:
schema_test.t1</pre>
<p id="EN-US_TOPIC_0000001098814594__p379744891020">The statistics about a single column are not collected.</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen18797448111011">Statistic Not Collect:
schema_test.t2(c1)</pre>
<p id="EN-US_TOPIC_0000001098814594__p147991248161018">The statistics about multiple columns are not collected.</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen1979914881015">Statistic Not Collect:
schema_test.t3((c1,c2))</pre>
<p id="EN-US_TOPIC_0000001098814594__p1680010488108">The statistics about a single column and multiple columns are not collected.</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen188008483108">Statistic Not Collect:
schema_test.t4(c1) schema_test.t5((c1,c2))</pre>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul158021484107"><li id="EN-US_TOPIC_0000001098814594__li4802148181010">SQL statements are not pushed down.<div class="p" id="EN-US_TOPIC_0000001098814594__p168039487109"><a name="EN-US_TOPIC_0000001098814594__li4802148181010"></a><a name="li4802148181010"></a>The cause details are displayed in the alarms. For details about the optimization, see <a href="dws_04_0447.html">Optimizing Statement Pushdown</a>.<ul id="EN-US_TOPIC_0000001098814594__ul28031481105"><li id="EN-US_TOPIC_0000001098814594__li1780414871010">If the pushdown failure is caused by functions, the function names are displayed in the alarm.</li><li id="EN-US_TOPIC_0000001098814594__li980424818107">If the pushdown failure is caused by syntax, the alarm indicates that the syntax does not support pushdown. For example, the syntax containing the <strong id="EN-US_TOPIC_0000001098814594__b1761331693119">With Recursive</strong>, <strong id="EN-US_TOPIC_0000001098814594__b7316112011316">Distinct On</strong>, or <strong id="EN-US_TOPIC_0000001098814594__b362123118311">Row</strong> expression, and syntax whose return value is of record type do not support pushdown.</li></ul>
</div>
<p id="EN-US_TOPIC_0000001098814594__p4805144811108">Example alarms:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen580512486107">SQL is not plan-shipping, reason : ""enable_stream_operator" is off"
SQL is not plan-shipping, reason : ""Distinct On" can not be shipped"
SQL is not plan-shipping, reason : ""v_test_unshipping_log" is VIEW that will be treated as Record type can't be shipped"</pre>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul6807448101012"><li id="EN-US_TOPIC_0000001098814594__li1980710481108">In a hash join, the larger table is used as the inner table.<p id="EN-US_TOPIC_0000001098814594__p7808134821017"><a name="EN-US_TOPIC_0000001098814594__li1980710481108"></a><a name="li1980710481108"></a>An alarm will be reported if the number of rows in the inner table reaches or exceeds 10 times of that in the foreign table, more than 100,000 inner-table rows are processed on each DN in average, and data has been flushed to disks. You can view the <strong id="EN-US_TOPIC_0000001098814594__b1224285413115">query_plan</strong> column in <a href="dws_04_0705.html">GS_WLM_SESSION_HISTORY</a> to check whether hash joins are used. In this scenario, you need to adjust the sequence of the HashJoin internal and foreign tables. For details, see <a href="dws_04_0456.html">Join Order Hints</a>.</p>
<p id="EN-US_TOPIC_0000001098814594__p1981034815106">Example alarm:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen13947193117383">PlanNode[7] Large Table is INNER in HashJoin "Vector Hash Aggregate"</pre>
<p id="EN-US_TOPIC_0000001098814594__p1394710318387">In the preceding command, <strong id="EN-US_TOPIC_0000001098814594__b10123536123510">7</strong> indicates the operator whose ID is <strong id="EN-US_TOPIC_0000001098814594__b216817425350">7</strong> in the <strong id="EN-US_TOPIC_0000001098814594__b1945594473512">query_plan</strong> column.</p>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul17811154811106"><li id="EN-US_TOPIC_0000001098814594__li118124480107"><strong id="EN-US_TOPIC_0000001098814594__b136648167564">nestloop</strong> is used in a large-table equivalent join.<p id="EN-US_TOPIC_0000001098814594__p108121248131010">An alarm will be reported if nested loop is used in an equivalent join where more than 100,000 larger-table rows are processed on each DN in average. You can view the <strong id="EN-US_TOPIC_0000001098814594__b11671557193018">query_plan</strong> column of <a href="dws_04_0705.html">GS_WLM_SESSION_HISTORY</a> to check whether nested loop is used. In this scenario, you need to adjust the table join mode and disable the NestLoop join mode between the current internal and foreign tables. For details, see <a href="dws_04_0457.html">Join Operation Hints</a>.</p>
<p id="EN-US_TOPIC_0000001098814594__p118142048161012">Example alarm:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen6814848181012">PlanNode[5] Large Table with Equal-Condition use Nestloop"Nested Loop"</pre>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul10815194841010"><li id="EN-US_TOPIC_0000001098814594__li178159483108">A large table is broadcasted.<p id="EN-US_TOPIC_0000001098814594__p381664819104"><a name="EN-US_TOPIC_0000001098814594__li178159483108"></a><a name="li178159483108"></a>An alarm will be reported if more than 100 thousand of rows are broadcasted on each DN in average. In this scenario, the broadcast operation of the BroadCast lower-layer operator needs to be disabled. For details about the optimization, see <a href="dws_04_0459.html">Stream Operation Hints</a>.</p>
<p id="EN-US_TOPIC_0000001098814594__p1381704815100">Example alarm:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen3817174811011">PlanNode[5] Large Table in Broadcast "Streaming(type: BROADCAST dop: 1/2)"</pre>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul1281820487104"><li id="EN-US_TOPIC_0000001098814594__li7818248171010">Data skew occurs.<p id="EN-US_TOPIC_0000001098814594__p3819154821018"><a name="EN-US_TOPIC_0000001098814594__li7818248171010"></a><a name="li7818248171010"></a>An alarm will be reported if the number of rows processed on any DN exceeds 100 thousand, and the number of rows processed on a DN reaches or exceeds 10 times of that processed on another DN. Generally, this alarm is generated due to storage layer skew or computing layer skew. For details about the optimization, see <a href="dws_04_0451.html">Optimizing Data Skew</a>.</p>
<p id="EN-US_TOPIC_0000001098814594__p2820948151017">Example alarm:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen1082094818101">PlanNode[6] DataSkew:"Seq Scan", min_dn_tuples:0, max_dn_tuples:524288</pre>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul19636114414589"><li id="EN-US_TOPIC_0000001098814594__li13636164410583">The index is improper.<p id="EN-US_TOPIC_0000001098814594__p930916541614"><a name="EN-US_TOPIC_0000001098814594__li13636164410583"></a><a name="li13636164410583"></a>During base table scanning, an alarm is reported if the following conditions are met:</p>
<ul id="EN-US_TOPIC_0000001098814594__ul2048918479205"><li id="EN-US_TOPIC_0000001098814594__li19363164514202">For row-store tables:<ul id="EN-US_TOPIC_0000001098814594__ul1296675572010"><li id="EN-US_TOPIC_0000001098814594__li15234115415203">When the index scanning is used, the ratio of the number of output lines to the number of scanned lines is greater than 1/1000 and the number of output lines is greater than 10,000.</li><li id="EN-US_TOPIC_0000001098814594__li123505420206">When sequential scanning is used, the number of output lines to the number of scanned lines is less than 1/1000, the number of output lines is less than or equal to 10,000, and the number of scanned lines is greater than 10,000.</li></ul>
</li><li id="EN-US_TOPIC_0000001098814594__li11368144513206">For column-store tables:<ul id="EN-US_TOPIC_0000001098814594__ul9716195902015"><li id="EN-US_TOPIC_0000001098814594__li8369645142017">When the index scanning is used, the ratio of the number of output lines to the number of scanned lines is greater than 1/10000 and the number of output lines is greater than 100.</li><li id="EN-US_TOPIC_0000001098814594__li337194552016">When sequential scanning is used, the number of output lines to the number of scanned lines is less than 1/10,000, the number of output lines is less than or equal to 100, and the number of scanned lines is greater than 10,000.</li></ul>
</li></ul>
<p id="EN-US_TOPIC_0000001098814594__p963684435813">For details about the optimization, see <a href="dws_04_0450.html">Optimizing Operators</a>. You can also refer to <a href="dws_04_0476.html">Case: Creating an Appropriate Index</a> and <a href="dws_04_0490.html">Case: Setting Partial Cluster Keys</a>.</p>
<p id="EN-US_TOPIC_0000001098814594__p19636114410584">Example alarms:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen1932122116599">PlanNode[4] Indexscan is not properly used:"Index Only Scan", output:524288, filtered:0, rate:1.00000
PlanNode[5] Indexscan is ought to be used:"Seq Scan", output:1, filtered:524288, rate:0.00000</pre>
</li></ul>
<ul id="EN-US_TOPIC_0000001098814594__ul382164871019"><li id="EN-US_TOPIC_0000001098814594__li13821124881013">Estimation is inaccurate.<p id="EN-US_TOPIC_0000001098814594__p18221448171018"><a name="EN-US_TOPIC_0000001098814594__li13821124881013"></a><a name="li13821124881013"></a>An alarm will be reported if the maximum number or the estimated maximum number of rows processed on a DN is over 100,000, and the larger number reaches or exceeds 10 times of the smaller one. In this scenario, you can refer to <a href="dws_04_0458.html">Rows Hints</a> to correct the estimation on the number of rows, so that the optimizer can re-design the execution plan based on the correct number.</p>
<p id="EN-US_TOPIC_0000001098814594__p1082234851011">Example alarm:</p>
<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen8823174831010">PlanNode[5] Inaccurate Estimation-Rows: "Hash Join" A-Rows:0, E-Rows:52488</pre>
</li></ul>
</div>
<div class="section" id="EN-US_TOPIC_0000001098814594__section728715105125"><h4 class="sectiontitle">Restrictions</h4><ol id="EN-US_TOPIC_0000001098814594__ol3766410193914"><li id="EN-US_TOPIC_0000001098814594__li376601043910">An alarm contains a maximum of 2048 characters. If the length of an alarm exceeds this value (for example, a large number of long table names and column names are displayed in the alarm when their statistics are not collected), a warning instead of an alarm will be reported.<pre class="screen" id="EN-US_TOPIC_0000001098814594__screen8176184920816">WARNING, "Planner issue report is truncated, the rest of planner issues will be skipped"</pre>
</li><li id="EN-US_TOPIC_0000001098814594__li927851164020">If a query statement contains the <strong id="EN-US_TOPIC_0000001098814594__b26611304576">Limit</strong> operator, alarms of operators lower than <strong id="EN-US_TOPIC_0000001098814594__b166637304571">Limit</strong> will not be reported.</li><li id="EN-US_TOPIC_0000001098814594__li6364189104016">For alarms about data skew and inaccurate estimation, only alarms on the lower-layer nodes in a plan tree will be reported. This is because the same alarms on the upper-level nodes may be triggered by problems on the lower-layer nodes. For example, if data skew occurs on the <strong id="EN-US_TOPIC_0000001098814594__b11886105310294">Scan</strong> node, data skew may also occur in operators (for example, <strong id="EN-US_TOPIC_0000001098814594__b568793414315">Hashagg</strong>) at the upper layer.</li></ol>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0445.html">Typical SQL Optimization Methods</a></div>
</div>
</div>
View Git Blame Copy Permalink