vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0450.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

64 lines
14 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001233761793"></a><a name="EN-US_TOPIC_0000001233761793"></a>
<h1 class="topictitle1">Optimizing Operators</h1>
<div id="body8662426"><div class="section" id="EN-US_TOPIC_0000001233761793__s347f3836f37846e6b92dd137ea0aa107"><h4 class="sectiontitle">What Is Operator Optimization</h4><p id="EN-US_TOPIC_0000001233761793__a123cd839a97d40a484bb6ff214260008">A query statement needs to go through multiple operator procedures to generate the final result. Sometimes, the overall query performance deteriorates due to long execution time of certain operators, which are regarded as bottleneck operators. In this case, you need to execute the <strong id="EN-US_TOPIC_0000001233761793__b842352706131458">EXPLAIN ANALYZE/PERFORMANCE</strong> command to view the bottleneck operators, and then perform optimization.</p>
<p id="EN-US_TOPIC_0000001233761793__abf45952235fa439a8bb1587ec03b334f">For example, in the following execution process, the execution time of the <strong id="EN-US_TOPIC_0000001233761793__b842352706131745">Hashagg</strong> operator accounts for about 66% [(51016-13535)/56476 ≈ 66%] of the total execution time. Therefore, the <strong id="EN-US_TOPIC_0000001233761793__b765568739132035">Hashagg</strong> operator is the bottleneck operator for this query. Optimize this operator first.</p>
<p id="EN-US_TOPIC_0000001233761793__p17461342114316"><span><img id="EN-US_TOPIC_0000001233761793__image2460104215435" src="figure/en-us_image_0000001233563361.jpg"></span></p>
</div>
<div class="section" id="EN-US_TOPIC_0000001233761793__sa87a94f567c344fda7bda3f0dcd3de5a"><h4 class="sectiontitle">Operator Optimization Example</h4><p id="EN-US_TOPIC_0000001233761793__en-us_topic_0073253804_p100769315447">1. Scan the base table. For queries requiring large volume of data filtering, such as point queries or queries that need range scanning, a full table scan using SeqScan will take a long time. To facilitate scanning, you can create indexes on the condition column and select IndexScan for index scanning.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001233761793__s22f7f1f3e566410791acd54a3a6232a7"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span></pre></div></td><td class="code"><div><pre><span></span><span class="w"> </span><span class="k">explain</span><span class="w"> </span><span class="p">(</span><span class="k">analyze</span><span class="w"> </span><span class="k">on</span><span class="p">,</span><span class="w"> </span><span class="n">costs</span><span class="w"> </span><span class="k">off</span><span class="p">)</span><span class="w"> </span><span class="k">select</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="n">store_sales</span><span class="w"> </span><span class="k">where</span><span class="w"> </span><span class="n">ss_sold_date_sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">2450944</span><span class="p">;</span>
<span class="w"> </span><span class="n">id</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">operation</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">A</span><span class="o">-</span><span class="k">time</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">A</span><span class="o">-</span><span class="k">rows</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Peak</span><span class="w"> </span><span class="n">Memory</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">A</span><span class="o">-</span><span class="n">width</span>
<span class="c1">----+--------------------------------+---------------------+--------+--------------+---------</span>
<span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">Streaming</span><span class="w"> </span><span class="p">(</span><span class="k">type</span><span class="p">:</span><span class="w"> </span><span class="n">GATHER</span><span class="p">)</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">3666</span><span class="p">.</span><span class="mi">020</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">3360</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">195</span><span class="n">KB</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">Seq</span><span class="w"> </span><span class="n">Scan</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="n">store_sales</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="p">[</span><span class="mi">3594</span><span class="p">.</span><span class="mi">611</span><span class="p">,</span><span class="mi">3594</span><span class="p">.</span><span class="mi">611</span><span class="p">]</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">3360</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="p">[</span><span class="mi">34</span><span class="n">KB</span><span class="p">,</span><span class="w"> </span><span class="mi">34</span><span class="n">KB</span><span class="p">]</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="n">Predicate</span><span class="w"> </span><span class="n">Information</span><span class="w"> </span><span class="p">(</span><span class="n">identified</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="n">plan</span><span class="w"> </span><span class="n">id</span><span class="p">)</span><span class="w"> </span>
<span class="c1">-----------------------------------------------</span>
<span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="c1">--Seq Scan on store_sales</span>
<span class="w"> </span><span class="n">Filter</span><span class="p">:</span><span class="w"> </span><span class="p">(</span><span class="n">ss_sold_date_sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">2450944</span><span class="p">)</span>
<span class="w"> </span><span class="k">Rows</span><span class="w"> </span><span class="n">Removed</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="n">Filter</span><span class="p">:</span><span class="w"> </span><span class="mi">4968936</span>
</pre></div></td></tr></table></div>
</div>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001233761793__sac2c9a001f284e47b8729093d1e6efa7"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span></pre></div></td><td class="code"><div><pre><span></span><span class="w"> </span><span class="k">create</span><span class="w"> </span><span class="k">index</span><span class="w"> </span><span class="n">idx</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="n">store_sales_row</span><span class="p">(</span><span class="n">ss_sold_date_sk</span><span class="p">);</span>
<span class="k">CREATE</span><span class="w"> </span><span class="k">INDEX</span>
<span class="w"> </span><span class="k">explain</span><span class="w"> </span><span class="p">(</span><span class="k">analyze</span><span class="w"> </span><span class="k">on</span><span class="p">,</span><span class="w"> </span><span class="n">costs</span><span class="w"> </span><span class="k">off</span><span class="p">)</span><span class="w"> </span><span class="k">select</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">from</span><span class="w"> </span><span class="n">store_sales_row</span><span class="w"> </span><span class="k">where</span><span class="w"> </span><span class="n">ss_sold_date_sk</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">2450944</span><span class="p">;</span>
<span class="w"> </span><span class="n">id</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">operation</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">A</span><span class="o">-</span><span class="k">time</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">A</span><span class="o">-</span><span class="k">rows</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Peak</span><span class="w"> </span><span class="n">Memory</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">A</span><span class="o">-</span><span class="n">width</span>
<span class="c1">----+------------------------------------------------+-----------------+--------+--------------+----------</span>
<span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="n">Streaming</span><span class="w"> </span><span class="p">(</span><span class="k">type</span><span class="p">:</span><span class="w"> </span><span class="n">GATHER</span><span class="p">)</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">81</span><span class="p">.</span><span class="mi">524</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">3360</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">195</span><span class="n">KB</span><span class="w"> </span><span class="o">|</span>
<span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">-&gt;</span><span class="w"> </span><span class="k">Index</span><span class="w"> </span><span class="n">Scan</span><span class="w"> </span><span class="k">using</span><span class="w"> </span><span class="n">idx</span><span class="w"> </span><span class="k">on</span><span class="w"> </span><span class="n">store_sales_row</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="p">[</span><span class="mi">13</span><span class="p">.</span><span class="mi">352</span><span class="p">,</span><span class="mi">13</span><span class="p">.</span><span class="mi">352</span><span class="p">]</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">3360</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="p">[</span><span class="mi">34</span><span class="n">KB</span><span class="p">,</span><span class="w"> </span><span class="mi">34</span><span class="n">KB</span><span class="p">]</span><span class="w"> </span><span class="o">|</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001233761793__a1adbec3966c54b8c9c48e89072e90bcd">In this example, the full table scan filters much data and returns 3360 records. After an index has been created on the <strong id="EN-US_TOPIC_0000001233761793__b842352706155245">ss_sold_date_sk</strong> column, the scanning efficiency is significantly boosted from 3.6s to 13 ms by using <strong id="EN-US_TOPIC_0000001233761793__b84235270615554">IndexScan</strong>.</p>
<p id="EN-US_TOPIC_0000001233761793__a76baf0df684e48988036e8dd736e3f88">2: If NestLoop is used for joining tables with a large number of rows, the join may take a long time. In the following example, NestLoop takes 181s. If <strong id="EN-US_TOPIC_0000001233761793__b843142163116">enable_mergejoin=off</strong> is set to disable merge join and <strong id="EN-US_TOPIC_0000001233761793__b1666891314312">enable_nestloop=off</strong> is set to disable NestLoop so that the optimizer selects hash join, the join takes more than 200 ms.</p>
<p id="EN-US_TOPIC_0000001233761793__p11840191418415"><span><img id="EN-US_TOPIC_0000001233761793__image88408142412" src="figure/en-us_image_0000001233883415.png"></span></p>
<p id="EN-US_TOPIC_0000001233761793__p71293102423"><span><img id="EN-US_TOPIC_0000001233761793__image20129161012426" src="figure/en-us_image_0000001233681853.png"></span></p>
<p id="EN-US_TOPIC_0000001233761793__a059528056ef34e0ca07fe9678e544f71">3. Generally, query performance can be improved by selecting <strong id="EN-US_TOPIC_0000001233761793__b84235270616655">HashAgg</strong>. If <strong id="EN-US_TOPIC_0000001233761793__b84235270616718">Sort</strong> and <strong id="EN-US_TOPIC_0000001233761793__b842352706193218">GroupAgg</strong> are used for a large result set, you need to set <strong id="EN-US_TOPIC_0000001233761793__b84235270616810">enable_sort</strong> to <strong id="EN-US_TOPIC_0000001233761793__b84235270616818">off</strong>. <strong id="EN-US_TOPIC_0000001233761793__b84235270616915">HashAgg</strong> consumes less time than <strong id="EN-US_TOPIC_0000001233761793__b1986216344193231">Sort</strong> and <strong id="EN-US_TOPIC_0000001233761793__b1973571036193231">GroupAgg</strong>.</p>
<p id="EN-US_TOPIC_0000001233761793__p1782144364812"><span><img id="EN-US_TOPIC_0000001233761793__image10821114354817" src="figure/en-us_image_0000001188482338.png"></span></p>
<p id="EN-US_TOPIC_0000001233761793__p13079260499"><span><img id="EN-US_TOPIC_0000001233761793__image183076265499" src="figure/en-us_image_0000001188642252.png"></span></p>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0445.html">Typical SQL Optimization Methods</a></div>
</div>
</div>
View Git Blame Copy Permalink