vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_04_0395.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

72 lines
18 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001188323572"></a><a name="EN-US_TOPIC_0000001188323572"></a>
<h1 class="topictitle1">Monitoring Memory Resources</h1>
<div id="body1500283119238"><div class="section" id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_en-us_topic_0066857609_section61066329171513"><h4 class="sectiontitle">Monitoring the Memory</h4><p id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_en-us_topic_0066857609_p47425651171513"><span id="EN-US_TOPIC_0000001188323572__ph1692512422530">GaussDB(DWS)</span> provides a view for monitoring the memory usage of the entire cluster.</p>
<div class="p" id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_en-us_topic_0066857609_p29111811171513">Query the pgxc_total_memory_detail view as a user with sysadmin permissions.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_en-us_topic_0066857609_screen60679707171513"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">pgxc_total_memory_detail</span><span class="p">;</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="p" id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_en-us_topic_0066857609_p16109207171513">If the following error message is returned during the query, enable the memory management function.<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_en-us_topic_0066857609_screen29777416171513"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">pgxc_total_memory_detail</span><span class="p">;</span>
<span class="n">ERROR</span><span class="p">:</span><span class="w"> </span><span class="n">unsupported</span><span class="w"> </span><span class="k">view</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">memory</span><span class="w"> </span><span class="n">protection</span><span class="w"> </span><span class="n">feature</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">disabled</span><span class="p">.</span>
<span class="n">CONTEXT</span><span class="p">:</span><span class="w"> </span><span class="n">PL</span><span class="o">/</span><span class="n">pgSQL</span><span class="w"> </span><span class="k">function</span><span class="w"> </span><span class="n">pgxc_total_memory_detail</span><span class="p">()</span><span class="w"> </span><span class="n">line</span><span class="w"> </span><span class="mi">12</span><span class="w"> </span><span class="k">at</span><span class="w"> </span><span class="k">FOR</span><span class="w"> </span><span class="n">over</span><span class="w"> </span><span class="k">EXECUTE</span><span class="w"> </span><span class="k">statement</span>
</pre></div></td></tr></table></div>
</div>
</div>
<div class="p" id="EN-US_TOPIC_0000001188323572__p1486513481171">You can set <strong id="EN-US_TOPIC_0000001188323572__b1531952419">enable_memory_limit</strong> and <strong id="EN-US_TOPIC_0000001188323572__b7412386414">max_process_memory</strong> on the GaussDB(DWS) console to enable memory management. The procedure is as follows:<ol id="EN-US_TOPIC_0000001188323572__en-us_topic_0000001082119785_ol142791348192513"><li id="EN-US_TOPIC_0000001188323572__en-us_topic_0000001082119785_l8f022bf0765f4461bc8d570483dcc35e">Log in to the GaussDB(DWS) management console.</li><li id="EN-US_TOPIC_0000001188323572__en-us_topic_0000001082119785_l2216db083add4f2f82560175b48a0cba">In the navigation pane on the left, click <span class="uicontrol" id="EN-US_TOPIC_0000001188323572__uicontrol1885118101015"><b>Clusters</b></span>.</li><li id="EN-US_TOPIC_0000001188323572__en-us_topic_0000001082119785_l347a21ae01ab4ead91c140de2c4844f7">In the cluster list, find the target cluster and click its name. The <span class="wintitle" id="EN-US_TOPIC_0000001188323572__wintitle119518920101"><b>Basic Information</b></span> page is displayed.</li><li id="EN-US_TOPIC_0000001188323572__en-us_topic_0000001082119785_l5e21be442af1463bb06101cacce34dd0">Click the <span class="parmname" id="EN-US_TOPIC_0000001188323572__parmname1321781115103"><b>Parameter Modification</b></span> tab, change the value of <strong id="EN-US_TOPIC_0000001188323572__b121711113103">enable_memory_limit</strong> to <strong id="EN-US_TOPIC_0000001188323572__b1217161114101">on</strong>, and click <span class="uicontrol" id="EN-US_TOPIC_0000001188323572__uicontrol202181811141016"><b>Save</b></span> to save the file.</li><li id="EN-US_TOPIC_0000001188323572__li085454618917">Change the value of <strong id="EN-US_TOPIC_0000001188323572__b19738192761020">max_process_memory</strong> to a proper one. For details about the modification suggestions, see <a href="dws_04_0893.html#EN-US_TOPIC_0000001188163786__sadc1e0e8c1c246a4a6cad3967deebaad">max_process_memory</a>. After it is done, click <span class="uicontrol" id="EN-US_TOPIC_0000001188323572__uicontrol2325195025817"><b>Save</b></span>.</li><li id="EN-US_TOPIC_0000001188323572__en-us_topic_0000001082119785_la56c2d28d9c446e39e3763c95cc387a6">In the <span class="uicontrol" id="EN-US_TOPIC_0000001188323572__uicontrol990392801817"><b>Modification Preview</b></span> dialog box, confirm the modifications and click <span class="uicontrol" id="EN-US_TOPIC_0000001188323572__uicontrol2090302801815"><b>Save</b></span>. After the modification, restart the cluster for the modification to take effect.</li></ol>
</div>
</div>
<div class="section" id="EN-US_TOPIC_0000001188323572__s662c3805a0f644fd8e8a5bd0183069b8"><h4 class="sectiontitle">Monitoring the Shared Memory</h4><p id="EN-US_TOPIC_0000001188323572__ad270a83cd9b94369ad9c95bd9007170d">You can query the context information about the shared memory on the pg_shared_memory_detail view.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188323572__s428dc7ef90404e398304791ab71db86b"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span>
<span class="normal">6</span>
<span class="normal">7</span>
<span class="normal">8</span>
<span class="normal">9</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">pg_shared_memory_detail</span><span class="p">;</span>
<span class="w"> </span><span class="n">contextname</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">level</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">parent</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">totalsize</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">freesize</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">usedsize</span><span class="w"> </span>
<span class="c1">---------------------------------+-------+---------------------------------+-----------+----------+----------</span>
<span class="w"> </span><span class="n">ProcessMemory</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">24576</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">9840</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">14736</span>
<span class="w"> </span><span class="n">Workload</span><span class="w"> </span><span class="n">manager</span><span class="w"> </span><span class="n">memory</span><span class="w"> </span><span class="n">context</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">1</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">ProcessMemory</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">2105400</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">7304</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">2098096</span>
<span class="w"> </span><span class="n">wlm</span><span class="w"> </span><span class="n">collector</span><span class="w"> </span><span class="n">hash</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Workload</span><span class="w"> </span><span class="n">manager</span><span class="w"> </span><span class="n">memory</span><span class="w"> </span><span class="n">context</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">8192</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">3736</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">4456</span>
<span class="w"> </span><span class="n">Resource</span><span class="w"> </span><span class="n">pool</span><span class="w"> </span><span class="n">hash</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Workload</span><span class="w"> </span><span class="n">manager</span><span class="w"> </span><span class="n">memory</span><span class="w"> </span><span class="n">context</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">24576</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">15968</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">8608</span>
<span class="w"> </span><span class="n">wlm</span><span class="w"> </span><span class="n">cgroup</span><span class="w"> </span><span class="n">hash</span><span class="w"> </span><span class="k">table</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">2</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Workload</span><span class="w"> </span><span class="n">manager</span><span class="w"> </span><span class="n">memory</span><span class="w"> </span><span class="n">context</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">24576</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">15968</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="mi">8608</span>
<span class="p">(</span><span class="mi">5</span><span class="w"> </span><span class="k">rows</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188323572__af0763b86c6964e68902130e0261ba03e">This view lists the context name of the memory, level, the upper-layer memory context, and the total size of the shared memory.</p>
<p id="EN-US_TOPIC_0000001188323572__a247c9711054445a5aa1ca109bd56a0e7">In the database, GUC parameter <strong id="EN-US_TOPIC_0000001188323572__en-us_topic_0058967736_b84235270610538">memory_tracking_mode</strong> is used to configure the memory statistics collecting mode, including the following options:</p>
<ul id="EN-US_TOPIC_0000001188323572__udc58917825804f78a086d1d8ce6c97d5"><li id="EN-US_TOPIC_0000001188323572__l7ecbd02d98444f5c99c42f03d9c0ef25"><strong id="EN-US_TOPIC_0000001188323572__b842352706105333">none:</strong> The memory statistics collecting function is not enabled.</li><li id="EN-US_TOPIC_0000001188323572__l3d4481464551428781d80fde7388a8b9"><strong id="EN-US_TOPIC_0000001188323572__b842352706105432">normal:</strong> Only memory statistics is collected in real time and no file is generated.</li><li id="EN-US_TOPIC_0000001188323572__lee95f586c2dd49aab084ad33befb0227"><strong id="EN-US_TOPIC_0000001188323572__b842352706105529">executor:</strong> The statistics file is generated, containing the context information about all allocated memory used on the execution layer.<p id="EN-US_TOPIC_0000001188323572__af0ec9c757314412c8f5599cbec3bdbbd">When the parameter is set to <strong id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_b84235270610566">executor</strong>, cvs files are generated under the <strong id="EN-US_TOPIC_0000001188323572__en-us_topic_0085031790_b842352706111914">pg_log</strong> directory of the DN process. The file names are in the format of <strong id="EN-US_TOPIC_0000001188323572__b84235270617447">memory_track_</strong><em id="EN-US_TOPIC_0000001188323572__i84235269717453">&lt;DN name&gt;</em><strong id="EN-US_TOPIC_0000001188323572__b84235270617457">_query_</strong><em id="EN-US_TOPIC_0000001188323572__i8423526971750">&lt;queryid&gt;</em><strong id="EN-US_TOPIC_0000001188323572__b8423527061755">.csv</strong>. The information about the operators executed by the postgres thread of the executor and all stream threads are input in this file during task execution.</p>
<p id="EN-US_TOPIC_0000001188323572__afa62eb4d46da4be5b95c4c6c5c0f279f">The instance is built with a file content similar to the following:</p>
<pre class="screen" id="EN-US_TOPIC_0000001188323572__s6d973bac4d63495fab90e959c10e2002">0, 0, ExecutorState, 0, PortalHeapMemory, 0, 40K, 602K, 23
1, 3, CStoreScan_29360131_25, 0, ExecutorState, 1, 265K, 554K, 23
2, 128, cstore scan per scan memory context, 1, CStoreScan_29360131_25, 2, 24K, 24K, 23
3, 127, cstore scan memory context, 1, CStoreScan_29360131_25, 2, 264K, 264K, 23
4, 7, InitPartitionMapTmpMemoryContext, 1, CStoreScan_29360131_25, 2, 31K, 31K, 23
5, 2, VecPartIterator_29360131_24, 0, ExecutorState, 1, 16K, 16K, 23
0, 0, ExecutorState, 0, PortalHeapMemory, 0, 24K, 1163K, 20
<strong id="EN-US_TOPIC_0000001188323572__acaa42aec637d4ca69a4ebec195d6dda9">1, 3, CStoreScan_29360131_22, 0, ExecutorState, 1, 390K, 1122K, 20</strong>
2, 20, cstore scan per scan memory context, 1, CStoreScan_29360131_22, 2, 476K, 476K, 20
3, 19, cstore scan memory context, 1, CStoreScan_29360131_22, 2, 264K, 264K, 20
4, 7, InitPartitionMapTmpMemoryContext, 1, CStoreScan_29360131_22, 2, 23K, 23K, 20
5, 2, VecPartIterator_29360131_21, 0, ExecutorState, 1, 16K, 16K, 20</pre>
<p id="EN-US_TOPIC_0000001188323572__a27c39f01943a4970a0c390b3c93ceab7">The fields include the output SN, SN of the memory allocation context within the thread, name of the current memory context, output SN of the parent memory context, name of the parent memory context, tree layer No. of the memory context, peak memory used by the current memory context, peak memory used by the current memory context and all its child memory contexts, and plan node ID of the query where the thread is executed.</p>
<p id="EN-US_TOPIC_0000001188323572__a976eb0d1fd2a4702a7aa92ae087b4e8f">In this example, the record "1, 3, CStoreScan_29360131_22, 0, ExecutorState, 1, 390K, 1122K, 20" represents the following information about Explain Analyze:</p>
<ul id="EN-US_TOPIC_0000001188323572__u6aa1f5ab96bc449989a810b13a417d74"><li id="EN-US_TOPIC_0000001188323572__le5117e52f5fd4a069a0169ec42cacafa"><strong id="EN-US_TOPIC_0000001188323572__b842352706112642">CstoreScan_29360131_22</strong> indicates the CstoreScan operator.</li><li id="EN-US_TOPIC_0000001188323572__l5b2618a1cff1435c872b9e211bdae883"><strong id="EN-US_TOPIC_0000001188323572__b842352706112657">1122K</strong> indicates the peak memory used by the CstoreScan operator.</li></ul>
</li><li id="EN-US_TOPIC_0000001188323572__ldbc1a3d84ddf4243a0d8ccbb6896113b"><strong id="EN-US_TOPIC_0000001188323572__b842352706112725">fullexec:</strong> The generated file includes the information about all memory contexts requested by the execution layer.<p id="EN-US_TOPIC_0000001188323572__a55fa01011f47429c805901437c20c632">If the parameter is set to <strong id="EN-US_TOPIC_0000001188323572__b19672132102811">fullexec</strong>, the output information will be similar to that for <strong id="EN-US_TOPIC_0000001188323572__b66751132202810">executor</strong>, except that some memory context allocation information may be returned because the information about all memory applications (no matter succeeded or not) is printed. As only the memory application information is recorded, the peak memory used by the memory context is recorded as <strong id="EN-US_TOPIC_0000001188323572__b90567601511346">0</strong>.</p>
</li></ul>
</div>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_04_0393.html">Resource Monitoring</a></div>
</div>
</div>
View Git Blame Copy Permalink