forked from docs/doc-exports
Yang, Tong
48706b7552
Reviewed-by: Kacur, Michal <michal.kacur@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
67 lines
6.3 KiB
HTML
67 lines
6.3 KiB
HTML
<a name="mrs_01_24849"></a><a name="mrs_01_24849"></a>
|
|
|
|
<h1 class="topictitle1">Solution to the "Too many parts" Error in Data Tables</h1>
|
|
<div id="body0000001532995214"><div class="section" id="mrs_01_24849__section18352810171513"><h4 class="sectiontitle">Troubleshooting</h4><ol id="mrs_01_24849__ol1392864032719"><li id="mrs_01_24849__li59281340122712">Log in to the ClickHouse client and check whether abnormal merge exists.<p id="mrs_01_24849__p254191763210"><a name="mrs_01_24849__li59281340122712"></a><a name="li59281340122712"></a><strong id="mrs_01_24849__b1292313190328">select database, table, elapsed, progress, merge_type from system.merges;</strong></p>
|
|
</li><li id="mrs_01_24849__li6929194092717">Do not perform the INSERT operation too frequently, do not insert a small amount of data, and increase the interval for inserting data.</li><li id="mrs_01_24849__li19298404273">The data table partitions are not properly allocated. As a result, too many partitions are generated and data tables need to be re-partitioned.</li><li id="mrs_01_24849__li109291240152714">If the MERGE operations are not triggered or slow, adjust the following parameters to accelerate them.<p id="mrs_01_24849__p4929840122716"><a name="mrs_01_24849__li109291240152714"></a><a name="li109291240152714"></a>For details, see <a href="mrs_01_24853.html">Accelerating Merge Operations</a>.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24849__table1929140122719" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24849__row15929174010278"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.1.2.4.2.1.3.1.1"><p id="mrs_01_24849__p8929140112717">Configuration Item</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.1.2.4.2.1.3.1.2"><p id="mrs_01_24849__p392934017274">Reference Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24849__row292964011274"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.1 "><p id="mrs_01_24849__p493064019274">max_threads</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.2 "><p id="mrs_01_24849__p17930114010272">Number of CPU cores x 2</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24849__row20930040152717"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.1 "><p id="mrs_01_24849__p9930140172716">background_pool_size</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.2 "><p id="mrs_01_24849__p1293018403271">Number of CPU cores</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24849__row1793018409276"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.1 "><p id="mrs_01_24849__p15930840172716">merge_max_block_size</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.2 "><p id="mrs_01_24849__p1493054032720">The value is an integer multiple of 8192 and is adjusted based on the CPU and memory resources.</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_24849__row13930440152717"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.1 "><p id="mrs_01_24849__p139301040122710">cleanup_delay_period</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.1.2.4.2.1.3.1.2 "><p id="mrs_01_24849__p1493004017278">Set this parameter to a value that is appropriately less than the default value 30.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li></ol>
|
|
</div>
|
|
<div class="section" id="mrs_01_24849__section1369542317156"><h4 class="sectiontitle">Changing the Value of parts_to_throw_insert</h4><div class="caution" id="mrs_01_24849__note19660112217191"><span class="cautiontitle"><img src="public_sys-resources/caution_3.0-en-us.png"> </span><div class="cautionbody"><p id="mrs_01_24849__p47087509168">Increase the value of this parameter only in special scenarios. This configuration acts as a warning for potential issues to some extent. If the cluster hardware resources are insufficient and this configuration is not adjusted properly, potential service issues cannot be detected in a timely manner, which may cause other faults and increase the difficulty of fault recovery.</p>
|
|
</div></div>
|
|
</div>
|
|
<p id="mrs_01_24849__p8818181614401">Log in to FusionInsight Manager and choose <strong id="mrs_01_24849__b91017883614">Cluster</strong> > <strong id="mrs_01_24849__b2102288361">Services</strong> > <strong id="mrs_01_24849__b41021389364">ClickHouse</strong>. Click <strong id="mrs_01_24849__b4103158133612">Configurations</strong> then <strong id="mrs_01_24849__b151032814363">All Configurations</strong>. Click <strong id="mrs_01_24849__b2104118153610">ClickHouseServer</strong> > <strong id="mrs_01_24849__b210417810363">Customization</strong>, find the <strong id="mrs_01_24849__b210417853615">clickhouse-config-customize</strong> parameter, add the following configuration, save it, and restart the service.</p>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_24849__table132600487403" frame="border" border="1" rules="all"><thead align="left"><tr id="mrs_01_24849__row1326144819407"><th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.1.3.1.1"><p id="mrs_01_24849__p5261164804017">Name</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="50%" id="mcps1.3.4.1.3.1.2"><p id="mrs_01_24849__p15261194884014">Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_24849__row42611648124012"><td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.1.3.1.1 "><p id="mrs_01_24849__p202611848104015">merge_tree.parts_to_throw_insert</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="50%" headers="mcps1.3.4.1.3.1.2 "><p id="mrs_01_24849__p152613485407">Memory of ClickHouse instances/32 GB x 300 (conservative estimation)</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<p id="mrs_01_24849__p4204094185">Verify the modification.</p>
|
|
<p id="mrs_01_24849__p5523415151720">Log in to the ClickHouse client and run the <strong id="mrs_01_24849__b069395619172">select * from system.merge_tree_settings where name = 'parts_to_throw_insert';</strong> command.</p>
|
|
<p id="mrs_01_24849__p8060118"></p>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_24848.html">ClickHouse Performance Tuning</a></div>
|
|
</div>
|
|
</div>
|
|
|