forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
46 lines
5.1 KiB
HTML
46 lines
5.1 KiB
HTML
<a name="mrs_01_0791"></a><a name="mrs_01_0791"></a>
|
|
|
|
<h1 class="topictitle1">Configuring Memory Management</h1>
|
|
<div id="body1590130530106"><div class="section" id="mrs_01_0791__s3636475a8fad49fba88e26f404606f97"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_0791__a4b582575e7e14b56978f0a61ff95cb37">In HDFS, each file object needs to register corresponding information in the NameNode and occupies certain storage space. As the number of files increases, if the original memory space cannot store the corresponding information, you need to change the memory size.</p>
|
|
</div>
|
|
<div class="section" id="mrs_01_0791__s6b006f89ff084c32b6da9ac09d646f5c"><h4 class="sectiontitle">Configuration Description</h4><p id="mrs_01_0791__a811cf9a508b0454eb17b74ca9dd85880"><strong id="mrs_01_0791__b203210473355">Navigation path for setting parameters:</strong></p>
|
|
<p id="mrs_01_0791__p39519895015">Go to the <strong id="mrs_01_0791__b1335215604317">All Configurations</strong> page of HDFS by referring to <a href="mrs_01_2125.html">Modifying Cluster Service Configuration Parameters</a>.</p>
|
|
</div>
|
|
|
|
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_0791__td1913adc9f3a4b09bdc3226f44204a92" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Parameter description</caption><thead align="left"><tr id="mrs_01_0791__rc7d4833f3d49472ca73912858fd5f2d0"><th align="left" class="cellrowborder" valign="top" width="22.62%" id="mcps1.3.3.2.4.1.1"><p id="mrs_01_0791__a7cdf534ff0fb40afafe4dee295445a73">Parameter</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="35.4%" id="mcps1.3.3.2.4.1.2"><p id="mrs_01_0791__ae6236a57fd1d48f4b396cfa527e759fb">Description</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="41.980000000000004%" id="mcps1.3.3.2.4.1.3"><p id="mrs_01_0791__a94af82cc20424031b9464e303a6007d4">Default Value</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_0791__r5e742f7deaf94b1086e96e349c70a1ba"><td class="cellrowborder" valign="top" width="22.62%" headers="mcps1.3.3.2.4.1.1 "><p id="mrs_01_0791__ab552978628f64083858a964656633904">GC_PROFILE</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35.4%" headers="mcps1.3.3.2.4.1.2 "><p id="mrs_01_0791__ac339b3162a5b46028eb1469e53b3956a">The NameNode memory size depends on the size of FsImage, which can be calculated based on the following formula: FsImage size = Number of files x 900 bytes. You can estimate the memory size of the NameNode of HDFS based on the calculation result.</p>
|
|
<p id="mrs_01_0791__a6c8d6d8dcc4e4a1ca951772cb42a6db8">The value range of this parameter is as follows:</p>
|
|
<ul id="mrs_01_0791__ua11c6dfe4187478aa289021d3e272459"><li id="mrs_01_0791__ld73aa715b4404566a3f7d8a061a4afc6"><strong id="mrs_01_0791__b594319775019">high</strong>: 4 GB</li><li id="mrs_01_0791__lb016a2b70c974113b8e3c83f943de3d7"><strong id="mrs_01_0791__b2710151455014">medium</strong>: 2 GB</li><li id="mrs_01_0791__l1ab159058f7943fca3ac1acf0626da38"><strong id="mrs_01_0791__b1874642211508">low</strong>: 256 MB</li><li id="mrs_01_0791__l2e5c9c075704495aac1a352869ddbb5d"><strong id="mrs_01_0791__b9749737185016">custom</strong>: The memory size can be set according to the data size in GC_OPTS.</li></ul>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.980000000000004%" headers="mcps1.3.3.2.4.1.3 "><p id="mrs_01_0791__a2461f3e335334774a9d3fd51d5d13a2a">custom</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_0791__row39558714101619"><td class="cellrowborder" valign="top" width="22.62%" headers="mcps1.3.3.2.4.1.1 "><p id="mrs_01_0791__p50139260101619">GC_OPTS</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="35.4%" headers="mcps1.3.3.2.4.1.2 "><p id="mrs_01_0791__p34748246101619">JVM parameter used for garbage collection (GC). This parameter is valid only when <strong id="mrs_01_0791__b4664758185013">GC_PROFILE</strong> is set to <strong id="mrs_01_0791__b667025819509">custom</strong>. Ensure that the <strong id="mrs_01_0791__b1041166155116">GC_OPT</strong> parameter is set correctly. Otherwise, the process will fail to be started.</p>
|
|
<div class="notice" id="mrs_01_0791__note954615441212"><span class="noticetitle"> NOTICE: </span><div class="noticebody"><p id="mrs_01_0791__p1438705918121">Exercise caution when you modify the configuration. If the configuration is incorrect, the services are unavailable.</p>
|
|
</div></div>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="41.980000000000004%" headers="mcps1.3.3.2.4.1.3 "><p id="mrs_01_0791__p31802318364">-Xms2G -Xmx4G -XX:NewSize=128M -XX:MaxNewSize=256M -XX:MetaspaceSize=128M -XX:MaxMetaspaceSize=128M -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=65 -XX:+PrintGCDetails -Dsun.rmi.dgc.client.gcInterval=0x7FFFFFFFFFFFFFE -Dsun.rmi.dgc.server.gcInterval=0x7FFFFFFFFFFFFFE -XX:-OmitStackTraceInFastThrow -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -Djdk.tls.ephemeralDHKeySize=2048</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_0790.html">Using HDFS</a></div>
|
|
</div>
|
|
</div>
|
|
|