:original_name: cce_10_0222.html .. _cce_10_0222: Managing a Node Pool ==================== Notes and Constraints --------------------- The default node pool DefaultPool does not support the following management operations. Configuring Kubernetes Parameters --------------------------------- CCE allows you to highly customize Kubernetes parameter settings on core components in a cluster. For more information, see `kubelet `__. This function is supported only in clusters of **v1.15 and later**. It is not displayed for clusters earlier than v1.15. #. Log in to the CCE console. #. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. #. Choose **More** > **Manage** next to the node pool name. #. On the **Manage Component** page on the right, change the values of the following Kubernetes parameters: .. table:: **Table 1** kubelet +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | Default Value | Remarks | +============================+====================================================================================================================================================================================================================================================================================================================================================================================================================+=================================================================================================================================+=======================================================================================================================================================================================================================================================================+ | cpu-manager-policy | Specifies the CPU core binding configuration. For details, see :ref:`CPU Core Binding `. | none | The values can be modified during the node pool lifecycle. | | | | | | | | - **none**: disables pods from exclusively occupying CPUs. Select this value if you want a large pool of shareable CPU cores. | | | | | - **static**: enables pods to exclusively occupy CPUs. Select this value if your workload is sensitive to latency in CPU cache and scheduling. | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | 100 | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | kube-api-burst | Burst to use while talking with kube-apiserver. | 100 | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | max-pods | Maximum number of pods managed by kubelet. | 40 | | | | | | | | | | 20 | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | pod-pids-limit | PID limit in Kubernetes | -1 | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | with-local-dns | Whether to use the local IP address as the ClusterDNS of the node. | false | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | event-qps | QPS limit for event creation | 5 | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | allowed-unsafe-sysctls | Insecure system configuration allowed. | [] | | | | | | | | | Starting from **v1.17.17**, CCE enables pod security policies for kube-apiserver. You need to add corresponding configurations to **allowedUnsafeSysctls** of a pod security policy to make the policy take effect. (This configuration is not required for clusters earlier than v1.17.17.) For details, see :ref:`Example of Enabling Unsafe Sysctls in Pod Security Policy `. | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | over-subscription-resource | Whether to enable node oversubscription. | true | ``-`` | | | | | | | | If this parameter is set to **true**, the node oversubscription feature is enabled. For details, see :ref:`Hybrid Deployment of Online and Offline Jobs `. | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | colocation | Whether to enable node hybrid deployment. | true | ``-`` | | | | | | | | If this parameter is set to **true**, the node hybrid deployment feature is enabled. For details, see :ref:`Hybrid Deployment of Online and Offline Jobs `. | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | kube-reserved-mem | Reserved node memory. | Depends on node specifications. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. | The sum of kube-reserved-mem and system-reserved-mem is less than half of the memory. | | | | | | | system-reserved-mem | | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | topology-manager-policy | Set the topology management policy. | none | The values can be modified during the node pool lifecycle. | | | | | | | | Valid values are as follows: | | .. important:: | | | | | | | | - **restricted**: kubelet accepts only pods that achieve optimal NUMA alignment on the requested resources. | | NOTICE: | | | - **best-effort**: kubelet preferentially selects pods that implement NUMA alignment on CPU and device resources. | | Exercise caution when modifying topology-manager-policy and topology-manager-scope will restart kubelet and recalculate the resource allocation of pods based on the modified policy. As a result, running pods may restart or even fail to receive any resources. | | | - **none** (default): The topology management policy is disabled. | | | | | - **single-numa-node**: kubelet allows only pods that are aligned to the same NUMA node in terms of CPU and device resources. | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | topology-manager-scope | Set the resource alignment granularity of the topology management policy. Valid values are as follows: | container | | | | | | | | | - **container** (default) | | | | | - **pod** | | | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | resolv-conf | DNS resolution configuration file specified by the container | The default value is null. | ``-`` | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. table:: **Table 2** kube-proxy +----------------------------------+-------------------------------------------------------------+---------------+------------------------------------------------------------+ | Parameter | Description | Default Value | Remarks | +==================================+=============================================================+===============+============================================================+ | conntrack-min | sysctl -w net.nf_conntrack_max | 131072 | The values can be modified during the node pool lifecycle. | +----------------------------------+-------------------------------------------------------------+---------------+------------------------------------------------------------+ | conntrack-tcp-timeout-close-wait | sysctl -w net.netfilter.nf_conntrack_tcp_timeout_close_wait | 1h0m0s | | +----------------------------------+-------------------------------------------------------------+---------------+------------------------------------------------------------+ .. table:: **Table 3** Network components (available only for CCE Turbo clusters) +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ | Parameter | Description | Default Value | Remarks | +===========================+======================================================================================================+=================+=================+ | nic-threshold | Low threshold of the number of bound ENIs:High threshold of the number of bound ENIs | Default: 0:0 | ``-`` | | | | | | | | .. note:: | | | | | | | | | | This parameter is being discarded. Use the dynamic pre-binding parameters of the other four ENIs. | | | +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ | nic-minimum-target | Minimum number of ENIs bound to the nodes in the node pool | Default: 10 | ``-`` | +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ | nic-maximum-target | Maximum number of ENIs pre-bound to a node at the node pool level | Default: 0 | ``-`` | +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ | nic-warm-target | Number of ENIs pre-bound to a node at the node pool level | Default: 2 | ``-`` | +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ | nic-max-above-warm-target | Reclaim number of ENIs pre-bound to a node at the node pool level | Default: 2 | ``-`` | +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ .. table:: **Table 4** Pod security group in a node pool (available only for CCE Turbo clusters) +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------------+ | Parameter | Description | Default Value | Remarks | +==============================+=====================================================================================================================================================================================================================================================================================================+=================+=================+ | security_groups_for_nodepool | - Default security group used by pods in a node pool. You can enter the security group ID. If this parameter is not set, the default security group of the cluster container network is used. A maximum of five security group IDs can be specified at the same time, separated by semicolons (;). | ``-`` | ``-`` | | | - The priority of the security group is lower than that of the security group configured for the :ref:`SecurityGroup ` resource object. | | | +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------------+ .. table:: **Table 5** Docker (available only for node pools that use Docker) +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ | Parameter | Description | Default Value | Remarks | +=======================+===============================================================+=================+========================================================================================================+ | native-umask | \`--exec-opt native.umask | normal | Cannot be changed. | +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ | docker-base-size | \`--storage-opts dm.basesize | 0 | Cannot be changed. | +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ | insecure-registry | Address of an insecure image registry | false | Cannot be changed. | +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ | limitcore | Maximum size of a core file in a container. The unit is byte. | 5368709120 | ``-`` | +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ | default-ulimit-nofile | Limit on the number of handles in a container | {soft}:{hard} | The value cannot exceed the value of the kernel parameter **nr_open** and cannot be a negative number. | | | | | | | | | | You can run the following command to obtain the kernel parameter **nr_open**: | | | | | | | | | | .. code-block:: | | | | | | | | | | sysctl -a | grep nr_open | +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ #. Click **OK**. Editing a Node Pool ------------------- #. Log in to the CCE console. #. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. #. Click **Edit** next to the name of the node pool you will edit. In the **Edit Node Pool** page, edit the following parameters: **Basic Settings** .. table:: **Table 6** Basic settings +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+=================================================================================================================================================================================================================================================================================================================================================================================================================================================+ | Node Pool Name | Name of the node pool. | +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Auto Scaling | By default, this parameter is disabled. | | | | | | After you enable autoscaler by clicking |image1|, nodes in the node pool are automatically created or deleted based on service requirements. | | | | | | - **Maximum Nodes** and **Minimum Nodes**: You can set the maximum and minimum number of nodes to ensure that the number of nodes to be scaled is within a proper range. | | | - **Priority**: A larger value indicates a higher priority. For example, if this parameter is set to **1** and **4** respectively for node pools A and B, B has a higher priority than A, and auto scaling is first triggered for B. If the priorities of multiple node pools are set to the same value, for example, **2**, the node pools are not prioritized and the system performs scaling based on the minimum resource waste principle. | | | - **Cooldown Period**: Required. The unit is minute. This parameter indicates the interval between the previous scale-out action and the next scale-in action. | | | | | | If the **Autoscaler** field is set to on, install the :ref:`autoscaler add-on ` to use the autoscaler feature. | +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ **Advanced Settings** .. table:: **Table 7** Advanced settings +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+================================================================================================================================================================================================================================================================+ | K8s label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | | | | | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | | | | | | .. note:: | | | | | | After a **K8s label** is modified, the inventory nodes in the node pool are updated synchronously. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Resource Tag | You can add resource tags to classify resources. | | | | | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | | | | | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | | | | | | .. note:: | | | | | | After a **resource tag** is modified, the modification automatically takes effect when a node is added. For existing nodes, you need to manually reset the nodes for the modification to take effect. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | | | | | | - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | | | | | | For details, see :ref:`Managing Node Taints `. | | | | | | .. note:: | | | | | | After a **taint** is modified, the inventory nodes in the node pool are updated synchronously. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Edit Key pair | Only node pools that use key pairs for login support key pair editing. You can select another key pair. | | | | | | .. note:: | | | | | | The edited key pair automatically takes effect when a node is added. For existing nodes, you need to manually reset the nodes for the key pair to take effect. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Click **OK**. In the node pool list, the node pool status becomes **Scaling**. After the status changes to **Completed**, the node pool parameters are modified successfully. The modified configuration will be synchronized to all nodes in the node pool. Deleting a Node Pool -------------------- Deleting a node pool will delete nodes in the pool. Pods on these nodes will be automatically migrated to available nodes in other node pools. If pods in the node pool have a specific node selector and none of the other nodes in the cluster satisfies the node selector, the pods will become unschedulable. #. Log in to the CCE console. #. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. #. Choose **More > Delete** next to a node pool name to delete the node pool. #. Read the precautions in the **Delete Node Pool** dialog box. #. In the text box, click **Yes** to confirm that you want to continue the deletion. .. _cce_10_0222__section550619571556: Copying a Node Pool ------------------- You can copy the configuration of an existing node pool to create a new node pool on the CCE console. #. Log in to the CCE console. #. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. #. Choose **More > Copy** next to a node pool name to copy the node pool. #. The configurations of the selected node pool are replicated to the **Clone Node Pool** page. You can edit the configurations as required and click **Next: Confirm**. #. On the **Confirm** page, confirm the node pool configuration and click **Create Now**. Then, a new node pool is created based on the edited configuration. Migrating a Node ---------------- Nodes in a node pool can be migrated. Currently, nodes in a node pool can be migrated only to the default node pool (defaultpool) in the same cluster. #. Log in to the CCE console and click the cluster name to access the cluster. #. In the navigation pane, choose **Nodes** and switch to the **Node Pools** tab page. #. Click **View Node** in the **Operation** column of the node pool to be migrated. #. Select the nodes to be migrated and choose **More** > **Migrate** to migrate the nodes to the default node pool in batches. You can also choose **More** > **Migrate** in the **Operation** column of a single node to migrate the node. #. In the displayed **Migrate Node** window, confirm the information. .. note:: The migration has no impacts on the original resource tags, Kubernetes labels, and taints of the node. .. |image1| image:: /_static/images/en-us_image_0000001528627005.png