diff --git a/umn/source/_static/images/en-us_image_0000001171703840.png b/umn/source/_static/images/en-us_image_0000001171703840.png deleted file mode 100644 index ddd83bb..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001171703840.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001199021274.png b/umn/source/_static/images/en-us_image_0000001199021274.png deleted file mode 100644 index a4bbbd0..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001199021274.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001199021280.png b/umn/source/_static/images/en-us_image_0000001199021280.png deleted file mode 100644 index 15ee848..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001199021280.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001199181292.png b/umn/source/_static/images/en-us_image_0000001199181292.png deleted file mode 100644 index bd21029..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001199181292.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001199341268.png b/umn/source/_static/images/en-us_image_0000001199341268.png new file mode 100644 index 0000000..672ccf8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001199341268.png differ diff --git a/umn/source/_static/images/en-us_image_0000001199501182.png b/umn/source/_static/images/en-us_image_0000001199501182.png deleted file mode 100644 index 57a44dd..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001199501182.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001199501252.png b/umn/source/_static/images/en-us_image_0000001199501252.png deleted file mode 100644 index bfcccfe..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001199501252.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001207036074.png b/umn/source/_static/images/en-us_image_0000001207036074.png deleted file mode 100644 index 33a41a2..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001207036074.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001207511384.png b/umn/source/_static/images/en-us_image_0000001207511384.png new file mode 100644 index 0000000..5a0e760 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001207511384.png differ diff --git a/umn/source/_static/images/en-us_image_0000001217183707.png b/umn/source/_static/images/en-us_image_0000001217183707.png index e8a2751..aa7b279 100644 Binary files a/umn/source/_static/images/en-us_image_0000001217183707.png and b/umn/source/_static/images/en-us_image_0000001217183707.png differ diff --git a/umn/source/_static/images/en-us_image_0000001276433425.png b/umn/source/_static/images/en-us_image_0000001276433425.png new file mode 100644 index 0000000..6a1cd90 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001276433425.png differ diff --git a/umn/source/_static/images/en-us_image_0000001280547449.png b/umn/source/_static/images/en-us_image_0000001280547449.png deleted file mode 100644 index 0492b6b..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001280547449.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001378942548.png b/umn/source/_static/images/en-us_image_0000001378942548.png new file mode 100644 index 0000000..a004b89 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001378942548.png differ diff --git a/umn/source/_static/images/en-us_image_0000001248666457.png b/umn/source/_static/images/en-us_image_0000001464878016.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001248666457.png rename to umn/source/_static/images/en-us_image_0000001464878016.png diff --git a/umn/source/_static/images/en-us_image_0000001248946053.png b/umn/source/_static/images/en-us_image_0000001465197524.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001248946053.png rename to umn/source/_static/images/en-us_image_0000001465197524.png diff --git a/umn/source/_static/images/en-us_image_0000001480191270.png b/umn/source/_static/images/en-us_image_0000001480191270.png new file mode 100644 index 0000000..290b94f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001480191270.png differ diff --git a/umn/source/_static/images/en-us_image_0000001482546084.png b/umn/source/_static/images/en-us_image_0000001482546084.png new file mode 100644 index 0000000..b2f9f12 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001482546084.png differ diff --git a/umn/source/_static/images/en-us_image_0000001482796460.png b/umn/source/_static/images/en-us_image_0000001482796460.png new file mode 100644 index 0000000..60b03ef Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001482796460.png differ diff --git a/umn/source/_static/images/en-us_image_0000001248946421.png b/umn/source/_static/images/en-us_image_0000001515838557.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001248946421.png rename to umn/source/_static/images/en-us_image_0000001515838557.png diff --git a/umn/source/_static/images/en-us_image_0000001249026401.png b/umn/source/_static/images/en-us_image_0000001515917789.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001249026401.png rename to umn/source/_static/images/en-us_image_0000001515917789.png diff --git a/umn/source/_static/images/en-us_image_0000001528627005.png b/umn/source/_static/images/en-us_image_0000001528627005.png new file mode 100644 index 0000000..d74a10f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001528627005.png differ diff --git a/umn/source/_static/images/en-us_image_0000001531373685.png b/umn/source/_static/images/en-us_image_0000001531373685.png new file mode 100644 index 0000000..2375222 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001531373685.png differ diff --git a/umn/source/_static/images/en-us_image_0000001531533045.png b/umn/source/_static/images/en-us_image_0000001531533045.png new file mode 100644 index 0000000..f963c97 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001531533045.png differ diff --git a/umn/source/_static/images/en-us_image_0000001531533921.png b/umn/source/_static/images/en-us_image_0000001531533921.png new file mode 100644 index 0000000..da5aed3 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001531533921.png differ diff --git a/umn/source/_static/images/en-us_image_0000001533585325.png b/umn/source/_static/images/en-us_image_0000001533585325.png new file mode 100644 index 0000000..c18b799 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001533585325.png differ diff --git a/umn/source/_static/images/en-us_image_0000001533586881.png b/umn/source/_static/images/en-us_image_0000001533586881.png new file mode 100644 index 0000000..92f9830 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001533586881.png differ diff --git a/umn/source/add-ons/autoscaler.rst b/umn/source/add-ons/autoscaler.rst index e8d3bfc..edb2c9d 100644 --- a/umn/source/add-ons/autoscaler.rst +++ b/umn/source/add-ons/autoscaler.rst @@ -121,7 +121,7 @@ Installing the Add-on +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Total Nodes | Maximum number of nodes that can be managed by the cluster, within which cluster scale-out is performed. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Total Cores | Maximum sum of CPU cores of all nodes in a cluster, within which cluster scale-out is performed. | + | Total CPUs | Maximum sum of CPU cores of all nodes in a cluster, within which cluster scale-out is performed. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Total Memory (GB) | Maximum sum of memory of all nodes in a cluster, within which cluster scale-out is performed. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst b/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst index 7e8502b..812cbf8 100644 --- a/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst +++ b/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst @@ -40,101 +40,101 @@ This add-on has been installed by default. If it is uninstalled due to some reas .. table:: **Table 1** coredns add-on parameters - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Add-on Specifications | Concurrent domain name resolution ability. Select add-on specifications that best fit your needs. | - | | | - | | If you select **Custom qps**, the domain name resolution QPS provided by CoreDNS is positively correlated with the CPU consumption. Adjust the number of pods and container CPU/memory quotas as required. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Pods | Number of pods that will be created to match the selected add-on specifications. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Containers | CPU and memory quotas of the container allowed for the selected add-on specifications. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameters | - **parameterSyncStrategy**: indicates whether to configure consistency check when an add-on is upgraded. | - | | | - | | - **ensureConsistent**: indicates that the configuration consistency check is enabled. If the configuration recorded in the cluster is inconsistent with the actual configuration, the add-on cannot be upgraded. | - | | - **force**: indicates that the configuration consistency check is ignored during an upgrade. Ensure that the current effective configuration is the same as the original configuration. After the add-on is upgraded, restore the value of **parameterSyncStrategy** to **ensureConsistent** and enable the configuration consistency check again. | - | | | - | | - **stub_domains**: A domain name server for a user-defined domain name. The format is a key-value pair. The key is a suffix of DNS domain name, and the value is one or more DNS IP addresses. | - | | | - | | - **upstream_nameservers**: IP address of the upstream DNS server. | - | | | - | | - **servers**: The servers configuration is available since coredns 1.23.1. You can customize the servers configuration. For details, see `dns-custom-nameservers `__. **plugins** indicates the configuration of each component in coredns (https://coredns.io/manual/plugins/). You are advised to retain the default configurations in common scenarios to prevent CoreDNS from being unavailable due to configuration errors. Each plugin component contains **name**, **parameters** (optional), and **configBlock** (optional). The format of the generated Corefile is as follows: | - | | | - | | $name $parameters { | - | | | - | | $configBlock | - | | | - | | } | - | | | - | | :ref:`Table 2 ` describes common plugins. | - | | | - | | Example: | - | | | - | | .. code-block:: | - | | | - | | { | - | | "servers": [ | - | | { | - | | "plugins": [ | - | | { | - | | "name": "bind", | - | | "parameters": "{$POD_IP}" | - | | }, | - | | { | - | | "name": "cache", | - | | "parameters": 30 | - | | }, | - | | { | - | | "name": "errors" | - | | }, | - | | { | - | | "name": "health", | - | | "parameters": "{$POD_IP}:8080" | - | | }, | - | | { | - | | "configBlock": "pods insecure\nfallthrough in-addr.arpa ip6.arpa", | - | | "name": "kubernetes", | - | | "parameters": "cluster.local in-addr.arpa ip6.arpa" | - | | }, | - | | { | - | | "name": "loadbalance", | - | | "parameters": "round_robin" | - | | }, | - | | { | - | | "name": "prometheus", | - | | "parameters": "{$POD_IP}:9153" | - | | }, | - | | { | - | | "configBlock": "policy random", | - | | "name": "forward", | - | | "parameters": ". /etc/resolv.conf" | - | | }, | - | | { | - | | "name": "reload" | - | | }, | - | | { | - | | "name": "log" | - | | } | - | | ], | - | | "port": 5353, | - | | "zones": [ | - | | { | - | | "zone": "." | - | | } | - | | ] | - | | } | - | | ], | - | | "stub_domains": { | - | | "acme.local": [ | - | | "1.2.3.4", | - | | "6.7.8.9" | - | | ] | - | | }, | - | | "upstream_nameservers": ["8.8.8.8", "8.8.4.4"] | - | | } | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ + | Add-on Specifications | Concurrent domain name resolution ability. Select add-on specifications that best fit your needs. | + | | | + | | If you select **Custom qps**, the domain name resolution QPS provided by CoreDNS is positively correlated with the CPU consumption. Adjust the number of pods and container CPU/memory quotas as required. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pods | Number of pods that will be created to match the selected add-on specifications. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Containers | CPU and memory quotas of the container allowed for the selected add-on specifications. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameters | - **parameterSyncStrategy**: indicates whether to configure consistency check when an add-on is upgraded. | + | | | + | | - **ensureConsistent**: indicates that the configuration consistency check is enabled. If the configuration recorded in the cluster is inconsistent with the actual configuration, the add-on cannot be upgraded. | + | | - **force**: indicates that the configuration consistency check is ignored during an upgrade. Ensure that the current effective configuration is the same as the original configuration. After the add-on is upgraded, restore the value of **parameterSyncStrategy** to **ensureConsistent** and enable the configuration consistency check again. | + | | | + | | - **stub_domains**: A domain name server for a user-defined domain name. The format is a key-value pair. The key is a suffix of DNS domain name, and the value is one or more DNS IP addresses. | + | | | + | | - **upstream_nameservers**: IP address of the upstream DNS server. | + | | | + | | - **servers**: The servers configuration is available since coredns 1.23.1. You can customize the servers configuration. For details, see `dns-custom-nameservers `__. **plugins** indicates the configuration of each component in coredns (https://coredns.io/manual/plugins/). You are advised to retain the default configurations in common scenarios to prevent CoreDNS from being unavailable due to configuration errors. Each plugin component contains **name**, **parameters** (optional), and **configBlock** (optional). The format of the generated Corefile is as follows: | + | | | + | | $name $parameters { | + | | | + | | $configBlock | + | | | + | | } | + | | | + | | :ref:`Table 2 ` describes common plugins. | + | | | + | | Example: | + | | | + | | .. code-block:: | + | | | + | | { | + | | "servers": [ | + | | { | + | | "plugins": [ | + | | { | + | | "name": "bind", | + | | "parameters": "{$POD_IP}" | + | | }, | + | | { | + | | "name": "cache", | + | | "parameters": 30 | + | | }, | + | | { | + | | "name": "errors" | + | | }, | + | | { | + | | "name": "health", | + | | "parameters": "{$POD_IP}:8080" | + | | }, | + | | { | + | | "configBlock": "pods insecure\nfallthrough in-addr.arpa ip6.arpa", | + | | "name": "kubernetes", | + | | "parameters": "cluster.local in-addr.arpa ip6.arpa" | + | | }, | + | | { | + | | "name": "loadbalance", | + | | "parameters": "round_robin" | + | | }, | + | | { | + | | "name": "prometheus", | + | | "parameters": "{$POD_IP}:9153" | + | | }, | + | | { | + | | "configBlock": "policy random", | + | | "name": "forward", | + | | "parameters": ". /etc/resolv.conf" | + | | }, | + | | { | + | | "name": "reload" | + | | }, | + | | { | + | | "name": "log" | + | | } | + | | ], | + | | "port": 5353, | + | | "zones": [ | + | | { | + | | "zone": "." | + | | } | + | | ] | + | | } | + | | ], | + | | "stub_domains": { | + | | "acme.local": [ | + | | "1.2.3.4", | + | | "6.7.8.9" | + | | ] | + | | }, | + | | "upstream_nameservers": ["8.8.8.8", "8.8.4.4"] | + | | } | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. _cce_10_0129__table1420814384015: diff --git a/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst b/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst index bfbb3f3..ac58001 100644 --- a/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst +++ b/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst @@ -26,7 +26,7 @@ This add-on has been installed by default. If it is uninstalled due to some reas #. Log in to the CCE console and access the cluster console. Choose **Add-ons** in the navigation pane, locate **everest** on the right, and click **Install**. -#. Select **Single**, **Custom**, or **HA** for **Add-on Specifications**. +#. Select **Standalone**, **HA**, or **Custom** for **Add-on Specifications**. The everest add-on contains the following containers. You can adjust the specifications as required. diff --git a/umn/source/add-ons/gpu-beta.rst b/umn/source/add-ons/gpu-beta.rst index 89f6a51..75ac615 100644 --- a/umn/source/add-ons/gpu-beta.rst +++ b/umn/source/add-ons/gpu-beta.rst @@ -26,7 +26,7 @@ Installing the Add-on .. important:: - - If the download link is a public network address, for example, **https://us.download.nvidia.com/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run**, bind an EIP to each GPU node. For details about how to obtain the driver link, see :ref:`Obtaining the Driver Link from Public Network `. + - If the download link is a public network address, for example, **https://us.download.nvidia.com/tesla/470.103.01/NVIDIA-Linux-x86_64-470.103.01.run**, bind an EIP to each GPU node. For details about how to obtain the driver link, see :ref:`Obtaining the Driver Link from Public Network `. - If the download link is an OBS URL, you do not need to bind an EIP to GPU nodes. - Ensure that the NVIDIA driver version matches the GPU node. - After the driver version is changed, restart the node for the change to take effect. @@ -68,7 +68,7 @@ Obtaining the Driver Link from Public Network .. _cce_10_0141__fig11696366517: - .. figure:: /_static/images/en-us_image_0000001280547449.png + .. figure:: /_static/images/en-us_image_0000001531533921.png :alt: **Figure 1** Setting parameters **Figure 1** Setting parameters @@ -77,20 +77,20 @@ Obtaining the Driver Link from Public Network .. _cce_10_0141__fig7873421145213: - .. figure:: /_static/images/en-us_image_0000001199501252.png + .. figure:: /_static/images/en-us_image_0000001531373685.png :alt: **Figure 2** Driver information **Figure 2** Driver information 6. Obtain the driver link in either of the following ways: - - Method 1: As shown in :ref:`Figure 3 `, find *url=/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run* in the browser address box. Then, supplement it to obtain the driver link https://us.download.nvidia.com/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run. By using this method, you must bind an EIP to each GPU node. + - Method 1: As shown in :ref:`Figure 3 `, find *url=/tesla/470.103.01/NVIDIA-Linux-x86_64-470.103.01.run* in the browser address box. Then, supplement it to obtain the driver link https://us.download.nvidia.com/tesla/470.103.01/NVIDIA-Linux-x86_64-470.103.01.run. By using this method, you must bind an EIP to each GPU node. - Method 2: As shown in :ref:`Figure 3 `, click **AGREE & DOWNLOAD** to download the driver. Then, upload the driver to OBS and record the OBS URL. By using this method, you do not need to bind an EIP to GPU nodes. .. _cce_10_0141__fig5901194614534: - .. figure:: /_static/images/en-us_image_0000001199181292.png + .. figure:: /_static/images/en-us_image_0000001531533045.png :alt: **Figure 3** Obtaining the link **Figure 3** Obtaining the link diff --git a/umn/source/add-ons/overview.rst b/umn/source/add-ons/overview.rst index daf120b..d5cab0a 100644 --- a/umn/source/add-ons/overview.rst +++ b/umn/source/add-ons/overview.rst @@ -9,18 +9,20 @@ CCE provides multiple types of add-ons to extend cluster functions and meet feat .. table:: **Table 1** Add-on list - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add-on Name | Introduction | - +=========================================================================+======================================================================================================================================================================================+ - | :ref:`coredns (System Resource Add-On, Mandatory) ` | The coredns add-on is a DNS server that provides domain name resolution services for Kubernetes clusters. coredns chains plug-ins to provide additional features. | - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`storage-driver (System Resource Add-On, Discarded) ` | storage-driver is a FlexVolume driver used to support IaaS storage services such as EVS, SFS, and OBS. | - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`everest (System Resource Add-On, Mandatory) ` | Everest is a cloud native container storage system. Based on the Container Storage Interface (CSI), clusters of Kubernetes v1.15.6 or later obtain access to cloud storage services. | - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`autoscaler ` | The autoscaler add-on resizes a cluster based on pod scheduling status and resource usage. | - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`metrics-server ` | metrics-server is an aggregator for monitoring data of core cluster resources. | - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`gpu-beta ` | gpu-beta is a device management add-on that supports GPUs in containers. It supports only NVIDIA drivers. | - +-------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Add-on Name | Introduction | + +=========================================================================+==============================================================================================================================================================================================================================================================================================+ + | :ref:`coredns (System Resource Add-On, Mandatory) ` | The coredns add-on is a DNS server that provides domain name resolution services for Kubernetes clusters. coredns chains plug-ins to provide additional features. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`storage-driver (System Resource Add-On, Discarded) ` | storage-driver is a FlexVolume driver used to support IaaS storage services such as EVS, SFS, and OBS. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`everest (System Resource Add-On, Mandatory) ` | Everest is a cloud native container storage system. Based on the Container Storage Interface (CSI), clusters of Kubernetes v1.15.6 or later obtain access to cloud storage services. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`autoscaler ` | The autoscaler add-on resizes a cluster based on pod scheduling status and resource usage. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`metrics-server ` | metrics-server is an aggregator for monitoring data of core cluster resources. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`gpu-beta ` | gpu-beta is a device management add-on that supports GPUs in containers. It supports only NVIDIA drivers. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`volcano ` | Volcano provides general-purpose, high-performance computing capabilities, such as job scheduling, heterogeneous chip management, and job running management, serving end users through computing frameworks for different industries, such as AI, big data, gene sequencing, and rendering. | + +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/add-ons/volcano.rst b/umn/source/add-ons/volcano.rst index 0a3d79d..34772c3 100644 --- a/umn/source/add-ons/volcano.rst +++ b/umn/source/add-ons/volcano.rst @@ -101,7 +101,7 @@ Installing the Add-on +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ | priority | The priority plugin schedules pods based on the custom workload priority. | ``-`` | ``-`` | +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ - | overcommit | Resources in a cluster are scheduled after being accumulated in a certain multiple to improve the workload enqueuing efficiency. If all workloads are Deployments, remove this plugin or set the raising factor to **2.0**. | **overcommit-factor**: Raising factor. Defaults to is **1.2**. | .. code-block:: | + | overcommit | Resources in a cluster are scheduled after being accumulated in a certain multiple to improve the workload enqueuing efficiency. If all workloads are Deployments, remove this plugin or set the raising factor to **2.0**. | **overcommit-factor**: Raising factor. Defaults to **1.2**. | .. code-block:: | | | | | | | | | | - plugins: | | | | | - name: overcommit | diff --git a/umn/source/auto_scaling/index.rst b/umn/source/auto_scaling/index.rst index bad3012..8715284 100644 --- a/umn/source/auto_scaling/index.rst +++ b/umn/source/auto_scaling/index.rst @@ -7,7 +7,7 @@ Auto Scaling - :ref:`Overview ` - :ref:`Scaling a Workload ` -- :ref:`Scaling a Cluster/Node ` +- :ref:`Scaling a Node ` - :ref:`Using HPA and CA for Auto Scaling of Workloads and Nodes ` .. toctree:: @@ -16,5 +16,5 @@ Auto Scaling overview scaling_a_workload/index - scaling_a_cluster_node/index + scaling_a_node/index using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/creating_a_node_scaling_policy.rst b/umn/source/auto_scaling/scaling_a_node/creating_a_node_scaling_policy.rst similarity index 99% rename from umn/source/auto_scaling/scaling_a_cluster_node/creating_a_node_scaling_policy.rst rename to umn/source/auto_scaling/scaling_a_node/creating_a_node_scaling_policy.rst index 839bad5..aa2a7fc 100644 --- a/umn/source/auto_scaling/scaling_a_cluster_node/creating_a_node_scaling_policy.rst +++ b/umn/source/auto_scaling/scaling_a_node/creating_a_node_scaling_policy.rst @@ -73,7 +73,7 @@ Procedure - **Periodic**: - **Triggered At**: You can select a specific time point every day, every week, every month, or every year. + **Trigger Time**: You can select a specific time point every day, every week, every month, or every year. **Action**: Set an action to be performed when the trigger condition is met. diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/index.rst b/umn/source/auto_scaling/scaling_a_node/index.rst similarity index 88% rename from umn/source/auto_scaling/scaling_a_cluster_node/index.rst rename to umn/source/auto_scaling/scaling_a_node/index.rst index 71adde3..220efbb 100644 --- a/umn/source/auto_scaling/scaling_a_cluster_node/index.rst +++ b/umn/source/auto_scaling/scaling_a_node/index.rst @@ -2,8 +2,8 @@ .. _cce_10_0291: -Scaling a Cluster/Node -====================== +Scaling a Node +============== - :ref:`Node Scaling Mechanisms ` - :ref:`Creating a Node Scaling Policy ` diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/managing_node_scaling_policies.rst b/umn/source/auto_scaling/scaling_a_node/managing_node_scaling_policies.rst similarity index 80% rename from umn/source/auto_scaling/scaling_a_cluster_node/managing_node_scaling_policies.rst rename to umn/source/auto_scaling/scaling_a_node/managing_node_scaling_policies.rst index 2616f4a..2c49103 100644 --- a/umn/source/auto_scaling/scaling_a_cluster_node/managing_node_scaling_policies.rst +++ b/umn/source/auto_scaling/scaling_a_node/managing_node_scaling_policies.rst @@ -17,11 +17,15 @@ You can view the associated node pool, rules, and scaling history of a node scal #. Log in to the CCE console and access the cluster console. #. Choose **Node Scaling** in the navigation pane and click |image1| in front of the policy to be viewed. -#. In the expanded area, the **Associated Node Pool**, **Execution Rules**, and **Scaling Records** tab pages are displayed. If the policy is abnormal, locate and rectify the fault based on the error information. +#. In the expanded area, the **Associated Node Pools**, **Rules**, and **Scaling History** tab pages are displayed. If the policy is abnormal, locate and rectify the fault based on the error information. .. note:: - You can also enable or disable auto scaling in **Node Pools**. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**, and click **Edit** in the upper right corner of the node pool to be operated. In the **Edit Node Pool** dialog box displayed, you can enable **Autoscaler** and set the limits of the number of nodes. + You can also disable or enable auto scaling on the **Node Pools** page. + + a. Log in to the CCE console and access the cluster console. + b. In the navigation pane, choose **Nodes** and switch to the **Node Pools** tab page. + c. Click **Edit** of the node pool to be operated. In the **Edit Node Pool** dialog box that is displayed, set the limits of the number of nodes. Deleting a Node Scaling Policy ------------------------------ diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/node_scaling_mechanisms.rst b/umn/source/auto_scaling/scaling_a_node/node_scaling_mechanisms.rst similarity index 100% rename from umn/source/auto_scaling/scaling_a_cluster_node/node_scaling_mechanisms.rst rename to umn/source/auto_scaling/scaling_a_node/node_scaling_mechanisms.rst diff --git a/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst b/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst index a344fbe..6974b41 100644 --- a/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst +++ b/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst @@ -17,7 +17,7 @@ Notes and Constraints - HPA policies can be created only for clusters of v1.13 or later. -- Only one policy can be created for each workload. That is, if you have created an HPA policy, you cannot create other HPA policies for the workload. You can delete the created HPA policy and create a new one. +- Only one policy can be created for each workload. You can create an HPA policy. - For clusters earlier than v1.19.10, if an HPA policy is used to scale out a workload with EVS volumes mounted, the existing pods cannot be read or written when a new pod is scheduled to another node. @@ -36,60 +36,60 @@ Procedure .. table:: **Table 1** HPA policy parameters - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================================================================================+ - | Policy Name | Name of the policy to be created. Set this parameter as required. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace to which the workload belongs. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Associated Workload | Workload with which the HPA policy is associated. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Pod Range | Minimum and maximum numbers of pods. | - | | | - | | When a policy is triggered, the workload pods are scaled within this range. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cooldown Period | Interval between a scale-in and a scale-out. The unit is minute. **The interval cannot be shorter than 1 minute.** | - | | | - | | **This parameter is available only for clusters of v1.15 and later. It is not supported in clusters of v1.13 or earlier.** | - | | | - | | This parameter indicates the interval between consecutive scaling operations. The cooldown period ensures that a scaling operation is initiated only when the previous one is completed and the system is running stably. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Rules | Policy rules can be based on system metrics or custom metrics. | - | | | - | | **System Policy** | - | | | - | | - **Metric**: You can select **CPU usage** or **Memory usage**. | - | | | - | | .. note:: | - | | | - | | Usage = CPUs or memory used by pods/Requested CPUs or memory. | - | | | - | | - **Expected Value**: Enter the expected average resource usage. | - | | | - | | This parameter indicates the expected value of the selected metric. The number of new pods required (rounded up) = Current metric value/Expected value x Number of current pods | - | | | - | | - **Threshold**: Enter the scaling thresholds. | - | | | - | | If the metric value is greater than the scale-in threshold and less than the scale-out threshold, no scaling is triggered. **This parameter is supported only in clusters of v1.15 or later.** | - | | | - | | **Custom policy (supported only in clusters of v1.15 or later)** | - | | | - | | - **Metric Name**: name of the custom metric. You can select a name as prompted. | - | | | - | | For details, see :ref:`Custom Monitoring `. | - | | | - | | - **Source**: Select an object type from the drop-down list. You can select **Pod**. | - | | | - | | - **Expected Value**: the average metric value of all pods. | - | | | - | | - **Threshold**: Triggers scaling only when the metric value is within the thresholds. | - | | | - | | .. note:: | - | | | - | | When calculating the number of pods to be added or reduced, the HPA policy uses the maximum number of pods in the last 5 minutes. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==============================================================+===========================================================================================================================================================================================================================+ + | Policy Name | Name of the policy to be created. Set this parameter as required. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Namespace | Namespace to which the workload belongs. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Associated Workload | Workload with which the HPA policy is associated. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pod Range | Minimum and maximum numbers of pods. | + | | | + | | When a policy is triggered, the workload pods are scaled within this range. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Cooldown Period | Interval between a scale-in and a scale-out. The unit is minute. **The interval cannot be shorter than 1 minute.** | + | | | + | | **This parameter is available only for clusters of v1.15 and later. It is not supported in clusters of v1.13 or earlier.** | + | | | + | | This parameter indicates the interval between consecutive scaling operations. The cooldown period ensures that a scaling operation is initiated only when the previous one is completed and the system is running stably. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | System Policy | - **Metric**: You can select **CPU usage** or **Memory usage**. | + | | | + | | .. note:: | + | | | + | | Usage = CPUs or memory used by pods/Requested CPUs or memory. | + | | | + | | - **Desired Value**: Enter the desired average resource usage. | + | | | + | | This parameter indicates the desired value of the selected metric. Number of pods to be scaled (rounded up) = (Current metric value/Desired value) x Number of current pods | + | | | + | | .. note:: | + | | | + | | When calculating the number of pods to be added or reduced, the HPA policy uses the maximum number of pods in the last 5 minutes. | + | | | + | | - **Tolerance Range**: Scaling is not triggered when the metric value is within the tolerance range. The desired value must be within the tolerance range. | + | | | + | | If the metric value is greater than the scale-in threshold and less than the scale-out threshold, no scaling is triggered. **This parameter is supported only in clusters of v1.15 or later.** | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Custom Policy (supported only in clusters of v1.15 or later) | .. note:: | + | | | + | | Before setting a custom policy, you need to install an add-on that supports custom metric collection in the cluster, for example, prometheus add-on. | + | | | + | | - **Metric Name**: name of the custom metric. You can select a name as prompted. | + | | | + | | For details, see :ref:`Custom Monitoring `. | + | | | + | | - **Metric Source**: Select an object type from the drop-down list. You can select **Pod**. | + | | | + | | - **Desired Value**: the average metric value of all pods. Number of pods to be scaled (rounded up) = (Current metric value/Desired value) x Number of current pods | + | | | + | | .. note:: | + | | | + | | When calculating the number of pods to be added or reduced, the HPA policy uses the maximum number of pods in the last 5 minutes. | + | | | + | | - **Tolerance Range**: Scaling is not triggered when the metric value is within the tolerance range. The desired value must be within the tolerance range. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. After the configuration is complete, click **Create**. If the system displays a message indicating that the request to create workload policy \**\* is successfully submitted, click **Back to Workload Scaling**. - -#. On the **Workload Scaling** tab page, you can view the newly created HPA policy. +#. Click **Create**. diff --git a/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst b/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst index 8f11ce0..d38baef 100644 --- a/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst +++ b/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst @@ -21,7 +21,11 @@ You can view the rules, status, and events of an HPA policy and handle exception .. note:: - You can also view the created HPA policy on the workload details page. Log in to the CCE console, choose **Workloads > Deployments** or **Workloads > StatefulSets** in the navigation pane, and choose **More** > **Scaling** in the **Operation** column. On the workload details page, click the **Scaling** tab. You can see the **Auto Scaling-HPA** pane, as well as the HPA policy you have configured on the **Auto Scaling** page. + You can also view the created HPA policy on the workload details page. + + a. Log in to the CCE console and access the cluster console. + b. In the navigation pane, choose **Workloads**. Click the workload name to view its details. + c. On the workload details page, swich to the **Auto Scaling** tab page to view the HPA policies. You can also view the scaling policies you configured in **Workload Scaling**. .. table:: **Table 1** Event types and names diff --git a/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst b/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst index a5fffd0..6a2ccb3 100644 --- a/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst +++ b/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst @@ -133,7 +133,7 @@ Creating a Node Pool and a Node Scaling Policy #. Set node pool parameters, add a node with 2 vCPUs and 4 GB memory, and enable auto scaling. - **Nodes**: Set it to **1**, indicating that one node is created by default when a node pool is created. - - Auto Scaling: Enable the option, meaning that nodes will be automatically created or deleted in the node pool based on the cluster loads. + - **Auto Scaling**: Enable the option, meaning that nodes will be automatically created or deleted in the node pool based on the cluster loads. - **Max. Nodes**: Set it to **5**, indicating the maximum number of nodes in a node pool. - **Specifications**: 2 vCPUs \| 4 GiB diff --git a/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst b/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst index 198412e..91a6a04 100644 --- a/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst +++ b/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst @@ -133,7 +133,7 @@ Creating a Node Pool and a Node Scaling Policy #. Set node pool parameters, add a node with 2 vCPUs and 4 GB memory, and enable auto scaling. - **Nodes**: Set it to **1**, indicating that one node is created by default when a node pool is created. - - Auto Scaling: Enable the option, meaning that nodes will be automatically created or deleted in the node pool based on the cluster loads. + - **Auto Scaling**: Enable the option, meaning that nodes will be automatically created or deleted in the node pool based on the cluster loads. - **Max. Nodes**: Set it to **5**, indicating the maximum number of nodes in a node pool. - **Specifications**: 2 vCPUs \| 4 GiB diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst index df1d142..37da7e9 100644 --- a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst @@ -156,4 +156,4 @@ The storage infrastructure of an on-premises cluster is different from that of a #. After the restoration is complete, check whether the application is running properly. If other adaptation problems may occur, rectify the fault by following the procedure described in :ref:`Updating Resources Accordingly `. -.. |image1| image:: /_static/images/en-us_image_0000001171703840.png +.. |image1| image:: /_static/images/en-us_image_0000001480191270.png diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst index bf93bff..21126fc 100644 --- a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst @@ -18,7 +18,7 @@ CCE allows you to customize cluster resources to meet various service requiremen +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ | Resource | Key Performance Parameter | Description | Example Value | +=================+===========================================================+==============================================================================================================================================================================================================================================================================================================================================================================================+=================================================================+ - | Cluster | **\***\ Cluster Type | - CCE cluster: supports VM nodes. You can run your containers in a secure and stable container runtime environment based on a high-performance network model. | CCE cluster | + | Cluster | **\***\ Cluster Type | - **CCE cluster**: supports VM nodes. You can run your containers in a secure and stable container runtime environment based on a high-performance network model. | CCE cluster | | | | - **CCE Turbo cluster**: runs on a cloud native infrastructure that features software-hardware synergy to support passthrough networking, high security and reliability, intelligent scheduling, and BMS nodes. | | +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ | | **\***\ Network Model | - **VPC network**: The container network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. | VPC network | diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst index 2acb129..0824c54 100644 --- a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst @@ -20,7 +20,7 @@ The WordPress and MySQL images used in this example can be pulled from SWR. Ther .. code-block:: - 'swr.{Region}.otc.t-systems.com/{Organization name}/{Image name}:{Tag} + 'swr.{Region}.otc.t-systems.com/{Organization name}/{Image name}:{Tag}' #. Run the following command to modify the workload and replace the **image** field in the YAML file with the image path: diff --git a/umn/source/helms/deploying_an_application_from_a_chart.rst b/umn/source/charts/deploying_an_application_from_a_chart.rst similarity index 99% rename from umn/source/helms/deploying_an_application_from_a_chart.rst rename to umn/source/charts/deploying_an_application_from_a_chart.rst index ab2b364..3897862 100644 --- a/umn/source/helms/deploying_an_application_from_a_chart.rst +++ b/umn/source/charts/deploying_an_application_from_a_chart.rst @@ -94,7 +94,7 @@ Creating a Release #. Log in to the CCE console, click the cluster name, and access the cluster console. In the navigation pane, choose **Charts**. -#. In the list of uploaded charts, click **Install Chart**. +#. In the list of uploaded charts, click **Install**. #. Set workload installation parameters by referring to :ref:`Table 2 `. diff --git a/umn/source/helms/index.rst b/umn/source/charts/index.rst similarity index 94% rename from umn/source/helms/index.rst rename to umn/source/charts/index.rst index 1264304..83d69ba 100644 --- a/umn/source/helms/index.rst +++ b/umn/source/charts/index.rst @@ -2,8 +2,8 @@ .. _cce_10_0019: -Helms -===== +Charts +====== - :ref:`Overview ` - :ref:`Deploying an Application from a Chart ` diff --git a/umn/source/helms/overview.rst b/umn/source/charts/overview.rst similarity index 100% rename from umn/source/helms/overview.rst rename to umn/source/charts/overview.rst diff --git a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst index d0bcfb6..28b686b 100644 --- a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst +++ b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst @@ -25,7 +25,7 @@ Resource Changes and Deprecations **Kubernetes 1.24 Release Notes** - Beta APIs are disabled by default. When some long-term beta APIs are removed from Kubernetes, 90% cluster administrators do not care about the beta APIs. Beta features are not recommended in the production environment. However, due to the default enabling policy, these APIs are enabled in the production environment, incurring risks. Therefore, in v1.24 and later versions, beta APIs are disabled by default except for the enabled beta APIs. -- The LegacyServiceAccountTokenNoAutoGeneration feature is in beta state. By default, this feature is enabled and no more secret token will be automatically generated for the service account. If you want to use a token that never expires, you need to create a secret and mount it. For details, see `Service account token Secrets `__. +- The LegacyServiceAccountTokenNoAutoGeneration feature is in beta state. By default, this feature is enabled and no more secret token will be automatically generated for the service account. If you want to use a token that never expires, you need to create a secret and mount it. For details, see `Service account token secrets `__. - **service.alpha.kubernetes.io/tolerate-unready-endpoints** is replaced by **Service.spec.publishNotReadyAddresses**. - The **Service.Spec.LoadBalancerIP** tag is deprecated and may be removed in later versions. Use a customized annotation. diff --git a/umn/source/clusters/creating_a_cce_cluster.rst b/umn/source/clusters/creating_a_cce_cluster.rst index ec0b6eb..34da41e 100644 --- a/umn/source/clusters/creating_a_cce_cluster.rst +++ b/umn/source/clusters/creating_a_cce_cluster.rst @@ -16,6 +16,7 @@ Notes and Constraints - You can create a maximum of 50 clusters in a single region. - After a cluster is created, the following items cannot be changed: + - Cluster type - Number of master nodes in the cluster - AZ of a master node - Network configuration of the cluster, such as the VPC, subnet, container CIDR block, Service CIDR block, and kube-proxy (forwarding) settings @@ -30,11 +31,11 @@ Procedure **Basic Settings** - - Cluster Name + - **Cluster Name** - - Cluster Version: Select the Kubernetes version used by the cluster. + - **Cluster Version**: Select the Kubernetes version used by the cluster. - - Cluster Scale: maximum number of nodes that can be managed by the cluster. + - **Cluster Scale**: maximum number of nodes that can be managed by the cluster. - **HA**: distribution mode of master nodes. By default, master nodes are randomly distributed in different AZs to improve DR capabilities. @@ -75,6 +76,8 @@ Procedure - The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. **If the certificate is invalid, the cluster cannot be created**. - Starting from v1.25, Kubernetes no longer supports certificate authentication generated using the SHA1WithRSA or ECDSAWithSHA1 algorithm. You are advised to use the SHA256 algorithm. + - **Description**: The value can contain a maximum of 200 English characters. + #. Click **Next: Add-on Configuration**. By default, :ref:`cordens ` and :ref:`everest ` add-ons are installed. diff --git a/umn/source/clusters/creating_a_cce_turbo_cluster.rst b/umn/source/clusters/creating_a_cce_turbo_cluster.rst index 3f06dc4..79cd235 100644 --- a/umn/source/clusters/creating_a_cce_turbo_cluster.rst +++ b/umn/source/clusters/creating_a_cce_turbo_cluster.rst @@ -40,7 +40,7 @@ Procedure - **HA**: distribution mode of master nodes. By default, master nodes are randomly distributed in different AZs to improve DR capabilities. - You can also expand advanced settings and customize the master node distribution mode. The following two modes are supported: + You can also expand advanced settings and customize the master node distribution mode. The following modes are supported: - **Host**: Master nodes are created on different hosts in the same AZ. - **Custom**: You can determine the location of each master node. @@ -82,6 +82,8 @@ Procedure - The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. **If the certificate is invalid, the cluster cannot be created**. - Starting from v1.25, Kubernetes no longer supports certificate authentication generated using the SHA1WithRSA or ECDSAWithSHA1 algorithm. You are advised to use the SHA256 algorithm. + - **Description**: The value can contain a maximum of 200 English characters. + #. Click **Next: Add-on Configuration**. By default, :ref:`cordens ` and :ref:`everest ` add-ons are installed. diff --git a/umn/source/clusters/index.rst b/umn/source/clusters/index.rst index 46196a3..cf95826 100644 --- a/umn/source/clusters/index.rst +++ b/umn/source/clusters/index.rst @@ -1,6 +1,6 @@ -:original_name: cce_10_0027.html +:original_name: cce_10_0091.html -.. _cce_10_0027: +.. _cce_10_0091: Clusters ======== diff --git a/umn/source/clusters/managing_a_cluster/cluster_overload_control.rst b/umn/source/clusters/managing_a_cluster/cluster_overload_control.rst new file mode 100644 index 0000000..2b33afd --- /dev/null +++ b/umn/source/clusters/managing_a_cluster/cluster_overload_control.rst @@ -0,0 +1,57 @@ +:original_name: cce_10_0602.html + +.. _cce_10_0602: + +Cluster Overload Control +======================== + +Scenario +-------- + +If overload control is enabled, concurrent requests are dynamically controlled based on the resource pressure of master nodes to keep them and the cluster available. + +Notes and Constraints +--------------------- + +The cluster version must be 1.23 or later. + +Enabling Overload Control +------------------------- + +**Method 1: Enabling it when creating a cluster** + +When creating a cluster of v1.23 or later, you can enable overload control during the cluster creation. + +**Method 2: Enabling it in an existing cluster** + +#. Log in to the CCE console and go to an existing cluster whose version is v1.23 or later. +#. On the cluster information page, view the master node information. If overload control is not enabled, a message is displayed. You can click **Start Now** to enable the function. + +Overload Monitoring +------------------- + +**Method 1: Using the CCE console** + +#. Log in to the CCE console and go to an existing cluster whose version is v1.23 or later. + +#. On the cluster information page, view the master node information. The overload level metric is displayed. + + The overload levels are as follows: + + - Circuit breaking: Rejects all external traffic. + - Severe overload: Rejects 75% external traffic. + - Moderate overload: Rejects 50% external traffic. + - Slight overload: Rejects 25% external traffic. + - Normal: Does not reject external traffic. + +**Method 2: Using the AOM concole** + +You can log in to the AOM console, create a dashboard, and add the metric named **vein_overload_level**. + +The meanings of the monitoring metrics are as follows: + +- 0: Circuit breaking: Rejects all external traffic. +- 1: Severe overload: Rejects 75% external traffic. +- 2: Moderate overload: Rejects 50% external traffic. +- 3: Slight overload: Rejects 25% external traffic. +- 4: Normal: Does not reject external traffic. diff --git a/umn/source/clusters/managing_a_cluster/index.rst b/umn/source/clusters/managing_a_cluster/index.rst index 58f376e..e4b3ac6 100644 --- a/umn/source/clusters/managing_a_cluster/index.rst +++ b/umn/source/clusters/managing_a_cluster/index.rst @@ -5,14 +5,16 @@ Managing a Cluster ================== +- :ref:`Managing Cluster Components ` - :ref:`Deleting a Cluster ` - :ref:`Hibernating and Waking Up a Cluster ` -- :ref:`Managing Cluster Components ` +- :ref:`Cluster Overload Control ` .. toctree:: :maxdepth: 1 :hidden: + managing_cluster_components deleting_a_cluster hibernating_and_waking_up_a_cluster - managing_cluster_components + cluster_overload_control diff --git a/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst b/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst index 85b3ad1..f6fc814 100644 --- a/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst +++ b/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst @@ -22,7 +22,7 @@ Procedure #. Click |image1| next to the target cluster. #. On the **Manage Component** page on the right, change the values of the following Kubernetes parameters: - .. table:: **Table 1** external-controller + .. table:: **Table 1** Extended controller parameters +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ | Parameter | Description | Value | @@ -35,43 +35,46 @@ Procedure .. table:: **Table 2** kube-apiserver parameters - +----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ - | Parameter | Description | Value | - +========================================+=======================================================================================================================================+=======================+ - | default-not-ready-toleration-seconds | notReady tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300s | - +----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ - | default-unreachable-toleration-seconds | unreachable tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300s | - +----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ - | max-mutating-requests-inflight | Maximum number of concurrent mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 1000 | - | | | | - | | The value **0** indicates no limitation. | | - | | | | - | | Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. | | - | | | | - | | - **200** for clusters with 50 or 200 nodes | | - | | - **500** for clusters with 1,000 nodes | | - | | - **1000** for clusters with 2,000 nodes | | - +----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ - | max-requests-inflight | Maximum number of concurrent non-mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 2000 | - | | | | - | | The value **0** indicates no limitation. | | - | | | | - | | Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. | | - | | | | - | | - **400** for clusters with 50 or 200 nodes | | - | | - **1000** for clusters with 1,000 nodes | | - | | - **2000** for clusters with 2,000 nodes | | - +----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ - | service-node-port-range | Range of node port numbers. | Default: | - | | | | - | | | 30000-32767 | - | | | | - | | | Options: | - | | | | - | | | min>20105 | - | | | | - | | | max<32768 | - +----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | Parameter | Description | Value | + +========================================+=============================================================================================================================================================================+=========================================+ + | default-not-ready-toleration-seconds | notReady tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300s | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | default-unreachable-toleration-seconds | unreachable tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300s | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | max-mutating-requests-inflight | Maximum number of concurrent mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 1000 | + | | | | + | | The value **0** indicates no limitation. | | + | | | | + | | Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. | | + | | | | + | | - **200** for clusters with 50 or 200 nodes | | + | | - **500** for clusters with 1,000 nodes | | + | | - **1000** for clusters with 2,000 nodes | | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | max-requests-inflight | Maximum number of concurrent non-mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 2000 | + | | | | + | | The value **0** indicates no limitation. | | + | | | | + | | Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. | | + | | | | + | | - **400** for clusters with 50 or 200 nodes | | + | | - **1000** for clusters with 1,000 nodes | | + | | - **2000** for clusters with 2,000 nodes | | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | service-node-port-range | Range of node port numbers. | Default: | + | | | | + | | | 30000-32767 | + | | | | + | | | Options: | + | | | | + | | | min>20105 | + | | | | + | | | max<32768 | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | support-overload | Cluster overload control. If enabled, concurrent requests are dynamically controlled based on the resource pressure of master nodes to keep them and the cluster available. | - false: Overload control is disabled. | + | | | - true: Overload control is enabled. | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ .. table:: **Table 3** kube-controller-manager parameters @@ -121,7 +124,7 @@ Procedure | kube-api-burst | Burst to use while talking with kube-apiserver. | Default: 100 | +----------------+------------------------------------------------------------------+--------------+ - .. table:: **Table 5** eni (supported only by CCE Turbo clusters) + .. table:: **Table 5** eni parameters (supported only by CCE Turbo clusters) +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ | Parameter | Description | Value | diff --git a/umn/source/clusters/obtaining_a_cluster_certificate.rst b/umn/source/clusters/obtaining_a_cluster_certificate.rst index d48f327..2bc42d4 100644 --- a/umn/source/clusters/obtaining_a_cluster_certificate.rst +++ b/umn/source/clusters/obtaining_a_cluster_certificate.rst @@ -13,7 +13,7 @@ This section describes how to obtain the cluster certificate from the console an Procedure --------- -#. Log in to the CCE console and click the cluster name to access the cluster. +#. Log in to the CCE console and access the cluster console. #. Choose **Cluster Information** from the navigation pane and click **Download** next to **Authentication Mode** in the **Connection Information** area. diff --git a/umn/source/clusters/upgrading_a_cluster/before_you_start.rst b/umn/source/clusters/upgrading_a_cluster/before_you_start.rst index f19e002..4e31c57 100644 --- a/umn/source/clusters/upgrading_a_cluster/before_you_start.rst +++ b/umn/source/clusters/upgrading_a_cluster/before_you_start.rst @@ -59,7 +59,7 @@ Notes and Constraints Upgrade Backup -------------- -Currently, there are two backup modes for cluster upgrade: +How to back up a node: - etcd database backup: CCE automatically backs up the etcd database during the cluster upgrade. - Master node backup (recommended, **manual confirmation required**): On the upgrade confirmation page, click **Backup** to back up the entire master node of the cluster. The backup process uses the Cloud Backup and Recovery (CBR) service and takes about 20 minutes. If there are many cloud backup tasks at the current site, the backup time may be prolonged. diff --git a/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst b/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst index 523b16b..fba3587 100644 --- a/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst +++ b/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst @@ -54,7 +54,7 @@ Procedure #. **Create a workload in the new cluster.** - The workload name and specifications remain unchanged. For details about how to create a workload, see :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `. For details about how to attach a storage volume to the workload, see :ref:`Creating a Pod Mounted with an EVS Volume `. + The workload name and specifications remain unchanged. For details about how to create a workload, see :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `. For details about how to attach a storage volume to the workload, see :ref:`Creating a Deployment Mounted with an EVS Volume `. #. **Create a Service in the new cluster.** diff --git a/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst b/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst index 1288da7..6dcb5f3 100644 --- a/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst +++ b/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst @@ -18,7 +18,7 @@ An upgrade flag will be displayed on the cluster card view if there is a new ver Log in to the CCE console and check whether the message "New version available" is displayed in the lower left corner of the cluster. If yes, the cluster can be upgraded. If no, the cluster cannot be upgraded. -.. figure:: /_static/images/en-us_image_0000001199501182.png +.. figure:: /_static/images/en-us_image_0000001482796460.png :alt: **Figure 1** Cluster with the upgrade flag **Figure 1** Cluster with the upgrade flag @@ -84,7 +84,7 @@ Precautions for Major Version Upgrade | | Root cause: X.509 `CommonName `__ is discarded in Go 1.15. kube-apiserver of CCE 1.19 is compiled using Go 1.15. If your webhook certificate does not have SANs, kube-apiserver does not process the **CommonName** field of the X.509 certificate as the host name by default. As a result, the authentication fails. | - If you do not have your own webhook server, you can skip this check. | | | | - If the field is not set, you are advised to use the SAN field to specify the IP address and domain name supported by the certificate. | +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| v1.15 to v1.19 | The control plane of CCE 1.19 is incompatible with Kubelet 1.15. If the master node fails to be upgraded or the node to be upgraded restarts after the master node is successfully upgraded, there is a high probability that the node is in the **NotReady** state. | #. In normal cases, this scenario is not triggered. | +| v1.15 to v1.19 | The control plane of CCE 1.19 is incompatible with Kubelet 1.15. If the master node fails to be upgraded or the node to be upgraded restarts after the master node is successfully upgraded, there is a high probability that the node is in the **NotReady** status. | #. In normal cases, this scenario is not triggered. | | | | #. After the master node is upgraded, do not suspend the upgrade. Upgrade the node quickly. | | | There is a high probability that kubelet restarts on the node that fails to be upgraded, triggering the node registration process. The default registration labels of kubelet 1.15 (**failure-domain.beta.kubernetes.io/is-baremetal** and **kubernetes.io/availablezone**) are regarded as an invalid label by kube-apiserver 1.19. | #. If a node fails to be upgraded and cannot be restored, evict applications on the node as soon as possible. Contact technical support and skip the node upgrade. After the upgrade is complete, reset the node. | | | | | diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst index d373f07..6a3e620 100644 --- a/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst @@ -325,7 +325,7 @@ To listen on ports 5000 and 6000 locally, forwarding data to/from ports 5000 and .. code-block:: - kubectl port -forward podname 5000:6000 + kubectl port-forward podname 5000:6000 **proxy\*** diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/customizing_a_cluster_certificate_san.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/customizing_a_cluster_certificate_san.rst new file mode 100644 index 0000000..d0e4f39 --- /dev/null +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/customizing_a_cluster_certificate_san.rst @@ -0,0 +1,42 @@ +:original_name: cce_10_0367.html + +.. _cce_10_0367: + +Customizing a Cluster Certificate SAN +===================================== + +Scenario +-------- + +A **Subject Alternative Name (SAN)** can be signed in to a cluster server certificate. A SAN is usually used by the client to verify the server validity in TLS handshakes. Specifically, the validity check includes whether the server certificate is issued by a CA trusted by the client and whether the SAN in the certificate matches the IP address or DNS domain name that the client actually accesses. + +If the client cannot directly access the private IP or EIP of the cluster, you can sign the IP address or DNS domain name that can be directly accessed by the client into the cluster server certificate to enable two-way authentication on the client, which improves security. Typical use cases include DNAT access and domain name access. + +Notes and Constraints +--------------------- + +This feature is available only to clusters of v1.19 and later. + +Customizing a SAN +----------------- + +#. Log in to the CCE console. +#. Click the target cluster in the cluster list to go to the cluster details page. +#. In the **Connection Information** area, click |image1| next to **Custom SAN**. In the dialog box displayed, add the IP address or domain name and click **Save**. + + .. note:: + + 1. This operation will restart kube-apiserver and update the **kubeconfig.json** file for a short period of time. Do not perform operations on the cluster during this period. + + 2. A maximum of 128 domain names or IP addresses, separated by commas (,), are allowed. + + 3. If a custom domain name needs to be bound to an EIP, ensure that an EIP has been configured. + +Typical Domain Name Access Scenarios +------------------------------------ + +- Add the response domain name mapping when specifying the DNS domain name address in the host domain name configuration on the client, or configuring **/etc/hosts** on the client host. +- Use domain name access in the intranet. DNS allows you to configure mappings between cluster EIPs and custom domain names. After an EIP is updated, you can continue to use two-way authentication and the domain name to access the cluster without downloading the **kubeconfig.json** file again. +- Add A records on a self-built DNS server. + +.. |image1| image:: /_static/images/en-us_image_0000001199341268.png diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst index b4e5b33..1e4e5b5 100644 --- a/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst @@ -6,6 +6,7 @@ Using kubectl to Run a Cluster ============================== - :ref:`Connecting to a Cluster Using kubectl ` +- :ref:`Customizing a Cluster Certificate SAN ` - :ref:`Common kubectl Commands ` .. toctree:: @@ -13,4 +14,5 @@ Using kubectl to Run a Cluster :hidden: connecting_to_a_cluster_using_kubectl + customizing_a_cluster_certificate_san common_kubectl_commands diff --git a/umn/source/conf.py b/umn/source/conf.py index 2ba353a..b502764 100644 --- a/umn/source/conf.py +++ b/umn/source/conf.py @@ -24,7 +24,7 @@ extensions = [ otcdocs_auto_name = False otcdocs_auto_version = False -project = 'Cloud Container Service' +project = 'Cloud Container Engine' otcdocs_repo_name = 'docs/cloud-container-engine' # Those variables are required for edit/bug links otcdocs_git_fqdn = 'gitea.eco.tsi-dev.otc-service.com' @@ -86,7 +86,7 @@ html_theme_options = { # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". -html_title = "Cloud Container Service - User Guide" +html_title = "Cloud Container Engine - User Guide" # Add any paths that contain custom static files (such as style sheets) here, @@ -101,6 +101,6 @@ html_copy_source = False latex_documents = [ ('index', 'None.tex', - u'Cloud Container Service - User Guide', + u'Cloud Container Engine - User Guide', u'OpenTelekomCloud', 'manual'), ] diff --git a/umn/source/index.rst b/umn/source/index.rst index fe7c686..5ab5adf 100644 --- a/umn/source/index.rst +++ b/umn/source/index.rst @@ -1,6 +1,6 @@ -==================================== -Cloud Container Service - User Guide -==================================== +=================================== +Cloud Container Engine - User Guide +=================================== .. toctree:: :maxdepth: 1 @@ -21,7 +21,7 @@ Cloud Container Service - User Guide configuration_center/index auto_scaling/index add-ons/index - helms/index + charts/index permissions_management/index cloud_trace_service_cts/index storage_flexvolume/index diff --git a/umn/source/monitoring_and_alarm/monitoring_overview.rst b/umn/source/monitoring_and_alarm/monitoring_overview.rst index adf2ada..2578d8d 100644 --- a/umn/source/monitoring_and_alarm/monitoring_overview.rst +++ b/umn/source/monitoring_and_alarm/monitoring_overview.rst @@ -53,8 +53,6 @@ Viewing Cluster Monitoring Data Click the cluster name and access the cluster console. In the navigation pane, choose **Cluster Information**. In the right pane, you can view the CPU and memory usage of all nodes (excluding master nodes) in the cluster in the last hour. -The cluster monitoring page displays the monitoring status of cluster resources, CPU, memory, and disk usage of all nodes in a cluster, and CPU and memory allocation rates. - **Explanation of monitoring metrics:** - CPU allocation rate = Sum of CPU quotas requested by pods in the cluster/Sum of CPU quotas that can be allocated of all nodes (excluding master nodes) in the cluster diff --git a/umn/source/namespaces/configuring_a_namespace-level_network_policy.rst b/umn/source/namespaces/configuring_a_namespace-level_network_policy.rst deleted file mode 100644 index 09d6fab..0000000 --- a/umn/source/namespaces/configuring_a_namespace-level_network_policy.rst +++ /dev/null @@ -1,71 +0,0 @@ -:original_name: cce_10_0286.html - -.. _cce_10_0286: - -Configuring a Namespace-level Network Policy -============================================ - -You can configure a namespace-level network policy after enabling network isolation. - -By default, **Network Isolation** is disabled for namespaces. For example, if network isolation is off for namespace **default**, **all workloads in the current cluster** can access the workloads in namespace **default**. - -To prevent other workloads from accessing the workloads in namespace **default**, perform the following steps: - -.. important:: - - - Only clusters that use the tunnel network model support network policies. - - - Network isolation is not supported for IPv6 addresses. - - - Network policies do not support egress rules except for clusters of v1.23 or later. - - Egress rules are supported only in the following operating systems: - - - EulerOS 2.9: kernel version 4.18.0-147.5.1.6.h541.eulerosv2r9.x86_64 - - CentOS 7.7: kernel version 3.10.0-1062.18.1.el7.x86_64 - - EulerOS 2.5: kernel version 3.10.0-862.14.1.5.h591.eulerosv2r7.x86_64 - - - If a cluster is upgraded to v1.23 in in-place mode, you cannot use egress rules because the node OS is not upgraded. In this case, reset the node. - -Prerequisites -------------- - -- You have created a Kubernetes cluster. For details, see :ref:`Creating a CCE Cluster `. -- You have created a namespace. For details, see :ref:`Creating a Namespace `. - -Procedure ---------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Namespaces**. - -#. Select the cluster to which the namespace belongs from the **Clusters** drop-down list. - -#. At the row of a namespace (for example, **default**), switch on **Network Isolation**. - - After network isolation is enabled, workloads in namespace **default** can access each other but they cannot be accessed by workloads in other namespaces. - - - .. figure:: /_static/images/en-us_image_0000001199021274.png - :alt: **Figure 1** Namespace-level network policy - - **Figure 1** Namespace-level network policy - -Network Isolation Description ------------------------------ - -Enabling network isolation is to create a network policy in a namespace. The network policy selects all pods in the namespace and prevents pods in other namespaces from accessing. - -.. code-block:: - - kind: NetworkPolicy - apiVersion: networking.k8s.io/v1 - metadata: - name: deny-default - namespace: default - spec: - ingress: - - from: - - podSelector: {} - podSelector: {} # {} indicates that all pods are selected. - -You can also customize a network policy. For details, see :ref:`Network Policies `. diff --git a/umn/source/namespaces/index.rst b/umn/source/namespaces/index.rst index 5289f73..ed86546 100644 --- a/umn/source/namespaces/index.rst +++ b/umn/source/namespaces/index.rst @@ -7,7 +7,6 @@ Namespaces - :ref:`Creating a Namespace ` - :ref:`Managing Namespaces ` -- :ref:`Configuring a Namespace-level Network Policy ` - :ref:`Setting a Resource Quota ` .. toctree:: @@ -16,5 +15,4 @@ Namespaces creating_a_namespace managing_namespaces - configuring_a_namespace-level_network_policy setting_a_resource_quota diff --git a/umn/source/networking/index.rst b/umn/source/networking/index.rst index 9159c55..b8697dc 100644 --- a/umn/source/networking/index.rst +++ b/umn/source/networking/index.rst @@ -8,7 +8,7 @@ Networking - :ref:`Overview ` - :ref:`Container Network Models ` - :ref:`Services ` -- :ref:`Ingress ` +- :ref:`Ingresses ` - :ref:`DNS ` - :ref:`Configuring Intra-VPC Access ` - :ref:`Accessing Public Networks from a Container ` @@ -22,7 +22,7 @@ Networking overview container_network_models/index services/index - ingress/index + ingresses/index dns/index configuring_intra-vpc_access accessing_public_networks_from_a_container diff --git a/umn/source/networking/ingress/index.rst b/umn/source/networking/ingresses/index.rst similarity index 78% rename from umn/source/networking/ingress/index.rst rename to umn/source/networking/ingresses/index.rst index 76e2899..36f8d50 100644 --- a/umn/source/networking/ingress/index.rst +++ b/umn/source/networking/ingresses/index.rst @@ -2,10 +2,10 @@ .. _cce_10_0248: -Ingress -======= +Ingresses +========= -- :ref:`Overview ` +- :ref:`Ingress Overview ` - :ref:`Using ELB Ingresses on the Console ` - :ref:`Using kubectl to Create an ELB Ingress ` @@ -13,6 +13,6 @@ Ingress :maxdepth: 1 :hidden: - overview + ingress_overview using_elb_ingresses_on_the_console using_kubectl_to_create_an_elb_ingress diff --git a/umn/source/networking/ingress/overview.rst b/umn/source/networking/ingresses/ingress_overview.rst similarity index 98% rename from umn/source/networking/ingress/overview.rst rename to umn/source/networking/ingresses/ingress_overview.rst index 8dce45d..11cc203 100644 --- a/umn/source/networking/ingress/overview.rst +++ b/umn/source/networking/ingresses/ingress_overview.rst @@ -2,8 +2,8 @@ .. _cce_10_0094: -Overview -======== +Ingress Overview +================ Why We Need Ingresses --------------------- diff --git a/umn/source/networking/ingress/using_elb_ingresses_on_the_console.rst b/umn/source/networking/ingresses/using_elb_ingresses_on_the_console.rst similarity index 90% rename from umn/source/networking/ingress/using_elb_ingresses_on_the_console.rst rename to umn/source/networking/ingresses/using_elb_ingresses_on_the_console.rst index f335b98..d05c1b6 100644 --- a/umn/source/networking/ingress/using_elb_ingresses_on_the_console.rst +++ b/umn/source/networking/ingresses/using_elb_ingresses_on_the_console.rst @@ -52,7 +52,7 @@ This section uses an Nginx workload as an example to describe how to add an ELB If there is already an HTTPS ingress for the chosen port on the load balancer, the certificate of the new HTTPS ingress must be the same as the certificate of the existing ingress. This means that a listener has only one certificate. If two certificates, each with a different ingress, are added to the same listener of the same load balancer, only the certificate added earliest takes effect on the load balancer. - - **SNI**: SNI is an extended protocol of TLS. It allows multiple TLS-based access domain names to be provided for external systems using the same IP address and port number. Different domain names can use different security certificates. After SNI is enabled, the client is allowed to submit the requested domain name when initiating a TLS handshake request. After receiving the TLS request, the load balancer searches for the certificate based on the domain name in the request. If the certificate corresponding to the domain name is found, the load balancer returns the certificate for authorization. Otherwise, the default certificate (server certificate) is returned for authorization. + - **SNI**: Server Name Indication (SNI) is an extended protocol of TLS. It allows multiple TLS-based access domain names to be provided for external systems using the same IP address and port. Different domain names can use different security certificates. After SNI is enabled, the client is allowed to submit the requested domain name when initiating a TLS handshake request. After receiving the TLS request, the load balancer searches for the certificate based on the domain name in the request. If the certificate corresponding to the domain name is found, the load balancer returns the certificate for authorization. Otherwise, the default certificate (server certificate) is returned for authorization. .. note:: @@ -74,7 +74,7 @@ This section uses an Nginx workload as an example to describe how to add an ELB - **Domain Name**: actual domain name. Ensure that the domain name has been registered and archived. Once a domain name rule is configured, you must use the domain name for access. - - Rule Matching + - **URL Matching Rule**: - **Prefix match**: If the URL is set to **/healthz**, the URL that meets the prefix can be accessed. For example, **/healthz/v1** and **/healthz/v2**. - **Exact match**: The URL can be accessed only when it is fully matched. For example, if the URL is set to **/healthz**, only /healthz can be accessed. @@ -94,7 +94,7 @@ This section uses an Nginx workload as an example to describe how to add an ELB **Destination Service Port**: Select the access port of the destination Service. - - Set ELB: + - **Set ELB**: - **Distribution Policy**: Three algorithms are available: weighted round robin, weighted least connections algorithm, or source IP hash. @@ -105,7 +105,7 @@ This section uses an Nginx workload as an example to describe how to add an ELB - **Source IP hash**: The source IP address of each request is calculated using the hash algorithm to obtain a unique hash key, and all backend servers are numbered. The generated key allocates the client to a particular server. This enables requests from different clients to be distributed in load balancing mode and ensures that requests from the same client are forwarded to the same server. This algorithm applies to TCP connections without cookies. - **Type**: This function is disabled by default. You can select **Load balancer cookie**. - - **Health Check**: This function is disabled by default. The health check is for the load balancer. When TCP is selected during the :ref:`port settings `, you can select either TCP or HTTP. By default, the service port (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a service port named **cce-healthz** will be added for the Service. + - **Health Check**: This function is disabled by default. The health check is for the load balancer. When TCP is selected during the :ref:`port settings `, you can choose either TCP or HTTP. Currently, UDP is not supported. By default, the service port (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a service port named **cce-healthz** will be added for the Service. - **Operation**: Click **Delete** to delete the configuration. diff --git a/umn/source/networking/ingress/using_kubectl_to_create_an_elb_ingress.rst b/umn/source/networking/ingresses/using_kubectl_to_create_an_elb_ingress.rst similarity index 84% rename from umn/source/networking/ingress/using_kubectl_to_create_an_elb_ingress.rst rename to umn/source/networking/ingresses/using_kubectl_to_create_an_elb_ingress.rst index 08c4a0e..b9af77d 100644 --- a/umn/source/networking/ingress/using_kubectl_to_create_an_elb_ingress.rst +++ b/umn/source/networking/ingresses/using_kubectl_to_create_an_elb_ingress.rst @@ -20,6 +20,25 @@ Prerequisites - A NodePort Service has been configured for the workload. For details about how to configure the Service, see :ref:`NodePort `. - Dedicated load balancers must be the application type (HTTP/HTTPS) supporting private networks (with a private IP). +.. _cce_10_0252__section084115985013: + +Ingress Description of networking.k8s.io/v1 +------------------------------------------- + +In CCE clusters of v1.23 or later, the ingress version is switched to networking.k8s.io/v1. + +Compared with v1beta1, v1 has the following differences in parameters: + +- The ingress type is changed from **kubernetes.io/ingress.class** in **annotations** to **spec.ingressClassName**. +- The format of **backend** is changed. +- The **pathType** parameter must be specified for each path. The options are as follows: + + - **ImplementationSpecific**: The matching method depends on Ingress Controller. The matching method defined by **ingress.beta.kubernetes.io/url-match-mode** is used in CCE, which is the same as v1beta1. + - **Exact**: exact matching of the URL, which is case-sensitive. + - **Prefix**: matching based on the URL prefix separated by a slash (/). The match is case-sensitive, and elements in the path are matched one by one. A path element refers to a list of labels in the path separated by a slash (/). + +|image1| + .. _cce_10_0252__section3675115714214: Creating an Ingress - Automatically Creating a Load Balancer @@ -33,6 +52,10 @@ The following describes how to run the kubectl command to automatically create a **vi ingress-test.yaml** + .. note:: + + Starting from cluster v1.23, the ingress version is switched from **networking.k8s.io/v1beta1** to **networking.k8s.io/v1**. For details about the differences between v1 and v1beta1, see :ref:`Ingress Description of networking.k8s.io/v1 `. + **Example of a shared load balancer (public network access) for clusters of v1.23 or later:** .. code-block:: @@ -107,141 +130,147 @@ The following describes how to run the kubectl command to automatically create a .. code-block:: apiVersion: networking.k8s.io/v1 - kind: Ingress - metadata: - name: ingress-test - namespace: default - annotations: - kubernetes.io/elb.class: performance - kubernetes.io/elb.port: '80' - kubernetes.io/elb.autocreate: - '{ - "type": "public", - "bandwidth_name": "cce-bandwidth-******", - "bandwidth_chargemode": "bandwidth", - "bandwidth_size": 5, - "bandwidth_sharetype": "PER", - "eip_type": "5_bgp", - "available_zone": [ - "eu-de-01" - ], - "l7_flavor_name": "L7_flavor.elb.s1.small" - }' - spec: - rules: - - host: '' - http: - paths: - - path: '/' - backend: - service: - name: # Replace it with the name of your target Service. - port: - number: 8080 # Replace 8080 with the port number of your target Service. - property: - ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH - pathType: ImplementationSpecific - ingressClassName: cce + kind: Ingress + metadata: + name: ingress-test + namespace: default + annotations: + kubernetes.io/elb.class: performance + kubernetes.io/elb.port: '80' + kubernetes.io/elb.autocreate: + '{ + "type": "public", + "bandwidth_name": "cce-bandwidth-******", + "bandwidth_chargemode": "bandwidth", + "bandwidth_size": 5, + "bandwidth_sharetype": "PER", + "eip_type": "5_bgp", + "available_zone": [ + "eu-de-01" + ], + "l7_flavor_name": "L7_flavor.elb.s1.small" + }' + spec: + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce - **Example of a dedicated load balancer (public network access) for clusters of v1.21 or earlier:** + **Example of a dedicated load balancer (public network access) for clusters of version 1.21 or earlier:** .. code-block:: apiVersion: networking.k8s.io/v1beta1 - kind: Ingress - metadata: - name: ingress-test - namespace: default - annotations: - kubernetes.io/elb.class: performance - kubernetes.io/ingress.class: cce - kubernetes.io/elb.port: '80' - kubernetes.io/elb.autocreate: - '{ - "type": "public", - "bandwidth_name": "cce-bandwidth-******", - "bandwidth_chargemode": "traffic", - "bandwidth_size": 5, - "bandwidth_sharetype": "PER", - "eip_type": "5_bgp", - "available_zone": [ - "eu-de-01" - ], - "l7_flavor_name": "L7_flavor.elb.s1.small" - }' - spec: - rules: - - host: '' - http: - paths: - - path: '/' - backend: - serviceName: # Replace it with the name of your target Service. - servicePort: 80 - property: - ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + kind: Ingress + metadata: + name: ingress-test + namespace: default + annotations: + kubernetes.io/elb.class: performance + kubernetes.io/ingress.class: cce + kubernetes.io/elb.port: '80' + kubernetes.io/elb.autocreate: + '{ + "type": "public", + "bandwidth_name": "cce-bandwidth-******", + "bandwidth_chargemode": "bandwidth", + "bandwidth_size": 5, + "bandwidth_sharetype": "PER", + "eip_type": "5_bgp", + "available_zone": [ + "eu-de-01" + ], + "l7_flavor_name": "L7_flavor.elb.s1.small" + }' + spec: + rules: + - host: '' + http: + paths: + - path: '/' + backend: + serviceName: # Replace it with the name of your target Service. + servicePort: 80 + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH .. table:: **Table 1** Key parameters - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Mandatory | Type | Description | - +===========================================+=========================================+=======================+======================================================================================================================================================================================+ - | kubernetes.io/elb.class | Yes | String | Select a proper load balancer type. | - | | | | | - | | | | The value can be: | - | | | | | - | | | | - **union**: shared load balancer | - | | | | - **performance**: dedicated load balancer.. | - | | | | | - | | | | Default: **union** | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/ingress.class | Yes | String | **cce**: The self-developed ELB ingress is used. | - | | | | | - | | (only for clusters of v1.21 or earlier) | | This parameter is mandatory when an ingress is created by calling the API. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | ingressClassName | Yes | String | **cce**: The self-developed ELB ingress is used. | - | | | | | - | | (only for clusters of v1.23 or later) | | This parameter is mandatory when an ingress is created by calling the API. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.port | Yes | Integer | This parameter indicates the external port registered with the address of the LoadBalancer Service. | - | | | | | - | | | | Supported range: 1 to 65535 | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.subnet-id | ``-`` | String | ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | - | | | | | - | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | - | | | | - Optional for clusters later than v1.11.7-r0. It is left blank by default. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.autocreate | Yes | elb.autocreate object | Whether to automatically create a load balancer associated with an ingress. For details about the field description, see :ref:`Table 2 `. | - | | | | | - | | | | **Example** | - | | | | | - | | | | - If a public network load balancer will be automatically created, set this parameter to the following value: | - | | | | | - | | | | {"type":"public","bandwidth_name":"cce-bandwidth-``******``","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"} | - | | | | | - | | | | - If a private network load balancer will be automatically created, set this parameter to the following value: | - | | | | | - | | | | {"type":"inner","name":"A-location-d-test"} | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | host | No | String | Domain name for accessing the Service. By default, this parameter is left blank, and the domain name needs to be fully matched. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | path | Yes | String | User-defined route path. All external access requests must match **host** and **path**. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Yes | String | Name of the target Service bound to the ingress. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | servicePort | Yes | Integer | Access port of the target Service. | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | ingress.beta.kubernetes.io/url-match-mode | No | String | Route matching policy. | - | | | | | - | | | | Default: **STARTS_WITH** (prefix match) | - | | | | | - | | | | Options: | - | | | | | - | | | | - **EQUAL_TO**: exact match | - | | | | - **STARTS_WITH**: prefix match | - | | | | - **REGEX**: regular expression match | - +-------------------------------------------+-----------------------------------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +===========================================+=========================================+=======================+=========================================================================================================================================================================================================================================+ + | kubernetes.io/elb.class | Yes | String | Select a proper load balancer type. | + | | | | | + | | | | The value can be: | + | | | | | + | | | | - **union**: shared load balancer | + | | | | - **performance**: dedicated load balancer.. | + | | | | | + | | | | Default: **union** | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/ingress.class | Yes | String | **cce**: The self-developed ELB ingress is used. | + | | | | | + | | (only for clusters of v1.21 or earlier) | | This parameter is mandatory when an ingress is created by calling the API. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ingressClassName | Yes | String | **cce**: The self-developed ELB ingress is used. | + | | | | | + | | (only for clusters of v1.23 or later) | | This parameter is mandatory when an ingress is created by calling the API. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.port | Yes | Integer | This parameter indicates the external port registered with the address of the LoadBalancer Service. | + | | | | | + | | | | Supported range: 1 to 65535 | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.subnet-id | ``-`` | String | ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | + | | | | | + | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | + | | | | - Optional for clusters later than v1.11.7-r0. It is left blank by default. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.autocreate | Yes | elb.autocreate object | Whether to automatically create a load balancer associated with an ingress. For details about the field description, see :ref:`Table 2 `. | + | | | | | + | | | | **Example** | + | | | | | + | | | | - If a public network load balancer will be automatically created, set this parameter to the following value: | + | | | | | + | | | | {"type":"public","bandwidth_name":"cce-bandwidth-``******``","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"} | + | | | | | + | | | | - If a private network load balancer will be automatically created, set this parameter to the following value: | + | | | | | + | | | | {"type":"inner","name":"A-location-d-test"} | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | host | No | String | Domain name for accessing the Service. By default, this parameter is left blank, and the domain name needs to be fully matched. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | path | Yes | String | User-defined route path. All external access requests must match **host** and **path**. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | serviceName | Yes | String | Name of the target Service bound to the ingress. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | servicePort | Yes | Integer | Access port of the target Service. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ingress.beta.kubernetes.io/url-match-mode | No | String | Route matching policy. | + | | | | | + | | | | Default: **STARTS_WITH** (prefix match) | + | | | | | + | | | | Options: | + | | | | | + | | | | - **EQUAL_TO**: exact match | + | | | | - **STARTS_WITH**: prefix match | + | | | | - **REGEX**: regular expression match | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | pathType | Yes | String | Path type. This field is supported only by clusters of v1.23 or later. | + | | | | | + | | | | - **ImplementationSpecific**: The matching method depends on Ingress Controller. The matching method defined by **ingress.beta.kubernetes.io/url-match-mode** is used in CCE. | + | | | | - **Exact**: exact matching of the URL, which is case-sensitive. | + | | | | - **Prefix**: matching based on the URL prefix separated by a slash (/). The match is case-sensitive, and elements in the path are matched one by one. A path element refers to a list of labels in the path separated by a slash (/). | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. _cce_10_0252__table268711532210: @@ -730,3 +759,5 @@ Ingresses can route requests to multiple backend Services based on different mat servicePort: 80 property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + +.. |image1| image:: /_static/images/en-us_image_0000001276433425.png diff --git a/umn/source/networking/network_policies.rst b/umn/source/networking/network_policies.rst index 5a56392..231d442 100644 --- a/umn/source/networking/network_policies.rst +++ b/umn/source/networking/network_policies.rst @@ -190,19 +190,39 @@ Creating a Network Policy on the Console .. _cce_10_0059__table166419994515: - .. table:: **Table 1** Parameters for adding a rule + .. table:: **Table 1** Adding an inbound rule - +------------------+--------------------------------------------------------------------------+ - | Parameter | Description | - +==================+==========================================================================+ - | Protocol & Port | Select the protocol type and port. Currently, TCP and UDP are supported. | - +------------------+--------------------------------------------------------------------------+ - | Source Namespace | Select a namespace whose objects can be accessed. | - +------------------+--------------------------------------------------------------------------+ - | Source Pod Label | Select the pod that allows this label to access. | - +------------------+--------------------------------------------------------------------------+ + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==================+==============================================================================================================================================================+ + | Protocol & Port | Select the protocol type and port. Currently, TCP and UDP are supported. | + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Source Namespace | Select a namespace whose objects can be accessed. If this parameter is not specified, the source object belongs to the same namespace as the current policy. | + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Source Pod Label | Allow access to the pods with this label, if not specified, all pods in the namespace can be accessed. | + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + - **Outbound Rule**: Click |image3| to add an outbound rule. For details about parameter settings, see :ref:`Table 1 `. + + |image4| + + .. table:: **Table 2** Adding an outbound rule + + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +========================+===================================================================================================================================================================================================================================================================================================================================================================================+ + | Protocol & Port | Select the protocol type and port. Currently, TCP and UDP are supported. If this parameter is not specified, the protocol type is not limited. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Destination CIDR Block | Allows requests to be routed to a specified CIDR block (and not to the exception CIDR blocks). Separate the destination and exception CIDR blocks by vertical bars (|), and separate multiple exception CIDR blocks by commas (,). For example, 172.17.0.0/16|172.17.1.0/24,172.17.2.0/24 indicates that 172.17.0.0/16 is accessible, but not for 172.17.1.0/24 or 172.17.2.0/24. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Destination Namespace | Select a namespace whose objects can be accessed. If this parameter is not specified, the source object belongs to the same namespace as the current policy. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Destination Pod Label | Allow access to the pods with this label, if not specified, all pods in the namespace can be accessed. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Click **OK**. .. |image1| image:: /_static/images/en-us_image_0000001251716033.png -.. |image2| image:: /_static/images/en-us_image_0000001207036074.png +.. |image2| image:: /_static/images/en-us_image_0000001533585325.png +.. |image3| image:: /_static/images/en-us_image_0000001533586881.png +.. |image4| image:: /_static/images/en-us_image_0000001482546084.png diff --git a/umn/source/networking/overview.rst b/umn/source/networking/overview.rst index adfd150..2211858 100644 --- a/umn/source/networking/overview.rst +++ b/umn/source/networking/overview.rst @@ -58,7 +58,7 @@ You can configure the following types of Services: - NodePort: used for access from outside a cluster. A NodePort Service is accessed through the port on the node. - LoadBalancer: used for access from outside a cluster. It is an extension of NodePort, to which a load balancer routes, and external systems only need to access the load balancer. -For details about the Service, see :ref:`Overview `. +For details about the Service, see :ref:`Service Overview `. .. _cce_10_0010__section1248852094313: @@ -73,7 +73,7 @@ Services forward requests using layer-4 TCP and UDP protocols. Ingresses forward **Figure 2** Ingress and Service -For details about the ingress, see :ref:`Overview `. +For details about the ingress, see :ref:`Ingress Overview `. .. _cce_10_0010__section1286493159: diff --git a/umn/source/networking/services/index.rst b/umn/source/networking/services/index.rst index 5a667c2..70d3e66 100644 --- a/umn/source/networking/services/index.rst +++ b/umn/source/networking/services/index.rst @@ -5,7 +5,7 @@ Services ======== -- :ref:`Overview ` +- :ref:`Service Overview ` - :ref:`Intra-Cluster Access (ClusterIP) ` - :ref:`NodePort ` - :ref:`LoadBalancer ` @@ -16,7 +16,7 @@ Services :maxdepth: 1 :hidden: - overview + service_overview intra-cluster_access_clusterip nodeport loadbalancer diff --git a/umn/source/networking/services/intra-cluster_access_clusterip.rst b/umn/source/networking/services/intra-cluster_access_clusterip.rst index 9851f28..38ff599 100644 --- a/umn/source/networking/services/intra-cluster_access_clusterip.rst +++ b/umn/source/networking/services/intra-cluster_access_clusterip.rst @@ -24,7 +24,7 @@ The cluster-internal domain name format is **.\ *`, you can choose either TCP or HTTP. When UDP is selected during the :ref:`port settings `, only UDP is supported.. By default, the service port (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a service port named **cce-healthz** will be added for the Service. + - **Health Check**: This function is disabled by default. The health check is for the load balancer. When TCP is selected during the :ref:`port settings `, you can choose either TCP or HTTP. When UDP is selected during the :ref:`port settings `, only UDP is supported.. By default, the service port (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a service port named **cce-healthz** will be added for the Service. - .. _cce_10_0014__li388800117144: diff --git a/umn/source/networking/services/service_annotations.rst b/umn/source/networking/services/service_annotations.rst index 0c46d09..b194ce2 100644 --- a/umn/source/networking/services/service_annotations.rst +++ b/umn/source/networking/services/service_annotations.rst @@ -16,7 +16,7 @@ The annotations of a Service are the parameters that need to be specified for co +===========================================+====================================================+=========================================================================================================================================================================================================+==============================+================================================+ | kubernetes.io/elb.class | String | Select a proper load balancer type. | performance | v1.9 or later | | | | | | | - | | | Value: | | | + | | | The value can be: | | | | | | | | | | | | - **union**: shared load balancer | | | | | | - **performance**: dedicated load balancer, which can be used only in clusters of v1.17 and later. | | | diff --git a/umn/source/networking/services/overview.rst b/umn/source/networking/services/service_overview.rst similarity index 98% rename from umn/source/networking/services/overview.rst rename to umn/source/networking/services/service_overview.rst index b738a5f..8d87e99 100644 --- a/umn/source/networking/services/overview.rst +++ b/umn/source/networking/services/service_overview.rst @@ -2,8 +2,8 @@ .. _cce_10_0249: -Overview -======== +Service Overview +================ Direct Access to a Pod ---------------------- diff --git a/umn/source/node_pools/creating_a_node_pool.rst b/umn/source/node_pools/creating_a_node_pool.rst index b731d54..73ede10 100644 --- a/umn/source/node_pools/creating_a_node_pool.rst +++ b/umn/source/node_pools/creating_a_node_pool.rst @@ -54,7 +54,7 @@ Procedure | | c. If multiple node pools have the same priority or no priority is configured for them, the system selects the node pool that will consume the least resources based on the configured VM specification. | | | d. If the VM specifications of multiple node pools are the same but the node pools are deployed in different AZs, the system randomly selects a node pool to trigger scaling. | | | | - | | - **Scale-In Cooling Interval**: Set this parameter in the unit of minute or hour. This field indicates the period during which the nodes added in the current node pool cannot be scaled in. | + | | - **Cooldown Period**: Requied. The unit is minute. This field indicates the period during which the nodes added in the current node pool cannot be scaled in. | | | | | | Scale-in cooling intervals can be configured in the node pool settings and the :ref:`autoscaler add-on ` settings. | | | | @@ -88,17 +88,13 @@ Procedure | | | | | You are advised to select **Random** to deploy your node in a random AZ based on the selected node flavor. | | | | - | | .. note:: | - | | | - | | In a CCE Turbo cluster, an AZ is randomly selected from available AZs, and all nodes will be created in the selected AZ. | - | | | | | An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. To enhance workload availability, create nodes in different AZs. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Node Type | For a CCE cluster, **ECS** and **BMS** are supported. | | | | | | CCE Turbo clusters support ECSs of the VM and physical types. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Engine | CCE clusters support Docker. | + | Container Engine | CCE clusters support Docker. Starting from CCE 1.23, containerd is supported. | | | | | | For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -110,11 +106,11 @@ Procedure | | | | | **Private image**: You can use private images. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Login Mode | **Key Pair** | + | Login Mode | - **Key Pair** | | | | - | | Select the key pair used to log in to the node. You can select a shared key. | + | | Select the key pair used to log in to the node. You can select a shared key. | | | | - | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ **Storage Settings** @@ -133,7 +129,7 @@ Procedure | | - **Encryption** is not selected by default. | | | - After you select **Encryption**, you can select an existing key in the displayed dialog box. If no key is available, click **View Key List** to create a key. After the key is created, click the refresh icon. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Data Disk | Data disk used by the container runtime and kubelet on the node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. | + | Data Disk | Data disk used by the container runtime and kubelet on the node. | | | | | | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | | | | @@ -165,11 +161,19 @@ Procedure .. table:: **Table 4** Configuration parameters - +-------------+-------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +=============+=============================================================================================================+ - | Node Subnet | The node subnet selected during cluster creation is used by default. You can choose another subnet instead. | - +-------------+-------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+======================================================================================================================================================================================+ + | Node Subnet | The node subnet selected during cluster creation is used by default. You can choose another subnet instead. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node IP Address | Random allocation is supported. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Associate Security Group | Security group used by the nodes created in the node pool. A maximum of 5 security groups can be selected. | + | | | + | | When a cluster is created, a node security group named **{Cluster name}-cce-node-{Random ID}** is created and used by default. | + | | | + | | Traffic needs to pass through certain ports in the node security group to ensure node communications. Ensure that you have enabled these ports if you select another security group. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ **Advanced Settings** @@ -180,7 +184,7 @@ Procedure +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+================================================================================================================================================================================================================================================================+ - | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 10 labels can be added. | + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | | | | | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/node_pools/managing_a_node_pool.rst b/umn/source/node_pools/managing_a_node_pool.rst index c298a59..ccac6d4 100644 --- a/umn/source/node_pools/managing_a_node_pool.rst +++ b/umn/source/node_pools/managing_a_node_pool.rst @@ -24,38 +24,60 @@ This function is supported only in clusters of **v1.15 and later**. It is not di .. table:: **Table 1** kubelet - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | Parameter | Description | Default Value | Remarks | - +========================+====================================================================================================================================================================================================================================================================================================================================================================================================================+=================================================================================================================================+=======================================================================================+ - | cpu-manager-policy | Specifies the CPU core binding configuration. For details, see :ref:`CPU Core Binding `. | none | The values can be modified during the node pool lifecycle. | - | | | | | - | | - **none**: disables pods from exclusively occupying CPUs. Select this value if you want a large pool of shareable CPU cores. | | | - | | - **static**: enables pods to exclusively occupy CPUs. Select this value if your workload is sensitive to latency in CPU cache and scheduling. | | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | 100 | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | kube-api-burst | Burst to use while talking with kube-apiserver. | 100 | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | max-pods | Maximum number of pods managed by kubelet. | 40 | | - | | | | | - | | | 20 | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | pod-pids-limit | PID limit in Kubernetes | -1 | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | with-local-dns | Whether to use the local IP address as the ClusterDNS of the node. | false | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | event-qps | QPS limit for event creation | 5 | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | allowed-unsafe-sysctls | Insecure system configuration allowed. | [] | | - | | | | | - | | Starting from **v1.17.17**, CCE enables pod security policies for kube-apiserver. You need to add corresponding configurations to **allowedUnsafeSysctls** of a pod security policy to make the policy take effect. (This configuration is not required for clusters earlier than v1.17.17.) For details, see :ref:`Example of Enabling Unsafe Sysctls in Pod Security Policy `. | | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | kube-reserved-mem | Reserved node memory. | Depends on node specifications. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. | The sum of kube-reserved-mem and system-reserved-mem is less than half of the memory. | - | | | | | - | system-reserved-mem | | | | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ - | resolv-conf | DNS resolution configuration file specified by the container | The default value is null. | ``-`` | - +------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | Default Value | Remarks | + +============================+====================================================================================================================================================================================================================================================================================================================================================================================================================+=================================================================================================================================+=======================================================================================================================================================================================================================================================================+ + | cpu-manager-policy | Specifies the CPU core binding configuration. For details, see :ref:`CPU Core Binding `. | none | The values can be modified during the node pool lifecycle. | + | | | | | + | | - **none**: disables pods from exclusively occupying CPUs. Select this value if you want a large pool of shareable CPU cores. | | | + | | - **static**: enables pods to exclusively occupy CPUs. Select this value if your workload is sensitive to latency in CPU cache and scheduling. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | 100 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kube-api-burst | Burst to use while talking with kube-apiserver. | 100 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | max-pods | Maximum number of pods managed by kubelet. | 40 | | + | | | | | + | | | 20 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | pod-pids-limit | PID limit in Kubernetes | -1 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | with-local-dns | Whether to use the local IP address as the ClusterDNS of the node. | false | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | event-qps | QPS limit for event creation | 5 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | allowed-unsafe-sysctls | Insecure system configuration allowed. | [] | | + | | | | | + | | Starting from **v1.17.17**, CCE enables pod security policies for kube-apiserver. You need to add corresponding configurations to **allowedUnsafeSysctls** of a pod security policy to make the policy take effect. (This configuration is not required for clusters earlier than v1.17.17.) For details, see :ref:`Example of Enabling Unsafe Sysctls in Pod Security Policy `. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | over-subscription-resource | Whether to enable node oversubscription. | true | ``-`` | + | | | | | + | | If this parameter is set to **true**, the node oversubscription feature is enabled. For details, see :ref:`Hybrid Deployment of Online and Offline Jobs `. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | colocation | Whether to enable node hybrid deployment. | true | ``-`` | + | | | | | + | | If this parameter is set to **true**, the node hybrid deployment feature is enabled. For details, see :ref:`Hybrid Deployment of Online and Offline Jobs `. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kube-reserved-mem | Reserved node memory. | Depends on node specifications. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. | The sum of kube-reserved-mem and system-reserved-mem is less than half of the memory. | + | | | | | + | system-reserved-mem | | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | topology-manager-policy | Set the topology management policy. | none | The values can be modified during the node pool lifecycle. | + | | | | | + | | Valid values are as follows: | | .. important:: | + | | | | | + | | - **restricted**: kubelet accepts only pods that achieve optimal NUMA alignment on the requested resources. | | NOTICE: | + | | - **best-effort**: kubelet preferentially selects pods that implement NUMA alignment on CPU and device resources. | | Exercise caution when modifying topology-manager-policy and topology-manager-scope will restart kubelet and recalculate the resource allocation of pods based on the modified policy. As a result, running pods may restart or even fail to receive any resources. | + | | - **none** (default): The topology management policy is disabled. | | | + | | - **single-numa-node**: kubelet allows only pods that are aligned to the same NUMA node in terms of CPU and device resources. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | topology-manager-scope | Set the resource alignment granularity of the topology management policy. Valid values are as follows: | container | | + | | | | | + | | - **container** (default) | | | + | | - **pod** | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | resolv-conf | DNS resolution configuration file specified by the container | The default value is null. | ``-`` | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. table:: **Table 2** kube-proxy @@ -98,25 +120,25 @@ This function is supported only in clusters of **v1.15 and later**. It is not di .. table:: **Table 5** Docker (available only for node pools that use Docker) - +-----------------------+------------------------------------------------------------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ - | Parameter | Description | Default Value | Remarks | - +=======================+==================================================================================================================+=================+========================================================================================================+ - | native-umask | \`--exec-opt native.umask | normal | Cannot be changed. | - +-----------------------+------------------------------------------------------------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ - | docker-base-size | \`--storage-opts dm.basesize | 0 | Cannot be changed. | - +-----------------------+------------------------------------------------------------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ - | insecure-registry | Address of an insecure image registry | false | Cannot be changed. | - +-----------------------+------------------------------------------------------------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ - | limitcore | The maximum number of cores. The total number of cores in a node pool cannot exceed the value of this parameter. | 5368709120 | ``-`` | - +-----------------------+------------------------------------------------------------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ - | default-ulimit-nofile | Limit on the number of handles in a container | {soft}:{hard} | The value cannot exceed the value of the kernel parameter **nr_open** and cannot be a negative number. | - | | | | | - | | | | You can run the following command to obtain the kernel parameter **nr_open**: | - | | | | | - | | | | .. code-block:: | - | | | | | - | | | | sysctl -a | grep nr_open | - +-----------------------+------------------------------------------------------------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | Parameter | Description | Default Value | Remarks | + +=======================+===============================================================+=================+========================================================================================================+ + | native-umask | \`--exec-opt native.umask | normal | Cannot be changed. | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | docker-base-size | \`--storage-opts dm.basesize | 0 | Cannot be changed. | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | insecure-registry | Address of an insecure image registry | false | Cannot be changed. | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | limitcore | Maximum size of a core file in a container. The unit is byte. | 5368709120 | ``-`` | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | default-ulimit-nofile | Limit on the number of handles in a container | {soft}:{hard} | The value cannot exceed the value of the kernel parameter **nr_open** and cannot be a negative number. | + | | | | | + | | | | You can run the following command to obtain the kernel parameter **nr_open**: | + | | | | | + | | | | .. code-block:: | + | | | | | + | | | | sysctl -a | grep nr_open | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ #. Click **OK**. @@ -129,42 +151,69 @@ Editing a Node Pool #. Click **Edit** next to the name of the node pool you will edit. In the **Edit Node Pool** page, edit the following parameters: - .. table:: **Table 6** Node pool parameters + **Basic Settings** + + .. table:: **Table 6** Basic settings +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+=================================================================================================================================================================================================================================================================================================================================================================================================================================================+ | Node Pool Name | Name of the node pool. | +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Nodes | Modify the number of nodes based on service requirements. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Auto Scaling | By default, autoscaler is disabled. | + | Auto Scaling | By default, this parameter is disabled. | | | | | | After you enable autoscaler by clicking |image1|, nodes in the node pool are automatically created or deleted based on service requirements. | | | | | | - **Maximum Nodes** and **Minimum Nodes**: You can set the maximum and minimum number of nodes to ensure that the number of nodes to be scaled is within a proper range. | | | - **Priority**: A larger value indicates a higher priority. For example, if this parameter is set to **1** and **4** respectively for node pools A and B, B has a higher priority than A, and auto scaling is first triggered for B. If the priorities of multiple node pools are set to the same value, for example, **2**, the node pools are not prioritized and the system performs scaling based on the minimum resource waste principle. | + | | - **Cooldown Period**: Required. The unit is minute. This parameter indicates the interval between the previous scale-out action and the next scale-in action. | | | | | | If the **Autoscaler** field is set to on, install the :ref:`autoscaler add-on ` to use the autoscaler feature. | +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Kubernetes Label | Click **Add** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 10 labels can be added. | - | | | - | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Resource Tag | You can add resource tags to classify resources. | - | | | - | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | - | | | - | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | - | | | - | | - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | - | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | - | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | - | | | - | | For details, see :ref:`Managing Node Taints `. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Advanced Settings** + + .. table:: **Table 7** Advanced settings + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+================================================================================================================================================================================================================================================================+ + | K8s label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | + | | | + | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | + | | | + | | .. note:: | + | | | + | | After a **K8s label** is modified, the inventory nodes in the node pool are updated synchronously. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resource Tag | You can add resource tags to classify resources. | + | | | + | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | + | | | + | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | + | | | + | | .. note:: | + | | | + | | After a **resource tag** is modified, the modification automatically takes effect when a node is added. For existing nodes, you need to manually reset the nodes for the modification to take effect. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | | | + | | - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | + | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | + | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | + | | | + | | For details, see :ref:`Managing Node Taints `. | + | | | + | | .. note:: | + | | | + | | After a **taint** is modified, the inventory nodes in the node pool are updated synchronously. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Edit Key pair | Only node pools that use key pairs for login support key pair editing. You can select another key pair. | + | | | + | | .. note:: | + | | | + | | The edited key pair automatically takes effect when a node is added. For existing nodes, you need to manually reset the nodes for the key pair to take effect. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Click **OK**. @@ -215,4 +264,4 @@ Nodes in a node pool can be migrated. Currently, nodes in a node pool can be mig The migration has no impacts on the original resource tags, Kubernetes labels, and taints of the node. -.. |image1| image:: /_static/images/en-us_image_0000001199021280.png +.. |image1| image:: /_static/images/en-us_image_0000001528627005.png diff --git a/umn/source/node_pools/node_pool_overview.rst b/umn/source/node_pools/node_pool_overview.rst index 2b864f7..d75ae98 100644 --- a/umn/source/node_pools/node_pool_overview.rst +++ b/umn/source/node_pools/node_pool_overview.rst @@ -30,7 +30,7 @@ Generally, all nodes in a node pool have the following same attributes: - Node runtime. - Startup parameters of Kubernetes components on a node - User-defined startup script of a node -- **K8S Labels** and **Taints** +- **K8s Labels** and **Taints** CCE provides the following extended attributes for node pools: @@ -66,28 +66,28 @@ The following table describes multiple scenarios of large-scale cluster manageme Functions and Precautions ------------------------- -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Function | Description | Notes | -+=======================================+========================================================================================================================================================+================================================================================================================================================================================================================+ -| Creating a node pool | Add a node pool. | It is recommended that a cluster contains no more than 100 node pools. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Deleting a node pool | Deleting a node pool will delete nodes in the pool. Pods on these nodes will be automatically migrated to available nodes in other node pools. | If pods in the node pool have a specific node selector and none of the other nodes in the cluster satisfies the node selector, the pods will become unschedulable. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Enabling auto scaling for a node pool | After auto scaling is enabled, nodes will be automatically created or deleted in the node pool based on the cluster loads. | You are advised not to store important data on nodes in a node pool because after auto scaling, data cannot be restored as nodes may be deleted. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Enabling auto scaling for a node pool | After auto scaling is disabled, the number of nodes in a node pool will not automatically change with the cluster loads. | / | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Adjusting the size of a node pool | The number of nodes in a node pool can be directly adjusted. If the number of nodes is reduced, nodes are randomly removed from the current node pool. | After auto scaling is enabled, you are not advised to manually adjust the node pool size. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Changing node pool configurations | You can modify the node pool name, node quantity, Kubernetes labels (and their quantity), and taints. | The modified Kubernetes labels and taints (as well as their quantity) will apply to all nodes in the node pool, which may cause pod re-scheduling. Therefore, exercise caution when performing this operation. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Removing a node from a node pool | Nodes in a node pool can be migrated to the default node pool of the same cluster. | Nodes in the default node pool cannot be migrated to other node pools, and nodes in a user-created node pool cannot be migrated to other user-created node pools. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Cloning a node pool | You can copy the configuration of an existing node pool to create a new node pool. | / | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Setting Kubernetes parameters | You can configure core components with fine granularity. | - This function is supported only in clusters of v1.15 and later. It is not displayed for versions earlier than v1.15. | -| | | - The default node pool DefaultPool does not support this type of configuration. | -+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Function | Description | Notes | ++=======================================+========================================================================================================================================================+========================================================================================================================================================================================================================+ +| Creating a node pool | Add a node pool. | It is recommended that a cluster contains no more than 100 node pools. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Deleting a node pool | Deleting a node pool will delete nodes in the pool. Pods on these nodes will be automatically migrated to available nodes in other node pools. | If pods in the node pool have a specific node selector and none of the other nodes in the cluster satisfies the node selector, the pods will become unschedulable. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Enabling auto scaling for a node pool | After auto scaling is enabled, nodes will be automatically created or deleted in the node pool based on the cluster loads. | You are advised not to store important data on nodes in a node pool because after auto scaling, data cannot be restored as nodes may be deleted. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Enabling auto scaling for a node pool | After auto scaling is disabled, the number of nodes in a node pool will not automatically change with the cluster loads. | / | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Adjusting the size of a node pool | The number of nodes in a node pool can be directly adjusted. If the number of nodes is reduced, nodes are randomly removed from the current node pool. | After auto scaling is enabled, you are not advised to manually adjust the node pool size. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Changing node pool configurations | You can modify the node pool name, node quantity, Kubernetes labels (and their quantity), and taints. | The deleted or added Kubernetes labels and taints (as well as their quantity) will apply to all nodes in the node pool, which may cause pod re-scheduling. Therefore, exercise caution when performing this operation. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Removing a node from a node pool | Nodes in a node pool can be migrated to the default node pool of the same cluster. | Nodes in the default node pool cannot be migrated to other node pools, and nodes in a user-created node pool cannot be migrated to other user-created node pools. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Cloning a node pool | You can copy the configuration of an existing node pool to create a new node pool. | / | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Setting Kubernetes parameters | You can configure core components with fine granularity. | - This function is supported only in clusters of v1.15 and later. It is not displayed for versions earlier than v1.15. | +| | | - The default node pool DefaultPool does not support this type of configuration. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Deploying a Workload in a Specified Node Pool --------------------------------------------- diff --git a/umn/source/nodes/adding_nodes_for_management.rst b/umn/source/nodes/adding_nodes_for_management.rst new file mode 100644 index 0000000..23e0311 --- /dev/null +++ b/umn/source/nodes/adding_nodes_for_management.rst @@ -0,0 +1,135 @@ +:original_name: cce_10_0198.html + +.. _cce_10_0198: + +Adding Nodes for Management +=========================== + +Scenario +-------- + +In CCE, you can :ref:`Creating a Node ` or add existing nodes (ECSs) into your cluster. + +.. important:: + + - While an ECS is being accepted into a cluster, the operating system of the ECS will be reset to the standard OS image provided by CCE to ensure node stability. The CCE console prompts you to select the operating system and the login mode during the reset. + - The system disk and data disk of an ECS will be formatted while the ECS is being accepted into a cluster. Ensure that information in the disks has been backed up. + - While an ECS is being accepted into a cluster, do not perform any operation on the ECS through the ECS console. + +Notes and Constraints +--------------------- + +- The cluster version must be 1.15 or later. +- If the password or key has been set when a VM node is created, the VM node can be accepted into a cluster 10 minutes after it is available. During the management, the original password or key will become invalid. You need to reset the password or key. +- Nodes in a CCE Turbo cluster must support sub-ENIs or be bound to at least 16 ENIs. For details about the node specifications, see the nodes that can be selected on the console when you create a node. + +Prerequisites +------------- + +A cloud server that meets the following conditions can be accepted: + +- The node to be accepted must be in the **Running** state and not used by other clusters. In addition, the node to be accepted does not carry the CCE-Dynamic-Provisioning-Node tag. +- The node to be accepted and the cluster must be in the same VPC. (If the cluster version is earlier than v1.13.10, the node to be accepted and the CCE cluster must be in the same subnet.) +- At least one data disk is attached to the node to be accepted. The data disk capacity is greater than or equal to 100 GB. +- The node to be accepted has 2-core or higher CPU, 4 GB or larger memory, and only one NIC. +- Only cloud servers with the same specifications, AZ, and data disk configuration can be added in batches. + +Procedure +--------- + +#. Log in to the CCE console and go to the cluster where the node to be managed resides. + +#. In the navigation pane, choose **Nodes**. On the displayed page, click **Accept Node** in the upper right corner. + +#. Specify node parameters. + + **Compute Settings** + + .. table:: **Table 1** Configuration parameters + + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=======================================================================================================================================================================================+ + | Specifications | Click **Select Cloud Server** and select the servers to be accepted. | + | | | + | | You can select multiple cloud servers for batch management. However, only the cloud servers with the same specifications, AZ, and data disk configuration can be added in batches. | + | | | + | | If a cloud server contains multiple data disks, select one of them for the container runtime and kubelet. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Container Engine | CCE clusters support Docker. Starting from CCE 1.23, containerd is supported. | + | | | + | | For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | OS | **Public image**: Select an OS for the node. | + | | | + | | **Private image**: You can use private images. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Login Mode | - **Key Pair** | + | | | + | | Select the key pair used to log in to the node. You can select a shared key. | + | | | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Storage Settings** + + Configure storage resources on a node for the containers running on it. + + .. table:: **Table 2** Configuration parameters + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+====================================================================================================================================================================================================================================================================================================+ + | System Disk | Directly use the system disk of the cloud server. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Data Disk | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | + | | | + | | Click **Expand** and select **Allocate Disk Space** to define the disk space occupied by the container runtime to store the working directories, container image data, and image metadata. For details about how to allocate data disk space, see :ref:`Data Disk Space Allocation `. | + | | | + | | For other data disks, a raw disk is created without any processing by default. You can also click **Expand** and select **Mount Disk** to mount the data disk to a specified directory. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Advanced Settings** + + .. table:: **Table 3** Advanced configuration parameters + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+================================================================================================================================================================================================================================================================+ + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | + | | | + | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resource Tag | You can add resource tags to classify resources. | + | | | + | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | + | | | + | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | | | + | | - **Key**: A key must contain 1 to 63 characters, starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | + | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | + | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | + | | | + | | .. important:: | + | | | + | | NOTICE: | + | | | + | | - If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-up may fail or pods cannot be scheduled onto the added nodes. | + | | - After a node pool is created, you can click **Edit** to modify its configuration. The modification will be synchronized to all nodes in the node pool. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Max. Pods | Maximum number of pods that can run on the node, including the default system pods. | + | | | + | | This limit prevents the node from being overloaded with pods. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pre-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Post-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed after Kubernetes software is installed and will not affect the installation. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Click **Next: Confirm**. Click **Submit**. diff --git a/umn/source/nodes/creating_a_node.rst b/umn/source/nodes/creating_a_node.rst index ffaa032..8c3f15f 100644 --- a/umn/source/nodes/creating_a_node.rst +++ b/umn/source/nodes/creating_a_node.rst @@ -47,9 +47,10 @@ After a cluster is created, you can create nodes for the cluster. | | | | | CCE Turbo clusters support Elastic Cloud Servers (ECSs) and bare metal servers (BMSs). | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Engine | CCE clusters support Docker. | + | Container Engine | CCE clusters support Docker and containerd in some scenarios. | | | | - | | For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | + | | - VPC network clusters of v1.23 and later versions support containerd. Container tunnel network clusters of v1.23.2-r0 and later versions support containerd. | + | | - For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Specifications | Select the node specifications based on service requirements. The available node specifications vary depending on AZs. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -65,11 +66,11 @@ After a cluster is created, you can create nodes for the cluster. | | | | | A node name must start with a lowercase letter and cannot end with a hyphen (-). Only digits, lowercase letters, and hyphens (-) are allowed. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Login Mode | **Key Pair** | + | Login Mode | - **Key Pair** | | | | - | | Select the key pair used to log in to the node. You can select a shared key. | + | | Select the key pair used to log in to the node. You can select a shared key. | | | | - | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ **Storage Settings** @@ -88,7 +89,7 @@ After a cluster is created, you can create nodes for the cluster. | | - **Encryption** is not selected by default. | | | - After you select **Encryption**, you can select an existing key in the displayed dialog box. If no key is available, click **View Key List** to create a key. After the key is created, click the refresh icon. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Data Disk | Data disk used by the container runtime and kubelet on the node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. | + | Data Disk | Data disk used by the container runtime and kubelet on the node. | | | | | | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | | | | @@ -137,7 +138,7 @@ After a cluster is created, you can create nodes for the cluster. +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+================================================================================================================================================================================================================================================================+ - | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 10 labels can be added. | + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | | | | | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/nodes/index.rst b/umn/source/nodes/index.rst index a15f692..28f26de 100644 --- a/umn/source/nodes/index.rst +++ b/umn/source/nodes/index.rst @@ -7,6 +7,7 @@ Nodes - :ref:`Node Overview ` - :ref:`Creating a Node ` +- :ref:`Adding Nodes for Management ` - :ref:`Removing a Node ` - :ref:`Resetting a Node ` - :ref:`Logging In to a Node ` @@ -23,6 +24,7 @@ Nodes node_overview/index creating_a_node + adding_nodes_for_management removing_a_node resetting_a_node logging_in_to_a_node diff --git a/umn/source/nodes/managing_node_labels.rst b/umn/source/nodes/managing_node_labels.rst index 6846610..22fbd55 100644 --- a/umn/source/nodes/managing_node_labels.rst +++ b/umn/source/nodes/managing_node_labels.rst @@ -60,8 +60,6 @@ After a node is created, some fixed labels exist and cannot be deleted. For deta +-----------------------------------------------------+-------------------------------------------------------------+ | node.kubernetes.io/container-engine | Container engine used by the node. | +-----------------------------------------------------+-------------------------------------------------------------+ - | NPU node labels. | | - +-----------------------------------------------------+-------------------------------------------------------------+ | accelerator | GPU node labels. | +-----------------------------------------------------+-------------------------------------------------------------+ | cce.cloud.com/cce-nodepool | The dedicated label of a node in a node pool. | @@ -78,6 +76,6 @@ Adding or Deleting a Node Label Enter the key and value of the label to be added or deleted, and click **OK**. - As shown in the figure, the key is **deploy_qa** and the value is **true**, indicating that the node is used to deploy the QA (test) environment. + For example, the key is **deploy_qa** and the value is **true**, indicating that the node is used to deploy the QA (test) environment. #. After the label is added, check the added label in node data. diff --git a/umn/source/nodes/node_overview/container_engine.rst b/umn/source/nodes/node_overview/container_engine.rst index c5a0e6e..340d1c0 100644 --- a/umn/source/nodes/node_overview/container_engine.rst +++ b/umn/source/nodes/node_overview/container_engine.rst @@ -17,24 +17,28 @@ Mapping between Node OSs and Container Engines .. table:: **Table 1** Node OSs and container engines in CCE clusters - +-------------+----------------+------------------+-----------------------------------------------------+-------------------+ - | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | - +=============+================+==================+=====================================================+===================+ - | CentOS 7.x | 3.x | Docker | Clusters of v1.19.16 and earlier use Device Mapper. | runC | - | | | | | | - | | | | Clusters of v1.19.16 and later use OverlayFS. | | - +-------------+----------------+------------------+-----------------------------------------------------+-------------------+ - | EulerOS 2.5 | 3.x | Docker | Device Mapper | runC | - +-------------+----------------+------------------+-----------------------------------------------------+-------------------+ - | EulerOS 2.9 | 4.x | Docker | OverlayFS | runC | - +-------------+----------------+------------------+-----------------------------------------------------+-------------------+ + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | + +=============+================+=================================================+=====================================================+===================+ + | CentOS 7.x | 3.x | Docker | Clusters of v1.19.16 and earlier use Device Mapper. | runC | + | | | | | | + | | | Clusters of v1.23 and later support containerd. | Clusters of v1.19.16 and later use OverlayFS. | | + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + | EulerOS 2.5 | 3.x | Docker | Device Mapper | runC | + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + | EulerOS 2.9 | 4.x | Docker | OverlayFS | runC | + | | | | | | + | | | Clusters of v1.23 and later support containerd. | | | + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ .. table:: **Table 2** Node OSs and container engines in CCE Turbo clusters +-----------------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ | Node Type | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | +=========================================+=============+================+==================+==========================+===================+ - | VM | CentOS 7.x | 3.x | Docker | OverlayFS | runC | + | Elastic Cloud Server (VM) | CentOS 7.x | 3.x | Docker | OverlayFS | runC | + | | | | | | | + | | EulerOS 2.9 | | | | | +-----------------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ | Elastic Cloud Server (physical machine) | EulerOS 2.9 | 4.x | containerd | Device Mapper | Kata | +-----------------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ diff --git a/umn/source/nodes/node_overview/data_disk_space_allocation.rst b/umn/source/nodes/node_overview/data_disk_space_allocation.rst index 8c37edb..b094458 100644 --- a/umn/source/nodes/node_overview/data_disk_space_allocation.rst +++ b/umn/source/nodes/node_overview/data_disk_space_allocation.rst @@ -56,7 +56,7 @@ You can log in to the node and run the **docker info** command to view the stora Using rootfs for container storage in CCE -- CCE cluster: EulerOS 2.5 nodes use Device Mapper, and EulerOS 2.9 nodes use OverlayFS. CentOS 7.6 nodes in clusters earlier than v1.19.16 use Device Mapper, and use OverlayFS in clusters of v1.19.16 and later. +- CCE cluster: EulerOS 2.5 nodes use Device Mapper and EulerOS 2.9 nodes use OverlayFS. CentOS 7.x nodes in clusters earlier than v1.19.16 use Device Mapper, and use OverlayFS in clusters of v1.19.16 and later. - CCE Turbo cluster: BMSs use Device Mapper. ECSs use OverlayFS. .. _cce_10_0341__section12119191161518: diff --git a/umn/source/nodes/node_overview/kata_containers_and_common_containers.rst b/umn/source/nodes/node_overview/kata_containers_and_common_containers.rst index 8efdd06..2101440 100644 --- a/umn/source/nodes/node_overview/kata_containers_and_common_containers.rst +++ b/umn/source/nodes/node_overview/kata_containers_and_common_containers.rst @@ -36,7 +36,7 @@ You can run common or Kata containers on a single node in a CCE Turbo cluster. T | | | | | | | It is recommended that the ratio of CPU (unit: core) to memory (unit: GiB) be in the range of 1:1 to 1:8. For example, if CPU is 0.5 cores, the memory should range form 512 MiB to 4 GiB. | | | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ -| Container engine CLI | crictl | docker | crictl | +| Container engine CLI | crictl | Docker | crictl | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ | Pod computing resources | The request and limit values must be the same for both CPU and memory. | The request and limit values can be different for both CPU and memory. | The request and limit values can be different for both CPU and memory. | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ diff --git a/umn/source/nodes/node_overview/precautions_for_using_a_node.rst b/umn/source/nodes/node_overview/precautions_for_using_a_node.rst index 2cbc5a9..63fa3f8 100644 --- a/umn/source/nodes/node_overview/precautions_for_using_a_node.rst +++ b/umn/source/nodes/node_overview/precautions_for_using_a_node.rst @@ -12,7 +12,7 @@ A container cluster consists of a set of worker machines, called nodes, that run .. note:: - A Kubernetes cluster consists of master nodes and node nodes. The nodes described in this section refer to **worker nodes**, the computing nodes of a cluster that run containerized applications. + A Kubernetes cluster consists of master nodes and worker nodes. The nodes described in this section refer to **worker nodes**, the computing nodes of a cluster that run containerized applications. CCE uses high-performance Elastic Cloud Servers (ECSs) as nodes to build highly available Kubernetes clusters. diff --git a/umn/source/nodes/resetting_a_node.rst b/umn/source/nodes/resetting_a_node.rst index 3534a0a..27fac2c 100644 --- a/umn/source/nodes/resetting_a_node.rst +++ b/umn/source/nodes/resetting_a_node.rst @@ -63,11 +63,11 @@ The new console allows you to reset nodes in batches. You can also use private i | | | | | **Private image**: You can use private images. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Login Mode | **Key Pair** | + | Login Mode | - **Key Pair** | | | | - | | Select the key pair used to log in to the node. You can select a shared key. | + | | Select the key pair used to log in to the node. You can select a shared key. | | | | - | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ **Storage Settings** @@ -95,7 +95,7 @@ The new console allows you to reset nodes in batches. You can also use private i +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+================================================================================================================================================================================================================================================================+ - | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 10 labels can be added. | + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | | | | | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/nodes/synchronizing_data_with_cloud_servers.rst b/umn/source/nodes/synchronizing_data_with_cloud_servers.rst index 0af6d36..60381f0 100644 --- a/umn/source/nodes/synchronizing_data_with_cloud_servers.rst +++ b/umn/source/nodes/synchronizing_data_with_cloud_servers.rst @@ -15,14 +15,18 @@ Some information about CCE nodes is maintained independently from the ECS consol Notes and Constraints --------------------- -If an ECS name is specified as the Kubernetes node name, the change of the ECS name cannot be synchronized to the CCE console. +- Data, including the VM status, ECS names, number of CPUs, size of memory, ECS specifications, and public IP addresses, can be synchronized. + + If an ECS name is specified as the Kubernetes node name, the change of the ECS name cannot be synchronized to the CCE console. + +- Data, such as the OS and image ID, cannot be synchronized. (Such parameters cannot be modified on the ECS console.) Procedure --------- #. Log in to the CCE console. -#. Click the cluster name to access the cluster details page. Choose **Nodes** in the navigation pane. +#. Click the cluster name to access the cluster console. Choose **Nodes** in the navigation pane. #. Choose **More** > **Sync Server Data** next to the node. diff --git a/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst b/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst index 158a1ff..684fa2c 100644 --- a/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst +++ b/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst @@ -54,11 +54,11 @@ You can create a read-only user group named **read_only** on the IAM console and - For the O&M engineer William, assign the read-only permission on clusters to him in this step. - The O&M team leader James already has the management permissions on all clusters. You can add him to the **read_only** user group to assign the read-only permission on clusters to him. -As shown in the following figure, users James, Robert, William, Linda, and Peter are added to the **read_only** user group. +Users James, Robert, William, Linda, and Peter are added to the **read_only** user group. Assign the read-only permission on clusters to the user group **read_only**. -Return to the CCE console, and add the read-only permission on namespaces to the user group **read_only** to which the five users belong. Choose **Permissions** > **Namespace-Level Permissions** on the CCE console, and assign the read-only policy to the user group **read_only** for each cluster. +Return to the CCE console, and add the read-only permission on namespaces to the user group **read_only** to which the five users belong. Choose **Permissions** on the CCE console, and assign the read-only policy to the user group **read_only** for each cluster. After the setting is complete, James has the cluster management permissions for all projects and the read-only permissions on all namespaces, and the Robert, William, Linda, and Peter have the read-only permission on all clusters and namespaces. diff --git a/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst b/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst index f10bcb2..8c42416 100644 --- a/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst +++ b/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst @@ -42,17 +42,17 @@ Users with different cluster permissions (assigned using IAM) have different nam .. table:: **Table 1** Differences in namespace permissions - +------------------------------------------------+-----------------------------------------+ - | User | Clusters of v1.13 and Later | - +================================================+=========================================+ - | User with the Tenant Administrator permissions | All namespace permissions | - +------------------------------------------------+-----------------------------------------+ - | IAM user with the CCE Administrator role | All namespace permissions | - +------------------------------------------------+-----------------------------------------+ - | IAM user with the CCE Viewer role | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+-----------------------------------------+ - | IAM user with the Tenant Guest role | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+-----------------------------------------+ + +-------------------------------------------------------------+-----------------------------------------+ + | User | Clusters of v1.13 and Later | + +=============================================================+=========================================+ + | User with the Tenant Administrator permissions | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE Administrator role | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE FullAccess or CCE ReadOnlyAccess role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the Tenant Guest role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ Precautions ----------- diff --git a/umn/source/permissions_management/permissions_overview.rst b/umn/source/permissions_management/permissions_overview.rst index 8b46683..190e97f 100644 --- a/umn/source/permissions_management/permissions_overview.rst +++ b/umn/source/permissions_management/permissions_overview.rst @@ -47,17 +47,17 @@ Users with different cluster permissions (assigned using IAM) have different nam .. table:: **Table 1** Differences in namespace permissions - +------------------------------------------------+-----------------------------------------+ - | User | Clusters of v1.13 and Later | - +================================================+=========================================+ - | User with the Tenant Administrator permissions | All namespace permissions | - +------------------------------------------------+-----------------------------------------+ - | IAM user with the CCE Administrator role | All namespace permissions | - +------------------------------------------------+-----------------------------------------+ - | IAM user with the CCE Viewer role | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+-----------------------------------------+ - | IAM user with the Tenant Guest role | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+-----------------------------------------+ + +-------------------------------------------------------------+-----------------------------------------+ + | User | Clusters of v1.13 and Later | + +=============================================================+=========================================+ + | User with the Tenant Administrator permissions | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE Administrator role | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE FullAccess or CCE ReadOnlyAccess role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the Tenant Guest role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ kubectl Permissions ------------------- diff --git a/umn/source/storage/deployment_examples/creating_a_pod_mounted_with_an_evs_volume.rst b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_evs_volume.rst similarity index 99% rename from umn/source/storage/deployment_examples/creating_a_pod_mounted_with_an_evs_volume.rst rename to umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_evs_volume.rst index 8a89c88..1e31282 100644 --- a/umn/source/storage/deployment_examples/creating_a_pod_mounted_with_an_evs_volume.rst +++ b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_evs_volume.rst @@ -2,8 +2,8 @@ .. _cce_10_0257: -Creating a Pod Mounted with an EVS Volume -========================================= +Creating a Deployment Mounted with an EVS Volume +================================================ Scenario -------- diff --git a/umn/source/storage/deployment_examples/index.rst b/umn/source/storage/deployment_examples/index.rst index f3999e9..d63fdf4 100644 --- a/umn/source/storage/deployment_examples/index.rst +++ b/umn/source/storage/deployment_examples/index.rst @@ -5,7 +5,7 @@ Deployment Examples =================== -- :ref:`Creating a Pod Mounted with an EVS Volume ` +- :ref:`Creating a Deployment Mounted with an EVS Volume ` - :ref:`Creating a Deployment Mounted with an OBS Volume ` - :ref:`Creating a StatefulSet Mounted with an OBS Volume ` - :ref:`Creating a Deployment Mounted with an SFS Volume ` @@ -15,7 +15,7 @@ Deployment Examples :maxdepth: 1 :hidden: - creating_a_pod_mounted_with_an_evs_volume + creating_a_deployment_mounted_with_an_evs_volume creating_a_deployment_mounted_with_an_obs_volume creating_a_statefulset_mounted_with_an_obs_volume creating_a_deployment_mounted_with_an_sfs_volume diff --git a/umn/source/storage/overview.rst b/umn/source/storage/overview.rst index 30b98fb..b2dd45e 100644 --- a/umn/source/storage/overview.rst +++ b/umn/source/storage/overview.rst @@ -8,9 +8,7 @@ Overview Volume ------ -On-disk files in a container are ephemeral, which will be lost when the container crashes and are difficult to be shared between containers running together in a pod. The Kubernetes volume abstraction solves both of these problems. Volumes cannot be independently created, but defined in the pod spec. - -All containers in a pod can access its volumes, but the volumes must have been mounted. Volumes can be mounted to any directory in a container. +On-disk files in a container are ephemeral, which will be lost when the container crashes and are difficult to be shared between containers running together in a pod. The Kubernetes volume abstraction solves both of these problems. Volumes cannot be independently created, but defined in the pod spec. All containers in a pod can access its volumes, but the volumes must have been mounted to any directory in a container. The following figure shows how a storage volume is used between containers in a pod. diff --git a/umn/source/storage/persistentvolumeclaims_pvcs.rst b/umn/source/storage/persistentvolumeclaims_pvcs.rst index af4d2c5..a90b650 100644 --- a/umn/source/storage/persistentvolumeclaims_pvcs.rst +++ b/umn/source/storage/persistentvolumeclaims_pvcs.rst @@ -77,7 +77,7 @@ StorageClass describes the storage class used in the cluster. You need to specif - Ultra-high I/O - **Access Mode**: **ReadWriteOnce** and **ReadWriteMany** are supported. For details, see :ref:`Volume Access Modes `. - - **Capacity (GiB)** (supported only by EVS and SFS): storage capacity. This parameter is not available for OBS. + - **Capacity (GiB)** (only EVS and SFS are supported): storage capacity. This parameter is not available for OBS. - **Encryption** (supported only for EVS and SFS): Select **Encryption**. After selecting this option, you need to select a key. - **Secret** (supported only for OBS): Select an access key for OBS. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. @@ -173,7 +173,7 @@ If a PV has been created, you can create a PVC to apply for PV resources. - **Storage Volume Claim Type**: Select a storage type as required. - **PVC Name**: name of a PVC. - **Creation Method**: Select **Existing storage volume**. - - **Associate Volume**: Select the volume to be associated, that is, the PV. + - **PV**: Select the volume to be associated, that is, the PV. #. Click **Create**. diff --git a/umn/source/storage/persistentvolumes_pvs.rst b/umn/source/storage/persistentvolumes_pvs.rst index 96141d6..3a447d0 100644 --- a/umn/source/storage/persistentvolumes_pvs.rst +++ b/umn/source/storage/persistentvolumes_pvs.rst @@ -157,6 +157,21 @@ Creating an SFS Volume - The SFS file system and the cluster must be in the same VPC. +**Using the CCE Console** + +#. Log in to the CCE console. +#. Click the cluster name and access the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumes (PVs)** tab. +#. Click **Create Volume** in the upper right corner. In the dialog box displayed, set the volume parameters. + + - **Volume Type**: Select **SFS**. + - Select SFS resources. + - **PV Name**: Enter a PV name. + - **Access Mode**: ReadWriteMany + - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. + - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. + +#. Click **Create**. + **Using YAML** .. code-block:: @@ -181,7 +196,7 @@ Creating an SFS Volume everest.io/share-export-location: # Shared path of the file storage storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner persistentVolumeReclaimPolicy: Retain # Reclaim policy. - storageClassName: csi-nas # Storage class name. + storageClassName: csi-nas # Storage class name mountOptions: [] # Mount options .. table:: **Table 3** Key parameters @@ -195,9 +210,9 @@ Creating an SFS Volume | | | | | This field is valid only when the everest version is 1.2.9 or later and the reclaim policy is Delete. If the reclaim policy is Delete and the current value is **retain-volume-only**, the associated PV is deleted while the underlying storage volume is retained, when a PVC is deleted. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeHandle | File system ID. | + | volumeHandle | - If SFS Capacity-Oriented file storage is used, enter the file storage ID. | | | | - | | On the management console, choose **Service List** > **Storage** > **Scalable File Service**. In the SFS file system list, click the name of the target file system and copy the content following **ID** on the page displayed. | + | | On the management console, choose **Service List** > **Storage** > **Scalable File Service**. In the SFS file system list, click the name of the target file system and copy the content following **ID** on the page displayed. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | everest.io/share-export-location | Shared path of the file system. | | | | @@ -251,7 +266,7 @@ Creating an OBS Volume - **PV Name**: Enter a PV name. - **Access Mode**: ReadWriteMany - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. - - **Key**: You can customize the access key (AK/SK) for mounting an OBS volume. You can use the AK/SK to create a secret and mount the secret to the PV. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. + - **Secret**: You can customize the access key (AK/SK) for mounting an OBS volume. You can use the AK/SK to create a secret and mount the secret to the PV. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. #. Click **Create**. diff --git a/umn/source/storage/setting_mount_options.rst b/umn/source/storage/setting_mount_options.rst index e7e5420..8d9508a 100644 --- a/umn/source/storage/setting_mount_options.rst +++ b/umn/source/storage/setting_mount_options.rst @@ -17,7 +17,7 @@ This section describes how to set mount options when mounting SFS and OBS volume SFS Volume Mount Options ------------------------ -The everest add-on in CCE presets the options described in :ref:`Table 1 ` for mounting SFS volumes. You can set other mount options if needed. For details, see `Mounting an NFS File System to ECSs (Linux) `__ +The everest add-on in CCE presets the options described in :ref:`Table 1 ` for mounting SFS volumes. You can set other mount options if needed. For details, see `Mounting an NFS File System to ECSs (Linux) `__. .. _cce_10_0337__table128754351546: @@ -78,7 +78,7 @@ When mounting file storage, the everest add-on presets the options described in +-----------------------+--------------------------------------------------------------------------------------------------------------------+ | Option | Description | +=======================+====================================================================================================================+ - | max_write=131072 | If specified, obsfs allocates the **inode** number. Enabled by default in read/write mode. | + | max_write=131072 | This parameter is valid only when **big_writes** is configured. The recommended value is **128 KB**. | +-----------------------+--------------------------------------------------------------------------------------------------------------------+ | ssl_verify_hostname=0 | Disables verifying the SSL certificate based on the host name. | +-----------------------+--------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage/using_local_disks_as_storage_volumes.rst b/umn/source/storage/using_local_disks_as_storage_volumes.rst index 01b9d17..2b23dbd 100644 --- a/umn/source/storage/using_local_disks_as_storage_volumes.rst +++ b/umn/source/storage/using_local_disks_as_storage_volumes.rst @@ -28,7 +28,7 @@ You can mount a path on the host to a specified container path. A hostPath volum #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image1|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **hostPath** from the drop-down list. #. Set parameters for adding a local volume, as listed in :ref:`Table 1 `. @@ -39,7 +39,7 @@ You can mount a path on the host to a specified container path. A hostPath volum +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Storage Type | Select **HostPath**. | + | Storage Type | Select **hostPath**. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Host Path | Path of the host to which the local volume is to be mounted, for example, **/etc/hosts**. | | | | @@ -75,7 +75,7 @@ You can mount a path on the host to a specified container path. A hostPath volum | | - **Read-only**: You can only read the data volumes mounted to the path. | | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image1| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. _cce_10_0377__section550555216467: @@ -87,7 +87,7 @@ emptyDir applies to temporary data storage, disaster recovery, and runtime data #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image2|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **emptyDir** from the drop-down list. #. Set the local volume type to **emptyDir** and set parameters for adding a local volume, as described in :ref:`Table 2 `. @@ -100,7 +100,7 @@ emptyDir applies to temporary data storage, disaster recovery, and runtime data +===================================+=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ | Storage Type | Select **emptyDir**. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Medium | - **Default**: Data is stored in hard disks, which is applicable to a large amount of data with low requirements on reading and writing efficiency. | + | Storage Medium | - **Default**: Data is stored in hard disks, which is applicable to a large amount of data with low requirements on reading and writing efficiency. | | | - **Memory**: Selecting this option can improve the running speed, but the storage capacity is subject to the memory size. This mode applies to scenarios where the data volume is small and the read and write efficiency is high. | | | | | | .. note:: | @@ -129,7 +129,7 @@ emptyDir applies to temporary data storage, disaster recovery, and runtime data | | - **Read-only**: You can only read the data volumes mounted to the path. | | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image2| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. _cce_10_0377__section18638191594712: @@ -141,7 +141,7 @@ The data stored in a ConfigMap can be referenced in a volume of type ConfigMap. #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image3|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **ConfigMap** from the drop-down list. #. Set the local volume type to **ConfigMap** and set parameters for adding a local volume, as shown in :ref:`Table 3 `. @@ -177,7 +177,7 @@ The data stored in a ConfigMap can be referenced in a volume of type ConfigMap. | | | | | c. Set the permission to **Read-only**. Data volumes in the path are read-only. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image3| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. _cce_10_0377__section10197243134710: @@ -189,7 +189,7 @@ You can mount a secret as a volume to the specified container path. Contents in #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image4|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **Secret** from the drop-down list. #. Set the local volume type to **Secret** and set parameters for adding a local volume, as shown in :ref:`Table 4 `. @@ -225,7 +225,7 @@ You can mount a secret as a volume to the specified container path. Contents in | | | | | c. Set the permission to **Read-only**. Data volumes in the path are read-only. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image4| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Mounting a hostPath Volume Using kubectl @@ -343,7 +343,7 @@ You can use kubectl to mount a file directory of the host where the container is -rw-r--r-- 1 root root 0 Jun 1 08:12 test1 -rw-r--r-- 1 root root 0 Jun 1 08:14 test2 -.. |image1| image:: /_static/images/en-us_image_0000001248946053.png -.. |image2| image:: /_static/images/en-us_image_0000001249026401.png -.. |image3| image:: /_static/images/en-us_image_0000001248666457.png -.. |image4| image:: /_static/images/en-us_image_0000001248946421.png +.. |image1| image:: /_static/images/en-us_image_0000001465197524.png +.. |image2| image:: /_static/images/en-us_image_0000001515917789.png +.. |image3| image:: /_static/images/en-us_image_0000001464878016.png +.. |image4| image:: /_static/images/en-us_image_0000001515838557.png diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst index f1c7ca7..7ea9ea4 100644 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst +++ b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst @@ -22,7 +22,7 @@ Procedure **touch pv-evs-example.yaml** **pvc-evs-example.yaml** +-------------------------------+--------------------------------+-----------------------------------------------------+ - | Kubernetes Version | Description | YAML Example | + | Kubernetes Cluster Version | Description | YAML Example | +===============================+================================+=====================================================+ | 1.11.7 <= K8s version <= 1.13 | Clusters from v1.11.7 to v1.13 | :ref:`Example YAML ` | +-------------------------------+--------------------------------+-----------------------------------------------------+ @@ -147,7 +147,7 @@ Procedure | volumeName | Name of the PV. | +-----------------------------------------------+---------------------------------------------------------------------------------------------+ - **Clusters from v1.11 to v1.11.7** + **1.11 <= K8s version < 1.11.7** - .. _cce_10_0313__li19211184720504: diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst index 9d5325e..5efd49b 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst @@ -27,7 +27,7 @@ Procedure **touch pv-obs-example.yaml** **pvc-obs-example.yaml** +-----------------------------+------------------------------+-----------------------------------------------------+ - | Kubernetes Version | Description | YAML Example | + | Kubernetes Cluster Version | Description | YAML Example | +=============================+==============================+=====================================================+ | 1.11 <= K8s version <= 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | +-----------------------------+------------------------------+-----------------------------------------------------+ diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst index 84c3232..bed800b 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst @@ -21,13 +21,13 @@ Procedure **touch pv-sfs-example.yaml** **pvc-sfs-example.yaml** - +-----------------------------+------------------------------+-----------------------------------------------------+ - | Kubernetes Version | Description | YAML Example | - +=============================+==============================+=====================================================+ - | 1.11 <= K8s version <= 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | - +-----------------------------+------------------------------+-----------------------------------------------------+ - | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | - +-----------------------------+------------------------------+-----------------------------------------------------+ + +----------------------------+------------------------------+-----------------------------------------------------+ + | Kubernetes Cluster Version | Description | YAML Example | + +============================+==============================+=====================================================+ + | 1.11 <= K8s version < 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | + +----------------------------+------------------------------+-----------------------------------------------------+ + | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | + +----------------------------+------------------------------+-----------------------------------------------------+ **Clusters from v1.11 to v1.13** diff --git a/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst b/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst index 6348d93..5a41319 100644 --- a/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst +++ b/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst @@ -23,7 +23,7 @@ Check Method This health check mode is applicable to containers that provide HTTP/HTTPS services. The cluster periodically initiates an HTTP/HTTPS GET request to such containers. If the return code of the HTTP/HTTPS response is within 200-399, the probe is successful. Otherwise, the probe fails. In this health check mode, you must specify a container listening port and an HTTP/HTTPS request path. - For example, for a container that provides HTTP services, the HTTP check path is **/health-check**, the port is 80, and the host address is optional (which defaults to the container IP address). Here, 172.16.0.186 is used as an example, and we can get such a request: GET http://172.16.0.186:80/health-check. The cluster periodically initiates this request to the container. + For example, for a container that provides HTTP services, the HTTP check path is **/health-check**, the port is 80, and the host address is optional (which defaults to the container IP address). Here, 172.16.0.186 is used as an example, and we can get such a request: GET http://172.16.0.186:80/health-check. The cluster periodically initiates this request to the container. You can also add one or more headers to an HTTP request. For example, set the request header name to **Custom-Header** and the corresponding value to **example**. - **TCP port** @@ -50,6 +50,16 @@ Check Method - Put the program to be executed in the container image so that the program can be executed. - If the command to be executed is a shell script, do not directly specify the script as the command, but add a script parser. For example, if the script is **/data/scripts/health_check.sh**, you must specify **sh/data/scripts/health_check.sh** for command execution. The reason is that the cluster is not in the terminal environment when executing programs in a container. +- **gRPC Check** + + gRPC checks can configure startup, liveness, and readiness probes for your gRPC application without exposing any HTTP endpoint, nor do you need an executable. Kubernetes can connect to your workload via gRPC and query its status. + + .. important:: + + - The gRPC check is supported only in CCE clusters of v1.25 or later. + - To use gRPC for check, your application must support the `gRPC health checking protocol `__. + - Similar to HTTP and TCP probes, if the port is incorrect or the application does not support the health checking protocol, the check fails. + Common Parameters ----------------- diff --git a/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst b/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst index f80a795..026dac6 100644 --- a/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst +++ b/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst @@ -24,7 +24,7 @@ Using the Console Create a secret for accessing a third-party image repository. - Click the cluster name and access the cluster details page. In the navigation pane, choose **ConfigMaps and Secrets**. On the **Secrets** tab page, click **Create Secret** in the upper right corner. Set **Secret Type** to **kubernetes.io/dockerconfigjson**. For details, see :ref:`Creating a Secret `. + Click the cluster name and access the cluster console. In the navigation pane, choose **ConfigMaps and Secrets**. On the **Secrets** tab page, click **Create Secret** in the upper right corner. Set **Secret Type** to **kubernetes.io/dockerconfigjson**. For details, see :ref:`Creating a Secret `. Enter the user name and password used to access the third-party image repository. diff --git a/umn/source/workloads/creating_a_cron_job.rst b/umn/source/workloads/creating_a_cron_job.rst index 8731232..1fb71f1 100644 --- a/umn/source/workloads/creating_a_cron_job.rst +++ b/umn/source/workloads/creating_a_cron_job.rst @@ -30,7 +30,7 @@ Using the CCE Console #. Log in to the CCE console. -#. Click the cluster name to access the cluster details page, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. +#. Click the cluster name to go to the cluster console, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. #. Set basic information about the workload. @@ -50,7 +50,6 @@ Using the CCE Console - **Basic Info**: See :ref:`Setting Basic Container Information `. - **Lifecycle**: See :ref:`Setting Container Lifecycle Parameters `. - **Environment Variables**: See :ref:`Setting an Environment Variable `. - - **Logging**: See :ref:`Using ICAgent to Collect Container Logs `. - **Image Access Credential**: Select the credential used for accessing the image repository. The default value is **default-secret**. You can use default-secret to access images in SWR. For details about **default-secret**, see :ref:`default-secret `. diff --git a/umn/source/workloads/creating_a_daemonset.rst b/umn/source/workloads/creating_a_daemonset.rst index 0eededf..c448262 100644 --- a/umn/source/workloads/creating_a_daemonset.rst +++ b/umn/source/workloads/creating_a_daemonset.rst @@ -30,7 +30,7 @@ Using the CCE Console #. Log in to the CCE console. -#. Click the cluster name to access the cluster details page, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. +#. Click the cluster name to go to the cluster console, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. #. Set basic information about the workload. @@ -69,7 +69,7 @@ Using the CCE Console A Service is used for pod access. With a fixed IP address, a Service forwards access traffic to pods and performs load balancing for these pods. - You can also create a Service after creating a workload. For details about the Service, see :ref:`Overview `. + You can also create a Service after creating a workload. For details about the Service, see :ref:`Service Overview `. **Advanced Settings** diff --git a/umn/source/workloads/creating_a_deployment.rst b/umn/source/workloads/creating_a_deployment.rst index c70b3ed..3972467 100644 --- a/umn/source/workloads/creating_a_deployment.rst +++ b/umn/source/workloads/creating_a_deployment.rst @@ -65,7 +65,7 @@ Using the CCE Console A Service is used for pod access. With a fixed IP address, a Service forwards access traffic to pods and performs load balancing for these pods. - You can also create a Service after creating a workload. For details about the Service, see :ref:`Overview `. + You can also create a Service after creating a workload. For details about the Service, see :ref:`Service Overview `. **Advanced Settings** diff --git a/umn/source/workloads/creating_a_job.rst b/umn/source/workloads/creating_a_job.rst index e8ad9f7..1e54800 100644 --- a/umn/source/workloads/creating_a_job.rst +++ b/umn/source/workloads/creating_a_job.rst @@ -28,7 +28,7 @@ Using the CCE Console #. Log in to the CCE console. -#. Click the cluster name to access the cluster details page, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. +#. Click the cluster name to go to the cluster console, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. #. Set basic information about the workload. diff --git a/umn/source/workloads/creating_a_statefulset.rst b/umn/source/workloads/creating_a_statefulset.rst index 4c514d2..609566a 100644 --- a/umn/source/workloads/creating_a_statefulset.rst +++ b/umn/source/workloads/creating_a_statefulset.rst @@ -64,9 +64,9 @@ Using the CCE Console - StatefulSets support dynamically provisioned EVS volumes. - Dynamic mounting is achieved by using the `volumeClaimTemplates `__ field and depends on the dynamic creation capability of StorageClass. A StatefulSet associates each pod with a unique PVC using the **volumeClaimTemplates** field, and the PVCs are bound to their corresponding PVs. Therefore, after the pod is rescheduled, the original data can still be mounted thanks to the PVC. + Dynamic mounting is achieved by using the `volumeClaimTemplates `__ field and depends on the dynamic creation capability of StorageClass. A StatefulSet associates each pod with a unique PVC using the **volumeClaimTemplates** field, and the PVCs are bound to their corresponding PVs. Therefore, after the pod is rescheduled, the original data can still be mounted thanks to the PVC. - - After a workload is created, the dynamic storage cannot be updated. + - After a workload is created, the storage that is dynamically mounted cannot be updated. - **Security Context**: Set container permissions to protect the system and other containers from being affected. Enter the user ID to set container permissions and prevent systems and other containers from being affected. - **Logging**: See :ref:`Using ICAgent to Collect Container Logs `. @@ -83,7 +83,7 @@ Using the CCE Console A Service is used for pod access. With a fixed IP address, a Service forwards access traffic to pods and performs load balancing for these pods. - You can also create a Service after creating a workload. For details about the Service, see :ref:`Overview `. + You can also create a Service after creating a workload. For details about the Service, see :ref:`Service Overview `. **Advanced Settings** diff --git a/umn/source/workloads/index.rst b/umn/source/workloads/index.rst index 53a4df3..fd71f38 100644 --- a/umn/source/workloads/index.rst +++ b/umn/source/workloads/index.rst @@ -16,6 +16,7 @@ Workloads - :ref:`GPU Scheduling ` - :ref:`CPU Core Binding ` - :ref:`Pod Labels and Annotations ` +- :ref:`Volcano Scheduling ` - :ref:`Security Group Policies ` .. toctree:: @@ -33,4 +34,5 @@ Workloads gpu_scheduling cpu_core_binding/index pod_labels_and_annotations + volcano_scheduling/index security_group_policies diff --git a/umn/source/workloads/managing_workloads_and_jobs.rst b/umn/source/workloads/managing_workloads_and_jobs.rst index 23fa540..93652dc 100644 --- a/umn/source/workloads/managing_workloads_and_jobs.rst +++ b/umn/source/workloads/managing_workloads_and_jobs.rst @@ -8,7 +8,7 @@ Managing Workloads and Jobs Scenario -------- -After a workload is created, you can, upgrade, monitor, roll back, or delete the workload, as well as edit its YAML file. +After a workload is created, you can upgrade, monitor, roll back, or delete the workload, as well as edit its YAML file. .. table:: **Table 1** Workload/Job management diff --git a/umn/source/workloads/volcano_scheduling/hybrid_deployment_of_online_and_offline_jobs.rst b/umn/source/workloads/volcano_scheduling/hybrid_deployment_of_online_and_offline_jobs.rst new file mode 100644 index 0000000..49a4ad0 --- /dev/null +++ b/umn/source/workloads/volcano_scheduling/hybrid_deployment_of_online_and_offline_jobs.rst @@ -0,0 +1,521 @@ +:original_name: cce_10_0384.html + +.. _cce_10_0384: + +Hybrid Deployment of Online and Offline Jobs +============================================ + +Online and Offline Jobs +----------------------- + +Jobs can be classified into online jobs and offline jobs based on whether services are always online. + +- **Online job**: Such jobs run for a long time, with regular traffic surges, tidal resource requests, and high requirements on SLA, such as advertising and e-commerce services. +- **Offline jobs**: Such jobs run for a short time, have high computing requirements, and can tolerate high latency, such as AI and big data services. + +Resource Oversubscription and Hybrid Deployment +----------------------------------------------- + +Many services see surges in traffic. To ensure performance and stability, resources are often requested at the maximum needed. However, the surges may ebb very shortly and resources, if not released, are wasted in non-peak hours. Especially for online jobs that request a large quantity of resources to ensure SLA, resource utilization can be as low as it gets. + +Resource oversubscription is the process of making use of idle requested resources. Oversubscribed resources are suitable for deploying offline jobs, which focus on throughput but have low SLA requirements and can tolerate certain failures. + +Hybrid deployment of online and offline jobs in a cluster can better utilize cluster resources. + + +.. figure:: /_static/images/en-us_image_0000001378942548.png + :alt: **Figure 1** Resource oversubscription + + **Figure 1** Resource oversubscription + +Oversubscription for Hybrid Deployment +-------------------------------------- + +Hybrid deployment is supported, and CPU and memory resources can be oversubscribed. The key features are as follows: + +- Offline jobs preferentially run on oversubscribed nodes. + + If both oversubscribed and non-oversubscribed nodes exist, the former will score higher than the latter and offline jobs are preferentially scheduled to oversubscribed nodes. + +- Online jobs can use only non-oversubscribed resources if scheduled to an oversubscribed node. + + Offline jobs can use both oversubscribed and non-oversubscribed resources of an oversubscribed node. + +- In the same scheduling period, online jobs take precedence over offline jobs. + + If both online and offline jobs exist, online jobs are scheduled first. When the node resource usage exceeds the upper limit and the node requests exceed 100%, offline jobs will be evicted. + +- CPU/memory isolation is provided by kernels. + + CPU isolation: Online jobs can quickly preempt CPU resources of offline jobs and suppress the CPU usage of the offline jobs. + + Memory isolation: When system memory resources are used up and OOM Kill is triggered, the kernel evicts offline jobs first. + +- kubelet offline jobs admission rules: + + After the the pod is scheduled to a node, kubelet starts the pod only when the node resources can meet the pod request (predicateAdmitHandler.Admit). kubelet starts the pod when both of the following conditions are met: + + - The total request of pods to be started and online running jobs < allocatable nodes + - The total request of pods to be started and online/offline running job < allocatable nodes+oversubscribed nodes + +- Resource oversubscription and hybrid deployment: + + If only hybrid deployment is used, you need to configure the label **volcano.sh/colocation=true** for the node and delete the node label **volcano.sh/oversubscription** or set its value to **false**. + + If the label **volcano.sh/colocation=true** is configured for a node, hybrid deployment is enabled. If the label **volcano.sh/oversubscription=true** is configured, resource oversubscription is enabled. The following table lists the available feature combinations after hybrid deployment or resource oversubscription is enabled. + + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | Hybrid Deployment Enabled (volcano.sh/colocation=true) | Resource oversubscription Enabled (volcano.sh/oversubscription=true) | Use Oversubscribed Resources? | Conditions for Evicting Offline Pods | + +========================================================+======================================================================+===============================+========================================================================================+ + | No | No | No | None | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | Yes | No | No | The node resource usage exceeds the high threshold. | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | No | Yes | Yes | The node resource usage exceeds the high threshold, and the node request exceeds 100%. | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | Yes | Yes | Yes | The node resource usage exceeds the high threshold. | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + +Notes and Constraints +--------------------- + +**Specifications** + +- Kubernetes version: + + - 1.19: 1.19.16-r4 or later + - 1.21: 1.21.7-r0 or later + - 1.23: 1.23.5-r0 or later + +- Cluster Type: CCE or CCE Turbo +- Node OS: EulerOS 2.9 (kernel-4.18.0-147.5.1.6.h729.6.eulerosv2r9.x86_64) +- Node Type: ECS +- The volcano add-on version: 1.7.0 or later + +**Constraints** + +- Before enabling the volcano oversubscription plug-in, ensure that the overcommit plug-in is not enabled. +- Modifying the label of an oversubscribed node does not affect the running pods. +- Running pods cannot be converted between online and offline services. To convert services, you need to rebuild pods. +- If the label **volcano.sh/oversubscription=true** is configured for a node in the cluster, the **oversubscription** configuration must be added to the volcano add-on. Otherwise, the scheduling of oversubscribed nodes will be abnormal. Ensure that you have correctly configure labels because the scheduler does not check the add-on and node configurations. For details about the labels, see :ref:`Configuring Oversubscription Labels for Scheduling `. +- To disable oversubscription, perform the following operations: + + - Remove the **volcano.sh/oversubscription** label from the oversubscribed node. + - Set **over-subscription-resource** to **false**. + - Modify the configmap of the volcano scheduler named **volcano-scheduler-configmap** and remove the oversubscription add-on. + +- If **cpu-manager-policy** is set to static core binding on a node, do not assign the QoS class of Guaranteed to offline pods. If core binding is required, change the pods to online pods. Otherwise, offline pods may occupy the CPUs of online pods, causing online pod startup failures, and offline pods fail to be started although they are successfully scheduled. +- If **cpu-manager-policy** is set to static core binding on a node, do not bind cores to all online pods. Otherwise, online pods occupy all CPU or memory resources, leaving a small number of oversubscribed resources. + +.. _cce_10_0384__section1940910414220: + +Configuring Oversubscription Labels for Scheduling +-------------------------------------------------- + +If the label **volcano.sh/oversubscription=true** is configured for a node in the cluster, the **oversubscription** configuration must be added to the volcano add-on. Otherwise, the scheduling of oversubscribed nodes will be abnormal. For details about the related configuration, see :ref:`Table 1 `. + +Ensure that you have correctly configure labels because the scheduler does not check the add-on and node configurations. + +.. _cce_10_0384__table152481219311: + +.. table:: **Table 1** Configuring oversubscription labels for scheduling + + +----------------------------+--------------------------------+----------------------------------------------------+ + | Oversubscription in Add-on | Oversubscription Label on Node | Scheduling | + +============================+================================+====================================================+ + | Yes | Yes | Triggered by oversubscription | + +----------------------------+--------------------------------+----------------------------------------------------+ + | Yes | No | Triggered | + +----------------------------+--------------------------------+----------------------------------------------------+ + | No | No | Triggered | + +----------------------------+--------------------------------+----------------------------------------------------+ + | No | Yes | Not triggered or failed. Avoid this configuration. | + +----------------------------+--------------------------------+----------------------------------------------------+ + +Using Hybrid Deployment +----------------------- + +#. Configure the volcano add-on. + + a. Use kubectl to connect to the cluster. + + b. Install the volcano plug-in and add the **oversubscription** plug-in to **volcano-scheduler-configmap**. Ensure that the plug-in configuration does not contain the **overcommit** plug-in. If **- name: overcommit** exists, delete this configuration. + + .. code-block:: + + # kubectl edit cm volcano-scheduler-configmap -n kube-system + apiVersion: v1 + data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: gang + - name: priority + - name: conformance + - name: oversubscription + - plugins: + - name: drf + - name: predicates + - name: nodeorder + - name: binpack + - plugins: + - name: cce-gpu-topology-predicate + - name: cce-gpu-topology-priority + - name: cce-gpu + +#. Enable the node oversubscription feature. + + A label can be configured to use oversubscribed resources only after the oversubscription feature is enabled for a node. Related nodes can be created only in a node pool. To enable the oversubscription feature, perform the following steps: + + a. Create a node pool. + b. Choose **More** > **Manage** in the **Operation** column of the created node pool. + c. In the **Manage Component** window that is displayed, set **over-subscription-resource** under **kubelet** to **true** and click **OK**. + + |image1| + +#. Set the node oversubscription label. + + The **volcano.sh/oversubscription** label needs to be configured for an oversubscribed node. If this label is set for a node and the value is **true**, the node is an oversubscribed node. Otherwise, the node is not an oversubscribed node. + + .. code-block:: + + kubectl label node 192.168.0.0 volcano.sh/oversubscription=true + + An oversubscribed node also supports the oversubscription thresholds, as listed in :ref:`Table 2 `. For example: + + .. code-block:: + + kubectl annotate node 192.168.0.0 volcano.sh/evicting-cpu-high-watermark=70 + + Querying the node information + + .. code-block:: + + # kubectl describe node 192.168.0.0 + Name: 192.168.0.0 + Roles: + Labels: ... + volcano.sh/oversubscription=true + Annotations: ... + volcano.sh/evicting-cpu-high-watermark: 70 + + .. _cce_10_0384__table1853397191112: + + .. table:: **Table 2** Node oversubscription annotations + + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Name | Description | + +===========================================+====================================================================================================================================+ + | volcano.sh/evicting-cpu-high-watermark | When the CPU usage of a node exceeds the specified value, offline job eviction is triggered and the node becomes unschedulable. | + | | | + | | The default value is **80**, indicating that offline job eviction is triggered when the CPU usage of a node exceeds 80%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/evicting-cpu-low-watermark | After eviction is triggered, the scheduling starts again when the CPU usage of a node is lower than the specified value. | + | | | + | | The default value is **30**, indicating that scheduling starts again when the CPU usage of a node is lower than 30%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/evicting-memory-high-watermark | When the memory usage of a node exceeds the specified value, offline job eviction is triggered and the node becomes unschedulable. | + | | | + | | The default value is **60**, indicating that offline job eviction is triggered when the memory usage of a node exceeds 60%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/evicting-memory-low-watermark | After eviction is triggered, the scheduling starts again when the memory usage of a node is lower than the specified value. | + | | | + | | The default value is **30**, indicating that the scheduling starts again when the memory usage of a node is less than 30%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/oversubscription-types | Oversubscribed resource type. The options are as follows: | + | | | + | | - CPU (oversubscribed CPU) | + | | - memory (oversubscribed memory) | + | | - cpu,memory (oversubscribed CPU and memory) | + | | | + | | The default value is **cpu,memory**. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + +#. Deploy online and offline jobs. + + The **volcano.sh/qos-level** label needs to be added to annotation to distinguish offline jobs. The value is an integer ranging from -7 to 7. If the value is less than 0, the job is an offline job. If the value is greater than or equal to 0, the job is a high-priority job, that is, online job. You do not need to set this label for online jobs. For both online and offline jobs, set **schedulerName** to **volcano** to enable the Volcano scheduler. + + .. note:: + + The priorities of online/online and offline/offline jobs are not differentiated, and the value validity is not verified. If the value of **volcano.sh/qos-level** of an offline job is not a negative integer ranging from -7 to 0, the job is processed as an online job. + + For an offline job: + + .. code-block:: + + kind: Deployment + apiVersion: apps/v1 + spec: + replicas: 4 + template: + metadata: + annotations: + metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"","path":"","port":"","names":""}]' + volcano.sh/qos-level: "-1" # Offline job label + spec: + schedulerName: volcano # The Volcano scheduler is used. + ... + + For an online job: + + .. code-block:: + + kind: Deployment + apiVersion: apps/v1 + spec: + replicas: 4 + template: + metadata: + annotations: + metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"","path":"","port":"","names":""}]' + spec: + schedulerName: volcano # The Volcano scheduler is used. + ... + +#. Run the following command to check the number of oversubscribed resources and the resource usage: + + kubectl describe node ** + + .. code-block:: + + # kubectl describe node 192.168.0.0 + Name: 192.168.0.0 + Roles: + Labels: ... + volcano.sh/oversubscription=true + Annotations: ... + volcano.sh/oversubscription-cpu: 2335 + volcano.sh/oversubscription-memory: 341753856 + Allocatable: + cpu: 3920m + memory: 6263988Ki + Allocated resources: + (Total limits may be over 100 percent, i.e., overcommitted.) + Resource Requests Limits + -------- -------- ------ + cpu 4950m (126%) 4950m (126%) + memory 1712Mi (27%) 1712Mi (27%) + +Hybrid Deployment Example +------------------------- + +The following uses an example to describe how to deploy online and offline jobs in hybrid mode. + +#. Assume that a cluster has two nodes: one oversubscribed node and one non-oversubscribed node. + + .. code-block:: + + # kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.173 Ready 4h58m v1.19.16-r2-CCE22.5.1 + 192.168.0.3 Ready 148m v1.19.16-r2-CCE22.5.1 + + - 192.168.0.173 is an oversubscribed node (with the **volcano.sh/oversubscirption=true** label). + - 192.168.0.3 is a non-oversubscribed node (without the **volcano.sh/oversubscirption=true** label). + + .. code-block:: + + # kubectl describe node 192.168.0.173 + Name: 192.168.0.173 + Roles: + Labels: beta.kubernetes.io/arch=amd64 + ... + volcano.sh/oversubscription=true + +#. Submit offline job creation requests. If resources are sufficient, all offline jobs will be scheduled to the oversubscribed node. + + The offline job template is as follows: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: offline + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: offline + template: + metadata: + labels: + app: offline + annotations: + volcano.sh/qos-level: "-1" #Offline job label + spec: + schedulerName: volcano # The Volcano scheduler is used. + containers: + - name: container-1 + image: nginx:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 500m + memory: 512Mi + limits: + cpu: "1" + memory: 512Mi + imagePullSecrets: + - name: default-secret + + Offline jobs are scheduled to the oversubscribed node. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + offline-69cdd49bf4-pmjp8 1/1 Running 0 5s 192.168.10.178 192.168.0.173 + offline-69cdd49bf4-z8kxh 1/1 Running 0 5s 192.168.10.131 192.168.0.173 + +#. Submit online job creation requests. If resources are sufficient, the online jobs will be scheduled to the non-oversubscribed node. + + The online job template is as follows: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: online + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: online + template: + metadata: + labels: + app: online + spec: + schedulerName: volcano # The Volcano scheduler is used. + containers: + - name: container-1 + image: resource_consumer:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 1400m + memory: 512Mi + limits: + cpu: "2" + memory: 512Mi + imagePullSecrets: + - name: default-secret + + Online jobs are scheduled to the non-oversubscribed node. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + online-ffb46f656-4mwr6 1/1 Running 0 5s 192.168.10.146 192.168.0.3 + online-ffb46f656-dqdv2 1/1 Running 0 5s 192.168.10.67 192.168.0.3 + +#. Improve the resource usage of the oversubscribed node and observe whether offline job eviction is triggered. + + Deploy online jobs to the oversubscribed node (192.168.0.173). + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: online + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: online + template: + metadata: + labels: + app: online + spec: + affinity: # Submit an online job to an oversubscribed node. + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/hostname + operator: In + values: + - 192.168.0.173 + schedulerName: volcano # The Volcano scheduler is used. + containers: + - name: container-1 + image: resource_consumer:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 700m + memory: 512Mi + limits: + cpu: 700m + memory: 512Mi + imagePullSecrets: + - name: default-secret + + Submit the online or offline jobs to the oversubscribed node (192.168.0.173) at the same time. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + offline-69cdd49bf4-pmjp8 1/1 Running 0 13m 192.168.10.178 192.168.0.173 + offline-69cdd49bf4-z8kxh 1/1 Running 0 13m 192.168.10.131 192.168.0.173 + online-6f44bb68bd-b8z9p 1/1 Running 0 3m4s 192.168.10.18 192.168.0.173 + online-6f44bb68bd-g6xk8 1/1 Running 0 3m12s 192.168.10.69 192.168.0.173 + + Observe the oversubscribed node (192.168.0.173). You can find that oversubscribed resources exist and the CPU allocation rate exceeds 100%. + + .. code-block:: + + # kubectl describe node 192.168.0.173 + Name: 192.168.0.173 + Roles: + Labels: … + volcano.sh/oversubscription=true + Annotations: … + volcano.sh/oversubscription-cpu: 2343 + volcano.sh/oversubscription-memory: 3073653200 + … + Allocated resources: + (Total limits may be over 100 percent, i.e., overcommitted.) + Resource Requests Limits + -------- -------- ------ + cpu 4750m (121%) 7350m (187%) + memory 3760Mi (61%) 4660Mi (76%) + … + + Increase the CPU usage of online jobs on the node. Offline job eviction is triggered. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + offline-69cdd49bf4-bwdm7 1/1 Running 0 11m 192.168.10.208 192.168.0.3 + offline-69cdd49bf4-pmjp8 0/1 Evicted 0 26m 192.168.0.173 + offline-69cdd49bf4-qpdss 1/1 Running 0 11m 192.168.10.174 192.168.0.3 + offline-69cdd49bf4-z8kxh 0/1 Evicted 0 26m 192.168.0.173 + online-6f44bb68bd-b8z9p 1/1 Running 0 24m 192.168.10.18 192.168.0.173 + online-6f44bb68bd-g6xk8 1/1 Running 0 24m 192.168.10.69 192.168.0.173 + +Handling Suggestions +-------------------- + +- After kubelet of the oversubscribed node is restarted, the resource view of the Volcano scheduler is not synchronized with that of kubelet. As a result, OutOfCPU occurs in some newly scheduled jobs, which is normal. After a period of time, the Volcano scheduler can properly schedule online and offline jobs. + +- After online and offline jobs are submitted, you are not advised to dynamically change the job type (adding or deleting annotation volcano.sh/qos-level: "-1") because the current kernel does not support the change of an offline job to an online job. + +- CCE collects the resource usage (CPU/memory) of all pods running on a node based on the status information in the cgroups system. The resource usage may be different from the monitored resource usage, for example, the resource statistics displayed by running the **top** command. + +- You can add oversubscribed resources (such as CPU and memory) at any time. + + You can reduce the oversubscribed resource types only when the resource allocation rate does not exceed 100%. + +.. |image1| image:: /_static/images/en-us_image_0000001207511384.png diff --git a/umn/source/workloads/volcano_scheduling/index.rst b/umn/source/workloads/volcano_scheduling/index.rst new file mode 100644 index 0000000..71363a4 --- /dev/null +++ b/umn/source/workloads/volcano_scheduling/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_10_0423.html + +.. _cce_10_0423: + +Volcano Scheduling +================== + +- :ref:`Hybrid Deployment of Online and Offline Jobs ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + hybrid_deployment_of_online_and_offline_jobs