Creating a CCE Cluster

On the CCE console, you can easily create Kubernetes clusters. Kubernetes can manage container clusters at scale. A cluster manages a group of node resources.

In CCE, you can create a CCE cluster to manage VMs as nodes. By using high-performance network models, hybrid clusters provide a multi-scenario, secure, and stable runtime environment for containers.

Notes and Constraints

Procedure

  1. Log in to the CCE console. On the Dashboard page, click Create Cluster. Alternatively, choose Resource Management > Clusters in the navigation pane and click Create next to CCE Cluster.
  2. Set cluster parameters by referring to Table 1. Pay attention to the parameters marked with an asterisk (*).

    Table 1 Parameters for creating a cluster

    Parameter

    Description

    Region

    Select a region near you to ensure the lowest latency possible.

    *Cluster Name

    Name of the new cluster, which cannot be changed after the cluster is created.

    A cluster name contains 4 to 128 characters starting with a letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed.

    Version

    Kubernetes community baseline version. The latest version is recommended.

    If a Beta version is available, you can use it for trial. However, it is not recommended for commercial use.

    Management Scale

    Maximum number of worker nodes that can be managed by the master nodes of the current cluster. You can select 50 nodes, 200 nodes, or 1,000 nodes for your cluster, or 2,000 nodes if you are buying a cluster of v1.15.11 or later.

    If you select 1000 nodes, the master nodes of the cluster can manage a maximum of 1000 worker nodes. The configuration fee varies depending on the specifications of master nodes for different management scales.

    Number of master nodes

    3: Three master nodes will be created to make the cluster highly available. If a master node is faulty, the cluster can still be available without affecting service functions. Click Change. In the dialog box displayed, you can configure the following parameters:

    Disaster recovery level

    • AZ: Master nodes are deployed in different AZs for disaster recovery.
    • Fault domain: Master nodes are deployed in different failure domains in the same AZ for disaster recovery. This option is displayed only when the environment supports failure domains.
    • Host computer: Master nodes are deployed on different hosts in the same AZ for disaster recovery.
    • Customize: You can select different locations to deploy different master nodes. In the fault domain mode, master nodes must be in the same AZ.

    1: Only one master node is created in the cluster, which cannot ensure SLA for the cluster. Single-master clusters (non-HA clusters) are not recommended for commercial scenarios. Click Change. In the AZ Settings dialog box, select an AZ for the master node.

    NOTE:
    • You are advised to create multiple master nodes to improve the cluster DR capability in commercial scenarios.
    • The multi-master mode cannot be changed after the cluster is created. A single-master cluster cannot be upgraded to a multi-master cluster. For a single-master cluster, if a master node is faulty, services will be affected.
    • To ensure reliability, the multi-master mode is enabled by default for a cluster with 1,000 or more nodes.

    *VPC

    VPC where the cluster is located. The value cannot be changed after the cluster is created.

    A VPC provides a secure and logically isolated network environment.

    If no VPC is available, click Create a VPC to create a VPC. After the VPC is created, click the refresh icon.

    *Subnet

    Subnet where the node VM runs. The value cannot be changed after the cluster is created.

    A subnet provides dedicated network resources that are logically isolated from other networks for network security.

    If no subnet is available, click Create Subnet to create a subnet. After the subnet is created, click the refresh icon. For details about the relationship between VPCs, subnets, and clusters, see Cluster Overview.

    During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name.

    The selected subnet cannot be changed after the cluster is created.

    Network Model

    After a cluster is created, the network model cannot be changed. Exercise caution when selecting a network model. For details about how to select a network model, see Overview.

    VPC network

    In this network model, each node occupies one VPC route. The number of VPC routes supported by the current region and the number of container IP addresses that can be allocated to each node (that is, the maximum number of pods that can be created) are displayed on the console.

    • The container network uses VPC routes to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. However, each node occupies one VPC route, and the maximum number of nodes allowed in a cluster depends on the VPC route quota.
    • Each node is assigned a CIDR block of a fixed size. VPC networks are free from packet encapsulation overheads and outperform container tunnel networks. In addition, as VPC routing includes routes to node IP addresses and the container CIDR block, container pods in the cluster can be directly accessed from outside the cluster.
      NOTE:
      • In the VPC network model, extended CIDR blocks and network policies are not supported.
      • When creating multiple clusters using the VPC network model in one VPC, select a CIDR block for each cluster that does not overlap with the VPC address or other container CIDR blocks.

    Tunnel network

    • The container network is an overlay tunnel network on top of a VPC network and uses the VXLAN technology. This network model is applicable when there is no high requirements on performance.
    • VXLAN encapsulates Ethernet packets as UDP packets for tunnel transmission. Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications.

    Container Network Segment

    An IP address range that can be allocated to container pods. After the cluster is created, the value cannot be changed.

    • If Automatically select is deselected, enter a CIDR block manually. If the CIDR block you specify conflicts with a subnet CIDR block, the system prompts you to select another CIDR block. The recommended CIDR blocks are 10.0.0.0/8-18, 172.16.0.0/16-18, and 192.168.0.0/16-18.

      If different clusters share a container CIDR block, an IP address conflict will occur and access to applications may fail.

    • If Automatically select is selected, the system automatically assigns a CIDR block that does not conflict with any subnet CIDR block.

    The mask of the container CIDR block must be appropriate. It determines the number of available nodes in a cluster. A too small mask value will cause the cluster to soon fall short of nodes. After the mask is set, the estimated maximum number of containers supported by the current CIDR block will be displayed.

    Service Network Segment

    An IP address range that can be allocated to Kubernetes Services. After the cluster is created, the value cannot be changed. The Service CIDR block cannot conflict with the created route. If they conflict, select another CIDR block.

    • Default: The default CIDR block 10.247.0.0/16 will be used.
    • Custom: Manually set a CIDR block and mask based on service requirements. The mask determines the maximum number of Service IP addresses available in the cluster.

    Authorization Mode

    RBAC is selected by default and cannot be deselected.

    After RBAC is enabled, IAM users access resources in the cluster according to fine-grained permissions policies. For details, see Namespace Permissions (Kubernetes RBAC-based).

    Authentication Mode

    The authentication mechanism controls user permission on resources in a cluster.

    The X.509-based authentication mode is enabled by default. X.509 is a commonly used certificate format.

    If you want to perform permission control on the cluster, select Enhanced authentication. The cluster will identify users based on the header of the request for authentication.

    You need to upload your own CA certificate, client certificate, and client certificate private key (for details about how to create a certificate, see Certificates), and select I have confirmed that the uploaded certificates are valid.

    CAUTION:
    • Upload a file smaller than 1 MB. The CA certificate and client certificate can be in .crt or .cer format. The private key of the client certificate can only be uploaded unencrypted.
    • The validity period of the client certificate must be longer than five years.
    • The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. If the certificate is invalid, the cluster cannot be created.

    Cluster Description

    Optional. Enter the description of the new container cluster.

    Advanced Settings

    Click Advanced Settings to expand the details page. The following functions are supported (unsupported functions in current AZs are hidden):

    Service Forwarding Mode

    • iptables: Traditional kube-proxy uses iptables rules to implement Service load balancing. In this mode, too many iptables rules will be generated when many Services are deployed. In addition, non-incremental updates will cause a latency and even obvious performance issues in the case of heavy service traffic.
    • ipvs: optimized kube-proxy mode to achieve higher throughput and faster speed, ideal for large-sized clusters. This mode supports incremental updates and can keep connections uninterrupted during Service updates.

      In this mode, when the ingress and Service use the same ELB instance, the ingress cannot be accessed from the nodes and containers in the cluster.

    NOTE:
    • ipvs provides better scalability and performance for large clusters.
    • Compared with iptables, ipvs supports more complex load balancing algorithms such as least load first (LLF) and weighted least connections (WLC).
    • ipvs supports server health checking and connection retries.

    CPU Policy

    This parameter is displayed only for clusters of v1.13.10-r0 and later.

    • On: Exclusive CPU cores can be allocated to workload pods. Select On if your workload is sensitive to latency in CPU cache and scheduling.
    • Off: Exclusive CPU cores will not be allocated to workload pods. Select Off if you want a large pool of shareable CPU cores.

    For details about CPU management policies, see Feature Highlight: CPU Manager.

    After CPU Policy is enabled, workloads cannot be started or created on nodes after the node specifications are changed.

    Open EIP

    An independent public IP address that is reachable from public networks. Select an EIP that has not been bound to any node. A cluster's EIP is preset in the cluster's certificate. Do no delete the EIP after the cluster has been created. Otherwise, two-way authentication will fail.

    • Do not configure: The cluster's master node will not have an EIP.
    • Configure now: If no EIP is available for selection, create one.

  3. Click Next: Create Node and set the following parameters.

    • Create Node
      • Create now: Create a node when creating a cluster. Currently, only VM nodes are supported. If a node fails to be created, the cluster will be rolled back.
      • Create later: No node will be created. Only an empty cluster will be created.
    • Current Region: geographic location of the nodes to be created.
    • AZ: Set this parameter based on the site requirements. An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network.

      You are advised to deploy worker nodes in different AZs after the cluster is created to make your workloads more reliable. When creating a cluster, you can deploy nodes only in one AZ.

    • Node Type
      • VM node: A VM node will be created in the cluster.
    • Node Name: Enter a node name. A node name contains 1 to 56 characters starting with a lowercase letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed.
    • Specifications: Select node specifications that best fit your business needs.
      • General-purpose: provides a balance of computing, memory, and network resources. It is a good choice for many applications, such as web servers, workload development, workload testing, and small-scale databases.
      • Memory-optimized: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive.
      • GPU-accelerated: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. GPU-accelerated nodes can be created only in clusters of v1.11 or later. GPU-accelerated nodes are available only in certain regions.
      • General computing-plus: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance.
      • Disk-intensive: supports local disk storage and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads.

      To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see Formula for Calculating the Reserved Resources of a Node.

    • OS: Select an OS for the node to be created.

      Reinstalling the OS or modifying OS configurations could make the node unavailable. Exercise caution when performing these operations.

    • System Disk: Set the system disk space of the worker node. The value ranges from 40GB to 1024 GB. The default value is 40GB.

      By default, system disks support Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) EVS disks.

      Encryption: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. This function is available only in certain regions.
      • Encryption is not selected by default.
      • After you select Encryption, you can select an existing key in the displayed Encryption Setting dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon.
    • Data Disk: Set the data disk space of the worker node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The EVS disk types provided for the data disk are the same as those for the system disk.

      If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk.

      • LVM: If this option is selected, CCE data disks are managed by the Logical Volume Manager (LVM). On this condition, you can adjust the disk space allocation for different resources. This option is selected for the first disk by default and cannot be unselected. You can choose to enable or disable LVM for new data disks.
        • This option is selected by default, indicating that LVM management is enabled.
        • You can deselect the check box to disable LVM management.
          • Disk space of the data disks managed by LVM will be allocated according to the ratio you set.
          • When creating a node in a cluster of v1.13.10 or later, if LVM is not selected for a data disk, follow instructions in Adding a Second Data Disk to a Node in a CCE Cluster to fill in the pre-installation script and format the data disk. Otherwise, the data disk will still be managed by LVM.
          • When creating a node in a cluster earlier than v1.13.10, you must format the data disks that are not managed by LVM. Otherwise, either these data disks or the first data disk will be managed by LVM.
      • Encryption: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function.
        This function is supported only for clusters of v1.13.10 or later in certain regions, and is not displayed for clusters of v1.13.10 or earlier.
        • Encryption is not selected by default.
        • After you select Encryption, you can select an existing key in the displayed Encryption Setting dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon.
      • Add Data Disk: Currently, a maximum of two data disks can be attached to a node. After the node is created, you can go to the ECS console to attach more data disks. This function is available only to clusters of certain versions.
      • Data disk space allocation: Click to specify the resource ratio for Kubernetes Space and User Space. Disk space of the data disks managed by LVM will be allocated according to the ratio you set. This function is available only to clusters of certain versions.
        • Kubernetes Space: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs.

          The Docker space cannot be less than 10%, and the space size cannot be less than 60 GB. The kubelet space cannot be less than 10%.

          The Docker space size is determined by your service requirements. For details, see Data Disk Space Allocation.

        • User Space: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted.

          Note that the mount path cannot be /, /home/paas, /var/paas, /var/lib, /var/script, /var/log, /mnt/paas, or /opt/cloud, and cannot conflict with the system directories (such as bin, lib, home, root, boot, dev, etc, lost+found, mnt, proc, sbin, srv, tmp, var, media, opt, selinux, sys, and usr). Otherwise, the system or node installation will fail.

      If the cluster version is v1.13.10-r0 or later and the node specification is Disk-intensive, the following options are displayed for data disks:
      • EVS: Parameters are the same as those when the node type is not Disk-intensive. For details, see Data Disk above.
      • Local disk: Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks.
        Local disk parameters are as follows:
        • Disk Mode: If the node type is disk-intensive, the supported disk mode is HDD.
        • Read/Write Mode: When multiple local disks exist, you can set the read/write mode. The serial and sequential modes are supported. Sequential indicates that data is read and written in linear mode. When a disk is used up, the next disk is used. Serial indicates that data is read and written in striping mode, allowing multiple local disks to be read and written at the same time.
        • Kubernetes Space: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs.
        • User Space: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted.
      • The ratio of disk space allocated to the Kubernetes space and user space must be equal to 100% in total. You can click to refresh the data after you have modified the ratio.
      • By default, disks run in the direct-lvm mode. If data disks are removed, the loop-lvm mode will be used and this will impair system stability.
    • VPC: A VPC where the current cluster is located. This parameter cannot be changed and is displayed only for clusters of v1.13.10-r0 or later.
    • Subnet: A subnet improves network security by providing exclusive network resources that are isolated from other networks. You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets.

      During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name.

    • EIP: an independent public IP address. If the nodes to be created require public network access, select Automatically assign or Use existing.
      An EIP bound to the node allows public network access. EIP bandwidth can be modified at any time. An ECS without a bound EIP cannot access the Internet or be accessed by public networks.
      • Do not use: A node without an EIP cannot be accessed from public networks. It can be used only as a cloud server for deploying services or clusters on a private network.
      • Automatically assign: An EIP with specified configurations is automatically assigned to each node. If the number of EIPs is smaller than the number of nodes, the EIPs are randomly bound to the nodes.

        Configure the EIP specifications, billing factor, bandwidth type, and bandwidth size as required. When creating an ECS, ensure that the elastic IP address quota is sufficient.

      • Use existing: Existing EIPs are assigned to the nodes to be created.

      By default, VPC's SNAT feature is disabled for CCE. If SNAT is enabled, you do not need to use EIPs to access public networks. For details about SNAT, see Custom Policies.

    • Login Mode:
      • Key pair: Select the key pair used to log in to the node. You can select a shared key.

        A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click Create a key pair.

        When creating a node using a key pair, IAM users can select only the key pairs created by their own, regardless of whether these users are in the same group. For example, user B cannot use the key pair created by user A to create a node, and the key pair is not displayed in the drop-down list on the CCE console.

    • Advanced ECS Settings (optional): Click to show advanced ECS settings.
      • ECS Group: An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group.
        • Anti-affinity: ECSs in an ECS group are deployed on different physical hosts to improve service reliability.

        Select an existing ECS group, or click Create ECS Group to create one. After the ECS group is created, click the refresh button.

      • Resource Tags: By adding tags to resources, you can classify resources.

        You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency.

        CCE will automatically create the "CCE-Dynamic-Provisioning-Node=node id" tag. A maximum of 5 tags can be added.

      • Agency: An agency is created by a tenant administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. To authorize an ECS or BMS to call cloud services, select Cloud service as the agency type, click Select, and then select ECS BMS.
      • Pre-installation Script: Enter a maximum of 1,000 characters.

        The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. The script is usually used to format data disks.

      • Post-installation Script: Enter a maximum of 1,000 characters.

        The script will be executed after Kubernetes software is installed and will not affect the installation. The script is usually used to modify Docker parameters.

      • Subnet IP Address: Select Automatically assign IP address (recommended) or Manually assigning IP addresses.
    • Advanced Kubernetes Settings: (Optional) Click to show advanced cluster settings.
      • Max Pods: maximum number of pods that can be created on a node, including the system's default pods. If the cluster uses the VPC network model, the maximum value is determined by the number of IP addresses that can be allocated to containers on each node.

        This limit prevents the node from being overloaded by managing too many pods. For details, see Maximum Number of Pods That Can Be Created on a Node.

      • Maximum Data Space per Container: maximum data space that can be used by a container. The value ranges from 10 GB to 500 GB. If the value of this field is larger than the data disk space allocated to Docker resources, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker resources. This parameter is displayed only for clusters of v1.13.10-r0 and later.
    • Nodes: The value cannot exceed the management scale you select when configuring cluster parameters. Set this parameter based on service requirements and the remaining quota displayed on the page. Click to view the factors that affect the number of nodes to be added (depending on the factor with the minimum value).

  4. Click Next: Install Add-on, and select the add-ons to be installed in the Install Add-on step.

    System resource add-ons must be installed. Advanced functional add-ons are optional.

    You can also install all add-ons after the cluster is created. To do so, choose Add-ons in the navigation pane of the CCE console and select the add-on you will install. For details, see Add-ons.

  5. Click Next: Confirm. Read the product instructions and select I am aware of the above limitations. Confirm the configured parameters, specifications, and fees.
  6. Click Submit.

    It takes about 6 to 10 minutes to create a cluster. You can click Back to Cluster List to perform other operations on the cluster or click Go to Cluster Events to view the cluster details. If the cluster status is Available, the cluster is successfully created.

Related Operations

Parent topic: Clusters