Enhanced CPU Policy

Kubernetes provides two CPU policies: none and static.

Based on the Kubernetes static core binding policy, the enhanced CPU policy (enhanced-static) supports burstable pods (whose CPU requests and limits must be positive integers) and allows them to preferentially use certain CPUs to ensure application stability. Example:

...
spec:
  containers:
  - name: nginx
    image: nginx
    resources:
      limits:
        memory: "300Mi"
        cpu: "2"
      requests:
        memory: "200Mi"
        cpu: "1"

This feature is built on the optimized CPU scheduling in the HCE OS 2.0 kernel. When the CPU usage preferentially used by a container exceeds 85%, the container is automatically allocated to other CPUs with low usage to ensure the response capability of applications.

  • When enhanced CPU policy is enabled, the application performance is better than that of the none) policy but worse than that of the static policy.
  • CPU would not be exclusively used by burstable pods, it is still in the shared CPU pool. When the burstable pods are in the low tide, other pods can share this CPU.

Constraints

To use this feature, the following conditions must be met:

Procedure

  1. Log in to the CCE console.
  2. Click the cluster name to access the cluster console. Choose Nodes in the navigation pane and click the Node Pools tab on the right.
  3. Select a node pool whose OS is HCE OS 2.0 and click Manage in the Operation column.
  4. In the Manage Components window that is displayed, change the cpu-manager-policy value of the kubelet component to enhanced-static.
  5. Click OK.

Verification

Take a node with 8 vCPUs and 32 GB memory as an example. Deploy a workload whose CPU request is 1 and limit is 2 in the cluster in advance.

  1. Log in to a node in the node pool and view the /var/lib/kubelet/cpu_manager_state output.

    cat /var/lib/kubelet/cpu_manager_state

    Command output:

     {"policyName":"enhanced-static","defaultCpuSet":"0,2-7","entries":{"6739f6f2-ebe5-48ae-945a-986d5d8919b9":{"container-1":"0-7,10001"}},"checksum":1638128523}
    • If the value of policyName is enhanced-static, the policy is configured successfully.
    • 10000 is used as the base for the CPU ID. In this example, 10001 indicates that the affinity CPU ID used by the container is CPU 1, and 0-7 indicates the set of CPUs that can be used by the container in the pod.

  2. Check the cgroup setting of cpuset.preferred_cpus of the container. The output is the ID of the CPU that is preferentially used.

    cat /sys/fs/cgroup/cpuset/kubepods/burstable/pod {pod uid} / {Container ID} /cpuset.preferred_cpus
    • {pod uid} indicates the pod UID, which can be obtained by running the following command on the host that has been connected to the cluster using kubectl:
      kubectl get po {pod name} -n {namespace} -ojsonpath='{.metadata.uid}{"\n"}'

      In the preceding command, {pod name} and {namespace} indicate the pod name and the namespace to which the pod belongs.

    • {Container id} must be a complete container ID. You can run the following command on the node where the container is running to obtain the container ID:
      Docker node pool: In the command, {pod name} indicates the pod name.
      docker ps --no-trunc | grep {pod name} | grep -v cce-pause | awk '{print $1}'

      containerd node pool: In the command, {pod name} indicates the pod name, {pod id} indicates the pod ID, and {container name} indicates the container name.

      # Obtain the pod ID.
crictl pods | grep {pod name} | awk '{print $1}'
# Obtain the complete container ID.
crictl ps --no-trunc | grep {pod id} | grep {container name} | awk '{print $1}'

    A complete example is as follows:

    cat /sys/fs/cgroup/cpuset/kubepods/burstable/pod6739f6f2-ebe5-48ae-945a-986d5d8919b9/5ba5603434b95fd22d36fba6a5f1c44eba83c18c2e1de9b52ac9b52e93547a13/cpuset.preferred_cpus

    If the following command output is displayed, CPU 1 is preferentially used.

    1

Parent topic: CPU Scheduling