Container Service for Kubernetes (ACK) strictly abides by the terms of the Certified Kubernetes Conformance Program. This topic describes the changes that ACK Lingjun has made to support Kubernetes 1.22.
Version updates
All components in ACK Lingjun clusters have been updated and optimized to support Kubernetes 1.22.
Key component | Version | Description |
Kubernetes | 1.22.15-aliyun.1 |
|
etcd | 3.5.1 | None |
CoreDNS | v1.9.3.6-32932850-aliyun | The update does not affect your workloads. The following features are provided:
|
CRI | containerd 1.5.13 | None |
CSI | v1.26.3-fc2ba2a-aliyun | None |
CNI | Terway v1.5.7 | None |
NVIDIA Container Runtime | 3.13.0 | None |
Ingress Controller | v1.8.0-aliyun.1 | The update may temporarily interrupt your workloads and cause compatibility issues with your workload configurations. We recommend that you evaluate the impact of the component update before you update to Kubernetes 1.22. |
Version details
Resource changes and deprecation
[Resource changes] The
admissionregisration.k8s.io/v1beta1API version for the MutatingWebhookConfiguration and ValidatingWebhookConfiguration resources is discontinued. Admission webhook configurations and mutating webhook configurations cannot be created by using this API version, which adversely affects the use of admission webhooks and mutating webhooks. You can use theadmissionregisration.k8s.io/v1API version instead.[Resource changes] The
apiextensions.k8s.io/v1beta1API version for the CustomResourceDefinition (CRD) resource is discontinued. CRDs cannot be created by using this API version, which adversely affects the reconciliation of controllers that use CRDs. You can use theapiextensions.k8s.io/v1API version instead.[Resource changes] The
apiregistration.k8s.io/v1beta1API version for the APIService resource is discontinued. Extended Kubernetes APIs that are managed by using this API version cannot be used. You can use theapiregistration.k8s.io/v1API version instead.[Resource changes] The
authentication.k8s.io/v1beta1API version for the TokenReview resource is discontinued. TokenReviews that are created by using this API version cannot be used for authentication, which adversely affects your applications. You can use theauthentication.k8s.io/v1API version instead.[Resource changes] The
authorization.k8s.io/v1beta1API version for the SubjectAccessReview resource is discontinued. SubjectAccessReviews that are created by using this API version cannot be used for authorization, which adversely affects your applications. You can use theauthorization.k8s.io/v1API version instead.[Resource changes] The
certificate.k8s.io/v1beta1API version for the CertificateSigningRequest (CSR) resource is discontinued. CSRs that are created by using this API version cannot be used in certificate signing and issuing. You can use thecertificates.k8s.io/v1API version instead.[Resource changes] The
coordination.k8s.io/v1beta1API version for the Lease resource is discontinued. Leases that are created by using this API version cannot be used for leader election, which adversely affects your applications. You can use thecoordination.k8s.io/v1API version instead.[Resource changes] The
networking.k8s.io/v1beta1andextensions/v1beta1API versions of the Ingress and IngressClass resources are discontinued. Ingresses that are created by using these API versions cannot be used to expose Services. You can use thenetworking.k8s.io/v1API version instead.[Resource changes] The
rbac.authorization.k8s.io/v1beta1API version for the ClusterRole, ClusterRoleBinding, Role, and RoleBinding resources is discontinued. Role-based access control (RBAC) resources that are managed by using this API version cannot be used to grant the permissions to manage applications and clusters. You can use therbac.authorization.k8s.io/v1API version instead.[Resource changes] The
storage.k8s.io/v1beta1API version for the CSIDriver, CSINode, StorageClass, and VolumeAttachment resources is discontinued. If you use this API version to manage resources that are related to the Container Storage Interface (CSI) plug-in, the CSI plug-in may not run as normal and storage services in your cluster are adversely affected. You can usestorage.k8s.io/v1instead.[Resource changes] The
scheduling.k8s.io/v1beta1API version for the PriorityClass resource is discontinued. PriorityClasses that are managed by using this API version cannot be used to configure pod priorities. You can use thescheduling.k8s.io/v1version instead.Dockershim is deprecated and will be removed in Kubernetes 1.22. For more information, see EP-2221 and cri-containerd.
Before you update to Kubernetes 1.22, we recommend that you perform the following steps to migrate workloads that run in Docker containers to containers that run other container runtimes:
Decide the node specifications and calculate the number of nodes that run container runtimes other than Docker based on the number of existing Docker containers.
Add new nodes to your cluster during off-peak hours.
Drain nodes that run the Docker runtime one after one. Each time a node is drained, verify that application pods on the node are successfully migrated to new nodes before you drain another node.
After all the nodes that run the Docker runtime are drained and no pod runs on the nodes, remove the nodes.
[Resource deprecation] In Kubernetes 1.22.10 and later versions, kube-proxy no longer listens on the ports of NodePort Services. After this update, TCP connections may occasionally fail if the port range of a NodePort Service (specified by the ServiceNodePortRange parameter of the API server) conflicts with the port range specified by the
net.ipv4.ip_local_port_rangekernel parameter. This may lead to health check failures and cause service exceptions on the node. Before you update the Kubernetes version of your cluster to 1.22.10 or later, make sure that the port ranges of all NodePort Services in the cluster do not conflict with the port range specified by thenet.ipv4.ip_local_port_rangekernel parameter. For more information about how to configure the port range of a NodePort Service, see How do I configure the port range of a NodePort Service? or Kubernetes community PR.
Feature enhancements
By default, the ImmutableEphemeralVolumes feature is enabled in Kubernetes 1.21 and later versions. You can use this feature to set ConfigMaps and Secrets as immutable, which significantly reduces the load on the Kubernetes API server of your cluster. For more information, see Secrets and ConfigMaps.
By default, the IPv4/IPv6 dual stack (IPv6DualStack) feature is enabled in Kubernetes 1.21 and later versions. To use IPv4/IPv6 dual stack, you must specify proper IPv4 CIDR blocks and IPv6 CIDR blocks when you create a cluster, and install a Container Network Interface (CNI) plug-in that supports IPv4/IPv6 dual stack. For more information, see IPv4/IPv6 dual stack.
By default, the GracefulNodeShutdown feature is enabled in Kubernetes 1.21 and later versions. This feature supports only Linux nodes. After this feature is enabled, kubelet is aware of node shutdown events that are about to take place and can evict the pods on a node within a specific shutdown period. For more information, see Graceful node shutdown.
By default, the EfficientWatchResumption feature is enabled in Kubernetes 1.21 and later versions. This feature can resume the watch cache of the Kubernetes API server in an efficient manner after the API server is restarted. This feature is suitable for large-scale clusters. For more information, see KEP-1904.
By default, the CSIStorageCapacity feature is enabled in Kubernetes 1.22 and later versions. This feature enables kube-scheduler to schedule a pod to a node whose storage capacity is sufficient for creating the volume that is used by the pod. For more information, see Storage capacity.
By default, the DaemonSetUpdateSurge feature is enabled in Kubernetes 1.22 and later versions. This feature allows you to use the
.spec.strategy.rollingUpdate.maxSurgefield to specify the percentage of pods that can be created above the expected number of pods during a rolling update on a DaemonSet. For more information, see Perform a Rolling Update on a DaemonSet.By default, the IndexedJob feature is enabled in Kubernetes 1.22 and later versions. This feature allows you to create an indexed Job by setting .spec.completionMode to Indexed in the Job configuration. This way, the annotation batch.kubernetes.io/job-completion-index and the JOB_COMPLETION_INDEX environment variable are added to each pod that is created by the Job. For more information, see Kubernetes.
By default, the MemoryManager feature is enabled in Kubernetes 1.22 and later versions. This feature supports only Linux nodes. You can use this feature to enable non-uniform memory access (NUMA)-aware memory management. This feature is suitable for applications that require guaranteed memory resources to significantly improve application performance. ACK does not configure memory reservation for this feature. For more information, see Memory maps at runtime and Utilize the NUMA-aware memory manager.
By default, the PodAffinityNamespaceSelector feature is enabled in Kubernetes 1.22 and later versions. This feature allows you to apply label selectors of pod affinity settings across namespaces instead of within the same namespace. This optimizes affinity-based pod scheduling. For more information, see KEP-2249.
By default, the PodDeletionCost feature is enabled in Kubernetes 1.22 and later versions. After this feature is enabled, pods with lower resource utilization incur lower pod deletion costs. For more information, see ReplicaSet.
By default, the PreferNominatedNode is enabled in Kubernetes 1.22 and later versions. After this feature is enabled, kube-scheduler preferably schedules pods to nominated nodes. kube-scheduler evaluates the other nodes only if all nominated nodes fail to match the pods. For more information, see KEP-1923.
The ProbeTerminationGracePeriod feature is enabled in Kubernetes 1.22 and later versions. This feature supports only liveness probes. This feature allows you to set a probe-level or pod-level teminationGracePeriodSeconds field to shorten the time period that a pod must wait to restart after the pod fails a liveness probe. For more information, see Configure liveness, readiness, and startup probes.
By default, the NetworkPolicyEndPort feature is enabled in Kubernetes 1.22 and later versions. This feature allows you to specify a port range in a NetworkPolicy. For more information, see Network policies.
By default, the LogarithmicScaleDown feature is enabled in Kubernetes 1.22 and later versions. This feature provides a randomized approach to scale in pods and therefore reduces the impact of issues caused by pod topology spread constraints. For more information, see Pod topology spread constraints should be taken into account on scale down and KEP-2185.
By default, the SuspendJob feature is enabled in Kubernetes 1.22 and later versions. This feature allows users to manage the lifecycle of Jobs in a more efficient manner. For example, you can use this feature to suspend and resume Jobs. For more information, see Introduce suspended Jobs.
By default, the ServiceInternalTrafficPolicy feature is enabled in Kubernetes 1.22 and later versions. You can use this feature to route internal traffic to node-local endpoints that are ready or all endpoints that are ready in the cluster. For more information, see Services.
By default, the ServiceLoadBalancerClass feature is enabled in Kubernetes 1.22 and later versions. You can use this feature to customize load balancing. For more information, see Specify the class of load balancer implementation.
By default, the ServiceLBNodePortControl feature is enabled in Kubernetes 1.22 and later versions. This feature allows you to disable node port allocation for a LoadBalancer Service by setting .spec.allocateLoadBalancerNodePorts to false in the Service configuration. This way, the Service routes traffic directly to pods. For more information, see Disable load balancer NodePort allocation.
By default, the SizeMemoryBackedVolumes feature is enabled in Kubernetes 1.22 and later versions. This feature supports only Linux nodes. You can use this feature to specify the size of an emptyDir memory-backed volume by setting the emptyDir.sizeLimit field. This improves the observability of pod scheduling. For more information, see KEP-1967.
By default, the Server-side Apply feature is enabled in Kubernetes 1.22 and later versions. This feature allows you to track changes to the fields of a resource configuration. You can track information about the change, such as the source, time, and operation. For more information, see Server-side apply.
The feature of integrating the CSI plug-in with Windows containers is stabilized in Kubernetes 1.22 and later versions. This feature allows you to use CSI Proxy to perform storage operations on the host whose operating system does not support privileged containers, such as Windows Server 2019 and Windows Server version 2004. To use this feature, make sure that the CSI plug-in that you use supports this feature. For more information, see CSI Proxy.
By default, the CSRDuration feature is enabled in Kubernetes 1.22 and later versions. After this feature is enabled, the validity period of a certificate to be signed and issued is set to the smaller value between the value of .spec.expirationSeconds in the CSR and the value of
--cluster-signing-durationin the kube-controller-manager configuration. In ACK clusters, the default value of --cluster-signing-duration in the kube-controller-manager configuration is 10 years. For more information, see Signers.In Kubernetes 1.22 and later, the BoundServiceAccountTokenVolume feature gate reaches General Availability (GA). When this feature gate is enabled, the default validity period of service account tokens that are not mounted as projected volumes to pods is one year. For more information, see Feature details.
New features
The volume health monitoring feature is supported in Kubernetes 1.21 and later versions. This feature helps detect the health status of persistent volumes (PVs) that are provisioned by using the CSI plug-in. This prevents data from being read from or written to unhealthy PVs. By default, this feature is enabled for ACK clusters that use the CSI plug-in. To use this feature, make sure that the CSI plug-in that you use supports this feature. For more information, see Volume health monitoring.
The memory Quality of Service (QoS) feature that is developed based on cgroups v2 is supported in Kubernetes 1.22 and later versions. In scenarios where computing resources are insufficient, such as scenarios where resource request spikes occur, CPU throttling is performed to ensure the availability of CPU resources. However, memory throttling is not supported. To support memory throttling, open source Linux kernel optimizes specific interfaces in cgroups v2. By default, the memory QoS feature is enabled for ACK clusters. This feature supports only Linux nodes. To use this feature, make sure that the OS kernels of the Linux nodes that you use support this feature. For more information, see Memcg QoS feature of the cgroup v1 interface and 2570-memory-qos.
Windows privileged containers can be created from HostProcess containers in Kubernetes 1.22 and later versions. By default, the Windows HostProcess container feature is enabled for ACK clusters. To use this feature, make sure that the OS kernels of the nodes that you use support this feature. For more information, see What's new for Windows containers on Windows Server 2022 and Create a Windows HostProcess Pod.
The swap memory feature is supported for workloads in Kubernetes 1.22 and later versions. This feature supports only Linux nodes. For scenarios in which the swap memory feature is required, you can use the swap memory feature to improve the performance of your application. For example, a node administrator wants to improve node performance or reduce stability issues that are caused by memory contention. The swap memory feature is disabled for ACK clusters. For more information, see Swap memory management and KEP-2400.
Default seccomp profiles are configured for workloads in Kubernetes 1.22 and later versions. This feature supports only Linux nodes. After this feature is enabled, the RuntimeDefault seccomp profile is used by default. Specific workloads may require fewer limits on system calls than other workloads. These workloads may fail after this feature is enabled. This feature is disabled for ACK clusters. For more information, see Enable the use of RuntimeDefault as the default seccomp profile for all workloads.
Feature updates
The PSP resource was deprecated in Kubernetes 1.21 and later versions, and will be removed in Kubernetes 1.25. By default, the pod security policy feature is enabled for ACK clusters. You can use ACK pod security policies as an alternative to the PSP resource in Kubernetes 1.22. For more information, see Pod security admission and PodSecurityPolicy deprecation: past, present, and future.
The topologyKeys field was deprecated in Kubernetes 1.21 and later versions. Instead, the Topology Aware Hints feature is used to enable the Service topology feature. By default, the Service topology feature is disabled for ACK clusters. If the Service topology feature is enabled for a cluster of Kubernetes 1.22, you can enable the Topology Aware Hints feature to achieve the same effect as the topologyKeys field. For more information, see Topology Aware Hints.
Enhancements to Kubernetes 1.22
Observability
More metrics about the access and requests to the Kubernetes API server are added. This improves the observability of the Kubernetes API server.
Key metrics of control plane components can be collected for ACK Lingjun clusters. This improves the observability of control plane components.
Stability
The following enhancements are provided for all types of ACK clusters:
Protection for storage resources is improved to reduce the load on etcd during cold starts.
Traffic throttling can be performed on the Kubernetes API server based on the combination of the sources, types, and routes of requests. This reduces the load on etcd during cold starts.
Performance improvements
kubelet: During the in-place upgrade of kubelet, the system prevents pod restarts with the best effort. For more information, see kubelet's calculation of whether a container has changed can cause cluster-wide outages.
kube-proxy: kube-proxy is compatible with Alibaba Cloud Linux 2 (kernel-4.19.91-23) and later versions. When the IP Virtual Server (IPVS) mode is enabled, conn_reuse_mode is not set to 0. For more information, see [ipvs] Set conn_reuse_mode=1 on Linux kernel version >= v5.9.
Fixed issues
The issue of EndpointSlice leakage in specific scenarios is fixed for kube-controller-manager. For more information, see Fixing how EndpointSlice Mirroring handles Service selector transitions.