如果您希望根據Pod實際的資源使用方式,動態調整和最佳化資源配置,以確保Pod能夠調度到資源充足的節點上,建議您為業務容器開啟垂直伸縮VPA(Vertical Pod Autoscaler)功能。VPA能夠監控Pod的資源消耗模式,靈活推薦CPU和記憶體資源分派的配置,並在適當的情況下自動進行調整,而不調整Pod的副本數量。這種能力更適用於需要穩定資源配置的有狀態應用的擴容等情境。
閱讀前提示
為了協助您更好地使用VPA功能,建議您在閱讀本文前參見Kubernetes社區VPA介紹瞭解VPA的基本資料、使用流程、已知限制等。
此外,ACK叢集提供了多種工作負載伸縮(調度層彈性)和節點伸縮(資源層彈性)方案,建議您在使用本文檔前閱讀Auto Scaling概述,瞭解不同方案的適用情境、使用限制等。
ack-vertical-pod-autoscaler組件介紹
VPA基於ack-vertical-pod-autoscaler組件實現,以根據實際資源需求推薦並自動調整Pod中容器的CPU及記憶體的Request和Limit參數。同時,VPA也會遵循容器初始配置中定義的資源Request和Limit佔比。ack-vertical-pod-autoscaler組件有以下組成部分:
Admission Controller組件:為建立Pod設定正確的資源請求值Requests。安裝admission-controller組件前,您需要用此指令碼為Webhook產生認證。
Recommender組件:監控容器目前和過去的資源使用方式,並基於檢測資料推薦資源配置,以更準確地匹配資源使用需求。
Updater組件:檢查VPA管理的Pod資源量配置是否正確。如果不正確,Updater會終止這些Pod,以便後續重建。
注意事項
VPA功能目前處於Beta階段,其效能未在大型叢集中測試,請謹慎使用。如遇相關問題或有相關產品建議,請
提交工單聯絡Container Service團隊。更新正在啟動並執行Pod資源配置會導致Pod的重建和重啟,且有可能會被調度到其他節點上。VPA動態更新現有Pod的資源Request時,支援無需重啟的更新機制,但此機制仍處於測試階段。
VPA不會驅逐沒有在副本控制器(Replication Controller)管理下的Pod。在VPA中,此類Pod的Auto模式等同於Initial模式,即VPA會為新的Pod設定資源請求和限制,但不會自動刪除和重新建立那些資源分派不當的運行中的Pod。
不推薦您為同一個工作負載部署多個VPA。多個VPA同時匹配同一個Pod時,可能會造成不可預測的行為。
VPA對Pod資源Request的修改值可能超過實際的資源上限,例如節點資源上限、空閑資源或資源配額,從而造成Pod處於Pending狀態無法被調度。此時,使用節點自動調整可能可以解決這個問題:當Pod的資源使用率超出擴容閾值後,Pod將擴容,但此時節點資源不足,導致Pod調度失敗並處於Pending狀態,繼而觸發節點擴容。更多資訊,請參見啟用節點自動調整。
目前,VPA和HPA的Controller無法完全相容。如果您同時使用VPA和HPA來監控CPU和記憶體使用量情況,這可能會產生衝突。如果您的HPA僅監控其他定製化的或者外部的資源使用方式,可以避免此衝突。
VPA使用Admission Webhook作為其准入控制器。請確保叢集中存在的其他Admission Webhook不會與VPA發生衝突。
說明准入控制器的執行順序定義可在API Server的配置參數中尋找。
VPA會處理出現的絕大多數OOM(Out Of Memory)的事件,但無法保證所有的情境下都有效。
關於VPA更多的已知限制,請參見VPA已知限制。
步驟一:安裝ack-vertical-pod-autoscaler組件
您可以通過控制台(v1.26及以上叢集)或kubectl(v1.26以下叢集)的方式安裝ack-vertical-pod-autoscaler組件。
通過控制台安裝(推薦)
推薦您通過Container Service管理主控台安裝ack-vertical-pod-autoscaler組件,屏蔽組件底層複雜性,使用門檻和維護成本更低。
前提條件
已建立v1.26及以上的ACK託管叢集。具體操作,請參見建立Kubernetes託管版叢集。如需升級叢集,請參見手動升級叢集。
已使用命令列工具串連叢集,請參見擷取叢集KubeConfig並通過kubectl工具串連叢集。
如果您之前通過kubectl的方式安裝了VPA,建議您卸載該VPA,通過控制台重新安裝。具體操作,請參見下文如何通過控制台管理使用kubectl安裝的VPA?。
安裝步驟
登入Container Service管理主控台,在左側導覽列選擇叢集。
在叢集列表頁面,單擊目的地組群名稱,然後在左側導覽列,選擇營運管理 > 組件管理。
在組件管理頁面,定位ack-vertical-pod-autoscaler組件,然後按照頁面提示完成安裝。
通過kubectl安裝
前提條件
已建立v1.26以下的ACK託管叢集。具體操作,請參見建立Kubernetes託管版叢集。
已使用命令列工具串連叢集,請參見擷取叢集KubeConfig並通過kubectl工具串連叢集。
如果您已在叢集中部署VPA,請卸載該VPA,以避免新安裝的VPA與舊版VPA衝突。
操作步驟
儲存RBAC許可權的YAML檔案,並執行命令,建立RBAC許可權檔案。
展開查看RBAC YAML檔案
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-reader rules: - apiGroups: - "metrics.k8s.io" resources: - pods verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-actor rules: - apiGroups: - "" resources: - pods - nodes - limitranges verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - get - list - watch - create - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-status-actor rules: - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers/status verbs: - get - patch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-checkpoint-actor rules: - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalercheckpoints verbs: - get - list - watch - create - patch - delete - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalercheckpoints verbs: - get - list - watch - create - patch - delete - apiGroups: - "" resources: - namespaces verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:evictioner rules: - apiGroups: - "apps" - "extensions" resources: - replicasets verbs: - get - apiGroups: - "" resources: - pods/eviction verbs: - create --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-reader roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-reader subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-status-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-status-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-checkpoint-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-checkpoint-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-target-reader rules: - apiGroups: - '*' resources: - '*/scale' verbs: - get - watch - apiGroups: - "" resources: - replicationcontrollers verbs: - get - list - watch - apiGroups: - apps resources: - daemonsets - deployments - replicasets - statefulsets verbs: - get - list - watch - apiGroups: - batch resources: - jobs - cronjobs verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-target-reader-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-target-reader subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system - kind: ServiceAccount name: vpa-admission-controller namespace: kube-system - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-evictioner-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:evictioner subjects: - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-admission-controller namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-recommender namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-admission-controller rules: - apiGroups: - "" resources: - pods - configmaps - nodes - limitranges verbs: - get - list - watch - apiGroups: - "admissionregistration.k8s.io" resources: - mutatingwebhookconfigurations verbs: - create - delete - get - list - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "coordination.k8s.io" resources: - leases verbs: - create - update - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-admission-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-admission-controller subjects: - kind: ServiceAccount name: vpa-admission-controller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-status-reader rules: - apiGroups: - "coordination.k8s.io" resources: - leases verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-status-reader-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-status-reader subjects: - kind: ServiceAccount name: vpa-updater namespace: kube-system
kubectl apply -f rbac.yaml
儲存CRD的YAML檔案,並執行命令,建立ack-vertical-pod-autoscaler組件的CRD。
說明CRD可以提高Kubernetes的擴充能力,詳情請參見Extend the Kubernetes API with CustomResourceDefinitions。
CRD YAML定義如下:
1.22≤ 叢集版本<1.26
展開查看CRD YAML
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 controller-gen.kubebuilder.io/version: v0.9.2 creationTimestamp: null name: verticalpodautoscalercheckpoints.autoscaling.k8s.io spec: group: autoscaling.k8s.io names: kind: VerticalPodAutoscalerCheckpoint listKind: VerticalPodAutoscalerCheckpointList plural: verticalpodautoscalercheckpoints shortNames: - vpacheckpoint singular: verticalpodautoscalercheckpoint scope: Namespaced versions: - name: v1 schema: openAPIV3Schema: description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal state of VPA that is used for recovery after recommender's restart. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: containerName: description: Name of the checkpointed container. type: string vpaObjectName: description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint object. type: string type: object status: description: Data of the checkpoint. properties: cpuHistogram: description: Checkpoint of histogram for consumption of CPU. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object firstSampleStart: description: Timestamp of the fist sample from the histograms. format: date-time nullable: true type: string lastSampleStart: description: Timestamp of the last sample from the histograms. format: date-time nullable: true type: string lastUpdateTime: description: The time when the status was last refreshed. format: date-time nullable: true type: string memoryHistogram: description: Checkpoint of histogram for consumption of memory. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object totalSamplesCount: description: Total number of samples in the histograms. type: integer version: description: Version of the format of the stored data. type: string type: object type: object served: true storage: true - name: v1beta2 schema: openAPIV3Schema: description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal state of VPA that is used for recovery after recommender's restart. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: containerName: description: Name of the checkpointed container. type: string vpaObjectName: description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint object. type: string type: object status: description: Data of the checkpoint. properties: cpuHistogram: description: Checkpoint of histogram for consumption of CPU. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object firstSampleStart: description: Timestamp of the fist sample from the histograms. format: date-time nullable: true type: string lastSampleStart: description: Timestamp of the last sample from the histograms. format: date-time nullable: true type: string lastUpdateTime: description: The time when the status was last refreshed. format: date-time nullable: true type: string memoryHistogram: description: Checkpoint of histogram for consumption of memory. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object totalSamplesCount: description: Total number of samples in the histograms. type: integer version: description: Version of the format of the stored data. type: string type: object type: object served: true storage: false --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 controller-gen.kubebuilder.io/version: v0.9.2 creationTimestamp: null name: verticalpodautoscalers.autoscaling.k8s.io spec: group: autoscaling.k8s.io names: kind: VerticalPodAutoscaler listKind: VerticalPodAutoscalerList plural: verticalpodautoscalers shortNames: - vpa singular: verticalpodautoscaler scope: Namespaced versions: - additionalPrinterColumns: - jsonPath: .spec.updatePolicy.updateMode name: Mode type: string - jsonPath: .status.recommendation.containerRecommendations[0].target.cpu name: CPU type: string - jsonPath: .status.recommendation.containerRecommendations[0].target.memory name: Mem type: string - jsonPath: .status.conditions[?(@.type=='RecommendationProvided')].status name: Provided type: string - jsonPath: .metadata.creationTimestamp name: Age type: date name: v1 schema: openAPIV3Schema: description: VerticalPodAutoscaler is the configuration for a vertical pod autoscaler, which automatically manages pod resources based on historical and real time resource utilization. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the behavior of the autoscaler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: recommenders: description: Recommender responsible for generating recommendation for this object. List should be empty (then the default recommender will generate the recommendation) or contain exactly one recommender. items: description: VerticalPodAutoscalerRecommenderSelector points to a specific Vertical Pod Autoscaler recommender. In the future it might pass parameters to the recommender. properties: name: description: Name of the recommender responsible for generating recommendation for this object. type: string required: - name type: object type: array resourcePolicy: description: Controls how the autoscaler computes recommended resources. The resource policy may be used to set constraints on the recommendations for individual containers. If not specified, the autoscaler computes recommended resources for all containers in the pod, without additional constraints. properties: containerPolicies: description: Per-container resource policies. items: description: ContainerResourcePolicy controls how autoscaler computes the recommended resources for a specific container. properties: containerName: description: Name of the container or DefaultContainerResourcePolicy, in which case the policy is used by the containers that don't have their own policy specified. type: string controlledResources: description: Specifies the type of recommendations that will be computed (and possibly applied) by VPA. If not specified, the default of [ResourceCPU, ResourceMemory] will be used. items: description: ResourceName is the name identifying various resources in a ResourceList. type: string type: array controlledValues: description: Specifies which resource values should be controlled. The default is "RequestsAndLimits". enum: - RequestsAndLimits - RequestsOnly type: string maxAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the maximum amount of resources that will be recommended for the container. The default is no maximum. type: object minAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the minimal amount of resources that will be recommended for the container. The default is no minimum. type: object mode: description: Whether autoscaler is enabled for the container. The default is "Auto". enum: - Auto - "Off" type: string type: object type: array type: object targetRef: description: TargetRef points to the controller managing the set of pods for the autoscaler to control - e.g. Deployment, StatefulSet. VerticalPodAutoscaler can be targeted at controller implementing scale subresource (the pod set is retrieved from the controller's ScaleStatus) or some well known controllers (e.g. for DaemonSet the pod set is read from the controller's spec). If VerticalPodAutoscaler cannot use specified target it will report ConfigUnsupported condition. Note that VerticalPodAutoscaler does not require full implementation of scale subresource - it will not use it to modify the replica count. The only thing retrieved is a label selector matching pods grouped by the target resource. properties: apiVersion: description: API version of the referent type: string kind: description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' type: string name: description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' type: string required: - kind - name type: object x-kubernetes-map-type: atomic updatePolicy: description: Describes the rules on how changes are applied to the pods. If not specified, all fields in the `PodUpdatePolicy` are set to their default values. properties: minReplicas: description: Minimal number of replicas which need to be alive for Updater to attempt pod eviction (pending other checks like PDB). Only positive values are allowed. Overrides global '--min-replicas' flag. format: int32 type: integer updateMode: description: Controls when autoscaler applies changes to the pod resources. The default is 'Auto'. enum: - "Off" - Initial - Recreate - Auto type: string type: object required: - targetRef type: object status: description: Current information about the autoscaler. properties: conditions: description: Conditions is the set of conditions required for this autoscaler to scale its target, and indicates whether or not those conditions are met. items: description: VerticalPodAutoscalerCondition describes the state of a VerticalPodAutoscaler at a certain point. properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another format: date-time type: string message: description: message is a human-readable explanation containing details about the transition type: string reason: description: reason is the reason for the condition's last transition. type: string status: description: status is the status of the condition (True, False, Unknown) type: string type: description: type describes the current condition type: string required: - status - type type: object type: array recommendation: description: The most recently computed amount of resources recommended by the autoscaler for the controlled pods. properties: containerRecommendations: description: Resources recommended by the autoscaler for each container. items: description: RecommendedContainerResources is the recommendation of resources computed by autoscaler for a specific container. Respects the container resource policy if present in the spec. In particular the recommendation is not produced for containers with `ContainerScalingMode` set to 'Off'. properties: containerName: description: Name of the container. type: string lowerBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Minimum recommended amount of resources. Observes ContainerResourcePolicy. This amount is not guaranteed to be sufficient for the application to operate in a stable way, however running with less resources is likely to have significant impact on performance/availability. type: object target: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Recommended amount of resources. Observes ContainerResourcePolicy. type: object uncappedTarget: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: The most recent recommended resources target computed by the autoscaler for the controlled pods, based only on actual resource usage, not taking into account the ContainerResourcePolicy. May differ from the Recommendation if the actual resource usage causes the target to violate the ContainerResourcePolicy (lower than MinAllowed or higher that MaxAllowed). Used only as status indication, will not affect actual resource assignment. type: object upperBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Maximum recommended amount of resources. Observes ContainerResourcePolicy. Any resources allocated beyond this value are likely wasted. This value may be larger than the maximum amount of application is actually capable of consuming. type: object required: - target type: object type: array type: object type: object required: - spec type: object served: true storage: true subresources: {} - deprecated: true deprecationWarning: autoscaling.k8s.io/v1beta2 API is deprecated name: v1beta2 schema: openAPIV3Schema: description: VerticalPodAutoscaler is the configuration for a vertical pod autoscaler, which automatically manages pod resources based on historical and real time resource utilization. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the behavior of the autoscaler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: resourcePolicy: description: Controls how the autoscaler computes recommended resources. The resource policy may be used to set constraints on the recommendations for individual containers. If not specified, the autoscaler computes recommended resources for all containers in the pod, without additional constraints. properties: containerPolicies: description: Per-container resource policies. items: description: ContainerResourcePolicy controls how autoscaler computes the recommended resources for a specific container. properties: containerName: description: Name of the container or DefaultContainerResourcePolicy, in which case the policy is used by the containers that don't have their own policy specified. type: string maxAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the maximum amount of resources that will be recommended for the container. The default is no maximum. type: object minAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the minimal amount of resources that will be recommended for the container. The default is no minimum. type: object mode: description: Whether autoscaler is enabled for the container. The default is "Auto". enum: - Auto - "Off" type: string type: object type: array type: object targetRef: description: TargetRef points to the controller managing the set of pods for the autoscaler to control - e.g. Deployment, StatefulSet. VerticalPodAutoscaler can be targeted at controller implementing scale subresource (the pod set is retrieved from the controller's ScaleStatus) or some well known controllers (e.g. for DaemonSet the pod set is read from the controller's spec). If VerticalPodAutoscaler cannot use specified target it will report ConfigUnsupported condition. Note that VerticalPodAutoscaler does not require full implementation of scale subresource - it will not use it to modify the replica count. The only thing retrieved is a label selector matching pods grouped by the target resource. properties: apiVersion: description: API version of the referent type: string kind: description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' type: string name: description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' type: string required: - kind - name type: object x-kubernetes-map-type: atomic updatePolicy: description: Describes the rules on how changes are applied to the pods. If not specified, all fields in the `PodUpdatePolicy` are set to their default values. properties: updateMode: description: Controls when autoscaler applies changes to the pod resources. The default is 'Auto'. enum: - "Off" - Initial - Recreate - Auto type: string type: object required: - targetRef type: object status: description: Current information about the autoscaler. properties: conditions: description: Conditions is the set of conditions required for this autoscaler to scale its target, and indicates whether or not those conditions are met. items: description: VerticalPodAutoscalerCondition describes the state of a VerticalPodAutoscaler at a certain point. properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another format: date-time type: string message: description: message is a human-readable explanation containing details about the transition type: string reason: description: reason is the reason for the condition's last transition. type: string status: description: status is the status of the condition (True, False, Unknown) type: string type: description: type describes the current condition type: string required: - status - type type: object type: array recommendation: description: The most recently computed amount of resources recommended by the autoscaler for the controlled pods. properties: containerRecommendations: description: Resources recommended by the autoscaler for each container. items: description: RecommendedContainerResources is the recommendation of resources computed by autoscaler for a specific container. Respects the container resource policy if present in the spec. In particular the recommendation is not produced for containers with `ContainerScalingMode` set to 'Off'. properties: containerName: description: Name of the container. type: string lowerBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Minimum recommended amount of resources. Observes ContainerResourcePolicy. This amount is not guaranteed to be sufficient for the application to operate in a stable way, however running with less resources is likely to have significant impact on performance/availability. type: object target: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Recommended amount of resources. Observes ContainerResourcePolicy. type: object uncappedTarget: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: The most recent recommended resources target computed by the autoscaler for the controlled pods, based only on actual resource usage, not taking into account the ContainerResourcePolicy. May differ from the Recommendation if the actual resource usage causes the target to violate the ContainerResourcePolicy (lower than MinAllowed or higher that MaxAllowed). Used only as status indication, will not affect actual resource assignment. type: object upperBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Maximum recommended amount of resources. Observes ContainerResourcePolicy. Any resources allocated beyond this value are likely wasted. This value may be larger than the maximum amount of application is actually capable of consuming. type: object required: - target type: object type: array type: object type: object required: - spec type: object served: true storage: false
叢集版本<1.22
展開查看CRD YAML
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: verticalpodautoscalers.autoscaling.k8s.io annotations: "api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/63797" spec: group: autoscaling.k8s.io scope: Namespaced names: plural: verticalpodautoscalers singular: verticalpodautoscaler kind: VerticalPodAutoscaler shortNames: - vpa version: v1beta1 versions: - name: v1beta1 served: false storage: false - name: v1beta2 served: true storage: true - name: v1 served: true storage: false validation: # openAPIV3Schema is the schema for validating custom objects. openAPIV3Schema: type: object properties: spec: type: object required: [] properties: targetRef: type: object updatePolicy: type: object properties: updateMode: type: string resourcePolicy: type: object properties: containerPolicies: type: array items: type: object properties: containerName: type: string controlledValues: type: string enum: ["RequestsAndLimits", "RequestsOnly"] mode: type: string enum: ["Auto", "Off"] minAllowed: type: object maxAllowed: type: object controlledResources: type: array items: type: string enum: ["cpu", "memory"] --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: verticalpodautoscalercheckpoints.autoscaling.k8s.io annotations: "api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/63797" spec: group: autoscaling.k8s.io scope: Namespaced names: plural: verticalpodautoscalercheckpoints singular: verticalpodautoscalercheckpoint kind: VerticalPodAutoscalerCheckpoint shortNames: - vpacheckpoint version: v1beta1 versions: - name: v1beta1 served: false storage: false - name: v1beta2 served: true storage: true - name: v1 served: true storage: false
kubectl apply -f crd.yaml
安裝ack-vertical-pod-autoscaler組件的Admission Controller、Recommender和Updater組件。
1.22≤ 叢集版本<1.26
Admission-controller組件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-admission-controller namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-admission-controller template: metadata: labels: app: vpa-admission-controller spec: serviceAccountName: vpa-admission-controller securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: admission-controller image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-admission-controller:0.13.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: tls-certs mountPath: "/etc/tls-certs" readOnly: true resources: limits: cpu: 200m memory: 500Mi requests: cpu: 50m memory: 200Mi ports: - containerPort: 8000 - name: prometheus containerPort: 8944 volumes: - name: tls-certs secret: secretName: vpa-tls-certs --- apiVersion: v1 kind: Service metadata: name: vpa-webhook namespace: kube-system spec: ports: - port: 443 targetPort: 8000 selector: app: vpa-admission-controller
Recommender組件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-recommender namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-recommender template: metadata: labels: app: vpa-recommender spec: serviceAccountName: vpa-recommender securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: recommender image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-recommender:0.13.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - name: prometheus containerPort: 8942
Updater組件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-updater namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-updater template: metadata: labels: app: vpa-updater spec: serviceAccountName: vpa-updater securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: updater image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-updater:0.13.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - name: prometheus containerPort: 8943
叢集版本<1.22
Admission-controller組件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-admission-controller namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-admission-controller template: metadata: labels: app: vpa-admission-controller spec: serviceAccountName: admin containers: - name: admission-controller image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-admission-controller:0.7.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: tls-certs mountPath: "/etc/tls-certs" readOnly: true resources: limits: cpu: 200m memory: 500Mi requests: cpu: 50m memory: 200Mi ports: - containerPort: 8000 volumes: - name: tls-certs secret: secretName: vpa-tls-certs --- apiVersion: v1 kind: Service metadata: name: vpa-webhook namespace: kube-system spec: ports: - port: 443 targetPort: 8000 selector: app: vpa-admission-controller
Recommender組件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-recommender namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-recommender template: metadata: labels: app: vpa-recommender spec: serviceAccountName: admin containers: - name: recommender image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-recommender:0.7.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - containerPort: 8080
Updater組件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-updater namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-updater template: metadata: labels: app: vpa-updater spec: serviceAccountName: admin containers: - name: updater image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-updater:0.7.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - containerPort: 8080
步驟二:驗證安裝VPA
儲存以下YAML檔案,執行命令建立名為nginx-deployment-basic的Deployment。
展開查看nginx-deployment-basic.yaml檔案
說明將Deployment中的資源
requests
和limits
留空。apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment-basic labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
kubectl apply -f nginx-deployment-basic.yaml
儲存以下YAML檔案,執行命令建立名為nginx-deployment-basic-vpa的VPA資源。
展開查看nginx-deployment-basic-vpa.yaml檔案
說明您可以按需設定
updateMode
為Off
或者Auto
。Off
(推薦):根據叢集資源消耗推薦Pod的資源Request和Limit配置,但不會自動更新Pod資源配置。Auto
:根據叢集資源消耗推薦Pod的資源Request和Limit配置,並自動更新Pod資源配置。
apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: nginx-deployment-basic-vpa spec: targetRef: apiVersion: "apps/v1" kind: Deployment name: nginx-deployment-basic updatePolicy: updateMode: "Off" # 設定為off。
kubectl apply -f nginx-deployment-basic-vpa.yaml
執行以下命令,查詢VPA為Deployment推薦的CPU和記憶體資源的
requests
值。kubectl describe vpa nginx-deployment-basic-vpa
預期輸出需要等待兩分鐘左右。正常情況下,預期輸出如下,展示了VPA為Deployment推薦的值。
展開查看預期輸出
Recommendation: Container Recommendations: Container Name: nginx Lower Bound: Cpu: 25m Memory: 262144k Target: Cpu: 25m Memory: 262144k Uncapped Target: Cpu: 25m Memory: 262144k Upper Bound: Cpu: 11601m Memory: 12128573170
您可以根據VPA的推薦值來配置Deployment中實際資源的
requests
。VPA會持續監控應用資源的使用方式,並提供最佳化建議。
如何通過控制台管理使用kubectl安裝的VPA?
對於v1.26及以上的叢集,推薦您使用控制台進行ack-vertical-pod-autoscaler組件的安裝和VPA的管理。為避免新舊VPA衝突,請卸載舊的VPA(通過kubectl安裝的VPA),並通過控制台重新安裝。
展開查看完整的操作步驟
步驟1:卸載叢集中已經部署的VPA
卸載叢集中已經部署的VPA,包括清理Deployment、RBAC、Secret、CRD、Service等資源,以避免新安裝的VPA與舊版VPA衝突而導致VPA不可用。
在舊版VPA的YAML檔案中(本小節以下方樣本nginx-deployment-basic-vpa.yaml為例),刪除
metadata
中的資訊,僅保留name
和namespace
欄位即可,同時刪除status
欄位資訊。然後,儲存此YAML檔案供後續步驟使用。apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"autoscaling.k8s.io/v1","kind":"VerticalPodAutoscaler","metadata":{"annotations":{},"name":"nginx-deployment-basic-vpa","namespace":"default"},"spec":{"targetRef":{"apiVersion":"apps/v1","kind":"Deployment","name":"nginx-deployment-basic"},"updatePolicy":{"updateMode":"Off"}}} creationTimestamp: "2024-02-29T06:03:35Z" generation: 1 name: nginx-deployment-basic-vpa namespace: default resourceVersion: "56264" uid: 9f128737-d12e-46f6-b254-c1a7505c19c6 spec: targetRef: apiVersion: apps/v1 kind: Deployment name: nginx-deployment-basic updatePolicy: updateMode: "Off" status: conditions: - lastTransitionTime: "2024-02-29T06:03:55Z" status: "True" type: RecommendationProvided recommendation: containerRecommendations: - containerName: nginx lowerBound: cpu: 25m memory: 262144k target: cpu: 25m memory: 262144k uncappedTarget: cpu: 25m memory: 262144k upperBound: cpu: 25m memory: 262144k
kubectl get vpa nginx-deployment-basic-vpa -oyaml
執行以下命令,刪除VPA組件安裝的資源。
// 刪除Deployment和Service kubectl delete deployment vpa-admission-controller vpa-recommender vpa-updater -n kube-system kubectl delete svc vpa-webhook -n kube-system // 刪除ClusterRole kubectl delete clusterrole system:metrics-reader system:vpa-actor system:vpa-status-actor system:vpa-checkpoint-actor system:evictioner system:vpa-target-reader system:vpa-admission-controller system:vpa-status-reader // 刪除ClusterRoleBinding kubectl delete clusterrolebinding system:metrics-reader system:vpa-actor system:vpa-status-actor system:vpa-checkpoint-actor system:vpa-target-reader-binding system:vpa-evictioner-binding system:vpa-admission-controller system:vpa-status-reader-binding // 刪除ServiceAccount kubectl delete sa vpa-admission-controller vpa-recommender vpa-updater -n kube-system // 刪除Secret kubectl delete secret vpa-tls-certs -n kube-system //刪除CRD kubectl delete crd verticalpodautoscalercheckpoints.autoscaling.k8s.io verticalpodautoscalers.autoscaling.k8s.io
步驟2:安裝ack-vertical-pod-autoscaler組件
登入Container Service管理主控台,在左側導覽列選擇叢集。
在叢集列表頁面,單擊目的地組群名稱,然後在左側導覽列,選擇營運管理 > 組件管理。
在組件管理頁面,定位ack-vertical-pod-autoscaler組件,然後按照頁面提示完成安裝。
步驟3:重新部署VPA YAML檔案
執行以下命令,重新部署步驟1:卸載叢集中已經部署的VPA儲存的VPA YAML檔案,本步驟以nginx-deployment-basic-vpa.yaml為例。
kubectl apply -f nginx-deployment-basic-vpa.yaml
相關文檔
如果您需要基於CPU使用率、記憶體使用量率或其他自訂指標實現Pod的擴縮容,請參見使用容器水平伸縮(HPA)。
如果您的應用資源使用率存在周期性變化,需要按照類似Crontab的策略定時對Pod進行擴縮容,請參見使用容器定時水平伸縮(CronHPA)。
如果您的應用資源使用率存在周期性變化,但難以通過規則定義,您可以選擇AHPA,以根據業務歷史指標自動識別業務水位周期,進行Pod擴縮容,請參見AHPA概述。
如需基於訊息佇列、定時策略、自訂指標等Kubernetes事件靈活自訂擴縮容策略,對Pod進行擴縮容,請參見使用事件驅動伸縮(KEDA)。