Container Service for Kubernetes:Comparison between Koordinator Descheduler and Kubernetes Descheduler

Before you start

This topic is intended for Kubernetes Descheduler users and describes the correlation between Koordinator Descheduler and Kubernetes Descheduler to help you migrate to Koordinator Descheduler. Before you read this topic, we recommend that you learn the features, interface attributes, and policies of Kubernetes Descheduler, such as the system configurations, template configurations, policy plug-in configurations, and evictor plug-in configurations.

Correlation between Koordinator Descheduler and Kubernetes Descheduler

Descheduling Framework was designed before Kubernetes Descheduler 0.25.0 is released. It allows you to manage the lifecycle of descheduling by using plug-ins. Descheduling Framework is developed to meet the following goals:

• Provides plug-ins based on the existing descheduling policies and pod eviction logic to improve the flexibility of descheduling.

• Builds a runtime environment where plug-ins can be loaded and run to manage the lifecycle of descheduling.

After Kubernetes Descheduler 0.26.0 is released, descheduling policies provided by Kubernetes Descheduler and DefaultEvictor are introduced in Koordinator Descheduler v1.2.0.

In addition, Koordinator Descheduler provides an evictor plug-in named MirgrationController.

Feature difference

Koordinator Descheduler applies to most use scenarios of Kubernetes Descheduler. However, the two deschedulers have differences in configuration methods, descheduling policies, evictors, and eviction control capabilities.

1. The development of Descheduler Framework by Koordinator Descheduler is prior to Kubenetes Descheduler. Therefore, Koordinator Descheduler uses a different configuration interface, which is v1alpha2/DeschedulerConfiguration.

2. The lowNodeLoad plug-in is introduced in Koordinator Descheduler. This descheduler plug-in is responsible for sensing the loads of each node. The Kubernetes-native descheduler plug-in LowNodeUtilization deschedules pods from overutilized nodes to underutilized nodes. The LowNodeLoad plug-in deschedules pods based on the actual resource utilization of nodes, which is more accurate. For more information about the LowNodeLoad plug-in, see Work with load-aware hotspot descheduling.

3. The pod evictor MigrationController is introduced in Koordinator Descheduler. MigrationController provides more eviction control capabilities with improved security. For more information, see Configure evictor plug-ins.

Configuration comparison

In the following example, the RemovePodsViolatingNodeTaints policy is enabled to compare the configurations of Kubernetes Descheduler and Koordinator Descheduler. The apiVersion and kind parameters define the version. The following table describes the components supported by different versions.

• Kubernetes Descheduler sample configuration

  • In versions earlier than v0.27.0, the descheduler/v1alpha1/DeschedulerPolicy version is used.

    # Interface attributes. 
    apiVersion: "descheduler/v1alpha1"
    kind: "DeschedulerPolicy"
    
    # System configurations. 
    nodeSelector: "node=node1"
    maxNoOfPodsToEvictPerNode: 10
    maxNoOfPodsToEvictPerNamespace: 10
    
    # Eviction configurations. In the other interfaces, the following parameters are evictor plug-in configurations. 
    evictFailedBarePods: false
    evictLocalStoragePods: true
    evictSystemCriticalPods: true
    evictDaemonSetPods: false
    ignorePvcPods: false
    
    # Descheduling policy configurations. In the other two interfaces, the descheduling policy configurations are replaced by descheduling template configurations, which consist of parameters for enabling/disabling and configuring plug-ins. 
    strategies:
      "RemovePodsViolatingNodeTaints":
        enabled: true
        params:
          nodeFit: true # nodeFit must be enabled in each policy. In the other two interfaces, nodeFit is an evictor plug-in parameter. 
          excludedTaints:
            - deschedule=not-allow # Ignore nodes whose taint key is "deschedule" and taint value is "not-allow".

  • In v0.27.0 and later versions, the descheduler/v1alpha2/DeschedulerPolicy version is used.

    # Interface attributes. 
    apiVersion: "descheduler/v1alpha2"
    kind: "DeschedulerPolicy"
    
    # System configurations. 
    nodeSelector: "node=node1" 
    maxNoOfPodsToEvictPerNode: 10
    maxNoOfPodsToEvictPerNamespace: 10
    # The preceding configurations are system configurations. 
    
    # The template list. 
    profiles:
    - name: kubernetes-descheduler  # The name of the descheduling template. 
        
      # Template configurations.  
      plugins:      
        deschedule:
          enabled:
            - "RemovePodsViolatingNodeTaints" # The enabled parameter is an array. 
        balance:
          disabled: 
            - "*"
      # The preceding configurations are template configurations.          
          
      # The plug-in list. 
      pluginConfig:
        
      # The configurations of the RemovePodsViolatingNodeTaints policy plug-in.     
      - name: RemovePodsViolatingNodeTaints # The configurations of the node taint verification plug-in. 
          args:        
            excludedTaints: 
            - deschedule=not-allow # Ignore nodes whose taint key is "deschedule" and taint value is "not-allow". 
        
      #The configurations of the DefaultEvictor evictor plug-in.   
      - name: "DefaultEvictor"
        args:                
          evictFailedBarePods: false
          evictLocalStoragePods: true
          evictSystemCriticalPods: true
          evictDaemonSetPods: false
          ignorePvcPods: false
          nodeFit: true # nodeFit is an evictor plug-in parameter.                

• Koordinator Descheduler sample configuration

  The descheduler/v1alpha2/DeschedulerConfiguration version is used.

  # Interface attributes. 
  apiVersion: descheduler/v1alpha2
  kind: DeschedulerConfiguration
  
  # System configurations. 
  dryRun: false  
  deschedulingInterval: 120s 
  nodeSelector: 
    node: node1 # The data type of the nodeSelector parameter differs from the other interfaces. 
  maxNoOfPodsToEvictPerNode: 10 
  maxNoOfPodsToEvictPerNamespace: 10 
  # The preceding configurations are system configurations. 
  
  # The template list.             
  profiles:
  - name: koord-descheduler
    
    # Template configurations. 
    plugins:
      deschedule: 
        enabled: 
          - name: RemovePodsViolatingNodeTaints # The enabled parameter is a structured list.           
      balance: 
        disabled: 
          - name: "*" 
      evict: # You can specify the evict parameter to use an evictor.           
        enabled:
          - name: MigrationController # Enable MigrationController by default.  
          # - name: DefaultEvictor # Specify DefaultEvictor.  
      filter: # You can specify the filter parameter to use an eviction filtering policy.  
        enabled:
          - name: MigrationController # Enable MigrationController by default. 
          # - name: DefaultEvictor # Specify DefaultEvictor. 
    # The preceding configurations are template configurations. 
            
    # The plug-in list. 
    pluginConfig:
      
    # The configurations of the RemovePodsViolatingNodeTaints policy plug-in.     
    - name: RemovePodsViolatingNodeTaints # The configurations of the node taint verification plug-in. 
        args:        
          excludedTaints: 
          - deschedule=not-allow # Ignore nodes whose taint key is "deschedule" and taint value is "not-allow". 
      
    #The configurations of the DefaultEvictor evictor plug-in.   
    - name: "DefaultEvictor"
      args:                
        evictFailedBarePods: false
        evictLocalStoragePods: true
        evictSystemCriticalPods: true
        evictDaemonSetPods: false
        ignorePvcPods: false
        nodeFit: true # nodeFit is an evictor plug-in parameter.     

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsViolatingNodeAffinity"
      args:
        nodeAffinityType:
        - "preferredDuringSchedulingIgnoredDuringExecution"
    plugins:
      deschedule:
        enabled:
          - "RemovePodsViolatingNodeAffinity"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsViolatingTopologySpreadConstraint"
      args:
        constraints:
          - DoNotSchedule
        topologyBalanceNodeFit: false  
    plugins:
      balance:
        enabled:
          - "RemovePodsViolatingTopologySpreadConstraint"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsHavingTooManyRestarts"
      args:
        states:
        - "CrashLoopBackOff"
    plugins:
      deschedule:
        enabled:
          - "RemovePodsHavingTooManyRestarts"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "PodLifeTime"
      args:
        states:
        - "CrashLoopBackOff"
        - "ImagePullBackOff"
    plugins:
      deschedule:
        enabled:
          - "PodLifeTime"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemoveFailedPods"
      args:
        exitCodes:
        - 1
    plugins:
      deschedule:
        enabled:
          - "RemoveFailedPods"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "DefaultEvictor"
      args:
        evictDaemonSetPods: false
        minReplicas: 2
    - name: "PodLifeTime"
      args:
        maxPodLifeTimeSeconds: 86400
    plugins:
      deschedule:
        enabled:
          - "PodLifeTime"

Migrate from Kubernetes Descheduler to Koordinator Descheduler

The method used to migrate from Kubernetes Descheduler to Koordinator Descheduler varies based on the version that is used. Perform the following steps to complete the migration.

1. Uninstall Kubernetes Descheduler. If multiple deschedulers exist in the cluster, unpredicable race conditions may exist.

2. Refer to Enable the descheduling feature and install and configure Koordinator Descheduler.

   1. Refer to System configuration difference and configure system configurations for Koordinator Descheduler.

   2. Create a descheduling template that adapts to Koordinator Descheduler.

      • If you use Kubernetes Descheduler 0.26.0 or earlier, only the v1alpha1/DeschedulerPolicy interface is supported. Refer to Template configuration difference and complete the configuration.

      • If you use a version later than Kubernetes Descheduler 0.26.0, you need to confirm the interface that is used.

        • If the interface is v1alpha1/DeschedulerPolicy, refer to Template configuration difference and create a descheduling template that adapts to Koordinator Descheduler.

        • If the interface is v1alpha2/DeschedulerPolicy, refer to Template configuration difference and modify the parameters in Koordinator Descheduler.

   3. Refer to Policy plug-in configuration difference and Evictor plug-in configuration difference and configure plug-in parameters in Koordinator Descheduler.

References

• For more information about how to install the ack-koordinator component and enable descheduling, see Enable the descheduling feature.

  You can also configure advanced parameters for Koordinator Descheduler, descheduling templates, descheduling policy plug-ins, and evictor plug-ins in a ConfigMap. For more information, see Configure advanced parameters.

• For more information about the introduction to ack-koordinator and the release notes, see ack-koordinator (ack-slo-manager).