實現應用Pod與快取資料之間的親和性調度 - Container Service for Kubernetes

通過Fluid提供的資料緩衝親和性調度最佳化能力，您可以設定應用Pod上的親和性配置，讓應用Pod優先訪問同節點、同可用性區域節點或同地區節點的快取資料，從而提高應用Pod訪問資料的效率。

使用限制

資料緩衝親和性調度最佳化功能僅支援在ACK叢集Pro版中使用。
資料緩衝親和性調度最佳化功能暫不支援與使用ECI彈性調度或自訂彈性資源優先順序調度共同使用。

前提條件

已建立ACK叢集Pro版，且叢集版本為1.18及以上。具體操作，請參見建立ACK Pro版叢集。
已安裝雲原生AI套件並部署ack-fluid組件，且ack-fluid版本為1.0.6及以上。具體操作，請參見安裝雲原生AI套件。
重要
若您已安裝開源Fluid，請卸載後再部署ack-fluid組件。
已通過kubectl串連Kubernetes叢集。具體操作，請參見擷取叢集KubeConfig並通過kubectl工具串連叢集。

功能介紹

基於Mutating Webhook機制，Fluid可以為應用Pod注入所需資料緩衝的親和性資訊。當Kubernetes調度器調度應用Pod時，可以優先選擇有快取資料的節點，或與快取資料位於相同可用性區域或地區的其他節點，自動實現應用Pod與資料緩衝之間的分層親和性調度。

重要

如果應用Pod在spec.affinity或spec.nodeSelector中自訂了與分層拓撲資訊相關的親和性資訊，此時以應用Pod自身配置為準，Fluid不會注入相關的親和性調度配置資訊。

調度策略配置

預設配置

Fluid預設提供了節點、可用性區域和地區三個層級的調度策略標識。您可以通過以下命令查看當前叢集中的調度策略配置。

kubectl get cm -n fluid-system webhook-plugins -oyaml

預期輸出結果：

apiVersion: v1
data:
  pluginsProfile: |
    pluginConfig:
    - args: |
        preferred:
          # fluid existed node affinity, the name can not be modified.
          - name: fluid.io/node
            weight: 100
          # runtime worker's zone label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/zone
            weight: 50
          # runtime worker's region label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/region
            weight: 20
        # used when app pod with label fluid.io/dataset.{dataset name}.sched set true
        required:
          - fluid.io/node
      name: NodeAffinityWithCache
    plugins:
      serverful:
        withDataset:
        - RequireNodeWithFuse
        - NodeAffinityWithCache
        - MountPropagationInjector
        withoutDataset:
        - PreferNodesWithoutCache
      serverless:
        withDataset:
        - FuseSidecar
        withoutDataset: []

上述ConfigMap資源定義的pluginsProfile欄位中參數解釋：

參數	解釋
`fluid.io/node`	Fluid預留的識別欄位。啟用該欄位後，Fluid會自動為應用Pod注入與快取資料在同一節點的優先親和性語義，語義權重為100。
`topology.kubernetes.io/zone`	Kubernetes叢集中標識所在可用性區域的通用欄位。啟用該欄位後，Fluid自動為應用Pod注入與快取資料在同一可用性區域的優先親和性語義，語義權重為50。
`topology.kubernetes.io/region`	Kubernetes叢集中標識所在地區的通用欄位。啟用該欄位後，Fluid自動為應用Pod注入與快取資料在同一地區的優先親和性語義，語義權重為20。

自訂配置

如果Kubernetes叢集中存在與其他叢集節點拓撲結構相關的節點標籤，並且希望Fluid根據此類節點標籤自動注入親和性資訊，可以執行以下操作完成相關配置。

執行以下命令，更新ConfigMap內容。

kubectl edit -n fluid-system cm webhook-plugins

將ConfigMap修改為如下內容。

您可以根據實際業務需求，刪除已存在的叢集拓撲標籤，請參見參考配置樣本一：忽略節點層級親和性。
您可以根據叢集節點拓撲結構，新增Kubernetes叢集拓撲相關的節點標籤（<topology_key>），並設定該標籤的親和性權重（<topology_weight>），請參見參考配置樣本二：添加叢集節點池層級親和性。

apiVersion: v1
data:
  pluginsProfile: |
    pluginConfig:
    - args: |
        preferred:
          # fluid existed node affinity, the name can not be modified.
          - name: fluid.io/node
            weight: 100
          # runtime worker's zone label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/zone
            weight: 50
          # runtime worker's region label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/region
            weight: 20
          - name: <topology_key>
            weight: <topology_weight>
        # used when app pod with label fluid.io/dataset.{dataset name}.sched set true
        required:
          - fluid.io/node
      name: NodeAffinityWithCache
    plugins:
      serverful:
        withDataset:
        - RequireNodeWithFuse
        - NodeAffinityWithCache
        - MountPropagationInjector
        withoutDataset:
        - PreferNodesWithoutCache
      serverless:
        withDataset:
        - FuseSidecar
        withoutDataset: []

參考配置樣本一：忽略節點層級親和性

apiVersion: v1
data:
  pluginsProfile: |
    pluginConfig:
    - args: |
        preferred:
          # fluid existed node affinity, the name can not be modified.
-         #- name: fluid.io/node
-         #  weight: 100
          # runtime worker's zone label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/zone
            weight: 50
          # runtime worker's region label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/region
            weight: 20
        # used when app pod with label fluid.io/dataset.{dataset name}.sched set true
        required:
          - fluid.io/node
      name: NodeAffinityWithCache
    plugins:
      serverful:
        withDataset:
        - RequireNodeWithFuse
        - NodeAffinityWithCache
        - MountPropagationInjector
        withoutDataset:
        - PreferNodesWithoutCache
      serverless:
        withDataset:
        - FuseSidecar
        withoutDataset: []

參考配置樣本二：添加叢集節點池層級親和性

apiVersion: v1
data:
  pluginsProfile: |
    pluginConfig:
    - args: |
        preferred:
          # fluid existed node affinity, the name can not be modified.
          - name: fluid.io/node
            weight: 100
+         - name: alibabacloud.com/nodepool-id
+           weight: 80
          # runtime worker's zone label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/zone
            weight: 50
          # runtime worker's region label name, can be changed according to k8s environment.
          - name: topology.kubernetes.io/region
            weight: 20
        # used when app pod with label fluid.io/dataset.{dataset name}.sched set true
        required:
          - fluid.io/node
      name: NodeAffinityWithCache
    plugins:
      serverful:
        withDataset:
        - RequireNodeWithFuse
        - NodeAffinityWithCache
        - MountPropagationInjector
        withoutDataset:
        - PreferNodesWithoutCache
      serverless:
        withDataset:
        - FuseSidecar
        withoutDataset: []

執行以下命令，重啟Fluid Webhook組件，使上述配置生效。
```
kubectl rollout restart deployment -n fluid-system fluid-webhook
```

操作樣本

樣本一：資料緩衝節點親和性調度

建立Secret。

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
stringData:
  fs.oss.accessKeyId: <ACCESS_KEY_ID>
  fs.oss.accessKeySecret: <ACCESS_KEY_SECRET>

建立Dataset和Runtime資源。

重要

該章節以JindoRuntime為例講解，如果需要使用其他Runtime作為緩衝運行時，請參見EFC加速NAS或CPFS檔案訪問。如需瞭解JindoFS如何加速OSS檔案訪問，請參見JindoFS加速OSS檔案訪問。

apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: demo-dataset
spec:
  mounts:
    - mountPoint: oss://<oss_bucket>/<bucket_dir>
      options:
        fs.oss.endpoint: <oss_endpoint>
      name: hadoop
      path: "/"
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: fs.oss.accessKeySecret
---
apiVersion: data.fluid.io/v1alpha1
kind: JindoRuntime
metadata:
  name: demo-dataset
spec:
  replicas: 2
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        quota: 10G
        high: "0.99"
        low: "0.8"

建立應用Pod。

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    fuse.serverful.fluid.io/inject: "true"
spec:
  containers:
    - name: nginx
      image: registry.openanolis.cn/openanolis/nginx:1.14.1-8.6
      volumeMounts:
        - mountPath: /data
          name: data-vol
  volumes:
    - name: data-vol
      persistentVolumeClaim:
        claimName: demo-dataset

與本文檔介紹功能相關的關鍵參數說明如下：

參數	說明
`fuse.serverful.fluid.io/inject: "true"`	該Pod需要由Fluid自動注入資料緩衝親和性相關配置資訊的標識。
`claimName`	該Pod聲明掛載的PVC。該PVC由Fluid自動建立，與Dataset同名。

查看應用Pod中注入的親和性調度資訊。

kubectl get pod nginx -oyaml

預期輸出：

apiVersion: v1
kind: Pod
metadata:
  labels:
    fuse.serverful.fluid.io/inject: "true"
  name: nginx
  namespace: default
  ...
spec:
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - preference:
          matchExpressions:
          - key: fluid.io/s-default-demo-dataset
            operator: In
            values:
            - "true"
        weight: 100

應用Pod Spec中自動注入了資料緩衝節點的親和性（fluid.io/s-default-demo-dataset），其權重取決於在調度策略配置中對於不同節點拓撲欄位的配置。

樣本二：資料緩衝同可用性區域親和性調度

建立Secret。

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
stringData:
  fs.oss.accessKeyId: <ACCESS_KEY_ID>
  fs.oss.accessKeySecret: <ACCESS_KEY_SECRET>

建立Dataset和Runtime資源。

重要

apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: demo-dataset
spec:
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: topology.kubernetes.io/zone
              operator: In
              values:
              - "<ZONE_ID>" # e.g. cn-beijing-i
  mounts:
    - mountPoint: oss://<oss_bucket>/<bucket_dir>
      options:
        fs.oss.endpoint: <oss_endpoint>
      name: hadoop
      path: "/"
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: fs.oss.accessKeySecret
---
apiVersion: data.fluid.io/v1alpha1
kind: JindoRuntime
metadata:
  name: demo-dataset
spec:
  replicas: 2
  master:
    nodeSelector:
      topology.kubernetes.io/zone: <ZONE_ID> # e.g. cn-beijing-i
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        quota: 10G
        high: "0.99"
        low: "0.8"

如果需要啟用資料緩衝同可用性區域親和性調度能力，需要在Dataset中顯式定義Dataset資料緩衝所在的可用性區域。例如，在nodeAffinity.required.nodeSelectorTerms中，本範例程式碼定義了資料集的可用性區域標籤topology.kubernetes.io/zone=cn-beijing-i。

建立應用Pod。

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    fuse.serverful.fluid.io/inject: "true"
spec:
  containers:
    - name: nginx
      image: registry.openanolis.cn/openanolis/nginx:1.14.1-8.6
      volumeMounts:
        - mountPath: /data
          name: data-vol
  volumes:
    - name: data-vol
      persistentVolumeClaim:
        claimName: demo-dataset

與本文檔介紹功能相關的關鍵參數說明如下：

參數	說明
`fuse.serverful.fluid.io/inject: "true"`	該Pod需要由Fluid自動注入資料緩衝親和性相關配置資訊的標識。
`claimName`	該Pod聲明掛載的PVC。該PVC由Fluid自動建立，與Dataset同名。

查看應用Pod中注入的親和性調度資訊。

kubectl get pod nginx -oyaml

預期輸出：

apiVersion: v1
kind: Pod
metadata:
  labels:
    fuse.serverful.fluid.io/inject: "true"
  name: nginx
  namespace: default
  ...
spec:
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - preference:
          matchExpressions:
          - key: fluid.io/s-default-demo-dataset
            operator: In
            values:
            - "true"
        weight: 100
      - preference:
          matchExpressions:
          - key: topology.kubernetes.io/zone
            operator: In
            values:
            - <ZONE_ID> # e.g. cn-beijing-i
        weight: 50
...

應用Pod Spec中自動注入了資料緩衝節點的親和性（fluid.io/s-default-demo-dataset）和可用性區域親和性（topology.kubernetes.io/zone），其權重取決於在調度策略配置中對於不同節點拓撲欄位的配置。

樣本三：資料緩衝節點強制親和性調度

建立Secret。

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
stringData:
  fs.oss.accessKeyId: <ACCESS_KEY_ID>
  fs.oss.accessKeySecret: <ACCESS_KEY_SECRET>

建立Dataset和Runtime資源。

重要

apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: demo-dataset
spec:
  mounts:
    - mountPoint: oss://<oss_bucket>/<bucket_dir>
      options:
        fs.oss.endpoint: <oss_endpoint>
      name: hadoop
      path: "/"
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: mysecret
              key: fs.oss.accessKeySecret
---
apiVersion: data.fluid.io/v1alpha1
kind: JindoRuntime
metadata:
  name: demo-dataset
spec:
  replicas: 2
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        quota: 10G
        high: "0.99"
        low: "0.8"

建立應用Pod。

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    fuse.serverful.fluid.io/inject: "true"
    fluid.io/dataset.demo-dataset.sched: required
spec:
  containers:
    - name: nginx
      image: registry.openanolis.cn/openanolis/nginx:1.14.1-8.6
      volumeMounts:
        - mountPath: /data
          name: data-vol
  volumes:
    - name: data-vol
      persistentVolumeClaim:
        claimName: demo-dataset

與本文檔介紹功能相關的關鍵參數說明如下：

參數	說明
`fuse.serverful.fluid.io/inject: "true"`	該Pod需要由Fluid自動注入資料緩衝親和性相關配置資訊的標識。
`fluid.io/dataset.<dataset_name>.sched: required`	該Pod需要注入名為`<dataset_name>`的資料緩衝的強制親和性資訊的標識。
`claimName`	該Pod聲明掛載的PVC。該PVC由Fluid自動建立，與Dataset同名。

查看應用Pod中注入的親和性調度資訊。

kubectl get pod nginx -oyaml

預期輸出：

apiVersion: v1
kind: Pod
metadata:
  labels:
    fluid.io/dataset.demo-dataset.sched: required
    fuse.serverful.fluid.io/inject: "true"
  name: nginx
  namespace: default
  ...
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: fluid.io/s-default-demo-dataset
            operator: In
            values:
            - "true"

應用Pod Spec中自動注入資料緩衝節點的強制親和性（fluid.io/s-default-demo-dataset）。