全部產品
Search
文件中心

Container Service for Kubernetes:MSE Ingress進階用法

更新時間:Aug 23, 2024

在Kubernetes叢集中,MSE Ingress對叢集服務(Service)中的外部可訪問的API對象進行管理,提供七層負載平衡能力。本文介紹MSE Ingress的進階用法,方便您對叢集入口流量進行治理。

灰階發布

MSE Ingress提供複雜的路由處理能力,支援基於Header、Query Parameter、Cookie以及權重的灰階發布功能。灰階發布功能可以通過設定註解來實現,為了啟用灰階發布功能,需要設定註解nginx.ingress.kubernetes.io/canary: "true",通過不同註解可以實現不同的灰階發布功能。

說明

當多種方式同時配置時,灰階方式選擇優先順序為:基於Header | 基於Query Parameter > 基於Cookie > 基於權重(從高到低)。

基於Header灰階發布

  • 只配置nginx.ingress.kubernetes.io/canary-by-header:基於Request Header的流量切分,當配置的header值為always時,請求流量會被分配到灰階服務入口;其他情況時,請求流量不會分配到灰階服務。

  • 同時配置nginx.ingress.kubernetes.io/canary-by-header-value和nginx.ingress.kubernetes.io/canary-by-header:當請求中的header和header-value與設定的值匹配時,請求流量會被分配到灰階服務;其他情況下,請求流量不會被分配到灰階服務。

說明

相比Nginx Ingress和ALB Ingress灰階發布時最多隻支援兩個版本服務,MSE Ingress灰階發布時支援多個版本服務(無上限)。

例如:

  • 請求Header為mse:always時會訪問灰階服務demo-service-canary;其他情況將訪問正式服務demo-service。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact          

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 請求Header為mse:v1時將訪問灰階服務demo-service-canary-v1;請求Header為mse:v2時將訪問灰階服務demo-service-canary-v2;其他情況將訪問正式服務demo-service。配置如下。

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary-v1
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v2"
      name: demo-canary-v2
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary-v2
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary-v1
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v2"
      name: demo-canary-v2
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary-v2
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

基於Query Parameter灰階發布

  • 僅配置mse.ingress.kubernetes.io/canary-by-query

    基於URL Query Parameter的流量切分,當請求的URL中Query Parameter的Key為該參數配置且Value為always時,請求流量會被分配到灰階服務入口。其他情況下,請求流量不會分配到灰階服務。

  • 同時配置mse.ingress.kubernetes.io/canary-by-query-value和mse.ingress.kubernetes.io/canary-by-query

    當請求中的query parameter keyquery parameter value與設定的值匹配時,請求流量會被分配到灰階服務。其他情況下,請求流量不會分配到灰階服務。

    說明

    基於Header的灰階發布可以和基於Query Parameter的灰階發布一起使用,同時滿足匹配條件,請求流量才會被分配到灰階服務。

樣本:

  • 請求URL的Query Parameter為canary:gray時會訪問灰階服務demo-service-canary,其他情況將訪問正式服務demo-service。相關配置如下。

    1.19及之後版本叢集

    apiVersion:networking.k8s.io/v1 
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        mse.ingress.kubernetes.io/canary-by-query: "canary"
        mse.ingress.kubernetes.io/canary-by-query-value: "gray"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact 

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        mse.ingress.kubernetes.io/canary-by-query: "canary"
        mse.ingress.kubernetes.io/canary-by-query-value: "gray"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 請求URL的Query Parameter為canary:gray,同時請求Header包含x-user-id: test時,會訪問灰階服務demo-service-canary,其他情況將訪問正式服務demo-service。相關配置如下。

    1.19及之後版本叢集

    apiVersion:networking.k8s.io/v1 
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        mse.ingress.kubernetes.io/canary-by-query: "canary"
        mse.ingress.kubernetes.io/canary-by-query-value: "gray"
        nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
        nginx.ingress.kubernetes.io/canary-by-header-value: "test"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact 

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        mse.ingress.kubernetes.io/canary-by-query: "canary"
        mse.ingress.kubernetes.io/canary-by-query-value: "gray"
        nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
        nginx.ingress.kubernetes.io/canary-by-header-value: "test"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

基於Cookie灰階發布

nginx.ingress.kubernetes.io/canary-by-cookie:基於Cookie的流量切分,當配置的cookie值為always時,請求流量會被分配到灰階服務;其他情況時,請求流量將不會分配到灰階服務。

說明

基於Cookie的灰階發布不支援設定自訂值,配置的cookie值只能為always

例如,請求的Cookie為demo=always時會訪問灰階服務demo-service-canary;其他情況將訪問正式服務demo-service。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "demo"
  name: demo-canary
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "demo"
  name: demo-canary
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

基於權重灰階發布

註解

說明

nginx.ingress.kubernetes.io/canary-weight

佈建要求到指定服務的百分比(值為0~100的整數)。

nginx.ingress.kubernetes.io/canary-weight-total

設定權重總和,預設為100。

例如,配置灰階服務demo-service-canary-v1的權重為30%,配置灰階服務demo-service-canary-v2的權重為20%,配置正式服務demo-service的權重為50%。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
  name: demo-canary-v1
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
  name: demo-canary-v2
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary-v2
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
  name: demo-canary-v1
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary-v1
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
  name: demo-canary-v2
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary-v2
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

服務Subset

服務Subset適用於一個Service關聯多個Deployment的情境,通過Ingress將請求轉寄至該Service下Pod集合的子集,常見情況是將請求轉寄至某個Service下含有某個Label的Pod集合,有如下兩種配置方式:

使用MSE Ingress約定的Pod Label

通過註解mse.ingress.kubernetes.io/service-subset設定Service版本。預設情況下,MSE Ingress約定配置的服務版本與Pod Label中以opensergo.io/canary為首碼的Label有對應關係。該註解含義如下:

  • 當配置為""或者base時,請求會被轉寄到Label中含有opensergo.io/canary: ""或不含有任何opensergo.io/canary為首碼的Label Key的Pod集合,即Label上打了空標或未打標的Pod集合。

  • 當配置為其他值,請求會被轉寄到Label中含有opensergo.io/canary-{其他值}: {其他值}的Pod集合。例如當配置為gray,請求會被轉寄到Label中含有opensergo.io/canary-gray: gray的Pod集合。

例如存在一個K8s Service go-httpbin關聯了兩個Deployment,其中一個Deployment管理的Pod不含有任何opensergo.io/canary為首碼的Label Key,另一個Deployment管理的Pod含有灰階標opensergo.io/canary-gray: gray,配置如下:

# go-httpbin k8s service
apiVersion: v1
kind: Service
metadata:
  name: go-httpbin
  namespace: default
spec:
  ports:
    - port: 8080
      protocol: TCP
  selector:
    app: go-httpbin
---
# go-httpbin base deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-httpbin-base
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-httpbin
  template:
    metadata:
      labels:
        app: go-httpbin
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/mse/go-httpbin
          args:
            - "--version=base"
          imagePullPolicy: Always
          name: go-httpbin
---
# go-httpbin gray deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-httpbin-gray
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-httpbin
  template:
    metadata:
      labels:
        app: go-httpbin
        opensergo.io/canary-gray: gray
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/mse/go-httpbin
          args:
            - "--version=gray"
          imagePullPolicy: Always
          name: go-httpbin

如果期望對於example.com/test請求,若請求Header包含x-user-id: test,則轉寄到go-httpbin-gray;否則轉寄到go-httpbin-base,配置如下:

1.19及之後版本叢集

apiVersion:networking.k8s.io/v1 
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
    nginx.ingress.kubernetes.io/canary-by-header-value: "test"
    # 轉寄請求到含有灰階標opensergo.io/canary-gray: gray的Pod集合
    mse.ingress.kubernetes.io/service-subset: gray
  name: demo-canary
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: go-httpbin
                port: 
                  number: 8080
            path: /test
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    # 轉寄請求到不含有以opensergo.io/canary為首碼的Label的Pod集合
    mse.ingress.kubernetes.io/service-subset: ""
  name: demo
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: go-httpbin
                port: 
                  number: 8080
            path: /test
            pathType: Exact 

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
    nginx.ingress.kubernetes.io/canary-by-header-value: "test"
    # 轉寄請求到含有灰階標opensergo.io/canary-gray: gray的Pod集合
    mse.ingress.kubernetes.io/service-subset: gray
  name: demo-canary
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /test
            backend:
              # 佈建服務為go-httpbin,但在註解中指定版本
              serviceName: go-httpbin
              servicePort: 8080
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    # 轉寄請求到不含有以opensergo.io/canary為首碼的Label的Pod集合
    mse.ingress.kubernetes.io/service-subset: ""
  name: demo
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /test
            backend:
              # 佈建服務為go-httpbin,但在註解中指定版本
              serviceName: go-httpbin
              servicePort: 8080

使用自訂Label

通過同時配置註解mse.ingress.kubernetes.io/service-subsetmse.ingress.kubernetes.io/subset-labels,設定自訂Label來定義Subset所屬Pod集合。

說明

此時該subset不再與opensergo.io/canary為首碼的Label有對應關係。

例如存在一個K8s Service go-httpbin關聯了兩個Deployment,其中一個Deployment管理的Pod不含有任何opensergo.io/canary為首碼的Label Key,另一個Deployment管理的Pod含有灰階標version: gray,配置如下:

# go-httpbin k8s service
apiVersion: v1
kind: Service
metadata:
  name: go-httpbin
  namespace: default
spec:
  ports:
    - port: 8080
      protocol: TCP
  selector:
    app: go-httpbin
---
# go-httpbin base deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-httpbin-base
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-httpbin
  template:
    metadata:
      labels:
        app: go-httpbin
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/mse/go-httpbin
          args:
            - "--version=base"
          imagePullPolicy: Always
          name: go-httpbin
---
# go-httpbin base gray
apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-httpbin-gray
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-httpbin
  template:
    metadata:
      labels:
        app: go-httpbin
        version: gray
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/mse/go-httpbin
          args:
            - "--version=gray"
          imagePullPolicy: Always
          name: go-httpbin

如果期望對於example.com/test的請求,若請求Header包含x-user-id: test,則轉寄到go-httpbin-gray;否則轉寄到go-httpbin-base。

1.19及之後版本叢集

apiVersion:networking.k8s.io/v1 
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
    nginx.ingress.kubernetes.io/canary-by-header-value: "test"
    # 轉寄請求到含有灰階標version: gray的Pod集合
    mse.ingress.kubernetes.io/service-subset: gray
    mse.ingress.kubernetes.io/subset-labels: version gray
  name: demo-canary
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: go-httpbin
                port: 
                  number: 8080
            path: /test
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    # 轉寄請求到不含有以opensergo.io/canary為首碼的Label的Pod集合
    mse.ingress.kubernetes.io/service-subset: ""
  name: demo
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: go-httpbin
                port: 
                  number: 8080
            path: /test
            pathType: Exact 

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
    nginx.ingress.kubernetes.io/canary-by-header-value: "test"
    # 轉寄請求到含有灰階標version: gray的Pod集合
    mse.ingress.kubernetes.io/service-subset: gray
    mse.ingress.kubernetes.io/subset-labels: version gray
  name: demo-canary
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /test
            backend:
              # 佈建服務為go-httpbin,但在註解中指定版本
              serviceName: go-httpbin
              servicePort: 8080
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    # 轉寄請求到不含有以opensergo.io/canary為首碼的Label的Pod集合
    mse.ingress.kubernetes.io/service-subset: ""
  name: demo
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /test
            backend:
              # 佈建服務為go-httpbin,但在註解中指定版本
              serviceName: go-httpbin
              servicePort: 8080

跨域

跨域資源共用CORS(Cross-Origin Resource Sharing)是指允許Web應用伺服器進行跨域存取控制,從而實現跨域資料安全傳輸。關於跨域的更多資訊,請參見跨源資源共用(CORS)

註解

說明

nginx.ingress.kubernetes.io/enable-cors

開啟或關閉跨域。

nginx.ingress.kubernetes.io/cors-allow-origin

允許的第三方網站,第三方網站之間使用英文逗號分隔,支援萬用字元*。預設值為*,即允許所有第三方網站。

nginx.ingress.kubernetes.io/cors-allow-methods

允許的要求方法,如GET、POST、PUT等,要求方法之間使用英文逗號分隔,支援萬用字元*。預設值為GET、PUT、POST、DELETE、PATCH、OPTIONS。

nginx.ingress.kubernetes.io/cors-allow-headers

允許的請求Header,Header之間使用英文逗號分隔,支援萬用字元*。預設值為DNT、X-CustomHeader、Keep-Alive、User-Agent、X-Requested-With、If-Modified-Since、Cache-Control、Content-Type、Authorization。

nginx.ingress.kubernetes.io/cors-expose-headers

允許暴露給瀏覽器的響應Header,響應Header之間使用英文逗號分隔。

nginx.ingress.kubernetes.io/cors-allow-credentials

是否允許攜帶憑證資訊。預設允許。

nginx.ingress.kubernetes.io/cors-max-age

預檢結果的最大緩衝時間,單位為秒。預設值為1728000秒。

例如,跨域請求被限制為只能來自example.com域的請求,並且HTTP的要求方法只能是GET和POST,允許的要求標頭部為X-Foo-Bar,不允許攜帶憑證資訊。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "example.com"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET,POST"
    nginx.ingress.kubernetes.io/cors-allow-headers: "X-Foo-Bar"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "false"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "example.com"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET,POST"
    nginx.ingress.kubernetes.io/cors-allow-headers: "X-Foo-Bar"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "false"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

正則匹配

標準的K8s Ingress只支援精確匹配和首碼匹配,MSE Ingress額外支援正則匹配,您可以通過註解nginx.ingress.kubernetes.io/use-regex: true使Ingress Spec中定義的Path匹配變為正則匹配。

如期望網域名稱為example.com,請求Path以/app或/test開頭的請求轉寄至服務demo,配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/use-regex: 'true'
  name: regex-match
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo
                port: 
                  number: 8080
            path: /(app|test)/(.*)
            pathType: Prefix

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/use-regex: 'true'
  name: regex-match
  namespace: default
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /(app|test)/(.*)
            backend:
              serviceName: demo
              servicePort: 8080

Rewrite重寫Path和Host

在請求轉寄給目標後端服務之前,重寫可以修改原始請求的路徑(Path)和主機域(Host)。

註解

說明

nginx.ingress.kubernetes.io/rewrite-target

重寫Path,支援擷取的群組(Capture Group)。

nginx.ingress.kubernetes.io/upstream-vhost

重寫Host。

Rewrite重寫Path

  1. 將請求example.com/test轉寄至後端服務之前,重寫為example.com/dev。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/dev"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/dev"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /test
                pathType: Exact
                backend:
                  serviceName: demo-service
                  servicePort: 80
  2. 將請求example.com/v1/xxx,即以/v1/為首碼的任意Path,轉寄至後端服務之前,去掉Path首碼/v1,重寫為example.com/xxx。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /v1/(.*)
                pathType: Prefix

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /v1/(.*) 
              	pathType: Prefix
                backend:
                  serviceName: demo-service
                  servicePort: 80
  3. 將請求example.com/v1/xxx,即以/v1/為首碼的任意Path,轉寄至後端服務之前,將Path首碼/v1更改為/v2,重寫為example.com/v2/xxx。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/v2/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /v1/(.*)
                pathType: Prefix

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/v2/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /v1/(.*)
                pathType: Prefix
                backend:
                  serviceName: demo-service
                  servicePort: 80

Rewrite重寫Host

例如,把請求example.com/test在轉寄至後端服務之前,重寫為test.com/test。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: "test.com"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: "test.com"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

重新導向

通過重新導向可以把原始用戶端請求更改為目標請求。

配置HTTP重新導向至HTTPS

註解

說明

nginx.ingress.kubernetes.io/ssl-redirect

HTTP重新導向到HTTPS

nginx.ingress.kubernetes.io/force-ssl-redirect

HTTP重新導向到HTTPS

說明

MSE Ingress對於以上兩個註解不區分對待,都是強制將HTTP重新導向到HTTPS。

例如,將請求http://example.com/test重新導向為https://example.com/test。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

永久重新導向

註解

說明

nginx.ingress.kubernetes.io/permanent-redirect

永久重新導向的目標URL,必須包含Scheme(HTTP或HTTPS)。

nginx.ingress.kubernetes.io/permanent-redirect-code

永久重新導向的HTTP狀態代碼,預設值為301。

例如,把請求http://example.com/test永久重新導向為http://example.com/app。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

臨時重新導向

nginx.ingress.kubernetes.io/temporal-redirect:臨時重新導向的目標URL,必須包含Scheme(HTTP或者HTTPS)。

例如,將請求http://example.com/test臨時重新導向為http://example.com/app。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/temporal-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/temporal-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

Header控制

通過Header控制,您可以在請求轉寄到後端服務之前對請求Header進行增刪改,在收到響應轉寄給用戶端時對響應Header進行增刪改。

請求Header控制

註解

說明

mse.ingress.kubernetes.io/request-header-control-add

請求在轉寄給後端服務時,添加指定Header。若該Header存在,則其值拼接在原有值後面。文法如下:

  • 單個Header:Key Value。

  • 多個Header:使用YAML特殊符號|,每對Key Value單獨處於一行。

mse.ingress.kubernetes.io/request-header-control-update

請求在轉寄給後端服務時,修改指定Header。若該Header存在,則其值覆蓋原有值。文法如下:

  • 單個Header:Key Value。

  • 多個Header:使用YAML特殊符號|,每對Key Value單獨處於一行。

mse.ingress.kubernetes.io/request-header-control-remove

請求在轉寄給後端服務時,刪除指定Header。文法如下:

  • 單個Header:Key。

  • 多個Header:使用英文逗號分隔。

例如:

  • 對於請求example.com/test添加兩個Header,分別是foo: bar和test: true。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: |
          foo bar
          test true
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: |
          foo bar
          test true
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com 
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • Header控制可以結合灰階發布,對灰階流量進行染色。請求Header為mse:v1時將訪問灰階服務demo-service-canary-v1,並添加Header(stage: gray);其他情況將訪問正式服務demo-service,並添加Header(stage: production)。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
        mse.ingress.kubernetes.io/request-header-control-add: "stage gray"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary-v1
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: "stage production"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
        mse.ingress.kubernetes.io/request-header-control-add: "stage gray"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary-v1
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: |
          foo bar
          test true
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com 
          http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

響應Header控制

註解

說明

mse.ingress.kubernetes.io/response-header-control-add

請求在收到後端服務響應之後並且轉寄響應給用戶端之前,添加指定Header。若該Header存在,則其值拼接在原有值後面。文法如下:

  • 單個Header:Key Value。

  • 多個Header:使用YAML特殊符號|,每對Key Value單獨處於一行。

mse.ingress.kubernetes.io/response-header-control-update

請求在收到後端服務響應之後並且轉寄響應給用戶端之前,修改指定Header。若該Header存在,則其值覆蓋原有值。文法如下:

  • 單個Header:Key Value。

  • 多個Header:使用YAML特殊符號|,每對Key Value單獨處於一行。

mse.ingress.kubernetes.io/response-header-control-remove

請求在收到後端服務響應之後並且轉寄響應給用戶端之前,刪除指定Header。文法如下:

  • 單個Header:Key。

  • 多個Header:使用英文逗號分隔。

例如,對於請求example.com/test的響應刪除Header:req-cost-time。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/response-header-control-remove: "req-cost-time"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/response-header-control-remove: "req-cost-time"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

重試

MSE Ingress提供路由層級的重試設定,可以自動為出錯的請求進行重試。您可以按需設定重試條件,例如建立串連失敗、後端服務不可用或者對指定HTTP狀態代碼的響應等進行請求重試。

註解

說明

nginx.ingress.kubernetes.io/proxy-next-upstream-tries

請求的最大重試次數。預設為3次。

nginx.ingress.kubernetes.io/proxy-next-upstream-timeout

請求重試的逾時時間,單位秒。預設未配置逾時時間。

nginx.ingress.kubernetes.io/proxy-next-upstream

請求重試條件,使用英文逗號作為分隔。預設值為error,timeout,合法值如下:

  • error:建立串連失敗,請求出錯5xx。

  • timeout:建立連線逾時,請求出錯5xx。

  • invalid_header:請求出錯5xx。

  • http_xxx:針對具體響應狀態代碼的情況進行重試。例如:http_502、http_403。

  • non_idempotent:對於非等冪請求出錯時進行重試。預設情況下,MSE Ingress針對非等冪POST、PATCH請求出錯時不會進行重試;如果配置non_idempotent,可以開啟重試。

  • off:關閉重試。

例如,設定example/test請求的最大重試次數為2次,重試逾時時間為5秒,只有在響應狀態代碼為502才重試,並且開啟非等冪重試。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "2"
    nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "5"
    nginx.ingress.kubernetes.io/proxy-next-upstream: "http_502,non_idempotent"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "2"
    nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "5"
    nginx.ingress.kubernetes.io/proxy-next-upstream: "http_502,non_idempotent"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

IP黑白名單存取控制

MSE Ingress提供網域名稱級和路由級的IP黑/白名單存取控制,且路由級的優先順序高於網域名稱級。

路由級IP存取控制

註解

說明

nginx.ingress.kubernetes.io/whitelist-source-range

指定路由上的IP白名單,支援IP地址或CIDR地址塊,以英文逗號分隔。

mse.ingress.kubernetes.io/blacklist-source-range

指定路由上的IP黑名單,支援IP地址或CIDR地址塊,以英文逗號分隔。

例如:

  • 僅允許用戶端IP為1.1.xx.xx訪問example.com/test。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.X.X
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.X.X
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com 
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 拒絕用戶端IP為2.2.xx.xx訪問example.com/test。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/blacklist-source-range: 2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/blacklist-source-range: 2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

網域名稱級IP存取控制

註解

說明

mse.ingress.kubernetes.io/domain-whitelist-source-range

指定網域名稱上的IP白名單,網域名稱優先順序低於路由層級,支援IP地址或CIDR地址塊,IP之間以英文逗號分隔。

mse.ingress.kubernetes.io/domain-blacklist-source-range

指定網域名稱上的IP黑名單,網域名稱優先順序低於路由層級,支援IP地址或CIDR地址塊,IP之間以英文逗號分隔。

例如:

  • 僅允許用戶端IP為1.1.xx.xx和2.2.xx.xx可以訪問example.com網域名稱下所有路由。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.X.X,2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
             - backend:
                  service:
                    name: app-service
                    port: 
                      number: 80
                path: /app
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.X.X,2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
              - path: /app
                backend:
                  serviceName: app-service
                  servicePort: 80
  • 網域名稱級和路由級IP存取控制可以結合使用,僅允許用戶端IP為1.1.xx.xx和2.2.xx.xx可以訪問example.com網域名稱下所有路由,但對於example.com/order這條路由,僅允許用戶端IP為3.3.xx.xx可以訪問。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.X.X,2.2.2.2
      name: demo-domain
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
             - backend:
                  service:
                    name: app-service
                    port: 
                      number: 80
                path: /app
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 3.3.X.X
      name: demo-route
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /order
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.X.X,2.2.2.2
      name: demo-domain
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
              - path: /app
                backend:
                  serviceName: app-service
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 3.3.X.X
      name: demo-route
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /order
                backend:
                  serviceName: demo-service
                  servicePort: 80

單機限流

MSE Ingress支援針對路由層級的單機限流策略,在設定的時間周期內,限制每個網關副本匹配在某個路由上的請求數量不大於閾值。

說明

該限流是針對單機層級,即配置的閾值在每個網關執行個體進行流控。如果希望限制某個路由在網關叢集上的全域流量,請使用全域限流量控制。

註解

說明

mse.ingress.kubernetes.io/route-limit-rpm

該Ingress定義的路由在每個網關執行個體上每分鐘最大請求次數。瞬時最大請求次數為該值乘以limit-burst-multiplier。

觸發限流時,響應Body內容為local_rate_limited,響應狀態代碼說明:

  • 網關版本小於1.2.23:狀態代碼為503。

  • 網關版本1.2.23及以上:狀態代碼為429。

mse.ingress.kubernetes.io/route-limit-rps

該Ingress定義的路由在每個網關執行個體上每秒最大請求次數。瞬時最大請求次數為該值乘以limit-burst-multiplier。

觸發限流時,響應Body內容為local_rate_limited,響應狀態代碼說明:

  • 網關版本小於1.2.23:狀態代碼為503。

  • 網關版本1.2.23及以上:狀態代碼為429。

mse.ingress.kubernetes.io/route-limit-burst-multiplier

瞬時最大請求次數的因子,預設為5。

例如:

  • 限制example.com/test的請求每分鐘最大請求數為100,瞬時請求數為200。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rpm: "100"
        mse.ingress.kubernetes.io/route-limit-burst-multiplier: "2"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rpm: "100"
        mse.ingress.kubernetes.io/route-limit-burst-multiplier: "2"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 限制example.com/test的請求每秒最大請求數為10,瞬時請求數50。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rps: "10"
        # 預設為5
        # mse.ingress.kubernetes.io/route-limit-burst-multiplier: "5"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rps: "10"
        # 預設為5
        # mse.ingress.kubernetes.io/route-limit-burst-multiplier: "5"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

全域限流量控制

MSE Ingress與Sentinel整合,提供路由層級的網關叢集全域限流,即限制某個路由在網關叢集全域的每秒最大請求數。

說明

該功能要求MSE Ingress網關的版本至少為1.2.25。

通過註解mse.ingress.kubernetes.io/rate-limit設定路由在網關叢集全域上每秒最大請求數。當觸發限流時,請求的響應結果的預設行為為:響應狀態代碼為429,響應Body為sentinel rate limited。目前MSE Ingress提供兩種方式自訂限流行為:自訂響應和重新導向,這兩種方式只能二選一。

自訂響應

  • mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-code:觸發限流時的響應狀態代碼,預設為429。

  • mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-body-type:觸發限流時的響應Body格式,預設為text

    • 配置為text時:響應的Content-Type值為text/plain; charset=UTF-8

    • 配置為json時:響應的Content-Type的值為application/json; charset=UTF-8

  • mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-body:觸發限流時的響應Body,預設為sentinel rate limited

範例一:期望限制example.com/test請求在網關叢集上每秒最大請求數為100,保持預設的限流行為,配置如下。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/rate-limit: "100"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/rate-limit: "100"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

範例二:期望限制example.com/test請求在網關叢集上每秒最大請求數為100,觸發限流時,響應狀態代碼為503,響應體為server is overload。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/rate-limit: "100"
    mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-code: 503
    mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-body: "server is overload"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/rate-limit: "100"
    mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-code: 503
    mse.ingress.kubernetes.io/rate-limit-fallback-custom-response-body: "server is overload"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

重新導向

  • mse.ingress.kubernetes.io/rate-limit-fallback-redirect-url:觸發限流時的重新導向地址。

範例一:期望限制example.com/test請求在網關叢集上每秒最大請求數為100,觸發限流時,重新導向到example.com/fallback。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/rate-limit: "100"
    mse.ingress.kubernetes.io/rate-limit-fallback-redirect-url: "example.com/fallback"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/rate-limit: "100"
    mse.ingress.kubernetes.io/rate-limit-fallback-redirect-url: "example.com/fallback"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

全域並發控制

MSE Ingress與Sentinel整合,提供路由層級的網關叢集全域並發控制,即限制某個路由在網關叢集全域的最大正在處理的請求數。

說明

該功能要求MSE Ingress網關的版本至少為1.2.25。

通過註解mse.ingress.kubernetes.io/concurrency-limit設定路由在網關叢集全域上最大處理請求數。當觸發全域並發控制時,請求響應狀態代碼為429,Body為sentinel rate limited。目前MSE Ingress提供兩種方式可以自訂並發行為:自訂響應和重新導向,這兩種方式只能二選一。

自訂響應

  • mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-code:觸發並發控制時的響應狀態代碼,預設為429。

  • mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-body-type:觸發並發控制時的響應Body的格式,預設為text

    • 配置為text時:響應的Content-Type值為text/plain; charset=UTF-8

    • 配置為json時:響應的Content-Type的值為application/json; charset=UTF-8

  • mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-body:觸發並發控制時的響應Body,預設為sentinel rate limited

範例一:期望限制example.com/test的請求在網關叢集全域上最大處理請求數為1000,保持預設的並發行為。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/concurrency-limit: "1000"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/concurrency-limit: "1000"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

範例二:期望限制example.com/test的請求在網關叢集全域最大處理請求數為1000,觸發並發控制時,響應狀態代碼為503,響應體為server is overloaded

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/concurrency-limit: "1000"
    mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-code: 503
    mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-body: "server is overload"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/concurrency-limit: "1000"
    mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-code: 503
    mse.ingress.kubernetes.io/concurrency-limit-fallback-custom-response-body: "server is overload"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

重新導向

  • mse.ingress.kubernetes.io/concurrency-limit-fallback-redirect-url:觸發並發控制時的重新導向地址。

期望限制example.com/test請求在網關叢集全域上最大處理請求數為1000,觸發並發控制時,重新導向到example.com/fallback。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/concurrency-limit: "1000"
    mse.ingress.kubernetes.io/concurrency-limit-fallback-redirect-url: "example.com/fallback"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/concurrency-limit: "1000"
    mse.ingress.kubernetes.io/concurrency-limit-fallback-redirect-url: "example.com/fallback"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

流量鏡像

通過配置流量鏡像,可以複製流量到指定服務,常用於Action Trail和流量測試等情境。

  • mse.ingress.kubernetes.io/mirror-target-service:複製流量轉寄到指定鏡像服務。服務格式為:namespace/name:port。

    • namespace: K8s Service所在的命名空間,可選,預設為Ingress所在的命名空間。

    • name:K8s Service的名稱,必選。

    • port:待轉寄至K8s Service的連接埠,可選,預設為第一個連接埠。

  • mse.ingress.kubernetes.io/mirror-percentage:複製流量的比例。可配置的值的範圍為:0~100,預設100。

說明

複製的流量在轉寄給目標服務時,原始請求中的Host會被自動加上-shadow尾碼。

例如,將example.com/test的流量複製並轉寄到目標服務:命名空間為test,服務名為app,連接埠為8080。

說明

本樣本中,複製的流量在轉寄給目標服務時,Host會被自動改寫為example.com-shadow。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/mirror-target-service: test/app:8080
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/mirror-target-service: test/app:8080
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

例如,將example.com/test的流量複製並轉寄到目標服務:命名空間為test,服務名為app,連接埠為8080,且複製比例為10%。

說明

本樣本中,複製的流量在轉寄給目標服務時,Host會被自動改寫為example.com-shadow。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/mirror-target-service: test/app:8080
    mse.ingress.kubernetes.io/mirror-percentage: 10
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/mirror-target-service: test/app:8080
    mse.ingress.kubernetes.io/mirror-percentage: 10
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

配置後端服務合約:HTTPS或gRPC

MSE Ingress預設使用HTTP協議轉寄請求到後端業務容器。當您的業務容器為HTTPS協議時,可以通過使用註解nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"來轉寄請求到後端業務容器;當您的業務容器為gRPC服務時,可以通過使用註解nginx.ingress.kubernetes.io/backend-protocol: "GRPC"來轉寄請求到後端業務容器。

說明

相比Nginx Ingress的優勢,如果您的後端服務所屬的K8s Service資源中關於Port Name的定義為gRPC或HTTP2,您無需配置註解nginx.ingress.kubernetes.io/backend-protocol: "GRPC",MSE Ingress會自動使用gRPC或者HTTP2。

例如:

  • 請求example/test轉寄至後端服務使用HTTPS協議。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 請求example/test轉寄至後端服務使用gRPC協議。此處列舉兩種做法,如下:

    • 方法1:通過註解,配置如下:

      1.19及之後版本叢集

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        annotations:
          nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - backend:
                    service:
                      name: demo-service
                      port: 
                        number: 80
                  path: /test
                  pathType: Exact

      1.19版本之前叢集

      apiVersion: networking.k8s.io/v1beta1
      kind: Ingress
      metadata:
        annotations:
          nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - path: /test
                  backend:
                    serviceName: demo-service
                    servicePort: 80
    • 方法2:通過Service Port Name,配置如下:

      1.19及之後版本叢集

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - backend:
                    service:
                      name: demo-service
                      port: 
                        number: 80
                  path: /order
                  pathType: Exact
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: demo-service
      spec:
        ports:
          - name: grpc
            port: 80
            protocol: TCP
        selector:
          app: demo-service

      1.19版本之前叢集

      apiVersion: networking.k8s.io/v1beta1
      kind: Ingress
      metadata:
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - path: /test
                  backend:
                    serviceName: demo-service
                    servicePort: 80
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: demo-service
      spec:
        ports:
          - name: grpc
            port: 80
            protocol: TCP
        selector:
          app: demo-service

配置後端服務的負載平衡演算法

負載平衡決定著網關在轉寄請求至後端服務時如何選擇節點。

普通負載平衡演算法

nginx.ingress.kubernetes.io/load-balance:後端服務的普通負載平衡演算法。預設為round_robin。合法值如下:

  • round_robin:基於輪詢的負載平衡。

  • least_conn:基於最小請求數的負載平衡。

  • random:基於隨機的負載平衡。

重要

雲原生網關不支援EWMA演算法,若配置為EWMA演算法,會回退到Round Robin演算法。

例如,設定後端服務demo-service的負載平衡演算法為least_conn。設定如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/load-balance: "least_conn"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /order
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/load-balance: "least_conn"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

基於一致性Hash的負載平衡演算法

基於一致性Hash的負載平衡演算法具備請求親和性,具有相同特徵的請求會始終負載到相同節點上。MSE Ingress支援基於部分Nginx變數的請求Header和請求路徑參數作為Hash Key。

nginx.ingress.kubernetes.io/upstream-hash-by:基於一致性Hash的負載平衡演算法,雲原生網關支援以下幾種形式:

  • 雲原生網關支援配置部分nginx變數:

    • $request_uri:請求的Path(包括路徑參數)作為Hash Key。

    • $host:請求的Host作為Hash Key。

    • $remote_addr:請求的用戶端IP作為Hash Key。

  • 基於請求Header的一致性Hash。您只需配置為$http_headerName。

  • 基於請求路徑參數的一致性Hash。您只需配置為$arg_varName。

例如:

  • 基於請求的用戶端IP作為Hash Key,同一個用戶端IP的請求始終負載到同一個節點。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 基於請求Header x-stage作為Hash key,帶有x-stage頭部的請求且值相同的請求始終負載到同一個節點。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$http_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$http_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 基於請求路徑參數 x-stage作為Hash key,帶有路徑參數x-stage的請求且值相同的請求始終負載到同一個節點。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$arg_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$arg_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

服務預熱(無損上線)

服務預熱可以保證新節點上線時,流量在指定預熱視窗內是逐步調大,充分保證新節點完成預熱。

mse.ingress.kubernetes.io/warmup:服務預熱時間,單位為秒。預設不開啟。

說明

服務預熱依賴於所選的負載平衡演算法,目前僅支援Round Robin和least_conn。

例如,對於後端服務demo-service開啟預熱,預熱視窗為30s。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/warmup: "30"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/warmup: "30"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

Cookie親和性(會話保持)

具備相同Cookie的請求會被網關始終負載到同一個節點,並且如果第一次訪問攜帶Cookie,MSE Ingress會在第一次響應時為用戶端產生一個Cookie,用來保證後續的請求被網關始終負載到相同節點。

註解

說明

nginx.ingress.kubernetes.io/affinity

親和性種類,目前只支援Cookie,預設為Cookie。

nginx.ingress.kubernetes.io/affinity-mode

親和性模式,雲原生網關目前只支援Balanced模式,預設為Balanced模式。

nginx.ingress.kubernetes.io/session-cookie-name

配置指定Cookie的值作為Hash Key,預設為INGRESSCOOKIE。

nginx.ingress.kubernetes.io/session-cookie-path

當指定Cookie不存在,產生的Cookie的Path值,預設為/。

nginx.ingress.kubernetes.io/session-cookie-max-age

當指定Cookie不存在,產生的Cookie的到期時間,單位為秒,預設為Session會話層級。

nginx.ingress.kubernetes.io/session-cookie-expires

當指定Cookie不存在,產生的Cookie的到期時間,單位為秒,預設為Session會話層級。

例如:

  • 開啟Cookie親和性,利用MSE Ingress的預設配置,即Cookie的名字為INGRESSCOOKIE,Path為/,Cookie的生命週期為Session會話層級。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
  • 開啟Cookie親和性,Cookie的名字為test,Path為/,Cookie的到期時間為10s。配置如下:

    1.19及之後版本叢集

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "test"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "10"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

    1.19版本之前叢集

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "test"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "10"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

網關與後端服務之間的串連池配置

通過在網關側對指定服務進行串連池配置,可以控制網關與後端服務之間的串連數量,有效防止後端服務過載,提高後端服務的穩定性和高可用。

  • mse.ingress.kubernetes.io/connection-policy-tcp-max-connection:網關與後端服務之間可以建立串連的最大數量。

  • mse.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint:網關與後端服務的單個節點之間可以建立串連的最大數量。

  • mse.ingress.kubernetes.io/connection-policy-http-max-request-per-connection:網關與後端服務之間單個串連上的最大請求數。

例如,對後端服務demo-service配置,網關與後端服務之間可以建立串連的最大數量為10,網關與後端服務的單個節點之間可以建立串連的最大數量為2。

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/connection-policy-tcp-max-connection:10
  	mse.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint:2
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/connection-policy-tcp-max-connection:10
  	mse.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint:2
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

配置用戶端到網關之間的TLS版本以及加密套件

目前,MSE Ingress預設最小TLS版本為TLSv1.0,預設最大TLS版本為TLSv1.3,預設加密套件為:

  • ECDHE-ECDSA-AES128-GCM-SHA256

  • ECDHE-RSA-AES128-GCM-SHA256

  • ECDHE-ECDSA-AES128-SHA

  • ECDHE-RSA-AES128-SHA

  • AES128-GCM-SHA256

  • AES128-SHA

  • ECDHE-ECDSA-AES256-GCM-SHA384

  • ECDHE-RSA-AES256-GCM-SHA384

  • ECDHE-ECDSA-AES256-SHA

  • ECDHE-RSA-AES256-SHA

  • AES256-GCM-SHA384

  • AES256-SHA

您可以通過以下註解為特定的網域名稱設定最小或者最大TLS版本以及加密套件。

註解

說明

mse.ingress.kubernetes.io/tls-min-protocol-version

指定TLS的最小版本,預設值為TLSv1.0。合法值如下:

  • TLSv1.0

  • TLSv1.1

  • TLSv1.2

  • TLSv1.3

mse.ingress.kubernetes.io/tls-max-protocol-version

指定TLS的最大版本,預設值為TLSv1.3。

nginx.ingress.kubernetes.io/ssl-cipher

指定TLS的加密套件,可以指定多個英文冒號分隔,僅當TLS握手時採用TLSv1.0~1.2生效。

例如,對於網域名稱example.com,設定TLS最小版本為TLSv1.2,最大版本為TLSv1.2。配置如下:

1.19及之後版本叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/tls-min-protocol-version: "TLSv1.2"
    mse.ingress.kubernetes.io/tls-max-protocol-version: "TLSv1.2"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/tls-min-protocol-version: "TLSv1.2"
    mse.ingress.kubernetes.io/tls-max-protocol-version: "TLSv1.2"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

網關與後端服務雙向認證 (mTLS)

MSE Ingress預設使用HTTP協議轉寄請求到後端業務容器。您可以通過使用註解nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"配置MSE Ingress訪問後端服務使用HTTPS協議,但這是單向TLS,也就是說只有MSE Ingress會驗證後端服務提供的認證,且一般後端服務使用的認證需要是權威CA(Certificate Authority)簽發的。另一種更安全的模式是零信任,網關會驗證後端服務的認證是否合法,同樣後端服務也會驗證網關提供的認證是否合法,這就是MTLS,網關與後端服務進行雙向認證。

註解

說明

nginx.ingress.kubernetes.io/proxy-ssl-secret

網關使用的用戶端認證,用於後端服務對網關進行身份認證,格式為secretNamespace/secretName。

nginx.ingress.kubernetes.io/proxy-ssl-name

TLS握手期間使用的SNI。

nginx.ingress.kubernetes.io/proxy-ssl-server-name

開啟或關閉TLS握手期間使用的SNI。

例如,網關與後端服務進行雙向認證,網關使用的secret name為gateway-cert,命名空間為default。配置如下:

1.19版本之後叢集

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-ssl-secret: "default/ateway-cert"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

1.19版本之前叢集

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-ssl-secret: "default/ateway-cert"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80