Elastic Container Instance:Manually deploy a VNode in a self-managed Kubernetes cluster

Prerequisites

• A Kubernetes cluster of version 1.13 to 1.30 is deployed.

• If your Kubernetes cluster is deployed in a data center, make sure that the data center and the virtual private cloud (VPC) to which the Elastic Container Instance-based pods belong are connected with each other by using Express Connect, Smart Access Gateway, or VPN Gateway. For more information, see the following topics:

  • What is Express Connect?

  • What is SAG?

  • What is VPN Gateway?

Preparations

Before you deploy a VNode in a self-managed Kubernetes cluster, prepare the parameter information that is required for creating a VNode and understand the permissions that are required to deploy the VNode in the cluster. The following table describes the parameters that are required to create a VNode.

To create a VNode, you must configure standard permissions for native Kubernetes nodes and additional permissions for the VNode, as shown in the following table.

Configure the cluster to generate a kubeconfig certificate

To deploy a VNode in a self-managed Kubernetes cluster, you must use a kubeconfig certificate. Before you create a VNode, you must generate a kubeconfig certificate for the VNode.

1. Connect to your Kubernetes cluster.

2. Download and execute a configuration script to generate a kubeconfig certificate.

   In TLS bootstrapping or ServiceAccount mode, run an appropriate command to generate a kubeconfig certificate based on the version of your Kubernetes cluster. TLS is the acronym of Transport Layer Security.

   TLS bootstrapping

   If you have security requirements, we recommend that you use the TLS bootstrapping mode. In this mode, the vnode-approver component is automatically deployed. The vnode-approver component is used to approve certificate signing requests (CSRs) that are sent by VNodes. vnode-approver has open source code on GitHub. For more information, see vnode-approver.

   • If the cluster version is from 1.14 to 1.22, run the following command to generate a kubeconfig certificate:

     curl -fsSL https://eci-docs.oss-cn-beijing.aliyuncs.com/vnode/vnode-deploy.sh | bash -s -- bootstrap

   • If the cluster version is 1.13, run the following command to generate a kubeconfig certificate:

   • For clusters of v1.13, the endpoint resource needs to select a leader. You must configure additional permissions by setting EXTRA_PRIVILEDGE=true in the script.

   curl -fsSL https://eci-docs.oss-cn-beijing.aliyuncs.com/vnode/vnode-deploy.sh | env EXTRA_PRIVILEDGE=true bash -s -- bootstrap

   ServiceAccount

   This mode does not support certificate rotation. When you generate a kubeconfig certificate, make sure that the token has a long enough or permanent validity period to prevent VNode failure due to expiration of the kubeconfig.

   • If the cluster version is from 1.14 to 1.22, run the following command to generate a kubeconfig certificate:

     curl -fsSL https://eci-docs.oss-cn-beijing.aliyuncs.com/vnode/vnode-deploy.sh | bash -s -- common

   • If the cluster version is 1.13, run the following command to generate a kubeconfig certificate:

     For clusters of v1.13, the endpoint resource needs to select a leader. You must configure additional permissions by setting EXTRA_PRIVILEDGE=true in the script.

     curl -fsSL https://eci-docs.oss-cn-beijing.aliyuncs.com/vnode/vnode-deploy.sh | env EXTRA_PRIVILEDGE=true bash -s -- common

3. View the content of the kubeconfig.

   The generated kubeconfig is named vnode-config. Run the following command to view the content of the kubeconfig:

   cat vnode-config

   When you view the content of the kubeconfig, take note of the following items:

   • Make sure that the VPC to which the VNode belongs can be connected to the server. To ensure the VPC can be connected to the server, the VNode and the server must reside in the same VPC, or the connection between the VPC and the server is established.

     • If the server address is an IP address, make sure that you can access the IP address from the VPC in which the VNode resides.

     • If the server address is a domain name, make sure that the domain name can be resolved in an Alibaba Cloud VPC. In addition, from the VPC in which the VNode resides, you must be able to access the IP address to which the domain name points.

   • If the content of certificate-authority-data is empty, check whether the PATH field is used in ~/.kube/config.

   The following code provides a sample content of the kubeconfig.

   TLS bootstrapping

   apiVersion: v1
   kind: Config
   current-context: kubernetes-admin@kubernetes
   contexts:
   - name: kubernetes-admin@kubernetes
     context:
       cluster: kubernetes-admin@kubernetes
       user: vnode
       namespace: kube-system
   clusters:
   - name: kubernetes-admin@kubernetes
     cluster:
       certificate-authority-data: "*****************************************=="
       server: https://10.16.XX.XX:6443
   users:
   - name: vnode
     user:
       token: ******.****************

   ServiceAccount

   apiVersion: v1
   kind: Config
   current-context: kubernetes-admin@kubernetes
   contexts:
   - name: kubernetes-admin@kubernetes
     context:
       cluster: kubernetes-admin@kubernetes
       user: vnode
       namespace: kube-system
   clusters:
   - name: kubernetes-admin@kubernetes
     cluster:
       certificate-authority-data: "*****************************************=="
       server: https://10.16.XX.XX:6443
   users:
   - name: vnode
     user:
       token: ***********************************************

Create a VNode

Procedure

You can use the Elastic Container Instance console or call an API operation to create a VNode.

Prevent DaemonSets from being scheduled to the VNode

DaemonSets cannot run on VNodes because VNodes are not real nodes. After you create a VNode, you need to modify the DaemonSet in kube-proxy and configure nodeAffinity to prevent DaemonSets from being scheduled to the VNode.

1. Modify the configurations of the DaemonSet.

   kubectl -n kube-system edit ds kube-proxy

2. Configure nodeAffinity.

   Add the following YAML content to spec > template > spec:

   affinity:
     nodeAffinity:
       requiredDuringSchedulingIgnoredDuringExecution:
         nodeSelectorTerms:
         - matchExpressions:
           - key: type
             operator: NotIn
             values:
             - virtual-kubelet

Schedule pods to the VNode

After you create a VNode, you can use one of the following methods to schedule pods to the VNode. Then, you can run the pods as elastic container instances in the VNode.

• Manual scheduling

  You can configure the nodeSelector and tolerations parameters or specify the nodeName parameter to schedule pods to the VNode. For more information, see Schedule pods to a VNode.

• Automatic Scheduling

  After you deploy the eci-profile component, you can specify the Selector parameter. This way, the system automatically schedules pods that meet the conditions specified by Selector to the VNode. For more information, see Use eci-profile to schedule pods to a VNode.