By Nandor Kracser, Alibaba Cloud MVP
Last year Alibaba Cloud joined CNCF and announced plans to create their own Kubernetes service - Alibaba Cloud Kubernetes (ACK). The service was launched more than a year ago, with its stated objective to make it easy to run Kubernetes on Alibaba Cloud without needing to install, operate, and maintain a Kubernetes control plane.
At Banzai Cloud we are committed to providing support for Kubernetes on all major cloud providers, thus one of our priorities was to enable Alibaba Cloud's Container Service for Kubernetes in Pipeline and take the DevOps experience to the next level by turning ACK into a feature-rich enterprise-grade application platform. At the same time, some Pipeline users are working in highly regulated markets with region based deployment constraints (e.g. the ability to deploy applications in the China region).
Given some Alibaba Cloud and Kubernetes knowledge, Alibaba ACK is already a smooth "I need a Kubernetes cluster" experience and Pipeline builds on top of this with additional features, such as:
When it comes to creating Alibaba ACK clusters, Pipeline reduces users' concerns to only two things:
The following diagram shows the various components that Pipeline pre-loads into the created cluster to provide the above mentioned rich feature set to users:
We have a Postman collection which also contains the Alibaba examples shown below for calling Pipeline, but for visual purposes cURL commands are shown in this blog post.
The Alibaba credentials that will be used to create the Alibaba ACK cluster must be registered in Pipeline first:
curl -X POST \
http://{{url}}/api/v1/orgs/{{orgId}}/secrets \
-H 'Authorization: Bearer {{token}}' \
-H 'Content-Type: application/json' \
-d '{
"name": "my-alibaba-secret",
"type": "alibaba",
"values": {
"ALIBABA_ACCESS_KEY_ID": "{{your_alibaba_access_key_id}}",
"ALIBABA_ACCESS_KEY_SECRET": "{{your_alibaba_access_key_secret}}"
}
}'your_alibaba_access_key_id: Alibaba access key id of the RAM user used for creating Alibaba ACK cluster
your_alibaba_access_key_secret: Alibaba secret access key of the RAM user used for creating Alibaba ACK cluster
The above REST request stores the passed in Alibaba credentials as a secret in Vault using Bank-Vaults for secure storage. Security is an important consideration for us and we take it seriously. You can find out more about how we handle security here.
The response will contain the id of the secret. We will use this id in subsequent requests where a secret_id is required and the credential is securely injected into the Alibaba client from Vault.
curl -X POST \
http://{{url}}/api/v1/orgs/{{orgId}}/clusters \
-H 'Authorization: Bearer {{token}}' \
-H 'Content-Type: application/json' \
-d '{
"name":"alibabacluster-{{username}}-{{$randomInt}}",
"location": "eu-central-1",
"cloud": "alibaba",
"secretId": "{{secret_id}}",
"properties": {
"acsk": {
"regionId": "eu-central-1",
"zoneId": "eu-central-1a",
"masterInstanceType": "ecs.sn1ne.large",
"masterSystemDiskCategory": "cloud_efficiency",
"nodePools": {
"pool1": {
"count": 1,
"image": "centos_7",
"instanceType": "ecs.sn1ne.large",
"systemDiskCategory": "cloud_efficiency"
}
}
}
}
}
'If we look at the fields the user has to provide in the create cluster request, we can see that the user is required to think of, and provide, only the location of the Alibaba ACK cluster for it to be created in, the size of the cluster, and the type of the nodes and features to be enabled. The rest is taken care of by Pipeline.
secret_id - the id of the secret in Vault that contains the Alibaba credentials to be used for provisioning all the resources needed for creating the Alibaba ACK cluster
location - the location of the Alibaba ACK cluster
nodePools - lists node pools that will be available for the worker nodes of the cluster (currently only one node pool is supported on Alibaba)
masterInstanceType - specifies which instance type to use for Kubernetes masters (3 masters will be created)
masterSystemDiskCategory - specifies which disk type to use for Kubernetes masters
Beside cluster autoscaling Pipeline also makes it possible for users to enable Horizontal Pod Autoscaling for deployments. We store all Alibaba ACK and deployment metrics in Prometheus which Horizontal Pod Autoscaler can feed metrics from to drive the autoscaling. You can find out more about the cluster autoscaling created with Pipeline by reading through these posts.
We can verify on the Alibaba Cloud console whether the cluster was created.
Currently, the Alibaba console only shows the state of the Kubernetes API server. We can verify the list of ECS instances that make up the cluster by searching for instances that have the cluster name in their name.
In order to start using the cluster with kubectl we need the cluster config. The cluster config can be retrieved through Pipeline using:
curl -X GET \
http://{{url}}/api/v1/orgs/{{orgId}}/clusters/{{clusterId}/config \
-H 'Authorization: Bearer {{token}}' \
-H 'Content-Type: application/json' \Save the retrieved cluster config to a file and point $KUBECONFIG environment variable to it so that the kubectl command will use this config by default.
Now we can verify the list of nodes that have joined the Kubernetes cluster using kubectl:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
eu-central-1.i-gw8i1a8sjmnmcgicj1ek Ready master 1h v1.10.4
eu-central-1.i-gw8i1a8sjmnmcmffv7bx Ready <none> 1h v1.10.4
eu-central-1.i-gw8i6d9k4vn8g9t012uk Ready master 1h v1.10.4
eu-central-1.i-gw8i6d9k4vn8gbs154tp Ready master 1h v1.10.4Similarly we can check running pods:
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default pipeline-traefik-59f9b59859-44fbw 1/1 Running 0 18m
kube-system alicloud-application-controller-558549f66d-6fhr2 1/1 Running 0 22m
kube-system alicloud-disk-controller-86bc486f98-d2zxh 1/1 Running 0 22m
kube-system alicloud-monitor-controller-67c84c6bb9-5452x 1/1 Running 0 22m
kube-system cloud-controller-manager-lv65t 1/1 Running 0 22m
kube-system cloud-controller-manager-sjkkq 1/1 Running 0 22m
kube-system cloud-controller-manager-wjkgf 1/1 Running 0 22m
kube-system coredns-7997f8864c-f6v2r 1/1 Running 0 27m
kube-system coredns-7997f8864c-xrqcs 1/1 Running 0 27m
kube-system dashboard-kubernetes-dashboard-5f657df58f-gzb5z 1/1 Running 0 18m
kube-system default-http-backend-5f89bdffd5-xr5qs 1/1 Running 0 22m
kube-system flexvolume-4z78x 1/1 Running 0 22m
kube-system flexvolume-dw2kx 1/1 Running 0 22m
kube-system flexvolume-gfzj9 1/1 Running 0 20m
kube-system flexvolume-zvbcb 1/1 Running 0 22m
kube-system heapster-6c46f88458-ftvqh 1/1 Running 0 22m
kube-system kube-apiserver-eu-central-1.i-gw8i1a8sjmnmcgicj1ek 1/1 Running 0 27m
kube-system kube-apiserver-eu-central-1.i-gw8i6d9k4vn8g9t012uk 1/1 Running 0 24m
kube-system kube-apiserver-eu-central-1.i-gw8i6d9k4vn8gbs154tp 1/1 Running 0 21m
kube-system kube-controller-manager-eu-central-1.i-gw8i1a8sjmnmcgicj1ek 1/1 Running 0 26m
kube-system kube-controller-manager-eu-central-1.i-gw8i6d9k4vn8g9t012uk 1/1 Running 0 25m
kube-system kube-controller-manager-eu-central-1.i-gw8i6d9k4vn8gbs154tp 1/1 Running 0 21m
kube-system kube-flannel-ds-4vhfg 2/2 Running 1 20m
kube-system kube-flannel-ds-fgtbf 2/2 Running 1 22m
kube-system kube-flannel-ds-kpr7x 2/2 Running 1 22m
kube-system kube-flannel-ds-vsqll 2/2 Running 1 22m
kube-system kube-proxy-master-784g5 1/1 Running 0 22m
kube-system kube-proxy-master-8rg4s 1/1 Running 0 22m
kube-system kube-proxy-master-wgg8c 1/1 Running 0 22m
kube-system kube-proxy-worker-pbwd2 1/1 Running 0 20m
kube-system kube-scheduler-eu-central-1.i-gw8i1a8sjmnmcgicj1ek 1/1 Running 0 28m
kube-system kube-scheduler-eu-central-1.i-gw8i6d9k4vn8g9t012uk 1/1 Running 0 25m
kube-system kube-scheduler-eu-central-1.i-gw8i6d9k4vn8gbs154tp 1/1 Running 0 21m
kube-system monitoring-influxdb-999f4f948-lwcn6 1/1 Running 0 22m
kube-system nginx-ingress-controller-6b6687fdd6-fx4b6 1/1 Running 0 22m
kube-system nginx-ingress-controller-6b6687fdd6-w8kjn 1/1 Running 0 22m
kube-system tiller-deploy-b67846f96-nn7m6 1/1 Running 0 22m
pipeline-infra pipeline-hpa-hpa-operator-66bfb9866f-rmncf 1/1 Running 0 18mThe whole of Pipeline is open-source and during the Alibaba integration period other open-source projects have been impacted as well:
One of the most important additions contributed in the interest of Pipeline integration was the Vault native Alibaba Object Storage Service support, which enables users to store secrets backed by Object Storage Service (OSS) through its native API. You can read more about how we run Vault on Alibaba with full automatic unsealing support and with Alibaba OSS storage backend in this previous blog post. We'd like to highlight the quick resolution of issues we faced during integration work by the Alibaba engineering team - they regularly managed to fix issues on the Golang SDK in less than one business day.
With Productinfo we can query the available instance types per region and their prices through a REST API. This service provides the necessary information for Telescopes - a cluster instance types and full cluster layout recommender engine. Telescopes, which is based on predefined resource requirements such as CPU, memory, GPU or network performance, and recommends a diverse set of cost-optimized Kubernetes node pool layouts. To follow the progress of Alibaba support in Productinfo please subscribe to this issue.
For users, it is easier to think in terms of the resource needs of their deployments/applications and not to deal with the cloud provider specific cluster layout at all. The initially provided cluster layout may not fit the resource needs of the deployed applications over time and the cluster layout must be changed dynamically. Enter autoscalers, which receive cluster and application metrics from Prometheus to scale the cluster up/down appropriately while taking into account recommendations from Telescopes.
Please note that the Alibaba Cloud Kubernetes support for some advanced scenarios is still a work in progress, and it can be tracked here.
Source: https://banzaicloud.github.io/blog/pipeline-alibaba-support/
How to Install NVIDIA GPU Cloud Virtual Machine Image on Alibaba Cloud
21 posts | 12 followers
FollowAlibaba Clouder - September 7, 2020
Alibaba Clouder - June 10, 2020
Data Geek - April 29, 2024
Data Geek - April 11, 2024
Alibaba Clouder - April 8, 2020
Apache Flink Community - May 10, 2024
21 posts | 12 followers
Follow
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn More
ACK One
Provides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resources
Learn More
Cloud-Native Applications Management Solution
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.
Learn More
Container Registry
A secure image hosting platform providing containerized image lifecycle management
Learn MoreMore Posts by Marketplace