A Study on the Deployment of Istio Microservice Application Based on ASK

By Wang Feilong (Buwu)

Kubernetes has become the de facto standard for container orchestration systems in the industry. The cloud-native application ecosystem (such as Helm, Istio, Knative, Kubeflow, and Spark on Kubernetes) based on Kubernetes has made Kubernetes a cloud operating system. Against this background, Serverless containers have become one of the evolutionary directions of the existing Container as a Service. On the one hand, adopting the Serverless model fundamentally solves the management complexity of Kubernetes, so users no longer have to suffer from Kubernetes cluster capacity planning, security maintenance, and fault diagnosis. On the other hand, it releases the power of cloud computing, enabling infrastructure to meet security, availability, scalability, and other requirements.

As Alibaba Cloud Serverless Kubernetes platform [1], ASK has significant capabilities (such as O&M-free, second-level elasticity, large Pod capacity, and elastic prediction) and remains a standard Kubernetes platform.

This article verifies the ASK compatibility with standard Kubernetes using Istio on ASK to deploy microservice applications. As the leading solution of Service Mesh, Istio is complex and representative enough on the one hand. On the other hand, it represents the trend of microservice architecture in the cloud-native era, which is of reference significance.

Let's get started!

Create Cluster

Before trying Istio, prepare an ASK cluster. Log in to the Alibaba Cloud console. Select Products and Services > Container Service for Kubernetes. On the left sidebar, select a Cluster to enter the cluster list page. Click the upper-right Create Cluster and configure the cluster parameters:

Cluster Name: hello-istio
Cluster Specification: Pro Edition
Region: USA (Silicon Valley)
Payment Method: Pay As You Go
Kubernetes Version: 1.20.11-aliyun.1
Private Network: Automatic Creation
Service CIDR: 172.21.0.0/20
API Server Access: Standard I (slb.s2.small)
Expose API Server with EIP: Yes
Time Zone: Aisa/Shanghai(UTC +08:00)
Service Discovery: CoreDNS
Use Log Service: Create a New Project

After confirming the configurations, click Create Cluster and wait for the cluster creation to be completed. Istio depends on the DNS services, so the CoreDNS component is installed by default when creating a cluster.

After the cluster is created, go to the Cluster List > hello-istio > Details > Cluster Information > Connection Information, copy the Internet access content to the local /tmp/kube/config file, and run the following commands to configure the kubelet:

$ export KUBECONFIG=/tmp/kube/config

Try Istio

After kubectl is configured, you can install and try Istio in the cluster.

Download Istio

Go to the Istio release page[2] to download the installation file for the operating system. You can also run the following commands to download and extract the latest version:

$ curl -L https://istio.io/downloadIstio | sh -

Since my native ~/bin directory has been put on the PATH, I will copy the extracted Istio directory to the ~/bin directory and create a soft link.

$ cp istio-1.13.3 ~/bin
$ cd ~/bin
$ ln -s istio-1.13.3/bin/istioctl
$ ls -al ~/bin/
total 28
drwxr-xr-x   5 feilong.wfl staff   160  5  4 22:40 ./
drwxr-xr-x+ 95 feilong.wfl staff  3040  5  8 22:30 ../
drwxr-x---   9 feilong.wfl staff   288  4 15 00:48 istio-1.13.3/
lrwxr-xr-x   1 feilong.wfl staff    25  5  4 22:40 istioctl -> istio-1.13.3/bin/istioctl*

If the output of the command istioctl --help is normal, istioctl is configured correctly .

Install Istio

1. In this installation, a demo profile[3] is used, which contains a set of functions specially prepared for testing and configuration combinations for user production or performance testing.

$ istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete

2. Add a tag to the namespace to instruct Istio to automatically inject the Envoy Sidecar proxy when deploying the application:

$ kubectl label namespace default istio-injection=enabled
namespace/default labeled

Sample Application Deployment

1. Deploy Bookinfo sample application[4]:

$ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/platform/kube/bookinfo.yaml
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created

2. Check that all pods are ready:

$ kubectl get pods
NAME                              READY   STATUS    RESTARTS   AGE
details-v1-79f774bdb9-t2jhq       2/2     Running   0          2m54s
productpage-v1-6b746f74dc-qc9lg   2/2     Running   0          2m46s
ratings-v1-b6994bb9-tmbh6         2/2     Running   0          2m51s
reviews-v1-545db77b95-xdhp4       2/2     Running   0          2m49s
reviews-v2-7bf8c9648f-4gn6f       2/2     Running   0          2m48s
reviews-v3-84779c7bbc-jfndj       2/2     Running   0          2m48s

Make sure all pods are in this state: the value of READY is 2/2 and the value of STATUS is Running. Depending on the platform, this operation may take several minutes.

3. Check that the Service is ready:

$ kubectl get services
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
details       ClusterIP   172.21.11.168   <none>        9080/TCP   59s
kubernetes    ClusterIP   172.21.0.1      <none>        443/TCP    33m
productpage   ClusterIP   172.21.0.124    <none>        9080/TCP   51s
ratings       ClusterIP   172.21.9.7      <none>        9080/TCP   57s
reviews       ClusterIP   172.21.13.223   <none>        9080/TCP   55s

4. Make sure the web service is working properly. If the command returns the page title, the application is already running in the cluster.

$ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- curl -s productpage:9080/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>

Open Services to the Public

Now, the BookInfo has been deployed, but it cannot be accessed by the outside world. You need to create an Istio Ingress Gateway to open access, which routes a path to services in the mesh.

Associate an application with an Istio gateway:

$ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/bookinfo-gateway.yaml
gateway.networking.istio.io/bookinfo-gateway created
virtualservice.networking.istio.io/bookinfo created

Ensure there is no problem with the configuration file:

$ istioctl analyze
✔ No validation issues found when analyzing namespace: default.

Determine the Ingress IP and Port

Run the following commands to set the INGRESS_HOST and INGRESS_PORT variables for accessing the gateway:

$ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')

Set the variable GATEWAY_URL and ensure that the IP address and port successfully assign value to the variable:

$ export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT
$ echo "$GATEWAY_URL"
47.88.21.82:80

Verify External Access

Run the following commands to obtain the external access address of the Bookinfo application:

$ echo "http://$GATEWAY_URL/productpage"
http://47.88.21.82:80/productpage

Copy the output address of the commands above to the browser and access it to confirm that Bookinfo is externally accessible. Refresh the page, and you can find that the display style of Book Reviews is constantly changing.

View Dashboards

The dashboard can help you understand the structure of the service mesh, display the topology of the network, and analyze the health status of the mesh.

1. Install Kiali and other plugins and wait for the deployment to complete:

$ kubectl apply -f ~/bin/istio-1.13.3/samples/addons
$ kubectl rollout status deployment/kiali -n istio-system
Waiting for deployment "kiali" rollout to finish: 0 of 1 updated replicas are available...
deployment "kiali" successfully rolled out

2. Access the Kiali dashboard:

$ istioctl dashboard kiali

On the left navigation menu, choose Graph and then select default from the Namespace drop-down list.

The Kiali dashboard shows the overview of Mesh and the relationships between the services of the Bookinfo sample application. It also provides filters to visualize the path of traffic.

Apply Default Destination Rule

Before you use Istio to control the Bookinfo version routing, you must define the available versions in the destination rule[5]. Run the following command to create default destination rules for the Bookinfo services:

$ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/destination-rule-all.yaml
destinationrule.networking.istio.io/productpage created
destinationrule.networking.istio.io/reviews created
destinationrule.networking.istio.io/ratings created
destinationrule.networking.istio.io/details created

Wait a few seconds for the destination rule to take effect. You can view the destination rules with the following command:

$ kubectl get destinationrules
NAME          HOST          AGE
details       details       30s
productpage   productpage   32s
ratings       ratings       31s
reviews       reviews       32s

Route All Traffic to Version v1

Run the following commands to create the Virtual Service to route all traffic to the v1 version of the microservice:

$ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/virtual-service-all-v1.yaml
virtualservice.networking.istio.io/productpage created
virtualservice.networking.istio.io/reviews created
virtualservice.networking.istio.io/ratings created
virtualservice.networking.istio.io/details created

You can test the new configuration by refreshing the /productpage of the Bookinfo application. Note: No matter how many times you refresh, the comments section of the page will not display the star rating. This is because Istio is currently configured to: All flow for the reviews service is routed to version reviews:v1, which does not support access to the star rating service.

Identity-Based Routing

Next, change the routing configuration to implement routing all traffic from a specific user to a specific service version. In this example, all traffic from the user (named Jason) is routed to service review: v2.

Istio does not have any special built-in mechanisms for user identities. In this example, the productpage service adds a custom end-user request header to all HTTP requests to implement identity-based routing.

1. Run the following commands to enable identity-based routing:

$ kubectl apply -f ~/bin/istio-1.13.3/samples/bookinfo/networking/virtual-service-reviews-test-v2.yaml
virtualservice.networking.istio.io/reviews created

2. Ensure the rule has been created:

$ kubectl get virtualservice reviews -o yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"reviews","namespace":"default"},"spec":{"hosts":["reviews"],"http":[{"match":[{"headers":{"end-user":{"exact":"jason"}}}],"route":[{"destination":{"host":"reviews","subset":"v2"}}]},{"route":[{"destination":{"host":"reviews","subset":"v1"}}]}]}}
  creationTimestamp: "2022-05-15T16:05:55Z"
  generation: 1
  name: reviews
  namespace: default
  resourceVersion: "1984849"
  uid: f3bd3dcb-d83c-4a75-9511-1fc9308ca05b
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

3. Log in as the user Jason on the /productpage of the Bookinfo application. Refresh your browser to see the star rating displayed next to each comment:

4. Log in as another user and refresh the browser. You can find that the star rating disappears.

Principles and Limitations

In the data plane, Istio is responsible for coordinating and controlling all the network communication between the microservices by injecting Sidecar proxy (istio-proxy) into the Pod. Istio needs to forward iptables rules to the Pod network to allow the Sidecar proxy (istio-proxy) to intercept the traffic of business containers. In the regular installation model, the iptables rules forwarding is completed by the initialization container istio-init injected by Istio into the Pod. Forwarding iptables rules to the Pod network requires the container to use the two high access capabilities: NET_ADMIN and NET_RAW. These two capabilities in the ASK cluster are affected by ASK Pod Security Policy[6] and ECI Container Security Policy[7].

The ASK Pod Security Policy CAPS is *, indicating that there are no limitations:

$ kubectl get psp
NAME             PRIV   CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
ack.privileged   true   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *

ECI Container Security Policy supports configuring the container security context. The Pod injection template file ~/bin/istio-1.13.3/manifests/charts/istio-control/istio-discovery/files/injection-template.yaml in Istio contains the following code: Not enable the Istio CNI plugin[8], and add NET_ADMIN and NET_RAW:

securityContext:
  allowPrivilegeEscalation: {{ .Values.global.proxy.privileged }}
  privileged: {{ .Values.global.proxy.privileged }}
  capabilities:
{{- if not .Values.istio_cni.enabled }}
    add:
    - NET_ADMIN
    - NET_RAW
{{- end }}
    drop:
- ALL

Therefore, in principle, there is no compatibility issue when using Istio in the current ASK cluster.

Summary

This time, I tried high-complexity software (such as Istio on ASK), and no compatibility issues were found. It is proven that the possibility of compatibility issues is low. Therefore, ASK has excellent compatibility with native Kubernetes. In the future, we will explore other functions of Istio on ASK clusters to further verify the compatibility of ASK with native Kubernetes.