Deploy and manage the ack-kserve component in an ACK cluster - Container Service for Kubernetes

KServe is an open source project that is designed to simplify the process of deploying and running machine learning models on Kubernetes. KServe supports multiple machine learning frameworks and provides auto scaling capabilities. KServe provides declarative API operations to deploy models based on the YAML files that you define. This way, KServe simplifies the configuration and management of models deployed as services. This topic describes how to deploy and manage the ack-kserve component in a Container Service for Kubernetes (ACK) cluster and provides a solution to a common issue.

Prerequisites

An ACK Pro cluster that runs Kubernetes 1.22 or later is created. For more information, see Create an ACK managed cluster.
The NGINX Ingress controller is installed. For more information, see Manage the NGINX Ingress controller.

Step 1: Install the cert-manager component

If the cert-manager component is installed in your ACK cluster, skip this step.

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
On the Helm page, click Deploy. In the Deploy panel, set the Application Name parameter to cert-manager. In the Chart section, search for cert-manager and select it. Then, click Next. In the Confirm message, click Yes. The chart will be installed in the cert-manager namespace by default.
In the Parameters step, check the value of the Chart Version parameter and the information displayed in the Parameters field and click OK.
After the cert-manager component is deployed, the details page of the cert-manager component appears. You can view the information about the cert-manager component on the Basic Information tab.

Step 2: Install the ack-kserve component

By default, the ack-kserve component is deployed in Raw Deployment mode and integrated with the Nginx Ingress controller.

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
On the Helm page, click Deploy. In the Deploy panel, set the Application Name parameter to ack-kserve. In the Chart section, search for ack-kserve and select it. Then, click Next. In the Confirm message, click Yes. The chart will be installed in the kserve namespace by default.
In the Parameters step, check the value of the Chart Version parameter and the information displayed in the Parameters field and click OK.
After the ack-kserve component is deployed, the details page of the ack-kserve component appears. You can view the information about the ack-kserve component on the Basic Information tab.
Check whether the ack-kserve component is running.
Run the following command to check whether the pod in the kserve namespace is in the running state:
```
kubectl get pod -n kserve
```
If running is returned for the STATUS parameter in the output as expected, the ack-kserve component is installed.

Step 3: (Optional) View or update the ack-kserve ️component

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
View the details of the ack-kserve component.
On the Helm page, find the ack-kserve component and click View Details in the Actions column. On the details page of the ack-kserve component, view the information on the Basic Information, Parameters, and History tabs.
Update the ack-kserve component.
On the Helm page, find the ack-kserve component and click Update in the Actions column. In the Update Release panel, change the version of the component and modify parameter settings.

Step 4: (Optional) Uninstall the ack-kserve component

Delete the KServe custom resources (CRs) and custom resource definitions (CRDs) in the cluster in which the ack-kserve component is installed before you uninstall the ack-kserve component. This prevents resource waste.
Important
Make sure that the CRs and CRDs are no longer required for your business before you delete the CRs and CRDs. If you delete a CRD, the CRs that are dependent on the CRD are also deleted and cannot be restored.
1. Delete all KServe CRs in the cluster after you make sure that you no longer require them. You can run the following commands to delete the CRs:
```
# View all CRs in the cluster. 
kubectl get isvc --all-namespaces

# Save all CRs in the cluster. 
kubectl get isvc --all-namespaces -oyaml > isvc.yaml.bak

# Delete the CRs after you make sure that you no longer require them. 
kubectl delete isvc --all
```
2. Delete the KServe CRDs in the cluster.
  Before you delete a CRD, make sure that you delete all CRs that are dependent on the CRD. Otherwise, the CRD fails to be deleted.
```
kubectl delete crd clusterservingruntimes.serving.kserve.io
kubectl delete crd clusterstoragecontainers.serving.kserve.io
kubectl delete crd inferencegraphs.serving.kserve.io
kubectl delete crd inferenceservices.serving.kserve.io
kubectl delete crd predictors.serving.kserve.io
kubectl delete crd servingruntimes.serving.kserve.io
kubectl delete crd trainedmodels.serving.kserve.io
```
Uninstall the ack-kserve component.
1. Log on to the ACK console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
3. On the Helm page, find the ack-kserve component and click Delete in the Actions column. In the Delete dialog box, select or clear Clear Release Records and click OK.

FAQ

Issue: The following error message appears when the ack-kserve component is being installed: failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": tls: failed to verify certificate: x509: certificate signed by unknown authority.

Cause: The ack-kserve component is highly dependent on the cert-manager component. If the cert-manager component is not installed or is not ready in your cluster, the preceding error occurs when you install the ack-kserve component.

Solution:

Run the following command to check whether the cert-manager component is installed in your cluster:
```
kubectl get crd |grep certificates.cert-manager.io
```
If the following output is returned, the cert-manager component is installed in the cluster.
```
certificates.cert-manager.io                         2024-05-06T07:09:17Z
```
If no CRD of the cert-manager component is available in your cluster, install the cert-manager component. For more information, see the Step 1: Install the cert-manager component section of this topic.
Run the following command to check whether the cert-manager component is ready:
```
kubectl -n cert-manager get po
```
If the following output is returned, the pods of the cert-manager component are ready.
```
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-7f4bb44d5b-jrrfn              1/1     Running   0          23h
cert-manager-cainjector-79544456cc-qp5pp   1/1     Running   0          23h
cert-manager-webhook-f74ccb647-7m5dt       1/1     Running   0          23h
```
If all pods are in the Ready state, you can uninstall the ack-kserve component and then reinstall it. For more information, see the "Uninstall the ack-kserve component" section of this topic.