Container Service for Kubernetes:Use ACK CoreDNS DNSTAP Analyser to diagnose DNS resolution errors

Step 1: Install ACK CoreDNS DNSTAP Analyser

1. Log on to the ACK console.

2. In the left-side navigation pane of the ACK console, choose Marketplace > Marketplace.

3. On the App Catalog page, enter ack-coredns-dnstap-analyser in the search box and click the search icon. In the search result, find and click ack-coredns-dnstap-analyser.

4. On the ack-coredns-dnstap-analyser page, click Deploy.

5. In the Deploy wizard, select a cluster and namespace, and then click Next. On the Parameters wizard, configure the parameters based on your business requirements and click OK.

Step 2: Configure the DNSTAP plug-in in CoreDNS

CoreDNS is pre-installed with the DNSTAP plug-in, which sends DNSTAP messages. The DNS messages that are sent and received by the DNSTAP plug-in can be forwarded to ACK CoreDNS DNSTAP Analyser. Before you can use ACK CoreDNS DNSTAP Analyser to diagnose CoreDNS resolution errors, you must configure and enable the DNSTAP plug-in in CoreDNS. After the DNSTAP plug-in is configured, CoreDNS forwards the DNS messages that are sent and received by the DNSTAP plug-in to ACK CoreDNS DNSTAP Analyser for automated diagnostics.

1. Run the following command to query and record the cluster IP address of ACK CoreDNS DNSTAP Analyser in the kube-system namespace. This IP address is used to configure the DNSTAP plug-in. In this example, the IP address is 172.21.0.10.

   kubectl -n kube-system get svc dnstap-analyser

2. Run the following command to modify the ConfigMap of CoreDNS:

   kubectl -n kube-system edit cm coredns -o yaml

3. Modify the default settings of CoreDNS based on the comments in the following snippet.

   In the ConfigMap, you must enable the DNSTAP plug-in and set its destination address to the cluster IP address of ACK CoreDNS DNSTAP Analyser. This way, the DNS messages that are sent and received by the DNSTAP plug-in can be forwarded to ACK CoreDNS DNSTAP Analyser.

   Corefile: |
           .:53 {
               #Details are not shown.
               ready
   
               # Add the following line. Replace 172.21.0.10 with the actual cluster IP address of CoreDNS DNSTAP Analyser. 
               dnstap tcp://172.21.0.10:6000 full
   
               kubernetes cluster.local in-addr.arpa ip6.arpa {
               #Details are not shown.
               }
               #Details are not shown.
               reload
               loadbalance
           }

4. Run the following command to query all pods that run CoreDNS:

   kubectl -n kube-system get pod | grep coredns

   Expected output:

   coredns-7d56l         1/1     Running   0          30m
   coredns-s7m2t         1/1     Running   0          30m

5. Run the following command to show the CoreDNS log:

   kubectl -n kube-system logs -f --tail=500 coredns-7d56l

   Note

   Replace coredns-7d56l with the name of a pod that is returned in preceding step 4.

   Wait for 1 minute. If the following output is returned and no exceptions occur, the configuration is modified.

   [INFO] Reloading complete

Step 3: View the log of ACK CoreDNS DNSTAP Analyser

1. Run the following command to query all pods that run ACK CoreDNS DNSTAP Analyser:

   kubectl -n kube-system get pod | grep dnstap-analyser

   Expected output:

   dnstap-analyser-bbdf879-g****         1/1     Running   0          30m

2. Run the following command to show the log of ACK CoreDNS DNSTAP Analyser:

   kubectl -n kube-system logs -f dnstap-analyser-bbdf879-g****

   Note

   Replace dnstap-analyser-bbdf879-gpfkm with the name of a pod that is returned in preceding step 1.

Step 4: Analyze the log of ACK CoreDNS DNSTAP Analyser

Step 5: (Optional) Uninstall ACK CoreDNS DNSTAP Analyser

After you identify the cause, we recommend that you uninstall ACK CoreDNS DNSTAP Analyser to avoid unnecessary costs. To uninstall ACK CoreDNS DNSTAP Analyser, perform the following steps:

1. Delete the line that contains dnstap from the ConfigMap of CoreDNS:

   1. Run the following command to modify the ConfigMap of CoreDNS:

      kubectl -n kube-system edit cm coredns -o yaml

   2. Delete the line that contains dnstap from the ConfigMap of CoreDNS based on the following content:

      Corefile: |
              .:53 {
                  #Details are not shown.
                  ready
      
                  # Delete the following line that you added earlier.
                  dnstap tcp://10.10.10.10:6000 full
      
                  kubernetes cluster.local in-addr.arpa ip6.arpa {
                  #Details are not shown.
                  }
                  #Details are not shown.
                  reload
                  loadbalance
              }

   3. Run the following command to query all pods that run CoreDNS:

      kubectl -n kube-system get pod | grep coredns

      Expected output:

      coredns-7d56l         1/1     Running   0          30m
      coredns-s7m2t         1/1     Running   0          30m

   4. Run the following command to show the CoreDNS log:

      kubectl -n kube-system logs -f --tail=500 coredns-7d56l

      Note

      Replace coredns-7d56l with the name of a pod that is returned in preceding step c.

      Wait for 1 minute. If the following output is returned and no exceptions occur, the configuration is modified.

      [INFO] Reloading complete

2. Uninstall ACK CoreDNS DNSTAP Analyser.

   1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

   2. On the Clusters page, click the name of the cluster that you want to manage and choose Applications > Helm in the left-side navigation pane.

   3. In the left-side navigation pane of cluster details page, choose Applications > Helm.

   4. On the Helm page, find ack-coredns-dnstap-analyser and click Delete in the Actions column. Follow the on-screen instructions to delete ack-coredns-dnstap-analyser: