Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Deployment Practices of General DBAudit in the Kubernetes Environment

Deployment Practices of General DBAudit in the Kubernetes Environment

This article introduces how to deploy General DBAudit (a lightweight, low-cost database security solution) in the Kubernetes environment.

By Huolang, from Alibaba Cloud Storage

An Introduction to General DBAudit

General DBAudit is a lightweight, low-cost database security solution provided by Log Service. Log Service offers out-of-the-box audit reports and alert configuration.

As cloud-native technology becomes more mature, more applications are deployed in the cloud-native environment. The dynamic and flexible cloud-native environment poses difficulties for packet capture tools. This article introduces how to deploy this lightweight, low-cost audit solution in the Kubernetes environment.

An Introduction to Network in Kubernetes Scenarios

Container Network

It is well known that the container is a kind of virtualization environment of an operating system. Essentially, it is one or a set of processes isolating from other parts of the system. It is implemented using two core features: Cgroup and Namespace. Cgroup controls resources, and Namespace isolates access.

Each container can have its own separate namespace that has CPU, PID, file systems, network, memory, and other resources. The namespace of all kinds of resources can make applications in the container operate independently.

Network namespace can realize network isolation of the container. A view of a complete network protocol stack can be seen in the namespace of the container. It contains network device interfaces, protocol stacks, etc. Docker containers mainly use Linux veth pair and bridge to realize the communication between containers and hosts and between containers.

As shown in the following figure, the host has two containers, and each container has a network device named eth0. In the container, eth0 communicates with the host by the corresponding veth pair, and veth1 and veth2 are mounted to the bridge Docker0 with an IP address. The network device is eth0 in the container environment. The network devices are veth1, veth2, docker0, and eth0 in the Host environment.

The network packet of applications uses the veth pair in the container environment to achieve ingress and egress. Therefore, it can use the two ends of veth pair, capturing a network packet of containers: one end is eth0 within the container, and the other end is veth1 within the host.

1

Kubernetes Network

The network architecture is more complex in the Kubernetes environment. Container Network Interface (CNI) is used in the continuous container management system and network plug-ins as a standard general interface. Kubernetes needs to deal with communications among different containers and cross-host communications. There are some container network solutions based on CNI, such as Flannel, Calico, Weave, and Terway.

The development process of a typical CNI plug-in is listed below:

2
The picture is from this Article by Xiheng.

Similar to container networks, CNI plug-ins of Kubernetes also need to expose the other end of the veth pair on the host. As a result, eth0 traffic in the pod can exchange with the outside world through veth1. However, eth0 in the container network is in the namespace of a container. A pod has multiple containers that share the same network namespace in the Kubernetes environment. Therefore, on the host, packet capture can be realized on the network traffic within the pod through the end-to-end veth1 of containers.

General DBAudit Practices in the Kubernetes Environment

General DBAudit of Log Service collects databases through the network traffic capture. Please refer to General DBAudit for the specific use method. In the Kubernetes environment, considering requirements in different scenarios, this article takes the Packetbeat subcontract program as an example to provide several cloud-native methods to deploy the packet capture program.

Deployment Methods

According to the network in Kubernetes scenarios mentioned above, there are two ways to realize the packet capture of pods:

  • The packet capture program and the container belong to the same network namespace.
  • The packet capture program is deployed on the host of the container and captures the packet on the one end of the host through the veth pair.

The foregoing two methods also correspond to two kinds of deployment methods in the Kubernetes environment:

  • Sidecar: Since Sidecar and business containers are naturally in the same pod and share the network namespace, the packet capture program deployed as Sidecar can capture the traffic of the business containers.

3

  • Daemonset: Since Daemonset is deployed in each host and has the capability for network monitor, it can capture the packet of the business containers through one end of the veth pair on the host.

4

Audit Data Reporting

According to the data above, the packet capture program can be deployed by the method of Sidecar or Daemonset. The packet capture program can choose Packetbeat that may be converted to various types of output. At this time, you can upload the captured traffic data through Kafka to Logstore of Log Service. Please refer to Use the Kafka Protocol to Upload Logs for the specific configuration.

Deployment Practices

Next, the article will introduce how to deploy Sidecar and Daemonset. Both are used for the deployment of the packet capture program. You can select one of them according to business requirements. The deployment of both depends on the configuration of ConfigMap, which includes the following process:

  • The premise is that the task configuration of general DBAudit is enabled. Please refer to General DBAudit for the configuration.
  • The configuration of ConfigMap mainly includes the configuration of packetbeat.yml.
  • You can choose Daemonset and Sidecar modes in Step 2 for the deployment of packetbeat.
  • Please refer to the GitHub link for the following configuration files.

Step 1

Edit packetbeat.yml

  • The packetbeat.flows is used to capture all traffic from the network device. If you do not need the packetbeat.flows, you can set the enabled:false to reduce the transmission of data.
  • Please refer to Use Kafka Protocol to Upload Logs to change the configuration of output.kafka. It is required to replace {project}, {region}, {ak}, and {sk}.
  • Please refer to the official configuration of packetbeat: Configure Packetbeat for additional configuration details.
packetbeat.interfaces.device: any
# Set `enabled: false` or comment out all options to disable flows reporting.
packetbeat.flows:
  timeout: 30s
  period: 10s
packetbeat.protocols:
- type: mysql
  ports: [3306,3307]
output.kafka: 
  hosts: ["{project}.{region}.log.aliyuncs.com:10012"] 
  username: "{project}" 
  password: "{ak}#{sk}" 
  ssl.certificate_authorities: 
  topic: 'general-db-logstore' 
  partition.round_robin: 
    reachable_only: false 
  required_acks: 1 
  compression: gzip 
  max_message_bytes: 1000000

Create a ConfigMap from Files

kubectl create configmap packetbeat-config --from-file=packetbeat.yml=packetbeat.yml

Alternative Step 2: Daemonset Mode

Configure a packetbeat-daemonset.yaml file

  • Enable the hostNetwork, allowing Daemonset to see the network device on the host machine
  • Add the capability of NET_ADMIN to allow Daemonset to monitor the traffic of the network device
  • Limit the occupancy of CPU and memory through resources
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: packetbeat-daemonset
  namespace: default
spec:
  selector:
    matchLabels:
      app: packetbeat-daemonset
  template:
    metadata:
      name: packetbeat-daemonset
      labels:
         app: packetbeat-daemonset
    spec:
      restartPolicy: Always
      hostNetwork: true
      containers:
      - name: packetbeat
        image: docker.elastic.co/beats/packetbeat:8.1.0
        resources:
          limits:
            cpu: 500m
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        command: ["/bin/sh", "-c"]
        args: ["/usr/share/packetbeat/packetbeat --strict.perms=false -c /etc/packetbeat/config/packetbeat.yml"]
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
        volumeMounts:
        - name: packetbeat-config
          mountPath: /etc/packetbeat/config
      terminationGracePeriodSeconds: 30
      volumes:
      - name: packetbeat-config
        configMap:
          name: packetbeat-config
          items:
          - key: packetbeat.yml
            path: packetbeat.yml

Create Daemonset

kubectl apply -f packetbeat-daemonset.yaml

Alternative Step 2: Sidecar Mode

Create a packetbeat-sidecar.yaml file

The mysql-test container is a test container. You can modify it based on business containers.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: packetbeat-sidecar
  namespace: default
spec:
  selector:
    matchLabels:
      app: packetbeat-sidecar
  template:
    metadata:
      name: packetbeat-sidecar
      labels:
         app: packetbeat-sidecar
    spec:
      restartPolicy: Always
      containers:
      - name: mysql-test
        image: imega/mysql-client
        command: ["/bin/sh", "-c"]
        args: ["mysql --host='xxx' --user=xxx --password='xxx' --database=mysql --execute='show tables;'; sleep 6000"]
      - name: packetbeat-sidecar
        image: docker.elastic.co/beats/packetbeat:8.1.0
        command: ["/bin/sh", "-c"]
        args: ["/usr/share/packetbeat/packetbeat --strict.perms=false -c /etc/packetbeat/config/packetbeat.yml"]
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
        volumeMounts:
        - name: packetbeat-config
          mountPath: /etc/packetbeat/config
      
      volumes:
      - name: packetbeat-config
        configMap:
          name: packetbeat-config
          items:
          - key: packetbeat.yml
            path: packetbeat.yml

Create Sidecar

kubectl apply -f packetbeat-sidecar.yaml

References

kafka Storage Log Service Containers Kubernetes Sidecar Container Network Interface DBAudit
0 0 0
Share on

Read previous post:

Use Terraform to Implement SLS Alert

Read next post:

Blogs of the Week – Ep. 16

Alibaba Cloud Community

1,079 posts | 265 followers

You may also like

Comments

Alibaba Cloud Community

1,079 posts | 265 followers

Related Products

More Posts by Alibaba Cloud Community

See All