Applications, containers, and infrastructure components in a cluster continuously generate a large amount of scattered log data, which complicates log collection and management. Container Service for Kubernetes (ACK) clusters are integrated with Simple Log Service (SLS) to provide a solution that simplifies log management.
For more information about the features and terms of SLS, see What is Simple Log Service? and Terms.
Introduction to the log collection component
The Logtail component is an agent provided by SLS to collect logs from ACK clusters. You can install this component in the ACK console to collect multiple types of logs in an efficient manner, including text logs, binlogs, and HTTP data. This component supports log data collection for standard containers, Kubernetes clusters, and other container clusters. If you want to improve ease of use, integration convenience, resource efficiency, and cost control, we recommend that you use Logtail to collect logs.
For more information about the differences between Logtail and open-source components, see What are the differences among log collection agents?.
Log collection
The following section describes the log collection method and process.
Collection process
In ACK clusters, Logtail is serves as a bridge to securely collect and transmit various data to SLS for storage and analysis. This method simplifies log collection management and improves flexibility and efficiency. The following figure shows the process and architecture of using Logtail to collect logs.
The following section describes the processes of using Logtail:
Deploy a Logtail container. For more information, see Install Logtail components in an ACK cluster.
When you install Logtail, the AliyunLogConfig CRD is created and the alibaba-log-controller component is installed by default to associate the AliyunLogConfig CRD with the collection configurations in SLS.
Logtail collects data based on the configuration. Logtail obtains and uses the collection configuration.
After you install Logtail, you must create a collection configuration. You can use the console, CRD configuration, and environments variables to collect logs. For more information, see Collect text logs from Kubernetes containers in DaemonSet mode, Collect stdout and stderr from Kubernetes containers in DaemonSet mode (old version), and Collect text logs from Kubernetes containers in Sidecar mode.
Logtail uploads the collected log data to SLS in real time.
Collection method
In a Kubernetes cluster, you can use the Sidecar or DaemonSet mode to collect logs. Each method has different collection processes and use scenarios.
DaemonSet mode: Logtail is installed in the kube-system namespace. The name of the DaemonSet is
logtail-ds. Logtail of each node collects data, including standard output and files, from all pods that run on the node. For more information, see Collect text logs from Kubernetes containers in DaemonSet mode.Sidecar mode: In addition to the primary container, the node runs the Sidecar container of Logtail. You must manually create the Logtail container in the business pod. The Logtail container and primary container share the log volume. When you use a virtual node, use the Sidecar mode. For more information, see Collect text logs from Kubernetes containers in Sidecar mode.
Category | DaemonSet mode | Sidecar mode |
Category | DaemonSet mode | Sidecar mode |
Scenarios | A cluster has clear log classification and a single function.
| Large, hybrid clusters If you require different log collection configurations for each pod, you can use this method to collect logs. However, a single pod in this mode generates a large amount of logs and consumes a large amount of resources. |
Resource usage | Run a Logtail container on each node. | Run a Logtail container on each pod. |
Deployment and O&M | You need to only maintain the DaemonSet. | A Logtail container must be deployed for each pod that needs to collect logs. |
Log type | Standard output and text logs. | Text logs. |
Supported log types
In an ACK cluster, you can collect log data from different sources and securely transmit the log data to SLS for storage and analysis.
Log type | Description | Reference |
Log type | Description | Reference |
Application (container) logs | Log data from containers of the ACK cluster can be quickly collected. The log data includes the standard output and text files. |
|
Control plane component logs | Logs are collected from control plane components, such as API server, kube-scheduler, kube-controller-manager, cloud-controller-manager, Controlplane-Events, and ALB Ingress controller. Logs are sent to a specific Logstore. This component facilitates centralized management and analysis of the running status of control plane components. This helps you quickly troubleshoot issues, monitor the health status of components, and verify the effect of custom parameter configurations. Each Logstore corresponds to a Kubernetes control plane component. | Collect logs of control plane components in ACK managed clusters |
Network component logs |
| |
System component logs | Log collection for components, such as Dashboard, network plug-ins, and storage plug-ins, is supported. |
Limits
Item | Description |
Item | Description |
Kubernetes version |
|
|
For more information about the limits of Logtail, see Logtail limits.
Billing
The costs incurred of SLS in the ACK cluster are the sum of cluster management fees and the SLS fees.
Cluster management fees are charged by ACK. The cluster management fee is charged only for ACK Pro clusters. No cluster management fee is charged for ACK Basic clusters or ACK dedicated clusters. For more information, see Billing.
The SLS fee is charged on a pay-as-you-go basis. The fees for the Logstore are charged by SLS.
For more information about the billing methods of Logstores, see Billing overview.
FAQ
How do I copy logs from a project to another project?
You can replicate the access logs from the logstore-a Logstore of the project-a project to the logstore-b Logstore of the project-b project to query and analyze the data in the project-b project in a centralized manner. For more information, see Replicate data from a Logstore.
What do I do if the log collection status is abnormal?
When you use Logtail to collect logs from standard Docker containers or Kubernetes containers, errors may occur. For more information about how to troubleshoot the errors and check the running status of Logtail, see What do I do if errors occur when I collect logs from containers?
FAQ about log loss, log deletion, and log cost reduction
For more information about how to change the log retention period and disable log collection, see FAQ about Logstores.
Logtail-related issues
For more information about the collection latency of Logtail, how to collect historical logs, and effective time of Logtail configurations, see FAQs about Logtail.
Contact us
If you have questions about ACK, contact us.
