Simple Log Service:Usage notes

Features

CloudLens for SLS supports the following features:

• The application allows you to manage all projects and Logstores that meet the specified conditions within your Alibaba Cloud account in a centralized manner. For more information about conditions, see Limits.

• The application allows you to enable log collection for instance logs and global logs with a few clicks and manage the log collection status in a centralized manner. Instance logs are classified into important logs, detailed logs, and job operational logs. Global logs are classified into audit logs, billing logs, error logs, and metric-related logs.

  Note

  You can view only instance logs and global logs that are generated after you enable the log collection feature.

  • Important logs record the consumption delay of consumer groups for each Logstore. Important logs also record the errors, heartbeats, and statistics of Logtail. The logs are stored in a Logstore named internal-diagnostic_log of a specified project.

  • Detailed logs record the operations that create, modify, update, and delete resources in a project. Detailed logs also record data read and write operations. The logs are stored in a Logstore named internal-operation_log of a specified project.

  • Job operational logs record the error and latency information about jobs such as data import jobs of the new version, data shipping jobs of the new version, and Scheduled SQL jobs. The logs are stored in a Logstore named internal-diagnostic_log of a specified project.

  • Audit logs record the operations that create, modify, update, and delete resources in all projects within your Alibaba Cloud account. The logs are stored in a Logstore named internal-audit_log of a specified project.

  • Billing logs record billable resource data. The logs are generated by Cost Manager. The logs are stored in Tablestore and are automatically updated on a daily basis. For more information, see Bill data.

    Note

    • The first time you enable the log collection feature for billing logs, the feature requires approximately 1 hour to 2 hours to import logs. The amount of time that is required to import logs varies based on the amount of billing logs.

    • Only Alibaba Cloud public cloud users can enable the log collection feature for billing logs.

    • After you enable the log collection feature for billing logs, you cannot disable the feature.

  • Error logs record the errors of the operations that create, modify, update, and delete resources and the errors of data read and write operations in all projects within your Alibaba Cloud account. The logs are stored in a Logstore named internal-error_log of a specified project.

  • Metric-related logs record the traffic, requests, and latency of each project and each Logstore. The metrics are stored in a Metricstore named internal-monitor-metric of a specified project.

  For more information, see Service logs.

  

• CloudLens for SLS provides alert rules such as baseline alert rules, interval-valued comparison alert rules, and periodicity-valued comparison alert rules. You can use the alert rules to monitor the usage of resources such as projects, Logtail, and consumer groups. CloudLens for SLS supports different notification methods, such as text messages, DingTalk messages, emails, voice calls, and custom webhooks. For more information, see Configure alerts.

• The application provides the following types of reports: Overview, Access Monitoring, Collection Monitoring, Operation Monitoring, Job Monitoring, Quota Monitoring, SQL Quality Monitoring, and Billable Resource Monitoring. For more information, see View data reports.

• The application allows you to view the quota information of all projects and Logstores that meet the specified conditions within your Alibaba Cloud account. For more information about conditions, see Limits.

  

Scenarios

• Development and O&M

  Developers can use CloudLens for SLS to monitor and manage assets, such as projects and Logstores in an efficient manner. In addition, the developers can query and analyze the instance logs of a project in CloudLens for SLS to troubleshoot errors in an efficient manner. Instance logs are classified into detailed logs, important logs, and job operational logs.

• IT O&M

  IT O&M personnel can use the alerting feature of CloudLens for SLS to identify and handle the exceptions of project service status, traffic, quotas, and Logtail metrics at the earliest opportunity.

• Security audit

  The audit logs of CloudLens for SLS record the operations that create, update, and delete resources in a project. This allows security auditors to analyze and audit operations within a specific period of time.

Assets

• Projects, Logstores, and Metricstores

  • When you enable log collection for important logs or job operational logs, you can select a project based on your business requirements. After log collection is enabled for important logs or job operational logs, Simple Log Service generates a Logstore named internal-diagnostic_log in the project.

  • When you enable log collection for detailed logs, you can select a project based on your business requirements. After log collection is enabled for detailed logs, Simple Log Service generates a Logstore named internal-operation_log in the project.

  • When you enable log collection for audit logs, you can select a project based on your business requirements. After log collection is enabled for audit logs, Simple Log Service generates a Logstore named internal-audit_log in the project.

  • When you enable log collection for error logs, you can select a project based on your business requirements. After log collection is enabled for error logs, Simple Log Service generates a Logstore named internal-error_log in the project.

  • When you enable log collection for metric-related logs, you can select a project based on your business requirements. After log collection is enabled for metric-related logs, Simple Log Service generates a Metricstore named internal-monitor-metric in the project.

• Dashboards

  Dashboard		Description
  Overview	Access Overview	Displays information about Simple Log Service projects in charts. The charts include Number of requests, Request Success Rate, Requests with Status Code 4xx, Requests with Status Code 5xx, Write traffic (raw size), Write traffic (after compression), Write traffic compression ratio, and Read Traffic.
  Access Monitoring	Access Traffic Monitoring	Displays information about access to Simple Log Service in charts. The charts include Total Requests, Clients, Users, Top 10 SourceIPs with Most Read Traffic, Top 10 SourceIPs with Most Write Traffic, Requests, Write Traffic, Trend of Write Traffic to Shards, Trend of Read Traffic from Shards, Read/Write Traffic/Minute, and Request IP Distribution.
  	Access Exception Monitoring	Displays information about abnormal access to Simple Log Service in charts. The charts include Total Requests, Percentage of Failed Requests, Quota Exceeded, Error Status Distribution, Logstore with Excessive Write Traffic, Error Requests, and Trend of Request Latency.
  	Consumer Group Monitoring	Displays information about consumer groups in charts. The charts include Consumer Group, Logstore, Shard, Fall Behind, Data, Consumer Group, Top 10 Latency, and Trend of Consumer Group Latency.
  Collection Monitoring	Logtail Overall Status	Displays information about Logtail in charts. The charts include Active Logtail, Data Traffic, Status, CPU, Logtail Overall Status, Trend of CPU, Trend of Memory, and Data Sending Traffic.
  	Logtail File Collection Monitoring	Displays information about the files from which you want to collect logs in charts. The charts include Logtail File Collection, Machine Collection, Collection File Distribution, Trend of Log Collection, Trend of Average Latency, Trend of Parse Failure, and Trend of Send.
  	Logtail Exception Monitoring	Displays information about Logtail exceptions in charts. The charts include Active Logtail, Restart Logtail List, Restart Clients, and All Alarm.
  Operation Monitoring	Operation Monitoring	Displays information about Simple Log Service operation records in charts. The charts include Logstore Related Operation Count, Logstore Related Fail Operation, and Clients.
  Job Monitoring	Data Processing Monitor	Displays information about data import jobs of the new version, data shipping jobs of the new version, and Scheduled SQL jobs in charts. The charts include Ingest Success, Ingest Failed, Deliver Success, Deliver Failed, Ingest PubNet, Deliver PubNet, Process Speed, Process Lags, Process Error, and Process State.
  Quota Monitoring	Quota Monitoring	Displays information about quota exceeded events in charts. The charts include Overview of Resource Quota (80%) Warning, QuotaExceed, LogStore Usage Distribution, MachineGroup Usage Distribution, Logtail Config Usage Distribution, Project Write Inflow Usage Distribution, Project Overview of Basic Quota Usage, Project Overview of Write/Read Quota Usage, and Project Quota Exceeded Statistics.
  SQL Quality Monitoring	SQL Quality Monitoring	Displays information about the quality of SQL statements in charts. The charts include SQL Health Score, SQL Metric, Running Metric, and SQL Pattern Analysis.
  Billable Resource Monitoring	Billable Resource Monitoring	Displays information about the billable resources of Simple Log Service in charts. The charts include Storage Size(Hot Storage), Index Size(Log), Read/Write Traffic, Billing Items, Top Project, and Top Logstore.

Billing

For more information, see Billable items of pay-by-feature.

Limits

The following table describes the regions where you can use CloudLens for SLS to monitor and manage assets.

Precautions

When you enable a CloudLens application, Simple Log Service automatically checks whether a project exists within your Alibaba Cloud account.