Simple Log Service:Best practices

Data collection

• Collect IoT or embedded development logs

• Use the web tracking feature to collect logs

• Build a service to upload logs from mobile apps to Simple Log Service

• Use Logtail to collect Zabbix data

• Use Logtail to collect logs across Alibaba Cloud accounts

• Use Logtail to collect container logs across Alibaba Cloud accounts

• Deliver Alibaba Cloud CDN real-time logs to Simple Log Service for analysis

Metric storage

Use Prometheus to collect Kubernetes metric data

Query and analysis

• Query and analyze website logs

• Query and analyze JSON logs

• Associate a Logstore with a MySQL database to perform query and analysis

• Associate a Logstore with an OSS external table to perform query and analysis

• Collect and analyze NGINX monitoring logs

• Collect and analyze NGINX access logs

• Analyze Apache access logs

• Analyze IIS access logs

• Analyze Log4j logs

• Analyze website logs

• Query and analyze application logs

• Analyze Layer 7 access logs of SLB

• Perform paged queries

• Analyze vehicle track logs

• Analyze sales system logs

Data transformation

• Cleanse data by using functions

• Check data by using functions

• Convert datetime

• Mask sensitive data

• Parse Syslog messages in standard formats

• Parse NGINX logs

• Parse Java error logs

• Extract dynamic key-value pairs from a string

• Transform logs in specific text formats

• Parse logs in a CSV-format log file

• Transform complex JSON data

• Convert logs to metrics

• Pull data from one Logstore to enrich log data in another Logstore

• Obtain the IPIP library from OSS and enrich IP address data

• Obtain the IP2Location library from OSS and enrich IP address data

• Pull a CSV file from OSS to enrich data

• Pull data from an ApsaraDB RDS for MySQL database to enrich data

• Obtain data from an ApsaraDB RDS for MySQL database over the internal network

• Use a resource function to obtain incremental data

• Use the e_dict_map and e_search_dict_map functions to enrich log data

• Pull data from a Hologres database to perform data enrichment

• Build dictionaries and tables for data enrichment

• Use the e_table_map function to enrich HTTP response status codes

• Replicate data from a Logstore

• Transfer data across regions

• Distribute data to multiple destination Logstores

• Aggregate data from multiple source Logstores

• Replicate and distribute data

Alerting

• Configure alerts based on log keywords

• Use custom query statements to analyze alert logs

Visualization

• Add a filter of the Replace Variable type

• Add a filter of the Filter type

Consumption and shipping

• Build a monitoring system

• Consume metering logs to generate bills

• Use a consumer library to consume logs in high reliability mode

Log applications

Trace

• Import trace data from Simple Log Service to Grafana

• Import trace data from Apache SkyWalking to Simple Log Service

• Import trace data from Apache SkyWalking to Simple Log Service based on an ACK cluster

• Import Ingress trace data from Kubernetes clusters to Simple Log Service