This topic describes how to collect logs and send the collected logs to Simple Log Service by using the web tracking feature. This topic also describes how to query and analyze the collected logs by using the web tracking feature.
Background information
When you send an important email, you can add the read receipt tag to the email. This way, you can receive a receipt when the recipient reads the email. The read receipt feature is widely used in the following scenarios:
Check whether a recipient reads a leaflet after the leaflet is sent to the recipient.
Check how many users click a promotional web page.
Analyze the page views (PVs) of a promotional event page on a mobile app.
Traditional solutions are developed for websites and webmasters. These solutions do not apply to data collection or analysis in the preceding scenarios due to the following reasons:
Traditional solutions cannot meet personalized requirements. The behavior data of users is not generated on mobile devices. The data includes parameters that are specific to personalized operations. The parameters include source, channel, environment, and behavior parameters.
Traditional solutions are difficult and expensive to develop. To collect and analyze data, you must purchase cloud hosts, public IP addresses, servers used to receive development data, and message-oriented middleware. You must configure mutual backup to ensure high service availability. In addition, you must develop and test the analysis service that you use.
Traditional solutions are difficult to use. After data is sent to the analysis service that you use, engineers must cleanse the data and import the data to the database that you can use. Then, you can generate data for your business operations.
Traditional solutions do not support auto scaling or the estimation of user resource usage. Therefore, a large resource pool must be reserved.
For these reasons, if you want to deliver content to intended users, you need an efficient method to collect and analyze the behavior data of the users.
Simple Log Service provides web tracking, JavaScript, and Tracking Pixel SDKs for the preceding scenarios. You can use the SDKs to collect data based on a lightweight tracking solution. This allows you to report tracking points and data within 1 minute.
Features
The solution provided in this topic is based on Simple Log Service. Simple Log Service is an end-to-end data logging service, which allows you to collect, consume, ship, query, and analyze log data without the need to invest in data collection or processing systems. Simple Log Service helps improve the O&M and operations efficiency. Simple Log Service provides the following features:
LogHub: This feature allows you to collect and consume log data in real time. LogHub can be integrated with Blink, Flink, Spark Streaming, Storm, and Kepler.
LogShipper: This feature allows you to ship data. LogShipper can be integrated with MaxCompute, E-MapReduce, Object Storage Service (OSS), and Function Compute.
LogSearch and Analytics: This feature allows you to query and analyze log data in real time. LogSearch and Analytics can be integrated with DataV, Grafana, Zipkin, and Tableau.
Benefits of data collection by Simple Log Service
Simple Log Service provides more than 30 types of data collection methods to collect data from servers, mobile devices, and embedded devices. Simple Log Service also provides solutions for various programming languages.
Logtail: the log collection agent for x86 servers
SDKs: SDKs for mobile Android and iOS devices
Producer Library: a library for smart devices and devices that have limited CPUs or memory
Web tracking is a lightweight collection solution. If you use the solution, you can send data to a Simple Log Service Logstore only by initiating an HTTP GET request. This solution is suitable for scenarios in which no verification is required to collect data from sources such as static web pages, online advertisements, promotional documents, and mobile devices. The following figure shows the benefits of the web tracking feature.
Web tracking process
Web tracking, also known as tracking pixel, is an HTML image tag. In web tracking, a 0-pixel image can be embedded into an HTML page, and the image is invisible to users by default. When you access the page and the image is loaded, a GET request is initiated to pass relevant parameters to the server. For more information, see Use web tracking to collect logs.
Scenarios
You want to send the new content that you generate, such as new features, promotional events, games, and articles, to users at the earliest opportunity. This is the first and most important step to attract users.
For example, your company spent a large amount of money and distributed 10,000 advertisements to promote a game. 2,000 advertisements were loaded by users, accounting for 20% of the total number of advertisements. 800 users viewed the loaded advertisements. Few users downloaded the game, created accounts, and tried the game.
If you can monitor the effectiveness of the promotion in real time, you can promote your game in a more efficient manner. To reach your promotion goals, you can use the following channels:
Internal messages, official blogs, and homepage banners
Text messages, emails, and leaflets
New media, such as Sina Weibo, DingTalk user groups, WeChat official accounts, Zhihu forum, and TouTiao
Procedure
Enable the web tracking feature.
Create a Logstore named myclick in Simple Log Service and enable the web tracking feature for the Logstore.
Create a web tracking tag.
Add an identifier to each promotion channel for an article named 1001. Then, create a web tracking tag named img.
Internal message (mailDec)
<img src="http://example.cn-hangzhou.log.aliyuncs.com/logstores/myclick/track_ua.gif?APIVersion=0.6.0&from=mailDec&article=1001" alt="" title="">Official website (aliyunDoc)
<img src="http://example.cn-hangzhou.log.aliyuncs.com/logstores/myclick/track_ua.gif?APIVersion=0.6.0&from=aliyundoc&article=1001" alt="" title="">Email (email)
<img src="http://example.cn-hangzhou.log.aliyuncs.com/logstores/myclick/track_ua.gif?APIVersion=0.6.0&from=email&article=1001" alt="" title="">
You can append more channels to the from parameter or add more parameters in the URL to collect the values of the parameters.
Add the img tag to the promotion content and release the content.
Analyze logs.
After you collect logs based on tracking points, you can use the LogSearch/Analytics feature of Simple Log Service to query and analyze a large amount of log data in real time. For more information, see LogSearch/Analytics. Simple Log Service can show log analysis results on built-in dashboards and connect to DataV, Grafana, and Tableau to visualize the log analysis results. For more information, see Create a dashboard, Connect Simple Log Service with DataV, and Connect to Simple Log Service by using Grafana.
The following figure shows the collected log data. You can enter a keyword in the search box to query logs.
You can also enter an analytic statement after the search statement to analyze the log data and visualize the analysis results in seconds.
Create query statements.
The following examples describe how to create query statements to obtain page clicks and PVs. For more information, see Log analysis overview.
To query the current total traffic and PVs, execute the following statement:
* | select count(1) as cTo query the trend of PVs on an hourly basis, execute the following statement:
* | select count(1) as c, date_trunc('hour',from_unixtime(__time__)) as time group by time order by time desc limit 100000To query the ratio of PVs in each channel, execute the following statement:
* | select count(1) as c, f group by f descTo query the devices on which PVs are initiated, execute the following statement:
* | select count_if(ua like '%Mac%') as mac, count_if(ua like '%Windows%') as win, count_if(ua like '%iPhone%') as ios, count_if(ua like '%Android%') as androidTo query the geographic locations in which PVs are initiated, execute the following statement:
* | select ip_to_province(__source__) as province, count(1) as c group by province order by c desc limit 100
Configure a dashboard to visualize the collected data in real time.