To gain insights into user behavior, enterprises collect logs from mini programs and analyze key data such as user devices and browsing history. This way, enterprises can improve features and optimize user experience. After the web tracking feature of Alibaba Cloud Simple Log Service is enabled, enterprises can automatically collect user logs from mini programs to Simple Log Service. This improves data processing efficiency and reduces the burden on application servers. After you enable the web tracking feature for a Logstore, the write permissions on the Logstore are granted to anonymous users. This can result in dirty data from configuration data leaks of mini programs. To reduce this risk and enhance data upload security, we recommend that you use Security Token Service (STS) to provide mini programs with tokens that are valid within a specified period of time and allow only specific operations. This topic describes how to use Simple Log Service and STS to build a secure and efficient direct data transfer service for mini programs to collect browsing behavior logs from mini programs to Simple Log Service.
Background information
Enterprise A develops a mini program that provides users with an online, feature-rich, and user-friendly platform. This platform is designed to attract users to browse and purchase products and services. Enterprise A acknowledges the importance of in-depth analysis on user behavior for better understanding user requirements, optimizing user experience, and further improving the conversion rate of the mini program. The user behavior includes clicks, swipes, dwelling, search habits, and purchases. Therefore, the technical team of Enterprise A collects logs from the mini program to Simple Log Service for efficient data analysis and processing.
Security risks
To reduce the burden on application servers and improve the efficiency of data processing, the mini program is designed to directly upload user logs to Simple Log Service without the need for data transfer and data collection to application servers.
The following figure shows how Enterprise A plans to build a direct data transfer service to collect mini program logs to Simple Log Service.
After the web tracking feature is enabled for a Simple Log Service Logstore, the write permissions on the Logstore are granted to anonymous users. In this case, the following risks may occur:
Dirty data generation: Malicious users may upload false or sabotage data, which compromises data quality and the accuracy of analysis results.
Service abuse: Unlimited write operations may be abused to initiate Denial of Service (DoS) attacks. Attackers may initiate a large number of write requests to exhaust resources.
Solution
To handle the risks, Enterprise A adds temporary access authorization. This way, Enterprise A can ensure the efficiency of direct data transfer and achieve the following benefits:
Enhanced authentication and authorization: STS generates tokens that are valid within a specified period of time. This significantly reduces security risks even if the tokens are leaked because the tokens quickly become invalid.
Fine-grained access control: STS allows you to configure permissions based on the principle of least privilege. You can grant only the required permissions to the mini program. This helps narrow the impact scope of token leaks and avoids granting excessive permissions to the mini program.
The following figure shows how Enterprise A builds a direct data transfer service to collect mini program logs to Simple Log Service.
Solution deployment
This section describes how to use Simple Log Service and STS to build a direct data transfer service for the mini program to collect browsing behavior logs from the mini program to Simple Log Service.
Manual deployment
Preparations
Before you start the deployment, you must create cloud resources.
Create a Simple Log Service project and a Logstore to store and analyze logs that are collected from the mini program.
Parameter
Example value
Region
China (Shanghai)
Project Name
sls-webtracking-mini
Logstore Name
web-tracking-logstore
For more information, see Step 1: Create a project and a Logstore.
Create an Elastic Compute Service (ECS) instance as your application server to generate STS tokens.
NoteIn your actual deployment, you do not need to create the ECS instance. You can directly integrate STS API into your application server.
Parameter
Example value
Billing Method
Pay-as-you-go
Region
China (Shanghai)
Public IP Address
Assign Public IPv4 Address
Security Group
HTTP-TCP:80-open
For more information, see Create and manage an ECS instance in the console (express version).
Procedure
To manually deploy the project, perform the following steps:
Step 1: Create a RAM user in the RAM console
Create a Resource Access Management (RAM) user and obtain the AccessKey pair of the RAM user. When you create the RAM user, set Access Mode to OpenAPI Access. The AccessKey pair is used as a long-term access credential for accessing and managing your application server.
Log on to the RAM console by using an Alibaba Cloud account or an account administrator.
In the left-side navigation pane, choose Identities > Users.
On the Users page, click Create User.
Configure the Logon Name and Display Name parameters.
In the Access Mode section, select OpenAPI Access. Then, click OK.
Click Copy in the Actions column and save the AccessKey pair of the RAM user.
Step 2: Grant the RAM user the permissions to call the AssumeRole operation in the RAM console
After the RAM user is created, grant the RAM user the permissions to call the AssumeRole operation of STS. This way, the RAM user can assume a RAM role to obtain STS tokens.
In the left-side navigation pane, choose Identities > Users.
On the Users page, find the RAM user and click Add Permissions in the Actions column.
In the Grant Permission panel, select the AliyunSTSAssumeRoleAccess system policy.
NoteTo grant a RAM user the permissions to call the AssumeRole operation, you must attach the AliyunSTSAssumeRoleAccess system policy to the RAM user. This policy does not grant the permissions to obtain STS tokens or initiate Simple Log Service requests by using the STS tokens.
Click Grant permissions.
Step 3: Create a RAM role in the RAM console
Create a RAM role that the RAM user can assume to obtain STS tokens and obtain the Alibaba Cloud Resource Name (ARN) of the RAM role. For more information, see ARN.
In the left-side navigation pane, choose Identities > Roles.
On the Roles page, click Create Role. In the Select Role Type step of the Create Role wizard, set Select Trusted Entity to Alibaba Cloud Account and click Next.
In the Configure Role step of the Create Role wizard, enter a name in the RAM Role Name field and set Select Trusted Alibaba Cloud Account to Current Alibaba Cloud Account.
Click OK. After the RAM role is created, click Close.
On the Roles page, enter the role name in the search box to search for the RAM role. Example:
sls-web-tracking
.On the details page of the RAM role, click Copy to save the ARN of the RAM role.
Step 4: Create a custom policy in the RAM console
Create a custom policy based on the principle of least privilege. The custom policy grants the RAM role the permissions to upload logs only to a specified Simple Log Service Logstore.
In the left-side navigation pane, choose Permissions > Policies.
On the Policies page, click Create Policy.
On the Create Policy page, click the JSON tab. Replace
<Project name>
and<Logstore name>
in the following policy document with the names of the project and Logstore that are created in the "Preparations" section of this topic.ImportantThe following policy document is provided only for reference. You must configure fine-grained RAM policies based on your business requirements to avoid granting excessive permissions to the RAM role. For more information about how to configure fine-grained RAM policies, see Examples of using custom policies to grant permissions to a RAM user.
{ "Version":"1", "Statement":[ { "Effect":"Allow", "Action":[ "log:PostLogStoreLogs" ], "Resource":[ "acs:log:*:*:project/<Project name>/logstore/<Logstore name>" ] } ] }
After you configure the policy, click Next to edit policy information.
In the Basic information section, specify a policy name. Then, click OK.
Step 5: Attach the custom policy to the RAM role in the RAM console
Attach the custom policy to the RAM role. This way, the RAM user can assume the RAM role to perform the operations that are allowed by the policy.
In the left-side navigation pane, choose Identities > Roles.
On the Roles page, find the RAM role and click Grant Permission in the Actions column.
In the Policy section of the Grant Permission panel, select Custom Policy from the policy type drop-down list and select the custom policy.
Click Grant permissions.
Step 6: Obtain STS tokens from your application server
Access your mini program and integrate STS SDK into your application server to implement the /get_sts_token operation. For more information, see STS SDK overview. When you call the /get_sts_token
operation by using the HTTP GET method, the system generates and returns an STS token.
The following procedure describes how to build a mini program by using the Flask framework on your ECS instance and obtain STS tokens from the application server:
Connect to the ECS instance.
For more information, see Create and manage an ECS instance in the console (express version).
Install Python 3. For more information, see Python 3.
Configure the
ALIBABA_CLOUD_ACCESS_KEY_ID
andALIBABA_CLOUD_ACCESS_KEY_SECRET
environment variables for the RAM user. For more information, see Configure environment variables in Linux, macOS, and Windows.Create a project folder and switch to the project directory.
mkdir my_web_sample cd my_web_sample
Install dependencies.
pip3 install Flask pip3 install attr pip3 install yarl pip3 install async_timeout pip3 install idna_ssl pip3 install attrs pip3 install aiosignal pip3 install charset_normalizer pip3 install alibabacloud_tea_openapi pip3 install alibabacloud_sts20150401 pip3 install alibabacloud_credentials
Write backend code.
Create a file named
main.py
.Add the following Python code to the file:
import json from flask import Flask, render_template from alibabacloud_tea_openapi.models import Config from alibabacloud_sts20150401.client import Client as Sts20150401Client from alibabacloud_sts20150401 import models as sts_20150401_models from alibabacloud_credentials.client import Client as CredentialClient app = Flask(__name__) # Replace <YOUR_ROLE_ARN> with the ARN of the RAM role. role_arn_for_oss_upload = '<YOUR_ROLE_ARN>' # Specify the region of STS. Example: cn-shanghai. region_id = 'cn-shanghai' @app.route('/get_sts_token', methods=['GET']) def get_sts_token(): # If you do not specify parameters when you initialize CredentialClient, the default credential chain is used. # If you run the file on your computer, you can specify the AccessKey pair by configuring the ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variables. # If you run the file on ECS, Elastic Container Instance (ECI), or Container Service for Kubernetes (ACK), you can specify the role of the bound instance by configuring the ALIBABA_CLOUD_ECS_METADATA environment variable. The SDK automatically obtains STS tokens. config = Config(region_id=region_id, credential=CredentialClient()) sts_client = Sts20150401Client(config=config) assume_role_request = sts_20150401_models.AssumeRoleRequest( role_arn=role_arn_for_oss_upload, # Replace <YOUR_ROLE_SESSION_NAME> with the custom name of the session. role_session_name='<YOUR_ROLE_SESSION_NAME>' ) response = sts_client.assume_role(assume_role_request) token = json.dumps(response.body.credentials.to_map()) return token app.run(host="0.0.0.0", port=80)
Replace
<YOUR_ROLE_ARN>
with the ARN that is obtained in Step 3.Replace
<YOUR_ROLE_SESSION_NAME>
with the custom name of the session. Example:role_session_test
.
Run the main.py file.
python3 main.py
Visit
http://<Public IP address of the ECS instance>/get_sts_token
in your browser.The following figure shows a success response.
Step 7: Use an STS token in your mini program to upload logs to Simple Log Service
For more information about the development process and code structure of mini programs, see Create a mini program in the documentation center of Alipay. In this example, you can write the client-side code of the mini program based on Todo App and Use the STS plug-in of web tracking SDK for JavaScript to upload logs.
After you implement the /get_sts_token operation on your application server, integrate the @aliyun-sls/web-track-mini instrumentation SDK and the @aliyun-sls/web-sts-plugin STS plug-in of Simple Log Service into the app.js
file of the mini program. This helps monitor user behavior in real time and upload related logs.
When you create a tracker, the tracker automatically calls the /get_sts_token
operation to obtain an STS token. This way, the user behavior data generated during mini program interactions can be securely transferred to Simple Log Service. The user behavior data includes the data of logons, product browsing, and order submission.
Download a developer tool of mini programs. For more information, see Developer tools of mini programs.
Create and use the Todo App mini program. For more information, see Todo App.
Install npm.
Install project dependencies.
npm install --save @aliyun-sls/web-track-mini npm install --save @aliyun-sls/web-sts-plugin
Modify the sample code of the Todo App mini program to upload user logs to the specified Logstore by using the web tracking feature. The following table describes the files that you need to modify.
File
Code modification description
app.js
Call the
initSlsTracker
method in theonLaunch
method. This way, when you start the mini program, the SlsTracker object is initialized.Configure parameters such as project and logstore in the
initSlsTracker
method. For more information about the parameters, see Web tracking parameters.Configure the
stsOpt
parameter and therefreshSTSToken
function to dynamically obtain and update STS tokens. For more information about STS token-related parameters, see STS parameters.
mini.project.json
The @aliyun-sls/web-sts-plugin STS plug-in supports ECMAScript 6 (ES6). When you modify the
mini.project.json
file, setcompileOptions.transpile
totrue
. This way, the developer tool of the mini program translates ES6 code to ECMAScript 5 (ES5) code to ensure compatibility.todo.xml
<view class="todo-footer"> <add-button text="Add Todo" onClickMe="handleAddTodo"></add-button> </view>
Define a button named
Add Todo
at the bottom of the user interface. You can useadd-button
to define the button.onClickMe="handleAddTodo"
specifies that the system calls thehandleAddTodo
method defined in thetodo.js
file when you click the button.todo.js
Use the
handleAddTodo()
function to handle events. When you add a to-do item on the user interface, this function is triggered. The function performs the following operations:Call the
this.addTodo()
method to add a to-do item.Call the
this.onAddTodoButtonClick()
method to send a log message indicating that a user clicks the Add Todo button.
Use the
addTodo()
function to call themy.navigateTo
function of the mini program. This way, you are navigated from the current page to the required page in the/pages/add-todo/add-todo
directory of the mini program.Use the
onAddTodoButtonClick()
method to call thesend
method of theSlsTracker
object to send a log. The log content iseventType: 'addtodo_click'
. The SlsTracker object is defined in theapp.js
file.
add-todo.js
Use the
add
method to perform the following operations:Create an object named
newTodo
. ThenewTodo
object specifies a to-do item, which contains the following attributes:text
: the content of the to-do item. You can obtain the content by calling thethis.data.inputValue
method. This call indicates that the content may come from user input.completed
: specifies whether the to-do item is completed. Initial value:false
. This value specifies that the to-do item is not completed.
Call the
send
method of theSlsTracker
object defined in theapp.js
file to send a log. The following code shows the content of the log:eventType: 'add_todo' todoText: newTodo.text
add-button.js
Define custom buttons. The
onClickMe
method first checks whetherthis.props.onClickMe
is a function. Ifthis.props.onClickMe
is a function, the method calls the function.Configure parameters in the
app.js
file.// app.js import SlsTracker from '@aliyun-sls/web-track-mini'; import createStsPlugin from '@aliyun-sls/web-sts-plugin'; App({ todos: [ { text: 'Learning Javascript', completed: true }, { text: 'Learning ES2016', completed: true }, { text: 'Learning the Alipay mini program', completed: false }, ], userInfo: null, tracker: null, onLaunch: function() { // When you start the mini program, initialize the SlsTracker object. this.initSlsTracker(); }, initSlsTracker: function() { const opts = { host: 'cn-shanghai.log.aliyuncs.com', // Replace the value with your endpoint. project: '${project}', // Replace the value with your project name. logstore: '${Logstore}', // Replace the value with your Logstore name. time: 10, count: 10, topic: 'topic', source: 'source', tags: { tags: 'tags', }, }; const stsOpt = { accessKeyId: '', accessKeySecret: '', securityToken: '', refreshSTSToken: () => new Promise((resolve, reject) => { my.request({ url: 'http://${Public IP address of the ECS instance}}/get_sts_token', // Replace the value with the actual address of your application server that generates the STS token. method: 'GET', success: (res) => { if (res.statusCode === 200) { let credential; // Check whether the data type of the returned data is string. If the data type is not string, the data may be an object. if (typeof res.data === "string") { // Parse the returned data into a JSON object. credential = JSON.parse(res.data); } else { // Use the object. credential = res.data; } stsOpt.accessKeyId = credential.AccessKeyId; stsOpt.accessKeySecret = credential.AccessKeySecret; stsOpt.securityToken = credential.SecurityToken; resolve(credential); } else { reject(new Error('Failed to refresh STS token with status code: ' + res.statusCode)); } }, fail: (err) => { reject(new Error('Failed to refresh STS token', err)); }, }); }), }; const tracker = new SlsTracker(opts); const stsPlugin = createStsPlugin(stsOpt); tracker.useStsPlugin(stsPlugin); // When you start the mini program, a log is uploaded to Simple Log Service. // tracker.send({ // eventType:'view_product', // productName: 'Tablet', // price: 500 // }); this.tracker = tracker; }, });
For more information about the
opts
andstsOpt
parameters, see Use the STS plug-in of web tracking SDK for JavaScript to upload logs.Replace
${project}
and${logstore}
with the names of the project and Logstore that are created in the "Preparations" section of this topic.Replace
<Public IP address of the ECS instance>
with the public IP address of the ECS instance that is created in the "Preparations" section of this topic. For more information about how to view the public IP address of an ECS instance, see View IP addresses.
Run the client-side code of the mini program. Click the icon in the upper-right corner of the developer tool page. On the preview page of the mini program, click the Add Todo button on the Todo App page. When you click the Add Todo button or enter a to-do item after you click the Add Todo button, the mini program directly uploads a log to the Simple Log Service Logstore. This precludes the need for data transfer or data collection to your application server.
Verification and cleaning
Solution verification
After you complete the preceding steps, you can preview logs to check whether the logs are collected to Simple Log Service.
In the Consumption Preview panel, view the collected logs.
Resource cleaning
In this example, you created an ECS instance, a Simple Log Service project, a Logstore, a RAM user, and a RAM role. After you verify the solution, you can perform the following operations to release the resources. This way, you can avoid unnecessary fees or security risks.
Release the ECS instance. For more information, see Release an instance.
Delete the Logstore and then delete the project. For more information, see Manage a Logstore and Manage a project.
Delete the RAM user and the RAM role. For more information, see Delete a RAM user and Delete a RAM role.
What to do next
The mini program used in this topic is only debugged by using the developer tool on your computer. To officially launch the mini program, you must perform the following operations:
Create a developer account and a mini program. Then, submit the mini program for review. For more information, see Register a developer account.
In your developer tool, select the required options to skip validity checks of domain names. This facilitates debugging. For more information, see Access preparations.
In the Alipay console, configure a domain name whitelist.
Add the domain name of your Simple Log Service server or valid domain names for requests to the whitelist. The domain name is in the
https://${project}.${host}
format.project
specifies the name of the project that is used.host
specifies the endpoint of the project.sls-webtracking-mini.cn-shanghai.log.aliyuncs.com
is used in this topic. For more information, see Domain name whitelist of servers.Add the domain name of the application server of the mini program. This topic provides only the client-side code of the mini program. To officially launch the mini program, you must develop the server-side code of the mini program. Only HTTPS is supported for domain names in the whitelist. To use HTTPS, you must install SSL certificates on the related servers. For more information, see Installation overview. In this example, the options to skip the validity checks of domain names are selected in the developer tool. Before you officially launch the mini program, you must configure the checks.
After you collect user logs from mini programs to Simple Log Service, you can perform the following operations:
Query and analyze the logs to gain insights into user behavior. For more information, see Query and analyze logs.
Display query and analysis results in dashboards, charts, and third-party visualization tools. For more information, see Overview of visualization.
Analyze user behavior logs to detect potential security threats, including failed logon attempts, abnormal access modes, and suspected data leaks. If the system detects the preceding threats, alerts are triggered. Then, the security team can respond to the threats at the earliest opportunity. For more information, see Introduction to the alerting feature.