All Products
Search
Document Center

Web Application Firewall:Enable and configure the bot management module

Last Updated:Sep 23, 2024

The bot management module of Web Application Firewall (WAF) mitigates attacks that are initiated by automation tools such as emulators and malicious scripts. The attacks include data crawling, fraud, credential stuffing attacks, spam user registrations, auto-purchase bots, promotion abuse, and SMS flood attacks. We recommend that you configure bot management rules based on analytical data of bot traffic to reduce data leaks and risks during marketing activities. This helps reduce server workloads and bandwidth costs. This topic describes how to enable the bot management module and configure bot management rules.

Feature description

The bot management module provides bot traffic analysis, basic protection, and scenario-specific protection. The bot management module helps you identify bot traffic and defend against crawlers to protect your web services from being crawled.

  • Bot traffic analysis: You can use the feature regardless of whether you enable the bot management module.

    You can view the results of bot traffic analysis without enabling the bot management module, including the bot traffic trends within a specific time range, top 20 risky clients, top 20 risky IP addresses, and analytical data of bot traffic to protected objects. This helps you identify and locate risky domain names in an efficient manner. For more information, see View analytical data of bot traffic.

    You can apply for a trial or enable the bot management module to configure scenario-specific protection for risky domain names. For more information about how to apply for a trial and how to enable the bot management module, see Enable the bot management module.

  • Scenario-specific Protection: You must enable the bot management module before you can use the feature.

    The scenario-specific protection feature provides SDKs that you can integrate to configure custom rules to protect your websites and apps from crawlers. For more information, see Create an anti-crawler rule for websites and Create an anti-crawler rule for apps.

    This feature is suitable for users who are sensitive to bot traffic.

  • Basic Protection: You must enable the bot management module before you can use the feature.

    The basic protection feature detects Layer 4 and Layer 7 bot traffic by using fingerprinting techniques. You can use the feature without the need to integrate SDKs. For more information, see Create a basic protection rule.

    This feature is suitable for users who want to defend against low-level crawlers by configuring protection rules in a simplified manner.

Prerequisites

View analytical data of bot traffic

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and region of the WAF instance. You can select Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Bot Management.

  3. On the Bot Traffic Analysis tab, view bot traffic trend, top 20 risky clients, top 20 risky IP addresses, and analytical data of bot traffic to protected objects. image.png

Enable the bot management module

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and region of the WAF instance. You can select Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Bot Management.

  3. Enable the bot management module.

    • Apply for a free trial

      Note
      • You can apply for a free trial of the bot management module only once. You can apply for a free trial only if you use a WAF Pro, Enterprise, or Ultimate instance.

      • You can receive a seven-day free trial after your application is approved. The analytical data that is generated during the trial period is available only during the trial period. If you want to retain the analytical data, enable the bot management module before the trial period ends.

      On the Bot Traffic Analysis tab, click Apply for Trial. On the page that appears, enter the application information and click Submit.

      After you submit your trial application, Alibaba Cloud engineers will contact you based on the contact information that you submit to confirm information that is related to your application. After the application is approved, the bot management module is automatically enabled for your WAF instance.

    • Enable the bot management module

      1. On the Bot Traffic Analysis, Scenario-specific Protection, or Basic Protection tab, click Purchase Now.

      2. On the buy page that appears, set the Bot Management - Web Application Protection or Bot Management - App Protection parameter to Enable and complete the payment.

        Note
        • After you enable bot management for web application protection, you can configure basic protection rules and anti-crawler rules for websites.

        • After you enable bot management for app protection, you can configure basic protection rules and anti-crawler rules for apps.

        • If you want to configure basic protection rules, anti-crawler rules for websites, and anti-crawler rules for apps, enable both bot management for web application protection and bot management for app protection.

After you enable the bot management module, you can configure scenario-specific protection rules on the Bot Traffic Analysis tab. To configure a scenario-specific protection rule, go to the Bot Traffic Analysis of Protected Objects section of the page, find the domain name of the website or app that you want to protect, and then click Configure Protection in the Actions column. For more information, see Create an anti-crawler rule for websites and Create an anti-crawler rule for apps.

If you want to configure basic protection rules to defend against low-level crawlers, go to the Basic Protection tab. For more information, see Create a basic protection rule.

Create an anti-crawler rule for websites

If you want to use WAF to mitigate the security threats that are caused by bot traffic on web pages, HTML5 pages, or HTML5 apps, we recommend that you create a protection template and configure an anti-crawler rule for websites.

Note
  • If a request from a client matches a protection rule in which the Action parameter is set to Run JavaScript Validation or Slider CAPTCHA, WAF performs JavaScript validation or slider CAPTCHA verification on the client. If the client passes the validation or verification, WAF adds the acw_sc__v2 or acw_sc__v3 cookie to the header of the request to indicate that the client passed the validation or verification.

  • If you configure a scenario-specific protection temple for the bot management module and enable the automatic integration of the Web SDK feature, WAF adds the ssxmod_itna, ssxmod_itna2, and ssxmod_itna3 cookies to the HTTP request header. The cookies are used to obtain fingerprint information about the browser on the client. The fingerprint information includes the host field in the HTTP request and the height and width of the browser window.

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and region of the WAF instance. You can select Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Bot Management.

  3. On the Scenario-specific Protection tab, click Create Template.

  4. In the Configure Scenarios step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Template Name

    Enter a name for the template.

    The name of the template must be 1 to 255 characters in length and can contain letters, digits, periods (.), underscores (_), and hyphens (-).

    Template Description

    Enter a description for the template.

    Service Type

    Select Websites. This way, WAF protects web pages, HTML5 pages, and HTML5 apps.

    Web SDK Integration

    Automatic Integration (recommended)

    WAF provides Web SDK for JavaScript to improve protection performance and prevent incompatibility issues.

    If you enable automatic integration, WAF automatically references the SDK in the HTML pages of the website that you want to protect. When automatic integration is enabled, the SDK collects information such as browser information, probe signatures, and malicious behaviors. Sensitive information is not collected. WAF detects and blocks malicious crawlers based on the collected information.

    If you want to access the current protected object from a different domain name, you must select Use Intermediate Domain Name. Then, select the intermediate domain name from the drop-down list. For example, if you want to access Domain Name A from Domain Name B, you must select Use Intermediate Domain Name and select Domain Name B from the drop-down list.

    Important

    The automatic integration of Web SDK is not supported for Application Load Balancer (ALB) instances, Microservices Engine (MSE) instances, or custom domain names bound to web applications in Function Compute that are added to WAF.

    Manual Integration

    If automatic integration is not supported, you can use manual integration. You can click Obtain SDK to obtain the scripts and place the scripts above the other scripts. This ensures that the scripts are loaded first. For more information, see Deployment methods.

    For more information, see Integrate the Web SDK into web applications.

    Traffic Characteristics

    Add match conditions to identify traffic that is destined for the domain name that you want to protect. To add a match condition, you must configure the match field, logical operator, and match content. Make sure that the match field is a header field of HTTP requests. You can add up to five conditions. The conditions are evaluated by using a logical AND. For more information about match fields, see Match conditions.

  5. In the Configure Protection Rules step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Risk Identification

    Select Business Security and enter relevant information. For more information, see Risk identification.

    The feature helps block requests from abnormal mobile phone numbers based on Fraud Detection. You are charged based on rule hits.

    Legitimate Bot Management

    Select Spider Whitelist. Then, select search engines from the drop-down list.

    After you select Spider Whitelist and then select search engines from the drop-down list, requests that are sent from the crawler IP addresses of the search engines are sent to the origin server. The bot management module no longer checks these requests.

    Bot Characteristic Detection

    Script-based Bot Block (JavaScript Validation)

    If you select Script-based Bot Block (JavaScript Validation), WAF performs JavaScript validation on clients. To prevent simple script-based attacks, WAF blocks requests from non-browser tools that cannot run JavaScript.

    Advanced Bot Protection (Dynamic Token-based Authentication)

    If you select Advanced Bot Protection (Dynamic Token-based Authentication), WAF verifies the signature of each request. Requests that fail signature verification are blocked. Valid values:

    • Signature Verification Exception: This option is required. Requests that do not contain signatures or requests that contain invalid signatures are blocked.

    • Signature Timestamp Exception: Requests that contain abnormal signature timestamps are blocked.

    • WebDriver Attack: Requests are blocked when WebDriver attacks occur.

    Bot Behavior Detection

    Intelligent Protection

    If you select Intelligent Protection, you must select Monitor, Slider CAPTCHA, or Add Tag as the action that you want WAF to perform on detected bot requests. If you select Add Tag, you must configure the Header Name and Header Content parameters.

    After you select Intelligent Protection, the intelligent protection engine analyzes access traffic and performs machine learning. Then, a blacklist or a protection rule is generated based on the analysis results and learned patterns.

    Custom Throttling

    You can configure custom throttling conditions to filter out crawler requests that are frequently initiated. This helps prevent HTTP flood attacks.

    • IP Address Throttling (Default)

      You can configure throttling conditions for IP addresses. If the number of requests that are sent from the same IP address within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Monitor, Slider CAPTCHA, or Block from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The conditions are evaluated by using a logical OR.

    • Custom Session Throttling

      You can configure throttling conditions for sessions. You can configure the Session Type parameter to specify the session type. If the number of requests of the same session type within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Monitor, Slider CAPTCHA, or Block from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The conditions are evaluated by using a logical OR.

      Valid values of the Session Type parameter are Custom Header, Custom Parameter, Custom Cookie, and Session.

    Bot Threat Intelligence

    Bot Threat Intelligence Library

    The library includes the IP addresses of attackers that have sent a large number of requests to crawl content from Alibaba Cloud users. If you select Bot Threat Intelligence Library, you can select Monitor, Slider CAPTCHA, or Add Tag as the action that you want WAF to perform on the requests. If you select Add Tag, you must configure the Header Name and Header Content parameters.

    Data Center Blacklist

    If you select Data Center Blacklist, requests that are sent from the IP addresses in the selected IP address libraries are blocked. You can select Monitor, Slider CAPTCHA, Block, or Add Tag as the action that you want WAF to perform on the requests. If you select Add Tag, you must configure the Header Name and Header Content parameters.

    Note

    If you use the source IP addresses of public clouds or data centers to access the website that you want to protect, you must add the IP addresses to the whitelist. For example, you must add the callback IP addresses of Alipay or WeChat and the IP addresses of monitoring applications to the whitelist.

    Fake Spider

    After you enable the feature, WAF performs the Block or Add Tag action on requests whose User-Agent header field matches the user agents of all search engines specified in the Legitimate Bot Management section. If requests are sent from the valid IP address of the specified search engines, the requests are allowed.

  6. In the Configure Effective Scope step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Apply To

    Select the protected objects or protected object groups to which you want to apply the protection template in the Objects to Select section and click the 移入 icon to move the protected objects or protected object groups to the Selected Objects section.

    Effective Time and Canary Release

    You must specify validity periods and configure canary release settings for the protection rules. If you do not specify validity periods or configure canary release settings, canary release is disabled for the rules and the rules are permanently valid.

    1. Find the rule whose configurations you want to modify and click Edit in the Actions column.

    2. Configure canary release settings and specify validity periods.

      • Canary Release: You can turn on Canary Release to apply the rule to a specific proportion of objects.

        If you turn on Canary Release, you must configure the Dimension and Canary Release Ratio parameters. Valid values of the Dimension parameter: IP Address, Custom Header, Custom Header, Custom Cookie, and Session.

      • Effective Mode

        • Permanently Effective (default): If you select Permanently Effective, the protection rule is permanently in effect.

        • Fixed Schedule: You can specify a time zone and a period of time during which you want the template to be in effect.

        • Recurring Schedule: You can specify a time zone, days of the week, and a period of time in a day. The template is in effect during the same period of time in a day on the same days of the week.

    You can select multiple rules to specify validity periods and configure canary release settings for the rules at the same time.

  7. In the Verify Protection Effect step, test the anti-crawler rule.

    To prevent false positives that are caused by improper rule configurations or compatibility issues, we recommend that you verify the protection effect of the rule before you publish an anti-crawler rule If the configurations are correct, click Skip.

    Test steps:

    1. Step 1: Enter a public IP address.

      Enter the public IP address of your test device. The test device can be a computer or mobile phone. The test takes effect only for the public IP address. The test does not affect your business.

      Important

      Do not enter the IP address that you obtain by running the ipconfig command. This command returns a private IP address. If you are unsure of the public IP address of your test device, you can use an online IP lookup tool to query the public IP address.

    2. Step 2: Select an action.

      Test the effectiveness of the actions that you specified in the Configure Protection Rules step. WAF generates a test rule only for the specified IP address. The actions include JavaScript Validation, Dynamic Token-based Authentication, Slider CAPTCHA Verification, and Block Verification.

      After you click Test for an action, WAF immediately generates a test rule and sends a test request to the test device. In the dialog box that appears, WAF provides the test procedure, expected result, and demonstration. We recommend that you carefully read the information in the dialog box.

      After the test is complete, click I Have Completed the Test to go to the next step. If the test result shows exceptions, click Go Back to optimize the anti-crawler rule based on the instructions that are provided in FAQ. Then, perform the test again.

By default, a newly created template is enabled. On the Scenario-specific Protection tab, you can perform the following operations:

  • Click a template card to view the rule information about the template.

  • Copy, Edit, or Delete a template.

  • Turn on or turn off the switch to enable or disable a template.

  • View the settings of the Action and Protected Object/Group parameters of a template.

Create an anti-crawler rule for apps

You can configure anti-crawler rules for native iOS or Android apps to protect your services against crawlers. HTML5 apps are not native iOS or Android apps.

Note

If a request from a client matches a protection rule in which the Action parameter is set to Run JavaScript Validation or Slider CAPTCHA, WAF performs JavaScript validation or slider CAPTCHA verification on the client. If the client passes the validation or verification, WAF adds the acw_sc__v2 or acw_sc__v3 cookie to the header of the request to indicate that the client passed the validation or verification.

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and region of the WAF instance. You can select Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Bot Management.

  3. On the Scenario-specific Protection tab, click Create Template.

  4. In the Configure Scenarios step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Template Name

    Enter a name for the template.

    The name of the template must be 1 to 255 characters in length and can contain letters, digits, periods (.), underscores (_), and hyphens (-).

    Template Description

    Enter a description for the template.

    Service Type

    Select App to configure anti-crawler rules for native iOS and Android apps. HTML5 apps are excluded.

    App SDK Integration

    WAF provides the Anti-Bot SDK to enhance protection capabilities for native Android and iOS apps. After the Anti-Bot SDK is integrated into apps, the Anti-Bot SDK collects the characteristics of clients and generates security signatures in requests. WAF identifies and blocks requests that are identified as unsafe based on the signatures.

    You can perform the following steps to integrate the Anti-Bot SDK.

    1. Obtain the SDK for iOS apps. To obtain the SDK, submit a ticket.

    2. Click Obtain and Copy AppKey to send SDK initialization requests.

    3. Integrate the Anti-Bot SDK into your apps. For more information, see Integrate the Anti-Bot SDK into iOS apps.

    Traffic Characteristics

    Add match conditions to identify traffic that is destined for the domain name that you want to protect. To add a match condition, you must configure the match field, logical operator, and match content. Make sure that the match field is a header field of HTTP requests. You can add up to five conditions. The conditions are evaluated by using a logical AND. For more information about match fields, see Match conditions.

  5. In the Configure Protection Rules step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Risk Identification

    Select Business Security and enter relevant information. For more information, see Risk identification.

    The feature helps block requests from abnormal mobile phone numbers based on Fraud Detection. You are charged based on rule hits.

    Bot Characteristic Detection

    • Detection rules

      Invalid App Signature (selected by default)

      By default, this option is selected and cannot be deselected. After the Anti-Bot SDK is integrated, WAF can detect requests that contain invalid signatures or do not contain signatures.

      Custom Signature Field (deselected by default)

      If you select Custom Signature Field, you must configure the Field Name parameter. You can select Cookie, Parameter, or Header from the drop-down list. Then, enter a custom value for the Value parameter. If the body of the request to be signed is empty, contains special characters, or exceeds the length limit, you can hash the signature and enter the returned string for the Value parameter.

      Abnormal Device Behavior

      After you enable this feature, WAF detects and controls requests from the devices that have abnormal behaviors. Valid values of this parameter include Expired Signature, Using Simulator, Using Proxy, Rooted Device, Debugging Mode, Hooking, Multiboxing, Simulated Execution, and Script Tools.

    • Protection action

      You can select Monitor, Block, or Strict Slider CAPTCHA Verification as the action that you want WAF to perform on the requests that match the rule specified below Bot Characteristic Detection.

    • Advanced protection

      Click Advanced Protection and configure the following parameters:

      Secondary Packaging Detection

      • Rule settings

        Requests that are sent from apps whose package names or package signatures are not in the whitelist are considered repackaging requests. Specify valid app packages.

        • Valid Package Name: Enter a valid app package name. Example: example.aliyundoc.com.

        • Signature: Specify the app package signature that needs to be verified. If the signature needs to be verified, submit a ticket. If the package signature does not need to be verified, leave this parameter empty. If the parameter is left empty, WAF verifies only the package name.

        • Important

          The value of the Signature parameter is not the certificate signature of your app.

        You can add up to five valid iOS or Android app packages. The package names must be unique. The logical operator between the conditions is OR.

      • Protection action

        You can select Monitor, Block, Slider CAPTCHA, or Strict Slider CAPTCHA Verification as the action that you want WAF to perform on the requests that match the rule specified below Repackaging Detection.

      Custom Rule

      If the default settings cannot meet your protection requirements, you can configure custom rules. To configure a custom rule, select Custom Rule, click Create Rule, and then configure the following parameters.

      • Match Condition: You can add up to five match conditions. The conditions are evaluated by using a logical AND.

        Click to view the supported match fields.

        eeid_is_root: specifies whether the device has root permissions.

        eeid_is_proxy: specifies whether the device is a proxy.

        eeid_is_simulator: specifies whether the device is a simulator.

        eeid_is_debugged: specifies whether the debugging mode is used.

        eeid_is_hook: specifies whether Hooking techniques are used.

        eeid_is_virtual: specifies whether multiple app processes are running on the device at the same time.

        eeid_is_new: specifies whether the device is a new device.

        eeid_is_wiped: specifies whether the device is suspected of brushing.

        eeid_short_uptime: specifies whether the startup time is excessively short.

        eeid_abnormal_time: specifies whether the system time is abnormal.

        eeid_running_frame_xposed: specifies whether Xposed is used.

        eeid_running_frame_frida: specifies whether Frida is used.

        eeid_running_frame_cydia: specifies whether Cydia is used.

        eeid_running_frame_fishhook: specifies whether fishhook is used.

        eeid_running_frame_va: specifies whether VA Framework is used.

        eeid_running_frame_magisk: specifies whether Magisk is used.

        eeid_running_frame_edxposed: specifies whether EdXposed is used.

        eeid_umid: specifies whether UMID is used.

        appname: specifies the application name.

        packagename: specifies the package name.

        appversion: specifies the version of the app.

        version: specifies the version of the SDK.

        brand: specifies the mobile phone brand.

        model: specifies the mobile phone model.

        product: specifies the product code.

        manufacture: specifies the mobile phone manufacturer.

        hardware: specifies the hardware name.

      • Action: You can select Monitor, Block, Slider CAPTCHA, Strict Slider CAPTCHA Verification, or Add Tag as the action that you want WAF to perform on the requests that match the rule. If you select Add Tag, you must configure the Header Name and Header Content parameters.

      You can add up to 10 custom rules. The rules are evaluated by using a logical OR.

    Bot Behavior Detection

    If you select Intelligent Protection, you must select Monitor, Slider CAPTCHA, Strict Slider CAPTCHA Verification, or Add Tag as the action that you want WAF to perform on detected bot requests. If you select Add Tag, you must configure the Header Name and Header Content parameters.

    After you select Intelligent Protection, the intelligent protection engine analyzes access traffic and performs machine learning. Then, a blacklist or a protection rule is generated based on the analysis results and learned patterns.

    Throttling

    You can configure custom throttling conditions to filter out crawler requests that are frequently initiated. This helps prevent HTTP flood attacks.

    • IP Address Throttling (Default)

      You can configure throttling conditions for IP addresses. If the number of requests that are sent from the same IP address within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Block, Monitor, Slider CAPTCHA, or Strict Slider CAPTCHA Verification from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The conditions are evaluated by using a logical OR.

    • Device Throttling

      You can configure throttling conditions for devices. If the number of requests that are sent from the same device within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Block, Monitor, Slider CAPTCHA, or Strict Slider CAPTCHA Verification from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The conditions are evaluated by using a logical OR.

    • Custom Session Throttling

      You can configure throttling conditions for sessions. You can configure the Session Type parameter to specify the session type. If the number of requests of the same session type within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Block, Monitor, Slider CAPTCHA, or Strict Slider CAPTCHA Verification from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The conditions are evaluated by using a logical OR.

      Valid values of the Session Type parameter are Custom Header, Custom Parameter, Custom Cookie, and Session.

    Bot Threat Intelligence

    Bot Threat Intelligence Library

    The library includes the IP addresses of attackers that have sent multiple requests to crawl content from Alibaba Cloud users. If you select Bot Threat Intelligence Library, you can select Monitor, Slider CAPTCHA, Strict Slider CAPTCHA Verification, or Add Tag as the action that you want WAF to perform on the requests. If you select Add Tag, you must configure the Header Name and Header Content parameters.

    Data Center Blacklist

    If you select Data Center Blacklist, requests that are sent from the IP addresses in the selected IP address libraries of data centers are blocked. You can select Monitor, Slider CAPTCHA, Block, Strict Slider CAPTCHA Verification, or Add Tag as the action that you want WAF to perform on the requests. If you select Add Tag, you must configure the Header Name and Header Content parameters.

    Note

    If you use the source IP addresses of public clouds or data centers to access the website that you want to protect, you must add the IP addresses to the whitelist. For example, you must add the callback IP addresses of Alipay or WeChat and the IP addresses of monitoring applications to the whitelist.

  6. In the Configure Effective Scope step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Apply To

    Select the protected objects or protected object groups to which you want to apply the protection template in the Objects to Select section and click the 移入 icon to move the protected objects or protected object groups to the Selected Objects section.

    Effective Time and Canary Release

    You must specify validity periods and configure canary release settings for the protection rules. If you do not specify validity periods or configure canary release settings, canary release is disabled for the rules and the rules are permanently valid.

    1. Find the rule whose configurations you want to modify and click Edit in the Actions column.

    2. Configure canary release settings and specify validity periods.

      • Canary Release: You can turn on Canary Release to apply the rule to a specific proportion of objects.

        If you turn on Canary Release, you must configure the Dimension and Canary Release Ratio parameters. Valid values of the Dimension parameter: IP Address, Custom Header, Custom Header, Custom Cookie, and Session.

      • Effective Mode

        • Permanently Effective (default): If you select Permanently Effective, the protection rule is permanently in effect.

        • Fixed Schedule: You can specify a time zone and a period of time during which you want the template to be in effect.

        • Recurring Schedule: You can specify a time zone, days of the week, and a period of time in a day. The template is in effect during the same period of time in a day on the same days of the week.

    You can select multiple rules to specify validity periods and configure canary release settings for the rules at the same time.

  7. In the Verify Protection Effect step, test the anti-crawler rule.

    To prevent false positives that are caused by improper rule configurations or compatibility issues, we recommend that you verify the protection effect of the rule before you publish an anti-crawler rule If the configurations are correct, click Skip.

    Test steps:

    1. Step 1: Enter a public IP address.

      Enter the public IP address of your test device. The test device can be a computer or mobile phone. The test takes effect only for the public IP address. The test does not affect your business.

      Important

      Do not enter the IP address that you obtain by running the ipconfig command. This command returns a private IP address. If you are unsure of the public IP address of your test device, you can use an online IP lookup tool to query the public IP address.

    2. Step 2: Select an action.

      Test the effectiveness of the actions that you specified in the Configure Protection Rules step. WAF generates a test rule only for the specified IP address. The actions include Block Verification and SDK Signature Verification.

      After you click Test for an action, WAF immediately generates a test rule and sends a test request to the test device. In the dialog box that appears, WAF provides the test procedure, expected result, and demonstration. We recommend that you carefully read the information in the dialog box.

      After the test is complete, click I Have Completed the Test to go to the next step. If the test result shows exceptions, click Go Back to optimize the anti-crawler rule based on the instructions that are provided in FAQ. Then, perform the test again.

By default, a newly created template is enabled. On the Scenario-specific Protection tab, you can perform the following operations:

  • Click a template card to view the rule information about the template.

  • Copy, Edit, or Delete a template.

  • Turn on or turn off the switch to enable or disable a template.

  • View the settings of the Action and Protected Object/Group parameters of a template.

Create a basic protection rule

You can configure basic protection rules to defend against medium- and low-level crawlers for your services. The bot management module does not provide a default basic protection rule template. Before you can enable the basic protection feature provided by the bot management module, you must create a basic protection rule template.

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and region of the WAF instance. You can select Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Bot Management.

  3. On the Basic Protection tab, click Create Template.

  4. In the Create Template - Bot Management panel, configure the parameters and click OK. The following table describes the parameters.

    Parameter

    Description

    Template Name

    Enter a name for the template.

    The name of the template must be 1 to 255 characters in length and can contain letters, digits, periods (.), underscores (_), and hyphens (-).

    Template Description

    Enter a description for the template.

    Action

    Specify an action that you want WAF to perform on the requests that match the rule. Valid values: Block and Monitor.

    Advanced Settings

    • Canary Release: You can turn on Canary Release to apply the rule to a specific proportion of objects.

      If you turn on Canary Release, you must configure the Dimension and Canary Release Ratio parameters. Valid values of the Dimension parameter: IP Address, Custom Header, Custom Header, Custom Cookie, and Session.

    • Effective Mode

      • Permanently Effective (default): If you select Permanently Effective, the protection rule is permanently in effect.

      • Fixed Schedule: You can specify a time zone and a period of time during which you want the template to be in effect.

      • Recurring Schedule: You can specify a time zone, days of the week, and a period of time in a day. The template is in effect during the same period of time in a day on the same days of the week.

    Apply To

    Select the protected objects and protected object groups to which you want to apply the template.

    You can apply only one template of a protection module to a protected object or protected object group. For more information about how to add protected objects and create protected object groups, see Configure protected objects and protected object groups.

By default, a newly created template is enabled. On the Basic Protection tab, you can perform the following operations:

  • View the IDs of the rules that are included in a template.

    Note

    A basic protection template includes two whitelist rules and one access control or HTTP flood protection rule. You can view the protection performance of the rules on the Security Reports page by using the rule IDs. For more information, see Security reports.

  • Copy, Edit, or Delete a template.

  • Turn on or turn off the switch to enable or disable a template.

  • View the settings of the Action and Protected Object/Group parameters of a template.

FAQ

If an error occurs in the Verify Protection Effect step, refer to the following table to fix the error.

Error

Cause

Solution

No valid test requests are detected. See WAF documentation or contact us to identify the possible causes.

The test request failed to be sent or is not sent to WAF.

Make sure that the test request is sent to the IP address that maps the CNAME provided by WAF.

The header fields in the test request do not match the header fields that you configured for the Traffic Characteristics parameter in the anti-crawler rule.

Modify the Traffic Characteristics parameter in the anti-crawler rule.

The source IP address of the test request is different from the public IP address that you specified in the anti-crawler rule.

Make sure that you use the correct public IP address. We recommend that you use Alibaba Network Diagnose Tool to obtain your public IP address.

The test requests failed the verification. See WAF documentation or contact us to identify the possible causes.

No real user access is simulated. For example, the debugging mode or automation tools are used.

Simulate a real user to access your website or app during the test.

An incorrect service type is selected. For example, Websites is selected when you configure an anti-crawler rule for apps.

Modify the value of the Service Type parameter.

An intermediate domain name is used, but an incorrect intermediate domain name is selected in the anti-crawler rule.

Select Use Intermediate Domain Name. Then, select the correct intermediate domain name from the drop-down list.

Compatibility issues occur in the frontend.

Submit a ticket to contact us.

No verification is triggered. See WAF documentation or contact us to identify the possible causes.

No test rules are generated.

Perform the test several times until a test rule is generated.

No valid test requests are detected or blocked. See WAF documentation or contact us to identify the possible causes.

The test request failed to be sent or is not sent to WAF.

Make sure that the test request is sent to the IP address that maps the CNAME provided by WAF.

The header fields in the test request do not match the header fields that you configured for the Traffic Characteristics parameter in the anti-crawler rule.

Modify the Traffic Characteristics parameter in the anti-crawler rule.

The source IP address of the test request is different from the public IP address that you specified in the anti-crawler rule.

Make sure that you use the correct public IP address. We recommend that you use Alibaba Network Diagnose Tool to obtain your public IP address.

What to do next

On the Security Reports page, you can query the protection details of the protection rules that you configured. For more information, see Security reports.