The bot threat intelligence feature provides information about suspicious IP addresses used by dialers, on-premises data centers, and malicious scanners. This feature also maintains an IP address library of malicious crawlers. You can configure bot threat intelligence rules to prevent malicious crawlers from accessing all pages under your domain name or specific directories.
Prerequisites
- A WAF instance is purchased, and the Bot Manager feature is enabled.
- Your website is added to WAF. For more information, see Tutorials.
Background information
When you configure bot threat intelligence rules, you can specify actions based on the types of threat intelligence libraries. For example, you can specify actions, such as block, JavaScript verification, or CAPTCHA verification. You can also configure bot threat intelligence rules to protect important endpoints against certain threats. This helps you minimize the negative impacts on the services.
Procedure
- Log on to the WAF console.
- In the top navigation bar, select the resource group and the region to which the WAF instance belongs. The region can be Chinese Mainland or Outside Chinese Mainland.
- In the left-side navigation pane, choose .
- In the upper part of the Website Protection page, select the domain name for which you want to configure a whitelist.
- Click the Bot Management tab, find the Bot Threat Intelligence section. Then, turn on Status and click Settings. Note After the bot threat Intelligence feature is enabled, all requests destined for your website are checked by the feature. You can configure the bot management allowlist so that the requests that match required conditions bypass the check of the feature. For more information, see Configure a whitelist for Bot Management.
- In the Bot Threat Intelligence rule list, find the threat intelligence library you want to use, and turn on the switch in the Status column.The following table lists the bot threat intelligence libraries that WAF supports.
Intelligence library Description Malicious Scanner Fingerprint Blacklist This library contains the characteristics of tens of thousands of scanners based on traffic analysis. Malicious Scanner IP Blacklist This library contains malicious IP addresses that are dynamically updated based on the source IP addresses of scan attacks detected on Alibaba Cloud. Credential Stuffing IP Blacklist This library contains hundreds of thousands of malicious IP addresses that are updated based on the source IP addresses of credential stuffing and brute-force attacks detected on Alibaba Cloud. Fake Crawler Blacklist This library identifies crawlers that use the User-Agent of authorized search engines, such as BaiduSpider, to disguise as authorized programs. Important Before you enable this library, make sure that a crawler allowlist is configured. Otherwise, false positives may occur. For more information, see Configure the allowed crawlers function.Malicious Crawler Blacklist This library contains millions of malicious IP addresses that are dynamically updated based on the source IP addresses of crawlers detected on Alibaba Cloud. This library is categorized into three severity levels: low, medium, and high. A higher severity indicates more IP addresses in the library and a higher false positive rate. Note We recommend that you set up two-factor authentication, such as CAPTCHA and JavaScript verification, for the high-severity library.In scenarios in which two-factor authentication cannot be implemented, we recommend that you configure threat intelligence rules based on the low-severity library.
IDC IP Lists These libraries contain IP addresses of public clouds and on-premises data centers, including Alibaba Cloud, Tencent Cloud, Meituan Open Services, and 21Vianet. Attackers typically use CIDR blocks of public clouds or on-premises data centers to deploy crawlers or as proxies to access websites. Regular users rarely access websites in this way. After you enable a default rule, WAF performs the Monitor action on requests initiated from IP addresses in the threat intelligence library that correspond to the rule to the directories of the protected domain name. This action allows the requests to the destination directories and records the requests in logs.If you need to modify a default rule, see the following section on how to configure a custom threat intelligence rule. For example, if you want to specify the protected URL or action, see the following section, step7.
- Optional:Configure a custom threat intelligence rule.