Overview
Abnormal traffic spikes in Object Storage Service (OSS) may be caused by malicious parties that hotlink your resources or send large amounts of requests to your servers. This topic describes how to detect and troubleshoot abnormal OSS traffic.
Solutions
To fix a traffic anomaly, you need to detect and analyze abnormal traffic. This issue is usually caused by traffic generated when malicious parties employ tactics such as Referer-based hotlinking and request flooding.
Malicious requests from IP addresses
Identify IP addresses from which malicious requests are sent.
Method 1: Check ranking statistics in the OSS console
Log on to the OSS console. In the left-side navigation pane, click Buckets. On the Buckets page, click the name of the bucket whose ranking statistics you want to check.
In the left-side navigation tree, choose Data Usage > Ranking Statistics. On the page that appears, click the Referer/IP Address Rankings tab.
On the IP Address (TOP 10) tab, check the number of visits with errors, volume of traffic, and PVs from the IP addresses in the rankings.
Method 2: Check bucket access logs
Identify malicious IP addresses by checking bucket access logs and block the IP addresses that send malicious requests. You can use the following methods to check and analyze logs of access to a bucket:
Logging: Make sure that the logging feature is enabled for the bucket. For more information, see Configure logging. You can use log analytics to analyze the access logs. For example, use the following
awkcommand to obtain a list of IP addresses that have sent large numbers of requests that are not processed as back-to-origin requests by CDN. Then, you can check the IP addresses to determine whether they have sent malicious requests.cat mylog-oss-example2017-09-10-04-00-00-0000 |awk '{if ($(NF-1) ~/-/) print $1}' | sort |uniq -c|sort -nr -k 1|head -20NoteFor more information about fields in OSS logs, see Logging.
Real-time log query: The real-time log query allows you to query the access logs of your bucket that were generated within the last seven days free of charge. You can analyze these logs to obtain IP addresses that have sent large numbers of requests that are not processed as back-to-origin requests by CDN.
Check the ranking statistics in the OSS console to identify malicious IP addresses. For more information, see View the resource usage of a bucket.
Defend against malicious requests from IP addresses based on bucket ACLs.
Private bucket: We recommend that you migrate data from the original private bucket to a new private bucket. Then, you can add the custom domain name of the bucket to Anti-DDoS Pro and Anti-DDoS Premium or Web Application Firewall (WAF). This provides protection for your external-facing services. For more information about data migration, see Overview. For more information about how to map custom domain names, see Map custom domain names.
Public-read bucket: Use one of the following methods to block malicious requests:
Change the ACL of the bucket to private and use signed URLs to provide external-facing services. Compared with a public-read bucket, downloading data from a private bucket is more complex and requires signing. For more information about signing, see Include a V1 signature in a URL.
(Recommended) Migrate data from the original bucket to a new bucket and provide external-facing services through a custom domain name protected by Anti-DDoS Pro and Anti-DDoS Premium or WAF. For more information about data migration, see Overview. For more information about how to map custom domain names, see Map custom domain names.
Migrate data from the original bucket to a new bucket and provide external-facing service through a custom domain name of the new bucket. Meanwhile, enable CDN-based acceleration for the new bucket and use the IP address blacklist feature of the CDN service to block IP addresses that send malicious requests. For more information, see Configure an IP address blacklist or whitelist.
NoteCDN limits the number of IP addresses that can be added to an IP address blacklist.
Referer-based hotlinking
Identify malicious Referers.
Method 1: Check ranking statistics in the OSS console
Log on to the OSS console. In the left-side navigation pane, click Buckets. On the Buckets page, click the name of the bucket whose ranking statistics you want to check.
In the left-side navigation tree, choose Data Usage > Ranking Statistics. On the page that appears, click the Referer/IP Address Rankings tab.
On the TOP 10 Referers tab, check whether the number of visits from the Referers is abnormal.
Method 2: Check bucket access logs
You can enable the real-time log query feature of OSS to detect whether objects in the bucket are requested by malicious Referers. You can use the following methods to check and analyze logs of access to a bucket:
Logging: Make sure that the logging feature is enabled for the bucket. For more information, see Configure logging. Then, use log analytics to analyze the access logs. For example, use the awk command to obtain a list of Referers that have sent large numbers of hotlinking requests.
NoteFor more information about fields in OSS logs, see Logging.
Real-time log query: The real-time log query allows you to query the access logs of your bucket that were generated within the last seven days free of charge. You can analyze these logs to obtain IP addresses that have sent large numbers of requests that are not processed as back-to-origin requests by CDN.
Configure hotlink protection.
Use the OSS console or OSS API to configure hotlink protection for the bucket. For more information, see Hotlink protection.