After your website is accelerated by Alibaba Cloud CDN, its content delivery may be slow. Many factors can cause slow content delivery. This topic describes how to troubleshoot the error.
Disclaimer: This topic may contain information about third-party products. The information is for reference only. Alibaba Cloud does not make a guarantee in any form for the performance and reliability of the third-party tools, and potential impacts of operations on these tools.
Problem description
After your website is accelerated by Alibaba Cloud CDN, its content delivery may be slow. Many factors can cause slow content delivery. This topic describes how to troubleshoot the error.
Solution
Note:
If content delivery is slow after your website is accelerated by Alibaba Cloud CDN, you can refer to the following content to resolve the issue.
Troubleshooting
Before you troubleshoot the issue, you need to understand how Alibaba Cloud CDN works. For more information, see How it works. This information helps you identify and analyze the possible causes of the issue. Alibaba Cloud CDN adds points of presence (POPs) to the existing network and caches resources retrieved from the origin server to the POPs closest to users. When the resources are requested again, Alibaba Cloud CDN returns the resources directly from the cache instead of retrieving them from the origin server. This helps reduce origin requests and improve page loading. The following items describe the possible causes behind the low access speed:
Network issues occur on the client, such as insufficient downstream bandwidth and DNS configuration errors.
Network connection between the client and the POP is poor and access latency is high.
An error occurs on the POP, causing a low response speed.
The size of the resource is large and the resource requires a longer period of time to be downloaded.
Network connectivity between POPs and the origin server is poor.
The response speed of the origin server is low.
Take note of symptoms to inform subsequent troubleshooting.
Determine if content delivery is slow for all users, individual users, users in a specific region, or users of a specific Internet service provider (ISP). Alibaba Cloud provides services such as Application Real-Time Monitoring Service (ARMS) and CloudMonitor. You can use the services for more accurate troubleshooting that targets a specific region or ISP:
If content delivery is slow for only a few users, the network of these users may be abnormal.
Determine whether users with problems are centralized. For example, content delivery may be slow for a large number of China Mobile users in a specific city, but normal for China Unicom or China Telecom users. In this case, the network of the ISP in the city may be abnormal. You can use detection tools for the city to check the network conditions.
If content delivery is slow for all users, the response or configuration of the origin server is likely the issue. It is very unlikely that all POPs or networks in all regions have problems at the same time. For example, you can check whether the acceleration region is incorrectly selected or whether the origin server responds slowly to dynamic requests or requests for resources that cannot be cached.
Check whether resources with slow or abnormal responses are cached on POPs.
If a request hits the cache, the cached resource is returned to the client and is not retrieved from the origin server. In this case, slow content delivery is not related to the origin server.
If a request does not hit the cache, you need to check whether slow content delivery is caused by the route between the client and the POP or caused by the origin server.
Observe indicators
When you use Alibaba Cloud CDN to accelerate content delivery, you can obtain scenario-specific indicators in addition to general indicators. You can learn how Alibaba Cloud CDN accelerates content delivery and how it is applied to your services by referencing these indicators. This helps you make adjustments and business decisions. For more information about Alibaba Cloud CDN indicators, see Performance indicators.
Collect information
A complete HTTP or HTTPS request process involves DNS resolution, TCP connection, SSL handshake (for HTTP requests), client request sending, and server responding. Understanding the process helps you better analyze the issue. Therefore, it is necessary to collect the following information on the clients:
Collect network conditions of clients and IP addresses of POPs
Run the ping command to connect to the accelerated domain name. Check whether the domain name is correctly resolved to Alibaba Cloud CDN, whether the network between the client and the POP is accessible, and whether the network latency is high. If you cannot connect to the accelerated domain name, you need to diagnose the network path. For more information, see Test network paths when packet loss or connection failures occur after the ping command is run. If you use a mobile client, you can use third-party applications to assist in diagnosis. For example, you can use Allegro Network Multimeter for Android and iNetTools for iOS.
Collect IP addresses and local DNS servers of clients
The POP routing policy is based on the local DNS server of the client. Therefore, you must confirm that the local DNS server is correctly configured. You can visit https://cdn.dns-detect.alicdn.com/ to obtain the IP address and DNS server of the client.
Identify URLs that have slow access speeds
Turn on the developer mode in the browser, click the Network tab, enter a URL, and then view all HTTP requests sent by the browser on the Network tab. Click the Time column to sort results by time, and view resources that have slow response speed. In the Domain column, find the URLs of these resources.
Note: In most cases, if a website loads a large number of resources, some resources may not be accelerated by Alibaba Cloud CDN. Resources that are not accelerated by Alibaba Cloud CDN have slow access speeds, and resources that are accelerated by Alibaba Cloud CDN have fast access speeds. However, the loading of resources that are not accelerated can slow down the response speed of the website as a whole. Therefore, you can sort the URLs by time to determine which URLs have slow access speeds.
Collect HTTP requests headers and response headers
On the Network tab, find a resource that has a low access speed and click the value in the Name column. On the Headers tab, you can view the General, Response Headers, and Request Headers information of the request for the resource. The request headers and response headers allow you to determine whether a request is a static request and whether it hits the cache.
Note: If you access a resource over a 4G network on a mobile phone and the response is slow, you need to capture packets on the mobile phone to obtain information. This operation may be difficult for regular users. Instead, you can turn on the personal hotspot of the mobile phone, connect to the hotspot from a PC, and collect information on the PC.
Collect the Timing information of HTTP requests
On the Timing tab, view information about the time spent on each part of the resource request process. For more information about Timing, see Understanding Resource Timing.
Common cases
After you understand how Alibaba Cloud CDN works and the HTTP request process, you can perform a preliminary analysis based on the symptoms. Then, you can use the information that is collected on clients to troubleshoot or identify the issue. The following section describes some common cases.
Case 1: poor network quality between the client and the POP
A client uses the ping command to connect to the accelerated domain name. The ping command screen shows that the network latency is high, and packet loss occurs. In this case, you need to collect the IP address and DNS server address of the client, the ping command screenshot, and the My Traceroute (MTR) screenshot. Alibaba Cloud CDN allocates POPs by using the DNS server of the client. Therefore, you can determine whether the routing is abnormal according to the IP address and DNS server of the client and the POP. You can also view the network latency and locate the part of the network link where the latency is generated based on the ping command screenshot and MTR screenshot. The issue in this case may be caused by the following reasons.
Invalid acceleration region
A user in the Chinese mainland is allocated to a POP outside the Chinese mainland, or a user outside the Chinese mainland is allocated to a POP in the Chinese mainland. Scenarios:
Note: In this case, we recommend that you set the acceleration region to Global.
If you set the acceleration region to Chinese Mainland Only, even requests from users outside the Chinese mainland are redirected to POPs in the Chinese mainland.
If you set the acceleration region to Global (Excluding the Chinese Mainland), even requests from users in the Chinese mainland are redirected to POPs outside the Chinese mainland.
Incorrect client DNS setting
The user needs to change to use the DNS server of the corresponding ISP and location.
If a China Mobile (Guangdong) user uses the DNS server of China Unicom, requests from the user are redirected to POPs of China Unicom, and long-distance routing extends the network link.
If a user of China Mobile (Guangdong) uses the DNS server of China Mobile (Harbin), requests from the user are redirected to POPs of China Mobile (Harbin), and long-distance routing extends the network link.
Note: If the acceleration region and DNS server are correct, the user is allocated to a proper POP, but the network quality is still poor. Then you need to collect traceroute and MTR information for further diagnosis.
Case 2: low cache hit ratio or frequent origin fetches
Alibaba Cloud CDN accelerates the delivery of static resources by caching the static resources on POPs that are nearest to the clients. When a user requests a resource, the POP returns the cached resource to the user. This prevents origin fetch over a long link. A low cache hit ratio increases the workload on origin servers and causes low access efficiency for static resources. Therefore, the cache hit ratio affects user experience. It is important to ensure a high cache hit ratio. You can optimize the cache hit ratio based on the causes contributing to the low hit ratio. For more information, see Increase the cache hit ratios of Alibaba Cloud CDN. You can determine whether the cache is hit based on the X-Cache header in the response.
Note:
Solutions for a low cache hit ratio or frequent origin fetches
Slow access to resources for the first time
The first access to an accelerated resource is slower than the direct access to the resource from the origin server, because the resource is not cached on POPs for the first access and needs to be retrieved from the origin server. In this case, we recommend that you use the URL-based prefetch feature to prefetch the content from the origin server to POPs. Then, the first time users access the prefetched content, the cache is hit. This accelerates content loading. For more information, see Purge and prefetch resources.
Low resource access frequency
If a resource is infrequently accessed, Alibaba Cloud CDN receives only a small number of requests for these resources, and they do not hit the cache. POPs are shared by all Alibaba Cloud CDN users. Cache rules in Alibaba Cloud CDN specify the maximum time-to-live (TTL) of resources on POPs. If the traffic of your Alibaba Cloud CDN-accelerated domain name is low, your resources may be removed from the cache of POPs before they expire, The least popular cached resources are the first to be stale in the cache. Popularity is indicated by how frequently a resource is accessed on POPs. If the resource has low popularity, the resource may be removed before it expires.
Improper cache configuration
An improper cache TTL is configured. Scenarios involving frequent origin fetches:
If you do not configure cache rules in Alibaba Cloud CDN and responses for static files do not contain the ETag or Last-modified headers, static files cannot be cached on POPs. To resolve this issue, you need to configure the response headers on the origin servers or configure cache rules in Alibaba Cloud CDN. For more information, see What is caching?
If you do not configure a cache rule in Alibaba Cloud CDN, the default cache policy is used. The maximum cache TTL is 3,600 seconds. This may cause frequent origin fetches after the cached resources expire. We recommend that you set a proper cache TTL based on your business requirements. For more information, see What is caching?.
Cache-Control directives on the origin have a higher priority. Even if you configure a cache rule, Alibaba Cloud CDN does not cache resources as long as the origin response contains Cache-Control directives that do not allow caching. If one of the maxage=0, max-age=0, no-cache, no-store, private, and Pragma: no-cache rules exists in cache rules of the origin server, Alibaba Cloud CDN does not cache resources. In this case, you need to modify the rules on the origin server. Change the value to Public, which allows caching. For more information, see Nginx cache policy settings and Apache cache policy settings.
URLs containing variable parameters
URLs for accessing resources contain variable parameters. When a client uses different URLs to access Alibaba Cloud CDN, Alibaba Cloud CDN considers that two different requests are initiated, and redirects the requests to the origin server to retrieve the file, even if the URLs point to the same file that is cached on POPs. In this case, we recommend that you enable the parameter filtering feature. For more information, see Ignore parameters.
Range origin fetch for large files
If you want to cache large files, we recommend that you enable the range origin fetch feature. For more information, see Range origin fetch.
Case 3: slow dynamic requests
If clients access a dynamic resource, every request is made to the origin server to retrieve the resource and then return the resource to clients. In this scenario, Alibaba Cloud CDN cannot cache dynamic content that changes in real time. Therefore, Alibaba Cloud CDN is not suitable for accelerating dynamic content. POPs redirect requests for dynamic content to origin servers. Therefore, delivery of dynamic content is not accelerated. If your website or app contains a large amount of dynamic content, such as API operations, you can use the following solutions:
Separate static and dynamic resources. Use Alibaba Cloud CDN to accelerate static resources. Redirect dynamic requests to another domain name that is directly resolved to the origin server.
Use Dynamic Content Delivery Network (DCDN) to accelerate dynamic requests. For more information, see What is DCDN? Note: DCDN accelerates dynamic requests by using dynamic acceleration technologies provided by Alibaba Cloud, such as route optimization and transmission optimization. This allows you to retrieve resources from the origin server as fast as possible. DCDN optimizes the network layer. If the response of the origin server is slow, you need to optimize the origin server.
Case 4: slow response of the origin server
Requests for resources that are not cached and dynamic requests are redirected to the origin server. If the response of the origin server is slow, the overall response speed is slow. In this case, you can map a local domain name to the origin server or test the response speed of the origin domain. Causes:
The processing performance of the origin server is limited. For example, the bandwidth or CPU of the origin server reaches a bottleneck, or the processing speed of the origin server is slow. In this case, you can optimize the origin server. If the performance is poor, you need to scale out the origin server.
The network of the origin server is poor or the origin server involves cross-border connections. For example, a user requests a resource from a POP in the Chinese mainland, but the origin server is outside the Chinese mainland. Alibaba Cloud CDN retrieves content from the origin server over the Internet. Therefore, if Alibaba Cloud CDN retrieves content from the origin server over a cross-border connection, content retrieval may be affected. This is because cross-border connections involve different ISPs in and outside the Chinese mainland, and use the international Internet egress. In this case, the networks of Alibaba Cloud CDN and the origin server cannot be controlled, and Alibaba Cloud CDN has a limited potential for optimization. We recommend that you deploy one origin server for requests from the Chinese mainland and one origin server for requests from outside the Chinese mainland.
Case 5: slow loading of website homepages
When a client accesses http://www.example.com/, the browser requests the homepage. After the request is successful, the server returns HTML code to the browser. Then, the browser requests resources to be introduced into the code, such as images, JavaScript files, and CSS files, based on the returned HTML code. If the homepage is a dynamic resource or a resource that does not need to be cached, Alibaba Cloud CDN retrieves the resource from the origin server for each request for the homepage. A slow response from the origin server causes slow loading of the homepage. As a result, requests remain in the Pending state for a long time. For information about whether the cache is hit, see Case 2: low cache hit ratio or frequent origin fetches.
In scenarios where the homepage is not cached and take a long time to load, requests for the homepage remain in the Pending state for a long time. When the homepage is finally loaded, static resources are quickly loaded.
Note: If the homepage is loading slowly, it may be inaccurate to use platforms such as chinaz and 17ce to verify the acceleration effect of Alibaba Cloud CDN. If the probe URL is http://[$Domain], the URL of the homepage is detected and static files on the website are not detected. The acceleration performance can be verified only if the probe URL is a specific static resource URL.
Case 6: large resource loading
If your website has a large amount of resources to be loaded, you can configure the performance optimization feature for the accelerated domain name. For more information, see Performance optimization overview. Intelligent compression supports the following formats: text/html, text/xml, text/plain, text/css, application/javascript, application/x-javascript, application/rss+xml, text/javascript, image/tiff, image/svg+xml, application/json, and application/xmltext.
Case 7: slow access by users of a specific ISP in a specific region
In some scenarios, clients have common problems. For example, a large number of China Mobile users in a specific city may report that access is slow or abnormal during a period of time, but access by China Unicom users is normal. This issue may be related to the network of the ISP or requested POPs in the region. In most cases, you can troubleshoot this issue by collecting ping information on the client and determine the network latency between the client and the POP.
In addition, you can bind the requested IP address to the POP for testing. The testing method is similar to testing on the origin server, except you need to change the IP address of the origin server to the IP address of the POP. Before you bind a POP, you can check whether the POP has a slow response speed. If the response speed is slow, check whether the request hits the cache and whether the loaded resources are too large. Then, analyze the issue based on the preceding cases. If you cannot identify the issue, submit a ticket to contact Alibaba Cloud.
Applicable scope
CDN
DCDN