Configure the Intra-zone Provider First feature

The Intra-zone Provider First feature provided by Microservices Engine (MSE) serves as a load balancing policy. Its core mechanism is to dynamically identify the attributes of the zones where consumers and providers are deployed and preferentially route requests to the providers in the same zone. Compared with the traditional polling algorithm, this feature helps reduce cross-zone traffic transfers. This can effectively reduce network latency, improve the service response speed, and enhance the disaster recovery capability of the system. This topic describes how to configure the Intra-zone Provider First feature in the MSE console.

Prerequisites

Microservices Governance of MSE is activated. For more information, see Activate Microservices Governance.
Microservices Governance of MSE is enabled for the desired application and its consumers. For more information, see Enable Microservices Governance for Java microservice applications in an ACK or ACS cluster and Enable Microservices Governance for microservice applications on ECS instances.

Important

The Intra-zone Provider First feature is supported by the Dubbo and Spring Cloud services. This feature is not supported by Kubernetes services.
Before you use the Intra-zone Provider First feature provided by MSE, you must set a proper security threshold. This feature is implemented only when the percentage of providers in a zone exceeds the specified security threshold.

Terms

Zone: A zone is a physical area in the same region that has independent power supplies and networks. For example, the China (Beijing) region has 12 zones, including Beijing Zone A and Beijing Zone B. The network latency between application instances that are deployed in the same zone is lower than the network latency between application instances that are deployed in different zones.
Consumers and providers: In microservices scenarios, the application instance that initiates a call is referred to as a consumer, and the application instance that provides services is referred to as a provider. In most cases, an application can work as a provider and a consumer at the same time.
RT: the round-trip time of a request, which is the period from the time the client initiates a request to the time the client receives a response from the server.
Instance: An instance is referred to as a node in a colloquial expression. In Kubernetes scenarios, a pod of an application workload is considered an application instance. In Elastic Compute Service (ECS) scenarios, a single application process on an ECS instance is considered an application instance.

Benefits

The Intra-zone Provider First feature serves as a traffic scheduling policy in the microservices architecture. This feature uses the load balancing mechanism to preferentially select a provider that is deployed in the same zone as its consumer. In the multi-zone architecture, the Intra-zone Provider First feature offers a wide range of benefits, including but not limited to:

Compared with cross-zone calls, intra-zone calls reduce the RT of the entire system.
Microservice calls must be made in the same zone. If a failure occurs in a single zone, the impact is limited to the zone.

Note

The Intra-zone Provider First feature is not mandatory. If you want to reduce the system-wide RT and improve the system-wide availability, we recommend that you enable this feature.

Feature description

Microservices Governance of MSE provides the Intra-zone Provider First feature. If you enable this feature for an application, its consumer preferentially initiates a call to an instance of the application in the same zone. For example, this feature is enabled for Application P. When Applications C1, C2, and C3 initiate calls to Application P, the applications preferentially call an instance of Application P in the zones where the applications are deployed.

The following figure shows the application call model used when the Intra-zone Provider First feature provided by MSE is not enabled.

The following figure shows the application call model used when the Intra-zone Provider First feature provided by MSE is enabled.

Important

The Intra-zone Provider First feature provided by MSE is effective only for a single provider. If you enable the Intra-zone Provider First feature for an application, all consumers of the application preferentially call application instances in the regions where the consumers are deployed.

Security threshold

When you use the Intra-zone Provider First feature provided by MSE, you may encounter the following issue: After you enable the Intra-zone Provider First feature for an application, all consumers call providers in the same zone. If the number of providers in a zone is less than the number of consumers, requests from specific consumers may fail. Actually, not all nodes of every application are evenly distributed across zones. In some cases, a small number of providers are deployed in a zone, and may not be able to handle traffic calls initiated by consumers in the zone.

The following figure shows a scenario in which three providers are deployed in Zone A, three providers are deployed in Zone B, and one provider is deployed in Zone C. If you enable the Intra-zone Provider First feature for the applications, one-third of the traffic is received in each zone on the assumption that the number of consumers in each zone is the same. However, in this scenario, only one provider is available in Zone C. The traffic that the only provider needs to handle is three times the traffic that each provider in other regions needs to handle. This may pose stability risks.

In this case, we expect that consumers in Zone C do not implement the logic of the Intra-zone Provider First feature. To address this issue, MSE allows you to configure a security threshold. When the Intra-zone Provider First feature is enabled for an application, if the percentage of application instances deployed in a zone is less than the specified security threshold, consumers in the zone do not implement the logic of the feature during calls to the application. Instead, the default random or round-robin policy of the microservices framework is applied.

For example, the numbers of application instances that are deployed in Zones A, B, and C are 2, 2, and 1, respectively, which account for 40%, 40%, and 20% of all application instances. If you enable the Intra-zone Provider First feature for the application and set the security threshold to 30%, consumers in Zone C do not preferentially call the providers in the same zone. Consumers in Zones A and B can still preferentially call the providers in the same zone.

Prerequisites

The Intra-zone Provider First feature is applied to calls between application instances that are deployed across zones. This is based on the assumption that the environment for resource deployment has multiple zones, and application instances are deployed across these zones.

Important

If instances of your application are not completely evenly deployed across zones, do not use the Intra-zone Provider First feature.

If instances of your application are evenly deployed across zones, we recommend that you use the Intra-zone Provider First feature.
If the number of application instances deployed in different zones is slightly different, we still recommend that you use the Intra-zone Provider First feature. However, you need to perform an estimation and configure a security threshold. For more information about how to perform an estimation and configure a security threshold, see Configure a security threshold.

Deploy application instances across zones

If you use Alibaba Cloud Container Service for Kubernetes (ACK) to manage and deploy applications, make sure that the network configuration of your node pool contains vSwitches in multiple zones. This ensures that your workloads can be deployed on nodes in different zones. In addition, we recommend that you deploy application instances across multiple zones. You can add a pod topology spread constraint to spec > template > spec of your workload. For more information about how to deploy application instances across zones, see Suggested configurations for creating HA clusters.
```
topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: "topology.kubernetes.io/zone"
    whenUnsatisfiable: ScheduleAnyway
```
If your application is deployed on an Elastic Compute Service (ECS) instance, you must select vSwitches in different zones for the ECS instance when you create the instance. When you deploy an application, you can manually deploy instances of the application across the zones.

Procedure

Log on to the MSE console, and select a region in the top navigation bar.
In the left-side navigation pane, choose Microservices Governance > Application Governance. On the page that appears, click the resource card of the application that you want to manage.
In the left-side navigation pane, click Traffic management. On the page that appears, click the Intra-zone Provider First tab.
Click Settings next to Configuration Information, turn on the Enable Status switch, specify Security Threshold, and then click OK.
After you perform the preceding steps, when a consumer initiates a call to the application, the consumer preferentially selects a provider in the same zone. After you click OK, the feature immediately takes effect without requiring you to restart the application.

Configure a security threshold

After you configure a security threshold, if the percentage of application provider instances in a single zone is less than the specified security threshold, consumers in the zone do not preferentially call the provider instances in the same zone. Instead, the default random or round-robin call policy of the microservices framework is used. For more information, see Security threshold in this topic.

You should properly configure a security threshold based on the deployment status of your business application. The core objective is to prevent a small number of application instances in a zone from being overwhelmed by excessive requests. You can configure a security threshold by referring to the following scenarios:

Scenario 1 (common): Application instances are evenly deployed across zones. In this scenario, we recommend that you set the security threshold to a value that is less than 1 divided by the number of zones. For example, an application has six instances, of which two instances are deployed in Zone A, two instances are deployed in Zone B, and two instances are deployed in Zone C. Application instances in each zone account for 33.33% of all application instances. In this case, you can set the security threshold to 33%.

Scenario 2: Application instances are not evenly deployed across zones. In this case, you must configure a security threshold based on the number of instances in each zone. For example, for an application, two application instances are deployed in Zone A, two application instances are deployed in Zone B, and one application instance is deployed in Zone C. It is expected that the Intra-zone Provider First feature is not implemented in Zone C. In this case, you can set the security threshold to 30%. Application instances in Zone A account for 40% of all application instances. This percentage is the same as the percentage of application instances in Zone B. Application instances in Zone C account for 20% of all application instances. Instead of the logic implemented by the Intra-zone Provider First feature, the default random or round-robin policy is applied in Zone C.

Important

The default value of the security threshold is 20%, which is usually used for verification in test environments and trial use. We recommend that you configure a security threshold based on the deployment status of application instances in zones.

Usage notes

If the percentage of providers in a zone is strictly greater than the specified security threshold, the Intra-zone Provider First feature takes effect. If the percentage of providers in a zone is equal to the security threshold, the Intra-zone Provider First feature does not take effect.
Ideally, providers and consumers are evenly deployed across zones. If providers and consumers are not evenly deployed but are centrally deployed in specific zones, traffic loads may be unbalanced in different zones after the Intra-zone Provider First feature is enabled.
When a service is released, the number of application instances in each zone may change within a short period of time, especially during a rolling update in Kubernetes. In this case, the percentage of application instances in some zones may not reach the security threshold, which may result in cross-zone provider calls within a short period of time.
The Intra-zone Provider First feature is suitable only for isolation at the traffic level, and should not be used for isolation at the business level. If you use this feature, you must make sure that cross-zone provider calls are allowed.
When the security threshold is in effect, the system checks whether the percentage of available application instances in a zone reaches the security threshold. The number of available application instances and the total number of application instances are calculated based on the logical filtering result of end-to-end canary release. (In most cases, a business system uses the end-to-end canary release feature only when a business version is released. You do not need to take note of this item. If your system uses the end-to-end canary release feature for normal route filtering in addition to version releases, you need to take note of this item.)

Note

For example, an application has five instances, of which two instances are deployed in Zone A, two instances are deployed in Zone B, and one instance is deployed in Zone C. In Zones A and B, one canary instance is available. For consumers that use the official version in Zone A, only one instance of the application in each zone can be called. If the security threshold is set to 35% (less than 40%), the Intra-zone Provider First feature does not take effect in Zones A and B for consumers that use the official version. For consumers that use the canary version in Zone A, one instance of the application in Zones A and B can be called. No application instance can be called in Zone C. The Intra-zone Provider First feature takes effect for consumers that use the canary version.

Traffic observation from perspectives of zones

Node and traffic distribution in zones

The Intra-zone Provider First feature provides the observation capability. After you enable MSE Microservices Governance for an application, you can view the instance deployment distribution of the current application and the traffic load in each zone on the Intra-zone Provider First tab.

You can also view the above-mentioned information if the Intra-zone Provider First feature is not enabled for the application.

Important

If you use Microservices Governance Professional Edition or if your namespace is a namespace of the Professional Edition, you cannot view the above-mentioned information. Use the following method to upgrade the namespace:

Upgrade the namespace from the Professional Edition to the Enterprise Edition. For more information, see Microservice namespace management.

Key points to note

In the Traffic Observation section of the Intra-zone Provider First tab, you can view the number of application instances deployed in each zone in Overall Data. Visual data can show the unbalanced distribution of application instances in different zones. We recommend that you evenly deploy the instances of your application across zones to improve system availability.

If the Intra-zone Provider First feature has been enabled for your application, you need to check the amount of traffic in zones, the amount of traffic processed in each zone, and the number of application instances deployed in each zone. For example, a large number of application instances are deployed in a zone, but the amount of traffic processed in the zone is small. Another example is that a small number of application instances are deployed in a zone, but the amount of traffic processed in the zone is large. In the two examples, you need to adjust the security threshold to prevent stability issues caused by unbalanced traffic distribution.

Reduced RT

If you do not enable the Intra-zone Provider First feature, the default random or round-robin policy is used for microservice calls. This causes a large number of microservice calls across zones. The following figure shows the data of an application for which the Intra-zone Provider First feature is not enabled. The displayed data shows that the average RT in the previous 5 minutes is 7.88 ms.

After the Intra-zone Provider First feature is enabled for all instances of the application in different zones, the data displayed in the following figure shows that the average RT changes to 6.85 ms.