Alibaba Released Sentinel Go 0.4.0 with Hot Parameter Throttling

Sentinel is an Alibaba open-source throttling component for distributed service architectures. Taking traffic as its breakout point, Sentinel helps developers ensure the stability of microservices in various ways, such as concurrency limiting, traffic shaping, circuit breaking, and adaptive system protection. Sentinel has supported core scenarios during Alibaba's Double 11 over the past ten years, such as flash sales, cold start, load shifting of messages, cluster throttling, and real-time circuit breaking of unavailable downstream services. It is a powerful tool to ensure highly available microservices and supports multiple programming languages, such as Java, Go, and C++. It also provides global throttling for Istio, Envoy, SOFA MOSN, and other service meshes to ensure reliability.

Sentinel Go 0.4.0 was officially released recently with the hot parameter throttling feature. It can automatically identify, count, and throttle "hot values" in input parameters. It is very useful in scenarios, such as click farming prevention and hot item access frequency control. It is oriented for highly available traffic. Next, let's look at the scenarios and principles of hot parameter throttling.

Hot Traffic Prevention

Traffic is random and unpredictable. We usually configure throttling rules for core APIs to prevent system crashes due to heavy traffic. However, common throttling rules are not enough in some scenarios. For example, a lot of popular items have extremely high transient page views (PVs) at peak hours during major promotions. Generally, we can predict some popular items and cache information about these requests. When there is a large number of PVs, product information can be returned quickly from a preloaded cache without accessing the database. However, a promotion may have some unexpected popular items whose information was not cached beforehand. When the number of PVs to these unexpected popular items increases abruptly, a large number of requests will be forwarded to the database. As a result, the database access speed will be slow, resource pools for common item requests will be occupied, and the system may crash. In this scenario, we can use the hot parameter throttling feature of Sentinel to automatically identify hot parameters and control the access QPS or concurrency of each hot value. This effectively prevents access requests with hot parameters from occupying normal scheduling resources.

Let's take a look at another example. When we want to restrict the frequency at which each user calls an API, it is not a good idea to use the API name and userId as the tracking resource name. In this scenario, we can use WithArgs(xxx) to transmit the userId as a parameter to the API tracking point and configure hotspot rules to restrict the call frequency of each user. Sentinel also supports independent throttling of specific values for more precise traffic control.

The following code shows tracking and rule examples for hot parameters:

// Tracking example
e, b := sentinel.Entry("my-api", sentinel.WithArgs(rand.Uint32()%3000, "sentinel", uuid.New().String()))

// Rule example
_, err = hotspot.LoadRules([]*hotspot.Rule{
    {
        Resource:          "my-api",
        MetricType:        hotspot.QPS, // Request quantity mode
        ControlBehavior:   hotspot.Reject,
        ParamIndex:        0, // Parameter index. 0 indicates the first parameter.
        Threshold:         50, // Threshold of each hot parameter value
        BurstCount:        0,
        DurationInSec:     1, // Statistics window duration, which is 1s in this example.
        SpecificItems: map[hotspot.SpecificValue]int64{
            // Configures throttling for a specific value. In this example, the restricted request quantity for value 9 is 0 (rejected).
            {ValKind: hotspot.KindInt, ValStr: "9"}: 0,
        },
    },
})

Hot parameter throttling rules can be dynamically configured based on dynamic data sources just like other rules.

The RPC framework integration modules, such as Dubbo and gRPC, provided by Sentinel Go will automatically carry the list of parameters that RPC calls in the tracking points. We can configure hot parameter throttling rules for corresponding parameter positions. Currently, hot parameter throttling rules can only be configured for parameters of the primitive and character string types. In the future, the community will provide support for more types.

Sentinel Go hot parameter throttling is implemented based on the cache termination mechanism and token bucket mechanism. Sentinel uses a cache replacement mechanism (such as LRU, LFU, or ARC policy) to identify hot parameters and the token bucket mechanism to control the PV quantity of each hot parameter. Sentinel Go 0.4.0 uses the LRU policy for hot parameter statistics. In future versions, the community will introduce more cache replacement mechanisms to adapt to different scenarios.

Best Practices of Hot Traffic Protection

In service provider scenarios, the service provider must be protected from overwhelming traffic peaks. Therefore, we often perform throttling based on the service capabilities of a service provider or concurrency limiting based on a specific service consumer. We can use Sentinel to configure QPS throttling rules based on a previous capacity evaluation to ensure the stability of service providers. When the number of requests per second exceeds the preset threshold, excessive requests are automatically rejected. QPS throttling rules can also be combined with hot parameter throttling for fine-grained traffic protection.

We need to protect the service consumers from being affected by unstable but service dependencies in service consumer scenarios. We can use the Sentinel semaphore isolation policy (concurrency throttling rule) to restrict the concurrent number of calls to a service to prevent a large number of slow calls from occupying normal request resources. In addition, with the circuit breaking rule, circuit breaking is performed when the exception ratio or slow service call ratio exceeds a threshold. In the circuit breaking period, we can provide the default processing logic (fallback) to return the fallback result for all calls instead of using unstable services. We still need to configure the request timeout time on the HTTP or RPC client for additional protection even when service consumers use the circuit breaking mechanism.

In scenarios that may involve abrupt increases in requests, such as consumption messages on the MQ client, we may not want to reject excessive messages but queue these messages for processing instead. This is a "load shifting" scenario. We can use constant speed + queuing in the Sentinel throttling rules to deal with this scenario. We can set a fixed interval for routing requests and queue excessive requests. This prevents the system from crashing when all requests are received or rejecting excessive requests.

Sentinel Go also provides global adaptive system protection. Based on a variety of metrics, such as the system load, CPU utilization, service entry QPS, response time, and concurrency, the adaptive system protection policy ensures a balance between the system entry traffic and load. This ensures the system runs stably at maximum throughput. System rules can be used as an additional protection policy to prevent service crashes.

Learn more about AHAS Sentinel at https://github.com/alibaba/Sentinel/wiki/AHAS-Sentinel-%E6%8E%A7%E5%88%B6%E5%8F%B0

Community

Alibaba Released Sentinel Go 0.4.0 with Hot Parameter Throttling

Hot Traffic Prevention

Best Practices of Hot Traffic Protection

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Microservices Engine (MSE)

Managed Service for Prometheus

Global Application Acceleration Solution

ADAM(Advanced Database & Application Migration)