Access Redis by Using Higress Based on Custom Plug-ins

This article introduces Higress and describes how Redis plugins can be utilized for implementing sophisticated features such as rate limiting, caching, and session management.

By Yucheng

This article introduces Higress, a gateway that leverages WebAssembly (WASM) for edge computing and allows users to create plug-ins in Go, C++, or Rust to expand its functionality. It specifically highlights how Redis®* plugins can be utilized for implementing sophisticated features such as rate limiting, caching, and session management.

Introduction

Higress provides excellent extensibility based on the WASM mechanism. Users can write WASM plugins using Go, C++, or Rust and customize the request processing logic to meet their personalized needs. Currently, the plugins have incorporated support for Redis® calls, enabling users to write stateful plugins and further enhancing the extensibility of Higress.

The documentation Use a Plug-in to Call Redis® Services [1] provides a complete example of how to use a gateway to call Redis® by using plug-ins, including the processes of creating and configuring an Tair (Redis® OSS-Compatible) instance, writing plug-in code, uploading and configuring plug-ins, and testing samples. Next, this article focuses on several plug-ins based on Redis®.

Global Throttling for Multiple Gateways

The gateway has already provided the sentinal throttling [2], which can effectively protect backend business applications. The Redis® plug-in throttling allows you to implement global quota management for multiple gateways.

The following code provides an example of how to use the plug-in to check the number of requests in the current period at the request header stage. If the number of requests exceeds the quota, the plug-in directly returns a 429 response.

func onHttpRequestHeaders(ctx wrapper.HttpContext, config RedisCallConfig, log wrapper.Log) types.Action {
    now := time.Now()
    minuteAligned := now.Truncate(time.Minute)
    timeStamp := strconv.FormatInt(minuteAligned.Unix(), 10)
    // If "err != nil" is returned by the Redis API, the gateway fails to find the Redis backend service. In this case, check whether the Redis backend service is deleted.
    err := config.client.Incr(timeStamp, func(response resp.Value) {
        if response.Error() != nil {
            log.Errorf("call redis error: %v", response.Error())
            proxywasm.ResumeHttpRequest()
        } else {
            ctx.SetContext("timeStamp", timeStamp)
            ctx.SetContext("callTimeLeft", strconv.Itoa(config.qpm-response.Integer()))
            if response.Integer() == 1 {
                err := config.client.Expire(timeStamp, 60, func(response resp.Value) {
                    if response.Error() != nil {
                        log.Errorf("call redis error: %v", response.Error())
                    }
                    proxywasm.ResumeHttpRequest()
                })
                if err != nil {
                    log.Errorf("Error occured while calling redis, it seems cannot find the redis cluster.")
                    proxywasm.ResumeHttpRequest()
                }
            } else {
                if response.Integer() > config.qpm {
                    proxywasm.SendHttpResponse(429, [][2]string{{"timeStamp", timeStamp}, {"callTimeLeft", "0"}}, []byte("Too many requests\n"), -1)
                } else {
                    proxywasm.ResumeHttpRequest()
                }
            }
        }
    })
    if err != nil {
        // Since the calling of Redis service fails, allow the request to pass and log this event.
        log.Errorf("Error occured while calling redis, it seems cannot find the redis cluster.")
        return types.ActionContinue
    } else {
        // Hold the request to wait for completing the calling of Redis service.
        return types.ActionPause
    }
}

The following figure shows the plug-in configuration.

Test results:

Implement Token Throttling by Combining Qwen

For developers who provide AI application services, user token quota management is a very critical feature. The following example shows how to use the gateway plug-in to implement token throttling for the backend services of Tongyi Qianwen (Qwen).

First of all, you need to apply for Qwen API access. For more information, see the reference [3]. Then, configure the corresponding services and routes on the MSE gateway as follows:

Write the plug-in code. In the plug-in, write the token quota used by the request during the response to the body stage, and read Redis® to check the current remaining token quota during the processing of requests stage. If there is no token quota, the response is directly returned and the request is terminated.

func onHttpRequestBody(ctx wrapper.HttpContext, config TokenLimiterConfig, body []byte, log wrapper.Log) types.Action {
  now := time.Now()
  minuteAligned := now.Truncate(time.Minute)
  timeStamp := strconv.FormatInt(minuteAligned.Unix(), 10)
  config.client.Get(timeStamp, func(response resp.Value) {
    if response.Error() != nil {
      defer proxywasm.ResumeHttpRequest()
      log.Errorf("Error occured while calling redis")
    } else {
      tokenUsed := response.Integer()
      if config.tpm < tokenUsed {
        proxywasm.SendHttpResponse(429, [][2]string{{"timeStamp", timeStamp}, {"TokenLeft", fmt.Sprint(config.tpm - tokenUsed)}}, []byte("No token left\n"), -1)
      } else {
        proxywasm.ResumeHttpRequest()
      }
    }
  })

  return types.ActionPause
}

func onHttpResponseBody(ctx wrapper.HttpContext, config TokenLimiterConfig, body []byte, log wrapper.Log) types.Action {
  now := time.Now()
  minuteAligned := now.Truncate(time.Minute)
  timeStamp := strconv.FormatInt(minuteAligned.Unix(), 10)
  tokens := int(gjson.ParseBytes(body).Get("usage").Get("total_tokens").Int())
  config.client.IncrBy(timeStamp, tokens, func(response resp.Value) {
    if response.Error() != nil {
      defer proxywasm.ResumeHttpResponse()
      log.Errorf("Error occured while calling redis")
    } else {
      if response.Integer() == tokens {
        config.client.Expire(timeStamp, 60, func(response resp.Value) {
          defer proxywasm.ResumeHttpResponse()
          if response.Error() != nil {
            log.Errorf("Error occured while calling redis")
          }
        })
      }
    }
  })
  return types.ActionPause
}

Test results:

Cookie-based Caching, Disaster Recovery, and Session Management

In addition to the preceding two throttling examples, more plug-ins can be implemented based on Redis® to extend the gateway. For example, cookie-based caching, disaster recovery, and session management can be implemented.

• Caching and disaster recovery: caching request responses based on user cookie information can reduce the pressure on backend services and implement disaster recovery when backend services are unavailable.

• Session management: Redis® is used to store user authentication information. When a request arrives, Redis® is first accessed to check whether the current user is authorized to access. If the current user is not authorized, the authentication service is then accessed so that the pressure on the authentication service can be reduced.

func onHttpRequestHeaders(ctx wrapper.HttpContext, config HelloWorldConfig, log wrapper.Log) types.Action {
  cookieHeader, err := proxywasm.GetHttpRequestHeader("cookie")
  if err != nil {
    proxywasm.LogErrorf("error getting cookie header: %v", err)
    // Implement your business logic.
  }
    // Process the cookie as needed.
  cookie := CookieHandler(cookieHeader)
  config.client.Get(cookie, func(response resp.Value) {
    if response.Error() != nil {
      log.Errorf("Error occured while calling redis")
      proxywasm.ResumeHttpRequest()
    } else {
      // Implement your business logic.
      proxywasm.ResumeHttpRequest()
    }
  })
  return types.ActionPause
}

Summary

Higress has significantly expanded the capabilities of its plug-ins by enabling Redis® calls, providing more flexibility to cater to the diverse personalized needs of developers. If you have further ideas or suggestions for Higress, feel free to reach out to us!

Reference

[1] Use a Plug-in to Call Tair (Redis® OSS-Compatible) Services
[2] Sentinal Throttling
[3] Link (Currently available in Chinese only)

*Redis is a registered trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by Alibaba Cloud is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and Alibaba Cloud.

Community

Access Redis by Using Higress Based on Custom Plug-ins

Introduction

Global Throttling for Multiple Gateways

Implement Token Throttling by Combining Qwen

Cookie-based Caching, Disaster Recovery, and Session Management

Summary

Reference

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Tair (Redis® OSS-Compatible)

Microservices Engine (MSE)

Managed Service for Prometheus

Function Compute