AI Gateway Introduces Credits Metering & Quota Capability: Making Consumer Usage Measurable and Constrainable

This article introduces AI Gateway's Credits metering and quota capability, which makes consumer LLM usage measurable and constrainable across models and providers.

TL;DR: The AI Gateway has introduced a Credits metering dimension, expanding consumer quotas from "Token only" to "Credits-based". It also adds two new resource types: model metadata and providers. The former carries the Credits unit price and specifications for each model, while the latter carries the provider ownership of the service. Whenever a call occurs, the gateway calculates the Credits consumption based on the model's unit price at that moment and charges it to the respective consumer's quota usage.

I. Why Measuring LLM Invocations is Difficult

If traditional API gateways manage the "traffic traffic lights," then the AI gateway has an extra responsibility — attaching a measurable scale to every invocation.

The biggest difference between LLMs and ordinary APIs is that the resources consumed by each invocation are different: a single Q&A can easily consume thousands or tens of thousands of Tokens, and the consumption among input, output, and cache varies significantly. There is also a lack of a unified baseline for comparison among different models. After running for a while, the three most common questions asked by business teams are:

How much has been consumed in total during this period?
Which business/consumer generated these consumption?
Can we set a "periodic usage limit" for a specific consumer, so that they are blocked directly if they exceed it?

Tokens are the smallest measurable unit, but exposing them directly to business teams is not user-friendly. The weights of Tokens for different models and levels are completely different; you cannot see the difference in resource occupancy just by looking at the number of Tokens. The Credits metering capability of the AI Gateway is designed to solve this by introducing a unified measurement scale that allows cross-model and cross-provider horizontal comparison — Credits, and completes the two tasks of "attribution" and "constraint" around it.

II. The Three Capabilities Released This Time

Capability	What Problem It Solves
Model Metadata Management	Maintains the model list within the instance along with their Credits unit price, capabilities, Token limits, and other specification details. This allows for quick model selection in Model APIs and routing, and tracks Credits consumption by model.
Provider Management	Allows each service and model to be clearly assigned under a provider name, managing and viewing resources from the provider dimension.
Credits Dimension Quota	Evolves quota rules from the "Token" dimension to the "Credits" dimension.

Let's break them down one by one.

III. Model Metadata: Creating a "Specification File" for Each Model

3.1 What is it

Before this, a model was just a string (such as qwen-max, gpt-4o) to the AI gateway. When performing Credits metering, the gateway had no idea what the corresponding input, output, and cache unit prices were for this string.

AI Gateway has added a new class of resource — Model Metadata. Simply put, it creates a "specification file" for each model.

A file contains:

Basic Information: Model name, bound provider
Credits Information (Key Point): Input / Output / Cache rate of Credits / 1M tokens unit price
Capability & Performance: Maximum input Tokens, total Token limit, supported modalities (text/image/audio/video), whether function calling is supported, streaming output, reasoning modes, and other capability toggles.
Available Path: Whether this model uses /v1/chat/completions or other paths, and whether it is OpenAI-compatible or Anthropic-compatible.

3.2 A Concrete Example

Suppose you maintain a model metadata entry in the gateway:

Name: qwen-max
Provider: Alibaba Cloud Bailian
Input Token Consumption: 8   Credits / 1M tokens
Output Token Consumption: 24  Credits / 1M tokens
Cache Token Consumption: 2   Credits / 1M tokens

Now there is an invocation that uses 1 million input Tokens, 200,000 output Tokens, and 50,000 cache Tokens. The gateway will calculate like this at the very moment the request occurs:

Input Credits = 1 million / 1 million × 8  = 8    credits
Output Credits = 200,000 / 1 million × 24 = 4.8  credits
Cache Credits = 50,000 / 1 million × 2  = 0.1  credits
────────────────────────────────
Total Consumption for this call     ≈ 12.9 credits

IV. Provider Resources: Evolving from "Labels" to "Attribution Dimensions"

4.1 Why was it not enough before

In previous versions of the AI Gateway, there was a "LLM Provider" field in the advanced settings of the service, which was essentially a protocol template (openai compatible / anthropic compatible) — it only told the gateway "what protocol to use to interface", but did not mean "which provider the usage of this call should be attributed to."

For example: you could have two "OpenAI-compatible" services, one actually routing to OpenAI and the other to DeepSeek. Their protocols are the same, but they belong to two different providers. The old field could not differentiate them.

4.2 New Capability: Independent Provider Resources

Now, the AI Gateway has isolated "Provider" as an independent resource object within the instance, and renamed the original field to avoid confusion:

Concept	New Name	Purpose
Original "LLM Provider" field	Renamed to Provider Configuration Template	Only describes the integration protocol
Newly added resource	Provider	Carries the dimension of usage attribution

The two concepts are completely decoupled, with no mutual filtering or inheritance. This way:

One provider can be associated with multiple services (1 : N)
A service can only belong to one provider at any given time (avoiding conflicts in statistical data)
A clear binding relationship is formed between services and providers, making it easy for administrators to manage and view resources from the provider dimension

V. Quota Unit Expansion: From "Restricting Tokens" to "Restricting Credits"

This is the most core capability upgrade.

5.1 Understand Its Value in One Sentence

Old Rule__: Set "10 million Tokens per day" for Consumer A — but is A using a lightweight model or a large model? The difference in consumption intensity is unclear.

New Rule__: Set "500 credits per day" for Consumer A — this directly limits the unified measurement scale. Lightweight models can run more times, while large models will naturally run fewer times.

5.2 UI Changes in Creating Quota Rules

Based on the original creation flow, a "Limit Unit" selection has been added to support configuring quotas in the Credits dimension:

Field	Type	Description
Rule Name	Text	Inherited
Limit Unit	Radio: token / credits, default token	New
Limit Value	Numeric, the suffix displays Tokens / Credits dynamically based on the unit	Inherited
Cycle Type	Radio + parameters (calendar day/week/month or custom 1~60 days)	Inherited
Time Zone	Dropdown, default UTC+8	Inherited

VI. Stringing the Three Capabilities Together: What Happens in a Real Invocation

Let's walk through the closed loop with a specific scenario:

Background__: Consumer 'Internal Inference Platform' Monthly Quota = 10000 credits (calendar month + credits)

The following have been configured in the Gateway:

Provider: Alibaba Cloud Bailian
Model Metadata: qwen-max (bound to Alibaba Cloud Bailian, Input 8 / Output 24 / Cache 2 credits per 1M tokens)
Service: qwen-service (associated with the Alibaba Cloud Bailian provider)

Step 1 · Request Arrival The consumer calls with an API Key. The gateway recognizes him as "Internal Inference Platform" with 9987 credits already consumed this month.

Step 2 · Pre-check This request has not been executed yet. The remaining quota is 13 credits, is it enough? Let it pass first.

Step 3 · Upstream Response Calling qwen-max consumed 50,000 input, 100,000 output, and 20,000 cache Tokens.

Step 4 · Conversion at Request Time

input  =  50,000 / 1 million × 8  = 0.4  credits
output = 100,000 / 1 million × 24 = 2.4  credits
cache  =  20,000 / 1 million × 2  = 0.04 credits
────────────────────────
Total approx. 2.84 credits

Step 5 · Quota Deduction The monthly cumulative consumption for the consumer "Internal Inference Platform" is updated to 9989.84 credits, with 10.16 credits remaining.

Step 6 · Next Request Another call with an estimated 20 credits arrives — the remaining 10.16 is not enough, and the gateway limits the request.

VII. Capability Overview

Focus Areas	The Solution Provided by AI Gateway
How many Credits does an invocation consume?	Real-time conversion at the request moment according to the Credits unit price in the model metadata
Which provider does the service belong to?	A clear binding relationship is established between services and providers, and a provider can be associated with multiple services.
How is consumer usage constrained?	Quota rules support both Token and Credits units, allowing administrators to choose flexibly.

The Credits metering and quota capabilities are now live. We welcome you to try them out and leave your usage governance needs and feedback in the comments section.

Community

AI Gateway Introduces Credits Metering & Quota Capability: Making Consumer Usage Measurable and Constrainable

I. Why Measuring LLM Invocations is Difficult

II. The Three Capabilities Released This Time

III. Model Metadata: Creating a "Specification File" for Each Model

3.1 What is it

3.2 A Concrete Example

IV. Provider Resources: Evolving from "Labels" to "Attribution Dimensions"

4.1 Why was it not enough before

4.2 New Capability: Independent Provider Resources

V. Quota Unit Expansion: From "Restricting Tokens" to "Restricting Credits"

5.1 Understand Its Value in One Sentence

5.2 UI Changes in Creating Quota Rules

VI. Stringing the Three Capabilities Together: What Happens in a Real Invocation

VII. Capability Overview

Read previous post:

Read next post:

Alibaba Cloud Native Community

You may also like

Comments

Alibaba Cloud Native Community

Related Products

Token Plan

Alibaba Cloud Model Studio

API Gateway

AgentBay