Alibaba Cloud Model Studio:Access model or application APIs over a private network using an endpoint

How it works

After you create an interface endpoint in a VPC, PrivateLink establishes a private, unidirectional connection between your VPC and Alibaba Cloud Model Studio. This connection only allows resources in your VPC to access Model Studio. Model Studio cannot access resources in your VPC.

When computing resources in the VPC access the endpoint, traffic is forwarded to the Model Studio service through PrivateLink and does not pass through the internet.

The endpoint must be in the same region as the Model Studio service. To use a VPC in another region, see Cross-region private access.

Model Studio service regions:

• Public cloud: Singapore and China (Beijing).

  Currently not supported in the US (Virginia) region.

Access APIs through PrivateLink endpoint

# Replace the original domain name with the domain name that you obtained in the previous step.
curl -X POST http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen-flash",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user", 
            "content": "Who are you?"
        }
    ]
}'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace the original domain name with the domain name that you obtained in the previous step.
    base_url="http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
    model="qwen-flash",
    messages=[
        {'role': 'system', 'content': 'You are a helpful assistant.'},
        {'role': 'user', 'content': 'Who are you?'}],
)
print(completion.model_dump_json())

import os
from http import HTTPStatus
# We recommend that you use DashScope SDK V1.14.0 or later.
import dashscope
from dashscope import Generation

# Replace the original domain name with the domain name that you obtained in the previous step.
dashscope.base_http_api_url = "http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/api/v1"
dashscope.api_key = os.getenv("DASHSCOPE_API_KEY")
messages = [{
    'role': 'user', 'content': 'Who are you?'
}]
response = Generation.call(
    model="qwen-flash",
    messages=messages,
    result_format='message'
)
if response.status_code == HTTPStatus.OK:
    print(response)
else:
    print('Request id: %s, Status code: %s, error code: %s, error message: %s' % (
        response.request_id, response.status_code,
        response.code, response.message
    ))

// We recommend that you use DashScope SDK V2.12.0 or later.
import java.util.Arrays;

import com.alibaba.dashscope.aigc.generation.Generation;
import com.alibaba.dashscope.aigc.generation.GenerationParam;
import com.alibaba.dashscope.aigc.generation.GenerationResult;
import com.alibaba.dashscope.common.Message;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.InputRequiredException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.utils.JsonUtils;

public class Main {
    public static GenerationResult callWithMessage() throws ApiException, NoApiKeyException, InputRequiredException {
        // Replace the original domain name with the domain name that you obtained in the previous step.
        Generation gen = new Generation(Protocol.HTTP.getValue(), "http://ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com/api/v1");
        Message systemMsg = Message.builder()
                .role(Role.SYSTEM.getValue())
                .content("You are a helpful assistant.")
                .build();
        Message userMsg = Message.builder()
                .role(Role.USER.getValue())
                .content("Who are you?")
                .build();
        GenerationParam param = GenerationParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-flash")
                .messages(Arrays.asList(systemMsg, userMsg))
                .resultFormat(GenerationParam.ResultFormat.MESSAGE)
                .build();
        return gen.call(param);
    }
    public static void main(String[] args) {
        try {
            GenerationResult result = callWithMessage();
            System.out.println(JsonUtils.toJson(result));
        } catch (ApiException | NoApiKeyException | InputRequiredException e) {
            // Print the error message.
            System.err.println("An error occurred while calling the generation service: " + e.getMessage());
        }
    }
}

Cross-region private access

In the proceeding steps, you have accessed Model Studio from an endpoint in the same region. But if your VPC is in another region, complete the following steps:

1. Follow the previous sections to create a PrivateLink endpoint.

2. Use Cloud Enterprise Network (CEN) to connect VPCs in different regions. Note the following:

   • Select VPCs with non-overlapping CIDR blocks to prevent failures due to CIDR block conflicts.

   • To connect VPCs between Chinese mainland and oversea regions, your account must pass enterprise identity verification.

3. In the security group that is associated with the PrivateLink endpoint, add an inbound rule to allow traffic from the requester VPC on ports 80 and 443.

After the configuration is complete, when you access the default service domain name of the endpoint from the requester VPC, the transit router (TR) routes the traffic to the endpoint in the Model Studio service region.

By default, the default domain name can be directly accessed from interconnected VPCs across regions. However, a custom service domain name is valid only within VPCs of the same region. To use the custom domain name, create an authoritative zone that is the same as the custom domain name, and then resolve it to the default domain name using a CNAME record:

1. Add an authoritative zone that is the same as the custom domain name, such as vpc-ap-southeast-1.dashscope.aliyuncs.com. For the Effective Scope section, select the requester VPC.

2. To add a DNS record, set Record Type to CNAME, Hostname to @, and Record Value to the default domain name. For example: ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com.

   Note: When you configure internal domain name resolution, do not use underscores (_) in the host record or the full domain name because this can cause API calls to fail. A domain name can contain only letters, numbers, and hyphens (-). For example, use test-for-dns.dashscope.aliyuncs.com instead of test_for_dns.dashscope.aliyuncs.com.

After the configuration is complete, you can access Model Studio APIs from the requester VPC using the custom domain name.

Billing

Using PrivateLink, CEN, and Private Zone may incur additional fees:

• Billing of PrivateLink

• Billing of CEN

• Billing of Private Zone

FAQ

1. Why can't my ECS instance access Model Studio APIs using PrivateLink?

   Follow these steps to troubleshoot:

   1. Confirm that the ECS instance and the endpoint are in the same VPC.

      If the ECS instance and the endpoint are in different VPCs, connect the VPCs first.

   2. Check the security group that is associated with the endpoint. Confirm that an inbound rule is added to allow traffic from the CIDR block of the source ECS instance on port 80 (HTTP) or 443 (HTTPS).

   3. Confirm the domain name.

      The default domain name supports only HTTP.

2. Can endpoints be accessed from the internet?

   No. PrivateLink is used only to establish private connections within the Alibaba Cloud internal network. Endpoints do not have internet access, and endpoint ENIs cannot be associated with elastic IP addresses (EIPs).

3. Why do I receive an error when I call a model using my custom domain name with internal domain name resolution?

   This issue usually occurs because the host record (or full domain name) used for internal domain name resolution contains invalid characters, such as an underscore (_). Domain names should only contain letters, numbers, and hyphens (-).

   Configure the DNS record as follows:

   1. Authoritative zone: Add a DNS record for the dashscope.aliyuncs.com authoritative zone.

   2. Host record: Set Record Type to CNAME, and enter your custom domain prefix, such as test-for-dns-right. Note: Host records cannot contain underscores (_).

      Correct example	Incorrect example

   3. Record value: Enter the default service domain name of the Model Studio endpoint. For example: ep-***.dashscope.ap-southeast-1.privatelink.aliyuncs.com.

   After completing the configuration, you can call the model using https://test-for-dns-right.dashscope.aliyuncs.com/api/v1. For OpenAI compatible mode, use https://test-for-dns-right.dashscope.aliyuncs.com/compatible-mode/v1/chat/completions.

   Using a domain name with an underscore, such as https://test_for_dns_wrong.dashscope.aliyuncs.com/api/v1, will cause an error.