Time-specific scaling of virtual warehouses (beta) - Hologres

Hologres virtual warehouse instances support the time-specific scaling feature. After you configure a scaling plan for a virtual warehouse, the system automatically scales out and scales in computing resources as scheduled based on traffic in peak hours and off-peak hours of your business. This helps improve the instance stability, increase resource utilization, and reduce costs. This topic describes how to use the time-specific scaling feature.

Scenarios

For example, your virtual warehouse instance includes three virtual warehouses: init warehouse, warehouse 1, and warehouse 2. The data mid-end team uses the init warehouse to perform large-scale extract, transform, and load (ETL) operations, import a large amount of data offline, import data within minutes, or import data in real time. Business team A uses warehouse 1 to query data. Business team B uses warehouse 2 to query data. In this example, you can use the time-specific scaling feature based on the following rules:

Data import within minutes and real-time data import: Use the init warehouse. If the amount of data to be imported throughout the day is stable, the time-specific scaling feature is not required. If the amount of data that is imported during a fixed period of time (less than 16 hours) every day is significantly·large, we recommend that you configure a time-specific scaling plan to provide elastic computing resources during peak hours.
Data queries: Different business teams use different follower virtual warehouses to perform queries. This ensures load isolation. If the numbers of query requests from the business teams are stable, time-specific scaling is not required. If the numbers of query requests per day periodically change, we recommend that you configure a time-specific scaling plan to provide elastic computing resources during peak hours.
Large-scale ETL operations and offline import of a large amount of data: We recommend that you use serverless computing resources. You can use serverless computing resources without the need to reserve computing resources. This helps improve the instance stability and lower the risk of out of memory (OOM) errors. You are charged only for the amount of serverless computing resources actually consumed. For more information about the Serverless Computing feature, see Overview of Serverless Computing. For more information about how to use the Serverless Computing feature, see User guide on Serverless Computing.

Limits

The time-specific scaling feature is supported only by virtual warehouse instances, but not general-purpose instances or read-only secondary instances.
Hologres V2.2.21 and later support the time-specific scaling feature.
The time-specific scaling feature is not available in the following regions: China East 2 Finance, China North 2 Ali Gov 1, China South 1 Finance, Japan (Tokyo), Malaysia (Kuala Lumpur), and Indonesia (Jakarta).

Note

The time-specific scaling feature is in public preview. If you want to use this feature, use your Alibaba Cloud account to submit a request for using the feature.

Usage notes

To use the time-specific scaling feature, you must use one of the following account types:
- Alibaba Cloud account or RAM users to which the AliyunHologresWarehouseFullAccess policy is attached. The AliyunHologresWarehouseFullAccess policy contains the read-only permissions on the Hologres console and the permissions to configure the time-specific scaling feature. For more information about permission authorization, see Grant permissions to a RAM user.
- Accounts with superuser permissions on the instance. For more information about permission authorization, see Grant the development permissions on a Hologres instance to a RAM user.
In Hologres V2.2, when you use the time-specific scaling feature to scale in or scale out a virtual warehouse as scheduled, data queries and writes on the virtual warehouse are interrupted for about 15 seconds. In Hologres V3.0 and later, when you use the time-specific scaling feature to scale out a virtual warehouse as scheduled, data queries and writes on the virtual warehouse are not affected. The virtual warehouse is in the processing state. When you use the time-specific scaling feature to scale in a virtual warehouse as scheduled, data queries and writes on the virtual warehouse are interrupted for about 15 seconds.
After you configure a scaling plan for a virtual warehouse, take note of the following items:
- You cannot suspend, delete, or scale in the virtual warehouse.
- You can scale out the virtual warehouse only in the Hologres console, not by manually executing the hg_alter_warehouse statement.
- You can create a virtual warehouse only in the Hologres console, not by manually executing the hg_create_warehouse statement.

Terms

Computing resources of an instance
- Reserved computing resources of an instance: The subscription and pay-as-you-go billing methods are supported. Reserved computing resources of an instance includes allocated resources and unallocated resources.
  - Allocated resources: the reserved resources that are allocated to virtual warehouses in the instance.
  - Unallocated resources: the reserved resources that are not allocated to virtual warehouses in the instance.
- Elastic computing resources of an instance: computing resources that are elastically generated for all virtual warehouses in the instance.
- Total computing resources of an instance: the total amount of computing resources that are used by an instance, which is the sum of the amount of reserved computing resources and the amount of elastic computing resources for the instance.
Computing resources of a virtual warehouse
- Reserved computing resources of a virtual warehouse: reserved computing resources that are allocated to a virtual warehouse in an instance.
- Elastic computing resources of a virtual warehouse: computing resources that are elastically generated for a virtual warehouse in an instance.
- Total computing resources of a virtual warehouse: the total amount of computing resources that are used by a virtual warehouse, which is the sum of the amount of reserved computing resources and the amount of elastic computing resources for the virtual warehouse.

The following figure shows the resources of a virtual warehouse instance, and the following table describes the resource details.

Item	Resource details
Instance	96 compute units (CUs) of reserved resources, of which 64 CUs are allocated and 32 CUs are not allocated
	32 CUs of elastic resources
	Total computing resources: 96 + 32 = 128 CUs
Virtual warehouse init_warehouse	32 CUs of reserved resources and 16 CUs of elastic resources, which are 48 CUs of computing resources in total
Virtual warehouse warehouse_2	32 CUs of reserved resources and 16 CUs of elastic resources, which are 48 CUs of computing resources in total

Pricing

Reserved computing resources of an instance: You are charged for exclusive computing resources of a virtual warehouse instance based on the subscription or pay-as-you-go billing method.
Elastic computing resources of an instance: The fees of elastic computing resources that are provided by the time-specific scaling feature are calculated by using the following formula: Fees = Amount of elastic computing resources that are consumed (unit: CU-Hour) × Unit price. For more information about the unit price, see Billing overview. The fees are calculated on an hourly basis. The system pushes a bill to you and automatically deducts the fees from your account balance.
Note
Elastic computing resources and unallocated computing resources of an instance are independent of each other. The time-specific scaling feature provides elastic computing resources for an instance as scheduled no matter whether the instance has unallocated computing resources.

Procedures

Manage virtual warehouse resources

Go to the Virtual Warehouse Management page.
1. Log on to the Hologres console. In the top navigation bar, select a region from the drop-down list.
2. In the left-side navigation pane, click Instances. On the Instances page, click the ID of the desired instance.
3. In the left-side navigation tree of the Instance Details page, click Virtual Warehouse Management.
View the resource information of the virtual warehouse instance.
On the Virtual Warehouse Resource Management tab, you can view the resource information of the virtual warehouse instance, including the amount of allocated reserved resources, the amount of unallocated reserved resources, and the amount of elastic resources.
Manage virtual warehouse resources.
- Click Create Virtual Warehouse. In the Create Virtual Warehouse dialog box, configure the Virtual Warehouse Name and Virtual Warehouse Resources parameters to add a virtual warehouse.
- Click Modify Configuration, Restart, Rebalance, or Stop for virtual warehouses whose status is Running. For more information about shard rebalancing, see Rebalance.
  Note
  You cannot suspend the default virtual warehouse init_warehouse of an instance.

Configure scaling plans for virtual warehouses

View time series charts of scaling plans

On the Virtual Warehouse Management page, click the Virtual Warehouse Scaling Plan tab.
Select an instance or a virtual warehouse from the Instance/Virtual Warehouse drop-down list to view time series charts. A time series chart indicates the scale-in and scale-out operations that are scheduled to be performed on an instance or virtual warehouse. No metrics that indicate historical resource usage are shown in the chart.

After you save the modification, the total amount of computing resources for Virtual Warehouse A is immediately scaled in from 96 CUs to 64 CUs.

Configure a scaling plan for a virtual warehouse

On the Virtual Warehouse Scaling Plan tab, click on the left of the desired virtual warehouse, and click Add Time Period. Configure the start time and end time of the effective time period, configure the amount of elastic computing resources, and then click Save in the Actions column.
You can also click Edit or Delete in the Actions column for an existing scaling plan.

Important

The amount of elastic computing resources cannot exceed the amount of reserved computing resources.
You can configure up to five scaling time periods for a virtual warehouse.
When you delete a scaling plan from a virtual warehouse, the virtual warehouse enters the processing state even if the resources of the virtual warehouse remain unchanged. In this process, data reads and writes on the virtual warehouse are not affected.

After you save a scaling plan for a virtual warehouse, the resources of the virtual warehouse are immediately scaled out or in if the current time falls in a related scaling time period. Examples:

Example 1:
Virtual Warehouse A has 64 CUs of reserved computing resources. You create and save a scaling plan for Virtual Warehouse A at 16:00.
In the scaling plan, 32 CUs of elastic computing resources are allocated, and the effective time period is from 15:00 to 18:00. In this case, the total amount of computing resources for Virtual Warehouse A is immediately scaled out to 96 CUs after you save the scaling plan.
Example 2:
Virtual Warehouse A has 64 CUs of reserved computing resources. You modify and save an existing scaling plan for Virtual Warehouse A at 16:00.
- Before the modification: 32 CUs of elastic computing resources are allocated, and the effective time period is from 15:00 to 18:00.
- After the modification: 32 CUs of elastic computing resources are allocated, and the effective time period is from 17:00 to 18:00.
After you save the modification, the total amount of computing resources for Virtual Warehouse A is immediately scaled in from 96 CUs to 64 CUs.

Monitor performance of the time-specific scaling feature

This section describes how to monitor the performance of the time-specific scaling feature.

Execution logs of scaling plans

On the Virtual Warehouse Management page, click the Scaling Plan Execution Log tab.
Select a time range. The execution information of scaling plans in the specified time range is displayed, including the execution time, virtual warehouse name, execution status, event type, amount of reserved computing resources, and amount of elastic computing resources.

Metrics

You can view the Warehouse_timed_elastic_cores(Count) metric in the Hologres console. This metric indicates the number of cores of elastic computing resources that are provided by an instance based on the time-specific scaling feature. You can also configure alert rules for this metric. For more information, see Hologres metrics.

ActionTrail

Operations such as modifications that are performed on scaling plans in the Hologres console and scale-in and scale-out operations that are performed based on scaling plans are logged in the ActionTrail console. For more information, see Query event logs.