To facilitate the management of DataWorks resources and improve user experience, DataWorks releases serverless resource groups. A serverless resource group can implement the core features of an exclusive resource group for scheduling, an exclusive resource group for Data Integration, and an exclusive resource group for DataService Studio at the same time. You can perform operations such as data synchronization, task scheduling and running, and API calling and management by using only one serverless resource group.
Introduction to serverless resource groups
Old-version resource groups consist of exclusive resource groups and shared resource groups. Compared with old-version resource groups, serverless resource groups provide more abundant capabilities, support a more unified sales mode, and reduce resource waste by improving the utilization of resource fragments. For more information about comparison between serverless resource groups and old-version resource groups, see Comparison between serverless resource groups and old-version resource groups.
Precautions
DataWorks supports various types of nodes. Some of the nodes are issued to the related compute engines for running, and others are run on DataWorks resource groups. Data computing fees generated by running nodes on compute engines are charged by the Alibaba Cloud services to which the compute engines belong. Data computing fees generated by running nodes on DataWorks resource groups are charged by DataWorks.
If you use a pay-as-you-go serverless resource group to run your tasks, the tasks may compete for resources during peak hours, and the timeliness of resource usage cannot be ensured.
The fees of a serverless resource group do not cover task scheduling fees regardless of whether you use a pay-as-you-go or subscription serverless resource group. You are separately charged for task scheduling based on the number of instances that are successfully run. For information about billing details of task scheduling, see the Appendix: Billing of task scheduling section in this topic.
You cannot change the billing method of a serverless resource group between subscription and pay-as-you-go. For example, if you choose the subscription billing method for a serverless resource group, you are charged based on this billing method when you use the resource group. You cannot change the billing method of the resource group to pay-as-you-go.
Billing scenarios
The resource fees of DataWorks consist of data computing fees and task scheduling fees.
Data computing fees: If you run data synchronization tasks, DataService Studio tasks, data computing tasks, and data quality monitoring rules in DataWorks, data computing fees are generated. The data synchronization tasks include batch synchronization tasks. DataService Studio tasks include tasks that are used to call DataService Studio APIs. The data computing tasks include ODPS SQL, PyODPS, and E-MapReduce (EMR) Hive tasks.
Task scheduling fees: If you deploy tasks to the production environment for periodic scheduling, scheduling fees are generated.
For data computing fees, you are charged based on compute units (CUs). For task scheduling fees, you are charged based on the number of instances that are successfully run, excluding dry-run instances. The fees of a serverless resource group include only data computing fees regardless of whether you purchase a pay-as-you-go or subscription serverless resource group. You are charged task scheduling fees separately. For information about billing details of task scheduling, see the Appendix: Billing of task scheduling section in this topic.
Performance metrics and purchase suggestions
Serverless resource groups whose billing method is subscription or pay-as-you-go are charged based on the number of CUs. 1 CU = 1 vCPU core + 4 GiB of memory.
You can purchase a serverless resource group and use the resource group to run data synchronization tasks, data development tasks, and DataService Studio tasks. For information about how to purchase a serverless resource group, associate the resource group with a workspace, and configure network connectivity for the resource group, see Create and use a serverless resource group.
The following tabs describe the recommended specifications and required minimum specifications of a serverless resource group when you use the resource group for different purposes.
Data synchronization
Batch synchronization tasks
Parallelism configured for a batch synchronization task | Recommended specifications | Required minimum specifications |
<4 | 0.5 CUs | 0.5 CUs |
>=4 |
|
Real-time synchronization tasks
Synchronization task type | Recommended specifications | Required minimum specifications | |
Real-time synchronization from MySQL | One source database | 2.5 CUs | Minimum specifications that are required to run such a real-time synchronization task: 1 CU |
Two to five source databases | 4 CUs | ||
Six or more source databases | 7 CUs | ||
Real-time synchronization from PolarDB-X 1.0 | 7 CUs | ||
Real-time synchronization from Kafka | 2.5 CUs | ||
Real-time synchronization of data in a single table of another source type | 2.5 CUs | ||
Real-time synchronization of all data in a database | - | Minimum specifications that are required to run such a synchronization task: 2 CUs | |
Task scheduling
The maximum number of parallel instances supported by a serverless resource group is 200.
If your scheduling tasks include data computing tasks such as PyODPS2 and EMR Hive tasks, the data computing tasks use a serverless resource group for computing. You must plan the specifications of the serverless resource group that you require based on your business requirements.
For information about the default number of CUs that are allowed for data computing tasks, see Appendix: Data computing tasks.
If your scheduling tasks do not include data computing tasks, the maximum number of parallel instances supported by a serverless resource group is 200, which is greater than the maximum number of parallel instances supported by an old-version resource group with the highest specifications. In this case, the default specifications of a serverless resource group can meet your business requirements and you do not need to adjust the specifications.
DataService Studio
Maximum QPS | Required minimum specifications |
500 | 4 CUs |
1000 | 8 CUs |
2000 | 16 CUs |
500 | 4 CUs |
1000 | 8 CUs |
2000 | 16 CUs |
Billing details
Billing methods
Serverless resource groups support the subscription and pay-as-you-go billing methods.
Subscription serverless resource group
You must determine the number of CUs that you require and the subscription duration in advance, and pay the fee before you can use such a resource group. After you purchase a subscription serverless resource group, you are not charged additional fees for using the resource group to synchronize data, perform data computing, and call and debug DataService Studio APIs in DataWorks.
Pay-as-you-go serverless resource group
You can use the related features and then pay the fee based on the total number of CUs that are used. If you use a pay-as-you-go serverless resource group to run tasks, such as batch synchronization tasks, DataService Studio tasks, and data development tasks, data computing fees are generated.
The following table compares the features of serverless resource groups that are charged based on different billing methods.
Category | Item | Pay-as-you-go serverless resource group | Subscription serverless resource group |
Quota | Total number of CUs that can be used in a resource group | The number of CUs that are actually used. | The number of CUs that you specify when you purchase a resource group. |
Use scenarios | Data computing, data synchronization, and DataService Studio | Supported. | Supported. |
Operations | Scale-out, scale-in, and renewal | N/A. | Supported. |
Quota management | This feature is used to control the maximum number of CUs that can be used in different scenarios. You can use this feature in data computing, data synchronization, and DataService Studio. | ||
Maximum number of parallel tasks allowed in data scheduling | A resource group supports a maximum of 200 parallel tasks. | ||
Network settings | Number of virtual private clouds (VPCs) that can be associated |
| The number of VPCs with which you can associate a resource group of this type varies based on the number of CUs that you purchase.
|
Billing rules
The fees of a serverless resource group do not cover task scheduling fees regardless of whether you use a pay-as-you-go or subscription serverless resource group. You are separately charged for task scheduling based on the number of instances that are successfully run. For information about billing details of task scheduling, see the Appendix: Billing of task scheduling section in this topic.
Subscription serverless resource group
Subscription serverless resource groups are charged based on the number of CUs that are purchased. You can calculate the fee of a resource group of this type based on the following formula: Fee to be paid = Unit price per month × Number of months × Number of CUs purchased per month.
For a subscription serverless resource group, you must purchase a minimum of 2 CUs per month. No upper limit is imposed. However, the maximum number of CUs that you can purchase may be affected by the inventory. If the inventory is insufficient, follow the instructions on the buy page.
If the resource group that you purchase does not meet your business requirements, you can scale out the resource group. For more information, see Scale out or in a resource group.
The minimum specifications that are required when you use a serverless resource group to run tasks vary based on the task type. For more information, see the Performance metrics and purchase suggestions section in this topic.
Pay-as-you-go serverless resource group
You are charged based on the following formula: Fee to be paid = Number of CU-hours × Unit price per CU-hour. Bills for the fee are generated on an hourly basis.
Calculation method of CU-hours: If two CUs are configured for a data computing task and the running of the task is complete 0.5 hours after it is started, the task consumes one CU-hour regardless of whether the running result is successful, failed, or manually terminated. The result is calculated based on the following formula: 2 CUs × 0.5 hours.
Unit price
Subscription serverless resource group
Region | Unit price (USD per month per CU) |
China (Zhangjiakou) | 27.91093 |
China (Chengdu) | 30.45686 |
China (Shanghai), China (Hangzhou), China (Beijing), and China (Shenzhen) | 37.1517 |
UK (London) | 51.01286 |
US (Virginia) | 53.92014 |
Malaysia (Kuala Lumpur) | 63.36534 |
China (Hong Kong), Singapore, Germany (Frankfurt), and Indonesia (Jakarta) | 67.61327 |
China East 2 Finance, China South 1 Finance, and China North 2 Finance | 70.58824 |
US (Silicon Valley) | 72.74794 |
Japan (Tokyo) | 77.45584 |
UAE (Dubai) | 81.09255 |
Pay-as-you-go serverless resource group
Region | Unit price (USD per CU-hour) | Example |
China (Zhangjiakou) | 0.058148 | For example, two CUs are configured for a data synchronization task in the China (Shanghai) region, the running duration of the task is 0.5 hours, and the unit price of each CU-hour in the China (Shanghai) region is USD 0.077399. The number of CU-hours consumed by the data synchronization task and the related fee are calculated based on the following formulas:
|
China (Shanghai), China (Hangzhou), China (Beijing), and China (Shenzhen) | 0.077399 | |
China (Chengdu) | 0.063452 | |
UK (London) | 0.106277 | |
US (Virginia) | 0.112334 | |
Malaysia (Kuala Lumpur) | 0.132011 | |
Germany (Frankfurt), Indonesia (Jakarta), China (Hong Kong), and Singapore | 0.140861 | |
China South 1 Finance, China North 2 Finance, and China East 2 Finance | 0.147059 | |
US (Silicon Valley) | 0.151558 | |
Japan (Tokyo) | 0.161366 | |
UAE (Dubai) | 0.168943 |
Appendix: Billing of task scheduling
Billing scenario
If you deploy data synchronization tasks, data development tasks, and DataService Studio tasks to the production environment for periodic scheduling, scheduling fees are generated. For information about the tasks that can be scheduled in DataWorks, see DataWorks nodes.
Billing unit and rules
You are charged based on the number of instances that are successfully run. DataWorks provides 12 billing tiers for calculating data scheduling fees based on the number of instances that are successfully run every day and charges you the fee by day based on the number of instances that are successfully run every day and the related billing tier.
You are not charged for dry-run instances. The system does not actually run a dry-run instance but directly returns a success result. The number of dry-run instances is not included in the total number of instances that are successfully run.
Example of statistics on the number of instances per day: If a task is scheduled by hour and is scheduled once per hour during the period of 00:00 to 23:59 every day, a total of 24 instances are generated every day.
Unit price for billing
What to do next
Purchase a serverless resource group
The first time you use DataWorks and resource groups, we recommend that you perform an evaluation by referring to Performance metrics and purchase suggestions and then purchase a serverless resource group based on your business requirements. For information about how to purchase a serverless resource group, see Create and use a serverless resource group.
If you already have an old-version resource group and want to migrate tasks that are run on the old-version resource group to a serverless resource group, you can refer to Change from an old-version resource group to a serverless resource group to evaluate the specifications of the serverless resource group that you need to purchase and perform the migration operation.
Scale out or in a serverless resource group
If you purchase a subscription serverless resource group and the number of CUs in the resource group does not meet your subsequent business requirements, you can scale out or in the resource group based on your business requirements. For more information about the scale-out or scale-in operation, see Scale out or in a resource group.
Pay-as-you-go serverless resource groups do not involve the scale-out or scale-in operation.
Change quotas for a serverless resource group
After you purchase a serverless resource group, you can configure the minimum CU quotas for data computing, data synchronization, and DataService Studio to ensure that your tasks can run as expected. For more information, see Manage quotas.
Renew a subscription serverless resource group
If your subscription serverless resource group is about to expire, you can renew the resource group. If you do not renew the resource group, the resource group stops providing services or is released after it expires. For more information about the renewal operation, see Expiration and renewal.