This topic describes the billing of resources in Data Lake Formation (DLF), including resource usage for data ingestion, storage of metadata objects, and metadata requests.
Billable items and billing methods
At present, all features of DLF are free of charge. You are not charged even if the number of metadata objects exceeds one million or the number of metadata requests exceeds one million.
Storage of metadata objects
Metadata objects are databases, tables, partitions, and indexes that are stored in the metadata of a data lake.
The first one million metadata objects are stored for free each month. If the number of metadata objects exceeds one million, the part that exceeds one million metadata objects costs you USD 1 per month for every 100,000 metadata objects. If the number of metadata objects in the excess part is not a multiple of 100,000, the number is rounded up to the nearest multiple of 100,000 for billing.
Metadata objects are counted by month and billed on the first day of each month. Payments are settled on the first day of each month.
If metadata objects are less than the minimum amount to be charged, you will not be charged.
Example:
On January 1, a data lake contains 100 databases, 10,000 tables, 500,000 partitions, and 5,000 indexes.
To sum up, a total of 515,100 metadata objects are stored in the data lake on January 1.
On January 31, 600,000 partitions are added, whereas the number of databases, tables, and indexes remains unchanged. In this case, a total of 1.1151 million metadata objects are stored in the data lake on January 31.
The bill that you receive on February 1 shows that you are charged USD 2 in total. To be specific, you are charged for free for the first one million metadata objects, but USD 2 for the part that exceeds one million metadata objects. The number of excess metadata objects is 115,100.
On February 1, USD 2 is deducted from your account as the storage fee that is generated in January for the metadata objects in the data lake.
Metadata requests
Metadata requests are requests to create, read, modify, and delete databases, tables, partitions, indexes, and functions in the metadata of a data lake.
You can execute DDL statements in the compute engines that are associated with DLF to initiate metadata requests. The compute engines can be E-MapReduce (EMR) Hive, EMR Spark SQL, EMR Presto, and MaxCompute. You can use the DLF console or call API operations to manage metadata objects.
The first one million metadata requests are free of charge each month. If the number of metadata requests exceeds one million, the part that exceeds one million metadata requests costs you USD 1 per month per million. If the number of metadata requests in the excess part is less than or equal to one million, one million is used for billing.
Metadata requests are counted by month and billed on the first day of each month. Payments are settled on the first day of each month.
If metadata requests are less than the minimum amount to be charged, you will not be charged.
Example:
In January, the compute engines such as EMR, MaxCompute, and Hologres are used to create and access the metadata in DLF. A total of 1.2 million requests are initiated to access the metadata. In this case, you are charged USD 1 in total in January. To be specific, you are charged for free for the first one million metadata requests, but USD 1 for the part that exceeds one million metadata requests. The number of excess metadata requests is 200,000.
On February 1, USD 1 is deducted from your account as the fee that is generated in January for requests to manage the metadata in the data lake.
Resource usage for data ingestion
Data ingestion is a process of extracting data from data sources to a data lake in DLF and storing the data in DLF. (This feature has stopped updating.)
You are not charged for data ingestion tasks. However, if other services are used for data ingestion, you may be charged for the services. For example, a Data Transmission Service (DTS) instance is required if you want to synchronize data from an ApsaraDB RDS database to a data lake. In this case, you are charged additional DTS fees.
Impacts of overdue payments
The tasks that are running and the data in DLF are not affected within 72 hours from the time when you have an overdue payment. However, during the 72-hour period, you cannot perform the following operations:
Create data sources.
Create data ingestion tasks.
Create databases and tables.
Start data ingestion tasks.
If you fail to settle your overdue payment after 72 hours, DLF further limits your access to resources:
DLF stops your tasks that are running.
You cannot call metadata operations.
If you fail to settle your overdue payment after 168 hours, DLF releases or deletes your resources:
DLF deletes the data sources, data ingestion tasks, and metadata objects such as databases, tables, and partitions that are within your Alibaba Cloud account.
You cannot use the DLF console.
Note: The data cannot be recovered after it is released. We recommend that you back up the important data in advance.