Manage .done partitions - Graph Compute - Alibaba Cloud Documentation Center

Create a .done partition

For more information, see Partition and Partition and column operations.

Trigger automatic data backflow for a .done partition

Graph Compute allows you to specify .done partitions to identify partitions on which data backflow can be performed.
- A .done partition is an empty partition and is used only for identification.
- Example
  - If you want to perform data backflow on a partition named 20160528, you must create the corresponding 20160528.done partition.
  - When both the 20160528 and 20160528.done partitions exist, data is stored in the 20160528 partition.
The partition name can contain only digits, and we recommend that you name a partition with a date, such as 20171111.
- Partitions whose names contain letters are not supported for data backflow.
- The partition name must be less than 16 bytes in length.
- The correctness of data backflow cannot be ensured for partitions whose names do not meet the requirements.

Warning

If the interval at which .done partitions are created is too short, automatic data backflow is not triggered.
Data backflow can be triggered only once for a .done partition.
Data backflow cannot be triggered if the partition that corresponds to a .done partition is empty.

For tables on which data backflow is performed on an hourly basis, you can create .done partitions only for level-1 partitions, such as the 2017101011.done or 2017101012.done partition.
If multi-level partitioning is used for tables on which data backflow is performed on an hourly basis, you must pay attention to the order of partition fields in partition names. The ds field must be placed before the hh field.
Important
(1) Make sure that the .done suffix is at the end of the partition string, such as ds=20110101/hh=11.done. Take note that hh=12.done/ds=20170909 is invalid.
(2) The system does not parse the data in partitions or the definition of fields such as the ds, dt, and hh fields. The system only detects .done partitions by name in reverse alphabetical order.
Advanced usage: For example, when you trigger data backflow for a MaxCompute partition and you want to specify the start timestamp at which incremental data is retrieved, you can create a .done partition named in the following format: 20150105.base_timestamp#1505147960.done. The timestamp indicates the time when the system retrieves incremental data after index switching for full data is complete.
- base_timestamp#1505147960
- indicates the start timestamp at which incremental data is retrieved. Unit: seconds.
- ds=20190606/hh=14.base_timestamp#1559802331.done is an example of a .done partition created for a partition that is generated on an hourly basis.

⚠️ If no start timestamp is specified when data backflow is triggered for a .done partition for full data update and incremental data update, the timestamp generated when the backflow is triggered is used by default. The default timestamp is also the timestamp generated when the .done partition is detected.