All Products
Search

This Product

    DataWorks:Dynamic scaling conditions for resource tuning

    Document Center

    DataWorks:Dynamic scaling conditions for resource tuning

    Last Updated:Sep 18, 2024

    DataWorks Data Integration supports the dynamic scaling of resources used by real-time synchronization tasks. This topic describes the rules for configuring scaling resources for a real-time synchronization task.

    Rules

    When the system dynamically scales out or in resources for a task, you must make sure that the number of compute units (CUs) occupied by a single worker does not change after the scaling operation.

    If the number of workers that are run by a task changes, the number of CUs for the task after scaling must comply with the following formula:

    Note

    The minimum sales unit of CUs for data synchronization is 0.5 CUs. 

    Number of CUs for a task after scaling = ceil( (Initial number of CUs configured for the task ± Integral multiple of the number of CUs occupied by a single worker)/0.5) × 0.5

    You can use the initial number of CUs configured for a task to calculate the number of workers and the number of CUs occupied by a single worker based on the following formulas:

    • Number of workers = floor(Initial number of CUs configured for the task/Empirical constant)

    • Number of CUs occupied by a single worker = (Initial number of CUs configured for a task - Number of CUs occupied by control processes)/Number of workers

    Note

    Description of configuration items in the preceding formulas:

    • The empirical constant is fixed as 3. The value may be adjusted in the future based on actual situations.

    • The number of CUs occupied by control processes is determined based on the following instructions. The values may be adjusted in the future based on actual situations.

      • If the initial number of CUs configured for a task is less than or equal to 2, the number of CUs occupied by control processes is fixed as 0.25.

      • If the initial number of CUs configured for a task is greater than 2 but less than or equal to 10, the number of CUs occupied by control processes is fixed as 0.5.

      • If the initial number of CUs configured for a task is greater than 10, the number of CUs occupied by control processes is fixed as 0.75.

    If the CU settings for scaling do not meet the preceding conditions, your task may be restarted during resource scaling. Whether your task is restarted depends on the setting of the number of CUs after scaling and the scheduling sequence of the task.