This topic describes the compatibility between a job and the state data used to start the job after you modify a regular join, interval join, temporal join, or lookup join query for the job.
Regular join
The state data of operators related to a regular join cannot be migrated. If you modify an attribute of a regular join for a job, such as the join type, join condition, or the input schema, the job becomes incompatible with the state data. However, if you modify the equality or non-equality conditions and the changes can be pushed down to an input stream of the join query to generate a Calc node that has conditions, you can specify the table.optimizer.state-compatibility.ignore-filter: true configuration in the Parameters section to ensure that the job remains fully compatible with the state data.
Interval join
The state data of operators related to an interval join cannot be migrated. If you modify an attribute of an interval join for a job, such as the join type, join condition, interval, or the input schema, the job becomes incompatible with the state data. If you perform the following modifications, the job remains fully compatible with the state data:
Modify the time unit without affecting the final value of the interval.
Modify the equality or non-equality conditions and the changes can be pushed down to an input stream of the join query to generate a Calc node that has conditions and specify the
table.optimizer.state-compatibility.ignore-filter: trueconfiguration in the Parameters section.
Temporal join
The state data of operators related to a temporal join cannot be migrated. If you modify an attribute of a temporal join for a job, such as the join type, join condition, or the input schema, the job becomes incompatible with the state data.
Lookup join
By default, a lookup join is performed by a non-stateful operator. Modifying a lookup join does not affect the state compatibility of the corresponding operator. However, if the output of a lookup join is changed, the state compatibility of downstream operators may be affected. In addition, if the following conditions are met, a lookup join may be performed by a stateful operator and compatibility issues may occur. In this case, the system may fail to detect the issues and incorrectly displays that the modifications are fully compatible. As a result, an error occurs when you start the job because the state data cannot be recovered.
You specify the
table.optimizer.non-deterministic-update.strategy: TRY_RESOLVEconfiguration for a job and the system detects non-deterministic updates from a lookup join.You configure asynchronous lookup for a lookup join.