All Products
Search
Document Center

Container Compute Service:Persist workflows

Last Updated:Dec 27, 2024

Workflow-relevant resources, such as workflow states, are periodically deleted. To analyze and trace the history of workflows, you can configure a persistence policy to persist workloads to a database. This way, you can view workflow logs even after the workflows or the pods of the workflows are deleted. This topic uses ApsaraDB RDS for MySQL as an example to describe how to configure a persitence policy to persistent workflows to a database.

Step 1: Create an ApsaraDB RDS for MySQL instance and create a database

You need to create an ApsaraDB RDS for MySQL instance, create a database on the instance, and configure a database account. Make sure that the ApsaraDB RDS for MySQL instance and the cluster reside in the same VPC and configure an IP whitelist to allow access from and to the CIDR block of the VPC. For more information, see Step 1: Create an ApsaraDB RDS for MySQL instance and configure a database.

After you complete the configuration, record the username and password of the database account.

Note

For more information about the billing of ApsaraDB RDS for MySQL, see Billable items.

Step 2: Add persistence settings

You can add persistence settings to the Argo Workflows ConfigMap to persist the workflow to the database. By default, you need to manually restart the Argo Workflow controller and Argo Server for the persistence settings to take effect.

  1. Create a Secret named argo-mysql-config in the argo namespace, which is used to store the username and password of the database account.

    apiVersion: v1
    stringData:
      username: database-username # Replace with the actual username. 
      password: database-password # Replace with the actual password. 
    kind: Secret
    metadata:
      name: argo-mysql-config
      namespace: argo
    type: Opaque
  2. Modify workflow-controller-configmap in the argo namespace. Add the persistence settings to the data field.

    data:  
      persistence: |
        connectionPool:
          maxIdleConns: 100
          maxOpenConns: 0
          connMaxLifetime: 0s     # 0 means connections don't have a max lifetime.
        archiveTTL: 30d
        archive: true
        mysql:
          host: rm-xxx.mysql.cn-beijing.rds.aliyuncs.com
          port: 3306
          database: argo-workflow
          tableName: argo_workflows
          userNameSecret:
            name: argo-mysql-config
            key: username
          passwordSecret:
            name: argo-mysql-config
            key: password

    Parameter

    Description

    host

    The endpoint of the ApsaraDB RDS for MySQL instance.

    database

    The name of the database.

    archive

    Set the value to true to enable workflow persistence.

    archiveTTL

    The retention period, which does not have an upper limit. In this example, the parameter is set to 30d, which means that workflows are persisted for 30 days in the database.

  3. Restart the Argo Workflows controller and Argo Server for the persistence settings to take effect.