All Products
Search
Document Center

:Best practices: Table management with the API

Last Updated:Oct 15, 2025

DataWorks provides a comprehensive set of API operations for metadata management. This guide demonstrates how to query tables, retrieve table details, and manage metadata using the DataWorks API.

Prerequisites

Before you begin, review the following resources:

1. List tables

This practice describes how to use the API to query the list under a MaxCompute project/schema and supports paged queries. The main procedure is as follows:

Backend: Use MetaServiceProxy to query table details

  1. Write a ListTables method in MetaServiceProxy to process frontend parameters and send requests to the ListTables API to obtain the list information of metadata tables.

    • Supported query parameters: Parent entity ID, name (fuzzy matching), comment (fuzzy matching), type, sorting parameters, and pagination parameters.

      Note

      Refer to Metadata entity-related concept description. The Parent entity ID may exist in two formats.

      • Parent entity ID is MaxCompute project ID, format: maxcompute-project:${Alibaba Cloud account ID}::{projectName}

      • Parent entity ID is MaxCompute schema ID, format: maxcompute-schema:${Alibaba Cloud account ID}::{projectName}:{schemaName}

    • Query results include: Total number of data tables, pagination information, and for each data table: ID, Parent entity ID, table name, comment, database name, schema name, type, partition field list, creation time, and modification time.

    • Get the Alibaba Cloud account ID: Whether you are using a RAM user or an Alibaba Cloud account, you can view the Alibaba Cloud account ID in the upper-right corner of the page.

      1. Log on to the DataWorks console using your Alibaba Cloud account or RAM user.

      2. Hover your mouse over the profile picture in the upper-right corner to view the Alibaba Cloud account ID.

        • Alibaba Cloud account: The Account ID is the Alibaba Cloud account ID that you need to obtain.

        • RAM user: You can directly view the Alibaba Cloud Account ID.

    • Sample code:

      /**
       * @author dataworks demo
       */
      @Service
      public class MetaServiceProxy {
           
          @Autowired
          private DataWorksOpenApiClient dataWorksOpenApiClient;
           
          /**
                * DataWorks OpenAPI : ListTables
                *
                * @param listTablesDto
                */
          public ListTablesResponseBodyPagingInfo listTables(ListTablesDto listTablesDto) {
            try {
              Client client = dataWorksOpenApiClient.createClient();
              ListTablesRequest listTablesRequest = new ListTablesRequest();
              // Parent entity ID (see metadata entity concepts)
              listTablesRequest.setParentMetaEntityId(listTablesDto.getParentMetaEntityId());
              // Table name, supports fuzzy matching
              listTablesRequest.setName(listTablesDto.getName());
              // Table comment, supports fuzzy matching
              listTablesRequest.setComment(listTablesDto.getComment());
              // Table type, returns all types by default
              listTablesRequest.setTableTypes(listTablesDto.getTableTypes());
              // Sort method, CreateTime (default) / ModifyTime / Name / TableType
              listTablesRequest.setSortBy(listTablesDto.getSortBy());
              // Sort direction, supports Asc (default) / Desc
              listTablesRequest.setOrder(listTablesDto.getOrder());
              // Page number, default is 1
              listTablesRequest.setPageNumber(listTablesDto.getPageNumber());
              // Page size, default is 10, maximum 100
              listTablesRequest.setPageSize(listTablesDto.getPageSize());
              ListTablesResponse response = client.listTables(listTablesRequest);
              // Get the total number of tables that meet the requirements
              System.out.println(response.getBody().getPagingInfo().getTotalCount());
              for (Table table : response.getBody().getPagingInfo().getTables()) {
                // Table ID
                System.out.println(table.getId());
                // Parent entity ID
                System.out.println(table.getParentMetaEntityId());
                // Table name
                System.out.println(table.getName());
                // Table comment
                System.out.println(table.getComment());
                // Database/MaxCompute project name
                System.out.println(table.getId().split(":")[3]);
                // Schema name
                System.out.println(table.getId().split(":")[4]);
                // Table type
                System.out.println(table.getTableType());
                // Table creation time
                System.out.println(table.getCreateTime());
                // Table modification time
                           System.out.println(table.getModifyTime());
                           // Partition field list of the table
                           System.out.println(table.getPartitionKeys());
                       }
                       return response.getBody().getPagingInfo();
                   } catch (TeaException error) {
                       // For demonstration only. Handle exceptions properly in production code.
                       // Error message
                       System.out.println(error.getMessage());
                       // Diagnostic address
                       System.out.println(error.getData().get("Recommend"));
                       com.aliyun.teautil.Common.assertAsString(error.message);
                   } catch (Exception _error) {
                       TeaException error = new TeaException(_error.getMessage(), _error);
                       // For demonstration only. Handle exceptions properly in production code.
                       // Error message
                       System.out.println(error.getMessage());
                       // Diagnostic address
                       System.out.println(error.getData().get("Recommend"));
                       com.aliyun.teautil.Common.assertAsString(error.message);
                   }
                   return null;
           }
      }
  2. Add a listTables method in MetaRestController as an entry point for frontend access.

    /**
     * Custom metadata platform using the DataWorks API
     *
     * @author dataworks demo
     */
    @RestController
    @RequestMapping("/meta")
    public class MetaRestController {
       
        @Autowired
        private MetaServiceProxy metaService;
       
        /**
            * Query table data by page
            *
            * @param listTablesDto
            * @return {@link ListTablesResponseBodyPagingInfo}
            */
        @CrossOrigin(origins = "http://localhost:8080")
        @GetMapping("/listTables")
        public ListTablesResponseBodyPagingInfo listTables(ListTablesDto listTablesDto) {
          System.out.println("listTablesDto = " + listTablesDto);
          return metaService.listTables(listTablesDto);
        }
    }

Frontend: Display table metadata information

import React from "react";
import cn from "classnames";
import {
  Table,
  Form,
  Field,
  Input,
  Button,
  Pagination,
  Dialog,
  Card,
  Grid,
  Select,
} from "@alifd/next";
import type { TableListInput, TableEntity } from "../services/meta";
import * as services from "../services";
import DetailContent from "./detailContent";
import LineageContent from "./lineageContent";
import classes from "../styles/app.module.css";
import moment from "moment";

export interface Props {}

const { Column } = Table;
const { Item } = Form;
const { Header, Content } = Card;
const { Row, Col } = Grid;
const { Option } = Select;

const App: React.FunctionComponent<Props> = () => {
  const field = Field.useField();
  const [datasource, setDatasource] = React.useState<TableEntity[]>([]);
  const [loading, setLoading] = React.useState<boolean>(false);
  const [total, setTotal] = React.useState<number>(0);
  const [current, setCurrent] = React.useState<number>(1);
  const onSubmit = React.useCallback(
    (pageNumber: number = 1) => {
      field.validate(async (errors, values) => {
        if (errors) {
          return;
        }
        setLoading(true);
        try {
          const response = await services.meta.getTableList({
            pageNumber,
            ...values,
          } as TableListInput);
          setDatasource(response.tables);
          setCurrent(pageNumber);
          setTotal(response.totalCount);
        } catch (e) {
          throw e;
        } finally {
          setLoading(false);
        }
      });
    },
    [field]
  );
  const onViewDetail = React.useCallback((record: TableEntity) => {
    Dialog.show({
      title: "Data Table Details",
      content: <DetailContent item={record} />,
      shouldUpdatePosition: true,
      footerActions: ["ok"],
    });
  }, []);
  const onViewLineage = React.useCallback((record: TableEntity) => {
    Dialog.show({
      title: "Table Lineage Relationship",
      content: <LineageContent item={record} />,
      shouldUpdatePosition: true,
      footerActions: ["ok"],
    });
  }, []);
  React.useEffect(() => {
    field.setValue("dataSourceType", "odps");
  }, []);
  return (
    <div className={cn(classes.appWrapper)}>
      <Card free hasBorder={false}>
        <Header title="Metadata Table Management Scenario Demo (MaxCompute)" />
        <Content style={{ marginTop: 24 }}>
          <Form field={field} colon fullWidth>
            <Row gutter="1">
              <Col>
                <Item
                  label="MaxCompute Project/Schema ID"
                  name="parentMetaEntityId"
                  required
                  help="ID"
                >
                  <Input />
                </Item>
              </Col>
            </Row>

            <Row gutter="4">
              <Col>
                <Item label="Name" name="name">
                  <Input placeholder="Enter table name, fuzzy matching" />
                </Item>
              </Col>
              <Col>
                <Item label="Comment" name="comment">
                  <Input placeholder="Enter table comment, fuzzy matching" />
                </Item>
              </Col>
              <Col>
                <Item label="Type" name="tableTypes">
                  <Select mode="multiple" style={{ width: "100%" }}>
                    <Option value="TABLE">Table</Option>
                    <Option value="VIEW">View</Option>
                    <Option value="MATERIALIZED_VIEW">Materialized View</Option>
                  </Select>
                </Item>
              </Col>
              <Col>
                <Item label="Sort Method" name="sortBy">
                  <Select defaultValue="CreateTime" style={{ width: "100%" }}>
                    <Option value="CreateTime">Creation Time</Option>
                    <Option value="ModifyTime">Modification Time</Option>
                    <Option value="Name">Name</Option>
                    <Option value="TableType">Type</Option>
                  </Select>
                </Item>
              </Col>
              <Col>
                <Item label="Sort Direction" name="order">
                  <Select defaultValue="Asc" style={{ width: "100%" }}>
                    <Option value="Asc">Ascending</Option>
                    <Option value="Desc">Descending</Option>
                  </Select>
                </Item>
              </Col>
            </Row>
            <div
              className={cn(
                classes.searchPanelButtonWrapper,
                classes.buttonGroup
              )}
            >
              <Button type="primary" onClick={() => onSubmit()}>
                Query
              </Button>
            </div>
          </Form>
          <div>
            <Table
              dataSource={datasource}
              loading={loading}
              className={cn(classes.tableWrapper)}
              emptyContent={
                <div className={cn(classes.noDataWrapper)}>No Data</div>
              }
            >
              <Column
                title="Project Name"
                dataIndex="id"
                cell={(value) => {
                  const parts = value.split(":");
                  return parts.length > 1 ? parts[parts.length - 3] : "";
                }}
              />
              <Column
                title="schema"
                dataIndex="id"
                cell={(value) => {
                  const parts = value.split(":");
                  return parts.length > 1 ? parts[parts.length - 2] : "";
                }}
              />
              <Column title="Table Name" dataIndex="name" />
              <Column title="Table Comment" dataIndex="comment" />
              <Column title="Table Type" dataIndex="tableType" />
              <Column
                title="Partition Status"
                dataIndex="partitionKeys"
                cell={(value) => {
                  return value != null && value.length > 0 ? "Yes" : "No";
                }}
              />
              <Column
                title="Creation Time"
                dataIndex="createTime"
                cell={(value) => {
                  return moment(value).format("YYYY-MM-DD HH:mm:ss");
                }}
              />
              <Column
                title="Modification Time"
                dataIndex="modifyTime"
                cell={(value) => {
                  return moment(value).format("YYYY-MM-DD HH:mm:ss");
                }}
              />
              <Column
                title="Operation"
                width={150}
                cell={(value, index, record) => (
                  <div className={cn(classes.buttonGroup)}>
                    <Button
                      type="primary"
                      onClick={() => onViewDetail(record)}
                      text
                    >
                      View Details
                    </Button>
                    <Button
                      type="primary"
                      onClick={() => onViewLineage(record)}
                      text
                    >
                      View Table Lineage
                    </Button>
                  </div>
                )}
              />
            </Table>
            <Pagination
              current={current}
              total={total}
              onChange={onSubmit}
              showJump={false}
              className={cn(classes.tablePaginationWrapper)}
            />
          </div>
        </Content>
      </Card>
    </div>
  );
};

export default App;

After completing the above code development, you can deploy and run the project code. You can enter the MaxCompute Project ID and other parameters to query and obtain the list of all tables under the project space, with support for paged queries.

2. Query table details

The following practice will combine three API operations: GetTable, ListColumns, and ListPartitions in metadata to query table details, and use the UpdateColumnBusinessMetadata API to update the business description of fields. The practice workflow is as follows.

Backend: Use MetaServiceProxy to query table details

  1. Create getTable, listColumns, listPartitions methods in MetaServiceProxy to call operations to obtain basic information of the table, field information of the table, and partition information of the table, and create the updateColumnBusinessMetadata method to update the business description of fields.

    /**
     * @author dataworks demo
     */
    @Service
     public class MetaServiceProxy {
       
        @Autowired
        private DataWorksOpenApiClient dataWorksOpenApiClient;
       
        /**
            * DataWorks OpenAPI : getTableDto
            *
            * @param getTableDto
            */
        public Table getTable(GetTableDto getTableDto) {
          try {
            Client client = dataWorksOpenApiClient.createClient();
            GetTableRequest getTableRequest = new GetTableRequest();
            // Table ID
            getTableRequest.setId(getTableDto.getId());
            // Whether to return business metadata
            getTableRequest.setIncludeBusinessMetadata(getTableDto.getIncludeBusinessMetadata());
       
            GetTableResponse response = client.getTable(getTableRequest);
            // Corresponding data table
            Table table = response.getBody().getTable();
            // Table ID
            System.out.println(table.getId());
            // Parent entity ID
            System.out.println(table.getParentMetaEntityId());
            // Table name
            System.out.println(table.getName());
            // Table comment
            System.out.println(table.getComment());
            // Database/MaxCompute project name
            System.out.println(table.getId().split(":")[3]);
            // Schema name
            System.out.println(table.getId().split(":")[4]);
            // Table type
            System.out.println(table.getTableType());
            // Table creation time
            System.out.println(table.getCreateTime());
            // Table modification time
            System.out.println(table.getModifyTime());
            // Check if table is partitioned
            System.out.println(table.getPartitionKeys() != null && !table.getPartitionKeys().isEmpty());
            // Technical metadata of the table
            TableTechnicalMetadata technicalMetadata = table.getTechnicalMetadata();
            // Table owner (name)
            System.out.println(technicalMetadata.getOwner());
            // Whether it is a compressed table
            System.out.println(technicalMetadata.getCompressed());
            // Input format (HMS, DLF and other types support)
            System.out.println(technicalMetadata.getInputFormat());
            // Output format (HMS, DLF and other types support)
            System.out.println(technicalMetadata.getOutputFormat());
            // Class used for table serialization (HMS, DLF and other types support)
            System.out.println(technicalMetadata.getSerializationLibrary());
            // Table storage location (HMS, DLF and other types support)
            System.out.println(technicalMetadata.getLocation());
            // Other parameters
            Map<String, String> parameters = technicalMetadata.getParameters();
                   // Last DDL update time (millisecond-level timestamp), only MaxCompute type supports
                   System.out.println(parameters.get("lastDDLTime"));
                   // Lifecycle (unit: days), only MaxCompute type supports
                   System.out.println(parameters.get("lifecycle"));
                   // Storage size (unit: bytes), not real-time, only MaxCompute type supports
                   System.out.println(parameters.get("dataSize"));
                   // Business metadata of the table
                   TableBusinessMetadata businessMetadata = table.getBusinessMetadata();
                   if (businessMetadata != null) {
                       // Usage instructions
                       System.out.println(businessMetadata.getReadme());
                       // Custom tag information
                       List<TableBusinessMetadataTags> tags = businessMetadata.getTags();
                       if (tags != null && !tags.isEmpty()) {
                           for (TableBusinessMetadataTags tag : tags) {
                               System.out.println(tag.getKey() + ":" + tag.getValue());
                           }
                       }
                       // Upstream production tasks
                       List<TableBusinessMetadataUpstreamTasks> upstreamTasks = businessMetadata.getUpstreamTasks();
                       if (upstreamTasks != null && !upstreamTasks.isEmpty()) {
                           for (TableBusinessMetadataUpstreamTasks upstreamTask : upstreamTasks) {
                               // Task ID and task name, task details can be obtained through GetTask API
                               System.out.println(upstreamTask.getId() + ":" + upstreamTask.getName());
                           }
                       }
                       // Categories
                       List<List<TableBusinessMetadataCategories>> categories = businessMetadata.getCategories();
                       if (categories != null && !categories.isEmpty()) {
                           // Traverse multi-level categories
                           for (List<TableBusinessMetadataCategories> category : categories) {
                               // Output a single multi-level category
                               System.out.println(
                                       category.stream()
                                               .map(TableBusinessMetadataCategories::getName)
                                               .collect(Collectors.joining("->"))
                               );
                           }
                       }
                       // Extension information, only MaxCompute type supports
                       TableBusinessMetadataExtension extension = businessMetadata.getExtension();
                       if (extension != null) {
                           // Project ID
                           System.out.println(extension.getProjectId());
                           // Environment (Prod: production / Dev: development)
                           System.out.println(extension.getEnvType());
                           // Favorite count
                           System.out.println(extension.getFavorCount());
                           // Read count
                           System.out.println(extension.getReadCount());
                           // View count
                           System.out.println(extension.getViewCount());
                       }
                   }
                   return table;
               } catch (TeaException error) {
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               } catch (Exception _error) {
                   TeaException error = new TeaException(_error.getMessage(), _error);
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               }
               return null;
           }
       
           /**
            * DataWorks OpenAPI : ListColumns
            *
            * @param listColumnsDto
            */
           public ListColumnsResponseBodyPagingInfo listColumns(ListColumnsDto listColumnsDto) {
               try {
                   Client client = dataWorksOpenApiClient.createClient();
                   ListColumnsRequest listColumnsRequest = new ListColumnsRequest();
                   // Table ID (from ListTables API)
                   listColumnsRequest.setTableId(listColumnsDto.getTableId());
                   // Field name, supports fuzzy matching
                   listColumnsRequest.setName(listColumnsDto.getName());
                   // Field comment, supports word segmentation matching
                   listColumnsRequest.setComment(listColumnsDto.getComment());
                   // Sort method, supports Name / Position (default)
                   listColumnsRequest.setSortBy(listColumnsDto.getSortBy());
                   // Sort direction, supports Asc (default) / Desc
                   listColumnsRequest.setOrder(listColumnsDto.getOrder());
                   // Page number, default is 1
                   listColumnsRequest.setPageNumber(listColumnsDto.getPageNumber());
                   // Page size, default is 10, maximum 100
                   listColumnsRequest.setPageSize(listColumnsDto.getPageSize());
                   ListColumnsResponse response = client.listColumns(listColumnsRequest);
                   // Get the total number of fields that meet the requirements
                   System.out.println(response.getBody().getRequestId());
                   System.out.println(response.getBody().getPagingInfo().getTotalCount());
                   for (Column column : response.getBody().getPagingInfo().getColumns()) {
                       // Field ID
                       System.out.println(column.getId());
                       // Field name
                       System.out.println(column.getName());
                       // Field comment
                       System.out.println(column.getComment());
                       // Field type
                       System.out.println(column.getType());
                       // Field position
                       System.out.println(column.getPosition());
                       // Whether it is a partition field
                       System.out.println(column.getPartitionKey());
                       // Whether it is a primary key, only MaxCompute type supports
                       System.out.println(column.getPrimaryKey());
                       // Whether it is a foreign key, only MaxCompute type supports
                       System.out.println(column.getForeignKey());
                       // Business description for the field (supported by MaxCompute, HMS, and DLF) 
                       if (column.getBusinessMetadata() != null) {
                           System.out.println(column.getBusinessMetadata().getDescription());
                       }
                   }
                   return response.getBody().getPagingInfo();
               } catch (TeaException error) {
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               } catch (Exception _error) {
                   TeaException error = new TeaException(_error.getMessage(), _error);
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               }
               return null;
           }
       
           /**
            * DataWorks OpenAPI : UpdateColumnBusinessMetadata
            *
            * @param updateColumnBusinessMetadataDto
            */
           public boolean updateColumnBusinessMetadata(UpdateColumnBusinessMetadataDto updateColumnBusinessMetadataDto) {
               try {
                   // Currently supports MaxCompute, DLF, HMS (EMR cluster) types
                   Client client = dataWorksOpenApiClient.createClient();
                   UpdateColumnBusinessMetadataRequest updateColumnBusinessMetadataRequest = new UpdateColumnBusinessMetadataRequest();
                   // Field ID
                   updateColumnBusinessMetadataRequest.setId(updateColumnBusinessMetadataDto.getId());
                   // Field business description
                   updateColumnBusinessMetadataRequest.setDescription(updateColumnBusinessMetadataDto.getDescription());
                   UpdateColumnBusinessMetadataResponse response = client.updateColumnBusinessMetadata(updateColumnBusinessMetadataRequest);
                   System.out.println(response.getBody().getRequestId());
                   // Whether the update is successful
                   return response.getBody().getSuccess();
               } catch (TeaException error) {
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               } catch (Exception _error) {
                   TeaException error = new TeaException(_error.getMessage(), _error);
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               }
               return false;
           }
       
           /**
            * DataWorks OpenAPI : ListPartitions
            *
            * @param listPartitionsDto
            */
           public ListPartitionsResponseBodyPagingInfo listPartitions(ListPartitionsDto listPartitionsDto) {
               try {
                   // Currently supports MaxCompute type and HMS (EMR cluster) type
                   Client client = dataWorksOpenApiClient.createClient();
                   ListPartitionsRequest listPartitionsRequest = new ListPartitionsRequest();
                   // Table ID (from ListTables API)
                   listPartitionsRequest.setTableId(listPartitionsDto.getTableId());
                   // Partition name, supports fuzzy matching
                   listPartitionsRequest.setName(listPartitionsDto.getName());
                   // Sort method, supports Name (HMS method default, MaxCompute type supports) / CreateTime (MaxCompute type default) / ModifyTime (MaxCompute type supports) / RecordCount (MaxCompute type supports) / DataSize (MaxCompute type supports)
                   listPartitionsRequest.setSortBy(listPartitionsDto.getSortBy());
                   // Sort direction, supports Asc (default) / Desc
                   listPartitionsRequest.setOrder(listPartitionsDto.getOrder());
                   // Page number, default is 1
                   listPartitionsRequest.setPageNumber(listPartitionsDto.getPageNumber());
                   // Page size, default is 10, maximum 100
                   listPartitionsRequest.setPageSize(listPartitionsDto.getPageSize());
                   ListPartitionsResponse response = client.listPartitions(listPartitionsRequest);
                   // Get the total number of partitions that meet the requirements
                   System.out.println(response.getBody().getPagingInfo().getTotalCount());
                   for (Partition partition : response.getBody().getPagingInfo().getPartitionList()) {
                       // Table ID
                       System.out.println(partition.getTableId());
                       // Partition name
                       System.out.println(partition.getName());
                       // Creation time (millisecond-level timestamp)
                       System.out.println(partition.getCreateTime());
                       // Modification time (millisecond-level timestamp)
                       System.out.println(partition.getModifyTime());
                       // Partition record count, only MaxCompute type supports
                       System.out.println(partition.getRecordCount());
                       // Partition storage size (unit: bytes), only MaxCompute type supports
                       System.out.println(partition.getDataSize());
                   }
                   return response.getBody().getPagingInfo();
               } catch (TeaException error) {
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               } catch (Exception _error) {
                   TeaException error = new TeaException(_error.getMessage(), _error);
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               }
               return null;
           }
       }
  2. Provide four methods in MetaRestController: getTable, listColumns, listPartitions, and updateColumnBusinessMetadata for frontend calls.

    /**
     * Custom metadata platform using the DataWorks API
     *
     * @author dataworks demo
     */
    @RestController
      @RequestMapping("/meta")
      public class MetaRestController {
    
        @Autowired
        private MetaServiceProxy metaService;
    
        /**
         * Get table details
         *
         * @param getTableDto
         * @return {@link Table}
         */
        @CrossOrigin(origins = "http://localhost:8080")
        @GetMapping("/getTable")
        public Table getTable(GetTableDto getTableDto) {
          System.out.println("getTableDto = " + getTableDto);
          return metaService.getTable(getTableDto);
        }
    
        /**
         * List columns with pagination
         *
         * @param listColumnsDto
         * @return {@link ListColumnsResponseBodyPagingInfo}
         */
        @CrossOrigin(origins = "http://localhost:8080")
        @GetMapping("/listColumns")
        public ListColumnsResponseBodyPagingInfo listColumns(ListColumnsDto listColumnsDto) {
          System.out.println("listColumnsDto = " + listColumnsDto);
          return metaService.listColumns(listColumnsDto);
        }
    
        /**
         * Update column business metadata
         *
         * @param updateColumnBusinessMetadataDto
         * @return {@link Boolean}
         */
        @CrossOrigin(origins = "http://localhost:8080")
        @PostMapping("/updateColumnBusinessMetadata")
        public Boolean updateColumnBusinessMetadata(@RequestBody UpdateColumnBusinessMetadataDto updateColumnBusinessMetadataDto) {
          System.out.println("updateColumnBusinessMetadataDto = " + updateColumnBusinessMetadataDto);
          return metaService.updateColumnBusinessMetadata(updateColumnBusinessMetadataDto);
        }
    
        /**
         * Query partition data by page
         *
         * @param listPartitionsDto
         * @return {@link ListPartitionsResponseBodyPagingInfo}
         */
        @CrossOrigin(origins = "http://localhost:8080")
        @GetMapping("/listPartitions")
        public ListPartitionsResponseBodyPagingInfo listPartitions(ListPartitionsDto listPartitionsDto) {
          System.out.println("listPartitionsDto = " + listPartitionsDto);
          return metaService.listPartitions(listPartitionsDto);
        }
      }

Frontend: Display basic table information, field information, and partition information

import React from "react";
import moment from "moment";
import {
  Form,
  Grid,
  Table,
  Pagination,
  Tag,
  Field,
  Input,
  Button,
  Select,
  Dialog,
} from "@alifd/next";
import cn from "classnames";
import * as services from "../services";
import {
  type ListColumnsInput,
  type TableColumn,
  type TableEntity,
  type TablePartition,
} from "../services/meta";
import classes from "../styles/detailContent.module.css";

export interface Props {
  item: TableEntity;
}

const formItemLayout = {
  labelCol: {
    fixedSpan: 6,
  },
  wrapperCol: {
    span: 18,
  },
  labelTextAlign: "left" as const,
  colon: true,
  className: cn(classes.formItemWrapper),
};
const { Row, Col } = Grid;
const { Item } = Form;
const { Column } = Table;
const { Option } = Select;
const DetailContent: React.FunctionComponent<Props> = (props) => {
  let columnsPageSize = 10;
  let partitionsPageSize = 5;
  const listColumnsField = Field.useField();
  const listPartitionsField = Field.useField();
  const labelAlign = "top";
  const [detail, setDetail] = React.useState<Partial<TableEntity>>({});
  const [columns, setColumns] = React.useState<Partial<TableColumn[]>>([]);
  const [partitions, setPartitions] = React.useState<Partial<TablePartition[]>>(
    []
  );
  const [columnsTotal, setColumnsTotal] = React.useState<number>(0);
  const [partitionTotal, setPartitionTotal] = React.useState<number>(0);
  const [editingKey, setEditingKey] = React.useState<string>("");
  const [editingDescription, setEditingDescription] =
    React.useState<string>("");

  const [updateColumnDialogVisible, setUpdateColumnDialogVisible] =
    React.useState<boolean>(false);
  const onUpdateColumnDialogOpen = () => {
    setUpdateColumnDialogVisible(true);
  };
  const onUpdateColumnDialogOk = () => {
    handleColumnDescriptionSave();
    onUpdateColumnDialogClose();
  };
  const onUpdateColumnDialogClose = () => {
    setUpdateColumnDialogVisible(false);
  };
  const handleEditColumnDescription = async (record: TableColumn) => {
    setEditingKey(record.id);
    setEditingDescription(record.businessMetadata?.description);
    onUpdateColumnDialogOpen();
  };
  const handleColumnDesrciptionChange = async (e: string) => {
    setEditingDescription(e);
  };
  const handleColumnDescriptionSave = async () => {
    try {
      const res = await services.meta.updateColumnDescription({
        id: editingKey,
        description: editingDescription,
      });
      if (!res) {
        console.log("Request update failed");
      }
    } catch (e) {
      console.error("Update failed", e);
    } finally {
      listColumns();
      setEditingKey("");
    }
  };
  const getTableDetail = React.useCallback(async () => {
    const response = await services.meta.getTableDetail({
      id: props.item.id,
      includeBusinessMetadata: true,
    });
    setDetail(response);
  }, [props.item.id]);
  const listColumns = React.useCallback(
    async (pageNumber: number = 1) => {
      listColumnsField.setValue("tableId", props.item.id);
      listColumnsField.setValue("pageSize", columnsPageSize);
      listColumnsField.validate(async (errors, values) => {
        if (errors) {
          return;
        }
        try {
          const response = await services.meta.getMetaTableColumns({
            pageNumber,
            ...values,
          } as ListColumnsInput);
          setColumns(response.columns);
          setColumnsTotal(response.totalCount);
        } catch (e) {
          throw e;
        } finally {
        }
      });
    },
    [listColumnsField]
  );
  const listPartitions = React.useCallback(
    async (pageNumber: number = 1) => {
      listPartitionsField.setValue("tableId", props.item.id);
      listPartitionsField.setValue("pageSize", partitionsPageSize);
      listPartitionsField.validate(async (errors, values) => {
        if (errors) {
          return;
        }
        try {
          const response = await services.meta.getTablePartition({
            pageNumber,
            ...values,
          } as ListColumnsInput);
          setPartitions(response.partitionList);
          setPartitionTotal(response.totalCount);
        } catch (e) {
          throw e;
        } finally {
        }
      });
    },
    [listPartitionsField]
  );
  React.useEffect(() => {
    if (props.item.id) {
      getTableDetail();
      listColumns();
      listPartitions();
    }
  }, [props.item.id]);
  return (
    <div className={cn(classes.detailContentWrapper)}>
      <Dialog
        v2
        title="Update Field Business Metadata"
        visible={updateColumnDialogVisible}
        onOk={onUpdateColumnDialogOk}
        onClose={onUpdateColumnDialogClose}
      >
        <Row>
          <Col>
            <Item
              {...formItemLayout}
              labelAlign={labelAlign}
              label="Field Business Description"
            >
              <Input
                value={editingDescription}
                onChange={(e) => handleColumnDesrciptionChange(e.toString())}
              ></Input>
            </Item>
          </Col>
        </Row>
      </Dialog>
      <Form labelTextAlign="left">
        <Row>
          <Col>
            <Item {...formItemLayout} label="Table ID">
              <span className={cn(classes.formContentWrapper)}>
                {detail.id}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="MaxCompute Project">
              <span className={cn(classes.formContentWrapper)}>
                {detail?.id?.split(":").slice(-3, -2)[0]}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="schema Name">
              <span className={cn(classes.formContentWrapper)}>
                {detail?.id?.split(":").slice(-2, -1)[0]}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Table Name">
              <span className={cn(classes.formContentWrapper)}>
                {detail.name}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Comment">
              <span className={cn(classes.formContentWrapper)}>
                {detail.comment}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Table Type">
              <span className={cn(classes.formContentWrapper)}>
                {detail.tableType}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Is Compressed Table">
              <span className={cn(classes.formContentWrapper)}>
                {detail.technicalMetadata?.compressed ? "Yes" : "No"}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Workspace ID">
              <span className={cn(classes.formContentWrapper)}>
                {detail.businessMetadata?.extension?.projectId}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Environment">
              <span className={cn(classes.formContentWrapper)}>
                {detail.businessMetadata?.extension?.envType === "Dev"
                  ? "Development"
                  : "Production"}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Table Lifecycle">
              <span className={cn(classes.formContentWrapper)}>
                {detail.technicalMetadata?.parameters["lifecycle"]
                  ? `${detail.technicalMetadata?.parameters["lifecycle"]} days`
                  : "Permanent"}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Storage Space">
              <span
                className={cn(classes.formContentWrapper)}
              >{`${detail.technicalMetadata?.parameters["dataSize"]} Bytes`}</span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="DDL Last Change Time">
              <span className={cn(classes.formContentWrapper)}>
                {moment(
                  Number(detail.technicalMetadata?.parameters["lastDDLTime"])
                ).format("YYYY-MM-DD HH:mm:ss")}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Partition Status">
              <span className={cn(classes.formContentWrapper)}>
                {detail.partitionKeys != null && detail.partitionKeys.length > 0
                  ? "Yes"
                  : "No"}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Tags">
              <div className={cn(classes.formContentWrapper)}>
                {detail.businessMetadata?.tags?.map((tag, index) => (
                  <span key={index} className={cn(classes.tag)}>
                    <Tag type="primary" size="small">
                      {tag.key + (tag.value ? ":" + tag.value : "")}
                    </Tag>
                  </span>
                ))}
              </div>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Categories">
              <span className={cn(classes.formContentWrapper)}>
                {detail.businessMetadata?.categories?.map((category, index) => (
                  <span key={index} className={cn(classes.tag)}>
                    <Tag type="normal" color="gray" size="small">
                      {category.map((obj) => obj.name).join("->")}
                    </Tag>
                  </span>
                ))}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Upstream Production Tasks">
              <span className={cn(classes.formContentWrapper)}>
                {detail.businessMetadata?.upstreamTasks?.map((task, index) => (
                  <span key={index} className={cn(classes.tag)}>
                    <Tag type="primary" size="small">
                      {task.name + " (" + task.id + ")"}
                    </Tag>
                  </span>
                ))}
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="View Count">
              <span className={cn(classes.formContentWrapper)}>
                <Tag type="primary" size="small">
                  {detail.businessMetadata?.extension?.viewCount}
                </Tag>
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Read Count">
              <span className={cn(classes.formContentWrapper)}>
                <Tag type="primary" size="small">
                  {detail.businessMetadata?.extension?.readCount}
                </Tag>
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Favorite Count">
              <span className={cn(classes.formContentWrapper)}>
                <Tag type="primary" size="small">
                  {detail.businessMetadata?.extension?.favorCount}
                </Tag>
              </span>
            </Item>
          </Col>
        </Row>
        <Row>
          <Col>
            <Item {...formItemLayout} label="Creation Time">
              <span className={cn(classes.formContentWrapper)}>
                {moment(detail.createTime).format("YYYY-MM-DD HH:mm:ss")}
              </span>
            </Item>
          </Col>
          <Col>
            <Item {...formItemLayout} label="Modification Time">
              <span className={cn(classes.formContentWrapper)}>
                {moment(detail.modifyTime).format("YYYY-MM-DD HH:mm:ss")}
              </span>
            </Item>
          </Col>
        </Row>
        <br />
        <Item label="Field Details" colon>
          <Form
            field={listColumnsField}
            colon
            fullWidth
            style={{ marginTop: 10, marginBottom: 10 }}
          >
            <Row gutter="4">
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Name"
                  name="name"
                >
                  <Input placeholder="Enter field name, fuzzy matching" />
                </Item>
              </Col>
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Comment"
                  name="comment"
                >
                  <Input placeholder="Enter field comment, fuzzy matching" />
                </Item>
              </Col>
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Sort Method"
                  name="sortBy"
                >
                  <Select defaultValue="Position" style={{ width: "100%" }}>
                    <Option value="Position">Position</Option>
                    <Option value="Name">Name</Option>
                  </Select>
                </Item>
              </Col>
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Sort Direction"
                  name="order"
                >
                  <Select defaultValue="Asc" style={{ width: "100%" }}>
                    <Option value="Asc">Ascending</Option>
                    <Option value="Desc">Descending</Option>
                  </Select>
                </Item>
              </Col>
              <Col>
                <div
                  className={cn(
                    classes.searchPanelButtonWrapper,
                    classes.buttonGroup
                  )}
                  style={{ marginTop: "20px", textAlign: "right" }}
                >
                  <Button type="primary" onClick={() => listColumns()}>
                    Query
                  </Button>
                </div>
              </Col>
            </Row>
          </Form>
          <Table dataSource={columns}>
            <Column title="Name" dataIndex="name" />
            <Column title="Type" dataIndex="type" />
            <Column title="Comment" dataIndex="comment" />
            <Column
              title="Business Description"
              dataIndex="businessMetadata"
              cell={(value, index, record) => (
                <div
                  style={{
                    display: "flex",
                    justifyContent: "space-between",
                    alignItems: "center",
                  }}
                >
                  <span>{value?.description}</span>
                  <Button
                    onClick={() => handleEditColumnDescription(record)}
                    size="small"
                    type="primary"
                    text
                    style={{ marginLeft: 8 }}
                  >
                    Edit
                  </Button>
                </div>
              )}
            />
            <Column
              title="Is Primary Key"
              dataIndex="primaryKey"
              cell={(value) => (value ? "Yes" : "")}
            />
            <Column
              title="Is Foreign Key"
              dataIndex="foreignKey"
              cell={(value) => (value ? "Yes" : "")}
            />
            <Column
              title="Is Partition Field"
              dataIndex="partitionKey"
              cell={(value) => (value ? "Yes" : "")}
            />
            <Column title="Position" dataIndex="position" />
          </Table>
          <Pagination
            total={columnsTotal}
            pageSize={columnsPageSize}
            onChange={listColumns}
            showJump={false}
            className={cn(classes.tablePaginationWrapper)}
          />
        </Item>
        <br />
        <Item label="Partition Details" style={{ marginBottom: 32 }} colon>
          <Form
            field={listPartitionsField}
            colon
            fullWidth
            style={{ marginTop: 10, marginBottom: 10 }}
          >
            <Row gutter="4">
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Name"
                  name="name"
                >
                  <Input placeholder="Enter partition name (partial matches allowed)" />
                </Item>
              </Col>
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Sort Method"
                  name="sortBy"
                >
                  <Select defaultValue="CreateTime" style={{ width: "100%" }}>
                    <Option value="CreateTime">Creation Time</Option>
                    <Option value="ModifyTime">Modification Time</Option>
                    <Option value="Name">Partition Name</Option>
                    <Option value="RecordCount">Partition Record Count</Option>
                    <Option value="DataSize">Partition Size</Option>
                  </Select>
                </Item>
              </Col>
              <Col>
                <Item
                  {...formItemLayout}
                  labelAlign={labelAlign}
                  label="Sort Direction"
                  name="order"
                >
                  <Select defaultValue="Asc" style={{ width: "100%" }}>
                    <Option value="Asc">Ascending</Option>
                    <Option value="Desc">Descending</Option>
                  </Select>
                </Item>
              </Col>
              <Col>
                <div
                  className={cn(
                    classes.searchPanelButtonWrapper,
                    classes.buttonGroup
                  )}
                  style={{ marginTop: "20px", textAlign: "right" }}
                >
                  <Button type="primary" onClick={() => listPartitions()}>
                    Query
                  </Button>
                </div>
              </Col>
            </Row>
          </Form>
          <Table dataSource={partitions} emptyContent={<span>Non-partitioned Table</span>}>
            <Column title="Name" dataIndex="name" />
            <Column
              title="Creation Time"
              dataIndex="createTime"
              cell={(value) => moment(value).format("YYYY-MM-DD HH:mm:ss")}
            />
            <Column
              title="Modification Time"
              dataIndex="modifyTime"
              cell={(value) => moment(value).format("YYYY-MM-DD HH:mm:ss")}
            />
            <Column
              title="Partition Size"
              dataIndex="dataSize"
              cell={(value) => `${value} bytes`}
            />
            <Column title="Partition Record Count" dataIndex="recordCount" />
          </Table>
          <Pagination
            total={partitionTotal}
            pageSize={partitionsPageSize}
            onChange={listPartitions}
            showJump={false}
            className={cn(classes.tablePaginationWrapper)}
          />
        </Item>
      </Form>
    </div>
  );
};

export default DetailContent;

After completing the above code development, you can deploy and run the project code locally. For deployment and running operations, see Common operations: Local deployment and running.

3. Query table lineage

When a table definition changes, you can use the DataWorks API to perform lineage analysis of upstream and downstream tasks to find the nodes corresponding to the table.

Note

Use ListLineages to query the upstream and downstream lineage of data tables.

Backend: Use MetaServiceProxy to query table lineage

  1. Create the listTableLineages method in MetaServiceProxy to call the API to obtain the upstream and downstream lineage entity and relationship information of the table.

    /**
     * @author dataworks demo
     */
    @Service
      public class MetaServiceProxy {
       
        @Autowired
        private DataWorksOpenApiClient dataWorksOpenApiClient;
       
        /**
            * DataWorks OpenAPI : ListLineages
            *
            * @param listTableLineagesDto
            */
        public ListLineagesResponseBodyPagingInfo listTableLineages(ListTableLineagesDto listTableLineagesDto) {
          try {
            Client client = dataWorksOpenApiClient.createClient();
            ListLineagesRequest listLineagesRequest = new ListLineagesRequest();
            // Lineage query direction, whether it is upstream
            boolean needUpstream = "up".equalsIgnoreCase(listTableLineagesDto.getDirection());
            // Table ID (from ListTables API)
            // Partition name, supports fuzzy matching
            if (needUpstream) {
              listLineagesRequest.setDstEntityId(listTableLineagesDto.getTableId());
              listLineagesRequest.setSrcEntityName(listTableLineagesDto.getName());
            } else {
              listLineagesRequest.setSrcEntityId(listTableLineagesDto.getTableId());
              listLineagesRequest.setDstEntityName(listTableLineagesDto.getName());
            }
            // Sort method, supports Name
            listLineagesRequest.setSortBy(listTableLineagesDto.getSortBy());
            // Sort direction, supports Asc (default) / Desc
            listLineagesRequest.setOrder(listTableLineagesDto.getOrder());
            // Page number, default is 1
            listLineagesRequest.setPageNumber(listTableLineagesDto.getPageNumber());
            // Page size, default is 10, maximum 100
            listLineagesRequest.setPageSize(listTableLineagesDto.getPageSize());
            // Whether to return specific lineage relationship information
            listLineagesRequest.setNeedAttachRelationship(true);
            ListLineagesResponse response = client.listLineages(listLineagesRequest);
            // Get the total number of lineage entities that meet the requirements
            System.out.println(response.getBody().getPagingInfo().getTotalCount());
            if (response.getBody().getPagingInfo().getLineages() == null) {
              response.getBody().getPagingInfo().setLineages(Collections.emptyList());
            }
            if (response.getBody().getPagingInfo().getLineages() != null) {
              for (ListLineagesResponseBodyPagingInfoLineages lineage : response.getBody().getPagingInfo().getLineages()) {
                LineageEntity entity;
                // Get the corresponding lineage entity
                if (needUpstream) {
                  entity = lineage.getSrcEntity();
                } else {
                  entity = lineage.getDstEntity();
                }
                // Entity ID
                System.out.println(entity.getId());
                // Entity name
                System.out.println(entity.getName());
                // Entity attribute information
                System.out.println(entity.getAttributes());
                // Lineage relationship list between entities
                           List<LineageRelationship> relationships = lineage.getRelationships();
                           if (relationships != null) {
                               relationships.forEach(relationship -> {
                                   // Lineage relationship ID
                                   System.out.println(relationship.getId());
                                   // Creation time
                                   System.out.println(relationship.getCreateTime());
                                   // Task corresponding to the lineage relationship
                                   LineageTask task = relationship.getTask();
                                   if (task != null) {
                                       // Task ID, for DataWorks tasks, task details can be obtained through GetTask
                                       System.out.println(task.getId());
                                       // Task type
                                       System.out.println(task.getType());
                                       // Task attribute information
                                       Map<String, String> attributes = task.getAttributes();
                                       if (attributes != null) {
                                           // For DataWorks tasks, task and instance attribute information can be obtained from it
                                           // Project ID
                                           System.out.println(attributes.get("projectId"));
                                           // Task ID
                                           String taskId = attributes.containsKey("taskId") ? attributes.get("taskId") : attributes.get("nodeId");
                                           System.out.println(taskId);
                                           // Instance ID
                                           String taskInstanceId = attributes.containsKey("taskInstanceId") ? attributes.get("taskInstanceId") : attributes.get("jobId");
                                           System.out.println(taskInstanceId);
                                       }
                                   }
                               });
                           }
                       }
                       return response.getBody().getPagingInfo();
                   }
               } catch (TeaException error) {
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               } catch (Exception _error) {
                   TeaException error = new TeaException(_error.getMessage(), _error);
                   // For demonstration only. Handle exceptions properly in production code.
                   // Error message
                   System.out.println(error.getMessage());
                   // Diagnostic address
                   System.out.println(error.getData().get("Recommend"));
                   com.aliyun.teautil.Common.assertAsString(error.message);
               }
               return null;
           }
       }
  2. Provide the listTableLineages method in MetaRestController for frontend calls.

    /**
     * Custom metadata platform using the DataWorks API
     *
     * @author dataworks demo
     */
    @RestController
      @RequestMapping("/meta")
      public class MetaRestController {
       
        @Autowired
        private MetaServiceProxy metaService;
       
        /**
            * Query the lineage relationship of the table
            *
            * @param listTableLineagesDto
            * @return
            */
        @CrossOrigin(origins = "http://localhost:8080")
        @GetMapping("/listTableLineages")
        public ListLineagesResponseBodyPagingInfo listTableLineages(ListTableLineagesDto listTableLineagesDto) {
          System.out.println("listTableLineagesDto = " + listTableLineagesDto);
          return metaService.listTableLineages(listTableLineagesDto);
        }
       
      }

Frontend: Display table lineage

import React from 'react';
import Graphin, { Behaviors } from '@antv/graphin';
import type { IUserNode, IUserEdge, GraphinData, GraphEvent } from '@antv/graphin';
import cn from 'classnames';
import * as services from '../services';
import type { TableEntity, ListTableLineageOutput, LineageEntity } from '../services/meta';
import classes from '../styles/lineageContent.module.css';
import '@antv/graphin/dist/index.css';

export interface Props {
  item: TableEntity;
}

function transNode(entity: LineageEntity | TableEntity, direction?: string): IUserNode {
  return {
    id: entity.id,
    label: entity.name,
    data: {
      ...entity,
      direction,
    },
    style: {
      label: {
        value: entity.name,
      }
    },
  }
}
function transEdge(source: LineageEntity | TableEntity, target: LineageEntity | TableEntity): IUserEdge {
  return {
    source: source.id,
    target: target.id,
    style: {
      keyshape: {
        lineDash: [8, 4],
        lineWidth: 2,
      },
      animate: {
        type: 'line-dash',
        repeat: true,
      },
    }
  };
}
function parse(
  source: LineageEntity | TableEntity,
  data: ListTableLineageOutput,
  direction: string,
): [IUserNode[], IUserEdge[]] {
  const nodes: IUserNode[] = [];
  const edges: IUserEdge[] = [];
  data.lineages.forEach((lineageRelationship) => {
    if (direction === 'down') {
      nodes.push(transNode(lineageRelationship.dstEntity, direction));
      edges.push(transEdge(source, lineageRelationship.dstEntity));
    }  else {
      nodes.push(transNode(lineageRelationship.srcEntity, direction));
      edges.push(transEdge(lineageRelationship.srcEntity, source));
    }
  });
  return [nodes, edges];
}
function mergeNodes(prev: IUserNode[], next: IUserNode[]) {
  const result: IUserNode[] = prev.slice();
  next.forEach((item) => {
    const hasValue = prev.findIndex(i => i.id === item.id) >= 0;
    !hasValue && result.push(item);
  });
  return result;
}
function mergeEdges(source: IUserEdge[], target: IUserEdge[]) {
  const result: IUserEdge[] = source.slice();
  target.forEach((item) => {
    const hasValue = source.findIndex(i => i.target === item.target && i.source === item.source) >= 0;
    !hasValue && result.push(item);
  });
  return result;
}

const { ActivateRelations, DragNode, ZoomCanvas } = Behaviors;
const LineageContent: React.FunctionComponent = (props) => {
  const ref = React.useRef();
  const [data, setData] = React.useState({ nodes: [], edges: [] });
  const getTableLineage = async (
    collection: GraphinData,
    target: TableEntity | LineageEntity,
    direction: string,
  ) => {
    if (!direction) {
      return collection;
    }
    const response = await services.meta.getTableLineage({
      direction,
      tableId: target.id,
    });
    const [nodes, edges] = parse(target, response, direction);
    collection.nodes = mergeNodes(collection.nodes!, nodes);
    collection.edges = mergeEdges(collection.edges!, edges);
    return collection;
  };
  const init = async () => {
    let nextData = Object.assign({}, data, { nodes: [transNode(props.item)] });
    nextData = await getTableLineage(nextData, props.item, 'up');
    nextData = await getTableLineage(nextData, props.item, 'down');
    setData(nextData);
    ref.current!.graph.fitCenter();
  };
  React.useEffect(() => {
    ref.current?.graph && init();
  }, [
    ref.current?.graph,
  ]);
  React.useEffect(() => {
    const graph = ref.current?.graph;
    const event = async (event: GraphEvent) => {
      const source = event.item?.get('model').data;
      let nextData = Object.assign({}, data);
      nextData = await getTableLineage(nextData, source, source.direction);
      setData(nextData);
    };
    graph?.on('node:click', event);
    return () => {
      graph?.off('node:click', event);
    };
  }, [
    data,
    ref.current?.graph,
  ]);
  return (
    
}
        layout={{
          type: 'dagre',
          rankdir: 'LR',
          align: 'DL',
          nodesep: 10,
          ranksep: 40,
        }}
      >
        




  );
};

export default LineageContent;

After completing the above code development, you can deploy and run the project code locally. For deployment and running operations, see Common operations: Local deployment and running.

4. Find tasks corresponding to the table

When a table definition changes, you can perform lineage analysis of downstream tasks through the DataWorks API, OpenData, and message subscription to find the nodes corresponding to the table. The specific operations are as follows.

Note

Messages currently support table changes, task changes, etc. Enterprise Edition users can connect to table change messages. When you receive a table change message, you can view the lineage relationship of the table.

Lineage tasks

  1. Use ListLineages to view the lineage of the table, and obtain the DataWorks task ID (and task instance ID) from the Task in the lineage relationship.

  2. Based on the obtained task ID, use GetTask to obtain the task details.

Production tasks

  1. Based on the entity ID, use the GetTable API to obtain the business metadata of the table, which contains the DataWorks task ID in the upstream production tasks.

  2. Based on the specified task ID, use GetTask to obtain the task details.

Common operations: Local deployment and running

  1. Prepare the dependencies.

    You need to prepare the following dependency environments: Java 8 and above, Maven build tool, Node environment, pnpm tool. You can execute the following commands to determine whether you have the above environments:

    npm -v // If the installation is successful, the version is displayed in the command output. If the installation fails, an error that indicates no command is available is reported.
    java -version // If the installation is successful, the version is displayed in the command output. If the installation fails, an error that indicates no command is available is reported.
    pnpm -v // If the installation is successful, the version is displayed in the command output. If the installation fails, an error that indicates no command is available is reported.
  2. Download the project code and execute the following command.

    Project code download link: meta-api-demo.zip.

    pnpm i
  3. Find the application.properties file in the backend/src/main/resources path of the sample project, and modify the core parameters in the file.

    • api.access-key-id and api.access-key-secret need to be modified to the AccessKey ID and AccessKey Secret of your Alibaba Cloud account.

      Note

      You can refer to AccessKey Pair Management to obtain AccessKey and other related information of your Alibaba Cloud account.

    • api.region-id and api.endpoint need to be modified to the region information of the API to be called.

    Other parameters can be modified according to the actual situation. The filling example after modification is as follows.本地部署示例

  4. Execute the following command in the root directory of the project to run the sample code.

    npm run dev