All Products
Search

This Product

    Cloud Parallel File Storage:CreateDataFlowTask

    Document Center

    Cloud Parallel File Storage:CreateDataFlowTask

    Last Updated:Dec 11, 2024

    Creates a dataflow task.

    Operation description

    • Only Cloud Parallel File Storage CPFS for LINGJUN V2.4.0 and later support data flows. You can view the version information on the file system details page in the console.
    • You can create a data flow task only for a data flow that is in the Running state.
    • Data flow tasks are executed asynchronously. You can call the DescribeDataFlowTasks operation to query the task execution status. The task duration depends on the amount of data to be imported and exported. If a large amount of data exists, we recommend that you create multiple tasks.
    • When you manually run a data flow task, the automatic data update task for the data flow is interrupted and enters the pending state.
    • When you create an export task, make sure that the total length of the absolute path of the files to be exported from a CPFS or CPFS for LINGJUN file system does not exceed 1,023 characters.
    • CPFS for LINGJUN supports two types of tasks: batch tasks and streaming tasks. For more information, see Task types.

    Debugging

    You can run this interface directly in OpenAPI Explorer, saving you the trouble of calculating signatures. After running successfully, OpenAPI Explorer can automatically generate SDK code samples.

    Authorization information

    The following table shows the authorization information corresponding to the API. The authorization information can be used in the Action policy element to grant a RAM user or RAM role the permissions to call this API operation. Description:

    • Operation: the value that you can use in the Action element to specify the operation on a resource.
    • Access level: the access level of each operation. The levels are read, write, and list.
    • Resource type: the type of the resource on which you can authorize the RAM user or the RAM role to perform the operation. Take note of the following items:
      • The required resource types are displayed in bold characters.
      • If the permissions cannot be granted at the resource level, All Resources is used in the Resource type column of the operation.
    • Condition Key: the condition key that is defined by the cloud service.
    • Associated operation: other operations that the RAM user or the RAM role must have permissions to perform to complete the operation. To complete the operation, the RAM user or the RAM role must have the permissions to perform the associated operations.
    OperationAccess levelResource typeCondition keyAssociated operation
    nas:CreateDataFlowTaskcreate
    *DataFlow
    acs:nas:{#regionId}:{#accountId}:filesystem/{#filesystemId}
      none
    		none

    Request parameters

    ParameterTypeRequiredDescriptionExample
    FileSystemIdstringYes

    The ID of the file system.

    • The IDs of CPFS file systems must start with cpfs-. Example: cpfs-125487****.
    • The IDs of CPFS for LINGJUN file systems must start with bmcpfs-. Example: bmcpfs-0015****.
    Note CPFS is not supported on the international site.
    		bmcpfs-290w65p03ok64ya****
    DataFlowIdstringYes

    The dataflow ID.

    		df-194433a5be31****
    SrcTaskIdstringNo

    If you specify SrcTaskId, the configurations of the TaskAction, DataType, and EntryList parameters are copied from the desired dataflow task. You do not need to specify them.

    		task-27aa8e890f45****
    TaskActionstringNo

    The type of the data flow task.

    Valid values:

    • Import: imports data stored in the source storage to a CPFS file system.
    • Export: exports specified data from a CPFS file system to the source storage.
    • StreamImport: batch imports the specified data from the source storage to a CPFS file system.
    • StreamExport: batch exports specified data from a CPFS file system to the source storage.
    Note Only CPFS for LINGJUN V2.6.0 and later support StreamImport and StreamExport.
    		Import
    DataTypestringNo

    The type of data on which operations are performed by the dataflow task.

    Valid values:

    • Metadata: the metadata of a file, including the timestamp, ownership, and permission information of the file. If you select Metadata, only the metadata of the file is imported. You can only query the file. When you access the file data, the file is loaded from the source storage as required.
    • Data: the data blocks of a file.
    • MetaAndData: the metadata and data blocks of the file.
    		Metadata
    DirectorystringNo

    The source directory of the data.

    Limits:

    • The directory must be 1 to 1,023 characters in length.
    • The directory must be encoded in UTF-8.
    • The directory must start and end with a forward slash (/).
    • Only one directory can be listed at a time.
    • If the TaskAction parameter is set to Export, the directory must be a relative path within the FileSystemPath.
    • If the TaskAction parameter is set to Import, the directory must be a relative path within the SourceStoragePath.
    • If the TaskAction parameter is set to StreamExport, the directory must be a relative path within the FileSystemPath.
    • If the TaskAction parameter is set to StreamImport, the directory must be a relative path within the SourceStoragePath.
    Note Only CPFS for LINGJUN V2.6.0 and later support StreamImport and StreamExport.
    		/path_in_cpfs/
    EntryListstringNo

    The list of files that are executed by the data flow task.

    Limits:

    • The list must be encoded in UTF-8.
    • The total length of the file list cannot exceed 64 KB.
    • The file list is in JSON format.
    • The path of a single file must be 1 to 1,023 characters in length and must start with a forward slash (/).
    • If the TaskAction parameter is set to Import, each element in the list represents an OSS object name.
    • If the TaskAction parameter is set to Export, each element in the list represents a CPFS file path.
    		["/path_in_cpfs/file1", "/path_in_cpfs/file2"]
    DryRunbooleanNo

    Specifies whether to perform a dry run.

    During the dry run, the system checks whether the request parameters are valid and whether the requested resources are available. During the dry run, no data flow task is created and no fee is incurred.

    Valid values:

    • true: performs a dry run. The system checks the required parameters, request syntax, service limits, and available File Storage NAS (NAS) resources. If the request fails the dry run, an error message is returned. If the request passes the dry run, the HTTP status code 200 is returned. No value is returned for the TaskId parameter.
    • false (default): performs a dry run and sends the request. If the request passes the dry run, a data flow task is created.
    		false
    ClientTokenstringNo

    The client token that is used to ensure the idempotence of the request. You can use the client to generate the token, but you must make sure that the token is unique among different requests.

    The token can contain only ASCII characters and cannot exceed 64 characters in length. For more information, see How do I ensure the idempotence?

    Note If you do not specify this parameter, the system automatically uses the request ID as the client token. The value of RequestId may be different for each API request.
    		123e4567-e89b-12d3-a456-42665544****
    ConflictPolicystringNo

    The conflict policy for files with the same name. Valid values:

    • SKIP_THE_FILE: skips files with the same name.
    • KEEP_LATEST: compares the update time and keeps the latest version.
    • OVERWRITE_EXISTING: forcibly overwrites the existing file.
    Note This parameter is required for CPFS for LINGJUN file systems.
    		SKIP_THE_FILE
    DstDirectorystringNo

    The directory mapped to the data flow task. Limits:

    • The directory must start and end with a forward slash (/). The directory cannot be /../.
    • The directory must be 1 to 1,023 characters in length.
    • The directory must be encoded in UTF-8.
    • Only one directory can be listed at a time.
    • If the TaskAction parameter is set to Export, the directory must be a relative path within the SourceStoragePath.
    • If the TaskAction parameter is set to Import, the directory must be a relative path within the FileSystemPath.
    • If the TaskAction parameter is set to StreamExport, the directory must be a relative path within the SourceStoragePath.
    • If the TaskAction parameter is set to StreamImport, the directory must be a relative path within the FileSystemPath.
    Note Only CPFS for LINGJUN V2.6.0 and later support StreamImport and StreamExport.
    		/path_in_cpfs/
    CreateDirIfNotExistbooleanNo

    Specifies whether to automatically create a directory if no directory exists. Valid values:

    • true: automatically creates a directory.
    • false (default): does not automatically create a directory.
    Note
  • This parameter is required if the TaskAction parameter is set to Import.
  • Only CPFS for LINGJUN V2.6.0 and later support this parameter.
    •
    		false

    Response parameters

    ParameterTypeDescriptionExample
    object
    RequestIdstring

    The request ID.

    		2D69A58F-345C-4FDE-88E4-BF518948****
    TaskIdstring

    The ID of the dataflow task.

    		task-38aa8e890f45****

    Examples

    Sample success responses

    JSONformat

    {
  "RequestId": "2D69A58F-345C-4FDE-88E4-BF518948****",
  "TaskId": "task-38aa8e890f45****"
}

    Error codes

    HTTP status codeError codeError messageDescription
    400IllegalCharactersThe parameter contains illegal characters.The parameter contains illegal characters.
    400MissingFileSystemIdFileSystemId is mandatory for this action.-
    400MissingDataFlowIdDataFlowId is mandatory for this action.-
    400InvalidFilesystemVersion.NotSupportThis Api does not support this fileSystem version.This Api does not support this fileSystem version.
    403OperationDenied.InvalidStateThe operation is not permitted when the status is processing.The operation is not permitted when the status is processing.
    403OperationDenied.DependencyViolationThe operation is denied due to dependancy violation.-
    403OperationDenied.DataFlowNotSupportedThe operation is not supported.-
    404InvalidParameter.InvalidFormatThe EntryList format is invalid.-
    404InvalidParameter.SizeTooLargeThe specified EntryList size exceeds 64 KB.-
    404InvalidDataFlow.NotFoundThe specified data flow does not exist.-
    404InvalidTaskAction.NotSupportedThe task action is not supported.-
    404InvalidTaskAction.PermissionDeniedThe task action is not allowed.-
    404InvalidSrcTaskId.NotFoundThe SrcTaskId is not found.-
    404InvalidDataType.NotSupportedThe data type is not supported.-
    404InvalidSrcTaskId.TaskIdInvalidSource task ID is invalid.-
    404InvalidSrcTaskId.TaskIdNotFoundSource task ID is not found.-

    For a list of error codes, visit the Service error codes.

    Change history

    Change timeSummary of changesOperation
    2024-09-09The Error code has changed. The request parameters of the API has changedView Change Details
    2024-02-29The Error code has changed. The request parameters of the API has changedView Change Details