ALIYUN::NAS::DataFlow - Resource Orchestration Service - Alibaba Cloud Documentation Center

ALIYUN::NAS::DataFlow is used to create a data flow between a Cloud Parallel File Storage (CPFS) file system and an Object Storage Service (OSS) bucket.

Syntax

{
  "Type": "ALIYUN::NAS::DataFlow",
  "Properties": {
    "FsetId": String,
    "SourceStorage": String,
    "Description": String,
    "SourceSecurityType": String,
    "FileSystemId": String,
    "Throughput": Integer,
    "AutoRefreshs": List,
    "AutoRefreshPolicy": String,
    "AutoRefreshInterval": Integer
  }
}

Properties

Property	Type	Required	Editable	Description	Constraint
FsetId	String	Yes	No	The fileset ID.	None.
SourceStorage	String	Yes	No	The access path of the OSS bucket.	Value format: `<storage type>://<path>`. Description of the value format: storage type: Only OSS is supported. path: the name of the OSS bucket. The following limits apply: The name can contain only lowercase letters, digits, and hyphens (-). It must start and end with a lowercase letter or digit. It must be 8 to 128 characters in length. It must be encoded in UTF-8. It cannot start with `http://` or `https://`. Note The OSS bucket must be an existing bucket in the specified region.
Description	String	No	Yes	The description of the data flow.	The following limits apply: The description must be 2 to 128 characters in length. It must start with a letter and cannot start with `http://` or `https://`. It can contain letters, digits, colons (:), underscores (_), and hyphens (-).
SourceSecurityType	String	No	No	The security mechanism type of the OSS bucket.	This property must be specified when the OSS bucket is accessed with a security mechanism. Valid values: Null (default): The OSS bucket can be accessed without a security mechanism. SSL: The OSS bucket is accessed with an SSL certificate.
FileSystemId	String	Yes	No	The ID of the file system.	None.
Throughput	Integer	Yes	Yes	The maximum transmission bandwidth of the data flow.	Unit: MB/s. Valid values: 600 1200 1500 Note The transmission bandwidth of the data flow must be less than the I/O bandwidth of the file system.
AutoRefreshs	List	No	No	The automatic update configurations.	For more information, see AutoRefreshs property.
AutoRefreshPolicy	String	No	No	The automatic update policy.	CPFS imports updated data in the OSS bucket to the CPFS file system based on the policy. Valid values: None (default): CPFS does not automatically import updated data in the OSS bucket to the CPFS file system. You can import the updated data by running a data flow task. ImportChanged: CPFS automatically imports updated data in the OSS bucket to the CPFS file system.
AutoRefreshInterval	Integer	No	No	The interval at which the automatic update operation is performed.	CPFS checks whether updated data exists in the directory at the interval. If updated data exists, CPFS automatically imports the updated data. Unit: minutes. Valid values: 5 to 525600. Default value: 10.

AutoRefreshs syntax

"AutoRefreshs": [
  {
    "RefreshPath": String
  }
]

AutoRefreshs property

Property

Type

Required

Editable

Description

Constraint

RefreshPath

String

Yes

The automatic update directory.

CPFS registers the data change events of the OSS bucket and checks whether updated data exists in the directory. If updated data exists, CPFS automatically imports the updated data.

This property is empty by default. If you leave this property empty, CPFS does not automatically import updated data in the OSS bucket to the CPFS file system. You must import the updated data by running a manual task.

The following limits apply:

The directory must be 2 to 1,024 characters in length.
It must be encoded in UTF-8.
It must start and end with a forward slash (/).
The directory must be an existing directory in CPFS and must belong to the fileset directory of the data flow.

Return values

Fn::GetAtt

FileSystemId: the ID of the file system.
DataFlowId: the ID of the data flow.

Examples

YAML format

ROSTemplateFormatVersion: '2015-09-01'
Parameters:
  FileSystemId:
    Default: cpfs-03c125afccb****
    Type: String
  FsetId:
    Default: fset-1902718ea0ae****
    Type: String
  FileSystemPath:
    Default: "/rostestfilesystempath/"
    Type: String
  SourceStorage:
    Default: oss://cpfs-data-flow-source-test-****
    Type: String
Resources:
  DataFlow:
    Type: ALIYUN::NAS::DataFlow
    Properties:
      SourceSecurityType: SSL
      Description: ros_test_data_flow
      AutoRefreshPolicy: ImportChanged
      FileSystemId:
        Ref: FileSystemId
      FsetId:
        Ref: FsetId
      Throughput: 600
      AutoRefreshs:
      - RefreshPath:
          Ref: FileSystemPath
      AutoRefreshInterval: 10
      SourceStorage:
        Ref: SourceStorage
Outputs:
  DataFlowId:
    Value:
      Fn::GetAtt:
      - DataFlow
      - DataFlowId
  FileSystemId:
    Value:
      Fn::GetAtt:
      - DataFlow
      - FileSystemId

JSON format

{
  "ROSTemplateFormatVersion": "2015-09-01",
  "Parameters": {
    "FileSystemId": {
      "Default": "cpfs-03c125afccb****",
      "Type": "String"
    },
    "FsetId": {
      "Default": "fset-1902718ea0ae****",
      "Type": "String"
    },
    "FileSystemPath": {
      "Default": "/rostestfilesystempath/",
      "Type": "String"
    },
    "SourceStorage": {
      "Default": "oss://cpfs-data-flow-source-test-****",
      "Type": "String"
    }
  },
  "Resources": {
    "DataFlow": {
      "Type": "ALIYUN::NAS::DataFlow",
      "Properties": {
        "SourceSecurityType": "SSL",
        "Description": "ros_test_data_flow",
        "AutoRefreshPolicy": "ImportChanged",
        "FileSystemId": {
          "Ref": "FileSystemId"
        },
        "FsetId": {
          "Ref": "FsetId"
        },
        "Throughput": 600,
        "AutoRefreshs": [
          {
            "RefreshPath": {
              "Ref": "FileSystemPath"
            }
          }
        ],
        "AutoRefreshInterval": 10,
        "SourceStorage": {
          "Ref": "SourceStorage"
        }
      }
    }
  },
  "Outputs": {
    "DataFlowId": {
      "Value": {
        "Fn::GetAtt": [
          "DataFlow",
          "DataFlowId"
        ]
      }
    },
    "FileSystemId": {
      "Value": {
        "Fn::GetAtt": [
          "DataFlow",
          "FileSystemId"
        ]
      }
    }
  }
}