ALIYUN::EMR::Cluster is used to create an E-MapReduce (EMR) cluster.
ALIYUN::EMR::Cluster supports EMR API of the previous version and is available for clusters of the HADOOP, KAFKA, DRUID, ZOOKEEPER, DATA_SCIENCE, and GATEWAY types. ALIYUN::EMR::Cluster2 supports EMR API of the new version (2021-03-20) and is available for clusters of DATALAKE, OLAP, DATAFLOW, and DATASERVING types. We recommend that you use ALIYUN::EMR::Cluster2 to create clusters of these types.
Syntax
{
"Type": "ALIYUN::EMR::Cluster",
"Properties": {
"SshEnable": Boolean,
"EasEnable": Boolean,
"WhiteListType": String,
"InitCustomHiveMetaDB": Boolean,
"IoOptimized": Boolean,
"HostGroup": List,
"Config": List,
"KeyPairName": String,
"VpcId": String,
"AutoRenew": Boolean,
"RelatedClusterId": String,
"BootstrapAction": List,
"InstanceGeneration": String,
"DepositType": String,
"VSwitchId": String,
"NetType": String,
"UserDefinedEmrEcsRole": String,
"Name": String,
"ClusterType": String,
"ZoneId": String,
"IsOpenPublicIp": Boolean,
"OptionSoftWareList": List,
"Configurations": String,
"MasterPwd": String,
"MachineType": String,
"EmrVer": String,
"SecurityGroupName": String,
"MetaStoreConf": String,
"SecurityGroupId": String,
"LogPath": String,
"Period": Integer,
"HighAvailabilityEnable": Boolean,
"UseCustomHiveMetaDB": Boolean,
"UserInfo": List,
"ChargeType": String,
"MetaStoreType": String,
"AuthorizeContent": String,
"UseLocalMetaDb": Boolean,
"ClickHouseConf": Map,
"ResourceGroupId": String,
"Tags": List
}
}
Properties
Property | Type | Required | Editable | Description | Constraint |
SshEnable | Boolean | No | No | Specifies whether to enable SSH. | Valid values:
|
EasEnable | Boolean | No | No | Specifies whether the cluster is a high-security cluster. | Valid values:
|
WhiteListType | String | No | No | The type of the whitelist. | Valid values:
|
InitCustomHiveMetaDB | Boolean | No | No | A reserved property. You do not need to specify this property. | None. |
IoOptimized | Boolean | No | No | Specifies whether to enable I/O optimization. | Valid values:
|
HostGroup | List | Yes | No | The node groups in the cluster. | For more information, see HostGroup properties. |
Config | List | No | No | Details of the custom configuration items. | For more information, see Config properties. |
KeyPairName | String | No | No | The name of the key pair. | None. |
VpcId | String | No | No | The ID of the virtual private cloud (VPC). | None. |
AutoRenew | Boolean | No | No | Specifies whether to enable auto-renewal for subscription clusters. | Valid values:
|
RelatedClusterId | String | No | No | The ID of the EMR cluster that is associated with the gateway cluster. | This property takes effect when ClusterType is set to GATEWAY. |
BootstrapAction | List | No | No | Details of the bootstrap actions that you want to configure for the cluster. | For more information, see BootstrapAction properties. |
InstanceGeneration | String | No | No | The Elastic Compute Service (ECS) instance family. | None. |
VSwitchId | String | No | No | The ID of the vSwitch. | None. |
NetType | String | Yes | No | The type of the network. | Set the value to VPC. |
UserDefinedEmrEcsRole | String | No | No | The ECS application role that allows internal access from ECS to other Alibaba Cloud services, such as Object Storage Service (OSS). | None. |
Name | String | Yes | Yes | The name of the cluster. | The name must be 1 to 64 characters in length, and can contain letters, digits, hyphens (-), and underscores (_). |
ClusterType | String | Yes | No | The type of the cluster. | Valid values:
|
ZoneId | String | Yes | No | The ID of the zone. | None. |
IsOpenPublicIp | Boolean | No | No | Specifies whether to use public IP addresses. | Valid values:
|
OptionSoftWareList | List | No | No | The list of available software. | None. |
Configurations | String | No | No | A reserved property. You do not need to specify this property. | None. |
MasterPwd | String | No | No | The SSH password that is used to access the master node. | The password must be 8 to 30 characters in length. The password must contain at least three of the following character types: uppercase letters, lowercase letters, digits, and special characters. |
MachineType | String | No | No | The type of the node. | None. |
EmrVer | String | Yes | No | The version of EMR. | None. |
SecurityGroupName | String | No | No | The name of the security group. | If you do not specify SecurityGroupId, the system creates a new security group based on the value of SecurityGroupName. After the cluster is created, you can query the ID of the security group on the cluster details page. The default security group policy is applied to the security group. The default policy allows inbound traffic only on port 22 and outbound traffic on all ports. |
DepositType | String | No | No | The hosting type of the cluster. | None. |
SecurityGroupId | String | No | No | The ID of the security group. | If you want to use an existing security group, the default security group policy is applied to the security group. The default policy allows inbound traffic only on port 22 and outbound traffic on all ports. |
LogPath | String | No | No | The OSS path in which you want to store EMR logs. | None. |
Period | Integer | No | No | The subscription duration of the cluster. | This property must be specified when ChargeType is set to PrePaid. Valid values: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 24, and 36. Unit: month. |
HighAvailabilityEnable | Boolean | No | No | Specifies whether to enable high availability for the cluster. | Valid values:
|
UseCustomHiveMetaDB | Boolean | No | No | A reserved property. You do not need to specify this property. | None. |
UserInfo | List | No | No | The information about the Knox account. | For more information, see UserInfo properties. |
ChargeType | String | Yes | No | The billing method of the cluster. | Valid values:
|
AuthorizeContent | String | No | No | A reserved property. You do not need to specify this property. | None. |
UseLocalMetaDb | Boolean | Yes | No | Specifies whether to use the built-in MySQL database of the cluster as the Hive metadatabase. | Valid values:
|
MetaStoreConf | String | No | No | The metadata configurations. | This property must be specified when MetaStoreType is set to user_rds. Specify MetaStoreConf in the following format: |
MetaStoreType | String | No | No | The type of the metadata. | Valid values:
|
ClickHouseConf | Map | No | No | The configurations of the ApsaraDB for ClickHouse cluster. | None. |
ResourceGroupId | String | No | Yes | The ID of the resource group. | None. |
Tags | List | No | Yes | The tags that you want to add to the cluster. | For more information, see Tags properties. |
Tags syntax
"Tags": [
{
"Value": String,
"Key": String
}
]
Tags properties
Property | Type | Required | Editable | Description | Constraint |
Key | String | Yes | No | The key of the tag. | The tag key must be 1 to 128 characters in length, and cannot contain |
Value | String | No | No | The value of the tag. | The tag value can be up to 128 characters in length, and cannot contain |
HostGroup syntax
"HostGroup": [
{
"Comment": String,
"SysDiskType": String,
"DiskCapacity": Integer,
"NodeCount": Integer,
"ClusterId": String,
"DiskCount": Integer,
"CreateType": String,
"DiskType": String,
"AutoRenew": Boolean,
"HostGroupType": String,
"SysDiskCapacity": Integer,
"VSwitchId": String,
"ChargeType": String,
"Period": Integer,
"HostKeyPairName": String,
"HostPassword": String,
"HostGroupId": String,
"InstanceType": String,
"GpuDriver": String,
"HostGroupName": String
}
]
HostGroup properties
Property | Type | Required | Editable | Description | Constraint |
Comment | String | No | No | A reserved property. You do not need to specify this property. | None. |
SysDiskType | String | Yes | No | The system disk category of the node group. | Valid values:
|
DiskCapacity | Integer | Yes | No | The data disk capacity of the node group. | Unit: GB. |
NodeCount | Integer | Yes | No | The number of nodes in the node group. | None. |
ClusterId | String | No | No | A reserved property. You do not need to specify this property. | None. |
DiskCount | Integer | Yes | No | The number of data disks that you want to attach to the node group. | None. |
CreateType | String | No | No | The mode based on which the node group is created. | Valid values:
|
DiskType | String | Yes | No | The data disk category of the node group. | Valid values:
|
AutoRenew | Boolean | No | No | Specifies whether to enable auto-renewal for subscription clusters. | Valid values:
|
HostGroupType | String | Yes | No | The type of the node group. | Valid values:
|
SysDiskCapacity | Integer | Yes | No | The system disk capacity of the node group. | Unit: GB. |
VSwitchId | String | No | No | The ID of the vSwitch. | None. |
ChargeType | String | Yes | No | The billing method. | Valid values:
|
Period | Integer | No | No | The subscription duration. | This property must be specified when ChargeType is set to PrePaid. Valid values: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 24, and 36. Unit: month. |
HostKeyPairName | String | No | No | The name of the key pair that is used to access the node group. | This property takes effect when ClusterType is set to GATEWAY. |
HostPassword | String | No | No | The password that is used to access the node. | This property takes effect when ClusterType is set to GATEWAY. |
HostGroupId | String | No | No | A reserved property. You do not need to specify this property. | None. |
InstanceType | String | Yes | No | The instance type. | None. |
GpuDriver | String | No | No | The GPU driver. | None. |
HostGroupName | String | No | No | The name of the node group. | None. |
Config syntax
"Config": [
{
"Encrypt": String,
"ConfigKey": String,
"FileName": String,
"ServiceName": String,
"Replace": String,
"ConfigValue": String
}
]
Config properties
Property | Type | Required | Editable | Description | Constraint |
Encrypt | String | No | No | A reserved property. You do not need to specify this property. | None. |
ConfigKey | String | No | No | The name of the custom configuration item. | None. |
FileName | String | No | No | The name of the file that contains the custom configuration item. | None. |
ServiceName | String | No | No | The name of the service to which the custom configuration item belongs. | None. |
Replace | String | No | No | A reserved property. You do not need to specify this property. | None. |
ConfigValue | String | No | No | The value of the custom configuration item. | None. |
BootstrapAction syntax
"BootstrapAction": [
{
"Path": String,
"Name": String,
"Arg": String
}
]
BootstrapAction properties
Property | Type | Required | Editable | Description | Constraint |
Path | String | No | No | The OSS path in which you want to store the bootstrap action script. | None. |
Name | String | No | No | The name of the bootstrap action. | None. |
Arg | String | No | No | The parameter of the bootstrap action. | None. |
UserInfo syntax
"UserInfo": [
{
"UserName": String,
"Password": String,
"UserId": String
}
]
UserInfo properties
Property | Type | Required | Editable | Description | Constraint |
UserName | String | No | No | The username of the Knox account. | None. |
Password | String | No | No | The password of the Knox account. | None. |
UserId | String | No | No | The RAM user ID of the Knox account. | None. |
Return values
Fn::GetAtt
ClusterId: the ID of the cluster.
HostGroups: the node groups in the cluster.
MasterNodePubIps: the public IP addresses of master nodes in the cluster.
MasterNodeInnerIps: the private IP addresses of master nodes in the cluster.