Common data structures - Data Lake Formation - Alibaba Cloud Documentation Center

Catalog

Parameter	Type	Example	Description
CatalogId	String	1344371	The catalog ID of the metadatabase. Default value: the user ID of your Alibaba Cloud account.
CreateTime	Integer	1608863962	The time when the catalog was created. The value is in the UNIX timestamp format. Unit: seconds.

ColumnStatistics

ColumnStatisticsList

Parameter	Type	Example	Description
ColumnStatisticsDesc	Object		The description of field statistics.
ColumnStatisticsObjList	Array of ColumnStatisticsObj		The statistics on the field.
Engine	String		The engine.
IsStatsCompliant	Boolean	false	IsStatsCompliant

ColumnStatisticsObj

Parameter	Type	Example	Description
ColumnName	String	col1	The name of the field.
ColumnType	String	int	The data type of the field. Valid values include but are not limited to array, bigint, binary, boolean, char, date, decimal, double, float, int, interval, map, set, smallint, string, struct, timestamp, tinyint, union, and varchar.
ColumnStatisticsData	Object		The statistics on the field.
StatisticsData	String	{"numNulls":0,"numTrues":1,"numFalses":5}	The statistical results of the field.
StatisticsType	String	BOOLEAN_STATS	The type of statistics. Valid values include but are not limited to BOOLEAN_STATS, LONG_STATS, DOUBLE_STATS, STRING_STATS, BINARY_STATS, DECIMAL_STATS, and DATE_STATS.

Database

The information about a metadatabase.

Parameter	Type	Example	Description
CreateTime	Integer	1608707407	The time when the metadatabase was created. The value is in the UNIX timestamp format. Unit: seconds.
Description	String	this is a hive database	The description of the metadatabase.
LocationUri	String	/hdfs/hivedb/data	The storage path of the metadatabase, such as a Hadoop Distributed File System (HDFS) path.
Name	String	database_test	The name of the metadatabase.
OwnerName	String	zhangsan	The owner of the metadatabase.
OwnerType	String	USER	The type of the owner. Valid values: USER, ROLE, and GROUP.
Parameters	Parameters		The key-value pairs (KVPs) that define the parameters and attributes of the metadatabase.
Privileges	PrincipalPrivilegeSet		The permissions on the metadatabase. This parameter is optional.
UpdateTime	Integer	1608707407	The time when the metadatabase was updated. The value is in the UNIX timestamp format. Unit: seconds.

DatabaseInput

The input information about a metadatabase.

Parameter	Type	Example	Description
Description	String	this is a hive database	The description of the metadatabase.
LocationUri	String	/hdfs/hivedb/data	The storage path of the metadatabase, such as an HDFS path.
Name	String	name	The name of the metadatabase. The name can be up to 128 characters in length. The name does not support special characters other than underscores (_).
OwnerName	String	zhangsan	The owner of the metadatabase.
OwnerType	String	USER	The type of the owner. Valid values: USER, ROLE, and GROUP.
Parameters	Parameters		The KVPs that define the parameters and attributes of the metadatabase.
Privileges	PrincipalPrivilegeSet		The permissions on the metadatabase. This parameter is optional.

FieldSchema

Parameter	Type	Example	Description
Comment	String	comment_day	The comments on the field.
Name	String	day	The name of the field.
Parameters	Parameters		The KVPs that define the attributes associated with the field.
Type	String	int	The data type of the field. Valid values include but are not limited to array, bigint, binary, boolean, char, date, decimal, double, float, int, interval, map, set, smallint, string, struct, timestamp, tinyint, union, and varchar.

Function

The information about a function.

Parameter	Type	Example	Description
CatalogId	String	1344371	The catalog ID of the metadatabase. Default value: the user ID of your Alibaba Cloud account.
ClassName	String	com.xxx.xxxxFunction	The name of the Java class to implement the function.
CreateTime	Integer	1608863962	The time when the function was created. The value is in the UNIX timestamp format. Unit: seconds.
DatabaseName	String	database_test	The name of the metadatabase.
FunctionName	String	function1	The name of the function.
FunctionType	String	JAVA	The type of the function. Set the value to JAVA.
OwnerName	String	zhangsan	The owner of the function.
OwnerType	String	USER	The type of the owner. Valid values: USER, ROLE, and GROUP.
ResourceUri	Array of ResourceUri		The resource uniform resource identifiers (URIs) of the function.
UpdateTime	Integer	1608863962	The time when the function was updated. The value is in the UNIX timestamp format. Unit: seconds.

FunctionInput

The input information about a function.

Parameter	Type	Example	Description
ClassName	String	com.xxx.xxxxFunction	The name of the Java class to implement the function.
FunctionName	String	function1	The name of the function.
FunctionType	String	JAVA	The type of the function. Set the value to JAVA.
OwnerName	String	zhangsan	The owner of the function.
OwnerType	String	USER	The type of the owner. Valid values: USER, ROLE, and GROUP.
ResourceUri	Array of ResourceUri		The resource URIs of the function.

ListFieldSchema

The information about table fields.

Parameter	Type	Example	Description
FieldSchema	FieldSchema		The definitions of table fields.

ListResourceUri

The list of resource URIs.

Parameter	Type	Example	Description
ResourceType	String	JAR	The type of resource. Valid values: JAR, FILE, and ARCHIVE.
Uri	String	hdfs:///tmp/jar1.jar	The URI of the resource.

LockObj

Parameter	Type	Example	Description
CatalogId	String	1344371	The catalog ID of the metadatabase. Default value: the user ID of your Alibaba Cloud account.
DatabaseName	String	database_test	The name of the metadatabase.
PartitionName	String	dt	The name of the partition. This parameter is optional.
TableName	String	test_table	The name of the metadata table.

LockStatus

Parameter	Type	Example	Description
LockId	Long	1	The unique ID of the lock.
LockState	String	ACQUIRED	The status of the lock. Valid values: ACQUIRED and NOT_ACQUIRED.

Order

Parameter	Type	Example	Description
Col	String	col1	The name of the field.
Order	Integer	1	A value of 1 indicates the ascending order, and a value of 0 indicates the descending order.

Parameters

Parameter	Type	Example	Description

Partition

Parameter	Type	Example	Description
CreateTime	Integer	1608730999	The time when the partition was created. The value is in the UNIX timestamp format. Unit: seconds.
DatabaseName	String	database_test	The name of the metadatabase.
LastAccessTime	Integer	1608730999	The time when the partition was last accessed. The value is in the UNIX timestamp format. Unit: seconds.
LastAnalyzedTime	Integer	1608730999	The time when the last statistical operation was performed. The value is in the UNIX timestamp format. Unit: seconds.
Parameters	Map		The KVPs that are related to the partition.
Privileges	PrincipalPrivilegeSet		The configurations of permissions. This parameter can be left empty.
Sd	StorageDescriptor		The descriptor of the storage.
TableName	String	test_table_20201223	The name of the metadata table.
Values	Array of String	part1	The values in the partition.

PartitionInput

Partition

Parameter	Type	Example	Description
DatabaseName	String	database_test	The name of the metadatabase.
LastAccessTime	Integer	1608730999	The time when the partition was last accessed. The value is in the UNIX timestamp format. Unit: seconds.
LastAnalyzedTime	Integer	1608730999	The time when the last statistical operation was performed. The value is in the UNIX timestamp format. Unit: seconds.
Parameters	Map		The KVPs that are related to the partition.
Privileges	PrincipalPrivilegeSet		The configurations of permissions. This parameter can be left empty.
Sd	StorageDescriptor		The descriptor of the storage.
TableName	String	test_table_20201223	The name of the metadata table.
Values	Array of String	part1	The values in the partition.

PartitionValueList

Parameter	Type	Example	Description
Values	ListString		The values in the partition.

PrincipalPrivilegeSet

PrincipalPrivilegeSet Object

Parameter	Type	Description
GroupPrivileges	Privileges	GroupPrivileges
RolePrivileges	Privileges	RolePrivileges
UserPrivileges	Privileges	UserPrivileges

PrivilegeGrantInfo

PrivilegeGrantInfo Object

Parameter	Type	Example	Description
CreateTime	Integer	1606226844	The time when the permission was granted. The value is in the UNIX timestamp format. Unit: seconds.
GrantOption	Boolean	true	GrantOption
Grantor	String	Grantor	Grantor
GrantorType	String	GrantorType	GrantorType
Privilege	String	Privilege	Privilege

Privileges

Privileges Object

Parameter	Type	Example	Description

ResourceUri

Parameter	Type	Example	Description
ResourceType	String	JAR	The type of resource. Valid values: JAR, FILE, and ARCHIVE.
Uri	String	hdfs:///tmp/jar1.jar	The URI of the resource.

SerDeInfo

Parameter	Type	Example	Description
Name	String	name	The name of Serde.
SerializationLib	String	org.apache.hadoop.hive.serde2.OpenCSVSerde	The implementation class of Serde.

SkewedInfo

Parameter	Type	Example	Description
SkewedColNames	Array of String	col1	The names of the skewed fields.
SkewedColValueLocationMaps	Map		The mappings between skewed values and fields.
SkewedColValues	Array of Array		The values of the skewed fields.

StorageDescriptor

Parameter	Type	Example	Description
BucketCols	ListString		The array of bucket fields.
Cols	Array of FieldSchema		All the fields in a table.
Compressed	Boolean	false	Specifies whether to enable compression.
InputFormat	String	org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat	The input format. For more information, see the description of the CreateTable operation.
Location	String	file:///tmp/table	The physical location of the table.
NumBuckets	Integer	10	The number of buckets. This parameter is required if the BucketCols parameter is specified.
OutputFormat	String	org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat	The output format. For more information, see the description of the CreateTable operation.
Parameters	Map		The mapping array of KVPs.
SerDeInfo	SerDeInfo		The information about serialization and deserialization.
SkewedInfo	SkewedInfo		The duplicate values or skewed values in a field.
SortCols	Array of Order		The sort fields.
StoredAsSubDirectories	Boolean	false	Specifies whether table data is stored in a subdirectory.

Table

Parameter	Type	Example	Description
Cascade	Boolean	false	Specifies whether to perform cascade deletion.
CreateTime	Integer	1608707407	The time when the table was created. The value is in the UNIX timestamp format. Unit: seconds.
CreatedBy	String	zhangsan	The name of the user who created the table.
DatabaseName	String	database_test	The name of the metadatabase.
LastAccessTime	Integer	1608707407	The time when the table was last accessed. The value is in the UNIX timestamp format. Unit: seconds.
LastAnalyzedTime	Integer	1608707407	The time when the last statistical operation was performed on columns. The value is in the UNIX timestamp format. Unit: seconds.
Owner	String	zhangsan	The owner of the table.
OwnerType	String	USER	The type of the owner. Valid values: USER, ROLE, and GROUP.
Parameters	Parameters		The KVPs that define the attributes related to the table.
PartitionKeys	ListFieldSchema		The partition key columns of the partitioned table.
Privileges	PrincipalPrivilegeSet		The configurations of permissions.
Retention	Integer	30000	The retention time of the table.
RewriteEnabled	Boolean	true	Specifies whether the view can be used to rewrite a query.
Sd	StorageDescriptor		The descriptor of the storage, which contains physical storage information about this table.
TableName	String	table_name	The name of the table.
TableType	String	MANAGED_TABLE	The type of the metadata table, such as MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW, INDEX_TABLE, or MATERIALIZED_VIEW.
Temporary	Boolean	false	Specifies whether the table is a temporary table.
UpdateTime	Integer	1608707407	The time when the table was updated. The value is in the UNIX timestamp format. Unit: seconds.
ViewExpandedText	String	''	The expanded text of the view.
ViewOriginalText	String	''	The original text of the view.

TableError

Parameter	Type	Example	Description
ErrorDetail	Object		The information about the error.
TableName	String		The name of the table in which the error occurs.

TableInput

Table

Parameter	Type	Example	Description
Cascade	Boolean	false	Specifies whether to perform cascade deletion.
CreatedBy	String	zhangsan	The name of the user who created the table.
DatabaseName	String	database_test	The name of the metadatabase.
LastAccessTime	Integer	1608707407	The time when the table was last accessed. The value is in the UNIX timestamp format. Unit: seconds.
LastAnalyzedTime	Integer	1608707407	The time when the last statistical operation was performed on columns. The value is in the UNIX timestamp format. Unit: seconds.
Owner	String	zhangsan	The owner of the table.
OwnerType	String	USER	The type of the owner. Valid values: USER, ROLE, and GROUP.
Parameters	Parameters		The KVPs that define the attributes related to the table.
PartitionKeys	ListFieldSchema		The partition key columns of the partitioned table. Only the partition keys of the basic types are supported. If you do not specify a partition key, you must leave the PartitionKeys parameter empty.
Privileges	PrincipalPrivilegeSet		The configurations of permissions. This parameter can be left empty.
Retention	Integer	30000	The retention time of the table.
RewriteEnabled	Boolean	true	Specifies whether the view can be used to rewrite a query.
Sd	StorageDescriptor		The descriptor of the storage, which contains physical storage information about this table.
TableName	String	table_name	The name of the table.
TableType	String	MANAGED_TABLE	The type of the metadata table, such as MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW, INDEX_TABLE, or MATERIALIZED_VIEW.
Temporary	Boolean	false	Specifies whether the table is a temporary table.
ViewExpandedText	String	''	The expanded text of the view.
ViewOriginalText	String	''	The original text of the view.

TablePartitionColumnStatisticsRequest

Parameter	Type	Description
CatalogId	String	The catalog ID of the metadatabase. Default value: the user ID of your Alibaba Cloud account.
ColumnNames	ListString	The names of fields.
DatabaseName	String	The name of the metadatabase.
PartitionNames	ListString	The names of partitions.
TableName	String	The name of the metadata table.

TableVersion

The schema of a table version.

Parameter	Type	Example	Description
Table	Table		The details about the metadata table.
VersionId	Integer		The number of the version.

UpdateTablePartitionColumnStatisticsRequest

UpdateTableColumnStatistics

Parameter	Type	Example	Description
CatalogId	String	1344371	The catalog ID of the metadatabase. Default value: the user ID of your Alibaba Cloud account.
ColumnStatisticsList	Array of ColumnStatistics		The list of field statistics.
DatabaseName	String	database_test	The name of the metadatabase.
Engine	String		The engine.
TableName	String		The name of the metadata table.
ValidWriteIdList	String		This parameter is not supported and can be left empty.
WriteId	String		This parameter is not supported and can be left empty.
IsStatsCompliant	Boolean	true	IsStatsCompliant