All Products
Search

This Product

    Simple Log Service:OSSExportConfiguration

    Document Center

    Simple Log Service:OSSExportConfiguration

    Last Updated:Nov 28, 2024
    ParameterTypeDescriptionExample
    object

    OSSExportConfiguration

    logstorestring

    The name of the Logstore.

    		logstore-demo
    roleArnstring

    The Alibaba Cloud Resource Name (ARN) of the Resource Access Management (RAM) role that is used to read data from Simple Log Service.

    		acs:ram::123456789:role/aliyunlogdefaultrole
    sinkobject

    The configurations of the Object Storage Service (OSS) data shipping job.

    endpointstring
    • The endpoint that is used to access OSS. You can specify only an internal OSS endpoint for the region where the Simple Log Service project resides. The value is in the http://+OSS endpoint format. For more information, see OSS regions and endpoints.
    • The endpoint that is used to access OSS-HDFS. You can specify only an internal OSS-HDFS endpoint for the region where the Simple Log Service project resides.
    		http://xxxxxxxx
    bucketstring

    The OSS bucket.

    		test-bucket
    prefixstring

    The prefix of the OSS object.

    		prefixxxx/
    suffixstring

    The suffix of the OSS object.

    		.json
    roleArnstring

    The ARN of the RAM role that is used to write data to OSS.

    		acs:ram::xxxxxxx
    pathFormatstring

    The partition format. For more information, see Partition formats.

    		%Y_%m_%d/good/bad
    pathFormatTypestring

    The partition format type.

    		only support time
    timeZonestring

    The time zone. For more information, see Time zones.

    		+0800
    contentTypestring

    The storage format of the OSS object. Valid values: json, parquet, csv, and orc.

    		json/parquet/csv/orc
    contentDetailobject

    The details of the OSS object. Note: The value of this parameter is in the JSON format and varies based on the value of contentType.

    any

    If the value of contentType is json, specify contentDetail based on the following description:

    • If tag fields can be shipped, set enableTag to true. Example: {"enableTag": true}.
    • If tag fields cannot be shipped, set enableTag to false. Example: {"enableTag": false}.

    If the value of contentType is csv, specify contentDetail based on the following description:

    • columns specifies the keys of fields in logs of the source Logstore. Example: ["bucket", "bucket_location", "cdn_in", "cdn_out"].
    • delimiter specifies the delimiters that are used. Valid values: ,, |, \t, and space.
    • header specifies whether to retain the headers of the OSS object. Valid values: true and false.
    • lineFeed specifies the line feeds that are used. Valid values: \t, \n, and space.
    • nullI specifies the data that is shipped when the specified field name does not exist. Example: -.
    • quote specifies the escape characters that are used. Valid values: ", ', and space.

    If the value of contentType is parquet, specify contentDetail based on the following description:

    • columns specifies the keys of fields in logs of the source Logstore. You must also specify the data types of the keys. Example: {'columns': [{'name': "bucket", 'type': "string"}, {'name': "bucket_location", 'type': "string"}, {'name': "cdn_in", 'type': "string"},{'name': "cdn_out", 'type': "string"}].

    If the value of contentType is orc, specify contentDetail based on the following description:

    • columns specifies the keys of fields in logs of the source Logstore. You must also specify the data types of the keys. Example: {'columns': [{'name': "bucket", 'type': "string"}, {'name': "bucket_location", 'type': "string"}, {'name': "cdn_in", 'type': "string"},{'name': "cdn_out", 'type': "string"}].
    		{ "columns": [ "your log key1", "your log key2", "your log key3", ], "delimiter": " ", "header": True, "lineFeed": "\n", "null": "-", "quote": "\"" },
    compressionTypestring

    The compression type. Valid values: snappy, gizp, zstd, and none.

    		snappy/gizp/zstd/none
    bufferIntervallong

    The interval between two data shipping operations. Valid values: 300 to 900. Unit: seconds.

    		300
    bufferSizelong

    The size of the OSS object to which data is shipped. Valid values: 5 to 256. Unit: MB.

    		256
    delaySecdeprecatedlong

    The latency of data shipping.

    Note
    • This parameter is deprecated.
    		123
    delaySecondslong

    The latency of data shipping. The value of this parameter cannot exceed the data retention period of the source Logstore.

    		900
    fromTimelong

    The beginning of the time range to ship data. The value 1 specifies that the data shipping job ships data from the first log in the Logstore.

    		123456789
    toTimelong

    The end of the time range to ship data. The value 0 specifies that the data shipping job continuously ships data until the job is manually stopped.

    		123456789