All Products
Search

This Product

    Simple Log Service:OSSIngestionConfigurationSource

    Document Center

    Simple Log Service:OSSIngestionConfigurationSource

    Last Updated:Nov 28, 2024
    ParameterTypeDescriptionExample
    object

    The configurations of the Object Storage Service (OSS) data import job.

    endpointstring

    The OSS endpoint.

    		oss-cn-hangzhou.aliyuncs.com
    bucketstring

    The OSS bucket.

    		ossbucket
    compressionCodecstring

    The compression type.

    		none,snappy,gzip
    encodingstring

    The encoding type.

    		UTF-8
    formatobject

    The format.

    any

    The data format of the OSS object. Valid values:

    • Single-line text log: {"type":"Line"}
    • CSV:
    {
  "type": "CSV",
  "fieldDelimiter": ",",   // The delimiter.
  "quoteChar": "\"",      // The quotation mark.
  "escapeChar": "\\",    // The escape character.
  "firstRowAsHeader": true,    // Specifies whether the first line is used as the field name.
  "maxLines": 1,    // The maximum number of log lines.
  "skipLeadingRows": 0   // The number of skipped lines.
}
    • Single-line JSON: {"type": "JSON"}
    • Multi-line text log:
    {
  "type": "Multiline",
  "match": "after", // The position at which the regular expression is used to match data.
  "pattern": "\\d+", // The regular expression.
  "maxLines": 10 // The maximum number of lines.
}
    • ORC: {"type": "ORC"}
    • Parquet: {"type": "Parquet"}
    • Alibaba Cloud OSS access log: {"type": "ossAccessLog"}
    • Alibaba Cloud CDN download log: {"type": "cdnDownloadedLog"}
    		{"type": "line"}
    intervalstring

    The interval at which the system checks for new files.

    		never
    patternstring

    The regular expression that is used to filter OSS objects by directory.

    		.*
    prefixstring

    The directory of the OSS objects.

    		prefix
    restoreObjectEnabledboolean

    Specifies whether to import OSS objects of the Archive class.

    		true
    startTimelong

    The point in time after which modified OSS objects are imported.

    		1714274081
    endTimelong

    The point in time before which modified OSS objects are imported.

    		1714360481
    timeFieldstring

    The time field to extract.

    		__time__
    timeFormatstring

    The format of the time field.

    		yyyy-MM-dd HH:mm:ss
    timePatternstring

    The regular expression that is used to extract time.

    		[0-9]{0,2}\/[0-9a-zA-Z]+\/[0-9:,]+
    timeZonestring

    The time zone of the time field.

    		GMT+08:00
    useMetaIndexboolean

    Specifies whether to enable OSS metadata indexing.

    		false
    roleARNstring

    The Alibaba Cloud Resource Name (ARN) of the role.

    		acs:ram::12345:role/aliyunlogdefaultrole