Result fields - Simple Log Service - Alibaba Cloud Documentation Center

The results of Intelligent Anomaly Analysis are stored in a Logstore named internal-ml-log. This topic describes the fields in the results.

Common tag fields

The results for different types of tasks include the following common fields:

Note You can query the results of a task based on the __tag__:__job_name__ and __tag__:__schedule_id__ fields.

__tag__:__apply_time__:1638414250
__tag__:__batch_id__:a8343****5b0fd
__tag__:__data_type__:anomaly_detect
__tag__:__instance_name__:29030-****7bcdd
__tag__:__job_name__:etl-1637****3966-398245
__tag__:__model_name__:d52b5****c45397
__tag__:__region__:chengdu
__tag__:__schedule_id__:2457f****ebcdd

Field	Description
__tag__:__apply_time__	The time that is required by a model to inspect a batch of data. Unit: seconds.
__tag__:__batch_id__	The ID of a batch. Data in the same batch is identified by the same batch ID.
__tag__:__data_type__	The type of data. job_statistic: the statistical runtime data of a task. job_progress: the output data of entity inspection progress. anomaly_detect: the result data of anomalies. detection_process: the detection result data of a model training task. eval_report: the result data of each entity validation set after a model training task is complete.
__tag__:__instance_name__	The name of the instance that is created for a task. The name consists of a project ID and a schedule ID. Each task is associated with an instance name on the backend server.
__tag__:__job_name__	The name of a task. The name of each task in a project must be unique.
__tag__:__model_name__	The name of a model. A model is created for each entity in a task. Each model is associated with a time series entity.
__tag__:__region__	The region of a task.
__tag__:__schedule_id__	The ID of the instance that is created for a task. Each task is associated with an instance ID on the backend server.

Intelligent inspection (model training)

The type of a log varies based on the value of the __tag__:__data_type__ field.

Statistical runtime data of a task

If the value of the __tag__:__data_type__ field in the result data of your model training task is job_statistic, the data is the statistical runtime data of the task.

Field	Description
meta	The project and Logstore to which the data source of the model training task belongs. The value is JSON-formatted data.
project_name	The project to which the data source of the model training task belongs.
logstore_name	The Logstore to which the data source of the model training task belongs.
result	The result content. The value is JSON-formatted data.
event_msg	The progress of the model training task at the specified timestamp.
occ_time	The timestamp that corresponds to the progress of the model training task.
tips	The overview of the progress for the model training task. For example, the model is stored.

Detection result data of a model training task

If the value of the __tag__:__data_type__ field in the result data of your model training task is detection_process, the data is the detection result data of the task.

Field	Description
meta	The project and Logstore to which the data source of the model training task belongs. The value is JSON-formatted data.
project_name	The project to which the data source of the model training task belongs.
logstore_name	The Logstore to which the data source of the model training task belongs.
result	The result content. The value is JSON-formatted data.
dim_name	A feature of an entity.
score	The anomaly score for the feature of an entity at a specific point in time.
value	The value size for the feature of an entity at a specific point in time.
is_train_step	Indicates whether the point belongs to the training set.

Result data of a validation set

If the value of the __tag__:__data_type__ field in the result data of your model training task is eval_report, the data is the result data of each entity validation set after the task is complete.

Field	Description
entity	The entity for which the model is created. The value is a key-value pair.
meta	The project and Logstore to which the data source of the model training task belongs. The value is JSON-formatted data.
project_name	The project to which the data source of the model training task belongs.
logstore_name	The Logstore to which the data source of the model training task belongs.
result	The result content. The value is JSON-formatted data.
evaluation_metrics.auc	The AUC of the validation set. The AUC is calculated by the supervision model that is trained for the entity.
evaluation_metrics.macro_f1	The macro-averaged F1 score of the validation set. The macro-averaged F1 scoreis calculated by the supervision model that is trained for the entity.
evaluation_metrics.precision	The precision of the validation set. The precision is calculated by the supervision model that is trained for the entity.
evaluation_metrics.recall	The recall of the validation set. The recall is calculated by the supervision model that is trained for the entity.
time_config.training_start_time	The start time of model training for the entity. Unit: seconds.
time_config.training_stop_time	The end time of model training for the entity. Unit: seconds.
time_config.validation_end_time	The end time of model validation for the entity. Unit: seconds.
time_config.predict_time	The duration of model verification for the entity. Unit: seconds.
time_config.train_time	The duration of model training for the entity. Unit: seconds.
statistic.train_data_meta.train_anomaly_num	The number of anomaly points in the training set for the entity.
statistic.train_data_meta.train_data_length	The length of the training set for the entity.
statistic.evaluation_data_meta.evaluation_anomaly_num	The number of anomalies of the validation set for the entity.
statistic.evaluation_data_meta.evaluation_data_length	The length of the validation set for the entity.

Intelligent inspection (real-time inspection)

The type of a log varies based on the value of the __tag__:__data_type__ field.

Statistical runtime data of a task

If the value of the __tag__:__data_type__ field in the result data of your real-time inspection task is job_statistic, the data is the statistical runtime data of the task.

{
  "__tag__:__job_name__": "etl-1637133966-398245",
  "__tag__:__region__": "chengdu",
  "__tag__:__data_type__": "job_statistic",
  "__tag__:__apply_time__": "1638415928",
  "__tag__:__instance_name__": "29030-2457fbbd724de9421da8c73d37debcdd",
  "result": {
    "maxEntity": {
      "host": "machine_001",
      "ip": "192.0.2.1"
    },
    "maxTime": 1638415994,
    "minEntity": {
      "host": "machine_001",
      "ip": "192.0.2.1"
    },
    "minTime": 1638415994,
    "nTotalEntity": 1
  }
}

Field	Description
result	The result item. The value is JSON-formatted data.
maxEntity	The information about the entity at the point in time that is the closest to the point in time of the current data consumption.
maxTime	The point in time of the entity that is the closest to the current data consumption.
nTotalEntity	The number of entities that are detected in the current task.

Output data of entity inspection progress

If the value of the __tag__:__data_type__ field in the result data of your real-time inspection task is job_progress, the data is the output data of entity inspection progress. If a log contains the output data of entity inspection progress, you can determine whether errors occur. For example, you can determine whether a new entity appears or whether an existing entity does not have data.

{
  "__tag__:__job_name__": "etl-1637133966-398245",
  "__tag__:__region__": "chengdu",
  "__tag__:__data_type__": "job_progress",
  "__tag__:__apply_time__": "1638415883",
  "__tag__:__instance_name__": "29030-2457fbbd724de9421da8c73d37debcdd",
  "result": {
    "new_entity": false,
    "recently_arrived_time": 1638415994
  },
  "meta": {
    "logstore_name": "machine_monitor",
    "project_name": "sls-ml-demo"
  },
  "entity": {
    "host": "machine_001",
    "ip": "192.0.2.1"
  }
}

Field	Description
meta	The project and Logstore of the current task. The value is JSON-formatted data.
project_name	The project to which the data source of the real-time inspection task belongs.
logstore_name	The Logstore to which the data source of the real-time inspection task belongs.
result	The result item. The value is JSON-formatted data.
new_entity	Indicates whether a new entity appears.
recently_arrived_time	The timestamp of the last valid data record in the current entity, which is specified by the entity field.
entity	The information about an entity. The information is of the dictionary data type.

Result data of anomalies

If the value of the __tag__:__data_type__ field in the result data of your real-time inspection task is anomaly_detect, the data is the result data of anomalies.

{
  "__time__": 1638416474,
  "__tag__:__batch_id__": "a5870979816fc507cbeebc6b1133af0a",
  "__tag__:__schedule_id__": "2457fbbd724de9421da8c73d37debcdd",
  "__tag__:__apply_time__": "1638416291",
  "__tag__:__job_name__": "etl-1637133966-398245",
  "__tag__:__model_name__": "d52b59a6bfb3adcf2ee62a5064c45397",
  "__tag__:__data_type__": "anomaly_detect",
  "__tag__:__region__": "chengdu",
  "__tag__:__instance_name__": "29030-2457fbbd724de9421da8c73d37debcdd",
  "result": {
    "anomaly_type": "None",
    "dim_name": "value",
    "is_anomaly": false,
    "score": 0,
    "value": "0.780000"
  },
  "meta": {
    "logstore_name": "machine_monitor",
    "project_name": "sls-ml-demo"
  },
  "entity": {
    "host": "machine_001",
    "ip": "192.0.2.1"
  }
}

Field	Description
entity	The entity item. The value is JSON-formatted data and is obtained from the source data. The value is used to identify an entity.
meta	The configuration item. The value is JSON-formatted data and is obtained from the configuration information about an intelligent inspection task.
project_name	The project to which the Logstore belongs.
logstore_name	The Logstore to which the data source belongs.
result	The result item. The value indicates the inspection result of data at each point in time.
dim_name	The name of the dimension in which the generated inspection result is presented. The name is obtained from the source data. The value of the result field is presented only in a single dimension regardless of whether one or more dimensions are specified.
value	The value of the generated inspection result in the specified dimension. The value is obtained from the source data. The dimension is specified by the result.dim_name parameter.
score	The anomaly score. Valid values: [0,1]. A higher score indicates a higher degree of anomaly.
is_anomaly	Indicates whether an anomaly is considered true. If the value of the result.score field is greater than 0.5, the anomaly is considered true. If the value of the result.score field is greater than 0.75, the anomaly is considered true, and an alert is triggered.
anomaly_type	The anomaly type. A model preliminarily classifies an anomaly into the following types: Stab, Shrift, Variance, Lack, and OverThreshold. For more information, see Anomaly types.

Text analysis

The results of a text analysis task include the common tag fields and the following common fields.

Field	Description
algo_type	The algorithm type.
result_type	The result type, which is of the JSON data type.
result	The result content, which is of the JSON data type. The value of the result field varies based on the value of the result_type field.
meta	The metadata. The value is JSON-formatted data.
project_name	The project to which the Logstore belongs.
logstore_name	The Logstore to which the data source belongs.
topic	The log topic of the data source.
query	The method that is used to pull data. For example, a consumer group can be used to pull data.
win_size	The length of a time window.
version	The algorithm version.

The value of the result field varies based on the value of the result_type field. The following sections describe the result fields.

cluster_info specified for the result_type field

If the value of the result_type field is cluster_info, the value of the result field includes information about a log category. The following example shows the structure of the result field in this scenario:

"result": {
  "cluster_id": "xxxx",
  "cluster_pattern": "xxxx",
  "cluster_active_age": 120,
  "cluster_alive_age": 150,
  "anomaly_score": 0.1,
  "count": 2,
  "source": []
}

Field	Description
result.cluster_id	The ID of the log category.
result.cluster_pattern	The log template of the log category.
result.cluster_active_age	The number of time windows in which the log category is active. If the logs of a log category are detected in a time window, the log category is considered active in the time window.
result.cluster_alive_age	The number of time windows that are counted from the first time the log category appears to the current time.
result.anomaly_score	The anomaly score of the log category.
result.count	The number of logs that are included in the log category.
result.source	The possible values of variables in the log template.

group_info specified for the result_type field

If the value of the result_type field is group_info, the value of the result field includes information about a log category group. The following example shows the structure of the result field in this scenario:

"result": {
  "group_anomaly_score": 0.1,
  "group_age": 10,
  "group_n_event": 190,
  "group_n_cluster": 10
}

Field	Description
result.group_anomaly_score	The anomaly score of the log category group.
result.group_age	The sequential number of the current time window.
result.group_n_event	The total number of logs in the group in the current time window.
result.group_n_cluster	The total number of log categories in the group in the current time window.

anomaly_info specified for the result_type field

If the value of the result_type field is anomaly_info, the value of the result field includes information about an anomaly event. The following example shows the structure of the result field in this scenario:

"result": {
  "anomaly_id": "xxxx",
  "anomaly_type": "xxxx",
  "value": 0,
  "anomaly_score": 0.0,
  "expect_lower": 0.0,
  "expect_upper": 0.0
}

Field	Description
result.anomaly_id	The log category ID for the anomaly.
result.anomaly_type	The anomaly type.
result.value	The event value. The meaning of the result.value field varies based on the value of the result.anomaly_type field.
result.anomaly_score	The anomaly score.
result.expect_lower	The lower limit of the expected event value, which is specified by the result.value field.
result.expect_upper	The upper limit of the expected event value, which is specified by the result.value field.

Time series forecasting

The results of a time series forecasting task include the common tag fields and the following common fields.

Field	Description
algo_type	The algorithm type. The value is fixed as series_prediction.
result_type	The result type. The value is JSON-formatted data. If a forecasting operation is successful, the value is prediction_ok. If a forecasting operation fails, the value is prediction_error.
result	The result content. The value is JSON-formatted data. The value of the result field varies based on the value of the result_type field.
meta	The metadata. The value is JSON-formatted data.
project_name	The project to which the Logstore belongs.
logstore_name	The Logstore to which the data source belongs.
topic	The log topic of the data source.
version	The algorithm version.

The value of the result field varies based on the value of the result_type field. The following sections describe the result fields.

prediction_ok specified for the result_type field

If the value of the result_type field is prediction_ok, the forecasting operation is successful, and each log includes the forecasting result of a point in the time series. The following example shows the structure of the result field in this scenario:

{
  "entity": "xxxx",
  "metric": "xxxx",
  "time": xxxx,
  "value": "xxxx",
  "expect_value": "xxxx",
  "expect_lower": "xxxx",
  "expect_upper": "xxxx"
}

Field	Description
result.entity	The entity ID of the forecasted time series.
result.metric	The metric in the forecasted time series.
result.time	The timestamp of the current point in the forecasted time series.
result.value	The actual value of the current point in the forecasted time series.
result.expect_value	The forecast value of the current point in the forecasted time series.
result.expect_lower	The forecast lower limit of the current point in the forecasted time series.
result.expect_upper	The forecast upper limit of the current point in the forecasted time series.

prediction_error specified for the result_type field

If the value of the result_type field is prediction_error and the value of the __tag__:__data_type__ field is job_error_message, an error occurs in the forecasting operation. The following example shows the structure of the result field in this scenario:

{
  "entity": "xxxx",
  "metric": "xxxx",
  "error_type": "xxxx",
  "error_msg": "xxxx"
}

Field	Description
result.entity	The entity ID of the forecasted time series. An error occurs in the forecasting operation.
result.metric	The metric of the forecasted time series. An error occurs in the forecasting operation.
result.error_type	The error type.
result.error_msg	The error details.

Drill-down analysis

The results of a drill-down analysis task include the common tag fields and the following common fields.

Field	Description
result	The result content. The value is JSON-formatted data. The value of the result field varies based on the value of the __tag__:__data_type__ field.

The type of a log varies based on the value of the __tag__:__data_type__ field.

Progress information about a drill-down analysis task

If the value of the __tag__:__data_type__ field is job_progress, the value of the result field includes progress information about a drill-down analysis task.

Field	Description
result.from_ts	The start time of the task.
result.to_ts	The end time of the task. The value inf indicates that the task is ongoing.
result.progress	The current progress of the task.
result.message	The status information about the current progress of the task.

Status information about a drill-down analysis task

If the value of the __tag__:__data_type__ field is job_status, the value of the result field includes status information about a drill-down analysis task.

Field	Description
result.from_ts	The start time of the task.
result.to_ts	The end time of the task. The value inf indicates that the task is ongoing.
result.status	The status of the task.
result.message	The status details of the task.

Root causes detected by a drill-down analysis task

If the value of the __tag__:__data_type__ field is root_cause, the value of the result field includes the root causes that are detected by a drill-down analysis task.

Field	Description
result.status	Indicates whether root causes are detected. Valid values: success fail
result.snapshot_time	The point in time of the multi-dimensional time series data that is used for drill-down analysis.
result.elapsed_time	The duration of troubleshooting that is performed on the event to detect root causes.
result.event_info	The event that triggers root cause analysis.
result.root_cause	If the value of the result.status field is success, the value of this field indicates the result of the root cause analysis.
result.reason	If the value of the result.status field is fail, the value of this field indicates the reason why no causes are detected.