After a CloudMonitor agent parses on-premises logs into specific metrics based on your settings, Hybrid Cloud Monitoring reports the metrics to the specified namespace and visualizes the related monitoring data in charts. You can use a monitoring dashboard in Hybrid Cloud Monitoring to create custom charts. You can also configure Prometheus alert rules to receive alert notifications.
Prerequisites
An alert contact and an alert contact group are created. For more information, see Create an alert contact or alert contact group.
A namespace is created. For more information, see Create a namespace.
Background information
The following figure shows the procedure for using the dashboard feature of Hybrid Cloud Monitoring to monitor on-premises logs.
Step 1: Install a CloudMonitor agent on the host from which on-premises logs are collected
The CloudMonitor agent can be manually or automatically installed on the host from which on-premises logs are collected.
Manually install the CloudMonitor agent
If you want to use the on-premises log monitoring feature of Hybrid Cloud Monitoring to monitor multiple hosts but the CloudMonitor agent fails to be automatically installed on the hosts or the hosts are inaccessible over the Internet, you must manually install the CloudMonitor agent on the hosts. Then, you must create an application group and add the hosts to the application group. This way, you can manage the hosts in a centralized manner.
Manually install the CloudMonitor agent on the host from which on-premises logs are collected.
Make sure that a CloudMonitor agent whose version is V3.5.5 or later is installed on the instances of the selected application group. We recommend that you install a CloudMonitor agent of the latest version. For more information, see Manually install the CloudMonitor agent for C++ on a host.
View the installation result of the CloudMonitor agent.
In the left-side navigation pane, click . On the page that appears, you can view the version and status of the CloudMonitor agent that is installed on your host.
Create an application group for the host from which on-premises logs are collected.
In the left-side navigation pane, choose .
In the upper-left corner of the Application Groups tab, click Create Application Group.
In the Create Application Group panel, set Creation Method to Manually Create, enter a name in the Application Group Name field, turn off Initialize Agent Installation, and then select an alert contact group from the Alert Contact Group drop-down list.
Click OK.
Add the host from which on-premises logs are collected to the application group.
On the Group Resources page of the application group, click Manage Services and Resources.
In the Add/Modify Group Resources panel, select the cloud services and resources that you want to add to the application group.
Click OK.
On the Group Resources page of the application group, you can view the information about the cloud services and instances that you created.
Automatically install the CloudMonitor agent
If you want to use the on-premises log monitoring feature of Hybrid Cloud Monitoring to monitor multiple hosts, you can create an application group and add the hosts to the application group. The CloudMonitor agent can be automatically installed on the hosts.
If you use an application group to automatically install a CloudMonitor agent, the CloudMonitor agent of the latest version is installed on the instances of the application group.
Create an application group for the host from which on-premises logs are collected.
Log on to the CloudMonitor console.
In the left-side navigation pane, choose .
In the upper-left corner of the Application Groups tab, click Create Application Group.
In the Create Application Group panel, set Creation Method to Manually Create, enter a name in the Application Group Name field, turn on Initialize Agent Installation, and then select an alert contact group from the Alert Contact Group drop-down list.
Click OK.
Add the host from which on-premises logs are collected to the application group.
On the Group Resources page of the application group, click Manage Services and Resources.
In the Add/Modify Group Resources panel, select the cloud services and resources that you want to add to the application group.
Click OK.
On the Group Resources page of the application group, you can view the information about the cloud services and instances that you created.
View the installation result of the CloudMonitor agent.
In the left-side navigation pane, click . On the page that appears, you can view the version and status of the CloudMonitor agent that is installed on your host.
Step 2: Create a metric import task for on-premises logs
After you create a metric import task, the CloudMonitor agent parses on-premises logs into specific metrics based on your settings for the metric import task. Then, Hybrid Cloud Monitoring reports the metrics to the specified namespace and visualizes the related monitoring data in charts.
In the left-side navigation pane, choose .
Click the On-premises Log Monitoring tab.
In the upper-left corner of the On-premises Log Monitoring tab, click Metric Data.
On the On-premises Log Monitoring tab, configure the parameters. The following section describes the parameters.
Task Name: Enter the name of the metric import task for on-premises logs.
Application Groups: Select the application group that you created in Step 1: Install a CloudMonitor agent on the host from which on-premises logs are collected.
Hosts: The rule that is used to match hosts from which on-premises logs are collected. Default value: All.
Log Path: The path where on-premises logs are stored.
Windows: The log path must start with a drive letter. Example:C:\UserData\logs\pre.cms.console.aliyun.com.@FUNC__DateFormat__YYYY-MM-DD@.log. You cannot specify wildcard characters in a log path.
Linux: The log path must start with a forward slash (/). Example: /home/logs/pre.cms.console.aliyun.com.@FUNC__DateFormat__YYYY-MM-DD@.log.
Notepre.cms.console.aliyun.com.YYYY-MM-DD.log indicates the log file from which on-premises logs are collected. If you want to exclude the error.log error log file, use
@FUNC__DateFormat__YYYY-MM-DD@to filter the log file whose name contains a date.Log parsing: On-premises logs are parsed into specific metrics. The following parsing modes are supported:
Full Regex Mode
In the Sample Log section, enter a sample log.
Click Confirm. Then, click Generate Regular Expression.
In the Log Extraction Results section, find the key for which you want to configure translation values. Then, click Column Value Translation.
Configure the Column Name and Translation Value parameters and click OK.
Replace the metric value that is parsed from the sample log with another value for display.
Delimiter Mode
In the Sample Log section, enter a sample log.
Click Confirm. Then, click Add Column.
Enter a name for the column in the Column Name field and select the field that you want to parse in the Sample Log section. The system automatically generates values for the Column Value Sampling and Extraction Rule parameters. Then, configure the Translation Value parameter and click OK.
Replace the metric value that is parsed from the sample log with another value for display.
JSON Mode
In the Sample Log section, enter a JSON-formatted sample log.
Click Confirm.
In the Log Extraction Results section, find the key for which you want to configure translation values and click Column Value Translation.
Configure the Translation Value parameter and click OK.
Replace the metric value that is parsed from the sample log with another value for display.
Metric Settings: queries statistics on the metrics that are parsed from the sample log and displays the results in charts.
Namespace: Select a namespace where you want to store on-premises logs.
Tags: classifies metrics.
Click OK.
Step 3: View the monitoring chart of on-premises logs
You can view the monitoring chart of a specified metric of on-premises logs.
On the On-premises Log Monitoring tab, find the metric import task whose charts you want to view and click View Data in the Actions column.
On the View Data page, select a metric.
Step 4: Create custom charts for on-premises logs
You can use the dashboard feature of Hybrid Cloud Monitoring to create a monitoring dashboard for the metrics of on-premises logs and create custom charts.
In the left-side navigation pane, choose .
On the Dashboards page, click View Dashboard.
In the left-side navigation pane of the dashboard, click the
icon. Then, click Add an empty panel. On the Edit Panel page, configure metrics and monitoring charts.
On the Query tab, select the namespace and select metrics.
On the Panel tab, specify a panel title and enter a description. Then, configure visualization and other settings.
In the upper-right corner of the Edit Panel page, click Apply.
The monitoring chart is created.
In the upper-right corner, click the
icon. Then, specify the name of the dashboard and the directory in which the dashboard resides. Click Save.
The dashboard is created.
In the left-side navigation pane of the dashboard, choose
> Manage. Then, click the dashboard name to view the dashboard.
Step 5: Configure an alert rule for on-premises logs
You can configure alert rules for the metrics of on-premises logs. If the specified conditions are met, you can receive alert notifications.
In the left-side navigation pane, choose .
In the upper-left corner of the Namespace page, click Create Alert Rule.
In the Create Alert Rule panel, enter a name in the Rule Name field, select a namespace from the Namespace drop-down list, configure the Alert Expression (PromQL) parameter, and then select an alert contact group from the Alert Contact Group drop-down list.
Click OK.