Simple Log Service:Collect Windows event logs

Implementation

To obtain event logs, Windows operating systems provide Windows Event Log API and Event Logging API. Windows Event Log API is an upgraded version of Event Logging API and is provided only in the Windows Vista operating system or later. For more information, see Windows Event Log and Event Logging. Logtail plug-ins automatically select an API based on the operating system to obtain Windows event logs. Windows Event Log API is preferentially selected.

The publish-subscribe model is used to collect Windows event logs. An application or kernel publishes event logs to a specified channel, such as an application, security, or system channel. Logtail uses the required plug-in to call Windows Event Log API or Event Logging API to subscribe to the channel. This way, Logtail can continuously collect event logs and send the logs to Simple Log Service.

Logtail allows you to collect event logs from multiple channels at a time. For example, you can collect event logs from the application and system channels at a time.

View the information about a channel

You can view the information about a channel in the Event Viewer window of your Windows server.

1. Click Start.

2. Search for and click Event Viewer. The Event Viewer window appears.

3. In the left-side navigation pane, expand Windows Logs.

4. View the full name of a channel.

   Right-click a channel below Windows Logs and select Properties. In the window that appears, you can view the full name of the channel. The following channels are provided:

   • Application

   • Security

   • Setup

   • System

5. View the information about a channel.

   Click the channel below Windows Logs. Then, you can view Level, Date and Time, Source, and Event ID of each event.

   In a Logtail configuration, you can filter logs based on the preceding information.

   

Procedure

1. Log on to the Simple Log Service console.

2. In the Import Data section, click Windows Event Logs.

3. Select the project and Logstore. Then, click Next.

4. In the Machine Group Configurations step, configure a machine group.

   1. Configure the Scenario and Installation Environment parameters based on your business requirements.

      Important

      You must configure the Scenario and Installation Environment parameters regardless of whether a machine group is available. The parameter settings affect subsequent configurations.

   2. Make sure that a machine group is displayed in the Applied Server Groups section and click Next.

      Machine group available

      Select a machine group from the Source Machine Group section.

      

      No machine group available

      Click Create Machine Group. In the Create Machine Group panel, configure the parameters. You can set the Machine Group Identifier parameter to IP Address or Custom Identifier. For more information, see Create a custom identifier-based machine group or Create an IP address-based machine group.

      Important

      If you apply a machine group immediately after you create the machine group, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Simple Log Service. To resolve this issue, you can click Retry. If the issue persists, see What do I do if no heartbeat connections are detected on Logtail?

5. In the Configure Data Source step, set the Configuration Name and Plug-in Configuration parameters, and then click Next.

   • inputs is required and is used to configure the data source settings for the Logtail configuration.

     Important

     You can specify only one type of data source in inputs.

   • processors is optional and is used to configure the data processing settings for the Logtail configuration to parse data. You can specify one or more processing methods.

     If your logs cannot be parsed based only on the setting of inputs, you can configure processors in the Plug-in Configuration field to add plug-ins for data processing. For example, you can extract fields, extract log time, mask data, and filter logs. For more information, see Use Logtail plug-ins to process data.

   For example, if you want to collect logs from the Application and System channels, you can add the following elements to the inputs parameter:

   {
       "inputs": [
           {
               "type": "service_wineventlog",
               "detail": {
                   "Name": "Application",
                   "IgnoreOlder": 259200
               }
           },
           {
               "type": "service_wineventlog",
               "detail": {
                   "Name": "System",
                   "IgnoreOlder": 259200
               }
           }
       ]
   }

   Parameter	Type	Required	Description
   type	String	Yes	The type of the data source. Set the value to service_wineventlog.
   Name	String	Yes	The name of the channel from which you want to collect event logs. Default value: Application. This value indicates that event logs are collected from the Application channel. You can view the full name of a channel in the Windows operating system. For more information, see Step 4.
   IgnoreOlder	UINT	No	The time offset. You can configure this parameter to filter logs by event time. Unit: seconds. If the event time of a log is earlier than the start time of collection minus the offset, the log is ignored. The event time indicates the time when a log is generated. Examples: If you set the value to 3600, logs that are generated 1 hour before the start time of collection are ignored. If you set the value to 14400, logs that are generated 4 hours before the start time of collection are ignored. By default, this parameter is left empty, which indicates that logs are not filtered by event time. In this case, all historical event logs on your server are collected. Note This parameter takes effect only when the first time you create a Logtail configuration. Logtail records checkpoints for the collection of event logs. This prevents logs from being repeatedly collected.
   Level	String	No	The event level. You can configure this parameter to filter logs by event level. Default value: information, warning, error, critical. This value indicates that logs of all event levels except verbose are collected. Available levels include information, warning, error, critical, and verbose. You can specify multiple event levels and separate the event levels with commas (,). Note This parameter is available only when Windows Event Log API is used. This parameter is supported only in the Windows Vista operating system or later.
   EventID	String	No	The event ID. You can configure this parameter to filter logs by event ID. You can specify an event ID or a range of event IDs for positive filtering. In this case, the log with the specified event ID or the logs with the specified event IDs are collected. You can also specify an event ID for negative filtering. In this case, the log with the specified event ID is not collected. By default, this parameter is left empty, which indicates that all event logs are collected. Examples: 1-200: Only the event logs whose event IDs are in the range from 1 to 200 are collected. 20: Only the event log whose event ID is 20 is collected. -100: All event logs except the event log whose event ID is 100 are collected. 1-200,-100: Event logs whose event IDs are in the range from 1 to 200 except 100 are collected. You can specify multiple event IDs or event ID ranges and separate them with commas (,). Note This parameter is available only when Windows Event Log API is used. This parameter is supported only in the Windows Vista operating system or later.
   Provider	String array	No	The event source. You can configure this parameter to filter logs by event source. For example, if you set the value to ["App1", "App2"], only event logs whose sources are App1 and App2 are collected. By default, this parameter is left empty, which indicates that event logs of all sources are collected. Note This parameter is available only when Windows Event Log API is used. This parameter is supported only in the Windows Vista operating system or later.
   IgnoreZeroValue	Boolean	No	Some fields in an event log may be empty. You can configure this parameter to filter out empty fields. You can define empty fields based on the data type of the fields. For example, if the data type of an empty field is integer, specify 0 for the field. Default value: false. This value indicates that empty fields are not filtered out.

6. Create indexes and preview data. Then, click Next. By default, full-text indexing is enabled in Simple Log Service. You can also manually create field indexes for the collected logs or click Automatic Index Generation. Then, Simple Log Service generates field indexes. For more information, see Create indexes.

   Important

   If you want to query all fields in logs, we recommend that you use full-text indexes. If you want to query only specific fields, we recommend that you use field indexes. This helps reduce index traffic. If you want to analyze fields, you must create field indexes. You must include a SELECT statement in your query statement for analysis.

7. Click Log Query. You are redirected to the query and analysis page of your Logstore.

   You must wait approximately 1 minute for the indexes to take effect. Then, you can view the collected logs on the Raw Logs tab. For more information, see Query and analyze logs.

Troubleshooting

If no data is displayed on the preview page or query page after logs are collected by using Logtail, you can troubleshoot the errors based on the instructions that are provided in What do I do if errors occur when I use Logtail to collect logs?