Configure log collection for an EAS resource group - Platform For AI

You can configure log collection for a resource group in Platform for AI (PAI). After the configuration is complete, the system collects the logs generated by Elastic Algorithm Service (EAS) services that are deployed in the resource group and stores the logs in your Simple Log Service Logstore. The logs include standard outputs or custom files. This topic describes how to configure log collection for a public or dedicated resource group.

Prerequisites

Simple Log Service is activated, and a project and a Logstore are created. For more information, see Getting Started.

Step 1: Create a machine group for a resource group

Create a machine group for a public resource group

You need to manually create a machine group for a public resource group.

Log on to the Simple Log Service console.
In the Projects section, click the project that you want to manage.
In the left-side navigation pane, choose Resources > Machine Groups.
On the right side of Machine Groups, select > Create Machine Group.
Create a machine group by using a custom identifier.
Important
The custom identifier dedicated to EAS is eas-log-group-{region_id}. For example, the custom identifier of the machine group for EAS in the China (Zhangjiakou) region is eas-log-group-cn-zhangjiakou.
After you deploy a service, go to the Machine Group Configurations page of the machine group. In the Machine Group Status section, view the heartbeat status of service instances. A value of OK indicates that the machine group works as expected.
Note
If no services are deployed, the instance list is empty.

Create a machine group for a dedicated resource group

To create a machine group for a dedicated resource group, you need to turn on Configure SLS on the group details page in the PAI console.

Log on to the PAI console. Select a region on the top of the page. Then, select the desired workspace and click Enter Elastic Algorithm Service (EAS).
On the Resource Group tab, click the name of the dedicated resource group that you want to manage. The details page of the dedicated resource group appears.
On the details page of the dedicated resource group, click Configure SLS.

In the Configure Log Service Settings dialog box, configure SLSProject and LogStore and click OK. The following table describes the parameters.

Parameter	Description
SLSProject	Log Service projects are used to isolate and manage resources in Simple Log Service. If no project is available in the drop-down list, you can click Create SLSProject to create one. For more information, see Create a project.
LogStore	Simple Log Service Logstores are used to collect, store, and query log data. If no Logstore is available in the drop-down list, you can click Create LogStore to create one. For more information, see Create a Logstore.

After you configure the preceding parameters, a Simple Log Service machine group whose name is in the eas-sls-{resource-id} format is automatically created. Sample name: eas-sls-eas-r-9u2lq6ij1pk5yvvh****. resource-id indicates the ID of the resource group. For information about how to obtain ID, see Manage dedicated resource groups.

EAS automatically creates a Logtail configuration in the specified Logstore to collect logs generated by EAS services.

Step 2: Create a custom Logtail configuration

You can create a custom Logtail configuration to collect the desired logs. The following sections describe two configuration items that are frequently used. For information about other complex configuration items, see Collect text logs from servers.

Collect container standard outputs

Log on to the Simple Log Service console.
In the Import Data dialog box, click Kubernetes - Standard Output.
Select the project and Logstore that you want to manage, and click Next.
Click Use Existing Machine Groups, select the machine group that you created in Step 1, and then click Next.
(Optional) In scenarios where services are deployed by using a custom image and you do not want EAS logs to be collected, enable Container Filtering and add a K8s Container Name Regular Matching configuration in the Input Configurations section. Enter ^(easworker)$ in the field of the K8s Container Name Regular Matching configuration.
Click Next on this page and the subsequent wizard page. Then, the configuration is completed.

Collect container file logs

In the Import Data dialog box, click Kubernetes - Object.
Select the project and Logstore that you want to manage, and click Next.
Click Use Existing Machine Groups, select the machine group that you created in Step 1, and then click Next.
On the Logtail Configuration wizard page, configure the parameters and click Next.
To collect files, you must specify the File Path parameter (the address of the files that you want to collect) in the Input Configurations section. For more information about other parameter settings, see Collect text logs from Kubernetes containers in DaemonSet mode.
Click Next on this page and the subsequent wizard page. Then, the configuration is completed.

References

After you configure log collection for the resource group, you can view the logs in the specified Logstore. For more information, see Query and analyze logs.
For information about how to access the Internet and other cloud services that allow access only from specific IP addresses, see Configure Internet access and a whitelist.