Realtime Compute for Apache Flink:View the logs of a historical job

Background information

Prerequisites

• The log archiving feature is enabled for the historical job. To check whether log archiving is enabled for the historical job, perform the following operations: Go to the job details page and click the Configuration tab. In the Logging section, check whether Enabled is displayed in the Log Archiving field. By default, log archiving is enabled.

• Before you view the logs of a historical job in the OSS console or the Simple Log Service console, you have configured parameters to export logs of the historical job to OSS or Simple Log Service. For more information, see Configure parameters to export logs of a deployment.

Limits

• In Ververica Runtime (VVR) 3.X, only VVR 3.0.7 and later allow you to view the logs of the TaskManagers of a historical job.

• In VVR 4.X, only VVR 4.0.11 and later allow you to view the logs of the TaskManagers of a historical job.

Usage notes

• When the log size of a job reaches the specified size, the system automatically archives the logs and periodically clears up the logs whose archiving validity periods expire. After the job is stopped, the archived logs that have not been cleared are saved. These logs will not be automatically cleared. If a job is deleted, the related logs are also deleted.

  Note

  After the new version of Realtime Compute for Apache Flink is released in November 2024, all existing job logs, including the archived logs that are not cleared after a job is stopped, in the management console of Realtime Compute for Apache Flink will be cleared after the archiving validity period expires.

• If a TaskManager or the JobManager is started, logs are written to Simple Log Service or OSS. However, if no TaskManagers are started and the JobManager is also not started, logs are not written to Simple Log Service or OSS.

• If you cannot find the relevant log file on OSS because the JobManager and TaskManagers fail to start, contact Alibaba Cloud technical support.

• Two log files are retained in the configurations of Realtime Compute for Apache Flink. The default size of each of the log files is 5 MB. We recommend that you do not modify the file size. If the file size does not meet your business requirements, you can export the logs to OSS.

  Note

  The log files displayed on the Configuration tab of the Deployments page do not include the log files to which data is being written. If you set the DefaultRolloverStrategy max parameter to 1, a maximum of two files are saved. One file is saved in the memory, and the other is saved to your local disk.

View the logs of a historical job on the Logs tab

1. Log on to the Realtime Compute for Apache Flink console.

2. Find the workspace that you want to manage and click Console in the Actions column.

3. In the left-side navigation pane of the development console of Realtime Compute for Apache Flink, choose O&M > Deployments.

4. On the Deployments page, find the deployment that you want to manage and click its name.

5. Click the Logs tab.

6. In the left-side pane, click the Logs tab. On the Logs tab, select a job from the Job drop-down list.

   

7. View the logs of the JobManager and TaskManagers

   Click the Job Manager tab to view the logs of the JobManager and click the Task Managers tab to view the logs of the TaskManagers.

View the logs of a historical job in the OSS console

1. Log on to the OSS console.

2. In the left-side navigation pane, click Buckets.

3. Find the bucket that you configure when you activate Realtime Compute for Apache Flink and click its name.

4. In the Objects section of the page that appears, view the logs in the related directories.

   Folder	Description
   artifacts	Stores the uploaded JAR packages.
   flink-jobs	Stores the checkpoints of Realtime Compute for Apache Flink deployments.
   flink-savepoints	Saves the final savepoint file that is generated by the savepoint operation when you click Savepoint in the upper-right corner of the development console of Realtime Compute for Apache Flink.
   logs	Stores the logs of the deployment for which the Logging Profile parameter is set to OSS. Note The logs of deployments in the logs folder are stored in the following path: logs/${Workspace ID}/${Deployment ID}/${Job ID}. You can find the logs of the JobManager and TaskManagers of the related job based on this directory structure.

View the logs of a historical job in the Simple Log Service console

1. Log on to the Simple Log Service console.

2. In the Projects section, find the project whose logs you want to view and click its name.

3. In the Logstores section, find the Logstore in which all logs are stored and click its name.

4. On the page that appears, enter the query and analysis statements, specify a time range, and then click Search & Analyze.

   For more information, see Query and analyze logs.