All Products
Search
Document Center

Alibaba Cloud Linux:Memory reclamation optimization for the memcg global minimum watermark rating feature

Last Updated:Aug 23, 2024

When latency-sensitive and resource-consuming tasks are deployed together in a Linux operating system, resource-consuming tasks instantly request a large amount of memory. This causes the free memory of the operating system to reach the global minimum watermark (global wmark_min), which results in performance jitters in latency-sensitive tasks. The memcg global minimum watermark rating feature is supported in Alibaba Cloud Linux 2 starting with kernel version 4.19.91-18.al7 and Alibaba Cloud Linux 3 starting with kernel version 5.10.134-12.al8. You can specify different levels of memory usage thresholds (watermarks) to meet the memory requirements of latency-sensitive and resource-consuming tasks and prevent unnecessary memory reclamations from affecting the system.

Interface description

The memcg global minimum watermark rating feature is implemented on the memory.wmark_min_adj interface. The value of this interface indicates the percentage of adjustment over the global minimum watermark. Valid values: -25 to 50.

  • When the interface is created, the interface inherits a value of 0 from its parent group. Therefore, the default value is 0.

  • A negative value is a percentage of adjustment over the [0, WMARK_MIN] range, in which WMARK_MIN represents the value of global wmark_min. Example:

    memory.wmark_min_adj=-25, memcg WMARK_MIN is "WMARK_MIN + (WMARK_MIN - 0) * (-25%)"
    Note

    A negative value indicates a decrease of the global minimum watermark. This can increase the memcg quality of service (QoS) of latency-sensitive tasks.

  • A positive value is a percentage of adjustment over the [WMARK_MIN, WMARK_LOW] range, in which WMARK_MIN and WMARK_LOW represent the values of global wmark_min and global wmark_low. Example:

    memory.wmark_min_adj=50, memcg WMARK_MIN is "WMARK_MIN + (WMARK_LOW - WMARK_MIN) * 50%"
    Note

    A positive value indicates an increase of global wmark_min. This can decrease the memcg QoS of resource-consuming tasks.

  • When the offset global minimum watermark (global wmark_min) is triggered, throttling is performed. The throttling time is linearly related to the excess memory usage. Valid values of the throttling time: 1 to 1000. Unit: microseconds.

    Note

    The interface file is not stored in the memcg root directory.

Precautions

A multi-level memcg contains effective memory.wmark_min_adj, which is the final effective value of memory.wmark_min_adj. The values of memory.wmark_min_adj at all levels are traversed to obtain the maximum value. Intermediate nodes that have 0 as the default value are excluded. The following hierarchy provides an example of a multi-level memcg:

         root
         / \
        A   D
       / \
      B   C
     / \
    E   F

The following table describes the mappings between the values of memory.wmark_min_adj at each level and the final effective value.

Level

Value at each level

Final effective value

A

-10

-10

B

-25

-10

C

0

0

D

50

50

E

-25

-10

F

50

50

Note
  • The value displayed in the output of the cat /sys/fs/cgroup/memory/<Memcg path>/memory.wmark_min_adj command is the final effective value. In the command, <Memcg path> indicates the root path of the memcg.

  • We recommend that you use the global minimum watermark rating feature together with a high global minimum watermark. For example, you can set global wmark_min to 2 GB or more. You can specify the global minimum watermark by using the /proc/sys/vm/min_free_kbytes parameter.

Configuration examples

Warning
  • Improper modifications to the memcg QoS may cause network performance issues, such as network congestion, increased forwarding latency, and packet loss. This affects the quality and availability of your business. Proceed with caution.

  • Run the following commands only as the root user.

Example 1: Configure global minimum watermark rating for the memcg of latency-sensitive tasks

  1. Create a test file.

    mkdir /sys/fs/cgroup/memory/test-lc
  2. Set the percentage of adjustment over the global minimum watermark to -25.

    echo -25 > /sys/fs/cgroup/memory/test-lc/memory.wmark_min_adj

Example 2: Configure global minimum watermark rating for the memcg of resource-consuming tasks

  1. Create a test file.

    mkdir /sys/fs/cgroup/memory/test-be
  2. Set the percentage of adjustment over the global minimum watermark to 25.

    echo 25 > /sys/fs/cgroup/memory/test-be/memory.wmark_min_adj