When latency-sensitive and resource-consuming tasks are deployed together in a Linux operating system, resource-consuming tasks instantly request a large amount of memory. This causes the free memory of the operating system to reach the global minimum watermark (global wmark_min), which results in performance jitters in latency-sensitive tasks. The memcg global minimum watermark rating feature is supported in Alibaba Cloud Linux 2 starting with kernel version 4.19.91-18.al7
and Alibaba Cloud Linux 3 starting with kernel version 5.10.134-12.al8
. You can specify different levels of memory usage thresholds (watermarks) to meet the memory requirements of latency-sensitive and resource-consuming tasks and prevent unnecessary memory reclamations from affecting the system.
Interface description
The memcg global minimum watermark rating feature is implemented on the memory.wmark_min_adj
interface. The value of this interface indicates the percentage of adjustment over the global minimum watermark. Valid values: -25 to 50.
When the interface is created, the interface inherits a value of 0 from its parent group. Therefore, the default value is 0.
A negative value is a percentage of adjustment over the
[0, WMARK_MIN]
range, in whichWMARK_MIN
represents the value ofglobal wmark_min
. Example:memory.wmark_min_adj=-25, memcg WMARK_MIN is "WMARK_MIN + (WMARK_MIN - 0) * (-25%)"
NoteA negative value indicates a decrease of the global minimum watermark. This can increase the memcg quality of service (QoS) of latency-sensitive tasks.
A positive value is a percentage of adjustment over the
[WMARK_MIN, WMARK_LOW]
range, in which WMARK_MIN andWMARK_LOW
represent the values ofglobal wmark_min
andglobal wmark_low
. Example:memory.wmark_min_adj=50, memcg WMARK_MIN is "WMARK_MIN + (WMARK_LOW - WMARK_MIN) * 50%"
NoteA positive value indicates an increase of
global wmark_min
. This can decrease the memcg QoS of resource-consuming tasks.When the offset global minimum watermark (
global wmark_min
) is triggered, throttling is performed. The throttling time is linearly related to the excess memory usage. Valid values of the throttling time: 1 to 1000. Unit: microseconds.NoteThe interface file is not stored in the memcg root directory.
Precautions
A multi-level memcg contains effective memory.wmark_min_adj
, which is the final effective value of memory.wmark_min_adj
. The values of memory.wmark_min_adj at all levels are traversed to obtain the maximum value. Intermediate nodes that have 0 as the default value are excluded. The following hierarchy provides an example of a multi-level memcg:
root
/ \
A D
/ \
B C
/ \
E F
The following table describes the mappings between the values of memory.wmark_min_adj at each level and the final effective value.
Level | Value at each level | Final effective value |
A | -10 | -10 |
B | -25 | -10 |
C | 0 | 0 |
D | 50 | 50 |
E | -25 | -10 |
F | 50 | 50 |
The value displayed in the output of the
cat /sys/fs/cgroup/memory/<Memcg path>/memory.wmark_min_adj
command is the final effective value. In the command,<Memcg path>
indicates the root path of the memcg.We recommend that you use the global minimum watermark rating feature together with a high global minimum watermark. For example, you can set
global wmark_min
to 2 GB or more. You can specify the global minimum watermark by using the/proc/sys/vm/min_free_kbytes
parameter.
Configuration examples
Improper modifications to the memcg QoS may cause network performance issues, such as network congestion, increased forwarding latency, and packet loss. This affects the quality and availability of your business. Proceed with caution.
Run the following commands only as the root user.
Example 1: Configure global minimum watermark rating for the memcg of latency-sensitive tasks
Create a test file.
mkdir /sys/fs/cgroup/memory/test-lc
Set the percentage of adjustment over the global minimum watermark to -25.
echo -25 > /sys/fs/cgroup/memory/test-lc/memory.wmark_min_adj
Example 2: Configure global minimum watermark rating for the memcg of resource-consuming tasks
Create a test file.
mkdir /sys/fs/cgroup/memory/test-be
Set the percentage of adjustment over the global minimum watermark to 25.
echo 25 > /sys/fs/cgroup/memory/test-be/memory.wmark_min_adj