Alibaba Cloud Linux:THP reclaim

Interfaces

Limits

Alibaba Cloud Linux images that contain the following kernel versions support the THP reclaim feature:

• Alibaba Cloud Linux 2: kernel 4.19.91-24.al7 or later

• Alibaba Cloud Linux 3: kernel 5.10.134-15.al8 or later

Configure the THP reclaim feature

This section describes how to configure the THP reclaim feature. In this example, a memory control group named test is used.

1. Run the following command to create a memory control group named test:

   sudo mkdir /sys/fs/cgroup/memory/test/

2. Run the following command to enable the THP reclaim feature for test:

   sudo sh -c 'echo reclaim > /sys/fs/cgroup/memory/test/memory.thp_reclaim'

3. Run the following command to check whether the THP reclaim feature is enabled for test:

   cat /sys/fs/cgroup/memory/test/memory.thp_reclaim

   The following command output is returned. The settings that take effect are enclosed in brackets ([]). [reclaim] in the command output indicates that the THP reclaim feature is enabled for test.

   

4. Run the following command to forcefully enable the THP reclaim feature by using the global interface:

   sudo sh -c 'echo reclaim > /sys/kernel/mm/transparent_hugepage/reclaim'

   Run the following command to forcefully disable the THP reclaim feature:

   sudo sh -c 'echo disable > /sys/kernel/mm/transparent_hugepage/reclaim'

   Note

   If you set the /sys/kernel/mm/transparent_hugepage/reclaim global interface to reclaim or disable, the global interface takes precedence over the memory.thp_reclaim interface. The memory.thp_reclaim interface configurations in memory control groups are not affected.

5. Run the following command to configure the threshold parameter in the memory.thp_reclaim_ctrl interface to limit the number of zero subpages in a THP for test:

   sudo sh -c 'echo "threshold 32" >  /sys/fs/cgroup/memory/test/memory.thp_reclaim_ctrl'

   In this example, the threshold parameter is set to 32. When the number of zero subpages in a THP exceeds 32, the zero subpage reclaim subfeature of the THP reclaim feature is triggered to reclaim zero subpages.

6. Trigger the zero subpage reclaim subfeature of the THP reclaim feature.

   After the zero subpage reclaim subfeature of the THP reclaim feature is triggered, the subfeature reclaims excess zero subpages when the number of zero subpages exceeds the value of threshold. Take note of the following items for the reclaim parameter:

   Note

   You can only write reclaim to the memory.thp_reclaim_ctrl interface to trigger the zero subpage reclaim subfeature of the THP reclaim feature. You cannot run the cat command to query the reclaim settings.

   • Run the following command to trigger the zero subpage reclaim subfeature of the THP reclaim feature for the current memory control group:

     sudo sh -c 'echo "reclaim 1" >  /sys/fs/cgroup/memory/test/memory.thp_reclaim_ctrl'

   • Run the following command to recursively trigger the zero subpage reclaim subfeature of the THP reclaim feature for the current memory control group and its child control groups:

     sudo sh -c 'echo "reclaim 2" >  /sys/fs/cgroup/memory/test/memory.thp_reclaim_ctrl'

     You can trigger the zero subpage reclaim subfeature of the THP reclaim feature by writing reclaim to the memory.thp_reclaim_ctrl interface. The THP reclaim feature can be triggered by memory reclamation.

     • If an OOM error occurs, the zero subpage reclaim subfeature of the THP reclaim feature is triggered.

     • If the memcg backend asynchronous reclaim feature is triggered, the zero subpage reclaim subfeature of the THP reclaim feature is triggered. For information about the memcg backend asynchronous reclaim feature, see Memcg backend asynchronous reclaim.

7. Run the following command to check the status of the THP reclaim feature for test:

   cat /sys/fs/cgroup/memory/test/memory.thp_reclaim_stat

   The following command output is returned:

   queue_length        14
   split_hugepage     523
   reclaim_subpage 256207

   In the preceding command output:

   • queue_length: indicates that 14 THPs are in the queue.

   • split_hugepage: indicates that 523 THPs are split by the THP reclaim feature.

   • reclaim_subpage: indicates that 256,207 zero subpages are reclaimed by the THP reclaim feature.

Test the THP reclaim feature

This section provides sample C code that is used to request THPs for applications. You can use the sample C code to perform a test when the THP reclaim feature is enabled and disabled, and obtain different test results.

1. Run the following command to set memory.limit_in_bytes to 1 GiB:

   sudo sh -c 'echo 1G > /sys/fs/cgroup/memory/test/memory.limit_in_bytes'

   Run the following command to query the value of memory.limit_in_bytes:

   cat /sys/fs/cgroup/memory/test/memory.limit_in_bytes

   The following command output is returned.

   

2. Run the following command to disable the memcg backend asynchronous reclaim feature.

   For information about the memcg backend asynchronous reclaim feature, see Memcg backend asynchronous reclaim.

   sudo sh -c 'echo 0 > /sys/fs/cgroup/memory/test/memory.wmark_ratio'

3. Run the following sample C code when the THP reclaim feature is enabled and disabled and check the test results:

   gcc -o test <test.c>

   Replace parameters in the <test.c> sample C code with actual values based on your business requirements.

   // The application requests 1 GiB of memory (512 THPs), in which 10 THPs contain zero subpages. 
   #include <stdlib.h>  // For posix_memalign
   #include <string.h>  // For memset
   #include <unistd.h>  // For pause
   
   #define HUGEPAGE_SIZE 4096 * 512
   int main()
   {
           int i, thp = 512;
           char *addr;
           posix_memalign((void **)&addr, HUGEPAGE_SIZE, HUGEPAGE_SIZE * thp);
   
           for (i = 0; i < 10; i++) {
                   memset(addr, 0xc, HUGEPAGE_SIZE >> 1);
                   addr += HUGEPAGE_SIZE;
           }
   
           for (; i < thp; i++) {
                   memset(addr, 0xc, HUGEPAGE_SIZE);
                   addr += HUGEPAGE_SIZE;
           }
   
           pause();
           return 0;
   }

   During the test, you can run the dmesg -wH command on another terminal to detect OOM errors and memory usage in the system. The test results vary based on whether the THP reclaim feature is enabled.

   • If the THP reclaim feature is enabled, the feature splits the previously allocated THPs into subpages and reclaims zero subpages. This reduces memory usage and prevents OOM errors.

   • If the THP reclaim feature is disabled, the system does not split THPs or reclaim zero subpages. As a result, OOM errors may occur. Kernel logs indicate insufficient memory and the details of the processes that may be terminated.

References

• How do I optimize Transparent Huge Pages (THP) related performance on an Alibaba Cloud Linux instance?

• Huge Pages