The persisted buffer pool (PBP) feature ensures that the previous shared buffer pools can be used when the cluster crashes or is restarted.
Prerequisites
Your PolarDB for PostgreSQL cluster runs the following engine:
PostgreSQL 14 (revision version is 14.5.2.0 or later)
PostgreSQL 11 (revision version is 1.1.1 or later)
You can run the following statement to view the minor version of your PolarDB for PostgreSQL cluster:
PostgreSQL 14
select version();PostgreSQL 11
show polar_version;
Background information
The memory of a PolarDB for PostgreSQL cluster consists of shared buffer pools, dynamic shared memory areas, and process global areas.
Shared buffer pool: a large part of the shared memory that is pre-allocated when the cluster is started. Allocations to various modules are determining by offset values.
Dynamic shared memory area: the shared memory area designed in PostgreSQL to implement inter-process parallel computing. It can be dynamically expanded.
Process global area: the memory area used by a process. It includes the following parts.
Memory context
Private memory
The following figure shows how the memory is divided.
Shared buffer pools occupy the most memory of a PolarDB for PostgreSQL cluster and have a direct impact on cluster performance. In the native PostgreSQL, shared buffer pools are cleared and then reinitialized when the cluster restarts or unexpectedly exits. When the cluster is restarted and enters the fault recovery state, data pages are modified based on WAL logs. As a result, the data needs to be reloaded or modified. This affects the availability of the cluster. In addition, when shared buffer pools are reinitialized, business-related data may be reloaded. This causes severe performance jittering.
To resolve the preceding issues, PolarDB for PostgreSQL provides the PBP feature to ensure that the original shared buffer pools can be used when the cluster restarts or unexpectedly exits. This feature has the following benefits:
Shorter RTO to improve cluster availability.
No obvious performance jittering before and after the cluster crashes.
How this feature works
PolarDB for PostgreSQL changes some shared buffer pools to persisted buffer pools that have the pod-level lifecycle. The PBP feature of PolarDB for PostgreSQL only involves buffer pools and buffer descriptors that have the pod-level lifecycle. Other memory parts still have the cluster-level lifecycle.
Pod-level lifecycle: PolarDB for PostgreSQL clusters are deployed on Kubernetes. The memory parts that have the pod-level lifecycle are not destroyed when the cluster unexpectedly exists.
Cluster-level lifecycle: The memory parts that have the pod-level lifecycle are cleared after the cluster crashes.
The following figure shows how the memory that has the pod- and cluster-level lifecycle is divided.
PBP availability-related metrics
The PBP feature is not applicable in all scenarios. The PBP feature cannot be used under the following conditions when the cluster is started:
The specifications of the cluster change and buffer pools of different sizes are required.
The current PBP is not created for this cluster.
The control information in the current PBP is invalid.
In addition to the overall availability of the PBP, each PBP page requires an availability check. The current page cannot be used under the following conditions:
Uncommitted transactions exist on the current page.
The descriptor of the current page is invalid.
Invalid log sequence numbers (LSNs) exist on the current page.
The properties of the current page are exceptional or invalid.
The following figure shows PBP availability-related metrics.
Improved performance of the PBP feature
The PBP feature ensures that the previous shared buffer pools can be used when the cluster is restarted. Therefore, you can quickly use cached data to improve performance when the cluster is either in the fault recovery state or running. The following figure shows the improved performance of the PBP feature.
Comparison of recovery performance
The following scenario is tested: the cluster crashes and 2093 MB of logs are replayed.
Item
Log replay duration
Recovery duration
PBP feature disabled
598s
746s
PBP feature enabled
68s
294s
The following figure shows the time consumed before and after the PBP feature is enabled.
Comparison of restart performance
Not all pages in buffer pools can be reused. For example, a process adds an X lock on the page. If the cluster crashes after that, no process can release the X lock. Therefore, all pages in the buffer pool are checked for availability after the cluster crashes and is restarted to eliminate pages that cannot be reused. The reclamation of the buffer pool also depends on Kubernetes. The PBP feature also can stabilize the performance before and after the cluster is restarted. The following figure shows the performance values before and after the cluster is restarted.
Usage notes
Set the following parameter to ON:
polar_enable_persisted_buffer_pool = ONAfter you change the parameter value, you must restart the cluster for the change to take effect.