This topic describes how to use the cloud migration feature of ApsaraDB RDS for PostgreSQL to migrate the data of a PostgreSQL instance that has a public IP address to an ApsaraDB RDS for PostgreSQL instance. The PostgreSQL instance can be a self-managed PostgreSQL instance or a PostgreSQL instance of a third-party cloud service provider.
Background information
You can create an RDS instance in a virtual private cloud (VPC). You can use the cloud migration feature to migrate the data of a PostgreSQL instance to the RDS instance over the internal network. If you want to use the cloud migration feature to migrate the data of a PostgreSQL instance that has a public IP address to the RDS instance, you must configure an Internet NAT gateway for the RDS instance and associate an elastic IP address (EIP) with the Internet NAT gateway.
This topic also describes how to configure an Internet NAT gateway for an RDS instance and associate an EIP with the Internet NAT gateway.
This configuration enables migration over the Internet helps ensure the network security of the RDS instance during the cloud migration.
Source network address translation (SNAT) entries that are created for an Internet NAT gateway allow an RDS instance to access the Internet but does not allow the RDS instance to provide services over the Internet. For more information about Internet NAT gateways and SNAT, see Use the SNAT feature of an Internet NAT gateway to access the Internet.
Scenarios
You want to migrate the data of a self-managed PostgreSQL instance that has a public IP address to an RDS instance.
You want to migrate the data of a PostgreSQL instance that is provided by a third-party cloud service provider, such as Google or Amazon, to an RDS instance.
NoteThird-party cloud service providers may use self-developed or custom PostgreSQL extensions. If an extension incompatibility issue occurs during the cloud migration, you can submit a ticket.
Prerequisites
The following requirements are met:
The major engine versions of the PostgreSQL instance and the RDS instance are the same. PostgreSQL 10 or later is supported.
NoteIf the major engine version of the PostgreSQL instance is earlier than PostgreSQL 10, you must upgrade the major version before you perform the cloud migration.
If the major engine version of the PostgreSQL instance is different from that of the RDS instance, the cloud migration assessment fails, and you cannot perform the cloud migration.
The RDS instance is a primary RDS instance. Read-only RDS instances do not support the cloud migration feature.
The RDS instance uses cloud disks.
The RDS instance is empty. The available storage of the RDS instance is greater than or equal to the size of the data in the PostgreSQL instance.
NoteIf data exists in the RDS instance, the cloud migration assessment fails, and you cannot perform the cloud migration. You must back up the data and clear the data on the RDS instance or create another RDS instance before you perform the cloud migration.
Limits
None.
Impacts
None.
Usage notes
During the cloud migration, make sure that the PostgreSQL instance can be accessed. Do not perform operations such as restart. If transient connections or switchovers that are required to achieve high availability occur during the cloud migration, the cloud migration fails.
Billing
You must configure an Internet NAT gateway for the RDS instance and associate an EIP with the Internet NAT gateway. For more information, see Billing of Internet NAT gateways.
Procedure
Step 1: Configure an Internet NAT gateway and associate an EIP with the Internet NAT gateway
This step describes how to associate a public IP address with the VPC of your RDS instance. After a public IP address is associated, your RDS instance can access a self-managed PostgreSQL instance that has a public IP address or a PostgreSQL instance of a third-party cloud service provider.
Make sure that no Internet NAT gateway is configured for and no public IP address is associated with the VPC of your RDS instance. Otherwise, the RDS instance may fail to be accessed.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, click Database Connection. In the Database Connection section of the page that appears, click the VPC link to the right of the Network Type parameter.
In the VPC console, click the Resources tab. In the Access to Internet section, click Create Now.
On the Internet NAT Gateway page, configure the following parameters and retain the default values for the parameters that are not described in the following table.
Parameter
Description
Example
EIP
If an EIP has been associated with the VPC, set this parameter to Select EIP. In this topic, Purchase EIP is selected.
Purchase EIP
Maximum Bandwidth
Specify the maximum bandwidth value based on your business requirements. To facilitate the cloud migration, you can set this parameter to the maximum value to prevent slow cloud migration issues due to bandwidth limits.
200Mbps
Click Buy Now.
Confirm the parameter settings, read and select Terms of Service, and then click Confirm.
If the page that is shown in the following figure appears, the Internet NAT gateway is created, an EIP is associated with the Internet NAT gateway, and SNAT entries are created for the Internet NAT gateway.
Click the resource ID of the EIP. On the Instance Information tab of the EIP, view the public IP address.
Step 2: Configure the self-managed PostgreSQL instance or the PostgreSQL instance of the third-party cloud service provider
Configure the self-managed PostgreSQL instance
Configure the postgresql.conf file of the self-managed PostgreSQL instance.
Create an account for cloud migration on the self-managed PostgreSQL instance.
Update the pg_hba.conf file of the self-managed PostgreSQL instance.
ImportantThe IP address configured in the pg_hba.conf file is the EIP that is associated for your RDS instance in Step 1.
Configure the firewall of the server on which the self-managed PostgreSQL instance resides.
Configure the PostgreSQL instance of a third-party cloud service provider
Change the value of the wal_keep_segments or wal_keep_size parameter. For more information, see the official documentation of each cloud service provider.
View the minimum size of log files that can be retained in the pg_wal directory. If the minimum size exceeds, WAL logs of the PostgreSQL instance are deleted after a full backup. In this case, you must back up the PostgreSQL instance again.
wal_keep_segments: This parameter is applicable to instances that run PostgreSQL 10, PostgreSQL 11, and PostgreSQL 12. We recommend that you set this parameter to a value that is greater than or equal to 4096.
wal_keep_size: This parameter is applicable to instances that run PostgreSQL 13, PostgreSQL 14, and PostgreSQL 15. We recommend that you set this parameter to a value that is greater than or equal to 65536.
Create an account for cloud migration.
CREATE USER migratetest CREATEROLE REPLICATION LOGIN PASSWORD '123456'; GRANT pg_monitor TO migratetest;
NoteIn the preceding statements,
migratetest
and123456
are examples. You need to replace them with actual values.Enable the public IP address and configure a security group or whitelist to allow the RDS instance to access the PostgreSQL instance of the third-party cloud service provider by using the EIP that is associated with the RDS instance in Step 1. For more information, see the official documentation of the cloud service provider.
ImportantYou must make sure that the Internet Control Message Protocol (ICMP) protocol is enabled for the PostgreSQL instance of the third-party cloud service provider. You can run the
ping <Public endpoint of the PostgreSQL instance of the third-party cloud service provider>
command to check whether the ICMP protocol is enabled.
Step 3: Perform a cloud migration assessment
Log on to the ApsaraDB RDS console and go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the instance ID.
In the left-side navigation pane, click Cloud Migration/DR Construction. On the page that appears, click the Feasibility assessment tab.
In the Configure Scenario and Source Type step of the configuration wizard, select Migration to Cloud for Scenario and Self-managed Instance or Other Instance for Source. Then, click Next.
In the Configure Destination Instance step of the configuration wizard, click Next.
In the Configure Source Instance step of the configuration wizard, select all listed items and click Next. Before you start the cloud migration, you must complete the preparations that are described in the listed items.
In the Start Feasibility Assessment step of the configuration wizard, configure the information about the self-managed PostgreSQL instance.
Parameter
Description
Migration Task Name
The system automatically generates a name for the cloud migration task. You do not need to modify the generated name.
Source VPC/DNS IP
If the source instance is a self-managed PostgreSQL instance that has a public IP address, set this parameter to the public IP address of the server on which the self-managed PostgreSQL instance is deployed.
If the source instance is a PostgreSQL instance of a third-party cloud service provider, set this parameter to the public endpoint of the PostgreSQL instance.
Port of Source Instance
If the source instance is a self-managed PostgreSQL instance that has a public IP address, you can run the
netstat -a | grep PGSQL
command to view the port.If the source instance is a PostgreSQL instance of a third-party cloud service provider, you can go to the console of the third-party cloud service provider to view the port.
Username
The username and password that are created for the cloud migration task in Step 2.
Password
Click Create Feasibility Assessment Task.
After the cloud migration assessment is complete, you can view the status of the cloud migration assessment task in Migration to Cloud section of the Feasibility assessment tab.
If the value in the Status column of the feasibility assessment task is Successful, you can start the cloud migration.
If the value in the Status column of the feasibility assessment task is Failed, you can click View Report in the Actions column to view and handle the reported errors. For more information about common errors, see Introduction to cloud migration assessment reports.
NoteThird-party cloud service providers may use self-developed or custom PostgreSQL extensions. If an extension incompatibility issue occurs during the migration, you can submit a ticket.
After you handle the reported errors, you can click Re-assess in the Actions column to run the cloud migration assessment task again.
Step 4: Start the cloud migration
You can start the cloud migration only when the status of the feasibility assessment task is Successful.
Go to the Instances page. In the top navigation bar, select the region in which the RDS instance resides. Then, find the RDS instance and click the ID of the instance.
In the left-side navigation pane, click Cloud Migration/DR Construction. On the page that appears, click the Migration to Cloud tab. On the tab that appears, click Create Cloud Migration Task.
In the Create Cloud Migration Task dialog box, select the migration assessment task whose status indicates a success in Step 3 from the Associated Assessment Task drop-down list.
After you select a migration assessment task from the Associated Assessment Task drop-down list, the system automatically obtains the values of the required parameters.
Click Initiate Migration to Cloud. The system automatically starts the cloud migration task.
WarningDuring the cloud migration, make sure that the PostgreSQL instance can be accessed. Do not perform operations such as restart. If transient connections or switchovers that are required to achieve high availability occur during the cloud migration, the cloud migration fails.
Switch the workloads.
Click the link in the Cloud Migration Phase column to view the progress of the cloud migration task.
When the cloud migration task enters the phase of Incremental Data Synchronization, click Switchover in the Actions column of the cloud migration task to switch the workloads of the PostgreSQL instance to the RDS instance.
In the Switchover dialog box, configure the PostgreSQL instance to process only read requests. Alternatively, stop the connected application from writing data to the PostgreSQL instance.
You can configure the PostgreSQL instance to process only read requests.
-- Configure the PostgreSQL instance to process only read requests. ALTER SYSTEM SET default_transaction_read_only=on; -- Reload the parameter configuration for the modification to take effect. SELECT pg_reload_conf(); -- Terminate all existing sessions. SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE usename not in ('replicator', 'monitor', 'pgsql', 'aurora') AND pid != pg_backend_pid();
Select all check boxes and click Switch Now. Then, wait until the cloud migration is complete.