×
Community Blog Data Migration: Steps and Solutions for Data Transmission

Data Migration: Steps and Solutions for Data Transmission

9 Easy Steps for Successful Data Migration Alibaba Cloud provides a convenient way for migration through its Data Transmission Service (DTS).

9 Easy Steps for Successful Data Migration

Alibaba Cloud provides a convenient way for migration through its Data Transmission Service (DTS). By migrating database through DTS, you can implemen.

When it comes to migration, there are many factors that need to be considered to ensure successful migration. Overlooking even the tiniest of aspects can cause failures, or worse, irreversible damages to businesses. There is no doubt that data migration is a daunting task for enterprises, regardless of size.

To address these issues, Alibaba Cloud provides a convenient way for migration through its Data Transmission Service (DTS). By migrating a local database to Alibaba Cloud ApsaraDB for RDS (RDS) using DTS, you can implement smooth database migration without any service interruption. In this article, we will illustrate how you can easily migrate your local data to the Alibaba Cloud RDS using DTS.

What is Data Transmission Service?

Data Transmission Service (DTS) is a data exchange service that streamlines data migration, data synchronization, and data subscription. DTS is capable of handling most widely used commercial and open-sourced databases, including RDBMS (Relational Database Management System) and NoSQL. DTS supports schema migration and full migration of SQL Server data.

Data Transmission Service supports schema migration of the following objects: tables, views, table triggers, synonyms, SQL stored procedures, SQL functions, custom types, plan GUIDs, rules, and defaults.

Full migration through Data Transmission Service will migrate all the data of the migration objects in a local database to the target instance. If there are any incremental updates during the migration process, the incremental data will not be migrated to the target database. Therefore, we recommend you use Data Transmission for full data migration when no data is being written to the local business database.

Restrictions on Migration

Migrating a local database to RDS is subject to the following restrictions:

  1. DDL operations are not supported during migration.
  2. Schema migration does not support migration of assemblies, database-level stored procedures, service brokers, full-text indexes, full-text directories, distributed schemas, distributed functions, CLR scalar functions, CLR value functions, inner tables, aggregate functions, and systems.
  3. If object name mapping is enabled, other objects dependent on this object may fail to be migrated.

Note: Before the migration, create a migration account in the local database and RDS instance, create the database to be migrated in the RDS instance, and grant the read/write permissions of the database to the migration account. Different migration types need different permissions.

Related Blogs

Data Migration Pre-Check Failures and Solutions

Alibaba Cloud Data Transmission service supports data transmission between database-centered structured storage products.

Abstract

Alibaba Cloud Data Transmission Service supports data transmission between database-centered structured storage products. It integrates a wide variety of data transmission capabilities, including data migration, data subscription, and real-time data synchronization. With Data Transmission Service, you can easily build a secure, scalable, and highly available data architecture while achieving a one-click cloud migration. Data Transmission Service supports data migration between common database types including Oracle, MySQL, SQL Server, and Postgres Plus Advanced Server. Meanwhile, it also supports incremental data subscription for ApsaraDB for RDS MySQL and real-time synchronization between ApsaraDB for RDS instances.

Before ApsaraDB for RDS initiates data migration, Data Transmission Service will run a series of pre-check tests to ensure successful migration. This article briefly introduces the content found during the pre-check phase, as well as the causes of failures and solutions for each pre-check item.

Source Database Connectivity Check

This check aims to test whether the Data Transmission server can connect to the source database to be migrated. Data Transmission creates a connection to the source database through the JDBC protocol. When the creation fails, this check item fails.

The source database connection pre-check may fail for the following reasons:

1.Incorrect account or password provided when the migration task is created

Check method: Connect to the source database through client software on any network-ready server that can connect to the source database with the account/password used for creating the migration task, and confirm whether the connection works normally. If the connection reports errors, and the error prompt contains Access deny, it indicates the account or password is incorrect.

Solution: Modify the migration task in the Data Transmission Service console and enter the correct account and password, and then re-run the pre-check.

2.Source database migration account places restrictions on the source IP address

Check method:

  1. Connect to the source database through client software on any network-ready server that can normally connect to the source database with the account/password used to create the migration task. This indicates the source database limits the source IP address and only allowed servers can connect to it. The Data Transmission server IP address is not included in the authorized list, so it cannot be connected normally.
  2. If the source database is MySQL, you can connect to the source database with MySQL client, and execute select host from mysql.user where user='migration account',password='migration account password'.
    If the query result is not "%", it indicates that the Data Transmission server IP address is not included in the authorized IP address list, causing the connection failure.
  3. If the source database is SQL Server, you are advised to check whether a firewall has been set on the SQL Server installation server, or check whether there is an endpoint or trigger in the source database that places source IP address limitations.
  4. If the source database is Oracle, you are advised to check the sqlnet.ora configuration file of the source database and confirm whether the configuration item TCP.VALIDNODE_CHECKING is set to yes. If so, it indicates that the source database places source IP address limitations.

Solution:

  1. If the source database is MySQL, you can execute grant all on . to "migration account"@"%" identified by "migration password" in the source database to re-authorize the migration account. Replace the migration account in this command with the real migration account and migration password with the real migration password. Re-run the pre-check after authorization.
  2. If the source database is SQLServer, you need to first shut down the firewall or disable the trigger, and re-run the pre-check.
  3. If the source database is Oracle, you are advised to first modify TCP.VALIDNODE_CHECKING=no and restart the listener and then re-run the pre-check.

Cloud Migration without Stopping Services: Alibaba Cloud DTS

Alibaba Cloud Data Transmission Service (DTS) lets you perform data migration without stopping services, ensuring data reliability while maintaining business continuity.

More and more enterprise customers are choosing to migrate to the cloud to fully benefit from its convenience and high reliability. However, for many customers, the first issue encountered when migrating to the cloud is to smoothly complete the cloud migration without affecting the business. Many traditional migration solutions require businesses to stop services during migration, which affects business continuity.

Alibaba Cloud Data Transmission Service (DTS) has the ability to perform data migration without stopping services, meaning the cloud migration is completed while the source database provides the normal services of the business. In this article, we will discuss the principles and process of cloud migration, as well as explain the benefits of using Alibaba Cloud's DTS.

Migration after Stopping Services - Traditional Database Migration Solution

A variety of database engines have emerged in the market to meet different user scenarios. To meet user requirements for importing data, database vendors typically have their own data migration tools. For example: mysqldump for MySQL and exp/imp for Oracle.

However, traditional migration tools realize the migration of existing data through data export and import. To ensure the integrity of the migrated data, the source database service is usually stopped and the amount of time for which the service remains stopped depends on how long it takes to migrate the existing data. This can be several hours, and results in an extremely high migration cost for users.

Migration without Stopping Services - Alibaba Cloud DTS

Alibaba Cloud provides a migration service called Data Transmission Service (DTS), to reduce the migration cost to users. DTS supports data transmission between data sources such as relational databases, NoSQL, and Big Data (OLAP). It is a data transmission service that integrates data migration, data log real-time subscription, and real-time data synchronization.

DTS provides data migration without stopping services, enabling the smooth completion of the migration of database data without stopping the services related to the source database. Users can apply data migration in various scenarios:

  1. Migrate a self-built database from an offline IDC room to the self-built database on Alibaba Cloud RDS or ECS.
  2. Migrate a self-built database from other cloud vendors to the self-built database on Alibaba Cloud RDS or ECS.
  3. Migrate a database on Alibaba Cloud across regions and accounts.
  4. Migrate a self-built database on Alibaba Cloud RDS or ECS to the offline IDC test environment.

Trouble-Free Cloud Migration 1: Migrate Data from On-Premises Systems to the Cloud

This document shows you how to migrate your local data to Alibaba Cloud. It provides actionable solutions to multiple migration scenarios.

With the rapid development of cloud computing services, migrating existing server systems to the cloud easily and quickly is of great significance for most enterprises. In response, Alibaba Cloud has launched the server migration services. This migration service is a one-click solution that provides easy-to-use server system migration tools, helping to reducing costs while streamlining the server migration process.

Using the migration services makes light work of system migration. For more information on conditions of use and basic usage, you can refer to the migration tool help documentation.

Migration Process

  1. Practice using the migration tools by performing trial migrations.
  2. Assess the migration time and costs involved, and make a migration plan.
  3. Migrate your systems. For inquiries, contact Alibaba Cloud's Support Team.

    1. Data Transmission Service (DTS)
    2. Migrate your server to Alibaba
      Cloud by using the Cloud Migration tool
  4. Create Pay-As-You-Go instances and debug the system services.
  5. Switch to a cloud system and upgrade your instances to subscription instances.

Related Courses

Unstructured Data Migration

Experts estimate that 80 to 90 percent of the data in any organization is unstructured. Many organizations believe that their unstructured data stores include information that could help them make better business decisions. but to analyze unstructured data, there are some main challenges in terms of data migration. This course introduces the basic concepts of unstructured data, Alibaba Cloud unstructured data migration tools and service, Alibaba Cloud unstructured data SDK and how to use OssImport tool for unstructured data migration.

Structured Data Migration

Most applications use structured database software for data storage, such as: My SQL, SQL Server, Oracle. This course introduces the basic concepts of structured data storage, storage modes, and data migrations, focusing on how to use the Alibaba Cloud data migration service and DTS(Data Transmission Service) for structured data migration to the cloud.

Migrate Your Application to Alibaba Cloud

This course is designed to help traditional IT companies who want to move their business to the cloud and have a comprehensive understanding of Alibaba Cloud's recommendations and available tools for the migration of the application environment. So that you can choose the most appropriate way to complete the migration.

Related Market Products

SaaSaMe Transport On-line Migration Edition

A workload mover software solution, supports Physical-to-Cloud, Virtual-to-Cloud, and Cloud-to-Cloud. The very first on-line migration solution tightly integrated with Alibaba Cloud.

How to Migrate Data to Big Data Platform on the Cloud

Quickly learn the basic concepts and architectures of MaxCompute and DataWorks, and how to perform data integration with multiple data sources and formats.

Related Documentation

Seamlessly migrate data from a cloud service to OSS

This topic describes how to migrate data of an Internet enterprise from a cloud service to Alibaba Cloud Object Storage Service (OSS).

Background information

Enterprise A is an Internet service provider that deploys the main business applications in a cloud service provided by Cloud Service Provider B. The main business applications of Enterprise A provide online editing services for media files, such as images and videos. The existing data that is stored on the servers of Cloud Service Provider B includes 100,000,000 files and has a total size of about 320 TB with a daily increase of 20 GB. The bandwidth for both the storage service of Cloud Service Provider B and OSS is 250 MB/s. The business applications require a maximum bandwidth of 50 MB/s.

Enterprise A wants to switch over the businesses to OSS based on the development needs. When you switch over businesses between two data stores, you must migrate existing data and incremental data to OSS. To ensure a successful migration of large amounts of historical data and business continuity, the following needs must be met:

  1. During migration, you must ensure business continuity and avoid the impacts on normal data access from users.
  2. After the migration job is complete, you must check data integrity to ensure a seamless switchover of the businesses to OSS.

Migration solution

Based on the background information, you can migrate data as follows:

  1. With Data Transport, you can migrate existing data from a cloud service to OSS. Before a migration job is complete, ensure that no updates occur on the customer side.
  2. After existing data is migrated, you can create back-to-origin rules in OSS for users to access the incremental data that has not been migrated.
  3. Switch over businesses to OSS.
  4. After the business switchover is complete, you can migrate incremental data to OSS by using Data Transport.
  5. After all data is migrated and validated, delete the data at the source data address.

Data migration

You can use DTS to migrate data between various types of data sources. Typical scenarios include data migration to Alibaba Cloud, data migration between instances within Alibaba Cloud, and database splitting and scale-out. DTS supports data migration between homogeneous and heterogeneous data sources. It also supports extract, transform, and load (ETL) features such as object name mapping and data filtering.

For more information about data migration in various scenarios, see Overview.

Related Products

Data Transmission Service

Supports data migration and data synchronization between data engines, such as relational database, NoSQL and OLAP

ADAM (Database and Application Migration)

Heterogeneous database migration has always been a pain point of database migration. Traditionally, assessment of the compatibility and workload is difficult to complete in a short time; and it is also difficult to automatically output the transformation scheme and sort out the relationship between the database and the application. ADAM decreases the difficulty of heterogeneous database migration with functions of compatibility assessment, structure transformation suggestion, migration workload evaluation, application transformation assessment and suggestion.

0 0 0
Share on

Alibaba Clouder

2,599 posts | 762 followers

You may also like

Comments