Real-Time Synchronization

Real-time synchronization refers to the real-time flow of key service data from sources to destinations while consistency of data can be ensured. It is different from migration. Migration means moving your overall database from one platform to another. Synchronization refers to the continuous flow of data between different services.

Supported Database Types

DRS supports real-time synchronization between databases of various types, and many-to-one synchronization.

Table 1 Database types

Synchronization Direction

Data Flow

Source DB

Destination DB

Destination DB Type

To the cloud

MySQL->MySQL
  • On-premises databases
  • ECS databases
  • Databases on other clouds
  • RDS MySQL DB instances

RDS MySQL DB instances
  • Single DB instance
  • Primary/Standby DB instance

To the cloud

MySQL-> GaussDB(for openGauss) distributed
  • On-premises databases
  • ECS databases
  • Databases on other clouds
  • RDS MySQL DB instances

GaussDB(for openGauss) distributed
  • Cluster

To the cloud

MySQL -> GaussDB(for MySQL) primary/standby
  • On-premises databases
  • ECS databases
  • Databases on other clouds
  • RDS MySQL DB instances

GaussDB(for MySQL) primary/standby
  • Primary/Standby DB instance

To the cloud

PostgreSQL->PostgreSQL
  • On-premises databases
  • ECS databases
  • Databases on other clouds
  • RDS PostgreSQL DB instances

RDS PostgreSQL DB instances
  • Single DB instance
  • Primary/Standby DB instance

From the cloud

MySQL->MySQL

RDS MySQL DB instances
  • On-premises databases
  • ECS databases
  • Databases on other clouds
  • RDS MySQL DB instances

-

From the cloud

GaussDB(for openGauss) distributed -> MySQL

GaussDB(for openGauss) distributed
  • On-premises databases
  • ECS databases
  • Databases on other clouds
  • RDS MySQL instances

-

Self-built -> Self-built

MySQL->MySQL
  • On-premises databases
  • ECS databases
  • On-premises databases
  • ECS databases
  • Single DB instance
  • Primary/Standby DB instance

Synchronization Methods

DRS supports three synchronization modes: full synchronization, incremental synchronization, and full+incremental synchronization.

Full synchronization: All objects and data in non-system databases are synchronized to the destination database at a time. This mode is applicable to scenarios where service interruption is acceptable.

Incremental synchronization: Through log parsing, DRS replicates incremental data to keep sources and destinations in sync.

Full+Incremental synchronization: DRS allows you to synchronize data in real time. After a full synchronization initializes the destination database, an incremental synchronization parses logs to ensure data consistency between the source and destination databases.

Table 2 Synchronization methods

Synchronization Direction

Data Flow

Incremental

Full

Full+Incremental

One-way/Two-way Sync

To the cloud

MySQL->MySQL

Supported

Not supported

Supported

One-way sync

To the cloud

MySQL-> GaussDB(for openGauss) distributed

Not supported

Not supported

Supported

One-way sync

To the cloud

MySQL -> GaussDB(for MySQL) primary/standby

Supported

Not supported

Supported

One-way sync

To the cloud

PostgreSQL->PostgreSQL

Supported

Supported

Supported

One-way sync

From the cloud

MySQL->MySQL

Supported

Not supported

Supported

One-way sync

From the cloud

GaussDB(for openGauss) distributed -> MySQL

Supported

Not supported

Not supported

One-way sync

Self-built -> Self-built

MySQL->MySQL

Supported

Not supported

Supported

One-way sync

Database Versions

Data cannot be synchronized from a newer version database to an older version database.

Table 3 Database versions

Synchronization Direction

Data Flow

Source Database Version

Destination DB Version

To the cloud

MySQL->MySQL
  • MySQL 5.5.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x

To the cloud

MySQL-> GaussDB(for openGauss) distributed
  • MySQL 5.5.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • GaussDB(for openGauss) 1.0.0 or later

To the cloud

MySQL -> GaussDB(for MySQL) primary/standby
  • MySQL 5.5.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x

GaussDB(for MySQL)-MySQL 8.0

To the cloud

PostgreSQL->PostgreSQL
  • PostgreSQL 9.4.x
  • PostgreSQL 9.5.x
  • PostgreSQL 9.6.x
  • PostgreSQL 10.x
  • PostgreSQL 11.x
  • PostgreSQL 12.x
  • PostgreSQL 13.x
    NOTE:

    If the source DB type is RDS PostgreSQL Enhanced Edition, the destination DB type must be the same.
  • PostgreSQL 9.5.x
  • PostgreSQL 9.6.x
  • PostgreSQL 10.x
  • PostgreSQL 11.x
  • PostgreSQL 12.x
  • PostgreSQL 13.x

From the cloud

MySQL->MySQL
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x

From the cloud

GaussDB(for openGauss) distributed -> MySQL

GaussDB(for openGauss)1.3
  • MySQL 5.5.x
  • MySQL 5.6.x
  • MySQL 5.7.x

Self-built -> Self-built

MySQL->MySQL
  • MySQL 5.5.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x
  • MySQL 5.6.x
  • MySQL 5.7.x
  • MySQL 8.0.x

Network Types

DRS supports real-time synchronization through a Virtual Private Cloud (VPC), Virtual Private Network (VPN), Direct Connect, or public network. Table 4 lists the application scenarios of each network type and required preparations.

Table 4 Network types

Network Type

Application Scenario

Preparations

VPC

Synchronization between cloud databases in the same region
  • The source and destination databases must be in the same region.
  • The source and destination databases can be in either the same VPC or in different VPCs.
  • If source and destination databases are in the same VPC, they can communicate with each other by default. Therefore, you do not need to configure a security group.
  • If the source and destination databases are not in the same VPC, the CIDR blocks of the source and destination databases cannot be duplicated or overlapped, and the source and destination databases are connected through a VPC peering connection. DRS automatically establishes a route through a single IP address when you test the network connectivity.

VPN

Synchronization from on-premises databases to cloud databases or between cloud databases across regions

Establish a VPN connection between your local data center and the VPC that hosts the destination database. Before synchronization, ensure that the VPN network is accessible.

Direct Connect

Synchronization from on-premises databases to cloud databases or between cloud databases across regions

Use a dedicated network connection to connect your data center to VPCs.

Public network

Synchronization from on-premises or external cloud databases to the destination databases.

To ensure network connectivity between the source and destination databases, perform the following operations:

  1. Enable public accessibility.

    Enable public accessibility for the source database based on your service requirements.

  2. Configure security group rules.
    • Add the EIPs of the synchronization instance to the whitelist of the source database for inbound traffic.
    • If destination databases and the synchronization instance are in the same VPC, they can communicate with each other by default. Therefore, you do not need to configure a security group.
    NOTE:
    • The IP address on the Configure Source and Destination Databases page is the EIP of the synchronization instance.
    • If SSL is not enabled, synchronizing confidential data is not recommended.
Table 5 Supported network types

Synchronization Direction

Data Flow

VPC

Public Network

VPN or Direct Connect

To the cloud

MySQL->MySQL

Supported

Supported

Supported

To the cloud

MySQL-> GaussDB(for openGauss) distributed

Supported

Supported

Supported

To the cloud

MySQL -> GaussDB(for MySQL) primary/standby

Supported

Supported

Supported

To the cloud

PostgreSQL->PostgreSQL

Supported

Supported

Supported

From the cloud

MySQL->MySQL

Supported

Supported

Supported

From the cloud

GaussDB(for openGauss) distributed -> MySQL

Not supported

Supported

Supported

Self-built -> Self-built

MySQL->MySQL

Not supported

Supported

Supported

Supported Synchronization Objects

DRS allows you to synchronize different objects. The following table lists the supported objects.

Table 6 Supported synchronization objects

Synchronization Direction

Data Flow

Table-level

Database-level

Importing an Object File

To the cloud

MySQL->MySQL

Supported

Supported

Supported

To the cloud

MySQL-> GaussDB(for openGauss) distributed

Supported

Not supported

Not supported

To the cloud

MySQL -> GaussDB(for MySQL) primary/standby

Supported

Supported

Supported

To the cloud

PostgreSQL->PostgreSQL

Supported

Supported

Supported

From the cloud

MySQL->MySQL

Supported

Supported

Not supported

From the cloud

GaussDB(for openGauss) distributed -> MySQL

Supported

Not supported

Not supported

Self-built -> Self-built

MySQL->MySQL

Supported

Supported

Not supported

Advanced Features

DRS supports multiple features to ensure successful data synchronization.

Table 7 Advanced features

Feature

Description

Synchronization level

DRS supports database- and table-level synchronization.

  • Database-level synchronization refers to a type of synchronization method using database as a unit. You do not need to select tables to be synchronized. New tables in the database are automatically added to the synchronization task.
  • Table-level synchronization uses table as a unit, indicating that you need to add new tables to the synchronization task manually.

Mapping object names

Allows the names of synchronization objects (including databases, schemas, tables, and columns) in the source database to be different from those in the destination database. If the synchronization objects in source and destination databases have different names, you can map the source object name to the destination one.

The following objects can be mapped: databases, schemas and tables.

Dynamically adding or deleting synchronization objects

During data synchronization, you can add or delete synchronization objects as required.

Conflict policy

DRS uses primary key or unique key conflict policies to ensure that tables with primary key or unique constraints in the source database can be synchronized to the destination database as expected.

The following conflict policies are supported:

  • Ignore

    The system will skip the conflicting data and continue the subsequent synchronization process.

  • Overwrite

    Conflicting data will be overwritten.

  • Report error

    The synchronization task will be stopped and fail.

Ignore and overwrite: Synchronization stability is prioritized, so tasks will not be interrupted as data conflicts occur.

Report error: Data quality is prioritized. Any data conflicts are not allowed, so once a conflict occurs, the synchronization task fails and an error is reported. You need to manually find the cause of the fault. If the task is in the failed state for a long time, the storage space may be used up and the task cannot be restored.

Structure synchronization

DRS does not provide data structure synchronization as an independent function during real-time synchronization. Instead, it directly synchronizes data and structures to the destination database.
Parent topic: Functions and Features