- About MDB
- Installation
- Upgrade
- Concepts
- User Interface
- Source Requirements
- Target Database Requirements
- Target Kafka Requirements
- Usage Restriction
- Frequently Exceptions
- Release Note
Concepts
This document introduces MDB-related basic concepts, such as node, channel, object, task, and others.
Node
In the MDB system, a node is an endpoint in the replication process and corresponds to a database system.
A node can be a local (located in the server where the MDB system is deployed) or remote (located in a remote server) node.
A node can be used as a source database (for capturing data) or a target database (for synchronizing data).
The node-related concepts are as follows.
| Concept | Description |
|---|---|
| Node type | Database type, such as Oracle, MogDB, DB2, and MySQL. Currently, the node type supports Oracle, MogDB, MySQL, PostgreSQL and SQL Server. |
| Node name | Customized name (composed of 2 to 30 letters, digits, underline, and strigula) which mast start with letters or digits. |
| IP | Listening IP address of the database service |
| Port | Listening port of the database service |
| Service name/Database name | Database service name, SID, or database name. |
| Service name/Database name type | Oracle connection type. Only Oracle supports service name or database name type selection. The value can be ServiceName or SID. |
| Username | Database connection user |
| Password | Password of database connection user |
| Number of associated channels | Number of associated channels |
| Update time | Operation time |
Channel
In the MDB system, channel is a logical unit and used for connecting a source node and a target node through a series of operations. Data replication, scheduling, error processing, performance adjustment, and other operations generate in the channel. MDB synchronizes data in the channel from a source data to a target data.
Currently, a channel supports only a source node and a target node, that is, one-to-one data synchronization and replication.
The channel-related concepts are as follows.
| Concept | Description |
|---|---|
| Channel name | Customized name (composed of 2 to 30 letters, digits, underline, and strigula) which mast start with letters or digits. |
| Source node | Used for data capturing or data synchronization |
| Target node | Used for synchronizing data. The data captured from the source node will be synchronized to this node. |
| status | Current channel status, including: Initialization Pre-check Structure synchronization Data synchronization |
| Number of abnormal objects | Number of objects to be synchronized on the channel |
| Last update time | Operation time |
Object
In the MDB system, object is a basic replication unit. It can be table, sequence, view, or synonym.
The object-related concepts are as follows.
| Concept | Description |
|---|---|
| Source schema | Schema name of the source node (username in Oracle and database name in MySQL) |
| Source name | Object name of the source node, which is selected in the object selection operation |
| Object type | Object type in the source database |
| Target schema | Name of the schema to be synchronized to the target node |
| Target name | Name of the object to be synchronized to the target node |
| Field change | Number of field names to be changed in the target node during field mapping configuration |
| Whether to skip structure | Whether to synchronize the object's structure |
| skip the full amount or not | Select whether to synchronize the full data of the object |
| Skip increments or not | Select whether to synchronize the incremental data of the object |
| Synchronization status | Displays the current synchronization status of the object |
Field Mapping
Table structure mapping of a node displays the table field, data type, precision. Schema name, table name, and field name of a target node can be modified. After being modified, they will not change based on rule mapping configuration.
Field mapping-related concepts are as follows.
| Concept | Description |
|---|---|
| Source node | Table structure of a source node |
| Target node | Table structure of a target node |
| Field name | Column name of a table, which can be modified in the target node |
| Type | Data type |
| Precision | Data length |
| Scale | Data scale |
| Allowed to be null | Whether to allow the data to be null (Y, N) |
Rule Mapping
The schema or object name changed in a channel in the target node does not take effect for scheme name or table name changed in field mapping in the target node.
The rule mapping-related concepts are as follows.
| Concept | Description |
|---|---|
| Schema of a source node | String. Schema name of the source node, which is case-sensitive (Upper case in Oracle, lower case in MogDB, MySQL, PostgreSQL and SQL Server) |
| Schema of a target node | String. Schema name of the target node, which is case-sensitive (Upper case in Oracle, lower case in MogDB, MySQL, PostgreSQL and SQL Server) |
| Replacement rule | Object mapping rule type: Character replacement: Replace the specified string on the target node with that in the object name of the source node. Regular replacement: Replace the specified string on the target node with that conforming to the regular expression in the object name of the source node. |
| Search in a source node | String or regular expression. Searching the object of the source node |
| Replacement in the target node | String. Replacing the content searched from the source node on the target node |
Channel-Advanced Parameters
In the MDB system, advanced parameters control the logic of the channel synchronization process. For example, skipping snapshots, synchronizing only incremental data, and overriding the existing objects on the target node.
The channel advanced parameter-related concepts are as follows.
| Concept | Description |
|---|---|
| Parameter | Advanced parameter name |
| Parameter value | Default value of a parameter, which can be modified |
| Parameter description | Parameter description |
Channel-Operation
In the MDB system, operation controls task synchronization, configuration check, object synchronization, data capturing and writing, synchronization task start and pause.
Synchronization Task
- One-key synchronization: Perform synchronization from the channel of the current status until the incremental data is synchronized.
- Pre-check: Check environment before synchronization, including node connection, character set, time zone, supplementary logs, database configuration, table conflict, and naming conventions.
- Structure synchronization: Executes object structure synchronization, including Schema, Table, Index, Constraint, Synonym, Sequence, View, Trigger, Procedure, Function, and Package
- Structure Synchronization reset: Reset the structure synchronization operation will delete the structure synchronization progress and records of all tables in the channel
- Data synchronization Pause: Pause data synchronization, stop data extraction and data delivery services
- Data synchronization Start: Start data synchronization, perform data synchronization, restore data extraction and data delivery services
- Data synchronization reset: Reset Data synchronization, the data synchronization progress and records for all tables in the channel are deleted.
- Capture pausing: Pause data capturing.
- Integrate pausing: Pause data integrating.
- Capture starting: Restore data capturing.
- Integrate starting: Restore data integrating.
Run Log
Run log describes the data capturing and target node writing process after data synchronization.
Pre-check detail
Concepts related to check item execution failure and warning during pre-check are as follows.
| Concept | Description |
|---|---|
| check item | The name of the check item |
| Check description | Check item description |
| check time | Check time |
| Status | Success, failure, Warning |
Abnormal Object Synchronization
Concepts related to the failure information during object synchronization are as follows.
| Concept | Description |
|---|---|
| Object type | Schema, table, index, constraint, synonym, sequence, view, trigger, procedure, function and package |
| Schema of the source node | Name of the schema on the source node |
| Name of the source node | Name of the object on the source node |
| Schema of the target node | Name of the schema on the target node |
| Name of the target node | Name of the object on the target node |
| Migration time | Object synchronization time |
| Synchronization result | Failure and warning |
| Operation | Re-execute the SQL on the target node |
| SQL on the target node | SQL executed on the target node, which supports editing, formatting, and replicating |
| Error information on the target node | Error information reported when SQLs are executed on the target node |
Task Synchronization Details
Concepts involved in the task details are described as follows:
| Concept | Description |
|---|---|
| Data capture | Capture data from source node, status: normal, pausing and abnormal |
| Data integrate | Integrate data to target node, status: normal, pausing |
| Schema of the target node | Name of the schema on the target node |
| Table of the target node | Name of the Table on the target node |
| Snapshot count | The snapshot number |
| Snapshot Synchronization status | Status: prepare, In progress, skip, complete, pausing and abnormal |
| Incremental count | Incremental data volume |
| Incremental Synchronization status | Status: normal, pausing and abnormal |
Data validation task details
Concepts involved in the task details are described as follows:
| Concept | Description |
|---|---|
| Schema of the source node | Name of the schema on the source node |
| Table of the source node | Name of the table on the source node |
| Schema of the target node | Name of the schema on the target node |
| Table of the target node | Name of the table on the target node |
| Job Type | Type: MVD |
| Source table total | Number of source table records |
| Source difference rows | Number of difference records between source table and target table |
| Target table total | Number of target table records |
| Target difference rows | Number of difference records between target table and source table |
| Start time | Task start time |
| End time | Task end time |
| Task status | Status name：NOT_STARTED、 PENDING、 RUNNING、 FINISHED、 ERROR、STOPPED、WAITING |
| Result status | Status name： Data consistency, Data inconsistency, Parsing failure |
Failure Data Details
Concepts involved in the task details are described as follows:
| Concept | Description |
|---|---|
| Source | Name of the schema.table on the source node |
| Target | Name of the schema.table on the target node |
| Before data | Before data |
| After data | After data |
| Operate Type | Type：FULL_SYNC、INSERT、 UPDATE、 DELETE |
| Cause of exception | Synchronization failure cause |
| Operate time | Operate time |
| Operate | Discard: No synchronization is performed. Retry: Synchronize again |