HomeMogDBMogDB StackUqbar
Ecological Tools
Doc Tools
中文
v2.0

Documentation:v2.0

Supported Versions:
Current(v2.0)

MDB Installation

Requirements

The system in which MDB is to be deployed must meet the following requirement.

  • Server: Linux x86_64
  • Repository: MogDB
  • Disk space: 500 G

Supported Version

Software Version
Java 11+
MogDB Database: 2.0+ Plug-ins: wal2json
Oracle 11g, 12c, 19c
MySQL 5.7, 8.0.x (partial compatible for MariaDB > 10.4)
PostgreSQL Database: 9.6, 10, 11, 12, 13, 14 Plug-ins: decoderbufs, wal2json
SQL Server 22016 Service Pack 1 (SP1) and later

Installation Procedure

Download Software

Download the latest MDB installation package： MDB Latest

Upload and Decompress the Software Package

Upload the MDB software package to the server and decompress it.

tar -zxvf mdb_v1.0.0.RELEASE_linux_x86_64.tar.gz

Prepare a Repository

The MDB repository must use a MogDB (openGauss) database, and the version cannot be earlier than 2.0.

For details about how to install MogDB, see MogDB Single Node Installation.

After MogDB is installed, use gsql to log in to the database as a super administrator.

gsql -r

In MogDB, create an MDB repository and an MDB repository user.

-- Create a repository user with the password changed based on the actual requirement.
create user mdb_repo with password 'MDB@password' sysadmin;

-- Create a repository with the repository owner that must be set to mdb_repo.
create database mdb_repo_db with owner mdb_repo dbcompatibility='A' ENCODING='UTF8';

Check the Java Version

MDB depends on Java 11. Therefore, make sure that the Java version is not earlier than 11 in the current server.

java -version

If the Java version is earlier, you can download the software package newer than openJDK 11 from the official website of Java. The download link is as follows

https://jdk.java.net/java-se-ri/11-MR3

Download the openjdk-11.0.0.2_linux-x64.tar.gz software package, and upload it to the server and decompress it.

tar -zxvf openjdk-11.0.0.2_linux-x64.tar.gz

# Obtain the current path: /usr/local
pwd 

# Configure environment variables. 
vi ~/.bash_profile

# Replace the JAVA_HOME path and then add the following content to the end of the file. 
export JAVA_HOME=/usr/local/jdk-11.0.0.2
export CLASSPATH=$:CLASSPATH:$JAVA_HOME/lib/
export PATH=$PATH:$JAVA_HOME/bin

# Save the modification and run the following command to update the configuration file. 
source ~/.bash_profile

Confirm the Java version and make sure that Java 11 is used.

java -version

# The following shows the command output: 
# --------------
# openjdk 11.0.0.2 2024-07-02
# OpenJDK Runtime Environment 18.9 (build 11.0.0.2+2-2)
# OpenJDK 64-Bit Server VM 18.9 (build 11.0.0.2+2-2, mixed mode)

Check the Web Port

MDB Server requires a web listening port (updatable, default: 55437). During installation, make sure that the port is available.

netstat -anp | grep LISTEN | grep 55437    # Make sure that no process is in the listening status.

Check the Residual File

For ZooKeeper and kafka installation, make sure that the <tempFilePath>/zookeeper and <tempFilePath>/kafka-logs directories do not exist. If they are residual files, run the following commands to delete them.

rm -rf <tempFilePath>/zookeeper # <tempFilePath>: MDB configuration item, default: ../mdb_data
rm -rf <tempFilePath>/kafka-logs

Modify Installation Settings

Switch to the decompressed MDB software package directory and check the file integrity.

cd mdb_v1.0.0.RELEASE_linux_x86_64

ls -ltr

The following describes the software package files.

File Description
kafka.zip Kafaka installation package used for synchronizing middle data queue
mdb MDB command execution file
mdb-data-worker.zip Software package related to data collection and delivery tasks
mdb-server.zip MDB main software package
snapshot-worker snapshot sync software package
mvd_package MVD command execution file
mvd-java.zip MVD software package
config.json Installation and configuration file
connectorlog.zip Software package related to connector log
mtk required to download mtk
sca required to download sca

Modify the config.json configuration file.

{
  "mdbServerPort": 55437,
  "dbHost": "127.0.0.1",
  "dbPort": 26000,
  "dbUser": "mdb_repo",
  "dbPassword": "MDB@password",
  "dbName": "mdb_repo_db",
  "dataFilePath": "../mdb_data",
  "tempFileMaxSize": 0,
  "tempFileRetentionTime": 168,
  "loadingSeconds": 300,
  "messageMaxSize": 50,
  "mvdServerPort": 9876
 }

The following describes the configuration items.

Configuration Item Default Value Description
mdbServerPort 55437 MDB server port
Used for access the web page. Make sure that the network between the MDB server and the web client is normal.
dbHost 127.0.0.1 Listening IP address of the repository
dbPort 26000 Listening server port of the repository
dbUser mdb_repo MDB repository user
dbPassword MDB@password MDB repository password
dbName mdb_repo_db MDB repository database name
dataFilePath ../mdb_data MDB data file
tempFileMaxSize 0 MDB temporary file size (unit: GB). Default value 0: Unlimited size
tempFileRetentionTime 168 Retention time of MDB temporary files (unit: hour)
loadingSeconds 300 Service startup wait time （unit: second）
messageMaxSize 50 topic message max size（unit: M）
mvdServerPort 9876 MVD server port

Configure ulimit

During the data synchronization process, some temporary data files need to be stored in the local file system of MDB. For tables containing LOB fields, LOB data files will also be stored additionally. It is necessary to adjust the maximum limit of open handles for the current MDB user to avoid the number of open file handles exceeding the default limit during the synchronization process.

# Note: The following command lifts the restrictions for all users. You can modify the '*' to the MDB installation user to only lift the MDB restrictions, and re-login to take effect.
cat >> /etc/security/limits.conf <<EOF

* soft nofile 10485760
* hard nofile 10485760
EOF

# Modify the total file limit of the whole system
cat >> /etc/sysctl.conf <<EOF

fs.file-max=167772160
EOF

sysctl -p

Configure MTK and SCA (Optional)

pwd         # Make sure that you have switched to the mdb_v1.0.0.RELEASE_linux_x86_64 directory.

# Download sca_linux_x86_64 to the mdb_v1.0.0.RELEASE_linux_x86_64/sca directory
cd mdb_v1.0.0.RELEASE_linux_x86_64/sca
chmod u+x sca_linux_x86_64

# Download mtk_2.8.3_linux_amd64.tar.gz to mdb_v1.0.0.RELEASE_linux_x86_64 directory
cd mdb_v1.0.0.RELEASE_linux_x86_64
tar -zxvf mtk_2.8.3_linux_amd64.tar.gz
mv mtk_2.8.3_linux_amd64/* mtk

Install MDB

Check and make sure that you have switched to the MDB software package directory.

pwd         # Make sure that you have switched to the mdb_v1.0.0.RELEASE_linux_x86_64 directory.

Install MDB.

./mdb install

The output is as follows:

   2024-04-30 16:37:48  ======BEGIN TO INSTALL MDB======
   2024-04-30 16:37:48    1.CHECK MDB FILE
   2024-04-30 16:37:48    2.CHECK MDB ENVIRONMENT
   2024-04-30 16:37:48     JAVA_VERSION: 11.0
   2024-04-30 16:37:48    3.UNZIP KAFKA
   2024-04-30 16:37:48    4.INSTALL MDB SERVER
   2024-04-30 16:37:49    5.INSTALL MDB DATA WORKER
   2024-04-30 16:37:50    6.INSTALL MDB CONNECTOR LOG
   2024-04-30 16:37:50    7.INSTALL MVD SERVER
   2024-04-30 16:37:51  

   ======END UPGRADE MDB RESULT======
   +------------+-----------+-------------+---------------+--------------------+
   | File_Check | Env_Check | MDB_Install | KAFKA_INSTALL | MVD_Server_Install |
   +------------+-----------+-------------+---------------+--------------------+
   |  Success   |  Success  |   Success   |    Success    |      Success       |
   +------------+-----------+-------------+---------------+--------------------+

Apply for a LICENSE

Apply for a license.

 ./mdb generate-license test@enmotech.com      #test@enmotech.com is a email address for receiving the license file which needs to be uploaded to the server.

The output is as follows:

   2022-07-07 14:06:22  The email has been successfully sent to test@enmotech.com. Please check your email.

Copy the LICENSE to the Server

Copy the license to the server.

 ./mdb copy-license /u01/mdb/license.json      #/u01/mdb/license.json is the path of the license file on the server.

The output is as follows:

   2022-07-04 11:26:01  copy license success

Start MDB

Start MDB.

 ./mdb start

The output is as follows:

   2024-04-30 16:21:09  start zookeeper success
   2024-04-30 16:21:09  start kafka success
   2024-04-30 16:21:09  start connector success
   2024-04-30 16:21:09  start mdb-server success
   2024-04-30 16:21:09  start connectorlog success
   2024-04-30 16:21:09  [MVD_SERVER] start mvd-server success
   2024-04-30 16:21:09  After MDB is started, the the ports occupied by programs are as follows: 
   +-------+-----------+-----------+------------+------------+
   | KAFKA | ZOOKEEPER | CONNECTOR | MDB_SERVER | MVD_SERVER |
   +-------+-----------+-----------+------------+------------+
   |  9092 |    2181   |    8083   |   55437    |    9876    |
   +-------+-----------+-----------+------------+------------+
   2024-04-30 16:21:10  MDB program service status
   +--------+-----------+-----------+------------+------------+
   | KAFKA  | ZOOKEEPER | CONNECTOR | MDB_SERVER | MVD_SERVER |
   +--------+-----------+-----------+------------+------------+
   | online |   online  |   online  |   online   |   online   |
   +--------+-----------+-----------+------------+------------+

Start child service command:

 ./mdb start --help 
 #  --server_name TEXT  Optional service name: zookeeper、kafka、connector、mdb_server、mvd_server
 ./mdb start --server_name zookeeper 
 ./mdb start --server_name kafka 
 ./mdb start --server_name connector 
 ./mdb start --server_name mdb_server
 ./mdb start --server_name mvd_server

Check the MDB Server Status

Check the MDB server status.

 ./mdb status

The output is as follows:

   2022-06-23 14:06:39  MDB program service status
   +--------+-----------+-----------+------------+------------+
   | KAFKA  | ZOOKEEPER | CONNECTOR | MDB_SERVER | MVD_SERVER |
   +--------+-----------+-----------+------------+------------+
   | online |   online  |   online  |   online   |   online   |
   +--------+-----------+-----------+------------+------------+

Stop MDB

Stop MDB.

 ./mdb stop

The output is as follows:

   2024-04-30 16:34:17  stop connector success
   2024-04-30 16:34:27  stop kafka success
   2024-04-30 16:34:37  stop zookeeper success
   2024-04-30 16:34:47  stop mdb-server success
   2024-04-30 16:34:47  stop mdb-data-worker success
   2024-04-30 16:34:47  stop connector_log success
   2024-04-30 16:34:57  stop mvd-server success
   2024-04-30 16:34:59  MDB program service status
   +---------+-----------+-----------+------------+------------+
   |  KAFKA  | ZOOKEEPER | CONNECTOR | MDB_SERVER | MVD_SERVER |
   +---------+-----------+-----------+------------+------------+
   | offline |  offline  |  offline  |  offline   |  offline   |
   +---------+-----------+-----------+------------+------------+

Stop child service command:

 ./mdb stop --help 
 #  --server_name TEXT  Optional service name: zookeeper、kafka、connector、mdb_server、mvd_server
 ./mdb stop --server_name zookeeper 
 ./mdb stop --server_name kafka 
 ./mdb stop --server_name connector 
 ./mdb stop --server_name mdb_server
 ./mdb stop --server_name mvd_server

Apply Configuration

When the config.json configuration file is modified, run the following command to update configuration files of all services. Before update, make sure that the service is offline (You can perform the stop command to make the service offline).

Apply the configuration.

 ./mdb apply-config

The output is as follows:

   2022-08-08 14:54:34  apply config success!

Log collection

Generate a dump.zip file in the current directory.

Log collection.

 ./mdb dump

The output is as follows:

   2022-09-23 10:33:46  dump log success

Login to MDB

After MDB is started, access the client browser (Google Chrome is recommended) to check whether the MDB interface is normal.

http://localhost:55437/mdb/index.html

img

HOME

Issue
Copyright © 2011-2025 www.enmotech.com All rights reserved.