Release Notes

The functionality of this tool is integrated into the MDB Heterogeneous Migration Synchronization Suite, so it is recommended to use the MDB product directly.

v5.6.1 (2024-03-21)

Improvements

Add suplemental object collection during Oracle SQL collection task
Sending callback before program abnormal exit and automatic cleanup sub-processes
Adjust MySQL collection SQL for MariaDB
Move some rewrite rules from Oracle to common

Bugfix

Fix table querying error in Oracle 12.1
Fix parsing error for database(schema) name which contains special chars in MySQL collection
Fix error with type conversation (::numeric) in MogDB B compatibility
Fix schema creation error in target database
Fix SQL rewrite does not running in MySQL compatibility analysis
Fix keyword (COLLATION) error in MySQL 8.0 data collection
Set UTF-8 encoding when write MySQL result file, avoid chinese mistake
Fix SQL error when use distinct combined with group/order by in target object creation

v5.6.0 (2023-07-03)

Features

Add new task D, can remove data in repository with given data_id or data_id list

Improvements

Use connections with autocommit mode to do initial and deletion tasks
Support database schema and user separation, need to re-initialize the repository database
Supports the initial upgrade of the database and automatically determines whether to perform the upgrade according to the database version number

Bugfix

Fix the operation exception caused by repeated password decryption
Fixed an issue where the superuser password was not decrypted during initialization

v5.5.2 (2023-06-15)

Features

MDB adaptation, add --passwd_encrypt option, provide password decryption
Offline HTML report update, support MySQL database portrait, built-in support for Japanese language
Support parsing and analysis of SQL files based on csv/Excel format

Improvements

Change data_id acquisition logic to get from sequence to avoid the problem of data_id reuse in MDB scenario
Adjust Oracle MTK object collection: built-in configuration file, adjust executable path lookup order
Add redundant filtering conditions when obtain the overall objects compatibility in Oracle report
Database portrait code optimization, add basic information pie chart, object information pie chart and histogram data
Optimize the classification of database portrait, and separate the column into a menu item to avoid too much table-related data
Move OS collection configuration variables to the AgentConfig configuration file
Automatically identify the files under the collection configuration when importing data and import them in reverse order of file size
When displaying the collection results, do not show the results of the file "not found"
Proactively send error message callbacks before exit (sys.exit) of collection
Adjusted Excel SQL exception summary statistics logic to identify error reports that aggregate more objects that do not exists
MySQL/Oracle HTML report adjustment to ignore errors related to non-existent objects in percentage statistics
MySQL SQL analysis with 3-step strategy (only do SCA replacement, do syntax conversion, do NULL value replacement)

Bugfix

Fix the error reported when there is no ID process number in the slow log in some cases of MySQL 5.5, add some time formatting logic
MySQL slow log parsing using backslashreplace to deal with garbled code decoding problem, avoiding garbled code resulting in collection errors.
Adjust the length of sca_os_network.nic_mac_addr column from 32 to 128 to fix the error caused by insufficient length.

v5.5.1 (2023-04-04)

Features

Add option --disable-rowid to control whether to disable rowid-related conversion and rewriting rules
Add automatic rowid judgment logic to automatically determine whether to disable rowid conversion based on whether target database supports rowid or not without actively using --disable-rowid
Add index evaluation collected in MTK
Add option --callback to support getting PID, task progress and resource exceptions through callbacks

Improvement

In the HTML report of Oracle compatibility, the value of head summary compatibility does not consider the unchecked object and does not consider the error of object does not exist
Optimize temporary directory location of MTK task, use unpacked directory to avoid overlapping temporary directories of multiple tasks
Optimize the unpacked code directory detection to avoid mutual interference when multiple processes are running
According to the custome production data, add new rewriting rules and optimize existing rewriting rules
Optimize the collection background monitoring, only kill the session after 3 consecutive abnormalities (abnormalities may last 6-9 seconds)
Rewrite rules use single line matching mode to avoid inaccurate matching at the SQL beginning and end
Optimize Oracle SQL compatibility check logic: first check the SQL without conversion, then check the converted SQL, and finally check the SQL after the same replacement of binding variables
Optimize the memory usage of Oracle SQL compatibility checking to avoid excessive memory consumption due to one-time return of large data volume
Add some PLSQL abnormal to normal determination
Optimize MTK object compatibility check: considered of object dependencies
In the process of SCA object complementation, optimize the detection expression for sequences in SQL, ignoring case and conditional qualification
Optimization of Excel report to not display succeed SQLs or objects for packages with large amount of data
Distinguish between functions and stored procedures when building SCA objects, and use "create procedure" to create stored procedures
Support analysis of user-defined renamed zip packages (zip package name is different from the folder inside the zip package)
Oracle compatibility optimization: consider using explain, consider the detection of SQL when no syntax rewriting is done
When Oracle query encounters decoding error, use backquotes plus hexadecimal to replace undecodable characters instead of reporting error directly

Bugfix

Fix the execution error caused by PLSQL objects creation collected by MTK PLSQL, which are ending with /
Update the front-end resources of offline HTML report to fix the problem that character fields cannot be sorted by clicking on the title
Support the capture of Oracle MTK collection exceptions, and MTK collection exceptions do not corrupted the whole collection
Repair the problem of insufficient length of view name and PLSQL text field in the database
Repair the problem of no data in rewriting SQLs in Excel report and fix query error in MySQL Excel report
Repair the error caused by getting CPU frequency in some scenarios (in Docker)
Repair the problem that the software upgrades itself with an alphabetic version number
Repair the problem that the beta version number gives error when generating Oracle database portrait report
Repair the problem that the number of MogDB executions and rows in the performance comparison details in Oracle report is 0
Repair the problem that the missing control.dat causes incomplete data and report error (divisor is 0) after the abnormal CTRL-C
Fix the problem of packing directory error
Fix the problem of insufficient length of sca_ora_users.password_versions field in the database

v5.5.0 (2022-12-14)

Features

Support DB potrait data collection, analysis and display in HTML report (only Oracle)
Support Oracle object DDL collection by inline MTK, and do object compatibility analysis/show base on it

Improvements

Replace SQL Complexity to Object Compatibility in HTML report (only support in Oracle source, other databases will supported soon)
Default directory of license file can be detected automaticly between current path and program path

Bugfixs

Fix error of is_hidden column missing in sys.columns dictionary table with MSSQL version below 13.0
Fix rewrite sheet error in Excel data report with MSSQL database
Fix data missing error on SQL detail page in multiple language environment

v5.4.0 (2022-11-13)

Features

Add SQL Server data collection and sql compatible analysis

Improvements

Expand column size of sca_database.db_version to fit original SQL Server version text

Bugfixs

Fix the error caused by missing DEFAULTD field in Oracle version 10.2 PLSQL collection

v5.3.2 (2022-10-24)

Features

Add option '--report-lang' to specify report languages, reports with different languages can be created at the same time

Improvements

Add a lot of transformation rules for MySQL database
License Certification optimization (Use License v31), older license file does not support any more
Add complexity number in SQL detail pages in Oracle Excel report
Adjust the criteria for Oracle data dictionary statements to determine the query statement of 'sys.|mdsys.|system.' as a dictionary query

Bugfixs

Fix the error caused by insufficient length of the mogdb_message field in the sca_sql_result table
Adjust Oracle performance comparison SQL, fix some logic problems

v5.3.1 (2022-09-23)

Features

Add -d/--data to specify multiple data packages to merge and analyze data that collected from same database.
Support for automaticly running compat-tools scripts on target database when the 'gsql' command is available locally, which can help to increase compatibility

Improvements

Logging updated, reducing some of the log output more than necessary
Partial code logic fine-tuning, global parameter fine-tuning

Bugfixs

Remove rewrite rule for openGauss/MogDB: systimestamp function without parentheses

v5.3.0 (2022-08-30)

Features

Support informix: including both SQL and objects collection, and adjust the structure of the repository.
Add "--upgrade" command line option to support online upgrade.

Improvements

Add OBJECT_ID field to PLSQL object information, compatible with function overloading
PLSQL rebuild logic adjustment, adapt to Informix database
Optimize MySQL slow log collection, increase the collection efficiency of larger slow logs.
Add the recognition of time data in DD/MM/YYYY HH24:MI:SS format (Oracle bind data).
Support the red mark of SQL rewriting part in Excel, and modify some regular matching rules.
Remove row_number() for Oracle collection SQL, which affects the collection performance
Rewrite rules adjustment, Oracle/DB2 uppercase to lowercase exact match

Bugfixs

Fix the problem of abnormal naming of custom SQL parsing in DB2.
Fix the problem that the default CON_ID=0 can not query data when querying SQL details in PDB after 12c.
Fix the problem that MySQL data cannot be collected when "--slow-log" is specified.
Fix the problem of performance comparison when the target is the PostgreSQL.
Capture non-database exceptions in check_sql in openGauss/PG to avoid the impact on the overall analysis task.

v5.2.3 (2022-06-20)

Features

None

Improvements

Optimize the display of floating point numbers to show the 0 before the decimal point between 0 and 1 in Oracle simulation report
Optimize the logic of permission checking in Oracle collection task
Optimize the performance of query SQL for Oracle SQL complexity analysis data.

Bugfixs

Fix the abnormal data problem in Oracle data collection where the procedure in package has no parameters.

v5.2.2 (2022-06-13)

Features

Add "--steps" option to support specifying each step in the runtime analysis process.
Add "--sql-config" option to support customizing the SQL used in each task (advanced feature, for internal use only).
Add "--ignore-bind-plan" option to ignore the collection of Oracle SQL binding variables and execution plans, this can avoid the bug of parsing some dictionary datas.
Add detailed data report with Excel format (including some summary pie chart, detailed SQL list and the list of SQL involved in each rewrite rule), and remove the old version of csv detailed file.
Support "-d" to specify zip files directly, so zip files or unzipped directories can be executed.

Improvements

Update the SQL that ends with "already exists" as supported.
Change the rule name, and use "SCA_" for SCA-related rewrites, and mask them in the reports.
Significantly add and modify Oracle SQL rewriting rules to achieve more accurate and extensive matching by using data from actual customer environments.
Remove the conversion rule of "user" to "current_user()".
License adjustment, add version,type property, feature field restriction to sca only.
Oracle collection: give specific error message and quit acquisition when user list is not detected.
Object completion adjustment: Debug mode does not clean up the completed objects to facilitate debugging.
In the object completion module, create Schema uniformly and add the creation and rollback for sequences appearing in SQL.
Remove the logic of data case flipping in double quotes in stmt_transformer function, and use POST regular rules to handle case uniformly.
Optimize the case problem during object reconstruction, and add the handling of mixed case names.

Bugfixs

Fix the problem that rollback affects search_path in Oracle performance simulation.
When rebuilding objects, functions are uniformly rebuilt as IMMUTABLE functions to avoid errors caused by function attributes in some cases, e.g.: [42703] Alias "label" reference with volatile function included is not supported.
Fix the error caused by object name without double quotes when object is deleted.
Repair the error reported by Oracle in handling bind variables when no information of bind variables is collected.
Repair the error of loading SQL information, due to the over-length of module and action column in some Oracle scenarios.
Repair the problem of missing column and plsql definition data in the absence of control.dat file.
Repair the problem of object collection error caused by abnormal data acquisition of MySQL version.
Repair the problem of object collection error when MySQL only collects from specified slow logs.
Repair the problem of incorrect length of progress bar in SQL simulation task with force mode.
Fix object completion problems: Create schema problem, progress data query problem, non-package function creation error problem.
Add rules and test cases for adding brackets to functions without parameters, adjust some rules with abnormal test cases, and ensure that all test cases of rules are normal.
Replace single quotes with double single quotes for character type variable values when binding variables are parsed.

v5.2 (2022-05-10)

Features

add --sql-modified option to support manual adjustment of the SQL to be executed in SQL simulation
SQL compatibility evaluation supports breakpoints and linkage with forced mode.

Improvements

In various analysis tasks, add the summary information output after the analysis is completed
Add the error details output of copy_db function in openGauss/MogDB database module
Adjust the SQL_Normalize function to support the SQL normalization operation by database
Adjust the scope of object completion logic to execute only in the tasks where SQL compatibility evaluation is necessary
Use Python custom function to calculate SQL_HASH value for SQL with FMS=0 during Oracle SQL collection to reduce the probability of partial duplicate DML

Bugfixs

Fix the problem that the data copy to database in csv format is wrong due to the presence of double quotes in the data
Fix some abnormalities caused by NULL value and empty string judgment in the database
Fix the compatibility analysis error caused by the empty Package_name information
Fix the problem of misjudgment of keyword rewriting rules when Oracle SQL rewriting (V$VERSION => V$"VERSION")

v5.1.0

Features

Add source database support: PostgreSQL, whose SQL data is collected from the pg_stat_statements plugin
Add object information collection: All source databases will automatically collect field information and PLSQL parameter information, and automatically create them for use in SQL compatibility audits.

Bugfixs

Fix the error reporting problem that the progress bar output will be abnormal due to the queue problem in some cases
Fix the problem that performance_schema is not recognized as SYSTEM_CATALOG during MySQL collection
Fix the problem that the system built-in objects are not filtered when MySQL object information is collected
Fix the error caused by the -D option after replacing the openGauss driver due to a type binding problem
Fix the problem of failing to verify the expiration time of license when verifying license.
Fix the problem that the SQL text contains the line starting with # in the slow log file, which causes the error in parsing.
Fix the problem of abnormal sorting of table data when the source database is Oracle in the SQL compatibility summary page.
Fix the problem that offline report cannot be displayed in some browsers due to JS problem.
Fix the abnormal exit caused by the failure to parse ID data when MySQL slow log collection.
Fix the problem that the --debug option is not recognized and requires specified value.
Fix the problem of Oracle data collection character set error reported in some scenarios ORA-29275 (character set conversion + splicing empty string)
Fix the error reported in Oracle's report for source database due to the GaussDB database does not have median function, resulting in complex data query.

Improvements

Set search_path to include all schema after connecting to the target database to reduce the probability of not finding tables in SQL compatibility analysis process.
Add SQL rewriting rules, add some SQL rewriting rules for Oracle database.
Adjust the detailed SQL list in csv format, use Chinese table header and part of Chinese content, add support category field, easy to read and understand.
Adjust the logic of SQL compatibility summary page data, NULL related rewriting is not considered as rewriting support.
Adjust the MySQL data acquisition logic, strictly follow the FILE/TABLE setting in log_output to get data.
Driver and supporting code adjustment, use psycopg2 driver for PostgreSQL database, use py_opengauss driver for openGauss database.
Remove some redundant format_size functions and their calls

v5.0.0

Feature

Code logic refactoring, command options and usage have changed considerably
Support MySQL connecting database collection, through the database table of slow log information, or through the server-side local slow log
Support DB2 data collection, collection mode is similar to Oracle, need to run for a long time, the default collection time is one week
Supports DB2 SQL compatibility analysis, provided that the target MogDB database needs to be prepared in advance
Support for export of Oracle SQL detail list files (sql_detail_list.csv)
Adjust Oracle data collection logic, remove unnecessary object information and system configuration information
Adjust the repository structure, only keep the basic information of the source database, SQL compatibility audit, SQL performance comparison and other related structures

v4.1.0

Feature

Support SQL compatibility analysis of the MySQL database based on the slow log and general log.
Support customization of SQL conversion rules.

v4.0.0

Feature

Integrate a newly designed offline analysis report.
Support data collection, compatibility analysis, and performance assessment of a single executable program.
Support the SHA256 encryption authentication mode of openGauss/MogDB in the Linux OS.