- 简介
- 环境依赖
- 快速上手
- 配置文件
- 命令介绍
- mtk
- init-project
- config
- license
- mig
- show
- sync
- sync-schema
- sync-sequence
- sync-object-type
- sync-domain
- sync-wrapper
- sync-server
- sync-user-mapping
- sync-queue
- sync-table
- sync-nickname
- sync-rule
- sync-table-data
- sync-table-data-estimate
- sync-index
- sync-constraint
- sync-db-link
- sync-view
- sync-mview
- sync-function
- sync-procedure
- sync-package
- sync-trigger
- sync-synonym
- sync-table-data-com
- sync-alter-sequence
- sync-coll-statistics
- check-table-data
- gen
- gen completion
- encrypt
- convert-plsql
- report
- self
- mvd
- usql
- 图形化
- 常见问题
- Release
使用MTK迁移MSSQL到MogDB/openGauss
支持版本
- 2017+
- 2016
- 2012
How to
-
下载安装MTK
tar -zxvf mtk_<version>_linux_<platform>.tar.gz
-
申请授权
cd mtk_<version>_linux_<platform> ./mtk license gen License File Not Found (default license.json) The License code is invalid, start applying ? Email: xxxx@xxxx.com -- 填写邮箱信息 >> please enter a valid email address # 复制邮箱里的文件`license.json`到此目录
-
准备MSSQL用户
-
MogDB 创建数据库和用户
gsql create database db DBCOMPATIBILITY='A' ENCODING='UTF8' LC_COLLATE='C' LC_CTYPE='C'; \c db create user mtk_mig with password "mtkMigAbc123" sysadmin;
注意兼容模式 注意字符集排序规则. 创建数据库语法参考create database
-
初始化项目
./mtk init-project -s sqlserver -t mogdb -n sqlserver2mg tree -f sqlserver2mg/ sqlserver2mg ├── sqlserver2mg/config │ └── sqlserver2mg/config/mtk.json ├── sqlserver2mg/data ├── sqlserver2mg/report └── sqlserver2mg/schema
编辑配置文件. 具体查看MTK配置说明章
vi sqlserver2mg/config/mtk.json
-
编辑
source
节点,定义源数据库连接信息. 配置参考source -
编辑
target
节点,定义目标数据库连接信息. 配置参考target修改connect和type信息,parameter根据运行在进行调整
-
编辑
object
节点,定义迁移对象. 配置参考object
编辑完成后,运行config-check检查配置文件是否正确
./mtk config-check -c sqlserver2mg/config/mtk.json
配置文件配置正常会输出类型信息
use config : sqlserver2mg/config/mtk.json There is no error in the configuration file
-
-
迁移
步骤 描述 命令 ./mtk mig-tab-pre -c sqlserver2mg/config/mtk.json 迁移表 mig-tab-pre ./mtk mig-tab-data -c sqlserver2mg/config/mtk.json 迁移表数据 mig-tab-data ./mtk mig-tab-post -c sqlserver2mg/config/mtk.json 迁移索引约束 mig-tab-post ./mtk mig-tab-other -c sqlserver2mg/config/mtk.json 迁移存储过程PLSQL相关 mig-tab-other -
查看报告
报告和运行日志会生成在
ora2og/report
目录下. 如下[2022-01-20 11:29:25.800978] INFO reportDir: sqlserver2mg/report/report_20220120110125 function=GenReport line=412 file=mtk/cmd/mtk/services/cmd.go [2022-01-20 11:29:26.426822] INFO the text report : sqlserver2mg/report/report_20220120110125.txt function=PrintReport line=270 file=mtk/cmd/mtk/services/cmd.go [2022-01-20 11:29:26.429545] INFO the warring report : sqlserver2mg/report/report_20220120110125.warring function=PrintReport line=281 file=mtk/cmd/mtk/services/cmd.go [2022-01-20 11:29:26.430118] INFO the error report : sqlserver2mg/report/report_20220120110125.err file=mtk/cmd/mtk/services/cmd.go function=PrintReport line=292
文件名 说明 sqlserver2mg/report/report_20220120110125 html报告 sqlserver2mg/report/report_20220120110125.txt 文本报告 sqlserver2mg/report/report_20220120110125.warring 只包含警告信息的文本报告 sqlserver2mg/report/report_20220120110125.err 只包含错误信息的文本报告
注意事项
序列
Type
- 自定义类型. 查询SQL
表
- 分区表不支持迁移成分区表
列
生成列
表达式未转换
Gis
- Geometry 通过查询转换为WKT. colname.STAsText()
不支持的列
- GeoGraphy
- HIERARCHYID
- SqlVARIANT
- TimeStamp
约束
索引
- Spatial index 没有安装Postgis会报错.
不支持的索引
- XML index
- Clustered column store index
- Nonclustered column store index
- Nonclustered hash index
Trigger
- ON DATABASE 触发器不转换改写
- ON ALL SERVER 触发器不转换改写
- FOR XXX 触发器不转换改写
- INSTEAD OF XXX 改成为 BEFORE XXX
其他问题
TLS Handshake failed: tls: server selected unsupported protocol version 301
手工配置连接字符信息增加?encrypt=disable
{
"source": {
"type": "sqlserver",
"connect": {
"dsn": "sqlserver://username:password@host/instance?encrypt=disable",
}
},
"target": {
"type": "mogdb",
"connect": {
}
}
}
Latin1 字符集存中文
2.9.8
借助 VARBINARY
类型处理数据.
-
确定数据字符集编码. 找到存在中文的字段并确切知道中文内容,使用
cast(xxx as VARBINARY)
查看16进制编码select CAST(列名 AS VARBINARY(列长度)) from 表名
打开网站查看汉字编码输入汉字查看编码.和数据库查询出来结果进行对比. 如上面的是GBK编码
-
source.parameter 配置
"columnNameData": { "dbo.yp_supplyer.supplyer_name": "CAST({{.ColName}} AS VARBINARY(列长度))" }
-
target.connect.clientCharset 配置 步骤1 得的编码
-
target.parameter 配置
"columnNameData": { "dbo.yp_supplyer.supplyer_name": "convert_from({{.ColName}}::bytea,'字符集')" }
注意 可能列需要扩充长度
2.9.9
方案 1
-
确定数据字符集. 找到存在中文的列并确切知道中文内容,使用
cast(xxx as VARBINARY)
查看16进制编码select CAST(列名 AS VARBINARY(列长度)) from 表名
打开网站查看汉字编码输入汉字查看编码.和数据库查询出来结果进行对比. 如上面的是GBK编码
-
source.parameter 配置
"columnNameData": { "dbo.yp_supplyer.supplyer_name": "CAST({{.ColName}} AS VARBINARY(列长度))" }
-
target.connect.clientCharset 配置 步骤1 得的编码
方案2
-
确定数据字符集. 找到存在中文的字段并确切知道中文内容,使用
cast(xxx as VARBINARY)
查看16进制编码select CAST(列名 AS VARBINARY(列长度)) from 表名
打开网站查看汉字编码输入汉字查看编码.和数据库查询出来结果进行对比. 如上面的是GBK编码
-
查看数据库的编码ID和列编码ID
select schema_name(tab.schema_id) as schema_name, tab.name as table_name, c.name as column_name, collation_name, SERVERPROPERTY('Collation') AS Collation, SERVERPROPERTY('SqlSortOrder') AS SqlSortOrder from sys.columns c JOIN sys.tables tab ON tab.object_id = c.object_id where tab.name = '表名' and c.name = '列名'
确定下库级别的
Collation
和 列级别的collation_name
是否一致 -
source.connect.clientCharset 配置 步骤2 得的编码 SqlSortOrder
-
target.connect.clientCharset 配置 步骤1 得的编码
部分SQL语句
Schema
SELECT name from sys.schemas where 1=1
Sequence
2012
select schema_name(s.schema_id) as schema_name,
s.name as seq_name,
s.minimum_value as MINVALUE,
s.maximum_value as MAXVALUE,
s.[increment] as INCREMENT,
-- '' as ORDER,
s.is_cycling as CYCLE ,
s.cache_size as CACHE,
s.current_value as current_value,
s.current_value as LAST_NUMBER,
s.start_value as START,
s.[precision] as PRECISION
from sys.sequences s where 1=1
2017
select schema_name(s.schema_id) as schema_name,
s.name as seq_name,
s.minimum_value as MINVALUE,
s.maximum_value as MAXVALUE,
s.[increment] as INCREMENT,
-- '' as ORDER,
s.is_cycling as CYCLE ,
s.cache_size as CACHE,
s.current_value as current_value,
s.last_used_value as LAST_NUMBER,
s.start_value as START,
s.[precision] as PRECISION
from sys.sequences s where 1=1
Type
SELECT
s.name as schema_name,
t.name as type_name,
t.system_type_id,
t.user_type_id,
t.max_length,
t.precision,
t.scale,
s1.name as base_schema_name,
base.name as base_type_name,
t.is_nullable,
t.is_table_type,
t.is_user_defined,
t.is_assembly_type,
t.default_object_id,
t.rule_object_id
FROM sys.types as t
left join sys.schemas as s on t.schema_id = s.schema_id
left join sys.types as base on t.system_type_id = base.user_type_id
left join sys.schemas as s1 on base.schema_id = s1.schema_id
where t.is_user_defined=1
Table
2012
select schema_name(t.schema_id) as schema_name,t.name as table_name,0 as t.is_external,
0 as t.is_node,
0 as t.is_edge,
0 as t.temporal_type,
t.is_filetable,
ep.value,
ic.partition_ordinal
from sys.tables t
LEFT OUTER JOIN sys.extended_properties ep on ep.class = 1 AND ep.major_id = t.object_id AND ep.minor_id = 0 AND ep.name = 'MS_Description'
LEFT JOIN sys.indexes AS i ON t.object_id = i.object_id AND i.[type] <= 1
LEFT JOIN sys.index_columns AS ic ON ic.[object_id] = i.[object_id] AND ic.index_id = i.index_id AND ic.partition_ordinal >= 1
where 1=1
2016+
select schema_name(t.schema_id) as schema_name,t.name as table_name,t.is_external,
t.is_node,
t.is_edge,
t.temporal_type,
t.is_filetable,
ep.value,
ic.partition_ordinal
from sys.tables t
LEFT OUTER JOIN sys.extended_properties ep on ep.class = 1 AND ep.major_id = t.object_id AND ep.minor_id = 0 AND ep.name = 'MS_Description'
LEFT JOIN sys.indexes AS i ON t.object_id = i.object_id AND i.[type] <= 1
LEFT JOIN sys.index_columns AS ic ON ic.[object_id] = i.[object_id] AND ic.index_id = i.index_id AND ic.partition_ordinal >= 1
where 1=1
Partition
SELECT sch.name AS SchemaName,t.name AS TableName,i.name AS IndexName,p.partition_number,
p.partition_id,
i.data_space_id,
f.function_id,
f.type_desc as partitioningType,
r.boundary_id,
r.value AS BoundaryValue,
ic.column_id AS PartitioningColumnID,
c.name AS PartitioningColumnName,
s.name as partition_scheme_name,
f.name as partition_function_name,
f.boundary_value_on_right
FROM
sys.tables AS t
JOIN sys.indexes AS i ON t.object_id = i.object_id AND i.[type] <= 1
JOIN sys.partitions AS p ON i.object_id = p.object_id AND i.index_id = p.index_id
JOIN sys.partition_schemes AS s ON i.data_space_id = s.data_space_id
JOIN sys.index_columns AS ic ON ic.[object_id] = i.[object_id] AND ic.index_id = i.index_id AND ic.partition_ordinal >= 1
-- because 0 = non-partitioning column
JOIN sys.columns AS c ON t.[object_id] = c.[object_id] AND ic.column_id = c.column_id
JOIN sys.partition_functions AS f ON s.function_id = f.function_id
LEFT JOIN sys.partition_range_values AS r ON f.function_id = r.function_id and r.boundary_id = p.partition_number
LEFT OUTER JOIN sys.schemas sch ON t.schema_id = sch.schema_id
WHERE 1=1
Table Size
SELECT
s.Name AS TABLE_SCHEMA,
t.NAME AS TABLE_NAME,
p.partition_id ,
a.type_desc ,
p.rows AS RowCounts,
a.used_pages * 8 as UsedSpaceKB,
ic.partition_ordinal,
t.max_column_id_used
FROM
sys.tables t
INNER JOIN sys.indexes i ON t.OBJECT_ID = i.object_id and i.[type] <= 1 -- 只统计堆/聚集索引
LEFT JOIN sys.index_columns AS ic ON ic.[object_id] = i.[object_id] AND ic.index_id = i.index_id AND ic.partition_ordinal >= 1
INNER JOIN sys.partitions p ON i.object_id = p.OBJECT_ID AND i.index_id = p.index_id
INNER JOIN sys.allocation_units a ON p.partition_id = a.container_id
LEFT OUTER JOIN sys.schemas s ON t.schema_id = s.schema_id
WHERE t.is_ms_shipped = 0 AND i.OBJECT_ID > 255 AND t.type = 'U'
Column
2012
SELECT schema_name(tab.schema_id) as schema_name,tab.name as table_name,c.name as column_name,
c.column_id,
t.name as data_type,
c.max_length,
c.precision,
c.scale,
0 as c.is_hidden,
c.is_nullable ,
0 as c.is_masked,
COLUMNPROPERTY(c.object_id,c.name,'charmaxlen') as char_max_length,
dc.definition as default_definition,
ep.value as description,
cc.definition,
c.is_identity ,
ic.seed_value ,
ic.last_value ,
ic.increment_value,
t.is_user_defined
FROM
sys.all_columns c
left join sys.types as t on c.user_type_id = t.user_type_id
JOIN sys.tables tab ON tab.object_id = c.object_id
LEFT OUTER JOIN sys.default_constraints dc ON dc.parent_object_id = tab.object_id AND dc.parent_column_id = c.column_id
LEFT OUTER JOIN master.sys.extended_properties ep ON ep.class = 1 AND ep.major_id = tab.object_id AND ep.minor_id = c.column_id AND ep.name = 'MS_Description'
left join sys.computed_columns cc on cc.object_id = tab.object_id and cc.column_id = c.column_id
left join sys.identity_columns ic on ic.object_id = tab.object_id and ic.column_id = c.column_id
where 1=1
2016+
SELECT schema_name(tab.schema_id) as schema_name,tab.name as table_name,c.name as column_name,
c.column_id,
t.name as data_type,
c.max_length,
c.precision,
c.scale,
c.is_hidden,
c.is_nullable ,
c.is_masked,
COLUMNPROPERTY(c.object_id,c.name,'charmaxlen') as char_max_length,
dc.definition as default_definition,
ep.value as description,
cc.definition,
c.is_identity ,
ic.seed_value ,
ic.last_value ,
ic.increment_value,
t.is_user_defined
FROM
sys.all_columns c
left join sys.types as t on c.user_type_id = t.user_type_id
JOIN sys.tables tab ON tab.object_id = c.object_id
LEFT OUTER JOIN sys.default_constraints dc ON dc.parent_object_id = tab.object_id AND dc.parent_column_id = c.column_id
LEFT OUTER JOIN master.sys.extended_properties ep ON ep.class = 1 AND ep.major_id = tab.object_id AND ep.minor_id = c.column_id AND ep.name = 'MS_Description'
left join sys.computed_columns cc on cc.object_id = tab.object_id and cc.column_id = c.column_id
left join sys.identity_columns ic on ic.object_id = tab.object_id and ic.column_id = c.column_id
where 1=1
Constraint
PK/UK
select schema_name(t.schema_id) as schema_name,t.[name] as table_name,
case
when t.[type] = 'U' then 'Table'
when t.[type] = 'V' then 'View'
end as [object_type],
case
when c.[type] = 'PK' then 'P'
-- 'Primary key'
when c.[type] = 'UQ' then 'U'
--'Unique constraint'
when i.[type] = 1 then 'Unique clustered index'
when i.[type] = 2 then 'Unique index'
end as constraint_type,
c.name as constraint_name,
i.name as index_name,
substring(column_names, 1, len(column_names)-1) as columns
from
sys.objects t
inner join sys.indexes i on
t.object_id = i.object_id
inner join sys.key_constraints c on
i.object_id = c.parent_object_id
and i.index_id = c.unique_index_id cross apply (
select
CAST(ic.key_ordinal AS varchar) + '||||' +
col.[name] + '||||' +
case
when ic.is_descending_key = 1 then 'desc'
else 'asc'
end + '||||' +
case
when col.is_nullable = 1 then '1'
else '0'
end +
'@ '
from
sys.index_columns ic
inner join sys.columns col on
ic.object_id = col.object_id
and ic.column_id = col.column_id
where
ic.object_id = t.object_id
and ic.index_id = i.index_id
order by
ic.key_ordinal for xml path ('') ) D (column_names)
where
is_unique = 1
and t.is_ms_shipped <> 1
and t.[type] = 'U'
Check
select schema_name(t.schema_id) as schema_name,t.[name] as table_name,'Table',
'C', -- Check constraint
con.[name] as constraint_name,
col.name ,
con.[definition] as SEARCH_CONDITION
from
sys.check_constraints con
left outer join sys.objects t on
con.parent_object_id = t.object_id
left outer join sys.all_columns col on
con.parent_column_id = col.column_id
and con.parent_object_id = col.object_id
FK
select fk.object_id,schema_name(fk_tab.schema_id) as owner,fk_tab.name table_name,
fk.name as constraint_name,
fk.is_disabled as con_is_disabled,
fk.delete_referential_action_desc ,
fk.update_referential_action_desc ,
schema_name(pk_tab.schema_id) as R_OWNER,
pk_tab.name as R_TABLE_NAME,
fk_cols.constraint_column_id as no,
fk_col.name as fk_column_name,
fk_col.is_nullable as fk_column_is_nullable,
pk_col.name as pk_column_name,
pk_col.is_nullable as pk_column_is_nullable
from sys.foreign_keys fk
inner join sys.tables fk_tab
on fk_tab.object_id = fk.parent_object_id
inner join sys.tables pk_tab
on pk_tab.object_id = fk.referenced_object_id
inner join sys.foreign_key_columns fk_cols
on fk_cols.constraint_object_id = fk.object_id
inner join sys.columns fk_col
on fk_col.column_id = fk_cols.parent_column_id
and fk_col.object_id = fk_tab.object_id
inner join sys.columns pk_col
on pk_col.column_id = fk_cols.referenced_column_id
and pk_col.object_id = pk_tab.object_id
Index
select schema_name(t.schema_id) as schema_name,t.[name] as tab_name,i.[name] as index_name,
substring(column_names, 1, len(column_names)-1) as [columns],
-- case
-- when i.[type] = 1 then 'Clustered index'
-- when i.[type] = 2 then 'Nonclustered unique index'
-- when i.[type] = 3 then 'XML index'
-- when i.[type] = 4 then 'Spatial index'
-- when i.[type] = 5 then 'Clustered columnstore index'
-- when i.[type] = 6 then 'Nonclustered columnstore index'
-- when i.[type] = 7 then 'Nonclustered hash index'
-- end as index_type,
i.[type] as index_type,
i.is_unique,
-- case
-- when i.is_unique = 1 then 'Unique'
-- else 'Not unique'
-- end as [unique],
t.[type] as object_type,
i.is_disabled
-- case
-- when t.[type] = 'U' then 'Table'
-- when t.[type] = 'V' then 'View'
-- end as [object_type]
from
sys.objects t
inner join sys.indexes i on
t.object_id = i.object_id cross apply (
select
CAST(ic.key_ordinal AS varchar) + '||||' + col.[name] + '||||' +
case
when ic.is_descending_key = 1 then 'desc'
else 'asc'
end + '||||' +
case
when col.is_nullable = 1 then '1'
else '0'
end + '@ '
from
sys.index_columns ic
inner join sys.columns col on
ic.object_id = col.object_id
and ic.column_id = col.column_id
where
ic.object_id = t.object_id
and ic.index_id = i.index_id
order by
key_ordinal for xml path ('') ) D (column_names)
where
t.is_ms_shipped <> 1
and index_id > 0
and i.is_primary_key = 0
and i.is_unique_constraint = 0
View
select schema_name(v.schema_id), v.name, m.definition from
sys.views v join sys.sql_modules m on m.object_id = v.object_id
where 1=1
Materialized View
Function
select
schema_name(obj.schema_id) as schema_name,
obj.object_id,
obj.name as function_name,
type,
mod.definition
from
sys.objects obj
join sys.sql_modules mod on mod.object_id = obj.object_id
where 1=1 AND obj.[type] in
('AF','FN','FS','FT','IF','TF')
Procedure
select
schema_name(obj.schema_id) as schema_name,
obj.object_id,
obj.name as function_name,
type,
mod.definition
from
sys.objects obj
join sys.sql_modules mod on mod.object_id = obj.object_id
where 1=1 AND obj.[type] in
('P','PC','RF','X')
Trigger
select t.object_id,
schema_name(o.schema_id) as trigger_owner,
t.name as trigger_name,
schema_name(tab.schema_id) as table_owner,
tab.name table_name,
t.is_disabled ,
-- te.is_trigger_event ,
case when is_instead_of_trigger = 1 then 'INSTEAD OF' else 'AFTER' end as [activation],
case when t.parent_class = 1 then 'TABLE' when t.parent_class = 0 then 'DATABASE' end [class],
-- te.type_desc ,
mod.definition
from
sys.triggers t
-- join sys.trigger_events te on t.object_id = te.object_id
join sys.all_objects o on o.object_id = t.object_id
join sys.all_objects tab on tab.object_id = t.parent_id
join sys.sql_modules mod on mod.object_id = t.object_id
where 1=1