MTK Limit

Limit

Field	Type	Description
parallel	int	Indicates the degree of parallelism.
fetchSize	int	Specifies the row size for one row fetch. Valid values are 1-50000
batchSize	int	Specify the size of batch insert or Copy and make Commit.
bufferSize	int	Specifies the cache size (in MB) when querying or bulk inserting.
cpBufferSize	int	Defines the cache size (in MB) used in the Copy command.
oracleSelectParallel	int	Add hint `/* +parallel(t,n) */` to the Oracle query statement. Please configure it carefully,
limit	int64	Define how many rows to migrate per table.
scn	string	Consistency export transaction consistency point. Before v2.7.3, it was oraScn
splitTabSize	int64	When the table size or partition size is larger than this value, the query task is split. Unit MB
splitTabMethod	string	A method for splitting a table query. Split table query method. The default split method is different for each database
splitTabTaskNum	int	The number of single-table/single-partition split tasks
splitTabSamplePercentage	float64	Percentage sampled.

Appears in:

Config.limit

Example:

Limit Example

{
  "parallel": 2,
  "fetchSize": 1000,
  "batchSize": 1000,
  "bufferSize": 8,
  "cpBufferSize": 8,
  "oracleSelectParallel": 2,
  "channelCacheNum": 2000,
  "limit": 0,
  "splitTabSize": 5120,
  "splitTabMethod": "",
  "splitTabTaskNum": 0,
  "splitTabSamplePercentage": 0.1
}

parallel

Type: int

Desc: Indicates the degree of parallelism.

Create tables, constraints, etc. in parallel.

Default: 1

fetchSize

Type: int

Desc: Specifies the row size for one row fetch. Valid values are 1-50000

Support Oracle.

Default: 1000

batchSize

Type: int

Desc: Specify the size of batch insert or Copy and make Commit.

Versions after 2.3.4 began to support batch Commit. The previous version only interacted with data in batches without committing

If the submission fails the program will exit and log the error data to the error file.

If you configure igErrorData the program will not exit and will log the error data to the error file and continue processing the data.

Valid values are 1-50000

Support Oracle, PostgreSQL, openGauss, MySQL.

Batch commit supports PostgreSQL/MogDB,MySQL.

Default: 1000

bufferSize

Type: int

Desc: Specifies the cache size (in MB) when querying or bulk inserting.

Valid values are 1-1024

Support PostgreSQL, openGauss, MySQL.

Default: 8

cpBufferSize

Type: int

Desc: Defines the cache size (in MB) used in the Copy command.

Valid values are 1-1024

Support PostgreSQL, openGauss, MySQL.

Default: 8

oracleSelectParallel

Type: int

Desc: Add hint /* +parallel(t,n) */ to the Oracle query statement. Please configure it carefully, considering the performance of the Oracle database.

limit

Type: int64

Desc: Define how many rows to migrate per table. Defining this parameter no longer does table concurrent migrations.

scn

Type: string

Desc: Consistency export transaction consistency point. Before v2.7.3, it was oraScn

DBType	Desc
Oracle	AS OF SCN/TIMESTAMP
MySQL	does not support configuration. Record the pos of the bin log
PostgreSQL	Does not support configuration. Record snapshot
openGauss	Not supported
SQLServer	Not supported
Informix	Not supported

Example:

Oracle AS OF SCN Example

{
  "scn": "41393786"
}

Oracle AS OF Timestamp Example

{
  "scn": "TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')"
}

Add: v2.8.0

splitTabSize

Type: int64

Desc: When the table size or partition size is larger than this value, the query task is split. Unit MB

Default: 5120

Add: v2.9.3

splitTabMethod

Type: string

Desc: A method for splitting a table query. Split table query method. The default split method is different for each database

Oracle
- Auto : Select * from schema.table sample [0.000001,100)
- ora_hash

Option:

auto
ora_hash
mod
rowid

Add: v2.9.3

splitTabTaskNum

Type: int

Desc: The number of single-table/single-partition split tasks

Add: v2.9.3

splitTabSamplePercentage

Type: float64

Desc: Percentage sampled.

Oracle : Select * from schema.table sample [0.000001,100)

Default: 0.1

Add: v2.9.3