MySql Bulk Loader
MySql Bulk Loader
MySql Bulk LoaderDescription
The MySql Bulk Loader transform uses the copy command to load data as opposed to sending individual insert statements.
It will create a local file which will then be loaded using the LOAD DATA command. More information here.
Hop Engine
✓
Spark
?
Flink
?
Dataflow
?
Options
Note
Bulk loading must be enabled on both server and client in the client connection add following option allowLoadLocalInfile=true. On the server side the following query should return "ON" show global variables like 'local_infile';
General
Connection
The database connection to use when bulk loading
Target Schema
(Optional) The schema containing the table being loaded.
Target Table
The name of the table being loaded.
Fifo file
Temporary file location
Delimiter
Delimiter that determines the field.
Enclosure
You can specify an enclosure string which when placed around a value allows delimiters to be present in it.
Escape character
To include delimiter characters in values sometimes an escape string is used like backslash, double backslash and so on.
Character set (load command)
The used character set in the load command (optional).
Character set (file creation)
The used character set when creating the file (optional).
Bulk size (rows)
This will split the data load in multiple chucks.
Use replace clause
With REPLACE, new rows that have the same value as a unique key value in an existing row replace the existing row.
Use Ignore clause
With IGNORE, new rows that duplicate an existing row on a unique key value are discarded.
Local data
If LOCAL is not specified, the file must be located on the server host.
Fields
Table field
Name of the field in the table.
Stream field
Name of the field in the stream.
Field format OK?
You can decide if the format should be kept (Don’t change formatting) or changed:
Format as Date (yyyy-MM-dd)
Format as a timestamp (yyyy-MM-dd HH:mm:ss)
Format as Number (grouping symbol is "," - decimal is ".")
Escape enclosure characters when found
Last updated