MonetDB Bulk Loader

MonetDB Bulk Loader

Description

The MonetDB Bulk Loader transform bulk loads data to MonetDB. This significantly speeds up data loading to MonetDB.

Hop Engine	✓
Spark	✓
Flink	✓
Dataflow	✓

Options

General

Field	Description
Transform name	Specify the unique name of the MongoDB Output transform in the pipeline.
Connection	Select your MonetDB database connection

General Settings tab

This tab contains the destination settings, buffer size and location for the logfile.

Field	Description
Target Schema	Specify the database schema that has to be used.
Target Table	Specify the database table, use the Browse button next to this field to use a menu to select the table and schema
Buffer size (rows)	Specify how many rows will be kept in memory before transferring to MonetDB
Log file	Specify the location for the Bulk command logs returned from MonetDB
Truncate table	Remove all data from the destination table before loading the data.
Fully quote all SQL statements	Forces quotes around all objects when executing

MonetDB Settings tab

This tab contains information about the temporary files that are generated to load the data.

Field	Description
Field separator	This is the separator that will be used in the Bulk copy command, it is not allowed to have this field in the input data.
Field enclosure	The enclosure character used around values.
Null values represented	Null values will be converted to this string, this allows to differentiate empty strings and null values.
Encoding	File encoding used when generating the files for the copy statement.

Output Fields tab

This tab contains the source to target mapping.

Field	Description
Target table field	Field containing the name of the field in the target table
Incoming stream field	Field containing the value we want to insert in target table
Format is ok	Set to Y if the incoming stream’s field is the correct format according to the target datatatype. NOTE: This setting is evaluated only when Lazy Conversion is applied. For example: imagine you are getting values from a text file, your incoming data contains numbers or dates and Lazy Conversion is enabled in the input transform. In this case, the data is not transformed internally to the target data type and is managed as a String by Hop. By setting this flag to Y, we are saying Hop that the incoming data’s value is already in a format clearly understandable by the target database according to the target datatype.