Beam File Output

Beam File Output

Description

The Beam File Output transform writes files using a file definition with the Beam execution engine.

Hop Engine	X
Spark	✓
Flink	✓
Dataflow	✓

Options

Option	Description
Transform name	Name of the transform, this name has to be unique in a single pipeline.
Output location	The output location.
File prefix	Text which has to precede the file name.
File suffix	Text which has to succeed the file name.
Windowed writes?	Writes a file per window. Use together with the Beam Window transform.
File definition to use	The file definition to use stored in a metastore.
Edit	Edits an existing file definition.
New	Creates a new file definition.
Manage	Opens the MetaStore Explorer.

File Definition

A file definition can be used to define a file structure definition.

Option	Description
Name	The file definition name.
Description	The file definition description.
Field separator	The character separating the fields.
Field enclosure	The character enclosing fields.
Field definitions	A list of fields.
Field name	The field name.
Type	The field data type.
Format	The field format.
Length	The field length.
Precision	The field precision.