Text File Output
Text File Output
Text File OutputDescription
The Text File Output transform is used to export data to text file format.
This is commonly used to generate Comma Separated Values (CSV files) that can be read by spreadsheet applications.
It is also possible to generate fixed width files by setting lengths on the fields in the fields tab.
You can choose to use a Schema Definition or to define the required fields' layout manually. If you decide to define the fields layout by using a Schema Definition, use the Schema Mapping transform to adjust the incoming stream according to the chosen Schema Definition.
Hop Engine
Spark
Flink
Dataflow
Options
File Tab
The File tab is where you define basic properties about the file being created, such as:
Transform name
Name of the transform. This name has to be unique in a single pipeline.
Filename
Specifies the filename and location of the output text file. Click Browse to select a location.
Pass output to servlet
Enable this to return the data via a web service instead writing into a file.
Create parent folder
Enable this to create the parent folder when the file is created. Otherwise the folder must already exist or the write fails.
Do not create file at start
Enable this to avoid empty files when no rows are getting processed. Otherwise the file is created before any data is processed, and might produce an empty file.
Accept file name from field?
Enable this to specify the file name(s) in a field in the input stream
File name field
When the previous option is enabled, specify the field that will contain the filename(s) at runtime.
Extension
Specifies the file extension. This automatically adds the period character in between the filename and the extension when the file is created. For example, name.txt.
Include transform nr in filename
If you run the transform in multiple copies (Launching several copies of a transform), the copy number is included in the filename, before the extension and separated from the file name with an underscore. For example, name_0.
Include partition nr in filename?
Includes the data partition number in the filename, or null if the data is not partitioned.
Include date in filename
Includes the system date in the filename. This is separated from the filename with an underscore, and by default uses the format YYYYMMDD. For example, name_20230405.
Include time in filename
Includes the system time in the filename. This is separated from the filename with an underscore, and by default uses the format hhmmss. For example, name_235959.
Specify Date time format
Enable to specify the date and time format. Otherwise, these components use the defaults.
Date time format
Select the date and time format from the list to apply to the date and time values appended to the filename.
Show filename(s)
View a list of the files that will be generated.
Add file name to result
This adds all processed filenames to the internal result filename set to allow for further processing.
Content Tab
The content tab contains the following options for describing the content being read:
Append
Check this to append lines to the end of the specified file. Otherwise the file is replaced.
Separator
Specify the character that separates the fields in a single line of text. Typically this is ; or a tab.
Insert TAB
Click to insert a tab into the Separator field.
Enclosure
A pair of strings can enclose some fields. This allows separator or enclosure characters in fields. The enclosure string is optional.
Force the enclosure around fields?
This option forces all fields of an incoming string type (independent of the eventually changed field type within the Text File Output field definition) to be enclosed with the character specified in the Enclosure property above.
Disable the enclosure fix?
When a string field contains an enclosure it gets enclosed and the enclosing characters are escaped. When a string field contains a separator, it gets enclosed. Check this option, if this logic is not wanted. It has also an extra performance burden since the strings are scanned for enclosures and separators. So when you are sure there is no such logic needed since your strings don’t have these characters in there and you want to improve performance, un-check this option.
Header
Enable this option if you want the text file to have a header row. (First line in the file).
Footer
Enable this option if you want the text file to have a footer row. (Last line in the file). Note: Be careful using this option when in Append mode, because it is not possible to remove footers from the file contents before appending new rows. There are use cases where this option is wanted, e.g. to have a footer after each run of a pipeline to separate sections within the file.
Format
This can be either DOS or UNIX. UNIX files have lines are separated by linefeeds.
DOS files have lines separated by carriage returns and line feeds.
The options are: CR+LF terminated (Windows, DOS) / LF terminated (Unix) / CR terminated / No new-line terminator
Encoding
Specify the text file encoding to use, or leave blank to use the default encoding on your system.
To use Unicode specify UTF-8 or UTF-16.
Compression
Allows you to compress the file and specify the type of compression, either .zip or .gzip, to use when compressing the output.
Note: Only one file is placed in a single compressed archive.
Right pad fields
Add spaces to the end of the fields (or remove characters at the end) until they have the length specific on the Fields tab.
Fast data dump (no formatting)
Improves the performance when dumping large amounts of data to a text file by not including any formatting information.
Split every … rows
Specify a number of rows to split the file into chunks of that many rows.
Add Ending line of file
Specify an alternate ending row to the output file.
Fields Tab
The fields tab is where you define properties for the fields being exported. The table below describes each of the options for configuring the field properties:
Schema Definition
Name of the Schema Definition that we want to reference.
Name
The name of the field.
Type
Data type of the field. This can be either String, Date or Number.
Format
Select a format to apply to the field value. Only applies to numbers and dates.
Length
The length option depends on the field type:
Number - Total number of significant figures in a number
String - total length of string
Date - length of printed output of the string (e.g. . only gives back year)
Precision
For numbers, specifies the number of floating point digits
Currency
Symbol used to represent currencies, such as dollars ($10,000.00) or Euros (E5.000,00)
Decimal
The character used to represent the decimal point in a floating-point number. A decimal point can be a "." (10,000.00) or "," (5.000,00).
Group
The character used to represent the thousands separator in numbers. This can be a "," (10,000.00) or "." (5.000,00)
Trim type
The trimming method used to remove blank spaces from string values. This can be left (removes leading spaces), right (removes trailing spaces), both, or none.
Note: Trimming only works when there is no field length given.
Null
Specify a value to insert in the field if the field value is null.
Get Fields
Click to retrieve the list of fields from the input fields stream(s).
Rounding Type
When writing numbers to a file you can specify the Rounding type being used by default Half Even is used, for more information look at the Rounding Types section
Minimal width
Alter the options in the fields tab in such a way that the resulting width of lines in the text file is minimal.
So instead of a string value being 0000001, we write 1, etc. String fields will no longer be padded to their specified length.
Unresolved directive in tmp.JXvzjukc3e - include::../../snippets/rounding-types.adoc[]
Last updated 2025-09-04 18:23:19 +0200
Last updated