Local Pipeline Engine
Local
The local runner runs on the local Hop engine. This is the runtime configuration you’ll use when testing pipelines on your local machine while developing.
Options
Row set size
The row set buffer size.
10.000
Safe mode
Checks every row passed through your pipeline and ensure all layouts are identical. If a row does not have the same layout as the first row, an error is generated and reported.
false
Collect metrics
Collects metrics to monitor the performance of your pipeline.
false
Log rows feedback
Enables log rows feedback, showing a log line every after processing multiple of 50.000 (default) rows.
false
Feedback size in rows
The number of rows to return as feedback.
50.000
Wait time buffer check (ms)
This indicates the polling frequency when there are no rows in the input buffer of a transform, a lower value will cause higher CPU load when the pipeline has many idle transforms.
20
Sample type while running in the GUI
Here you can specify which rows you want to see when you click on the little grid icons on the transform icons during a pipeline execution. You can choose: None: do not show, First : show the first rows, Last : show the last rows or Random: a random selection of rows (reservoir sampling)
Last
Number of rows to sample in the GUI
The number of rows which will be collected
100
Make this pipeline transactional
If you enable this there will always be just one connection used per database. At the end of the pipeline there will be a commit or rollback (in case of error) on all named connections at the same time. Note that child pipelines or workflows are also automatically run as a transaction. Also be aware that while you can commit and rollback across multiple databases that this still means that you can have a successful commit on one database and a failure on another. This is not a two-phase-commit system.
false
Last updated