Pipeline variable resolver
Functionality
We can create a new variable resolver metadata element of type Pipeline Variable Resolver. The options used are fairly simple:
The pipeline filename to use: The file name of the pipeline to run to resolve a variable expression.
Pipeline Run configuration name: this has to reference a run configuration of type: Native Local.
The name of the variable in the pipeline that will contain the expression to resolve
The name of the output field to use to pick up the value of the fields.
This setup gives you a lot of flexibility. Maybe you have a configuration file per database, per system, … In that case you can set the variable to contain the type or name of the database, system, … to read a configuration file for. You can retrieve multiple values from these alternative configuration stores and send them to the output transform in a single row.
Only the first output row in the pipeline is considered.
The variable expression is (as always) in the following format:
#{name:variable:field}
name: the name of the variable resolver metadata element to use
variable: the variable value to set in the variable specified in the metadata (see above)
field: the name of the field for which we want to retrieve a string in the output transform.
Example
Suppose we have a number of databases we have configuration files for in separate properties files:
db1.properties
db2.properties
db3.properties …
These all contain the same 5 keys, with different values of-course:
username=user1
password=pwd1
hostname=hostname1
port=port1
db=db1We can then have a pipeline that reads and processes the appropriate file per type:
The Variable Resolver metadata would look like this:
Here are some examples of expressions and the results they give back:
#{conf:db1:username}: user1#{conf:db2:db}: db2#{conf:db3:port}: port3
This gives you a lot of flexibility.
Last updated