Specific attribute types
Last updated
Last updated
Here is a brief overview of complex attribute types. These attribute types are common for various groups of components.
Links to corresponding sections follow:
When you need to specify a file in a component, you need to use .
Some components use a specific metadata structure on their ports. The connected edges can be easily assigned metadata from predefined templates. See .
Some components can be configured with a time interval (usually a delay or a timeout). For an overview of the syntax of time interval specification, see .
In some of the components, records must be grouped according to values of a group key. In this key, neither the order of key fields nor the sort order are of importance. See .
In some of the components, records must be grouped and sorted according to the values of a sort key. In this key, both the order of key fields and the sort order are of importance. See .
In many components from different groups of components, a transformation can be or must be defined. See .
The following time units may be used when specifying time intervals:
w
week (7 days)
d
day (24 hours)
h
hour (60 minutes)
m
minute (60 seconds)
s
second (1000 milliseconds)
ms
millisecond
The units may be arbitrarily combined, but their order must be from the largest to the smallest one.
Example 3. Time Interval Specification
1w 2d 5h 30m 5s 100ms = 797405100 milliseconds
1h 30m = 5400000 milliseconds
120s = 120000 milliseconds
When no time unit is specified, the number is assumed to denote the default unit, which is component-specific (usually milliseconds).
Sometimes you need to select fields that will create a grouping key. This can be done in the Edit key dialog. After opening the dialog, you need to select the fields that should create the group key.
Select the fields you want and drag and drop each of the selected key fields to the Key parts pane on the right. (You can also use the Arrow buttons.)
After selecting the fields, you can click the OK button and the selected field names will turn to a sequence of the same field names separated by a semicolon. This sequence can be seen in the corresponding attribute row.
The resulting group key is a sequence of field names separated by a semicolon. It looks like this: FieldM;…​FieldN.
When a key is defined and used in a component, input records are gathered together into a group of the records with equal key values.
The key is ordered in following ways:
Ascending - if the input records are sorted in ascending order
Descending - if the input records are sorted in descending order
Auto - the sorting order of the input records is guessed from the first two records with different value in the key field, i.e., from the first records of the first two groups.
Ignore - if the input records with the same key field value(s) are not sorted
Group key is used in the following components:
In some of the components you need to define a sort key. Like a group key, this sort key can also be defined by selecting key fields using the Edit key dialog. There you can also choose what sort order should be used for each of the selected fields.
In the Edit key dialog, select the fields you want and drag and drop each of the selected key fields to the Key column of the Key parts pane on the right. (You can also use the Arrow buttons.)
Unlike in the case of a group key, in any sort key the order in which the fields are selected is of importance.
In every sort key, the field at the top has the highest sorting priority. Then the sorting priority descends down from top. The field at the bottom has the lowest sorting priority.
When you click the OK button, the selected fields will turn to a sequence of the same field names and an a or d letter in parentheses (with the meaning: ascending or descending, respectively) separated by a semicolon.
It can look like this: FieldM(a);…​FieldN(d).
This sequence can be seen in the corresponding attribute row. (The highest sorting priority has the first field in the sequence. The priority descends towards the end of the sequence.)
As you can see, in this kind of key, the sort order is expressed separately for each key field (either Ascending or Descending). The default sort order is Ascending. The default sort order can also be changed in the Order column of the Key parts pane.
Example 4. Sorting
If your sort key is the following: Salary(d);LastName(a);FirstName(a). The records will be sorted according to the Salary values in descending order, then the records will be sorted according to LastName within the same Salary value and they will be sorted according to FirstName within the same LastName and the same Salary (both in ascending order) value.
Thus, any person with Salary of 25000 will be processed after any other person with a salary of 28000. And, within the same Salary, any Brown will be processed before any Smith. And again, within the same salary, any John Smith will be processed before any Peter Smith. The highest priority is Salary, the lowest is FirstName.
Sort key is used in the following cases:
See also: , .
In this kind of key, no sort order is shown unlike in Sort key. By default, the order is ascending for all fields and priority of these fields descends down from top in the dialog pane and to the right from the left in the attribute row. For more detailed information, see .
Group key in
Merge key in
Aggregate key in
Key in
Group key in
Also Partition key that serves for distributing data records among different output ports (or Cluster nodes in case of clusterpartition) is of this type. See
Sort key in
Sort key in
Sort key in
