Primeur Online Docs
Data Shaper
Data Shaper
  • 🚀GETTING STARTED
    • What is Primeur Data Shaper
      • What is the Data Shaper Designer
      • What is the Data Shaper Server
      • What is the Data Shaper Cluster
    • How does the Data Shaper Designer work
      • Designer Views and Graphs
      • Data Shaper Graphs
      • Designer Projects and Sandboxes
      • Data Shaper Designer Reference
    • How do the Data Shaper Server and Cluster work
      • Data Shaper Server and Cluster
      • Data Shaper Server Reference
    • VFS Graph Components
      • DataOneFileDescriptor (DOFD) metadata
      • Passing parameters from Data One Contract to Data Shaper graph
      • Inheriting Data One context attributes in Data Shaper graph
  • DATA SHAPER DESIGNER
    • Configuration
      • Runtime configuration
        • Logging
        • Master Password
        • User Classpath
      • Data Shaper Server Integration
      • Execution monitoring
      • Java configuration
      • Engine configuration
      • Refresh Operation
    • Designer User Interface
      • Graph Editor with Palette of Components
      • Project Explorer Pane
      • Outline Pane
      • Tabs Pane
      • Execution Tab
      • Keyboard Shortcuts
    • Projects
      • Creating Data Shaper projects
      • Converting Data Shaper projects
      • Structure of Data Shaper projects
      • Versioning of server project content
      • Working with Data Shaper Server Projects
      • Project configuration
    • Graphs
      • Creating an empty graph
      • Creating a simple graph
        • Placing Components
        • Placing Components from Palette
        • Connecting Components with Edges
    • Execution
      • Successful Graph Execution
      • Run configuration
      • Connecting to a running job
      • Graph states
    • Common dialogs
      • URL file dialog
      • Edit Value dialog
      • Open Type dialog
    • Import
      • Import Data Shaper projects
      • Import from Data Shaper server sandbox
      • Import graphs
      • Import metadata
    • Export
      • Export graphs to HTML
      • Export to Data Shaper Server sandbox
      • Export image
      • Export Project as Library
    • Graph tracking
      • Changing record count font size
    • Search functionality
    • Working with Data Shaper server
      • Data Shaper server project basic principles
      • Connecting via HTTP
      • Connecting via HTTPS
      • Connecting via Proxy Server
    • Graph components
      • Adding components
      • Finding components
      • Edit component dialog box
      • Enable/disable component
      • Passing data through disabled component
      • Common properties of components
      • Specific attribute types
      • Metadata templates
    • Edges
      • Connecting Components with Edges
      • Types of Edges
      • Assigning Metadata to Edges
      • Colors of Edges
      • Debugging Edges
      • Edge Memory Allocation
    • Metadata
      • Records and Fields
        • Record Types
        • Data Types in Metadata
        • Data Formats
        • Locale and Locale Sensitivity
        • Time Zone
        • Autofilling Functions
      • Metadata Types
        • Internal Metadata
        • External (Shared) Metadata
        • SQL Query Metadata
        • Reading Metadata from Special Sources
      • Auto-propagated Metadata
        • Sources of Auto-Propagated Metadata
        • Explicitly Propagated Metadata
        • Priorities of Metadata
        • Propagation of SQL Query Metadata
      • Creating Metadata
        • Extracting Metadata from a Flat File
        • Extracting Metadata from an XLS(X) File
        • Extracting Metadata from a Database
        • Extracting Metadata from a DBase File
        • Extracting Metadata from Salesforce
        • SQL Query Metadata
        • User Defined Metadata
      • Merging Existing Metadata
      • Creating Database Table from Metadata and Database Connection
      • Metadata Editor
        • Opening Metadata Editor
        • Basics of Metadata Editor
        • Record Pane
        • Field Name vs. Label vs. Description
        • Details Pane
      • Changing and Defining Delimiters
      • Editing Metadata in the Source Code
      • Multi-value Fields
        • Lists and Maps Support in Components
        • Joining on multivalue fields (Comparison Rules)
    • Connections
      • Database Connections
        • Internal Database Connections
        • External (Shared) Database Connections
        • Database Connections Properties
        • Encryption of Access Password
        • Browsing Database and Extracting Metadata from Database Tables
        • Windows Authentication on Microsoft SQL Server
        • Snowflake Connection
        • Hive Connection
        • Troubleshooting
      • JMS Connections
      • QuickBase Connections
      • Hadoop Connections
      • Kafka Connections
      • OAuth2 Connections
      • MongoDB Connections
      • Salesforce Connections
    • Lookup Tables
      • Lookup Tables in Cluster Environment
      • Internal Lookup Tables
      • External (Shared) Lookup Tables
      • Types of Lookup Tables
    • Sequences
      • Persistent Sequences
      • Non Persistent Sequences
      • Internal Sequences
      • External (Shared) Sequences
      • Editing a Sequence
      • Sequences in Cluster Environment
    • Parameters
      • Internal Parameters
      • External (Shared) Parameters
      • Secure Graph Parameters
      • Graph Parameter Editor
      • Secure Graph Parameters
      • Parameters with CTL2 Expressions (Dynamic Parameters)
      • Environment Variables
      • Canonicalizing File Paths
      • Using Parameters
    • Internal/External Graph Elements
    • Dictionary
      • Creating a Dictionary
      • Using a Dictionary in Graphs
    • Execution Properties
    • Notes in Graphs
      • Placing Notes into Graph
      • Resizing Notes
      • Editing Notes
      • Formatted Text
      • Links from Notes
      • Folding Notes
      • Notes Properties
    • Transformations
      • Defining Transformations
      • Transform Editor
      • Common Java Interfaces
    • Data Partitioning (Parallel Running)
    • Data Partitioning in Cluster
      • High Availability
      • Scalability
      • Graph Allocation Examples
      • Example of Distributed Execution
      • Remote Edges
    • Readers
      • Common Properties of Readers
      • ComplexDataReader
      • DatabaseReader
      • DataGenerator
      • DataOneVFSReader
      • EDIFACTReader
      • FlatFileReader
      • JSONExtract
      • JSONReader
      • LDAPReader
      • MultiLevelReader
      • SpreadsheetDataReader
      • UniversalDataReader
      • X12Reader
      • XMLExtract
      • XMLReader
      • XMLXPathReader
    • Writers
      • Common Properties of Writers
      • DatabaseWriter
      • DataOneVFSWriter
      • EDIFACTWriter
      • FlatFileWriter
      • JSONWriter
      • LDAPWriter
      • SpreadsheetDataWriter
      • HIDDEN StructuredDataWriter
      • HIDDEN TableauWriter
      • Trash
      • UniversalDataWriter
      • X12Writer
      • XMLWriter
    • Transformers
      • Common Properties of Transformers
      • Aggregate
      • Concatenate
      • DataIntersection
      • DataSampler
      • Dedup
      • Denormalizer
      • ExtSort
      • FastSort
      • Filter
      • Map
      • Merge
      • MetaPivot
      • Normalizer
      • Partition
      • Pivot
      • Rollup
      • SimpleCopy
      • SimpleGather
      • SortWithinGroups
      • XSLTransformer
    • Joiners
      • Common Properties of Joiners
      • Combine
      • CrossJoin
      • DBJoin
      • ExtHashJoin
      • ExtMergeJoin
      • LookupJoin
      • RelationalJoin
    • Others
      • Common Properties of Others
      • CheckForeignKey
      • DBExecute
      • HTTPConnector
      • LookupTableReaderWriter
      • WebServiceClient
    • CTL2 - Data Shaper Transformation Language
    • Language Reference
      • Program Structure
      • Comments
      • Import
      • Data Types in CTL2
      • Literals
      • Variables
      • Dictionary in CTL2
      • Operators
      • Simple Statement and Block of Statements
      • Control Statements
      • Error Handling
      • Functions
      • Conditional Fail Expression
      • Accessing Data Records and Fields
      • Mapping
      • Parameters
      • Regular Expressions
    • CTL Debugging
      • Debug Perspective
      • Importing and Exporting Breakpoints
      • Inspecting Variables and Expressions
      • Examples
    • Functions Reference
      • Conversion Functions
      • Date Functions
      • Mathematical Functions
      • String Functions
      • Mapping Functions
      • Container Functions
      • Record Functions (Dynamic Field Access)
      • Miscellaneous Functions
      • Lookup Table Functions
      • Sequence Functions
      • Data Service HTTP Library Functions
      • Custom CTL Functions
      • CTL2 Appendix - List of National-specific Characters
      • HIDDEN Subgraph Functions
    • Tutorial
      • Creating a Transformation Graph
      • Filtering the records
      • Sorting the Records
      • Processing Speed-up with Parallelization
      • Debugging the Java Transformation
  • DATA SHAPER SERVER
    • Introduction
    • Administration
      • Monitoring
    • Using Graphs
      • Job Queue
      • Execution History
      • Job Inspector
    • Cluster
      • Sandboxes in Cluster
      • Troubleshooting
  • Install Data Shaper
    • Install Data Shaper
      • Introduction to Data Shaper installation process
      • Planning Data Shaper installation
      • Data Shaper System Requirements
      • Data Shaper Domain Master Configuration reference
      • Performing Data Shaper initial installation and master configuration
        • Creating database objects for PostgreSQL
        • Creating database objects for Oracle
        • Executing Data Shaper installer
        • Configuring additional firewall rules for Data Shaper
Powered by GitBook
On this page
  • Arithmetic Operators
  • Relational Operators
  • Logical Operators
  • Assignment Operator
  • Ternary Operator
  • Conditional Fail Expression
  1. DATA SHAPER DESIGNER
  2. Language Reference

Operators

The operators serve to perform operations in the same way as functions do, but using operators, your code is more compact and legible.

Operators can be arithmetic, relational and logical. The arithmetic operators can be used in all expressions, not only the logical ones. The relational and logical operators serve to create expressions with resulting boolean value.

All operators can be grouped into four categories:

  • Arithmetic Operators (\+ - * / % ++ --)

  • Relational Operators (> >= == <= < != ~= ?=)

  • Logical Operators (&& || ! == !=)

  • Assignment Operator (= += -= *= /= %=)

Arithmetic Operators

The arithmetic operators perform basic mathematical operation (addition, subtraction, etc.), concatenate strings or lists or merge content of two maps.

The operators can be used more times in one expression. The result depends on the order of operators within the expressions. In such a case, you can express priority of operations by parentheses.

Hint! If you are unsure about priority of operators or associativity, the safest way is to use parentheses.

Addition

+

numeric type +( numeric type left, numeric type right );
string +( string left, string right );
list +( list left, list right );
map +( map left, map right );

The operator + serves to sum the values of two expressions, concatenate two string values, concatenate two lists or merge content of two maps.

Nevertheless, if you want to add any data type to a string, the second data type is converted to a string automatically and it is concatenated with the first (string) summand. But remember that the string must be on the first place.

Naturally, two strings can be summed in the same way.

The addition of two boolean values or two date data types is not possible. To create a new value from two boolean values, you must use logical operators instead.

integer in01 = 1;
integer in02 = 2;
integer in03 = in02 + in01; // 3

// string concatenation
string s1 = "Hello";
string s2 = " World!";
string s3 = s1 + s2; // Hello World!

decimal price = 1.50d;
string order = "turnip " + price; // turnip 1.50

variant mapVar = {1 -> {2 -> 3}, 4 -> {5 -> 6}};
printLog(info, "mapVar = " + mapVar); // prints "mapVar = {1={2=3}, 4={5=6}}"

// concatenation of two lists
integer [] il1 = [2];
integer [] il2 = [3,5];
integer [] il3 = il1 + il2; // [2,3,5]

// merge of two maps
map[string,string] m1;
map[string,string] m2;
map[string,string] m3;
m1["d"] = "Delta";
m1["f"] = "Foxtrot";
m2["e"] = "Echo";
m3 = m1 + m2;

If you concatenate several strings, use the following approach instead of the plus sign:

string[] buffer;

buffer.append("<html>\n");
buffer.append("<head>\n");
buffer.append("<title>String concatenation example</title>\n");
buffer.append("</head>\n");

// append multiple strings at once
buffer.copy(["<body>", "\n", "Sample content", "\n", "</body>", "\n"]);
buffer.append("</html>");

// concatenates the list into a single string, null list elements are converted to the string "null"
string result = join("", buffer);

Subtraction and Unitary minus

-

numeric type -( numeric type left, numeric type right );

The operator - subtracts one numeric data type from another.

If the numeric types of operands differ, firstly, automatic conversions are applied and then subtraction is performed.

integer i1 = 5 - 3;

Multiplication

*

numeric type *( numeric type left, numeric type right );

The operator * multiplies two numbers.

Numbers can be of different data types. If data types of operands differ, automatic conversion is applied.

integer i1 = 2 * 3;
decimal d1 = 1.5 * 3.5;
double  d2 = 2.5 * 2;

Division

/

numeric type /( numeric type left, numeric type right );

Operator / serves to divide two numeric data types. Remember that you must not divide by zero. Division by zero throws TransformLangExecutorRuntimeException or returns Infinity (in the case of double (number) data type).

integer i1 = 7  / 2;      // i1 == 3
long    l2 = 9L / 4L;     // l2 == 2L
decimal d3 = 6.75D / 1.5D // d3 == 4.5D
double d4  = 6.25  / 2.5  // d4 == 2.5

Modulus

%

numeric type %( numeric type left, numeric type right );

Operator % returns the remainder of division. The operator can be used for floating-point, fixed-point and integral data types.

integer in1 = 7 % 3;         // in1 == 1
long    lo1 = 8 % 5;         // lo1 == 3
decimal de1 = 15.75D % 3.5D  // de1 == 1.75D
double  do1 = 6.25 % 2.5     // do1 == 1.25

Incrementing

++

Operator ++ serves to increment numeric data type value by one. The operator can be used for both floating-point data types and integer data types.

If it is used as a prefix, the number is incremented first and then it is used in the expression.

If it is used as a postfix, first, the number is used in the expression and then it is incremented.

Remember that the incrementing operator cannot be applied on literals, record fields, map, or list values of integer data type.

integer i1 = 20;
integer i2 = ++i1; // i1 = i1 + 1; i2 = i1;     i1 == 21 and i2 == 21
integer i3 = i++   // i3 = i1;     i1 = i1 + 1; i1 == 22 and i3 == 21

Decrementing

--

Operator -- serves to decrement numeric data type value by one. The operator can be used for floating-point, fixed-point and integral data types. If it is used as a prefix, the number is decremented first and then it is used in the expression. If it is used as a postfix, first, the number is used in the expression and then it is decremented.

Remember that the decrementing operator cannot be applied on literals, record fields, map, or list values of integer data type.

integer i1 = 20;
integer i2 = --i1; // i1 = i1 - 1; i2 = i1;     i1 == 19 and i2 == 19
integer i3 = i1--; // i3 = i1;     i1 = i1 - 1; i1 == 18 and i3 == 19

Relational Operators

The following operators serve to compare some subexpressions when you want to obtain a boolean value result. Each of the mentioned signs can be used. These signs can be used more times in one expression. In such a case you can express priority of comparisons by parentheses. If you choose the .operator. syntax, operator must be surrounded by white spaces. Example syntax for the eq operator:

CODE
WORKING?

5 .eq. 3

✓

5 == 3

✓

5 eq 3

x

5.eq(3)

x

  • Greater than Each of the two signs below can be used to compare expressions consisting of numeric, date and string data types. Both data types in the expressions must be comparable. The result can depend on the order of the two expressions if they are of different data types.

>

.gt.

boolean a = 4 > 3;
a = "dog" > "cat";
if ( date1 > date2 ) {}
  • Greater than or equal to Each of the three signs below can be used to compare expressions consisting of the numeric, date and string data types. Both data types in the expressions must be comparable. The result can depend on the order of the two expressions if they are of different data types.

>=

=>

.ge.

boolean a = 3.5 >= 3.5;
a = "ls" >= "lsof";
a = date1 >= date2;
  • Less than Each of the two signs below can be used to compare expressions consisting of numeric, date and string data types. Both data types in the expressions must be comparable. The result can depend on the order of the two expressions if they are of different data types.

<

.lt.
  • Less than or equal to Each of the three signs below can be used to compare expressions consisting of the numeric, date and string data types. Both data types in the expressions must be comparable. The result can depend on the order of the two expressions if they are of different data types.

<=
=<

.le.

int a = 7L < 8L;
if ( "awk" < "java" ) {}
a = date1 < date2;
  • Equal to Each of the two signs below can be used to compare expressions of any data type. Both data types in the expressions must be comparable. The result can depend on the order of the two expressions if they are of different data types.

==

.eq.

if( 5 == 5 ) {}
  • Not equal to Each of the three signs below can be used to compare expressions of any data type. Both data types in the expressions must be comparable. The result can depend on the order of the two expressions if they are of different data types.

!=

<>

.ne.

if ( 9 != 8 ) {}
boolean b = "cat" ~= "[a-z]{3}";
~=

.regex.
boolean b1 = "new bookcase" ~= ".*book.*";  // true
boolean b2 = "new bookcase" ~= "book";      // false
boolean b3 = "new bookcase" ~= null;        // fails
?=
boolean b = "miredo" ?= "redo";

"typeof" Operator

boolean <value> typeof <type or metadata name>

Tests if a value (left operand) is of the specified type (right operand).

Returns false if the value is null.

For lists and maps, does not check the type of elements.

Example 69. Usage of typeof

variant myVariant = 5;
if (myVariant typeof integer) { } // TRUE
if (myVariant typeof number) { } // FALSE
if (myVariant typeof string) { } // FALSE

variant someObject = {"a" -> 1, true -> false};
if (someObject typeof map) { // TRUE
     // handle map
} else if (someObject typeof list) { // FALSE
     // handle list
}

variant nullVariant = null;
if (nullVariant typeof string) { } // null returns FALSE for all types

myMetadata myRecord;
variant recordVariant = myRecord;
if (recordVariant typeof record) { } // TRUE - generic record
if (recordVariant typeof myMetadata) { } // TRUE - specific metadata
if (recordVariant typeof otherMetadata) { } // FALSE - specific metadata

Logical Operators

If the expression whose value must be of boolean data type is complex, it can consist of some subexpressions (see above) that are put together by logical conjunctions (AND, OR, NOT, .EQUAL TO, NOT EQUAL TO). If you want to express priority in such an expression, you can use parentheses. From the conjunctions mentioned below, you can choose either form (for example, && or and, etc.). Every sign of the form .operator. must be surrounded by a white space.

  • Logical AND

&&

and
  • Logical OR

||

or
  • Logical NOT

!

not
  • Logical EQUAL TO

==

.eq.
  • Logical NOT EQUAL TO

!=

<>

.ne.

Assignment Operator

Assignment operator assigns a value of expression on the right side of the operator to a variable on the left side of the operator.

int i = 5;

Compound Operators

Compound operators allow you to use a variable as an accumulator. CTL2 supports the following compound assignment operators: += (addition, string concatenation, list concatenation and map union), -= (subtraction), *= (multiplication), /= (division), and %= (modulus). If the original value of the left-hand side variable is null, the default value for the target type (0, empty string, empty list, empty map) is used for the evaluation instead. See variables nsand ns2in the example below. Example 70. Compound assignment operators

integer i = 5;
  i += 4; // i == 9

  integer ni = null;
  ni += 5; // ni == 5

  string s = "hello ";
  s += "world "; // s == "hello world "
  s += 123; // s == "hello world 123"

  string ns = null;
  ns += "hello"; // ns == "hello"

  string ns2 = null;
  ns2 = ns2 + "hello"; // ns2 == "nullhello"

  integer[] list1 = [1, 2, 3];
  integer[] list2 = [4, 5];
  list1 += list2; // list1 == [1, 2, 3, 4, 5]

  map[string, integer] map1;
  map1["1"] = 1;
  map1["2"] = 2;
  map[string, integer] map2;
  map2["2"] = 22;
  map2["3"] = 3;
  map1 += map2; // map1: "1"->1, "2"->22, "3"->3

  long l = 10L;
  l -= 4; // l == 6L;

  decimal d = 12.34D;
  d *= 2; // d == 24.68D;

  number n = 6.15;
  n /= 1.5; // n ~ 4.1

  long r = 27;
  r %= 10; // r == 7L

CTL2 does not perform any counter-intuitive conversion of the right operand of +=. If you need to add double to integer, you should convert it explicitly:

integer i = 3;
i += double2integer(1.0);

It works with **-=, *=, /= ** and %= as well.

The = operator does not just pass object references, but performs a deep copy of values. That is of course more demanding in terms of performance. Deep copy is only performed for mutable data types, i.e. lists, maps, records and dates. Other types are considered immutable, as CTL2 does not provide any means of changing the state of an existing object (even though the object is mutable in Java). Therefore it is safe to pass a reference instead of copying the value. Note that this assumption may not be valid for custom CTL2 function libraries. Example 71. Modification of a copied list, map and record

integer[] list1 = [1, 2, 3];
  integer[] list2;
  list2 = list1;

  list1.clear(); //  only list1 is cleared (older implementation: list2 was cleared, too)

  map[string, integer] map1;
  map1["1"] = 1;
  map1["2"] = 2;
  map[string, integer] map2;
  map2 = map1;

  map1.clear(); //  only map1 is cleared (older implementation: map2 was cleared, too)

  myMetadata record1;
  record1.field1 = "original value";
  myMetadata record2;
  record2 = record1;

  record1.field1 = "updated value"; // only record1 will be updated (older implementation: record2 was updated, too)

Ternary Operator

Ternary operator is a compact conditional assignment.

It serves to set a value of a variable depending on a boolean expression or a boolean variable.

a = b ? c : d;

The expression above is same as:

if ( b ) {
    a = c;
} else {
    a = d;
}

The a, c and d variables must be of the same data type (or type of c and d must be convertible to type of a using automatic conversion). The b variable is boolean.

b, c or d do not have to be variables. They may be constants or expressions. c has to be a variable.

For example, you can use a ternary operator to assign minimum of c and d into a in a compact way:

a = c < d ? c : d;

Conditional Fail Expression

The conditional fail expression allows the user to conditionally execute a piece of code depending on a failure occurred in the previous part of the code. variable = expr1 : expr2 : …​ : exprN;

integer count = getCachedValue() : refreshCacheAndGetCachedValue() : defaultValue;

Conditional expression is available only in an interpreted mode. It is not available in a compiled mode.

PreviousDictionary in CTL2NextSimple Statement and Block of Statements

Last updated 1 month ago

Note also that the concat() function is faster than +. You should use this function instead of adding any summand to a string. See .

This example is analogous to using a .

Avoid for concatenation of a large number of strings.

You can also use or to concatenate strings. The difference is that join allows storing intermediate results in a list of strings, while concat requires that all operands are passed as parameters simultaneously.

Matches regular expression The operator serves to compare string and some . It returns true, if the whole string matches the regular expression, otherwise returns false. If the right operand is null, operator fails.

Contains regular expression The operator serves to compare string and some . It returns true, if the string contains a substring that matches the regular expression, otherwise returns false.

See also: , ,

See also: .

concat
java.lang.StringBuilder
Schlemiel the Painter’s algorithm
Regular Expressions
Regular Expressions
raiseError
cast
getType
variant
concat
concatWithSeparator