Data Types in Metadata
Each metadata field can be of different data type.
The following types of record fields are used in metadata. If you need to see data types used in CTL, see Data Types in CTL2.
| DATA TYPE | SIZE [1] | VALUES | DEFAULT VALUE |
|---|---|---|---|
| boolean | Represents 1 bit. Its size is not precisely defined. | true | false | 1 | 0 | false | 0 |
| byte | Depends on the actual data length. | from -128 to 127 | null |
| cbyte | Depends on the actual data length and success of compression. | from -128 to 127 | null |
| date | 64 bits [2] | Zero date corresponds to 1st January 1970, 00:00:00 GMT. The precision of this data type is 1 ms. | 1970-01-01, 00:00:00 GMT |
| decimal | Depends on Length and Scale. (Length is the maximum number of all digits. Scale is the maximum number of digits after the decimal dot. Default values are 12 and 2, respectively.) [3][4] | Range of values depends on length and scale. For example, decimal(6,2) can have values from -9999.99 to 9999.99. | 0.00 |
| integer | 32 bits [3] | From Integer.MIN_VALUE to Integer.MAX_VALUE (according to the Java integer data type): From -231 to 231-1. Integer.MIN_VALUE is interpreted as null. | 0 |
| long | 64 bits [3] | From Integer.MIN_VALUE to Integer.MAX_VALUE (according to the Java integer data type): From -263 to 263-1. Integer.MIN_VALUE is interpreted as null. | 0 |
| number | 64 bits [3] | Negative values are from -(2-2-52)x21023 to -2-1074, another value is 0, and positive values are from 2-1074 to (2-2-52)x21023. Three special values: NaN, -Infinity, and Infinity are defined. | 0.0 |
| string | Depends on the actual data length. Each character from the basic Unicode plane is stored in 16 bits. Characters from other planes require 32 bits per character. | A string takes (number of characters) * 2 bytes of memory (or 4 bytes if you process characters from other Unicode planes). At the same time, no record can take more than MAX_RECORD_SIZE of bytes, see Engine Configuration. | null |
| variant | Depends on the actual data length. Variant can contain all other data types. | Any value. Variant field can contain any other data type, including list or map. Lists and maps can be nested, forming arbitrary tree structure. | null |
[1] Lets you estimate how much memory your records are going to need. To do that, take a look at how many fields your record has, which data types they are and then compare the result to the MAX_RECORD_SIZE property (the maximum size of a record in bytes, see Engine Configuration). If your records are likely to have more bytes than that, simply raise the value (otherwise buffer overflow will occur).
[2] Any date can be parsed and formatted using date and time format pattern. See Date and Time Format. Parsing and formatting can also be influenced by locale. See Locale.
[3] Any numeric data type can be parsed and formatted using numeric format pattern. See Numeric Format. Parsing and formatting may also be influenced by locale. See Locale.
[4] The default length and scale of a decimal are 12 and 2, respectively. These default values of DECIMAL_LENGTH and DECIMAL_SCALE are contained in the org.jetel.data.defaultProperties file and can be changed to other values.
[5] By default, if the value of any string metadata field is an empty string, the value is converted to null instead of an empty string (""). If you want a specific value to be converted to null, use the field’s Null value property.
For other information about these data types and other data types used in Data Shaper Transformation Language (CTL), see Data Types in CTL2.
Updated 5 months ago