Primeur Online Docs
Data Shaper
Data Shaper
  • 🚀GETTING STARTED
    • What is Primeur Data Shaper
      • What is the Data Shaper Designer
      • What is the Data Shaper Server
      • What is the Data Shaper Cluster
    • How does the Data Shaper Designer work
      • Designer Views and Graphs
      • Data Shaper Graphs
      • Designer Projects and Sandboxes
      • Data Shaper Designer Reference
    • How do the Data Shaper Server and Cluster work
      • Data Shaper Server and Cluster
      • Data Shaper Server Reference
    • VFS Graph Components
      • DataOneFileDescriptor (DOFD) metadata
      • Passing parameters from Data One Contract to Data Shaper graph
      • Inheriting Data One context attributes in Data Shaper graph
  • DATA SHAPER DESIGNER
    • Configuration
      • Runtime configuration
        • Logging
        • Master Password
        • User Classpath
      • Data Shaper Server Integration
      • Execution monitoring
      • Java configuration
      • Engine configuration
      • Refresh Operation
    • Designer User Interface
      • Graph Editor with Palette of Components
      • Project Explorer Pane
      • Outline Pane
      • Tabs Pane
      • Execution Tab
      • Keyboard Shortcuts
    • Projects
      • Creating Data Shaper projects
      • Converting Data Shaper projects
      • Structure of Data Shaper projects
      • Versioning of server project content
      • Working with Data Shaper Server Projects
      • Project configuration
    • Graphs
      • Creating an empty graph
      • Creating a simple graph
        • Placing Components
        • Placing Components from Palette
        • Connecting Components with Edges
    • Execution
      • Successful Graph Execution
      • Run configuration
      • Connecting to a running job
      • Graph states
    • Common dialogs
      • URL file dialog
      • Edit Value dialog
      • Open Type dialog
    • Import
      • Import Data Shaper projects
      • Import from Data Shaper server sandbox
      • Import graphs
      • Import metadata
    • Export
      • Export graphs to HTML
      • Export to Data Shaper Server sandbox
      • Export image
      • Export Project as Library
    • Graph tracking
      • Changing record count font size
    • Search functionality
    • Working with Data Shaper server
      • Data Shaper server project basic principles
      • Connecting via HTTP
      • Connecting via HTTPS
      • Connecting via Proxy Server
    • Graph components
      • Adding components
      • Finding components
      • Edit component dialog box
      • Enable/disable component
      • Passing data through disabled component
      • Common properties of components
      • Specific attribute types
      • Metadata templates
    • Edges
      • Connecting Components with Edges
      • Types of Edges
      • Assigning Metadata to Edges
      • Colors of Edges
      • Debugging Edges
      • Edge Memory Allocation
    • Metadata
      • Records and Fields
        • Record Types
        • Data Types in Metadata
        • Data Formats
        • Locale and Locale Sensitivity
        • Time Zone
        • Autofilling Functions
      • Metadata Types
        • Internal Metadata
        • External (Shared) Metadata
        • SQL Query Metadata
        • Reading Metadata from Special Sources
      • Auto-propagated Metadata
        • Sources of Auto-Propagated Metadata
        • Explicitly Propagated Metadata
        • Priorities of Metadata
        • Propagation of SQL Query Metadata
      • Creating Metadata
        • Extracting Metadata from a Flat File
        • Extracting Metadata from an XLS(X) File
        • Extracting Metadata from a Database
        • Extracting Metadata from a DBase File
        • Extracting Metadata from Salesforce
        • SQL Query Metadata
        • User Defined Metadata
      • Merging Existing Metadata
      • Creating Database Table from Metadata and Database Connection
      • Metadata Editor
        • Opening Metadata Editor
        • Basics of Metadata Editor
        • Record Pane
        • Field Name vs. Label vs. Description
        • Details Pane
      • Changing and Defining Delimiters
      • Editing Metadata in the Source Code
      • Multi-value Fields
        • Lists and Maps Support in Components
        • Joining on multivalue fields (Comparison Rules)
    • Connections
      • Database Connections
        • Internal Database Connections
        • External (Shared) Database Connections
        • Database Connections Properties
        • Encryption of Access Password
        • Browsing Database and Extracting Metadata from Database Tables
        • Windows Authentication on Microsoft SQL Server
        • Snowflake Connection
        • Hive Connection
        • Troubleshooting
      • JMS Connections
      • QuickBase Connections
      • Hadoop Connections
      • Kafka Connections
      • OAuth2 Connections
      • MongoDB Connections
      • Salesforce Connections
    • Lookup Tables
      • Lookup Tables in Cluster Environment
      • Internal Lookup Tables
      • External (Shared) Lookup Tables
      • Types of Lookup Tables
    • Sequences
      • Persistent Sequences
      • Non Persistent Sequences
      • Internal Sequences
      • External (Shared) Sequences
      • Editing a Sequence
      • Sequences in Cluster Environment
    • Parameters
      • Internal Parameters
      • External (Shared) Parameters
      • Secure Graph Parameters
      • Graph Parameter Editor
      • Secure Graph Parameters
      • Parameters with CTL2 Expressions (Dynamic Parameters)
      • Environment Variables
      • Canonicalizing File Paths
      • Using Parameters
    • Internal/External Graph Elements
    • Dictionary
      • Creating a Dictionary
      • Using a Dictionary in Graphs
    • Execution Properties
    • Notes in Graphs
      • Placing Notes into Graph
      • Resizing Notes
      • Editing Notes
      • Formatted Text
      • Links from Notes
      • Folding Notes
      • Notes Properties
    • Transformations
      • Defining Transformations
      • Transform Editor
      • Common Java Interfaces
    • Data Partitioning (Parallel Running)
    • Data Partitioning in Cluster
      • High Availability
      • Scalability
      • Graph Allocation Examples
      • Example of Distributed Execution
      • Remote Edges
    • Readers
      • Common Properties of Readers
      • ComplexDataReader
      • DatabaseReader
      • DataGenerator
      • DataOneVFSReader
      • EDIFACTReader
      • FlatFileReader
      • JSONExtract
      • JSONReader
      • LDAPReader
      • MultiLevelReader
      • SpreadsheetDataReader
      • UniversalDataReader
      • X12Reader
      • XMLExtract
      • XMLReader
      • XMLXPathReader
    • Writers
      • Common Properties of Writers
      • DatabaseWriter
      • DataOneVFSWriter
      • EDIFACTWriter
      • FlatFileWriter
      • JSONWriter
      • LDAPWriter
      • SpreadsheetDataWriter
      • HIDDEN StructuredDataWriter
      • HIDDEN TableauWriter
      • Trash
      • UniversalDataWriter
      • X12Writer
      • XMLWriter
    • Transformers
      • Common Properties of Transformers
      • Aggregate
      • Concatenate
      • DataIntersection
      • DataSampler
      • Dedup
      • Denormalizer
      • ExtSort
      • FastSort
      • Filter
      • Map
      • Merge
      • MetaPivot
      • Normalizer
      • Partition
      • Pivot
      • Rollup
      • SimpleCopy
      • SimpleGather
      • SortWithinGroups
      • XSLTransformer
    • Joiners
      • Common Properties of Joiners
      • Combine
      • CrossJoin
      • DBJoin
      • ExtHashJoin
      • ExtMergeJoin
      • LookupJoin
      • RelationalJoin
    • Others
      • Common Properties of Others
      • CheckForeignKey
      • DBExecute
      • HTTPConnector
      • LookupTableReaderWriter
      • WebServiceClient
    • CTL2 - Data Shaper Transformation Language
    • Language Reference
      • Program Structure
      • Comments
      • Import
      • Data Types in CTL2
      • Literals
      • Variables
      • Dictionary in CTL2
      • Operators
      • Simple Statement and Block of Statements
      • Control Statements
      • Error Handling
      • Functions
      • Conditional Fail Expression
      • Accessing Data Records and Fields
      • Mapping
      • Parameters
      • Regular Expressions
    • CTL Debugging
      • Debug Perspective
      • Importing and Exporting Breakpoints
      • Inspecting Variables and Expressions
      • Examples
    • Functions Reference
      • Conversion Functions
      • Date Functions
      • Mathematical Functions
      • String Functions
      • Mapping Functions
      • Container Functions
      • Record Functions (Dynamic Field Access)
      • Miscellaneous Functions
      • Lookup Table Functions
      • Sequence Functions
      • Data Service HTTP Library Functions
      • Custom CTL Functions
      • CTL2 Appendix - List of National-specific Characters
      • HIDDEN Subgraph Functions
    • Tutorial
      • Creating a Transformation Graph
      • Filtering the records
      • Sorting the Records
      • Processing Speed-up with Parallelization
      • Debugging the Java Transformation
  • DATA SHAPER SERVER
    • Introduction
    • Administration
      • Monitoring
    • Using Graphs
      • Job Queue
      • Execution History
      • Job Inspector
    • Cluster
      • Sandboxes in Cluster
      • Troubleshooting
  • Install Data Shaper
    • Install Data Shaper
      • Introduction to Data Shaper installation process
      • Planning Data Shaper installation
      • Data Shaper System Requirements
      • Data Shaper Domain Master Configuration reference
      • Performing Data Shaper initial installation and master configuration
        • Creating database objects for PostgreSQL
        • Creating database objects for Oracle
        • Executing Data Shaper installer
        • Configuring additional firewall rules for Data Shaper
Powered by GitBook
On this page
  • Short Description
  • Ports
  • Metadata
  • LDAPReader Attributes
  • Details
  • Alias Handling
  • Examples
  • Reading Data from LDAP
  • Looking up a Record from LDAP
  • Reading binary attributes
  • Best Practices
  • See also
  1. DATA SHAPER DESIGNER
  2. Readers

LDAPReader

PreviousJSONReaderNextMultiLevelReader

Short Description

LDAPReader reads information from an LDAP directory converting it to Data Shaper Data Records.

COMPONENT
DATA SOURCE
INPUT PORTS
OUTPUT PORTS
EACH TO ALL INPUTS
DIFFERENT TO DIFFERENT OUTPUTS
TRANSFORMATION
TRANSF. REQ.
JAVA
CTL
AUTO-PROPAGATED METADATA

LDAPReader

LDAP directory tree

1

1-n

x

x

x

x

x

x

x

Ports

PORT TYPE
NUMBER
REQUIRED
DESCRIPTION
METADATA

Input

0

x

Input records used for defining base and filter. If the input port is connected then for each input record one query is assembled and sent to the LDAP server. If such query returns no result then one empty record is sent out (with autofilling fields populated); This behavior requires the input port to be connected.

Any

Output

0

✓

For correct data records. Results of the search must have the same objectClass.

Any [1]

1-n

x

For correct data records

Output 0

[1] Metadata on the output must precisely describe the structure of the read object.

Metadata

LDAPReader does not propagate metadata. LDAPReader has no metadata template. Metadata on the output must precisely describe the structure of the read object. Only Clover fields of types string and byte/compressedByte are supported.

Note that metadata field names have strict naming conventions; therefore, to map an LDAP attribute containing special characters (e.g. a dash) in its name, use a metadata field label. Metadata field labels can contain special characters and have a higher priority than field names. For example, to read from the msDS-PrincipalName LDAP attribute, use a field with label msDS-PrincipalName and any name that follows the naming convention (e.g. msDS-PrincipalName).

Metadata can use . The autofilling attribute filename is set to complete the URL (includes base, filter).

LDAPReader Attributes

ATTRIBUTE

REQ.

DESCRIPTION

POSSIBLE VALUES

Basic

LDAP URL

Yes

LDAP URL of the directory.

ldap://host:port/

Base DN

Yes

Base Distinguished Name (the root of your LDAP tree) used for LDAP search. It is a comma separated list of attribute=value pairs referring to any location with the directory, e.g. if ou=Humans,dc=example,dc=com is the root of the subtree to be searched, entries representing people from example.com domain are to be found. Optional references to input record’s fields in the form $field_name are resolved.

Filter

Yes

Filter used for the LDAP connection. attribute=value pairs as a filtering condition for the search. All entries matching the filter will be returned, e.g. mail=* returns every entry which has an email address, while objectclass=* is the standard method for returning all entries matching a given base and scope because all entries have values for objectclass. Optional references to input record’s fields in the form $field_name are resolved.

Scope

Scope of the search request. By default, only one object is searched. If onelevel, the level immediately below the distinguished name is searched. If subtree, the whole subtree below the distinguished name is searched.

object (default) | onelevel | subtree

User

No

The user DN to be used when connecting to the LDAP directory. Similar to the following: cn=john.smith,dc=example,dc=com

Password

No

The password to be used when connecting to the LDAP directory.

Advanced

Multi-value separator

No

The character/string to be used when mapping multi-value attribute on simple Clover field as concatenation of string values. LDAPReader can handle keys with multiple values. These are delimited by this string or character. is special escape value which turns off this functionality, then only the first value is read. This attribute can only be used for string data type. When byte type is used, the first value is the only one that is read.

"|" (default) | other character or string

Alias handling

To control how aliases (leaf entries pointing to another object in the namespace) are dereferenced.

always | never | finding (default)| searching

Referral handling

By default, links to other servers are ignored. If follow, the referrals are processed.

ignore (default) | follow

Page size

No

The size of the page used in paging. If >0 then LDAP server is queried in paging mode and this attribute defines how many records are returned on one page.

e.g. 256

All attributes

No

The query LDAP for all available attributes or only those directly mappable on output fields. When using defaultField then this should be set to True.

True | False

Default field

No

The name of the output field of type MAP(string) where attributes without explicit mapping (corresponding field names on the output port) will be stored.

e.g. field15

Binary attributes

No

The list of field names containing binary attributes. By default, the objectGUID is added to the list of binary attributes.

e.g. objectGUID

LDAP Connection Properties

No

Java Property-like style of key-value definitions which will be added to LDAP connection environment.

Details

LDAPReader provides the logic to extract the search results and transform them into Data Shaper Data Records. The results of the search must have the same objectClass. The metadata provided on the output port/edge (field names) are used when mapping from LDAP attributes to fields. Only string and byte (cbyte) Data Shaper data fields are supported. String is compatible with most of LDAP usual types, byte is necessary; for example, for userPassword LDAP type reading. Multi-value attributes are mapped onto target fields in two ways:

  • if target field is of type List then individual values are stored as individual items.

  • If target field is simple type (and multiValueSeparator is set) then values are concatenated with the defined separator and stored as a single value.

When the defaultMapping field is set (must be of type Map) then all unmapped attributes returned from LDAP server are stored in the map in a key→value manner. Multi-values are stored concatenated.

Alias Handling

Searching the entry to which an alias entry points is known as dereferencing an alias. Setting the Alias handling attribute, you can control the extent to which entries are searched:

  • always: Always dereference aliases.

  • never: Never dereference aliases.

  • finding: Dereference aliases in locating the base of the search but not in searching subordinates of the base.

  • searching: Dereference aliases in searching subordinates of the base but not in locating the base

Examples

Reading Data from LDAP

Read records with uid=* from ou=people,dc=foo,dc=? subtree on foobar.com (port 389). Use credentials: user uid=Manager,dc=foo,dc=bar and password manager_password. The values for dc=? will be received from the input edge in the dc field.

Solution

ATTRIBUTE
VALUE

LDAP URL

ldap://example.com:389

Base DN

ou=people,dc=foo,dc=$dc

Filter

uid=*

Scope

subtree

Looking up a Record from LDAP

Retrieve information about particular person identified by UID. The UID is received from the input edge. The information about persons is in cn=people,dc=uninett,dc=no subtree on LDAP server example.com (port 389). The metadata on output port has following fields: cn (string), displayName (string), mail (list of strings),uid(string), objectClass (list of strings), default (map of strings).

Solution

ATTRIBUTE
VALUE

LDAP URL

ldap://example.com:389

Base DN

ou=people,dc=example,dc=com

Filter

uid=$userId

Scope

subtree

The filter parameter contains a reference to the input field name userId. This reference will be resolved for all input records and LDAP query executed (and result parsed) for each input record.

Reading binary attributes

This example shows a way to read binary attributes from LDAP. Read the records from the Reading Data from LDAP example above. In addition to the example, the records contain binary field objectGUID.

//#CTL2

function integer transform() {
    $out.0.* = $in.0.*;
    $out.0.logonHours = byte2hex($in.0.logonHours);

    return ALL;
}

Similarly, you can use the byte2hex function with a prefix argument to get a hexadecimal string representation of the objectGUID attribute.

String strObjectGUID = byte2hex($in.0.objectGUID,"\\");

Best Practices

Improving search performance: If there are no alias entries in the LDAP directory that require dereferencing, choose Alias handling never option.

See also

Solution The output metadata of LDAPReader should contain a byte field for objectGUID. Use the and byte2hex function to convert the byte field to string.

Autofilling Functions
Map
LDAPWriter
Common Properties of Components
Specific Attribute Types
Common Properties of Readers