Primeur Online Docs
Data Shaper
Data Shaper
  • 🚀GETTING STARTED
    • What is Primeur Data Shaper
      • What is the Data Shaper Designer
      • What is the Data Shaper Server
      • What is the Data Shaper Cluster
    • How does the Data Shaper Designer work
      • Designer Views and Graphs
      • Data Shaper Graphs
      • Designer Projects and Sandboxes
      • Data Shaper Designer Reference
    • How do the Data Shaper Server and Cluster work
      • Data Shaper Server and Cluster
      • Data Shaper Server Reference
    • VFS Graph Components
      • DataOneFileDescriptor (DOFD) metadata
      • Passing parameters from Data One Contract to Data Shaper graph
      • Inheriting Data One context attributes in Data Shaper graph
  • DATA SHAPER DESIGNER
    • Configuration
      • Runtime configuration
        • Logging
        • Master Password
        • User Classpath
      • Data Shaper Server Integration
      • Execution monitoring
      • Java configuration
      • Engine configuration
      • Refresh Operation
    • Designer User Interface
      • Graph Editor with Palette of Components
      • Project Explorer Pane
      • Outline Pane
      • Tabs Pane
      • Execution Tab
      • Keyboard Shortcuts
    • Projects
      • Creating Data Shaper projects
      • Converting Data Shaper projects
      • Structure of Data Shaper projects
      • Versioning of server project content
      • Working with Data Shaper Server Projects
      • Project configuration
    • Graphs
      • Creating an empty graph
      • Creating a simple graph
        • Placing Components
        • Placing Components from Palette
        • Connecting Components with Edges
    • Execution
      • Successful Graph Execution
      • Run configuration
      • Connecting to a running job
      • Graph states
    • Common dialogs
      • URL file dialog
      • Edit Value dialog
      • Open Type dialog
    • Import
      • Import Data Shaper projects
      • Import from Data Shaper server sandbox
      • Import graphs
      • Import metadata
    • Export
      • Export graphs to HTML
      • Export to Data Shaper Server sandbox
      • Export image
      • Export Project as Library
    • Graph tracking
      • Changing record count font size
    • Search functionality
    • Working with Data Shaper server
      • Data Shaper server project basic principles
      • Connecting via HTTP
      • Connecting via HTTPS
      • Connecting via Proxy Server
    • Graph components
      • Adding components
      • Finding components
      • Edit component dialog box
      • Enable/disable component
      • Passing data through disabled component
      • Common properties of components
      • Specific attribute types
      • Metadata templates
    • Edges
      • Connecting Components with Edges
      • Types of Edges
      • Assigning Metadata to Edges
      • Colors of Edges
      • Debugging Edges
      • Edge Memory Allocation
    • Metadata
      • Records and Fields
        • Record Types
        • Data Types in Metadata
        • Data Formats
        • Locale and Locale Sensitivity
        • Time Zone
        • Autofilling Functions
      • Metadata Types
        • Internal Metadata
        • External (Shared) Metadata
        • SQL Query Metadata
        • Reading Metadata from Special Sources
      • Auto-propagated Metadata
        • Sources of Auto-Propagated Metadata
        • Explicitly Propagated Metadata
        • Priorities of Metadata
        • Propagation of SQL Query Metadata
      • Creating Metadata
        • Extracting Metadata from a Flat File
        • Extracting Metadata from an XLS(X) File
        • Extracting Metadata from a Database
        • Extracting Metadata from a DBase File
        • Extracting Metadata from Salesforce
        • SQL Query Metadata
        • User Defined Metadata
      • Merging Existing Metadata
      • Creating Database Table from Metadata and Database Connection
      • Metadata Editor
        • Opening Metadata Editor
        • Basics of Metadata Editor
        • Record Pane
        • Field Name vs. Label vs. Description
        • Details Pane
      • Changing and Defining Delimiters
      • Editing Metadata in the Source Code
      • Multi-value Fields
        • Lists and Maps Support in Components
        • Joining on multivalue fields (Comparison Rules)
    • Connections
      • Database Connections
        • Internal Database Connections
        • External (Shared) Database Connections
        • Database Connections Properties
        • Encryption of Access Password
        • Browsing Database and Extracting Metadata from Database Tables
        • Windows Authentication on Microsoft SQL Server
        • Snowflake Connection
        • Hive Connection
        • Troubleshooting
      • JMS Connections
      • QuickBase Connections
      • Hadoop Connections
      • Kafka Connections
      • OAuth2 Connections
      • MongoDB Connections
      • Salesforce Connections
    • Lookup Tables
      • Lookup Tables in Cluster Environment
      • Internal Lookup Tables
      • External (Shared) Lookup Tables
      • Types of Lookup Tables
    • Sequences
      • Persistent Sequences
      • Non Persistent Sequences
      • Internal Sequences
      • External (Shared) Sequences
      • Editing a Sequence
      • Sequences in Cluster Environment
    • Parameters
      • Internal Parameters
      • External (Shared) Parameters
      • Secure Graph Parameters
      • Graph Parameter Editor
      • Secure Graph Parameters
      • Parameters with CTL2 Expressions (Dynamic Parameters)
      • Environment Variables
      • Canonicalizing File Paths
      • Using Parameters
    • Internal/External Graph Elements
    • Dictionary
      • Creating a Dictionary
      • Using a Dictionary in Graphs
    • Execution Properties
    • Notes in Graphs
      • Placing Notes into Graph
      • Resizing Notes
      • Editing Notes
      • Formatted Text
      • Links from Notes
      • Folding Notes
      • Notes Properties
    • Transformations
      • Defining Transformations
      • Transform Editor
      • Common Java Interfaces
    • Data Partitioning (Parallel Running)
    • Data Partitioning in Cluster
      • High Availability
      • Scalability
      • Graph Allocation Examples
      • Example of Distributed Execution
      • Remote Edges
    • Readers
      • Common Properties of Readers
      • ComplexDataReader
      • DatabaseReader
      • DataGenerator
      • DataOneVFSReader
      • EDIFACTReader
      • FlatFileReader
      • JSONExtract
      • JSONReader
      • LDAPReader
      • MultiLevelReader
      • SpreadsheetDataReader
      • UniversalDataReader
      • X12Reader
      • XMLExtract
      • XMLReader
      • XMLXPathReader
    • Writers
      • Common Properties of Writers
      • DatabaseWriter
      • DataOneVFSWriter
      • EDIFACTWriter
      • FlatFileWriter
      • JSONWriter
      • LDAPWriter
      • SpreadsheetDataWriter
      • HIDDEN StructuredDataWriter
      • HIDDEN TableauWriter
      • Trash
      • UniversalDataWriter
      • X12Writer
      • XMLWriter
    • Transformers
      • Common Properties of Transformers
      • Aggregate
      • Concatenate
      • DataIntersection
      • DataSampler
      • Dedup
      • Denormalizer
      • ExtSort
      • FastSort
      • Filter
      • Map
      • Merge
      • MetaPivot
      • Normalizer
      • Partition
      • Pivot
      • Rollup
      • SimpleCopy
      • SimpleGather
      • SortWithinGroups
      • XSLTransformer
    • Joiners
      • Common Properties of Joiners
      • Combine
      • CrossJoin
      • DBJoin
      • ExtHashJoin
      • ExtMergeJoin
      • LookupJoin
      • RelationalJoin
    • Others
      • Common Properties of Others
      • CheckForeignKey
      • DBExecute
      • HTTPConnector
      • LookupTableReaderWriter
      • WebServiceClient
    • CTL2 - Data Shaper Transformation Language
    • Language Reference
      • Program Structure
      • Comments
      • Import
      • Data Types in CTL2
      • Literals
      • Variables
      • Dictionary in CTL2
      • Operators
      • Simple Statement and Block of Statements
      • Control Statements
      • Error Handling
      • Functions
      • Conditional Fail Expression
      • Accessing Data Records and Fields
      • Mapping
      • Parameters
      • Regular Expressions
    • CTL Debugging
      • Debug Perspective
      • Importing and Exporting Breakpoints
      • Inspecting Variables and Expressions
      • Examples
    • Functions Reference
      • Conversion Functions
      • Date Functions
      • Mathematical Functions
      • String Functions
      • Mapping Functions
      • Container Functions
      • Record Functions (Dynamic Field Access)
      • Miscellaneous Functions
      • Lookup Table Functions
      • Sequence Functions
      • Data Service HTTP Library Functions
      • Custom CTL Functions
      • CTL2 Appendix - List of National-specific Characters
      • HIDDEN Subgraph Functions
    • Tutorial
      • Creating a Transformation Graph
      • Filtering the records
      • Sorting the Records
      • Processing Speed-up with Parallelization
      • Debugging the Java Transformation
  • DATA SHAPER SERVER
    • Introduction
    • Administration
      • Monitoring
    • Using Graphs
      • Job Queue
      • Execution History
      • Job Inspector
    • Cluster
      • Sandboxes in Cluster
      • Troubleshooting
  • Install Data Shaper
    • Install Data Shaper
      • Introduction to Data Shaper installation process
      • Planning Data Shaper installation
      • Data Shaper System Requirements
      • Data Shaper Domain Master Configuration reference
      • Performing Data Shaper initial installation and master configuration
        • Creating database objects for PostgreSQL
        • Creating database objects for Oracle
        • Executing Data Shaper installer
        • Configuring additional firewall rules for Data Shaper
Powered by GitBook
On this page
  1. DATA SHAPER DESIGNER
  2. Functions Reference

Data Service HTTP Library Functions

PreviousSequence FunctionsNextCustom CTL Functions

Functions from Data Service HTTP Library are available in context of Data API jobs.

addResponseHeader

void addResponseHeader( string name, string value  );

The addResponseHeader function adds an HTTP response header field. If the function is called multiple times, multiple headers will be added. The name parameter is a header field name. See . If name is null or empty string, the response header field is not added. The value parameter is a value of the header field. If value is empty string, empty string is used. If value is null the header field is not added.

Example 343. Usage of addResponseHeader The addResponseHeader("Content-Language", "fr") adds an HTTP header field Content-Language with value fr.

Content-Language: fr

The addResponseHeader("foo", "") adds header field with empty value

foo:

The addResponseHeder("foo", null) does not add an HTTP header field because of null.

See also:

containsResponseHeader

boolean containsResponseHeader( string headerField );

The containsResponseHeader() function checks for presence of a user-added header field. It does not check existence of header fields not added by user, e.g. Server: Apache-Coyote/1.1. The check is case insensitive. The headerField parameter is a name of HTTP header field.

Example 344. Usage of containsResponseHeader There is no Content-Language header. The containsResponseHeader("Content-Language") returns false. If the header was added by setResponseHeader() function. The function containsResponseHeader() returns true.

setResponseHeader("Content-Language", "fr");
boolean a = containsResponseHeader("Content-Language"); // true

The containsResponseHeader("Server") returns false. The header was not added by user.

See also: addResponseHeader

getRequestBody

string getRequestBody(  );

The getRequestBody() function returns the request body.

Example 345. Usage of getRequestBody If you query the data service with

wget \
    --user=userName \
    --password=password \
    --method=POST \
    --body-data="Once upon a time" \
    "http://${HOST_PORT}/clover/data-service/getRequestBody"

the getRequestBody() returns Once upon a time.

See also: getRequestEncoding

getRequestClientIPAddress

string getRequestClientIPAddress(  );

The getRequestClientIPAddress() function returns the IP address of client performing the request.

Example 346. Usage of getRequestClientIPAddress If you run the Data Shaper Server locally, the getRequestClientIPAddress() returns IP address corresponding to localhost: "127.0.0.1" or "0:0:0:0:0:0:0:1".

See also: getRequestBody

getRequestContentType

string getRequestContentType(  );

The getRequestContentType() function returns the content type.

Example 347. Usage of getRequestContentType If you query the data service API with

wget \
    --user=... \
    --password=... \
    --method=POST \
    --body-data="Once upon a time" \
    "http://${HOST_PORT}/clover/data-service/getRequestContentType"

the getRequestContentType() returns application/x-www-form-urlencoded.

See also: getResponseContentType

getRequestEncoding

string getRequestEncoding(  );

The getRequestEncoding() function encoding specified in Content-Type header. If the header does not exist, the function returns null.

Example 348. Usage of getRequestEncoding If you query the data with

wget \
    --user=... \
    --password=... \
    --header="Content-Type: text/html; charset=UTF-8" \
    "http://${HOST_PORT}/clover/data-service/getRequestEncoding"

the getRequestEncoding() function returns UTF-8.

See also: setRequestEncoding

getRequestHeader

string getRequestHeader( string headerField );

The getRequestHeader() function returns value of the header field. The headerField parameter is HTTP header field name. If the header field does not exist, the function returns null.

Example 349. Usage of getRequestHeader If you query the service with

wget \
    --user=... \
    --password=... \
    --header="Accept-Language: de" \
    "http://${HOST_PORT}/clover/data-service/getRequestHeader"

the getRequestHeader("Accept-Language") returns de.

See also: getRequestHeaderNames

getRequestHeaderNames

string[] getRequestHeaderNames(  );

The getRequestHeaderNames() function returns names of request header fields.

Example 350. Usage of getRequestHeaderNames If the data service receives

GET /clover/data-service/getRequestHeaderNames HTTP/1.1
User-Agent: Wget/1.19.1 (cygwin)
Accept: */*
Accept-Encoding: identity
Host: 172.22.2.71:33754
Connection: Keep-Alive
Accept-Language: es
Authorization: Basic Y2xvdmVyOmNsb3Zlcg==
Cookie: JSESSIONID=AE6A63EA112A0BADD046CDE0D068DCC1

The getRequestHeaderNames() function returns user-agent,accept,accept-encoding,host,connection,accept-language,authorization,cookie (as a list of strings).

See also: getRequestHeader, getRequestHeaders

getRequestHeaders

map[string,string] getRequestHeaders(  );
string[] getRequestHeaders( string param );

The getRequestHeaders() function returns a map with request headers. The header name is a key, header field value is value. The param parameter is header field name.

Example 351. Usage of getRequestHeaders If you query the data service with:

curl.exe \
    --user clover:clover \
    --header "Accept-Language: de, en, es, fr" \
    --header "Accept: text/plain" \
    --header "Accept: text/html" \
    --header "Accept: application/xml" \
    --header "Accept: application/json" \
    http://${HOST_PORT}/clover/data-service/getRequestHeaders

the getRequestHeaders() function returns map with key-value pairs: User-Agent=curl/7.54.1;Accept=application/json;Accept-Language=de, en, es, fr;…​. As the function does not return a multimap, the Accept header field contains only one of the received header fields values. To get all received header field values, use the getRequestHeaders(string) function.

The getRequestHeaders("Accept") function returns list of strings: text/plain;text/html;application/xml;application/json.

See also: getRequestHeader, getRequestHeaderNames

getRequestMethod

string getRequestMethod(  );

The getRequestMethod() function returns the HTTP method: GET, POST, PUT, PATCH or DELETE.

Example 352. Usage of getRequestMethod If you query the data service with:

wget \
    --user=clover \
    --password=clover \
    --method=GET \
    "http://${HOST_PORT}/clover/data-service/getRequestMethod"

the getRequestMethod() returns GET.

getRequestParameter

string getRequestParameter( string param );

The getRequestParameter() function returns value of GET or POST parameter. The param parameter is the parameter name.

Example 353. Usage of getRequestParameter If you query the data service with:

wget \
    --user=clover \
    --password=clover \
    "http://${HOST_PORT}/clover/data-service/getRequestParameter?id=1234&"

the getRequestParameter("id") returns 1234 (as string). If you query data service on .../getRequestParameter/id/{id} URL with

wget \
    --user=clover \
    --password=clover \
    "http://${HOST_PORT}/clover/data-service/getRequestParameter/id/123"

the getRequestParameter("id") returns 123 (as string). Note parameters in the URL in data service configuration. If you query data service with

wget \
    --user=clover \
    --password=clover \
    --post-data "id=234&" \
    "http://${HOST_PORT}/clover/data-service/getRequestParameter"

the getRequestParameter("id") returns 234 (as string).

See also: getRequestParameters, getRequestParameterNames

getRequestParameterNames

string[] getRequestParameterNames(  );

Example 354. Usage of getRequestParameterNames If you query the data service with:

wget \
    --user=clover \
    --password=clover \
    "http://${HOST_PORT}/clover/data-service/getRequestParameterNames?id=123&name=doe&"

the getRequestParameterNames() returns list containing id and name.

If you query data service on ./getRequestParameterNames2/id/{id}/name/{name} with:

wget \
    --user=clover \
    --password=clover \
    "http://${HOST_PORT}/clover/data-service/getRequestParameterNames2/id/123/name/doe"

the getRequestParameterNames() function returns list containing id and name.

If you query data service with:

wget \
    --user=clover \
    --password=clover \
    --post-data "name=doe&" \
    "http://${HOST_PORT}/clover/data-service/getRequestParameterNames3?id=234&"

the getRequestParameterNames() returns list containing id and name.

See also: getRequestParameter, getRequestParameters

getRequestParameters

string[] getRequestParameters( string name );
map[string,string] getRequestParameters(  );

Example 355. Usage of getRequestParameters If you query data service with:

wget \
    --user=clover \
    --password=clover \
    "http://${HOST_PORT}/clover/data-service/getReqPar?id=123&name=doe&name=john&"

The getRequestParameters() returns map. The key is parameter name, the value is the parameter value. The getRequestParameters("name") returns list containing doe and john.

See also: getRequestParameter, getRequestParameterNames

getRequestPartFilename

string getRequestPartFilename( string paramName );

The getRequestPartFilename() function returns name of file received in multipart entity. Usually, it is a file name from HTML form. The paramName parameter is name of HTML input field containing the file.

Example 356. Usage of getRequestPartFilename If you query the web service with:

curl -F name=@/tmp/filename \
    http://example.com:8080/clover/data-service/getRequestPartFilename \
    --user clover:clover

the getRequestPartFilename("name") returns somefile.

getResponseContentType

string getResponseContentType(  );

The getResponseContentType() function returns response content type - the value of Content-Type response header field.

Example 357. Usage of getResponseContentType The getResponseContentType() returns for example application/json.

See also: getRequestContentType, setResponseContentType

getResponseEncoding

string getResponseEncoding(  );

The getResponseEncoding() function returns the response encoding.

Example 358. Usage of getResponseEncoding E.g. the getResponseEncoding() returns iso-8859-1.

See also: getRequestEncoding, setResponseEncoding

setRequestEncoding

void setRequestEncoding( string encoding );

The setRequestEncoding() function sets the encoding to be used in POST request parsing. Call this function in init().

The encoding parameter is encoding.

Example 359. Usage of setRequestEncoding The setRequestEncoding("utf-8") sets request encoding to UTF-8. The setRequestEncoding("iso-8859-2") sets request encoding to latin2. The setRequestEncoding("cp1250") sets request encoding to code page 1250.

See also: getRequestEncoding

setResponseBody

void setResponseBody( string body );

The setResponseBody() function sets HTTP response body. Consider setting the response body encoding explicitly. The body parameter is the content of the body. If you try to create response body with setResponsebody() function and with writing to response:body, the later one will be used. You should use only one way to create the response body.

Example 360. Usage of getResponseBody The setResponseBody("The response") sets the response body.

See also: setResponseEncoding

setResponseContentType

void setResponseContentType( string contentType );

The setResponseContentType() function sets the response content type - the value of Content-Type response header field. The contentType parameter is the value of Content-Type response header field.

Example 361. Usage of setResponseContentType setResponseContentType("text/plain");

See also: setResponseBody, getResponseContentType

setResponseEncoding

void setResponseEncoding( string encoding );

The setResponseEncoding() function sets HTTP response encoding. This encoding is used if you set the response body with the setResponseBody() function. The encoding parameter is the response body encoding.

Example 362. Usage of setResponseEncoding The setResponseEncoding("UTF-8"); sets response body encoding to UTF-8.

See also: getResponseEncoding, setResponseBody

setResponseHeader

void setResponseHeader( string field, string value );

The setResponseHeader() function sets the response header. If the header does not exist, it will be created. The field parameter is HTTP header field name. The value parameter is HTTP header field value.

Example 363. Usage of setResponseHeader setResponseHeader("Server", "BOA")

See also: addResponseHeader

setResponseStatus

void setResponseStatus( integer statusCode );
void setResponseStatus( integer statusCode, string message );

The setResponseStatus() function sets the response status code. The statusCode parameter is the returned status code. The message parameter is a message.

Example 364. Usage of setResponseStatus The setResponseStatus(403) sets the response status to 403. The setResponseStatus(414, "URI Too Long") returns a status code 414.

HTTP/1.1 414 URI Too Long

The getRequestParameterNames() function returns names of GET or POST parameters., e.g.

The getRequestParameters() function return map of GET or POST request parameters and request parameter values. The parameters are from URL: The name parameter is name of the parameter.

See also: ,

list of header field names
containsResponseHeader
http://example.com/?id=123&
www.example.com/getRequestParameters?id=123&name=doe&name=john&
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes
https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml