Primeur Online Docs
Data Mover 1.20
Data Mover 1.20
  • 🚀GETTING STARTED
    • What is Primeur Data Mover
    • Main features of Primeur Data Mover
    • Primeur Data Mover deployment
    • Navigate through Primeur Data Mover
  • 👥Actors
    • Who are the actors
    • Create your first actor
    • Configure an actor 🚀
      • Users Tab
      • Groups Tab
      • VFS Tab
      • File Resource Tab
      • Connection Contract Tab
      • Client Connections Tab
    • Search files by actor
    • Actor Lineage 🚀
      • Aggregation of flows by protocol 🚀
      • Lineage with connection contracts 🚀
      • Lineage with input, mediation and output contracts 🚀
      • Lineage with any contract type 🚀
  • 📝Contracts
    • What is a contract
    • Create your first contract
      • Create an Input Contract
        • Define the contract info
        • Associate the contract with the actor
        • Define the contract actions
        • Set the contract variables
      • Create a Mediation Contract
      • Create an Output Contract
      • Create a Connection Contract
        • Create a contract clause
        • Associate the VFS with file processing rules
        • File Processing Rules
    • Managing contracts 🚀
  • 🧱Workflows
    • What is a workflow
    • Create your first workflow template
    • Trigger types
      • Trigger types for input contracts
      • Trigger types for mediation and output contracts
    • Service tasks
      • Standard service tasks
      • Triggerable service tasks 🚀
      • Spazio selectors and filebox metadata management
      • Error management
    • Variables
      • Variables in workflows and contracts
      • Handling process variables
    • Workflow templates
      • System workflow templates
        • Workflow templates for input contracts
        • Workflow templates for mediation contracts
        • Workflow templates for output contracts
      • Custom workflow templates
        • Workflow template toolbar
        • Workflow template Shape repository panel
        • Workflow template working area
        • Workflow template BPMN-diagram panel
      • Error workflow templates
    • Editing workflow templates
    • DataFlow Instance Context (DFIC) 🚀
  • 🔓Security
    • Identity and Access Management
    • Users & Groups
      • Setting the password policy
      • Creating Internal Users 🚀
      • Creating Internal Groups
      • Creating External Users
      • Creating External Groups
    • Key Stores and Trust Stores
      • Key Store 🚀
        • Creating a Key 🚀
        • Creating a Certificate 🚀
        • Importing a Key or a Certificate
        • Creating a Symmetric key
        • Examples
      • Trust Store 🚀
        • Importing Keys 🚀
        • Importing Certificates
      • Untrusted Cache 🚀
      • Trusting an element
        • When do I use the Keys tab?
        • When do I use the Certificates tab?
      • PGP Key Store / PGP Trust Store
        • Configuring the PGP Key Store
        • Importing keys into the PGP Trust Store
  • 🛸TRANSPORT PROTOCOLS AND CONNECTORS
    • Data Mover client and server roles
    • Client Connections
      • Client Connection: FTP
      • Client Connection: FTPS
      • Client Connection: SFTP
      • Client Connection: HTTP
      • Client Connection: HTTPS
      • Client Connection: PESIT
      • Client Connection: SMB v3 or later versions
      • Client Connection: POP3 IMAP
      • Client Connection: SMTP
      • Client Connection: PR4/PR4S
      • Client Connection: PR5
      • Client Connection: PR5S
      • Client Connection: HDFS
      • Client Connection: HDFSS
      • Client Connection: Amazon S3 🚀
      • Client Connection: Google Cloud Storage
        • Credentials
      • Client Connection: Azure Blob Storage
      • Client Connection: IBM Sterling Connect:Direct
      • Appendix
    • Server Connections 🚀
      • Server Connection: FTP
      • Server Connection: FTPS
      • Server Connection: SFTP
      • Server Connection: HTTP
      • Server Connection: HTTPS
      • Server Connection: PeSIT
      • Server Connection: PR4
      • Server Connection: PR5
      • Server Connection: PR5S 🚀
      • Server Connection: IBM Sterling Connect:Direct
    • Stopping all servers in one go
  • 🛰️DMZ GATEWAYS
    • DMZ Gateways
    • DMZ Clusters
  • 🎧FILE EVENT LISTENER
    • What is the File Event Listener
    • Configuring File Event Listeners
      • Setting the File Event Listener Engine
      • Defining a contract for the File Event Listener
      • Setting events to be monitored
    • RegEx Rules 🚀
    • Monitoring File Event Listeners
  • 🔍ICAP
    • ICAP Engines
    • Configuring an ICAP Engine
    • Defining an ICAP rule
  • 📚CLUSTERING
    • STENG, Clusters and Servers
    • Adding a cluster and a STENG
    • Deleting a STENG
  • 🕒MONITORING
    • Jobs
      • Details about Jobs 🚀
      • jobman.sh CLI
    • Job Manager
    • Job Queues
      • Managing Job Queues
    • File Transfers
      • Ongoing
      • Finished
      • Reports
    • File Transfers Rules
      • Configuring Rules
  • 🤓ADMINISTRATION
    • Storage Classes 🚀
      • Storage Class: File System 🚀
      • Storage Class: SMB v3 or later versions 🚀
      • Storage Class: Amazon S3 🚀
      • Storage Class: Google Cloud Storage 🚀
      • Storage Class: Azure Blob Storage 🚀
    • Retention Classes
    • Virtual File Systems (VFS) 🚀
      • Creating a VFS 🚀
      • Configuring a VFS
      • Adding Virtual Paths
      • Modifying and Deleting a VFS
      • Searching files in all VFS
    • File Resources
      • Creating File Resources
      • Navigating File Resources
      • How to use File Resources
    • Advanced Settings
  • 👑FILE MANAGER
    • Getting started
    • Logging into File Manager
    • Managing the File Manager 🚀
      • The list of results
      • Creating new folders
      • Uploading files
      • Downloading files 🚀
      • Searching for files and folders
      • Deleting files 🚀
      • Bulk actions 🚀
    • File Manager and VFS
    • Customizing File Manager externals
      • The configuration-wui.json file 🚀
      • How to customize the Login window and the logo
      • How to customize the footer
      • How to configure the Upload with Metadata option
      • How to customize bulk actions 🚀
  • 🧑‍⚖️FILE ROUTING
    • What is File Routing 🚀
    • Routing Rules page
      • The Rules tab
      • The Categories tab
      • The Output tab
    • How to create a rule 🚀
      • Add metadata 🚀
      • Select ACTIONS
      • Select OUTPUTS
      • Policy for the selection of metadata rules
    • Configuration of the environment in Data One
      • Set up Storage Classes
      • Set up Retention Classes
      • Configure the Actor
      • Set up File Resources
    • Associate the Routing Rule with a Contract
    • Example
  • 💬LOGS & AUDIT
    • Logs 🚀
      • Logs options 🚀
      • Troubleshooting error analysis in Logs
    • Audit Options 🚀
      • Export audit logs 🚀
      • List of Audit entity types 🚀
      • Audit message codes 🚀
    • Log Notifiers 🚀
      • FEL message codes
  • 📩NOTIFICATION CHANNELS
    • What are Notification Channels
    • Configuring the default Email Notification Channel
    • Configuring a new Email Notification Channel
    • Trusting Certificates
    • Managing Templates
      • Data Watcher Macros
      • Contract Macros
      • ICAP Macros
      • Central Log Macros
      • Email Templates
      • Editing default templates
      • Loading a new template
  • 🟣DATA MOVER + DATA WATCHER
    • Data Mover in a bundle with Data Watcher
    • Attributes 🚀
    • Cut-off Board
      • Cut-off Calendars
    • Dataflow Inquiry
  • 🟠DATA MOVER + DATA SHAPER
    • Data Mover in a bundle with Data Shaper
    • Monitoring
    • Execution History
    • Sandboxes
  • 💻API
    • HTTP MFT Rest API
    • Job Manager APIs 🚀
    • SFTP Server sessions APIs 🚀
    • Audit Logs APIs 🚀
  • 🧐HOW TO...
    • ... use different DNS names - NEW! 🚀
    • ... configure a Cron Expression
    • ... configure an Application
    • ... customize a header
    • ... run searches in Data Watcher 🚀
    • ... use Data Shaper graphs in Data Mover contracts
    • ... modify DMCFG and deploy it
    • ... tune Data One data retention
  • 🗒️RELEASE NOTES
    • Data One 1.20.10
    • Data One 1.20.9
    • Data One 1.20.8
    • Data One 1.20.7
      • Data One 1.20.7.1
    • Data One 1.20.6
    • Data One 1.20.5
    • Data One 1.20.4
    • Data One 1.20.3
    • Data One 1.20.2
    • Data One 1.20.1
    • Data One 1.20.0
Powered by GitBook
On this page
  • Connect:Direct Configuration
  • Share folders between PRIMEUR Data One and the Connect:Direct Node
  • Correct management of Connect:Direct process in status HOLD 🚀
  1. TRANSPORT PROTOCOLS AND CONNECTORS
  2. Client Connections

Client Connection: IBM Sterling Connect:Direct

Fields with the asterisk * are mandatory.

Properties
Description

NAME *

Name of the Connect:Direct connection you are creating. The maximum number of characters is 128. Blanks and these special characters: ' " | are not supported.

DESCRIPTION

Description of the Connect:Direct connection you are creating. The maximum number of characters is 2048.

STATUS

Enable or disable the client connection. Possible values:

  • Enabled (default value)

  • Disabled

NODE NAME *

The Connect:Direct node name of the counterpart.

USERNAME

Insert the credentials to access the counterpart Connect:Direct node.

PASSWORD

Insert the credentials to access the counterpart Connect:Direct node.

DATA ONE C:D SERVER NAME *

This fields must contain one of the "Connect:Direct" protocol type servers defined in Setup → STENG, Clusters and Servers → New Server.

SCRIPT TO SEND FILES

The Connect:Direct CDP template to send a file to this Actor. See the note below this table.

SCRIPT TO RECEIVE FILES

The Connect:Direct CDP template to send a file to this Actor. See the note below this table.

CONFIGURATION TEST

The receive and send file process template fields must contain the file name of CDP located in ${SPENG_HOME}/cfg/cdptemplates.

Connect:Direct Configuration

Share folders between PRIMEUR Data One and the Connect:Direct Node

When PRIMEUR Data One and the Connect:Direct Node reside on different machines, the first and most important step is to correctly manage shared folders between them. Connect:Direct sends and receives files based on folder policies and the cdp script must contain a direct reference to the filesystems. When Connect:Direct acts as a server, files are received on its filesystem in a folder specified by the sender. If Connect:Direct and PRIMEUR Data One do not reside on the same machine, files can be received only if they share a folder.

To configure the Data One Steng for the Connect:Direct server, follow these steps:

  1. If the Connect:Direct server is remote, you must mount shared filesystems to exchange files. See How to remap shared folders below.

  2. If the Connect:Direct server is remote, go to /ghibli-home/wlp/usr/servers/steng/cfg to configure the speng.cd.pathremap.table file according to the shared folder defined in the previous step.

  3. Copy the Connect:Direct API library (CDJAI.jar) in this folder: /ghibli-home/wlp/usr/servers/steng/lib/cd.

Important Note: Once the installation and the configuration are completed, restart the STENG before starting the server!

How to remap shared folders

Once the two file systems are shared, you must remap the shared folders. To do this, edit the path remap configuration speng.cd.pathremap.table file in the directory /ghibli-home/wlp/usr/servers/steng/cfg.

This configuration file consists of several pairs of folder paths to remap incoming and outgoing files. Below, you can find some examples. Note: Paths may slightly change due to differences between Windows and Unix.

Paths may slightly change due to differences between Windows and Unix.

#########
## SpEng Path <----> Local CD Path
## ex:
## C:\share\cd=/home/connect/share/cd
##
## SpEng Path <----> Remote CD Path
## ex:
## /winCdrive/Out/=C:\Out\
#########

# First (incoming files)
C:\CDReceive=Z:\CDReceive

# Second (push files)
Z:\CDShared=C:\CDShared

# Third (pull files)
/winCdrive/Test/Out/=C:\Test\Out\
/masterInDirCd=C:\CDShared
C:/CDShared=Z:\CDShared

The first example applies to incoming files:

• "Z:\CDReceive" is the shared partition defined on the local machine (DataOne) • "C:/CDReceive" is the shared partition on the remote Connect:Direct server

The File received by the Connect:Direct server will be placed in the "C:/CDReceive" folder. The DataOne\CD connector will be prompted that a new file has arrived. In the DataOne machine, the file is stored in the "Z:\CDReceive" folder, where it is taken and moved in the DataOne filesystem.

The second example applies to push operations. Outgoing files are copied from the DataOne filesystem to the shared folder "Z:\CDShared". When the DataOne\CD connector submits the “cdp” script to the remote Connect:Direct server, the correct remapped folder "C:\CDShared" will be specified in the Connect:Direct server.

The third example applies to pull operations. For Windows convention, we will use "/winCdrive" to indicate the "C:" drive path. The first step is to define a valid path for the Connect:Direct machine, for example: "/winCdrive/Test/Out/SourceFile.txt" and a virtual destination path, for example: "/masterInDirCd/DestFile.txt". Here below, you can find the remapping operations that are executed:

• From /winCdrive/Test/Out/ to C:\Test\Out\ (remapped path to fetch the source file on the C:D remote node) • From /masterInDirCd to C:\CDShared (remapped path where the file is received on the C:D server) • From C:/CDShared to Z:\CDShared (remapped path when the file is received, see first example)

Suppose you need to use several ConnectDirect scripts (CDP) to connect with the same actor. In that case, you must define one or more REMOTE CONNECTION configurations, each one with the relevant RECEIVE FILE PROCESS TEMPLATE and SEND FILE PROCESS TEMPLATE.

Correct management of Connect:Direct process in status HOLD 🚀

When Connect:Direct detects a transport problem (e.g., an inactive partner), the operation to be performed at the end of the retry attempts is configured in the Connect:Direct netmap.cfg file. The operation to be performed can be either to delete or to hold:

  • If delete is set, the process is deleted, and a record with the identifier PERR is issued in the statistics. Data One intercepts this record and acts accordingly, setting the transport to FAILED.

  • If hold is set, the process is not deleted and set to HOLD. Data One intercepts this status and, when the set timeout expires, sets the transport to FAILED. If a process in HOLD is restarted in Connect:Direct (even after a long time), a mismatch between Connect:Direct and Data One could occur. To avoid the mismatch, a hold timeout can be set in jvm.options manually:

    • connect.direct.hold.timeout.sec=<seconds>

    • connect.direct.hold.delete=true|false

The addition and tuning of these parameters depend on the client's environment and system policies.

connect.direct.hold.timeout.sec=<seconds> It sets the number of seconds after which Data One considers a transport failed. Values:

  • 0 / not defined / default: wait forever.

  • > 0: seconds after which Data One considers a transport failed.

connect.direct.hold.delete=true|false It is evaluated only if connect.direct.hold.timeout.sec is set to a value higher than 0. Values:

  • true: when the Connect:Direct process is restarted during the timeout, the Data One transport will also be restarted. If the timeout expires and the process is still on HOLD, the Data One transport will be set to FAILED and the Connect:Direct process will be deleted.

  • false (default): after the transport is declared FAILED, the process in Connect:Direct is left in HOLD.

PreviousClient Connection: Azure Blob StorageNextAppendix

Last updated 11 days ago

Use this section to test that the Cluster and the STENG are connected correctly. Select a Cluster and a STENG you want to test and click the TEST button. Data One automatically selects a Cluster and a STENG to test the connection. Note that a Cluster will be selected only if it is the only one available. A message with a green check will confirm a successful connection. A message with a red exclamation mark will warn about an unsuccessful connection. The message also details the reason why the connection fails. The connection might not be successful for different reasons, each of them depending on the transport type and the configuration. The first time you try to test the connection, if trusting certificates are missing, an error message detailing the unsuccessful connection will appear. In this case, go to the Setup → Untrusted Cache → Certificates tab and trust the relevant certificates. Note that trusting certificates in the Untrusted Cache page may not be sufficient to establish a successful connection. Refer to the section of this documentation for all the details.

🛸
Untrusted Cache