Primeur Online Docs
Data Mover 1.20
Data Mover 1.20
  • 🚀GETTING STARTED
    • What is Primeur Data Mover
    • Main features of Primeur Data Mover
    • Primeur Data Mover deployment
    • Navigate through Primeur Data Mover
  • 📚INFRASTRUCTURE
    • STENG, clusters and servers
      • Adding a cluster and a STENG
      • Deleting a STENG
    • DMZ Gateways
      • Blocking users automatically at the DMZ level 🚀
    • DMZ Clusters
      • Load balancing across active DMZ clusters for outbound file transfers 🚀
  • 👥Actors
    • Who are the actors
    • Create your first actor
    • Configure an actor 🚀
      • Users Tab
      • Groups Tab
      • VFS Tab
      • File Resource Tab
      • Connection Contract Tab
      • Client Connections Tab
    • Search files by actor
    • Actor Lineage 🚀
      • Aggregation of flows by protocol 🚀
      • Lineage with connection contracts 🚀
      • Lineage with input, mediation and output contracts 🚀
      • Lineage with any contract type 🚀
  • 🗄️VIRTUAL FILE SYSTEMS
    • Virtual File Systems (VFS) 🚀
      • Creating a VFS 🚀
      • Configuring a VFS
      • Adding Virtual Paths
      • Modifying and Deleting a VFS
    • Searching files in all VFS
    • Storage Classes 🚀
      • Storage Class: SMB v3 or later versions 🚀
      • Storage Class: Azure Blob Storage 🚀
      • Storage Class: Amazon S3 🚀
      • Storage Class: Google Storage 🚀
      • Storage Class: Local File System 🚀
    • Retention Classes
  • 📝Contracts
    • What is a contract
    • Create your first contract
      • Create an Input Contract
        • Define the contract info
        • Associate the contract with the actor
        • Define the contract actions
        • Set the contract variables
      • Create a Mediation Contract
      • Create an Output Contract
      • Create a Connection Contract
        • Create a contract clause
        • Associate the VFS with file processing rules
        • File Processing Rules
    • Managing contracts 🚀
    • File Resources
      • Creating File Resources
      • Navigating File Resources
      • How to use File Resources
  • 🧱Workflows
    • What is a workflow
    • Create your first workflow template
    • Trigger types
      • Trigger types for input contracts
      • Trigger types for mediation and output contracts
    • Service tasks
      • Standard service tasks
      • Triggerable service tasks 🚀
      • Spazio selectors and filebox metadata management
      • Error management
    • Variables
      • Variables in workflows and contracts
      • Handling process variables
    • Workflow templates
      • System workflow templates
        • Workflow templates for input contracts
        • Workflow templates for mediation contracts
        • Workflow templates for output contracts
      • Custom workflow templates
        • Workflow template toolbar
        • Workflow template Shape repository panel
        • Workflow template working area
        • Workflow template BPMN-diagram panel
      • Error workflow templates
    • Editing workflow templates
    • DataFlow Instance Context (DFIC) 🚀
  • 🛸TRANSPORT PROTOCOLS AND CONNECTORS
    • Data Mover client and server roles
    • Client Connections
      • Client Connection: FTP
      • Client Connection: FTPS
      • Client Connection: SFTP
      • Client Connection: HTTP
      • Client Connection: HTTPS
      • Client Connection: PESIT
      • Client Connection: SMB v3 or later versions
      • Client Connection: POP3 or IMAP
      • Client Connection: SMTP
      • Client Connection: PR4/PR4S
      • Client Connection: PR5
      • Client Connection: PR5S
      • Client Connection: HDFS
      • Client Connection: HDFSS
      • Client Connection: Amazon S3 🚀
      • Client Connection: Google Cloud Storage
        • Credentials
      • Client Connection: Azure Blob Storage
      • Client Connection: IBM Sterling Connect:Direct
      • Client Connections retry settings
    • Server Connections 🚀
      • Server Connection: FTP
      • Server Connection: FTPS
      • Server Connection: SFTP
      • Server Connection: HTTP
      • Server Connection: HTTPS
      • Server Connection: PeSIT
      • Server Connection: PR4
      • Server Connection: PR5
      • Server Connection: PR5S 🚀
      • Server Connection: IBM Sterling Connect:Direct
    • Protocol Connections Reference
    • Stopping all servers in one go
  • 💻API
    • HTTP MFT Rest API
    • Job Manager APIs 🚀
    • SFTP Server sessions APIs 🚀
    • Audit Logs APIs 🚀
  • 🔓Security
    • Identity and Access Management
    • Users & Groups
      • Setting the password policy
      • Creating Internal Users 🚀
      • Creating Internal Groups
      • Creating External Users
      • Creating External Groups
    • Key Stores and Trust Stores
      • Key Store 🚀
        • Creating a Key 🚀
        • Creating a Certificate 🚀
        • Importing a Key or a Certificate
        • Creating a Symmetric key
        • Examples
      • Trust Store 🚀
        • Importing Keys 🚀
        • Importing Certificates
      • Untrusted Cache 🚀
      • Trusting Keys and Certificates
      • PGP Key Store and PGP Trust Store
        • PGP Key Store
        • Importing keys into the PGP Trust Store
    • ICAP
      • Configuring ICAP
      • Defining an ICAP rule
  • 🎧FILE EVENT LISTENER
    • What is the File Event Listener
    • Configuring File Event Listeners
      • Setting the File Event Listener Engine
      • Defining a contract for the File Event Listener
      • Setting events to be monitored 🚀
    • RegEx Rules 🚀
    • Monitoring File Event Listeners
  • 👑FILE MANAGER
    • What is the File Manager
    • Logging into File Manager
    • Managing the File Manager 🚀
      • The list of results
      • Creating new folders
      • Uploading files
      • Downloading files 🚀
      • Searching for files and folders
      • Deleting files 🚀
      • Bulk actions 🚀
    • File Manager and VFS
    • Customizing File Manager externals
      • The configuration-wui.json file 🚀
      • How to customize the Login window and the logo
      • How to customize the footer
      • How to configure the Upload with Metadata option
      • How to customize bulk actions 🚀
  • 🧑‍⚖️FILE ROUTING
    • What is File Routing 🚀
    • Routing Rules
      • The Rules tab
      • The Categories tab
      • The Output tab
    • How to create a rule 🚀
      • Add metadata 🚀
      • Select ACTIONS
      • Select OUTPUTS
      • Policy for the selection of metadata rules
    • Configuration of the environment in Data One
      • Set up Storage Classes
      • Set up Retention Classes
      • Configure the Actor
      • Set up File Resources
    • Associate the Routing Rule with a Contract
    • Example
  • 📩NOTIFICATION CHANNELS
    • What are Notification Channels
    • Configuring the default Email Notification Channel
    • Configuring a new Email Notification Channel
    • Trusting Certificates
    • Managing Templates
      • Data Watcher Macros
      • Contract Macros
      • ICAP Macros
      • Central Log Macros
      • Email Templates
      • Editing default templates
      • Loading a new template
  • 💬LOGS & AUDIT
    • Logs 🚀
      • Logs options 🚀
      • Troubleshooting error analysis in Logs
    • Audit Options 🚀
      • Export audit logs 🚀
      • List of Audit entity types 🚀
      • Audit message codes 🚀
    • Log Notifiers 🚀
      • FEL message codes
  • 🕒MONITORING
    • Jobs
      • Details about Jobs 🚀
      • jobman.sh CLI
    • Job Manager
    • Job Queues
      • Managing Job Queues
    • File Transfers
      • Ongoing
      • Finished
      • Reports
    • File Transfers Rules
      • Configuring Rules
  • 🧐HOW TO...
    • ... use different DNS names 🚀
    • ... configure a Cron Expression
    • ... configure an application
    • ... customize the header
    • ... run searches in Data Watcher 🚀
    • ... use Data Shaper graphs in Data Mover contracts
    • ... modify DMCFG and deploy it
    • ... tune Data One data retention
    • ... fine tune Data Mover
      • Email sender for the Notification Channel
  • 🗒️RELEASE NOTES
    • Data One 1.20.10
    • Data One 1.20.9
    • Data One 1.20.8
    • Data One 1.20.7
      • Data One 1.20.7.1
    • Data One 1.20.6
    • Data One 1.20.5
    • Data One 1.20.4
    • Data One 1.20.3
    • Data One 1.20.2
    • Data One 1.20.1
    • Data One 1.20.0
Powered by GitBook
On this page
  • Connect:Direct Configuration
  • Share folders between PRIMEUR Data Mover and the Connect:Direct Node
  • Correct management of Connect:Direct process in status HOLD 🚀
  1. TRANSPORT PROTOCOLS AND CONNECTORS
  2. Client Connections

Client Connection: IBM Sterling Connect:Direct

Fields with the asterisk * are mandatory.

Properties
Description

NAME *

Name of the Connect:Direct connection you are creating. The maximum number of characters is 128. Blanks and these special characters: ' " | are not supported.

DESCRIPTION

Description of the Connect:Direct connection you are creating. The maximum number of characters is 2048.

STATUS

Enable or disable the client connection. Possible values:

  • Enabled (default value)

  • Disabled

NODE NAME *

The Connect:Direct node name of the counterpart.

USERNAME

Insert the credentials to access the counterpart Connect:Direct node.

PASSWORD

Insert the credentials to access the counterpart Connect:Direct node.

DATA ONE C:D SERVER NAME *

This fields must contain one of the "Connect:Direct" protocol type servers defined in Setup → STENG, Clusters and Servers → New Server.

SCRIPT TO SEND FILES

The Connect:Direct CDP template to send a file to this Actor. See the note below this table.

SCRIPT TO RECEIVE FILES

The Connect:Direct CDP template to receive a file from this Actor. See the note below this table.

CONFIGURATION TEST

Not enabled.

The receive and send file process template fields must contain the file name of CDP located in ${STENG_HOME}/cfg/cdptemplates.

Connect:Direct Configuration

Share folders between PRIMEUR Data Mover and the Connect:Direct Node

When PRIMEUR Data Mover and the Connect:Direct Node reside on different machines, the first and most important step is to correctly manage shared folders between them. Connect:Direct sends and receives files based on folder policies and the cdp script must contain a direct reference to the filesystems. When Connect:Direct acts as a server, files are received on its filesystem in a folder specified by the sender. If Connect:Direct and PRIMEUR Data Mover do not reside on the same machine, files can be received only if they share a folder.

To configure the Data Mover Steng for the Connect:Direct server, follow these steps:

  1. If the Connect:Direct server is remote, you must mount shared filesystems to exchange files. See How to remap shared folders below.

  2. If the Connect:Direct server is remote, go to ${STENG_HOME}/cfg to configure the speng.cd.pathremap.table file according to the shared folder defined in the previous step.

  3. Copy the Connect:Direct API library (CDJAI.jar) in this folder: ${STENG_HOME}/lib/cd.

Once the installation and the configuration are completed, restart the STENG before starting the server!

How to remap shared folders

Once the two file systems are shared, you must remap the shared folders. To do this, edit the path remap configuration speng.cd.pathremap.table file in the directory ${STENG_HOME}/cfg.

This configuration file consists of several pairs of folder paths to remap incoming and outgoing files. Below, you can find some examples.

Paths may slightly change due to differences between Windows and Unix.

#########
## SpEng Path <----> Local CD Path
## ex:
## C:\share\cd=/home/connect/share/cd
##
## SpEng Path <----> Remote CD Path
## ex:
## /winCdrive/Out/=C:\Out\
#########

# First (incoming files)
C:\CDReceive=Z:\CDReceive

# Second (push files)
Z:\CDShared=C:\CDShared

# Third (pull files)
/winCdrive/Test/Out/=C:\Test\Out\
/masterInDirCd=C:\CDShared
C:/CDShared=Z:\CDShared

The first example applies to incoming files:

• "Z:\CDReceive" is the shared partition defined on the local machine (DataOne) • "C:/CDReceive" is the shared partition on the remote Connect:Direct server

The File received by the Connect:Direct server will be placed in the "C:/CDReceive" folder. The DataOne\CD connector will be prompted that a new file has arrived. In the DataOne machine, the file is stored in the "Z:\CDReceive" folder, where it is taken and moved in the DataOne filesystem.

The second example applies to push operations. Outgoing files are copied from the DataOne filesystem to the shared folder "Z:\CDShared". When the DataOne\CD connector submits the “cdp” script to the remote Connect:Direct server, the correct remapped folder "C:\CDShared" will be specified in the Connect:Direct server.

The third example applies to pull operations. For Windows convention, we will use "/winCdrive" to indicate the "C:" drive path. The first step is to define a valid path for the Connect:Direct machine, for example: "/winCdrive/Test/Out/SourceFile.txt" and a virtual destination path, for example: "/masterInDirCd/DestFile.txt". Here below, you can find the remapping operations that are executed:

• From /winCdrive/Test/Out/ to C:\Test\Out\ (remapped path to fetch the source file on the C:D remote node) • From /masterInDirCd to C:\CDShared (remapped path where the file is received on the C:D server) • From C:/CDShared to Z:\CDShared (remapped path when the file is received, see first example)

Suppose you need to use several ConnectDirect scripts (CDP) to connect with the same actor. In that case, you must define one or more REMOTE CONNECTION configurations, each one with the relevant RECEIVE FILE PROCESS TEMPLATE and SEND FILE PROCESS TEMPLATE.

Correct management of Connect:Direct process in status HOLD 🚀

When Connect:Direct detects a transport problem (e.g., an inactive partner), the operation to be performed at the end of the retry attempts is configured in the Connect:Direct netmap.cfg file. The operation to be performed can be either to delete or to hold:

  • If delete is set, the process is deleted, and a record with the identifier PERR is issued in the statistics. Data One intercepts this record and acts accordingly, setting the transport to FAILED.

    • connect.direct.hold.timeout.sec=<seconds>

    • connect.direct.hold.delete=true|false

The addition and tuning of these parameters depend on the client's environment and system policies.

connect.direct.hold.timeout.sec=<seconds> It sets the number of seconds after which Data One considers a transport failed. Values:

  • 0 / not defined / default: wait forever.

  • > 0: seconds after which Data One considers a transport failed.

connect.direct.hold.delete=true|false It is evaluated only if connect.direct.hold.timeout.sec is set to a value higher than 0. Values:

  • true: when the Connect:Direct process is restarted during the timeout, the Data One transport will also be restarted. If the timeout expires and the process is still on HOLD, the Data One transport will be set to FAILED and the Connect:Direct process will be deleted.

  • false (default): after the transport is declared FAILED, the process in Connect:Direct is left in HOLD.

PreviousClient Connection: Azure Blob StorageNextClient Connections retry settings

Last updated 2 days ago

If hold is set, the process is not deleted and set to HOLD. Data One intercepts this status and, when the set timeout expires, sets the transport to FAILED. If a process in HOLD is restarted in Connect:Direct (even after a long time), a mismatch between Connect:Direct and Data One could occur. To avoid the mismatch, a hold timeout can be set in DMCFG ext_cfg manually (see for details):

🛸
How to modify DMCFG and deploy it