Primeur Online Docs
Data Mover 1.20
Data Mover 1.20
  • 🚀GETTING STARTED
    • What is Primeur Data Mover
    • Main features of Primeur Data Mover
    • Primeur Data Mover deployment
    • Navigate through Primeur Data Mover
  • 📚INFRASTRUCTURE
    • STENG, clusters and servers
      • Adding a cluster and a STENG
      • Deleting a STENG
    • DMZ Gateways
      • Blocking users automatically at the DMZ level 🚀
    • DMZ Clusters
      • Load balancing across active DMZ clusters for outbound file transfers 🚀
  • 👥Actors
    • Who are the actors
    • Create your first actor
    • Configure an actor 🚀
      • Users Tab
      • Groups Tab
      • VFS Tab
      • File Resource Tab
      • Connection Contract Tab
      • Client Connections Tab
    • Search files by actor
    • Actor Lineage 🚀
      • Aggregation of flows by protocol 🚀
      • Lineage with connection contracts 🚀
      • Lineage with input, mediation and output contracts 🚀
      • Lineage with any contract type 🚀
  • 🗄️VIRTUAL FILE SYSTEMS
    • Virtual File Systems (VFS) 🚀
      • Creating a VFS 🚀
      • Configuring a VFS
      • Adding Virtual Paths
      • Modifying and Deleting a VFS
    • Searching files in all VFS
    • Storage Classes 🚀
      • Storage Class: SMB v3 or later versions 🚀
      • Storage Class: Azure Blob Storage 🚀
      • Storage Class: Amazon S3 🚀
      • Storage Class: Google Storage 🚀
      • Storage Class: Local File System 🚀
    • Retention Classes
  • 📝Contracts
    • What is a contract
    • Create your first contract
      • Create an Input Contract
        • Define the contract info
        • Associate the contract with the actor
        • Define the contract actions
        • Set the contract variables
      • Create a Mediation Contract
      • Create an Output Contract
      • Create a Connection Contract
        • Create a contract clause
        • Associate the VFS with file processing rules
        • File Processing Rules
    • Managing contracts 🚀
    • File Resources
      • Creating File Resources
      • Navigating File Resources
      • How to use File Resources
  • 🧱Workflows
    • What is a workflow
    • Create your first workflow template
    • Trigger types
      • Trigger types for input contracts
      • Trigger types for mediation and output contracts
    • Service tasks
      • Standard service tasks
      • Triggerable service tasks 🚀
      • Spazio selectors and filebox metadata management
      • Error management
    • Variables
      • Variables in workflows and contracts
      • Handling process variables
    • Workflow templates
      • System workflow templates
        • Workflow templates for input contracts
        • Workflow templates for mediation contracts
        • Workflow templates for output contracts
      • Custom workflow templates
        • Workflow template toolbar
        • Workflow template Shape repository panel
        • Workflow template working area
        • Workflow template BPMN-diagram panel
      • Error workflow templates
    • Editing workflow templates
    • DataFlow Instance Context (DFIC) 🚀
  • 🛸TRANSPORT PROTOCOLS AND CONNECTORS
    • Data Mover client and server roles
    • Client Connections
      • Client Connection: FTP
      • Client Connection: FTPS
      • Client Connection: SFTP
      • Client Connection: HTTP
      • Client Connection: HTTPS
      • Client Connection: PESIT
      • Client Connection: SMB v3 or later versions
      • Client Connection: POP3 or IMAP
      • Client Connection: SMTP
      • Client Connection: PR4/PR4S
      • Client Connection: PR5
      • Client Connection: PR5S
      • Client Connection: HDFS
      • Client Connection: HDFSS
      • Client Connection: Amazon S3 🚀
      • Client Connection: Google Cloud Storage
        • Credentials
      • Client Connection: Azure Blob Storage
      • Client Connection: IBM Sterling Connect:Direct
      • Client Connections retry settings
    • Server Connections 🚀
      • Server Connection: FTP
      • Server Connection: FTPS
      • Server Connection: SFTP
      • Server Connection: HTTP
      • Server Connection: HTTPS
      • Server Connection: PeSIT
      • Server Connection: PR4
      • Server Connection: PR5
      • Server Connection: PR5S 🚀
      • Server Connection: IBM Sterling Connect:Direct
    • Protocol Connections Reference
    • Stopping all servers in one go
  • 💻API
    • HTTP MFT Rest API
    • Job Manager APIs 🚀
    • SFTP Server sessions APIs 🚀
    • Audit Logs APIs 🚀
  • 🔓Security
    • Identity and Access Management
    • Users & Groups
      • Setting the password policy
      • Creating Internal Users 🚀
      • Creating Internal Groups
      • Creating External Users
      • Creating External Groups
    • Key Stores and Trust Stores
      • Key Store 🚀
        • Creating a Key 🚀
        • Creating a Certificate 🚀
        • Importing a Key or a Certificate
        • Creating a Symmetric key
        • Examples
      • Trust Store 🚀
        • Importing Keys 🚀
        • Importing Certificates
      • Untrusted Cache 🚀
      • Trusting Keys and Certificates
      • PGP Key Store and PGP Trust Store
        • PGP Key Store
        • Importing keys into the PGP Trust Store
    • ICAP
      • Configuring ICAP
      • Defining an ICAP rule
  • 🎧FILE EVENT LISTENER
    • What is the File Event Listener
    • Configuring File Event Listeners
      • Setting the File Event Listener Engine
      • Defining a contract for the File Event Listener
      • Setting events to be monitored 🚀
    • RegEx Rules 🚀
    • Monitoring File Event Listeners
  • 👑FILE MANAGER
    • What is the File Manager
    • Logging into File Manager
    • Managing the File Manager 🚀
      • The list of results
      • Creating new folders
      • Uploading files
      • Downloading files 🚀
      • Searching for files and folders
      • Deleting files 🚀
      • Bulk actions 🚀
    • File Manager and VFS
    • Customizing File Manager externals
      • The configuration-wui.json file 🚀
      • How to customize the Login window and the logo
      • How to customize the footer
      • How to configure the Upload with Metadata option
      • How to customize bulk actions 🚀
  • 🧑‍⚖️FILE ROUTING
    • What is File Routing 🚀
    • Routing Rules
      • The Rules tab
      • The Categories tab
      • The Output tab
    • How to create a rule 🚀
      • Add metadata 🚀
      • Select ACTIONS
      • Select OUTPUTS
      • Policy for the selection of metadata rules
    • Configuration of the environment in Data One
      • Set up Storage Classes
      • Set up Retention Classes
      • Configure the Actor
      • Set up File Resources
    • Associate the Routing Rule with a Contract
    • Example
  • 📩NOTIFICATION CHANNELS
    • What are Notification Channels
    • Configuring the default Email Notification Channel
    • Configuring a new Email Notification Channel
    • Trusting Certificates
    • Managing Templates
      • Data Watcher Macros
      • Contract Macros
      • ICAP Macros
      • Central Log Macros
      • Email Templates
      • Editing default templates
      • Loading a new template
  • 💬LOGS & AUDIT
    • Logs 🚀
      • Logs options 🚀
      • Troubleshooting error analysis in Logs
    • Audit Options 🚀
      • Export audit logs 🚀
      • List of Audit entity types 🚀
      • Audit message codes 🚀
    • Log Notifiers 🚀
      • FEL message codes
  • 🕒MONITORING
    • Jobs
      • Details about Jobs 🚀
      • jobman.sh CLI
    • Job Manager
    • Job Queues
      • Managing Job Queues
    • File Transfers
      • Ongoing
      • Finished
      • Reports
    • File Transfers Rules
      • Configuring Rules
  • 🧐HOW TO...
    • ... use different DNS names 🚀
    • ... configure a Cron Expression
    • ... configure an application
    • ... customize the header
    • ... run searches in Data Watcher 🚀
    • ... use Data Shaper graphs in Data Mover contracts
    • ... modify DMCFG and deploy it
    • ... tune Data One data retention
    • ... fine tune Data Mover
      • Email sender for the Notification Channel
  • 🗒️RELEASE NOTES
    • Data One 1.20.10
    • Data One 1.20.9
    • Data One 1.20.8
    • Data One 1.20.7
      • Data One 1.20.7.1
    • Data One 1.20.6
    • Data One 1.20.5
    • Data One 1.20.4
    • Data One 1.20.3
    • Data One 1.20.2
    • Data One 1.20.1
    • Data One 1.20.0
Powered by GitBook
On this page
  1. FILE EVENT LISTENER

Monitoring File Event Listeners

PreviousRegEx Rules 🚀NextWhat is the File Manager

Last updated 5 months ago

In the Central Log, a new STENG File Event Listener Engine entry is available in the Module column:

To describe how the File Event Listener logs are managed and how to read them, let’s suppose you have the File Event Listener Engine installed on 2 different peers, Peer1 and Peer7:

  • On Peer1, the File Event Listener is the leader and monitors directories.

  • On Peer7, the File Event Listener is correctly configured and keeps waiting as long as needed. This happens because only one Peer leader can exist and in our case the leader is Peer1.

The Message column of the Central Log lists the phases the File Event Listener Engine goes through. The last entry shows that it is running as leader on Peer1:

On Peer7 as well the File Event Listener Engine has several entries listing the phases. Since it is not running as leader, no entry about having the leader lock will appear:

If Peer1 is shut down, the File Event Listener Engine configured on Peer7 becomes the leader and starts monitoring the directories. The Central Log for Peer1 will show: FEL engine terminated

And for Peer7: FEL Engine running with leader lock acquired

From now on, the File Event Listener Engine installed on Peer7 is the leader. Note that it will remain leader until it is shut down. Therefore, the roles of the File Event Listener Engines on Peer1 and Peer7 are now reversed:

  • File Event Listener on Peer7 is the leader and monitors directories.

  • File Event Listener on Peer1 is active and will keep waiting as long as needed.

When Peer1 is up and running again, the last entry in the Central Log will be: FEL Engine started

Each File Event Listener controls two actions that can occur in the directories it monitors:

  • new files added to the directories

  • existing files updated in the directories

When one of these actions occurs, an existing on-demand FEL input contract is triggered to perform actions. To give an example, the File Event Listener Engine configured on Peer7 can be associated to an input Contract called ContractNameNew that moves files to a virtual directory in Data One. Let’s see what we can find in the Central Log:

  • the on-demand FEL input Contract associated to Peer7 is in the Contract column.

  • the name of the on-demand FEL input contract triggered (ContractNameNew), the action executed (ActionNameNew), the Peer (Peer7), the workerId and other information is listed in the Message column.

In the Central Log the entries of a File Event Listener can be filtered by name. To do so, enter the name of the File Event Listener in the JOB column. Additionally, you can select STENG File Event Listener Engine in the MODULE column. Doing so, all Logs for the File Event Listener specified will be listed.

Note that when selecting the STENG File Event Listener Engine option in the Module column, the Job column will list File Event Listeners. Since a JobID is not available in this phase, the Job column is used to identify anything happening in the STENG related to a specific worker.

Logs can also be grouped by correlation ID, clicking the GROUP BY LCID (CORRELATION ID) button. In this way, logs belonging to the same instance are displayed together and issues can be checked more easily.

The File Event Listener sets multiple identities in the Central Log:

  • A long-running identity that groups together all the activities of a given File Event Listener for each STENG cluster peer, using the LCID column.

  • A short-running identity applied to each contract invocation (i.e., to each event of each monitored file), using the DFIID column.

🎧