What is the File Event Listener

The File Event Listener Engine is one of the components of Primeur Data Mover whose goal is to monitor changes to files residing on shared file systems, SMB3 shares, and SFTP Servers accessible from STENG peers of a STENG cluster in an efficient way.

Monitored locations are not directly managed by Data Mover but simply use a source for customer-provided input files to be processed by Data Mover.

The File Event Listener Engine detects all changes in the monitored directories, i.e., new files appearing in those directories as well as existing files being updated.

Once the File Event Listener detects a change, an on-demand FEL input contract is triggered to process the file.

A file with the same filename, file size, and identical last modification timestamp, appearing twice in the same location within 24 hours, will be ignored.

The File Event Listener Engine operates in a parallel fashion, with one distinct thread for the monitoring of a group of directories. These worker threads are called File Event Listeners.

The File Event Listener Engine runtime behavior is active/passive: peers coordinate to elect the leader and ensure that at any given time just one peer holds the leader role and polls directories, while all other peers will stand-by, waiting to take over the role of leader in case of crash or shutdown of the current leader.

Irrespective of monitored location types, the File Event Listener persists internal state information on a dedicated shared file system, which requires an NFS 4.1 and these mandatory mount settings:

nfsvers=4.1,sec=sys,sync,intr,noac,hard,lookupcache=none,timeo=50,retrans=2