How to use File Resources

To understand the File Resource within Data Mover, it is essential to recognize its function in managing files by setting metadata attributes for both local and remote files. This involves specifying details like content type (binary or text), data format, and security features such as encryption. Local files are generated within the system through input operations in the Virtual File System, while remote files can either be incoming or outgoing, managed via output operations. These functionalities allow seamless and secure data management and conversion in-stream during file transfers.

The File Resource is a powerful feature of Data Mover for setting file metadata into the Virtual File System and to perform in-stream data conversion while reading or writing file from/to the Virtual File System.

As previously described, the File Resource attributes (local and remote) contain:

• Information declaring the content of a plain file in terms of binary or textual data, charset, record format, end of line character(s), etc.

• Information declaring if the file is compressed and/or secured (e.g., encrypted and/or signed).

The above attributes apply to the local file and the remote file.

The local file is always related to the file that will be created in the Virtual File System. We refer to the creation of a file entry in the Virtual File System as an Input Virtual File System operation. During an input Virtual File System operation, if the File Resource is used, the local file attribute will be used to set the Virtual File System file entry metadata.

Depending on the context of the File Resource usage, the remote file is the receiving file to be inserted in the Virtual File System or the file to be sent to its destination. We refer to the read operation of the file entry from the Virtual File System as an Output Virtual File System operation.

To clarify this behavior, we can consider two usage contexts: a transport sending a file to a remote destination, and a transport receiving a file from a remote origin.

File Resource usage when a transport sends a file

When a transport sends a file, it reads the file from the Virtual File System (Output operation) and, in this case, the File Resource local attributes related to the file contents are not used (because the only valid local file attributes are the ones associated with the Virtual File System file entry). The File Resource remote file attributes are, instead, used to convert the read file before it is sent to its destination.

If, for example, in the Virtual File System, the file is defined with an "819" charset name, and in the remote attribute of the File Resource, the charset name is "1047", a charset conversion from "819" to "1047" will be executed in-stream before data is sent. If the remote file defines security and/or compression attributes, data is secured and/or compressed as well.

File Resource usage when a transport receives a file

When a transport receives a file, its contents are described by the remote attributes of the File Resource, but the final format of the file that will be written and inserted into the Virtual File System (Input operation) is described by the local attributes of the File Resource. If, for example, in the remote attributes of the File Resource the file is defined with an "819" charset name, and in the local attribute of the File Resource the charset name is "1047", a charset conversion from "819" to "1047" will be executed in-stream before data is written. If the local file defines security and/or compression attributes, data will be secured and/or compressed as well. It is important to state that the local file attributes of the File Resource are then copied into the Virtual File System entry metadata. When a File Resource is created, its usage context is not defined at first. For this reason, it is mandatory to define both local and remote attributes. Typically, the File Resource is used by the transport when the file is received from a remote origin or sent to a remote destination, allowing for an automatic conversion of file content. Once the File Resource is defined, to use it, it is necessary to create other configuration items where the created File Resource is referred. The file resource is referred to in the Contract during the setting of the File Processing Rules (see the Contract section).

Notes

• When the File Resource is used during an operation where a Virtual File System file entry is read (for example when a transport reads a file to be sent to a remote destination) then the local attributes of the File Resource are ignored because the file attributes used are the metadata associated to the Virtual File System entry.

• Some transport protocols (e.g., PR4 and PR5), when receiving a file, ignore the remote attributes of the File Resource because the content and format type of the receiving file is already known (via metadata sent by the client to the server).

• The Remote Attribute related to the secure processing is used differently depending on the context:

  • If the file is to be sent to a remote destination, then all the remote secure attributes are used to encode the data sent.

  • If the file is received from a remote origin, then only the remote encryption file format (e.g., PGP, CMS, etc.) is used to decode the receiving file.

• The Remote Attribute related to the compression processing is used differently depending on the usage context:

  • If the file is to be sent to a remote destination, then all the remote compression attributes are used to compress the data to be sent.

  • If the file is received from a remote origin, then only the remote compression type is used to decompress the receiving file.

  • If the remote and local attributes of the File Resource are the same, then no data conversion is applied.

Examples

No conversion

File Resource is used to receive a text file with an end-of-line without any conversion.

Local Attributes	Value	Remote Attributes	Value
Content Type	text/plain	Content type	text/plain
Character Set	ISO-8859-1	Character Set	ISO-8859-1
Record Format	UV	Record Format	UV
Eol	LF	Eol	LF

EoL conversion

File Resource is used to receive a text file with End of Line conversion.

Local Attributes	Value	Remote Attributes	Value
Content Type	text/plain	Content type	text/plain
Character Set	ISO-8859-1	Character Set	ISO-8859-1
Record Format	UV	Record Format	UV
Eol	LF	Eol	CRLF

PGP sign during file receive

File Resource is used to sign via PGP a received text file.

Local Attributes	Value	Remote Attributes	Value
Content Type	text/plain	Content type	text/plain
Character Set	ISO-8859-1	Character Set	ISO-8859-1
Record Format	UV	Record format	UV
Eol	CRLF	Eol	CRLF
Execute Security Operation	SIGN	Execute Security Operation	NONE
Security Operation File Format	PGP
Security Operation Signature Key Label	keylabel
Security Operation Signature Hash Algorithm	MD5

Charset+EoL and PGP sign and encrypt during file reception

File Resource is used to perform a Charset and an End of Line conversion, and sign and encrypt via PGP a received text file.

Local Attributes	Value	Remote Attributes	Value
Content Type	text/plain	Content type	text/plain
Character Set	UTF-8	Character Set	x-windows-874
Record Format	UV	Record format	UV
Eol	LF	Eol	CRLF
Execute Security Operation	SIGN	Execute Security Operation	NONE
Security Operation File Format	PGP
Security Operation Signature Key Label	keylabel
Security Operation Signature Hash Algorithm	MD5

PGP decrypt during file reception

File Resource is used to decrypt via PGP a received text file.

Local Attributes	Value	Remote Attributes	Value
Content Type	text/plain	Content type	text/plain
Character Set	ISO-8859-1	Character Set	ISO-8859-1
Record Format	UV	Record format	UV
Eol	CRLF	Eol	CRLF
Execute Security Operation	NONE	Execute Security Operation	OPEN
		Execute Security Operation	PGP