How to use File Resources
To understand the File Resource within Data Mover, it is essential to recognize its function in managing files by setting metadata attributes for both local and remote files. This involves specifying details like content type (binary or text), data format, and security features such as encryption. Local files are generated within the system through input operations in the Virtual File System, while remote files can either be incoming or outgoing, managed via output operations. These functionalities allow seamless and secure data management and conversion in-stream during file transfers.
The File Resource is a powerful feature of Data Mover for setting file metadata into the Virtual File System and to perform in-stream data conversion while reading or writing file from/to the Virtual File System.
As previously described, the File Resource attributes (local and remote) contain:
Information declaring the content of a plain file in terms of binary or textual data, charset, record format, end of line character(s), etc.
Information declaring if the file is compressed and/or secured (e.g., encrypted and/or signed).
The above attributes apply to the local file and the remote file.
The local file is always related to the file that will be created in the Virtual File System. We refer to the creation of a file entry in the Virtual File System as an Input Virtual File System operation. During an input Virtual File System operation, if the File Resource is used, the local file attribute will be used to set the Virtual File System file entry metadata.
Depending on the context of the File Resource usage, the remote file is the receiving file to be inserted in the Virtual File System or the file to be sent to its destination. We refer to the read operation of the file entry from the Virtual File System as an Output Virtual File System operation.
To clarify this behavior, we can consider two usage contexts: sending a file to a remote destination, and receiving a file from a remote origin.
Using File Resources when sending a file
When a file is sent, it is read from the Virtual File System (Output operation) and File Resource local attributes related to the file contents are not used since the only valid local file attributes are those associated with the Virtual File System file entry.
File Resource remote file attributes are used to convert the file before sending it to its destination. For example, if in the Virtual File System the file is defined with an "819" charset name, and in the remote attribute of the File Resource the charset name is "1047", a charset conversion from "819" to "1047" will be executed in-stream before data is sent.
If the remote file defines security and/or compression attributes, data is secured and/or compressed as well.
Using File Resource when receiving a file
When a file is received, its contents are described by the remote attributes of the File Resource but the final format of the file that will be written and sent to the Virtual File System (Input operation) is described by the local attributes of the File Resource. For example, if in the remote attributes of the File Resource the file is defined with an "819" charset name and in the local attribute of the File Resource the charset name is "1047", a charset conversion from "819" to "1047" will be executed in-stream before data is written.
If the local file defines security and/or compression attributes, data will be secured and/or compressed as well.
The local file attributes of the File Resource are then copied into the Virtual File System entry metadata.
When a File Resource is created, its usage context is not defined at first. For this reason, it is mandatory to define both local and remote attributes.
A File Resource is typically used when a file is received from a remote source or sent to a remote destination, enabling the automatic conversion of its contents.
Once the File Resource is defined, it is necessary to create other configuration items that refer to the created file resource in order to use it. The file resource is referenced in the contract when setting the file processing rules.
Notes
When the File Resource is used during an operation where a Virtual File System file entry is read (for example when a file must be sent to a remote destination) then the local attributes of the File Resource are ignored because the file attributes used are the metadata associated to the Virtual File System entry.
Some transport protocols (e.g., PR4 and PR5), when receiving a file, ignore the remote attributes of the File Resource because the content and format type of the receiving file is already known (via metadata sent by the client to the server).
The Remote Attribute related to the secure processing is used differently depending on the context:
If the file must be sent to a remote destination, all the remote secure attributes are used to encode the data sent.
If the file is received from a remote origin, only the remote encryption file format (e.g., PGP, CMS, etc.) is used to decode the receiving file.
The Remote Attribute related to the compression processing is used differently depending on the usage context:
If the file is sent to a remote destination, all the remote compression attributes are used to compress the data to be sent.
If the file is received from a remote origin, only the remote compression type is used to decompress the receiving file.
If the remote and local attributes of the File Resource are the same, no data conversion is applied.
Examples
No conversion
File Resource is used to receive a text file with an end-of-line without any conversion.
Content Type
text/plain
Content type
text/plain
Character Set
ISO-8859-1
Character Set
ISO-8859-1
Record Format
UV
Record Format
UV
Eol
LF
Eol
LF
EoL conversion
File Resource is used to receive a text file with End of Line conversion.
Content Type
text/plain
Content type
text/plain
Character Set
ISO-8859-1
Character Set
ISO-8859-1
Record Format
UV
Record Format
UV
Eol
LF
Eol
CRLF
PGP sign during file receive
File Resource is used to sign via PGP a received text file.
Content Type
text/plain
Content type
text/plain
Character Set
ISO-8859-1
Character Set
ISO-8859-1
Record Format
UV
Record format
UV
Eol
CRLF
Eol
CRLF
Execute Security Operation
SIGN
Execute Security Operation
NONE
Security Operation File Format
PGP
Security Operation Signature Key Label
keylabel
Security Operation Signature Hash Algorithm
MD5
Charset+EoL and PGP sign and encrypt during file reception
File Resource is used to perform a Charset and an End of Line conversion, and sign and encrypt via PGP a received text file.
Content Type
text/plain
Content type
text/plain
Character Set
UTF-8
Character Set
x-windows-874
Record Format
UV
Record format
UV
Eol
LF
Eol
CRLF
Execute Security Operation
SIGN
Execute Security Operation
NONE
Security Operation File Format
PGP
Security Operation Signature Key Label
keylabel
Security Operation Signature Hash Algorithm
MD5
PGP decrypt during file reception
File Resource is used to decrypt via PGP a received text file.
Content Type
text/plain
Content type
text/plain
Character Set
ISO-8859-1
Character Set
ISO-8859-1
Record Format
UV
Record format
UV
Eol
CRLF
Eol
CRLF
Execute Security Operation
NONE
Execute Security Operation
OPEN
Execute Security Operation
PGP
Last updated