DataOneVFSWriter
Short Description
The main purpose of DataOneVFSWriter is to store the content streamed by the upstream Data Shaper graph in an output file residing on Data One VFS. The content is stored together with a set of file metadata, namely DataOneFileDescriptor (DOFD), which are documented at this link.
The resulting graph performs the required processing on the file by combining DataOneVFSReader and DataOneVFSWriter with any of the other components available in the Data Shaper Designer palette.
The graph itself is invoked from within a mediation contract by using the specialized service task named Data Shaper Processor that submits the graph and monitors its outcome.
Data Shaper developers can expose several parameters at graph level, namely graph parameters. Such parameters (namely Data One Contract Parameters, DOP_*) will be configured by the Data One user at mediation contract configuration time and dynamically injected at graph execution time together with another set of implicitly injected context attributes (namely Data One Context Attributes, DOX_*). Details about DOP_* et al. exposed graph parameters can be found here. Details about DOX_* attributes can be found here.
Data Shaper graphs based on DataOneVFSReader and DataOneVFSWriter are not constrained to process a single input file and produce a single output file, both fan-out and fan-in data processing graphs can be easily implemented using the approach described above.
In the next few sections, a detailed discussion of each of the components highlighted above will be provided.
| COMPONENT | DATA OUTPUT | INPUT PORTS | OUTPUT PORTS | EACH TO ALL OUTPUTS | DIFFERENT TO DIFFERENT OUTPUTS | TRANSFORMATION | TRANSF. REQUIRED | JAVA | CTL | AUTO-PROPAGATED METADATA |
|---|---|---|---|---|---|---|---|---|---|---|
| DataOneVFSWriter | Data One | 0 | 1-2 | x | ✓ | x | x | x | x | ✓ |
Ports
| PORT TYPE | NUMBER | NAME | REQUIRED | DESCRIPTION | METADATA |
|---|---|---|---|---|---|
| Output | 0 | content | ✓ | File data content, streamed in chunks to the upstream graph | none |
| 1 | metadata | Data One File Descriptor metadata propagated once per file right before opening the output Data One file streamed in chunks by the upstream graph | Data One File Descriptor. Read this documentation |
DataOneVFSWriter Attributes
| ATTRIBUTE | REQ | DESCRIPTION | POSSIBLE VALUES |
|---|---|---|---|
| Basic | |||
| VFS Path | ✓ | Name of Data One VFS Path in the form: vfsname:/vpath (e.g. "ordersvfs:/receivables/20221H"). By default this attribute will be implicitly set to the value of ${DOP_VPATH_OUTPUT} graph parameter, if defined. When the special value _INHERITED_ is specified, the value is taken from the vfsFullPath field of DOFD on the metadata input port, when wired. | |
| File Name | ✓ | Name of the output file on the above specified VFS Path. By default this attribute will be implicitly set to the value of ${DOP_FILENAME_OUTPUT} graph parameter, if defined. When the special value _INHERITED_ is specified, the value is taken from the fileName field of DOFD on the metadata input port, when wired. | |
| Description | File description. By default this attribute will be implicitly set to the value of ${DOP_DESCRIPTION_OUTPUT} graph parameter, if defined. When the special value _INHERITED_ is specified, the value is taken from the description field of DOFD on the metadata input port, when wired. | ||
| Topic | String attribute attached to a file, it can be freely used to store any information that can drive specific processing for the file. By default this attribute will be implicitly set to the value of ${DOP_TOPIC_OUTPUT} graph parameter, if defined. When the special value _INHERITED_ is specified, the value is taken from the topic field of DOFD on the metadata input port, when wired. | ||
| Advanced | |||
| File Resource name | Name of a Data One File Resource to be applied to file content after having read it from the input port. Data One File Resource are definition of basic data processing operations for file content, examples include: line/record terminator character (EOR) conversion, charset conversion, compression/decompression, encryption/decryption. By default this attribute will be implicitly set to the value of ${DOP_FILERES_OUTPUT} graph parameter, if defined. | ||
| Expected charset | Specific charset that the file content read from the input port is expected to have. This value is used to set the charset for the file in Data One repository unless modified by the application of a File Resource. On the other hand, DataOneVFSWriter simply takes the provided value as-is, without attempting a validation of it versus the actual data being written. If the file is binary the charset should be set to NONE value and no charset conversion will be attempted. By default this attribute will be implicitly set to the value of ${DOP_CHARSET} graph parameter, if defined and not empty, or to UTF-8 otherwise. When the special value _INHERITED_ is specified, the value is taken from the charset field of DOFD on the metadata input port, when wired. | UTF-8 | <other encodings> |
Updated about 1 year ago