# Apache Hop Virtual File System

### Description <a href="#description" id="description"></a>

[Apache Virtual File System (VFS)](https://commons.apache.org/proper/commons-vfs/) is part of the Apache Commons project. Through VFS, Hop users can access various files from different sources such as files on the local disk, on a HTTP(S) server, inside a ZIP archive and so through a url format.

Apache Hop makes fervent usage of VFS. Beyond the standard VFS file system types, we have added a number which are present in the various technology stacks supported by Hop. Like the standard file systems each has its own unique name scheme which you can use.

### Apache Hop VFS File Systems

The table below provides a quick overview of the VFS file systems supported by Apache Hop. Click the File system name to access more detailed file system documentation.

| File System                                                                                                                                                | Description                                     | URI Format       |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------- | ---------------- |
| [AWS S3](https://docs.primeur.com/data-shaper-1.20/knowing-the-data-shaper-designer/apache-hop-virtual-file-system/aws-s3-vfs)                             | Provides access to Amazon S3 Buckets            | `s3://`          |
| [Azure Blob Storage](https://hop.apache.org/manual/2.14.0/vfs/azure-blob-storage-vfs.html)                                                                 | Provides access to Azure Blob Storage           | `azure://`       |
| [Dropbox](https://docs.primeur.com/data-shaper-1.20/knowing-the-data-shaper-designer/apache-hop-virtual-file-system/dropbox-vfs)                           | Provides access to Dropbox                      | `dropbox://`     |
| [Google Cloud Storage](https://docs.primeur.com/data-shaper-1.20/knowing-the-data-shaper-designer/apache-hop-virtual-file-system/google-cloud-storage-vfs) | Provides access to Google Cloud Storage buckets | `gs://`          |
| [Google Drive](https://docs.primeur.com/data-shaper-1.20/knowing-the-data-shaper-designer/apache-hop-virtual-file-system/google-drive-vfs)                 | Provides access to Google Drive folders         | `googledrive://` |

### Apache VFS File System Types

The table below lists the file system types provided by the default Apache VFS implementation.

Check the Apache VFS [file system types](https://commons.apache.org/proper/commons-vfs/filesystems.html) for more information on the supported functionality per files system.

| File System      | Description                                                                                                                                                                                                                            | URI Format                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| BZIP2            | Provides read-only access to the contents of gzip and bzip2 files.                                                                                                                                                                     | <p>URI Format</p><p><code>gz:// compressed-file-uri</code></p><p><code>bz2:// compressed-file-uri</code></p><p>Where compressed-file-uri refers to a file of any supported type. There is no need to add a <code>!</code> part to the URI if you read the content of the file you always will get the uncompressed version.</p><p>Examples</p><ul><li><code>gz:/my/gz/file.gz</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| CIFS\*           |                                                                                                                                                                                                                                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| File             | Provides access to the files on the local physical file system.                                                                                                                                                                        | <p>URI Format</p><p><code>\[file://] absolute-path</code></p><p>Where <code>absolute-path</code> is a valid absolute file name for the local platform. UNC names are supported under Windows.</p><p>Examples</p><ul><li><code>file:///home/someuser/somedir</code></li><li><code>file:///C:/Documents and Settings</code></li><li><code>file://///somehost/someshare/afile.txt</code></li><li><code>/home/someuser/somedir</code></li><li><code>c:\program files\some dir</code></li><li><code>c:/program files/some dir</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| FTP              | Provides access to the files on an FTP server.                                                                                                                                                                                         | <p>URI Format</p><p><code>tp\://\[ username\[: password]@] hostname\[: port]\[ relative-path]</code></p><p>Examples</p><p><code>ftp\://myusername:mypassword\@somehost/pub/downloads/somefile.tgz</code></p><p>By default, the path is relative to the user’s home directory. This can be changed with:</p><p><code>FtpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(options, false);</code></p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| FTPS             | Provides access to the files on an FTP server over SSL.                                                                                                                                                                                | <p>URI Format</p><p><code>ftps\://\[ username\[: password]@] hostname\[: port]\[ absolute-path]</code></p><p>Examples</p><p><code>ftps\://myusername:mypassword\@somehost/pub/downloads/somefile.tgz</code></p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| GZIP             | see 'bzip2'                                                                                                                                                                                                                            |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| HDFS             | Provides access to files in an Apache Hadoop File System (HDFS). On Windows the integration test is disabled by default, as it requires binaries.                                                                                      | <p>URI Format</p><p><code>hdfs\:// hostname\[: port]\[ absolute-path]</code></p><p>Examples</p><ul><li><code>hdfs\://somehost:8080/downloads/some\_dir</code></li><li><code>hdfs\://somehost:8080/downloads/some\_file.ext</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| HTTP(S)          | Provides access to files on an HTTP server.                                                                                                                                                                                            | <p>URI Format</p><p><code>http\://\[ username\[: password]@] hostname\[: port]\[ absolute-path]</code></p><p><code>https\://\[ username\[: password]@] hostname\[: port]\[ absolute-path]</code></p><p>File System Options</p><ul><li><strong>proxyHost</strong> The proxy host to connect through.</li><li><strong>proxyPort</strong> The proxy port to use.</li><li><strong>proxyScheme</strong> The proxy scheme (http/https) to use.</li><li><strong>cookies</strong> An array of Cookies to add to the request.</li><li><strong>maxConnectionsPerHost</strong> The maximum number of connections allowed to a specific host and port. The default is 5.</li><li><strong>maxTotalConnections</strong> The maximum number of connections allowed to all hosts. The default is 50.</li><li><strong>keystoreFile</strong> The keystore file for SSL connections.</li><li><strong>keystorePass</strong> The keystore password.</li><li><strong>keystoreType</strong> The keystore type.</li></ul><p>Examples</p><ul><li><code><http://somehost:8080/downloads/somefile.jar></code></li><li><code><http://myusername@somehost/index.html></code></li></ul> |
| Jar, Zip and Tar | Provides read-only access to the contents of Zip, Jar and Tar files.                                                                                                                                                                   | <p>URI Format</p><p><code>zip\:// arch-file-uri\[! absolute-path]</code></p><p><code>jar:// arch-file-uri\[! absolute-path]</code></p><p><code>tar:// arch-file-uri\[! absolute-path]</code></p><p><code>tgz:// arch-file-uri\[! absolute-path]</code></p><p><code>tbz2:// arch-file-uri\[! absolute-path]</code></p><p>Where <code>arch-file-uri</code> refers to a file of any supported type, including other zip files. Note: if you would like to use the ! as normal character it must be escaped using <code>%21</code>. <code>tgz</code> and <code>tbz2</code> are convenience for <code>tar:gz</code> and <code>tar:bz2</code>.</p><p>Examples</p><ul><li><code>jar:../lib/classes.jar!/META-INF/manifest.mf</code></li><li><code>zip:<http://somehost/downloads/somefile.zip></code></li><li><code>jar:zip:outer.zip!/nested.jar!/somedir</code></li><li><code>jar:zip:outer.zip!/nested.jar!/some%21dir</code></li><li><code>tar:gz:<http://anyhost/dir/mytar.tar.gz!/mytar.tar!/path/in/tar/README.txt></code></li><li><code>tgz:file://anyhost/dir/mytar.tgz!/somepath/somefile</code></li></ul>                                             |
| mime\*           | This (sandbox) filesystem can read mails and its attachements like archives. If a part in the parsed mail has no name, a dummy name will be generated. The dummy name is: \_body\_part\_X where X will be replaced by the part number. | <p>URI Format</p><p><code>mime:// mime-file-uri\[! absolute-path]</code></p><p>Examples</p><ul><li><code>mime:file:///your/path/mail/anymail.mime!/</code></li><li><code>mime:file:///your/path/mail/anymail.mime!/filename.pdf</code></li><li><code>mime:file:///your/path/mail/anymail.mime!/\_body\_part\_0</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| RAM              | A filesystem which stores all the data in memory (one byte array for each file content).                                                                                                                                               | <p>URI Format</p><p><code>ram://\[ path]</code></p><p>File System Options</p><ul><li><strong>maxsize</strong> Maximum filesystem size (total bytes of all file contents).</li></ul><p>Examples</p><ul><li><code>ram:///any/path/to/file.txt</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| RES              | This is not really a filesystem, it just tries to lookup a resource using javas ClassLoader.getResource() and creates a VFS url for further processing.                                                                                | <p>URI Format</p><p><code>res\://\[ path]</code></p><p>Examples</p><ul><li><code>+res\://path/in/classpath/image.png</code> might result in <code>jar:file://my/path/to/images.jar!/path/in/classpath/image.png+</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| SFTP             | Provides access to the files on an SFTP server (that is, an SSH or SCP server).                                                                                                                                                        | <p>URI Format</p><p><code>sftp\://\[ username\[: password]@] hostname\[: port]\[ relative-path]</code></p><p>Examples</p><ul><li><code>sftp\://myusername:mypassword\@somehost/pub/downloads/somefile.tgz</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| Tar              | see 'jar'                                                                                                                                                                                                                              |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| Temp             | Provides access to a temporary file system, or scratchpad, that is deleted when Commons VFS shuts down. The temporary file system is backed by a local file system.                                                                    | <p>URI Format</p><p><code>tmp\://\[ absolute-path]</code></p><p>Examples</p><ul><li><code>tmp\://dir/somefile.txt</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| WebDAV           | Provides access to files on a WebDAV server through the modules `commons-vfs2-jackrabbit1` and `commons-vfs2-jackrabbit2`.                                                                                                             | <p>URI Format</p><p><code>webdav://\[ username\[: password]@] hostname\[: port]\[ absolute-path]</code></p><p>File System Options</p><ul><li><strong>versioning</strong> true if versioning should be enabled</li><li><strong>creatorName</strong> the user name to be identified with changes to a file. If not set the user name used to authenticate will be used.</li></ul><p>Examples</p><ul><li><code>webdav://somehost:8080/dist</code></li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| Zip              | see 'jar'                                                                                                                                                                                                                              |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |

\*) VFS file system type in development


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.primeur.com/data-shaper-1.20/knowing-the-data-shaper-designer/apache-hop-virtual-file-system.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.