# Redshift Bulk Loader

## ![](/files/GGUT1a6qRhsDYnxDODs3) Redshift Bulk Loader

### Description <a href="#description" id="description"></a>

The Redshift Bulk Loader transform loads data from Apache Hop to AWS Redshift using the [`COPY`](https://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html) command.

{% hint style="info" %}
Tip: make sure your target Redshift table has a layout that is compatible with Parquet data types, e.g. use `int8` instead of `int4` data types.
{% endhint %}

| Hop Engine | <sup>✓</sup> |
| ---------- | ------------ |
| Spark      | ?            |
| Flink      | ?            |
| Dataflow   | ?            |

{% hint style="warning" %}
Important: The Redshift Bulk Loader is linked to the database type. It will fetch the JDBC driver from the hop/lib/jdbc folder.<br>
{% endhint %}

### General Options

| Option                   | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Transform name           | Name of the transform.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Connection               | Name of the database connection on which the target table resides.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Target schema            | The name of the target schema to write data to.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Target table             | The name of the target table to write data to.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| AWS Authentication       | <p>choose which authentication method to use with the <code>COPY</code> command. Supported options are <code>AWS Credentials</code> and <code>IAM Role</code>.<br></p><ul><li>check the <a href="https://docs.aws.amazon.com/redshift/latest/dg/copy-usage_notes-access-permissions.html#copy-usage_notes-access-key-based">Key-based access control</a> for more information on the <code>Credentials</code> option.</li><li>check the <a href="https://docs.aws.amazon.com/redshift/latest/dg/copy-usage_notes-access-permissions.html#copy-usage_notes-access-role-based">IAM Role</a> docs for more information on the <code>IAM Role</code> option.</li></ul> |
| Use AWS system variables | (`Credentials` only!) pick up the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` values from your operating system’s environment variables.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| AWS\_ACCESS\_KEY\_ID     | (if `Credentials` is selected and `Use AWS system variables` is unchecked) specify a value or variable for your `AWS_ACCESS_KEY_ID`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| AWS\_SECRET\_ACCESS\_KEY | (if `Credentials` is selected and `Use AWS system variables` is unchecked) specify a value or variable for your `AWS_SECRET_ACCESS_KEY`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| IAM Role                 | (if `IAM Role` is selected) specify the IAM Role to use, in the syntax `arn:aws:iam::<aws-account-id>:role/<role-name>`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Truncate table           | Truncate the target table before loading data.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| Truncate on first row    | Truncate the target table before loading data, but only when a first data row is received (will not truncate when a pipeline runs an empty stream (0 rows)).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| Specify database fields  | Specify the database and stream fields mapping                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |

### Main Options

| Option                                | Description                                                                                                                                                                  |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Stream to S3 CSV                      | write the current pipeline stream to a CSV file in an S3 bucket before performing the `COPY` load.                                                                           |
| Load from existing file               | do not stream the contents of the current pipeline, but perform the `COPY` load from a pre-existing file in S3. Supported formats are `CSV` (comma delimited) and `Parquet`. |
| Copy into Redshift from existing file | path to the file in S3 to `COPY` load the data from.                                                                                                                         |

### Database fields

Map the current stream fields to the Redshift table’s columns.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.primeur.com/data-shaper-1.21/knowing-the-data-shaper-designer/pipelines/transforms/redshift-bulkloader.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.