# Unique Rows

## <img src="/files/wIdt5BH73SzbVAqXQQp1" alt="" data-size="line"> Unique Rows

### Description <a href="#description" id="description"></a>

The Unique Rows transform removes duplicate rows from the (sorted) input stream(s).

For unsorted input streams, check the [Unique Rows By Hashset](/data-shaper-1.21/knowing-the-data-shaper-designer/pipelines/transforms/uniquerowsbyhashset.md) transform.

| Hop Engine | <sup>✓</sup> |
| ---------- | ------------ |
| Spark      | X            |
| Flink      | X            |
| Dataflow   | X            |

### Options

| Option                  | Description                                                                                                                                                                                  |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Transform Name          | Name of the transform this name has to be unique in a single pipeline.                                                                                                                       |
| Add counter to output?  | Check this option to add a counter field to the stream.                                                                                                                                      |
| Counter field           | Define the counter field name.                                                                                                                                                               |
| Redirect duplicate row  | Processes duplicate rows as an error and redirect rows to the error stream of the transform. Requires you to set error handling for this transform.                                          |
| Error Description       | Sets the error handling description to display when duplicate rows are detected. Only available when Redirect duplicate row is checked.                                                      |
| Fields to compare table | Specify the field names on which you want to force uniqueness or click Get to insert all fields from the input stream(s) You can choose to ignore case by setting the Ignore case flag to Y. |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.primeur.com/data-shaper-1.21/knowing-the-data-shaper-designer/pipelines/transforms/uniquerows.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.