AWS SQS Reader

AWS SQS Reader

Description

The AWS SQS Reader transform enables you to receive messages from Amazon Web Services Simple Queue Service within an Apache Hop pipeline.

Hop Engine	✓
Spark	?
Flink	?
Dataflow	?

Preconditions

Before the first execution you need to create an IAM-Role (e.g. for usage on EC2/ECS) or an IAM-User with an AWS Key and Secret and attach the required policies for pushing notifications via SNS.

You also need to create one ore more subscription topics you want to push the messages to.

Options

AWS Settings tab

option	description
Use AWS Credentials chain	Apache Hop tries to pick up the AWS credentials from the host environment. For more information, check the Credentials docs.
AWS Access Key	your AWS Access Key (AWS_ACCESS_KEY_ID)
AWS Secret Access Key	the secret for your AWS Access Key (AWS_SECRET_ACCESS_KEY)
AWS Region	the AWS-Region the service is running in.
SQS Queueu URL	the URL of the SQS Queue (starting with https:// - not the ARN!)

Output definition

On the Output Definition Tab you can define the output fields for the informations read from the SQS message as well as some initial settings for receiving the messages.

Output Settings

Option	Description
Delete Message(s)	delete the message(s) be from the queue after receiving?
Maximum messages to retrieve	finish after reaching this maximum number of messages. Zero (0) will receive all messages in the queue.

Output Settings

Option	Description
MessageID	For each message an unique ID is retrieved from SQS. This can be written to an output field defined here.
MessageBody	Complete content of the message.
ReceiptHandle	Unique identifier for receiving a message.
MD5 Hash	MD5 hash of the message body
Message Node from SNS	if the message was sent from SNS this field will contain the content of the JSON node "Message" within the Message body.