camel-aws-s3-source-kafka-connector source configuration

Connector Description: Receive data from AWS S3 Bucket. Access Key/Secret Key are the basic method for authenticating to the AWS S3 Service. These parameters are optional, because the Kamelet provide also the 'useDefaultCredentialsProvider'. When using a default Credentials Provider the S3 client will load the credentials through this provider and won’t use the static credential. This is reason for not having the access key and secret key as mandatory parameter for this Kamelet.

When using camel-aws-s3-source-kafka-connector as source make sure to use the following Maven dependency to have support for the connector:

<dependency>
  <groupId>org.apache.camel.kafkaconnector</groupId>
  <artifactId>camel-aws-s3-source-kafka-connector</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel Kafka connector version -->
</dependency>

To use this source connector in Kafka connect you’ll need to set the following connector.class

connector.class=org.apache.camel.kafkaconnector.awss3source.CamelAwss3sourceSourceConnector

The camel-aws-s3-source source connector supports 12 options, which are listed below.

Name Description Default Priority

camel.kamelet.aws-s3-source.bucketNameOrArn

Required The S3 Bucket name or ARN.

HIGH

camel.kamelet.aws-s3-source.deleteAfterRead

Delete objects after consuming them.

true

MEDIUM

camel.kamelet.aws-s3-source.accessKey

The access key obtained from AWS.

MEDIUM

camel.kamelet.aws-s3-source.secretKey

The secret key obtained from AWS.

MEDIUM

camel.kamelet.aws-s3-source.region

Required The AWS region to connect to Example: eu-west-1.

HIGH

camel.kamelet.aws-s3-source.autoCreateBucket

Setting the autocreation of the S3 bucket bucketName.

false

MEDIUM

camel.kamelet.aws-s3-source.includeBody

If it is true, the exchange will be consumed and put into the body and closed. If false the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata.

true

MEDIUM

camel.kamelet.aws-s3-source.prefix

The AWS S3 bucket prefix to consider while searching Example: folder/.

MEDIUM

camel.kamelet.aws-s3-source.ignoreBody

If it is true, the S3 Object Body will be ignored completely, if it is set to false the S3 Object will be put in the body. Setting this to true, will override any behavior defined by includeBody option.

false

MEDIUM

camel.kamelet.aws-s3-source.useDefaultCredentialsProvider

Set whether the S3 client should expect to load credentials through a default credentials provider or to expect static credentials to be passed in.

false

MEDIUM

camel.kamelet.aws-s3-source.uriEndpointOverride

Set the overriding endpoint URI. This option needs to be used in combination with overrideEndpoint option.

MEDIUM

camel.kamelet.aws-s3-source.overrideEndpoint

Set the need for overiding the endpoint URI. This option needs to be used in combination with uriEndpointOverride setting.

false

MEDIUM

The camel-aws-s3-source source connector has no converters out of the box.

The camel-aws-s3-source source connector has no transforms out of the box.

The camel-aws-s3-source source connector has no aggregation strategies out of the box.

Back to top