AWS S3 Storage Service

Store and retrieve objects from AWS S3 Storage Service.

What’s inside

AWS S3 Storage Service component, URI syntax: aws2-s3://bucketNameOrArn

Please refer to the above links for usage and configuration details.

Maven coordinates

<dependency>
    <groupId>org.apache.camel.springboot</groupId>
    <artifactId>camel-aws2-s3-starter</artifactId>
</dependency>

Spring Boot Auto-Configuration

The starter supports 65 options, which are listed below.

Name	Description	Default	Type
camel.component.aws2-s3.access-key	Amazon AWS Access Key		String
camel.component.aws2-s3.amazon-s3-client	Reference to a com.amazonaws.services.s3.AmazonS3 in the registry. The option is a software.amazon.awssdk.services.s3.S3Client type.		S3Client
camel.component.aws2-s3.amazon-s3-presigner	An S3 Presigner for Request, used mainly in createDownloadLink operation. The option is a software.amazon.awssdk.services.s3.presigner.S3Presigner type.		S3Presigner
camel.component.aws2-s3.auto-create-bucket	Setting the autocreation of the S3 bucket bucketName. This will apply also in case of moveAfterRead option enabled, and it will create the destinationBucket if it doesn’t exist already.	false	Boolean
camel.component.aws2-s3.autoclose-body	If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically.	true	Boolean
camel.component.aws2-s3.autowired-enabled	Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.	true	Boolean
camel.component.aws2-s3.aws-k-m-s-key-id	Define the id of KMS key to use in case KMS is enabled		String
camel.component.aws2-s3.batch-message-number	The number of messages composing a batch in streaming upload mode	10	Integer
camel.component.aws2-s3.batch-size	The batch size (in bytes) in streaming upload mode	1000000	Integer
camel.component.aws2-s3.bridge-error-handler	Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions (if possible) occurred while the Camel consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. Important: This is only possible if the 3rd party component allows Camel to be alerted if an exception was thrown. Some components handle this internally only, and therefore bridgeErrorHandler is not possible. In other situations we may improve the Camel component to hook into the 3rd party component and make this possible for future releases. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.	false	Boolean
camel.component.aws2-s3.buffer-size	The buffer size (in bytes) in streaming upload mode	1000000	Integer
camel.component.aws2-s3.conditional-writes-enabled	Uploads the object only if the object key name does not already exist in the bucket specified.	false	Boolean
camel.component.aws2-s3.configuration	The component configuration. The option is a org.apache.camel.component.aws2.s3.AWS2S3Configuration type.		AWS2S3Configuration
camel.component.aws2-s3.customer-algorithm	Define the customer algorithm to use in case CustomerKey is enabled		String
camel.component.aws2-s3.customer-key-id	Define the id of the Customer key to use in case CustomerKey is enabled		String
camel.component.aws2-s3.customer-key-m-d5	Define the MD5 of Customer key to use in case CustomerKey is enabled		String
camel.component.aws2-s3.delete-after-read	Delete objects from S3 after they have been retrieved. The deleting is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieved over and over again in the polls. Therefore, you need to use the Idempotent Consumer EIP in the route to filter out duplicates. You can filter using the AWS2S3Constants#BUCKET_NAME and AWS2S3Constants#KEY headers, or only the AWS2S3Constants#KEY header.	true	Boolean
camel.component.aws2-s3.delete-after-write	Delete file object after the S3 file has been uploaded	false	Boolean
camel.component.aws2-s3.delimiter	The delimiter which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in.		String
camel.component.aws2-s3.destination-bucket	Define the destination bucket where an object must be moved when moveAfterRead is set to true.		String
camel.component.aws2-s3.destination-bucket-prefix	Define the destination bucket prefix to use when an object must be moved, and moveAfterRead is set to true.		String
camel.component.aws2-s3.destination-bucket-suffix	Define the destination bucket suffix to use when an object must be moved, and moveAfterRead is set to true.		String
camel.component.aws2-s3.done-file-name	If provided, Camel will only consume files if a done file exists.		String
camel.component.aws2-s3.enabled	Whether to enable auto configuration of the aws2-s3 component. This is enabled by default.		Boolean
camel.component.aws2-s3.file-name	To get the object from the bucket with the given file name		String
camel.component.aws2-s3.force-path-style	Set whether the S3 client should use path-style URL instead of virtual-hosted-style	false	Boolean
camel.component.aws2-s3.health-check-consumer-enabled	Used for enabling or disabling all consumer based health checks from this component	true	Boolean
camel.component.aws2-s3.health-check-producer-enabled	Used for enabling or disabling all producer based health checks from this component. Notice: Camel has by default disabled all producer based health-checks. You can turn on producer checks globally by setting camel.health.producersEnabled=true.	true	Boolean
camel.component.aws2-s3.ignore-body	If it is true, the S3 Object Body will be ignored completely if it is set to false, the S3 Object will be put in the body. Setting this to true will override any behavior defined by includeBody option.	false	Boolean
camel.component.aws2-s3.include-body	If it is true, the S3Object exchange will be consumed and put into the body and closed. If false, the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to the autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed, while in case of includeBody false then it will be up to the caller to close the S3Object stream. However, setting autocloseBody to true when includeBody is false it will schedule to close the S3Object stream automatically on exchange completion.	true	Boolean
camel.component.aws2-s3.include-folders	If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those	true	Boolean
camel.component.aws2-s3.key-name	Setting the key name for an element in the bucket through endpoint parameter		String
camel.component.aws2-s3.lazy-start-producer	Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.	false	Boolean
camel.component.aws2-s3.move-after-read	Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation, the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved.	false	Boolean
camel.component.aws2-s3.multi-part-upload	If it is true, camel will upload the file with multipart format. The part size is decided by the partSize option. Camel will only do multipart uploads for files that are larger than the part-size thresholds. Files that are smaller will be uploaded in a single operation.	false	Boolean
camel.component.aws2-s3.naming-strategy	The naming strategy to use in streaming upload mode	progressive	AWSS3NamingStrategyEnum
camel.component.aws2-s3.operation	The operation to do in case the user don’t want to do only an upload		AWS2S3Operations
camel.component.aws2-s3.override-endpoint	Set the need for overriding the endpoint. This option needs to be used in combination with the uriEndpointOverride option	false	Boolean
camel.component.aws2-s3.part-size	Set up the partSize which is used in multipart upload, the default size is 25 MB. The minimum size in AWS is 5 MB. Camel will only do multipart uploads for files that are larger than the part-size thresholds. Files that are smaller will be uploaded in a single operation.	26214400	Long
camel.component.aws2-s3.pojo-request	If we want to use a POJO request as body or not	false	Boolean
camel.component.aws2-s3.policy	The policy for this queue to set in the com.amazonaws.services.s3.AmazonS3#setBucketPolicy() method.		String
camel.component.aws2-s3.prefix	The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in.		String
camel.component.aws2-s3.profile-credentials-name	If using a profile credentials provider, this parameter will set the profile name		String
camel.component.aws2-s3.proxy-host	To define a proxy host when instantiating the SQS client		String
camel.component.aws2-s3.proxy-port	Specify a proxy port to be used inside the client definition.		Integer
camel.component.aws2-s3.proxy-protocol	To define a proxy protocol when instantiating the S3 client	https	Protocol
camel.component.aws2-s3.region	The region in which the S3 client needs to work. When using this parameter, the configuration will expect the lowercase name of the region (for example, ap-east-1) You’ll need to use the name Region.EU_WEST_1.id()		String
camel.component.aws2-s3.remove-prefix-on-move	Remove the contents of the prefix configuration string from the new S3Object key before copying. For example, if prefix is set to 'demo/notify' and the destinationBucketPrefix is set to 'demo/archive', an S3Object with a key of 'demo/notify/example.txt' will be copied to 'demo/archive/example.txt', rather than the default behavior where the new key is 'demo/archive/demo/notify/example.txt'. Only applicable when moveAfterRead is true.	false	Boolean
camel.component.aws2-s3.restarting-policy	The restarting policy to use in streaming upload mode	override	AWSS3RestartingPolicyEnum
camel.component.aws2-s3.secret-key	Amazon AWS Secret Key		String
camel.component.aws2-s3.session-token	Amazon AWS Session Token used when the user needs to assume an IAM role		String
camel.component.aws2-s3.storage-class	The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request.		String
camel.component.aws2-s3.streaming-upload-mode	When stream mode is true, the upload to bucket will be done in streaming	false	Boolean
camel.component.aws2-s3.streaming-upload-timeout	While streaming upload mode is true, this option set the timeout to complete upload		Long
camel.component.aws2-s3.timestamp-grouping-enabled	Enable timestamp-based grouping of messages into time windows for file creation. When enabled, messages are grouped by their timestamp header into time windows.	false	Boolean
camel.component.aws2-s3.timestamp-header-name	The name of the message header containing the timestamp for grouping. Default is CamelMessageTimestamp.	CamelMessageTimestamp	String
camel.component.aws2-s3.timestamp-window-size-millis	The size of the time window in milliseconds for timestamp-based grouping. Messages within the same time window will be written to the same file. Default is 300000 (5 minutes).	300000	Long
camel.component.aws2-s3.trust-all-certificates	If we want to trust all certificates in case of overriding the endpoint	false	Boolean
camel.component.aws2-s3.uri-endpoint-override	Set the overriding uri endpoint. This option needs to be used in combination with overrideEndpoint option		String
camel.component.aws2-s3.use-aws-k-m-s	Define if KMS must be used or not	false	Boolean
camel.component.aws2-s3.use-customer-key	Define if Customer Key must be used or not	false	Boolean
camel.component.aws2-s3.use-default-credentials-provider	Set whether the S3 client should expect to load credentials through a default credentials provider.	false	Boolean
camel.component.aws2-s3.use-profile-credentials-provider	Set whether the S3 client should expect to load credentials through a profile credentials provider.	false	Boolean
camel.component.aws2-s3.use-s-s-e-s3	Define if SSE S3 must be used or not	false	Boolean
camel.component.aws2-s3.use-session-credentials	Set whether the S3 client should expect to use Session Credentials. This is useful in a situation in which the user needs to assume an IAM role for doing operations in S3.	false	Boolean

Name

Description

Default

Type

camel.component.aws2-s3.access-key

Amazon AWS Access Key

String

camel.component.aws2-s3.amazon-s3-client

Reference to a com.amazonaws.services.s3.AmazonS3 in the registry. The option is a software.amazon.awssdk.services.s3.S3Client type.

S3Client

camel.component.aws2-s3.amazon-s3-presigner

An S3 Presigner for Request, used mainly in createDownloadLink operation. The option is a software.amazon.awssdk.services.s3.presigner.S3Presigner type.

S3Presigner

camel.component.aws2-s3.auto-create-bucket

Setting the autocreation of the S3 bucket bucketName. This will apply also in case of moveAfterRead option enabled, and it will create the destinationBucket if it doesn’t exist already.

false

Boolean

camel.component.aws2-s3.autoclose-body

If this option is true and includeBody is false, then the S3Object.close() method will be called on exchange completion. This option is strongly related to includeBody option. In case of setting includeBody to false and autocloseBody to false, it will be up to the caller to close the S3Object stream. Setting autocloseBody to true, will close the S3Object stream automatically.

true

Boolean

camel.component.aws2-s3.autowired-enabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

Boolean

camel.component.aws2-s3.aws-k-m-s-key-id

Define the id of KMS key to use in case KMS is enabled

String

camel.component.aws2-s3.batch-message-number

The number of messages composing a batch in streaming upload mode

Integer

camel.component.aws2-s3.batch-size

The batch size (in bytes) in streaming upload mode

1000000

Integer

camel.component.aws2-s3.bridge-error-handler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions (if possible) occurred while the Camel consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. Important: This is only possible if the 3rd party component allows Camel to be alerted if an exception was thrown. Some components handle this internally only, and therefore bridgeErrorHandler is not possible. In other situations we may improve the Camel component to hook into the 3rd party component and make this possible for future releases. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

Boolean

camel.component.aws2-s3.buffer-size

The buffer size (in bytes) in streaming upload mode

1000000

Integer

camel.component.aws2-s3.conditional-writes-enabled

Uploads the object only if the object key name does not already exist in the bucket specified.

false

Boolean

camel.component.aws2-s3.configuration

The component configuration. The option is a org.apache.camel.component.aws2.s3.AWS2S3Configuration type.

AWS2S3Configuration

camel.component.aws2-s3.customer-algorithm

Define the customer algorithm to use in case CustomerKey is enabled

String

camel.component.aws2-s3.customer-key-id

Define the id of the Customer key to use in case CustomerKey is enabled

String

camel.component.aws2-s3.customer-key-m-d5

Define the MD5 of Customer key to use in case CustomerKey is enabled

String

camel.component.aws2-s3.delete-after-read

Delete objects from S3 after they have been retrieved. The deleting is only performed if the Exchange is committed. If a rollback occurs, the object is not deleted. If this option is false, then the same objects will be retrieved over and over again in the polls. Therefore, you need to use the Idempotent Consumer EIP in the route to filter out duplicates. You can filter using the AWS2S3Constants#BUCKET_NAME and AWS2S3Constants#KEY headers, or only the AWS2S3Constants#KEY header.

true

Boolean

camel.component.aws2-s3.delete-after-write

Delete file object after the S3 file has been uploaded

false

Boolean

camel.component.aws2-s3.delimiter

The delimiter which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in.

String

camel.component.aws2-s3.destination-bucket

Define the destination bucket where an object must be moved when moveAfterRead is set to true.

String

camel.component.aws2-s3.destination-bucket-prefix

Define the destination bucket prefix to use when an object must be moved, and moveAfterRead is set to true.

String

camel.component.aws2-s3.destination-bucket-suffix

Define the destination bucket suffix to use when an object must be moved, and moveAfterRead is set to true.

String

camel.component.aws2-s3.done-file-name

If provided, Camel will only consume files if a done file exists.

String

camel.component.aws2-s3.enabled

Whether to enable auto configuration of the aws2-s3 component. This is enabled by default.

Boolean

camel.component.aws2-s3.file-name

To get the object from the bucket with the given file name

String

camel.component.aws2-s3.force-path-style

Set whether the S3 client should use path-style URL instead of virtual-hosted-style

false

Boolean

camel.component.aws2-s3.health-check-consumer-enabled

Used for enabling or disabling all consumer based health checks from this component

true

Boolean

camel.component.aws2-s3.health-check-producer-enabled

Used for enabling or disabling all producer based health checks from this component. Notice: Camel has by default disabled all producer based health-checks. You can turn on producer checks globally by setting camel.health.producersEnabled=true.

true

Boolean

camel.component.aws2-s3.ignore-body

If it is true, the S3 Object Body will be ignored completely if it is set to false, the S3 Object will be put in the body. Setting this to true will override any behavior defined by includeBody option.

false

Boolean

camel.component.aws2-s3.include-body

If it is true, the S3Object exchange will be consumed and put into the body and closed. If false, the S3Object stream will be put raw into the body and the headers will be set with the S3 object metadata. This option is strongly related to the autocloseBody option. In case of setting includeBody to true because the S3Object stream will be consumed then it will also be closed, while in case of includeBody false then it will be up to the caller to close the S3Object stream. However, setting autocloseBody to true when includeBody is false it will schedule to close the S3Object stream automatically on exchange completion.

true

Boolean

camel.component.aws2-s3.include-folders

If it is true, the folders/directories will be consumed. If it is false, they will be ignored, and Exchanges will not be created for those

true

Boolean

camel.component.aws2-s3.key-name

Setting the key name for an element in the bucket through endpoint parameter

String

camel.component.aws2-s3.lazy-start-producer

Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing.

false

Boolean

camel.component.aws2-s3.move-after-read

Move objects from S3 bucket to a different bucket after they have been retrieved. To accomplish the operation, the destinationBucket option must be set. The copy bucket operation is only performed if the Exchange is committed. If a rollback occurs, the object is not moved.

false

Boolean

camel.component.aws2-s3.multi-part-upload

If it is true, camel will upload the file with multipart format. The part size is decided by the partSize option. Camel will only do multipart uploads for files that are larger than the part-size thresholds. Files that are smaller will be uploaded in a single operation.

false

Boolean

camel.component.aws2-s3.naming-strategy

The naming strategy to use in streaming upload mode

progressive

AWSS3NamingStrategyEnum

camel.component.aws2-s3.operation

The operation to do in case the user don’t want to do only an upload

AWS2S3Operations

camel.component.aws2-s3.override-endpoint

Set the need for overriding the endpoint. This option needs to be used in combination with the uriEndpointOverride option

false

Boolean

camel.component.aws2-s3.part-size

Set up the partSize which is used in multipart upload, the default size is 25 MB. The minimum size in AWS is 5 MB. Camel will only do multipart uploads for files that are larger than the part-size thresholds. Files that are smaller will be uploaded in a single operation.

26214400

Long

camel.component.aws2-s3.pojo-request

If we want to use a POJO request as body or not

false

Boolean

camel.component.aws2-s3.policy

The policy for this queue to set in the com.amazonaws.services.s3.AmazonS3#setBucketPolicy() method.

String

camel.component.aws2-s3.prefix

The prefix which is used in the com.amazonaws.services.s3.model.ListObjectsRequest to only consume objects we are interested in.

String

camel.component.aws2-s3.profile-credentials-name

If using a profile credentials provider, this parameter will set the profile name

String

camel.component.aws2-s3.proxy-host

To define a proxy host when instantiating the SQS client

String

camel.component.aws2-s3.proxy-port

Specify a proxy port to be used inside the client definition.

Integer

camel.component.aws2-s3.proxy-protocol

To define a proxy protocol when instantiating the S3 client

https

Protocol

camel.component.aws2-s3.region

The region in which the S3 client needs to work. When using this parameter, the configuration will expect the lowercase name of the region (for example, ap-east-1) You’ll need to use the name Region.EU_WEST_1.id()

String

camel.component.aws2-s3.remove-prefix-on-move

Remove the contents of the prefix configuration string from the new S3Object key before copying. For example, if prefix is set to 'demo/notify' and the destinationBucketPrefix is set to 'demo/archive', an S3Object with a key of 'demo/notify/example.txt' will be copied to 'demo/archive/example.txt', rather than the default behavior where the new key is 'demo/archive/demo/notify/example.txt'. Only applicable when moveAfterRead is true.

false

Boolean

camel.component.aws2-s3.restarting-policy

The restarting policy to use in streaming upload mode

override

AWSS3RestartingPolicyEnum

camel.component.aws2-s3.secret-key

Amazon AWS Secret Key

String

camel.component.aws2-s3.session-token

Amazon AWS Session Token used when the user needs to assume an IAM role

String

camel.component.aws2-s3.storage-class

The storage class to set in the com.amazonaws.services.s3.model.PutObjectRequest request.

String

camel.component.aws2-s3.streaming-upload-mode

When stream mode is true, the upload to bucket will be done in streaming

false

Boolean

camel.component.aws2-s3.streaming-upload-timeout

While streaming upload mode is true, this option set the timeout to complete upload

Long

camel.component.aws2-s3.timestamp-grouping-enabled

Enable timestamp-based grouping of messages into time windows for file creation. When enabled, messages are grouped by their timestamp header into time windows.

false

Boolean

camel.component.aws2-s3.timestamp-header-name

The name of the message header containing the timestamp for grouping. Default is CamelMessageTimestamp.

CamelMessageTimestamp

String

camel.component.aws2-s3.timestamp-window-size-millis

The size of the time window in milliseconds for timestamp-based grouping. Messages within the same time window will be written to the same file. Default is 300000 (5 minutes).

300000

Long

camel.component.aws2-s3.trust-all-certificates

If we want to trust all certificates in case of overriding the endpoint

false

Boolean

camel.component.aws2-s3.uri-endpoint-override

Set the overriding uri endpoint. This option needs to be used in combination with overrideEndpoint option

String

camel.component.aws2-s3.use-aws-k-m-s

Define if KMS must be used or not

false

Boolean

camel.component.aws2-s3.use-customer-key

Define if Customer Key must be used or not

false

Boolean

camel.component.aws2-s3.use-default-credentials-provider

Set whether the S3 client should expect to load credentials through a default credentials provider.

false

Boolean

camel.component.aws2-s3.use-profile-credentials-provider

Set whether the S3 client should expect to load credentials through a profile credentials provider.

false

Boolean

camel.component.aws2-s3.use-s-s-e-s3

Define if SSE S3 must be used or not

false

Boolean

camel.component.aws2-s3.use-session-credentials

Set whether the S3 client should expect to use Session Credentials. This is useful in a situation in which the user needs to assume an IAM role for doing operations in S3.

false

Boolean