AWS Kinesis
Consume and produce records from and to AWS Kinesis Streams.
What’s inside
-
AWS Kinesis component, URI syntax:
aws2-kinesis:streamName -
AWS Kinesis Firehose component, URI syntax:
aws2-kinesis-firehose:streamName
Please refer to the above links for usage and configuration details.
Maven coordinates
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-aws2-kinesis-starter</artifactId>
</dependency>Spring Boot Auto-Configuration
The starter supports 58 options, which are listed below.
| Name | Description | Default | Type |
|---|---|---|---|
camel.component.aws2-kinesis-firehose.access-key | Amazon AWS Access Key | String | |
camel.component.aws2-kinesis-firehose.amazon-kinesis-firehose-client | Amazon Kinesis Firehose client to use for all requests for this endpoint. The option is a software.amazon.awssdk.services.firehose.FirehoseClient type. | FirehoseClient | |
camel.component.aws2-kinesis-firehose.autowired-enabled | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | Boolean |
camel.component.aws2-kinesis-firehose.cbor-enabled | This option will set the CBOR_ENABLED property during the execution | true | Boolean |
camel.component.aws2-kinesis-firehose.configuration | Component configuration. The option is a org.apache.camel.component.aws2.firehose.KinesisFirehose2Configuration type. | KinesisFirehose2Configuration | |
camel.component.aws2-kinesis-firehose.enabled | Whether to enable auto configuration of the aws2-kinesis-firehose component. This is enabled by default. | Boolean | |
camel.component.aws2-kinesis-firehose.lazy-start-producer | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean |
camel.component.aws2-kinesis-firehose.operation | The operation to do in case the user don’t want to send only a record | KinesisFirehose2Operations | |
camel.component.aws2-kinesis-firehose.override-endpoint | Set the need for overriding the endpoint. This option needs to be used in combination with uriEndpointOverride option | false | Boolean |
camel.component.aws2-kinesis-firehose.profile-credentials-name | If using a profile credentials provider this parameter will set the profile name. | String | |
camel.component.aws2-kinesis-firehose.proxy-host | To define a proxy host when instantiating the Kinesis Firehose client | String | |
camel.component.aws2-kinesis-firehose.proxy-port | To define a proxy port when instantiating the Kinesis Firehose client | Integer | |
camel.component.aws2-kinesis-firehose.proxy-protocol | To define a proxy protocol when instantiating the Kinesis Firehose client | https | Protocol |
camel.component.aws2-kinesis-firehose.region | The region in which Kinesis Firehose client needs to work. When using this parameter, the configuration will expect the lowercase name of the region (for example ap-east-1) You’ll need to use the name Region.EU_WEST_1.id() | String | |
camel.component.aws2-kinesis-firehose.secret-key | Amazon AWS Secret Key | String | |
camel.component.aws2-kinesis-firehose.session-token | Amazon AWS Session Token used when the user needs to assume a IAM role | String | |
camel.component.aws2-kinesis-firehose.trust-all-certificates | If we want to trust all certificates in case of overriding the endpoint | false | Boolean |
camel.component.aws2-kinesis-firehose.uri-endpoint-override | Set the overriding uri endpoint. This option needs to be used in combination with overrideEndpoint option | String | |
camel.component.aws2-kinesis-firehose.use-default-credentials-provider | Set whether the Kinesis Firehose client should expect to load credentials through a default credentials provider or to expect static credentials to be passed in. | false | Boolean |
camel.component.aws2-kinesis-firehose.use-profile-credentials-provider | Set whether the Kinesis Firehose client should expect to load credentials through a profile credentials provider. | false | Boolean |
camel.component.aws2-kinesis-firehose.use-session-credentials | Set whether the Kinesis Firehose client should expect to use Session Credentials. This is useful in situation in which the user needs to assume a IAM role for doing operations in Kinesis Firehose. | false | Boolean |
camel.component.aws2-kinesis.access-key | Amazon AWS Access Key | String | |
camel.component.aws2-kinesis.amazon-kinesis-async-client | Supply a pre-constructed Amazon Kinesis async client to use for the KCL Consumer. The option is a software.amazon.awssdk.services.kinesis.KinesisAsyncClient type. | KinesisAsyncClient | |
camel.component.aws2-kinesis.amazon-kinesis-client | Amazon Kinesis client to use for all requests for this endpoint. The option is a software.amazon.awssdk.services.kinesis.KinesisClient type. | KinesisClient | |
camel.component.aws2-kinesis.application-name | Name of the KCL application. This defaults to the stream name. | String | |
camel.component.aws2-kinesis.async-client | If we want to a KinesisAsyncClient instance set it to true | false | Boolean |
camel.component.aws2-kinesis.autowired-enabled | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | Boolean |
camel.component.aws2-kinesis.bridge-error-handler | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions (if possible) occurred while the Camel consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. Important: This is only possible if the 3rd party component allows Camel to be alerted if an exception was thrown. Some components handle this internally only, and therefore bridgeErrorHandler is not possible. In other situations we may improve the Camel component to hook into the 3rd party component and make this possible for future releases. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | Boolean |
camel.component.aws2-kinesis.cbor-enabled | This option will set the CBOR_ENABLED property during the execution | true | Boolean |
camel.component.aws2-kinesis.cloud-watch-async-client | If we want to a KCL Consumer, we can pass an instance of CloudWatchAsyncClient. The option is a software.amazon.awssdk.services.cloudwatch.CloudWatchAsyncClient type. | CloudWatchAsyncClient | |
camel.component.aws2-kinesis.configuration | Component configuration. The option is a org.apache.camel.component.aws2.kinesis.Kinesis2Configuration type. | Kinesis2Configuration | |
camel.component.aws2-kinesis.dynamo-db-async-client | If we want to a KCL Consumer, we can pass an instance of DynamoDbAsyncClient. The option is a software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient type. | DynamoDbAsyncClient | |
camel.component.aws2-kinesis.enabled | Whether to enable auto configuration of the aws2-kinesis component. This is enabled by default. | Boolean | |
camel.component.aws2-kinesis.health-check-consumer-enabled | Used for enabling or disabling all consumer based health checks from this component | true | Boolean |
camel.component.aws2-kinesis.health-check-producer-enabled | Used for enabling or disabling all producer based health checks from this component. Notice: Camel has by default disabled all producer based health-checks. You can turn on producer checks globally by setting camel.health.producersEnabled=true. | true | Boolean |
camel.component.aws2-kinesis.iterator-type | Defines where in the Kinesis stream to start getting records | trim-horizon | ShardIteratorType |
camel.component.aws2-kinesis.kcl-disable-cloudwatch-metrics-export | If we want to use a KCL Consumer and disable the CloudWatch Metrics Export | false | Boolean |
camel.component.aws2-kinesis.lazy-start-producer | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean |
camel.component.aws2-kinesis.max-results-per-request | Maximum number of records that will be fetched in each poll | 1 | Integer |
camel.component.aws2-kinesis.message-timestamp | The message timestamp to start polling from. Required if iteratorType is set to AT_TIMESTAMP | String | |
camel.component.aws2-kinesis.override-endpoint | Set the need for overriding the endpoint. This option needs to be used in combination with uriEndpointOverride option | false | Boolean |
camel.component.aws2-kinesis.profile-credentials-name | If using a profile credentials provider this parameter will set the profile name. | String | |
camel.component.aws2-kinesis.proxy-host | To define a proxy host when instantiating the Kinesis client | String | |
camel.component.aws2-kinesis.proxy-port | To define a proxy port when instantiating the Kinesis client | Integer | |
camel.component.aws2-kinesis.proxy-protocol | To define a proxy protocol when instantiating the Kinesis client | https | Protocol |
camel.component.aws2-kinesis.region | The region in which Kinesis Firehose client needs to work. When using this parameter, the configuration will expect the lowercase name of the region (for example ap-east-1) You’ll need to use the name Region.EU_WEST_1.id() | String | |
camel.component.aws2-kinesis.secret-key | Amazon AWS Secret Key | String | |
camel.component.aws2-kinesis.sequence-number | The sequence number to start polling from. Required if iteratorType is set to AFTER_SEQUENCE_NUMBER or AT_SEQUENCE_NUMBER | String | |
camel.component.aws2-kinesis.session-token | Amazon AWS Session Token used when the user needs to assume a IAM role | String | |
camel.component.aws2-kinesis.shard-closed | Define what will be the behavior in case of shard closed. Possible value are ignore, silent and fail. In case of ignore a WARN message will be logged once and the consumer will not process new messages until restarted,in case of silent there will be no logging and the consumer will not process new messages until restarted,in case of fail a ReachedClosedStateException will be thrown | ignore | Kinesis2ShardClosedStrategyEnum |
camel.component.aws2-kinesis.shard-id | Defines which shardId in the Kinesis stream to get records from | String | |
camel.component.aws2-kinesis.shard-monitor-interval | The interval in milliseconds to wait between shard polling | 10000 | Long |
camel.component.aws2-kinesis.trust-all-certificates | If we want to trust all certificates in case of overriding the endpoint | false | Boolean |
camel.component.aws2-kinesis.uri-endpoint-override | Set the overriding uri endpoint. This option needs to be used in combination with overrideEndpoint option | String | |
camel.component.aws2-kinesis.use-default-credentials-provider | Set whether the Kinesis client should expect to load credentials through a default credentials provider or to expect static credentials to be passed in. | false | Boolean |
camel.component.aws2-kinesis.use-kcl-consumers | If we want to a KCL Consumer set it to true | false | Boolean |
camel.component.aws2-kinesis.use-profile-credentials-provider | Set whether the Kinesis client should expect to load credentials through a profile credentials provider. | false | Boolean |
camel.component.aws2-kinesis.use-session-credentials | Set whether the Kinesis client should expect to use Session Credentials. This is useful in situation in which the user needs to assume a IAM role for doing operations in Kinesis. | false | Boolean |