camel-azure-cosmosdb-kafka-connector source configuration

Connector description: Azure Cosmos DB is Microsofts globally distributed, multi-model database service for operational and analytics workloads. It offers multi-mastering feature by automatically scaling throughput, compute, and storage. This component interacts with Azure CosmosDB through Azure SQL API

When using camel-azure-cosmosdb-kafka-connector as source make sure to use the following Maven dependency to have support for the connector:

<dependency>
  <groupId>org.apache.camel.kafkaconnector</groupId>
  <artifactId>camel-azure-cosmosdb-kafka-connector</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel Kafka connector version -->
</dependency>

To use this Source connector in Kafka connect you’ll need to set the following connector.class

connector.class=org.apache.camel.kafkaconnector.azurecosmosdb.CamelAzurecosmosdbSourceConnector

The camel-azure-cosmosdb source connector supports 48 options, which are listed below.

Name Description Default Required Priority

camel.source.path.databaseName

The name of the Cosmos database that component should connect to. In case you are producing data and have createDatabaseIfNotExists=true, the component will automatically auto create a Cosmos database.

null

false

MEDIUM

camel.source.path.containerName

The name of the Cosmos container that component should connect to. In case you are producing data and have createContainerIfNotExists=true, the component will automatically auto create a Cosmos container.

null

false

MEDIUM

camel.source.endpoint.clientTelemetryEnabled

Sets the flag to enable client telemetry which will periodically collect database operations aggregation statistics, system information like cpu/memory and send it to cosmos monitoring service, which will be helpful during debugging. DEFAULT value is false indicating this is opt in feature, by default no telemetry collection.

false

false

MEDIUM

camel.source.endpoint.connectionSharingAcross ClientsEnabled

Enables connections sharing across multiple Cosmos Clients. The default is false. When you have multiple instances of Cosmos Client in the same JVM interacting to multiple Cosmos accounts, enabling this allows connection sharing in Direct mode if possible between instances of Cosmos Client. Please note, when setting this option, the connection configuration (e.g., socket timeout config, idle timeout config) of the first instantiated client will be used for all other client instances.

false

false

MEDIUM

camel.source.endpoint.consistencyLevel

Sets the consistency levels supported for Azure Cosmos DB client operations in the Azure Cosmos DB service. The requested ConsistencyLevel must match or be weaker than that provisioned for the database account. Consistency levels by order of strength are STRONG, BOUNDED_STALENESS, SESSION and EVENTUAL. Refer to consistency level documentation for additional details: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels One of: [Strong] [BoundedStaleness] [Session] [Eventual] [ConsistentPrefix]

"SESSION"

false

MEDIUM

camel.source.endpoint.containerPartitionKeyPath

Sets the container partition key path.

null

false

MEDIUM

camel.source.endpoint.contentResponseOnWrite Enabled

Sets the boolean to only return the headers and status code in Cosmos DB response in case of Create, Update and Delete operations on CosmosItem. In Consumer, it is enabled by default because of the ChangeFeed in the consumer that needs this flag to be enabled and thus is shouldn’t be overridden. In Producer, it advised to disable it since it reduces the network overhead

true

false

MEDIUM

camel.source.endpoint.cosmosAsyncClient

Inject an external CosmosAsyncClient into the component which provides a client-side logical representation of the Azure Cosmos DB service. This asynchronous client is used to configure and execute requests against the service.

null

false

MEDIUM

camel.source.endpoint.createContainerIfNotExists

Sets if the component should create Cosmos container automatically in case it doesn’t exist in Cosmos database

false

false

MEDIUM

camel.source.endpoint.createDatabaseIfNotExists

Sets if the component should create Cosmos database automatically in case it doesn’t exist in Cosmos account

false

false

MEDIUM

camel.source.endpoint.databaseEndpoint

Sets the Azure Cosmos database endpoint the component will connect to.

null

true

HIGH

camel.source.endpoint.multipleWriteRegionsEnabled

Sets the flag to enable writes on any regions for geo-replicated database accounts in the Azure Cosmos DB service. When the value of this property is true, the SDK will direct write operations to available writable regions of geo-replicated database account. Writable regions are ordered by PreferredRegions property. Setting the property value to true has no effect until EnableMultipleWriteRegions in DatabaseAccount is also set to true. DEFAULT value is true indicating that writes are directed to available writable regions of geo-replicated database account.

true

false

MEDIUM

camel.source.endpoint.preferredRegions

Sets the preferred regions for geo-replicated database accounts. For example, East US as the preferred region. When EnableEndpointDiscovery is true and PreferredRegions is non-empty, the SDK will prefer to use the regions in the container in the order they are specified to perform operations.

null

false

MEDIUM

camel.source.endpoint.readRequestsFallbackEnabled

Sets whether to allow for reads to go to multiple regions configured on an account of Azure Cosmos DB service. DEFAULT value is true. If this property is not set, the default is true for all Consistency Levels other than Bounded Staleness, The default is false for Bounded Staleness. 1. endpointDiscoveryEnabled is true 2. the Azure Cosmos DB account has more than one region

true

false

MEDIUM

camel.source.endpoint.throughputProperties

Sets throughput of the resources in the Azure Cosmos DB service.

null

false

MEDIUM

camel.source.endpoint.bridgeErrorHandler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

false

MEDIUM

camel.source.endpoint.changeFeedProcessorOptions

Sets the ChangeFeedProcessorOptions to be used. Unless specifically set the default values that will be used are: maximum items per page or FeedResponse: 100 lease renew interval: 17 seconds lease acquire interval: 13 seconds lease expiration interval: 60 seconds feed poll delay: 5 seconds maximum scale count: unlimited

null

false

MEDIUM

camel.source.endpoint.createLeaseContainerIfNot Exists

Sets if the component should create Cosmos lease container for the consumer automatically in case it doesn’t exist in Cosmos database

false

false

MEDIUM

camel.source.endpoint.createLeaseDatabaseIfNot Exists

Sets if the component should create Cosmos lease database for the consumer automatically in case it doesn’t exist in Cosmos account

false

false

MEDIUM

camel.source.endpoint.hostName

Sets the hostname. The host: a host is an application instance that uses the change feed processor to listen for changes. Multiple instances with the same lease configuration can run in parallel, but each instance should have a different instance name. If not specified, this will be a generated random hostname.

null

false

MEDIUM

camel.source.endpoint.leaseContainerName

Sets the lease container which acts as a state storage and coordinates processing the change feed across multiple workers. The lease container can be stored in the same account as the monitored container or in a separate account. It will be auto created if createLeaseContainerIfNotExists is set to true.

"camel-lease"

false

MEDIUM

camel.source.endpoint.leaseDatabaseName

Sets the lease database where the leaseContainerName will be stored. If it is not specified, this component will store the lease container in the same database that is specified in databaseName. It will be auto created if createLeaseDatabaseIfNotExists is set to true.

null

false

MEDIUM

camel.source.endpoint.exceptionHandler

To let the consumer use a custom ExceptionHandler. Notice if the option bridgeErrorHandler is enabled then this option is not in use. By default the consumer will deal with exceptions, that will be logged at WARN or ERROR level and ignored.

null

false

MEDIUM

camel.source.endpoint.exchangePattern

Sets the exchange pattern when the consumer creates an exchange. One of: [InOnly] [InOut] [InOptionalOut]

null

false

MEDIUM

camel.source.endpoint.accountKey

Sets either a master or readonly key used to perform authentication for accessing resource.

null

true

HIGH

camel.component.azure-cosmosdb.clientTelemetry Enabled

Sets the flag to enable client telemetry which will periodically collect database operations aggregation statistics, system information like cpu/memory and send it to cosmos monitoring service, which will be helpful during debugging. DEFAULT value is false indicating this is opt in feature, by default no telemetry collection.

false

false

MEDIUM

camel.component.azure-cosmosdb.configuration

The component configurations

null

false

MEDIUM

camel.component.azure-cosmosdb.connectionSharing AcrossClientsEnabled

Enables connections sharing across multiple Cosmos Clients. The default is false. When you have multiple instances of Cosmos Client in the same JVM interacting to multiple Cosmos accounts, enabling this allows connection sharing in Direct mode if possible between instances of Cosmos Client. Please note, when setting this option, the connection configuration (e.g., socket timeout config, idle timeout config) of the first instantiated client will be used for all other client instances.

false

false

MEDIUM

camel.component.azure-cosmosdb.consistencyLevel

Sets the consistency levels supported for Azure Cosmos DB client operations in the Azure Cosmos DB service. The requested ConsistencyLevel must match or be weaker than that provisioned for the database account. Consistency levels by order of strength are STRONG, BOUNDED_STALENESS, SESSION and EVENTUAL. Refer to consistency level documentation for additional details: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency-levels One of: [Strong] [BoundedStaleness] [Session] [Eventual] [ConsistentPrefix]

"SESSION"

false

MEDIUM

camel.component.azure-cosmosdb.containerPartition KeyPath

Sets the container partition key path.

null

false

MEDIUM

camel.component.azure-cosmosdb.contentResponseOn WriteEnabled

Sets the boolean to only return the headers and status code in Cosmos DB response in case of Create, Update and Delete operations on CosmosItem. In Consumer, it is enabled by default because of the ChangeFeed in the consumer that needs this flag to be enabled and thus is shouldn’t be overridden. In Producer, it advised to disable it since it reduces the network overhead

true

false

MEDIUM

camel.component.azure-cosmosdb.cosmosAsyncClient

Inject an external CosmosAsyncClient into the component which provides a client-side logical representation of the Azure Cosmos DB service. This asynchronous client is used to configure and execute requests against the service.

null

false

MEDIUM

camel.component.azure-cosmosdb.createContainerIf NotExists

Sets if the component should create Cosmos container automatically in case it doesn’t exist in Cosmos database

false

false

MEDIUM

camel.component.azure-cosmosdb.createDatabaseIfNot Exists

Sets if the component should create Cosmos database automatically in case it doesn’t exist in Cosmos account

false

false

MEDIUM

camel.component.azure-cosmosdb.databaseEndpoint

Sets the Azure Cosmos database endpoint the component will connect to.

null

true

HIGH

camel.component.azure-cosmosdb.multipleWrite RegionsEnabled

Sets the flag to enable writes on any regions for geo-replicated database accounts in the Azure Cosmos DB service. When the value of this property is true, the SDK will direct write operations to available writable regions of geo-replicated database account. Writable regions are ordered by PreferredRegions property. Setting the property value to true has no effect until EnableMultipleWriteRegions in DatabaseAccount is also set to true. DEFAULT value is true indicating that writes are directed to available writable regions of geo-replicated database account.

true

false

MEDIUM

camel.component.azure-cosmosdb.preferredRegions

Sets the preferred regions for geo-replicated database accounts. For example, East US as the preferred region. When EnableEndpointDiscovery is true and PreferredRegions is non-empty, the SDK will prefer to use the regions in the container in the order they are specified to perform operations.

null

false

MEDIUM

camel.component.azure-cosmosdb.readRequests FallbackEnabled

Sets whether to allow for reads to go to multiple regions configured on an account of Azure Cosmos DB service. DEFAULT value is true. If this property is not set, the default is true for all Consistency Levels other than Bounded Staleness, The default is false for Bounded Staleness. 1. endpointDiscoveryEnabled is true 2. the Azure Cosmos DB account has more than one region

true

false

MEDIUM

camel.component.azure-cosmosdb.throughput Properties

Sets throughput of the resources in the Azure Cosmos DB service.

null

false

MEDIUM

camel.component.azure-cosmosdb.bridgeErrorHandler

Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored.

false

false

MEDIUM

camel.component.azure-cosmosdb.changeFeedProcessor Options

Sets the ChangeFeedProcessorOptions to be used. Unless specifically set the default values that will be used are: maximum items per page or FeedResponse: 100 lease renew interval: 17 seconds lease acquire interval: 13 seconds lease expiration interval: 60 seconds feed poll delay: 5 seconds maximum scale count: unlimited

null

false

MEDIUM

camel.component.azure-cosmosdb.createLease ContainerIfNotExists

Sets if the component should create Cosmos lease container for the consumer automatically in case it doesn’t exist in Cosmos database

false

false

MEDIUM

camel.component.azure-cosmosdb.createLeaseDatabase IfNotExists

Sets if the component should create Cosmos lease database for the consumer automatically in case it doesn’t exist in Cosmos account

false

false

MEDIUM

camel.component.azure-cosmosdb.hostName

Sets the hostname. The host: a host is an application instance that uses the change feed processor to listen for changes. Multiple instances with the same lease configuration can run in parallel, but each instance should have a different instance name. If not specified, this will be a generated random hostname.

null

false

MEDIUM

camel.component.azure-cosmosdb.leaseContainerName

Sets the lease container which acts as a state storage and coordinates processing the change feed across multiple workers. The lease container can be stored in the same account as the monitored container or in a separate account. It will be auto created if createLeaseContainerIfNotExists is set to true.

"camel-lease"

false

MEDIUM

camel.component.azure-cosmosdb.leaseDatabaseName

Sets the lease database where the leaseContainerName will be stored. If it is not specified, this component will store the lease container in the same database that is specified in databaseName. It will be auto created if createLeaseDatabaseIfNotExists is set to true.

null

false

MEDIUM

camel.component.azure-cosmosdb.autowiredEnabled

Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc.

true

false

MEDIUM

camel.component.azure-cosmosdb.accountKey

Sets either a master or readonly key used to perform authentication for accessing resource.

null

true

HIGH

The camel-azure-cosmosdb source connector has no converters out of the box.

The camel-azure-cosmosdb source connector has no transforms out of the box.

The camel-azure-cosmosdb source connector has no aggregation strategies out of the box.