Kafka Source

Provided by: "Apache Software Foundation"

Support Level for this Kamelet is: "Stable"

Receive data from Kafka topics through Plain Login Module.

Configuration Options

The following table summarizes the configuration options available for the kafka-source Kamelet:

Property	Name	Description	Type	Default	Example
bootstrapServers	Bootstrap Servers	Required Comma separated list of Kafka Broker URLs.	string
password	Password	Required Password to authenticate to kafka.	string
topic	Topic Names	Required Comma separated list of Kafka topic names.	string
user	Username	Required Username to authenticate to Kafka.	string
allowManualCommit	Allow Manual Commit	Whether to allow doing manual commits.	boolean	false
autoCommitEnable	Auto Commit Enable	If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer.	boolean	true
autoOffsetReset	Auto Offset Reset	What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none.	string	latest
consumerGroup	Consumer Group	A string that uniquely identifies the group of consumers to which this source belongs.	string		my-group-id
deserializeHeaders	Automatically Deserialize Headers	When enabled the Kamelet source will deserialize all message headers to String representation.	boolean	true
pollOnError	Poll On Error Behavior	What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP.	string	ERROR_HANDLER
saslMechanism	SASL Mechanism	The Simple Authentication and Security Layer (SASL) Mechanism used.	string	PLAIN
securityProtocol	Security Protocol	Protocol used to communicate with brokers. SASL_PLAINTEXT, PLAINTEXT, SASL_SSL and SSL are supported.	string	SASL_SSL
topicIsPattern	Topic Is Pattern	Whether the topic is a pattern (regular expression). This can be used to subscribe to dynamic number of topics matching the pattern.	boolean	false

Property

Name

Description

Type

Default

Example

bootstrapServers

Bootstrap Servers

Required Comma separated list of Kafka Broker URLs.

string

password

Password

Required Password to authenticate to kafka.

string

topic

Topic Names

Required Comma separated list of Kafka topic names.

string

user

Username

Required Username to authenticate to Kafka.

string

allowManualCommit

Allow Manual Commit

Whether to allow doing manual commits.

boolean

false

autoCommitEnable

Auto Commit Enable

If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer.

boolean

true

autoOffsetReset

Auto Offset Reset

What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none.

string

latest

consumerGroup

Consumer Group

A string that uniquely identifies the group of consumers to which this source belongs.

string

my-group-id

deserializeHeaders

Automatically Deserialize Headers

When enabled the Kamelet source will deserialize all message headers to String representation.

boolean

true

pollOnError

Poll On Error Behavior

What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP.

string

ERROR_HANDLER

saslMechanism

SASL Mechanism

The Simple Authentication and Security Layer (SASL) Mechanism used.

string

PLAIN

securityProtocol

Security Protocol

Protocol used to communicate with brokers. SASL_PLAINTEXT, PLAINTEXT, SASL_SSL and SSL are supported.

string

SASL_SSL

topicIsPattern

Topic Is Pattern

Whether the topic is a pattern (regular expression). This can be used to subscribe to dynamic number of topics matching the pattern.

boolean

false

Dependencies

At runtime, the kafka-source Kamelet relies upon the presence of the following dependencies:

mvn:org.apache.camel.kamelets:camel-kamelets-utils:4.8.3-SNAPSHOT
camel:core
camel:kafka
camel:kamelet

Camel JBang usage

Prerequisites

You’ve installed JBang.
You have executed the following command:

jbang app install camel@apache/camel

Supposing you have a file named route.yaml with this content:

- route:
    from:
      uri: "kamelet:kafka-source"
      parameters:
        .
        .
        .
      steps:
        - to:
            uri: "kamelet:log-sink"

You can now run it directly through the following command

camel run route.yaml

Kamelet source file

https://github.com/apache/camel-kamelets/blob/main/kamelets/kafka-source.kamelet.yaml