Blog post featured image

Apache Camel Kafka Connector 0.7.0 has just been released.

This is based on the LTS release of Apache Camel 3.7.0, this means we will provide patch releases, as Camel 3.7.x is an LTS release.

So what’s in this release?

This release introduce bug fixes, improvements, new features and new connectors obviously

New connectors

The new connectors introduced in this release are the following:

  • AtlasMap: Transforms the message using an AtlasMap transformation
  • Kubernetes Custom Resources: Perform operations on Kubernetes Custom Resources and get notified on Deployment changes
  • Vert.X Kafka: Sent and receive messages to/from an Apache Kafka broker using vert.x Kafka client
  • JSON JSON-B: Marshal POJOs to JSON and back using JSON-B
  • CSimple: Evaluate a compiled simple expression language
  • DataSonnet: To use DataSonnet scripts in Camel expressions or predicates
  • jOOR: Evaluate a jOOR (Java compiled once at runtime) expression language

Support for idempotent repository

In this release we introduced the idempotency support.

The initial support will provide:

  • in-memory idempotent repository
  • kafka idempotent repository

The idempotency will be supported on both source and sink connector, this means:

  • From a source point of view, by enabling the idempotency, you’ll be able to avoid ingesting the same payload to the target Kafka topic. This can be done at body or header level.
  • From a sink point of view, by enabling the idempotency, you’ll be able to avoid sending the same payload to an external system, through checking body or header of the Kafka record from the Kafka source topic.

Support for remove headers option

There are situation in which you’ll be using a Camel-Kafka-Connector source connector and sink connector together. Sometimes you’ll need to remove some of the Camel headers to achieve a particular behavior. This option should be used in this case. The value of this option can be a list of headers or a regexp.

In the camel-kafka-connector-examples repository we provided a little example based on 0.7.0 SNAPSHOT version.

Json marshal/unmarshal present by default in each connector

We added camel-jackson in each connector so that is possible to use json-jackson marshal/unmarshal out of the box.

New Archetypes

We added two new archetypes:

  • Apicurio Archetype: To be able to leveraging the apicurio service registry
  • Dataformat Archetype: To extend a connector by adding a dataformat to it

Improved integration tests

We improved the integration tests by leveraging the really good work by Otavio R. Piske on the camel-test-infra platform. We are now basing the integration on the camel-test-infra modules coming from the Camel release.

Documentation

On the Archetypes side we added more documentation about the base extensible connector and on the apicurio archetype and dataformat archetype (added in this release)

Strimzi and Kafka Version

This release has been based on Kafka 2.6.0 and Strimzi 0.20.1

Examples

We added a bunch of new examples to the camel-kafka-connector-examples repository

What to expect

We are already working on the next release based on the upcoming 3.8.0 Camel release. We are working on improving the documentation and adding new stuff.

Apache Camel Kafka Connector 0.5.0: What's New

 , by ,

Apache Camel Kafka Connector 0.5.0 has just been released. This is based on the non-LTS release of Apache Camel 3.5.0, this means we will not provide patch releases, but use the release as-is. So what’s in this release? This release introduce bug fixes, improvements and new connectors obviously New connectors The new connectors introduced in this release are the following: ArangoDB: Perform operations on ArangoDb when used as a Document Database, or as a Graph Database.

Continue reading ❯

RELEASES

Camel Kafka Connector 0.4.0 Released

 , by ,

We recently released camel-kafka-connector 0.4.0. This is the first release of the latest project in the Camel’s ecosystem, based on an LTS camel release. The main features of this release are: Introduction of aggregation support Introduction of marshalling and unmarshalling support on both sink and source connectors Upgrade to the latest Apache Camel release 3.4.2 Addition of new examples in the camel-kafka-connector-examples repository Integration tests added for HDFS (sink), Cassandra/CQL (source), Slack (Source), JDBC (sink) and MongoDB (sink/source).

Continue reading ❯

RELEASES

Apache Camel Kafka Connector 0.1.0 Released

 , by ,

The Apache Camel community is pleased to announce the first release (0.1.0) of Camel-Kafka-connector project. This release is an early opportunity for the community to try the project and share feedback about usage of the autogenerated connectors as well as features ideas and use cases for the next development iterations. The project provides a tiny integration layer between camel and kafka connect frameworks and generate one kafka connector for each existing camel component.

Continue reading ❯

RELEASES

Introducing Idempotency Support in Camel Kafka Connector

 , by

In the next Camel Kafka connector release (0.7.0, on vote soon) there will be a new feature: the idempotency support on both source and sink connectors. The aim of this post is giving some hints on how and when to use the idempotency feature. What is Idempotency? The Idempotent Consumer from the EIP patterns is used to filter out duplicate messages: it essentially acts like a Message Filter to filter out duplicates, as reported in the Camel documentation

Continue reading ❯

RELEASESCAMEL-KAFKA-CONNECTOR

Apache Camel 3.5 What's New

 , by ,

Apache Camel 3.5 has just been released. This is a non-LTS release which means we will not provide patch releases, but use the release as-is. The next planned LTS release is 3.7 scheduled towards end of the year. So what’s in this release? This release introduces new set of features and noticeable improvements that will we cover in this blog post. Java 14 This is the first release that supports Java 14.

Continue reading ❯

RELEASES