Camel Integration with Kubernetes

Please make sure to meet these prerequisites for running Camel integrations on Kubernetes:

  • Connect to namespace on a Kubernetes cluster where you want to run the integration

  • Camel K operator must be installed on the Kubernetes cluster (either installed on the same namespace or as global operator for the whole cluster)

Running Camel routes on Kubernetes is quite simple with Camel JBang. In fact, you can develop and test your Camel route locally with Camel JBang and then promote the same source to running it as an integration on Kubernetes.

The Camel JBang Kubernetes functionality is provided as a command plugin. This means you need to enable the camel-k plugin first to use the subcommands in Camel JBang.

camel plugin add camel-k

You should see the camel-k plugin listed as installed plugin.

camel plugin get

Now Camel JBang is able to run the subcommands offered by the plugin.

Run integrations

Simply run the integration using the k subcommand in Camel JBang.

camel k run route.yaml

The command runs the Camel integration on Kubernetes. More precisely it creates a Camel K Integration custom resource in the current namespace. The Camel K operator makes sure to create a proper runtime image and run the integration (usually as a Pod).

The Camel K operator will automatically manage and configure this integration. In particular the operator takes care on exposing services, configuring health endpoints, providing metrics, updating image streams and much more.

By default, the run command will not wait for the integration to in state running. You need to add -w or --wait option in order to wait for the integration to become ready.

The --logs option makes the command also print the integration output once the integration Pod is running.

The run command offers a lot more options that you may use to configure the Camel K integration.

Option Description

--name

The integration name. Use this when the name should not get derived from the source file name.

--image

An image built externally (for instance via CI/CD). Enabling it will skip the integration build phase.

--kit, -k

The kit used to run the integration.

--profile

The trait profile to use for the deployment.

--service-account

The service account used to run this Integration.

--pod-template

The path of the YAML file containing a PodSpec template to be used for the integration pods.

--operator-id

Operator id selected to manage this integration. (default=camel-k)

--dependency, -d

Adds dependency that should be included, use "camel:" prefix for a Camel component, "mvn:org.my:app:1.0" for a Maven dependency.

--property, -p

Add a runtime property or properties file from a path, a config map or a secret (syntax: [my-key=my-value,file:/path/to/my-conf.properties,[configmap,secret]:name]).

--build-property

Add a build time property or properties file from a path, a config map or a secret (syntax: [my-key=my-value,file:/path/to/my-conf.properties,[configmap,secret]:name]]).

--config

Add a runtime configuration from a ConfigMap or a Secret (syntax: [configmap,secret]:name[/key], where name represents the configmap/secret name and key optionally represents the configmap/secret key to be filtered).

--resource

Add a runtime resource from a Configmap or a Secret (syntax: [configmap,secret]:name[/key][@path], where name represents the configmap/secret name, key optionally represents the configmap/secret key to be filtered and path represents the destination path).

--open-api

Add an OpenAPI spec (syntax: [configmap,file]:name).

--env, -e

Set an environment variable in the integration container, for instance "-e MY_VAR=my-value".

--volume, -v

Mount a volume into the integration container, for instance "-v pvcname:/container/path".

--connect, -c

A Service that the integration should bind to, specified as [[apigroup/]version:]kind:[namespace/]name.

--source

Add source file to your integration, this is added to the list of files listed as arguments of the command.

--maven-repository

Add a maven repository used to resolve dependencies.

--annotation

Add an annotation to the integration. Use name values pairs like "--annotation my.company=hello".

--label

Add a label to the integration. Use name values pairs like "--label my.company=hello".

--traits, -t

Add a trait configuration to the integration. Use name values pairs like "--trait trait.name.config=hello".

--use-flows

Write yaml sources as Flow objects in the integration custom resource (default=true).

--compression

Enable storage of sources and resources as a compressed binary blobs.

--wait, -w

Wait for the integration to become ready.

--logs, -l

Print logs after integration has been started.

--output, -o

Just output the generated integration custom resource (supports: yaml or json).

List integrations

You can list the available integration resources with the following command.

camel k get
NAME      PHASE    KIT            READY
my-route  Running  kit-123456789   1/1

This looks for all integrations in the current namespace and lists their individual status.

Show integration logs

To inspect the log output of a running integration call:

camel k logs my-route

The command connects to the running integration Pod and prints the log output. Just terminate the process to stop printing the logs.

Delete integrations

Of course, you may also delete an integration resource from the cluster.

camel k delete my-route

To remove all available integrations on the current namespace use the --all option.

camel k delete --all

Create integration pipes

In some contexts (for example "serverless") users often want to leverage the power of Apache Camel to be able to connect to various sources/sinks, with focus on connectivity to 3rd party technologies and services and less focus on doing complex processing (such as transformations or other enterprise integration patterns).

Pipe resources represent a special form of Camel integrations where a source gets bound to a sink. The operation to create such a Pipe resource is often related to as the process of binding a source to a sink.

You can use the Camel JBang subcommand bind to create Pipe resources. The result of this Pipe resource being created on a Kubernetes cluster is a running Camel integration.

The Camel K bind command supports the following options:

Option Description

--operator-id

Operator id selected to manage this integration. (default=camel-k)

--source

Source (from) such as a Kamelet or Camel endpoint uri that provides data..

--sink

Sink (to) such as a Kamelet or Camel endpoint uri where data should be sent to.

--step

Add optional 1-n steps to the pipe processing. Each step represents a reference to a Kamelet of type action.

--property

Add a pipe property in the form of [source,sink,error-handler,step-<n>].<key>=<value> where <n> is the step number starting from 1.

--error-handler

Add error handler (none,log,sink:<endpoint>). Sink endpoints are expected in the format [[apigroup/]version:]kind:[namespace/]name, plain Camel URIs or Kamelet name.

--annotation

Add an annotation to the integration. Use name values pairs like "--annotation my.company=hello".

--connect

A Service that the integration should bind to, specified as [[apigroup/]version:]kind:[namespace/]name.

--traits

Add a trait configuration to the integration. Use name values pairs like "--trait trait.name.config=hello".

--wait

Wait for the integration to become ready.

--logs

Print logs after integration has been started.

--output

Just output the generated pipe custom resource (supports: file, yaml or json).

Sources and sinks in a pipe may be Camel endpoint URIs, a Kamelet or a references to a Kubernetes resource (e.g. Knative brokers, Kafka topics).

Binding Kamelets

In a typical use case a Pipe connects Kamelets of type source and sink. Usually a Kamelet gets identified by its name (e.g. timer-source, log-sink).

camel k bind my-pipe --source timer-source --sink log-sink --property source.message="Camel rocks!" --property sink.showHeaders=true

The bind command receives the name of the pipe as a command argument and uses several options to specify the source and the sink. In addition to that the user is able to specify properties on the individual source and sink (e.g. the message property on the timer-source Kamelet).

The result of this command is a Pipe custom resource that you can apply to a Kubernetes cluster.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
  annotations:
    camel.apache.org/operator.id: camel-k
spec:
  source: (1)
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: timer-source
    properties:
      message: "Camel rocks!"
  sink: (2)
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink
    properties:
      showHeaders: true
1 Reference to the source that provides data
2 Reference to the sink where data should be sent to

Each Pipe resource uses an operator id annotation to specify which operator on the cluster should handle the resource.

The bind command is able to inspect the properties defined in the Kamelet specification in order to set default values. In case the Kamelet defines a required property that is not explicitly set by the user the bind command automatically creates a property placeholder with an example value.

Add binding steps

You can specify 1-n additional steps that get executed between the source and sink.

camel k bind my-pipe --source timer-source --sink log-sink --step set-body-action --property step-1.value="Camel rocks!"
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
spec:
  source:
# ...
  steps:
  - ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: set-body-action
    properties:
      value: "Camel rocks!"
  sink:
# ...
Each step should reverence a Kamelet of type action. The properties for a step can be set with the respective step-<n> prefix where n is the step number beginning with 1.

Binding Camel endpoint URIs

Instead of referencing a Kamelet or Kubernetes resource you can also configure the source/sink to be an explicit Camel URI. For example, the following bind command is allowed:

camel k bind my-pipe --source timer:tick --sink https://mycompany.com/the-service --property source.period=5000

This will use the Camel endpoint URIs timer:tick and log:info as source and sink in the Pipe. The properties are set as endpoint parameters.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
spec:
  source:
    uri: timer:tick (1)
    properties:
      period: 5000
  sink:
    uri: https://mycompany.com/the-service (2)
1 Pipe with explicit Camel endpoint URI as source
2 Pipe with explicit Camel endpoint URI as sink where the data gets pushed to

This Pipe explicitly defines Camel endpoint URIs that act as a source and sink.

You can also specify endpoint parameters directly on the source/sink like --source timer:tick?period=5000

Binding to Knative broker

You can reference Knative eventing resources as source or sink in a Pipe resource. The reference to the Knative resource is identified by the apiVersion, kind and resource name. Users may add properties to the object reference as usual.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
  annotations:
    camel.apache.org/operator.id: camel-k
spec:
  source: (1)
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: timer-source
    properties:
      message: "Camel rocks!"
  sink: (2)
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: default
    properties:
      type: org.apache.camel.event.my-event (3)
1 Reference to the source that provides data
2 Reference to the Knative broker where data should be sent to
3 The CloudEvents event type that is used for the events
Knative eventing uses CloudEvents data format by default. Camel provides the concept of data types that is able to transform from many different component data formats to CloudEvents data type. The data type transformation will set proper event properties such as ce-type, ce-source or ce-subject.

When referencing a Knative broker as a source the type property is mandatory in order to filter the event stream.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
  annotations:
    camel.apache.org/operator.id: camel-k
spec:
  source: (1)
    ref:
      kind: Broker
      apiVersion: eventing.knative.dev/v1
      name: default
    properties:
      type: org.apache.camel.event.my-event (2)
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink
1 Reference to the source Knative broker that provides the events
2 Filter the event stream for events with the given CloudEvents event type

Binding to Knative channels

Knative eventing provides the channel resource for a subscription consumer model. Camel K is able to automatically manage the subscription when referencing Knative eventing channels as a source or sink in a Pipe.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
  annotations:
    camel.apache.org/operator.id: camel-k
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: timer-source
    properties:
      message: "Camel rocks!"
  sink: (1)
    ref:
      kind: InMemoryChannel
      apiVersion: messaging.knative.dev/v1
      name: my-messages
1 Reference to the Knative message channel that receives the events

The same approach can be used to subscribe to a message chanel as a consumer to act as a source in a Pipe.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
  annotations:
    camel.apache.org/operator.id: camel-k
spec:
  source: (1)
    ref:
      kind: InMemoryChannel
      apiVersion: messaging.knative.dev/v1
      name: my-messages
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink
1 Reference to the source Knative message channel that provides the events

Binding to Kafka topics

Kafka topic resources may act as a source or sink in a Pipe. Strimzi provides KafkaTopic resources that you can reference in your Pipe.

The reference to the Strimzi resource is identified by the apiVersion, kind and resource name. Users may add properties to the object reference such as the brokers bootstrap URI.

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
  annotations:
    camel.apache.org/operator.id: camel-k
spec:
  source: (1)
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: timer-source
    properties:
      message: "Camel rocks!"
  sink: (2)
    ref:
      kind: KafkaTopic
      apiVersion: kafka.strimzi.io/v1beta2
      name: my-topic
    properties:
      brokers: "my-cluster-kafka-bootstrap:9092" (3)
1 Reference to the source that provides data
2 Reference to the Strimzi Kafka topic where data should be sent to
3 The Kafka brokers bootstrap URI
Camel K is able to auto resolve the Kafka broker bootstrap URI by resolving the Strimzi Kafka resources in the same namespace. The operator may perform a lookup of the bootstrap URI and inject this as a property to the Camel component at runtime.

You can set the brokers property to explicitly point to the Strimzi Kafka broker.

Error handling

You can configure an error handler in order to specify what to do when some event ends up with failure. Pipes offer a mechanism to specify an error policy to adopt in case an event processing fails.

In case of an exception thrown during the pipe processing the respective error handler will perform its actions.

The Pipe knows different types of error handlers none, log and sink:

  • none → Explicit noErrorHandler is set and the error is ignored.

  • log → Errors get logged to the output.

  • sink → Errors get pushed to a specified endpoint in the form of dead letter queue.

The error handler may be configured with special properties that allow you to define the error handling behavior such as redelivery or delay policy.

No error handler

There may be certain cases where you want to just ignore any failure happening on your integration. In this situation just use a none error handler.

camel k bind my-pipe --source timer-source --sink log-sink --error-handler none

This results in following error handler configuration on the pipe:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
spec:
  source:
# ...
  sink:
# ...
  errorHandler:
    none: {}

Log error handler

Apache Camel offers a default behavior for handling failure: log to standard output. However, you can use the log error handler to specify other behaviors such as redelivery or delay policy.

camel k bind my-pipe --source timer-source --sink log-sink --error-handler log --property error-handler.maximumRedeliveries=3 --property error-handler.redeliveryDelay=2000

This results in the error handler configuration on the Pipe:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
spec:
  source:
# ...
  sink:
# ...
  errorHandler:
    log:
      parameters: (1)
        redeliveryDelay: 2000
        maximumRedeliveries: 3
1 Parameters belonging to the log error handler type

Sink error handler

The sink error handler is probably the most interesting error handler type as it allows you to redirect failing events to other components, such as a third party URI, a queue or topic or even another Kamelet which will be performing certain logic with the failing event.

The sink error handler expects a proper endpoint URI which may be a reference to another Kamelet, a fully qualified custom resource reference or an arbitrary Camel endpoint URI.

camel k bind my-pipe --source timer-source --sink log-sink --error-handler sink:my-error-handler --property error-handler.sink.message=ERROR! --property error-handler.maximumRedeliveries=1
apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
spec:
  source:
# ...
  sink:
# ...
  errorHandler:
    sink:
      endpoint:
        ref: (1)
          kind: Kamelet
          apiVersion: camel.apache.org/v1
          name: my-error-handler
        properties:
          message: "ERROR!" (2)
          # ...
      parameters:
        maximumRedeliveries: 1 (3)
        # ...
1 You can use ref or uri. ref will be interpreted by the operator according the kind, apiVersion and name. You can use any Kamelet, KafkaTopic channel or Knative destination.
2 Properties targeting the sink endpoint (in this example a property on the Kamelet named my-error-handler). Properties targeting the sink endpoint need to use the error-handler.sink.* prefix.
3 Parameters for the error handler (such as redelivery or delay policy). Error handler parameters need to use the error-handler.* prefix.
The error handler properties are divided into properties that target the error handler sink endpoint and properties that should be set on the Camel error handler component (e.g. maximumRedeliveries). You need to specify the respective property prefix (error-handler. or error-handler.sink.) to decide where the property should be set.

As an alternative to referencing a Kamelet as an error handler sink you may also use an arbitrary Camel endpoint URI.

camel k bind my-pipe --source timer-source --sink log-sink --error-handler sink:log:error --property error-handler.sink.showHeaders=true

It creates the error handler specification as follows:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: my-pipe
spec:
  source:
# ...
  sink:
# ...
  errorHandler:
    sink:
      endpoint:
        uri: log:error
        properties:
          showHeaders: true
Back to top