Blog post featured image

Testing is probably one of those operations we use to repeat most of the time while building any application. Applications in Camel world are no difference. With the advent of Camel JBang, we have a unified place that can be used to perform our testing/fine tuning locally before moving to a higher environment.

During the last years of development, we have noticed that testing or fine tuning an integration directly connected to a Cloud Native environment can result a bit cumbersome. You need to be connected to the cluster, or alternatively you need a local Kubernetes cluster running on your machine (Minikube, Kind and the likes). Most of the time, the aspects inherent to the cluster fine tuning are coming very late in the development. Therefore, it would be nice to have a ligther way of testing our applications locally and, as soon as we’re happy with the result, move to a deployment stage where we can apply those tuning tipical of a cloud native environment.

kamel local is a command we have used to test an Integration locally in the past. However, it overlaps the effort done by Camel community to have a single CLI used to test locally any Camel application, independently where this is going to be deployed. For this reason we may deprecate it in the near future in favour of camel CLI provided by Camel JBang.

Camel JBang installation

First of all, we need to install and get familiar with jbang and camel CLIs. You can follow the official documentation about Camel JBang in order to install the CLIs to your local environment. Once this is done, we can see how to test an Integration for Camel K with Camel JBang.

Simple application development

The first application we develop is a simple one, but it defines the process you should follow when testing any Integration that will be eventually deployed in Kubernetes via Camel K. A good approach is to verify the target version of Camel in your Camel K installation. With this information we’ll make sure to test locally against the very same version that we’ll be later deploying in a cluster.

$ kamel version -a -v | grep Runtime
Runtime Version: 1.15.2

$ kubectl get camelcatalog camel-catalog-1.15.2 -o yaml | grep camel\.version
      camel.version: 3.18.3

The commands above are useful to find out what is the Camel version used by the runtime in my cluster Camel K installation. Our target is Camel version 3.18.3. Let’s keep this in mind.

The easiest way to initialize a Camel route is to run camel init command:

$ camel init HelloJBang.java

At this stage, we could edit the file with the logic we need for our integration, or, simply run it:

$ camel run HelloJBang.java
2022-11-23 12:11:05.407  INFO 52841 --- [           main] org.apache.camel.main.MainSupport        : Apache Camel (JBang) 3.18.1 is starting
2022-11-23 12:11:05.470  INFO 52841 --- [           main] org.apache.camel.main.MainSupport        : Using Java 11.0.17 with PID 52841. Started by squake in /home/squake/workspace/jbang/camel-blog
2022-11-23 12:11:07.537  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) is starting
2022-11-23 12:11:07.675  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext : Routes startup (started:1)
2022-11-23 12:11:07.676  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started java (timer://java)
2022-11-23 12:11:07.676  INFO 52841 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) started in 397ms (build:118ms init:140ms start:139ms JVM-uptime:3s)
2022-11-23 12:11:08.705  INFO 52841 --- [ - timer://java] HelloJBang.java:14                       : Hello Camel from java
2022-11-23 12:11:09.676  INFO 52841 --- [ - timer://java] HelloJBang.java:14                       : Hello Camel from java
...

A local java process will start with a Camel application running. No need to create a Maven project, all the boilerplate is on Camel JBang! However, you may notice that the Camel version used is different from the one we want to target. This is because my Camel JBang is using a different version of Camel. No worry, we can re-run this application but specifying the Camel version we want to run:

$ jbang run -Dcamel.jbang.version=3.18.3 camel@apache/camel run HelloJBang.java
...
[1] 2022-11-23 11:13:02,825 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.18.3 (camel-1) started in 70ms (build:0ms init:61ms start:9ms)
...

This is a workaround, as, at the moment, the camel CLI has not available that option (yet). This time, we have the correct version running. Let’s pretend we’re happy with the integration as is. The next step is to run it in a Kubernetes cluster where Camel K is installed.

$ kamel run HelloJBang.java --dev
Integration "hello-j-bang" created
...
[1] 2022-11-23 11:13:01,971 INFO  [org.apa.cam.k.Runtime] (main) Apache Camel K Runtime 1.15.2
[1] 2022-11-23 11:13:01,997 INFO  [org.apa.cam.qua.cor.CamelBootstrapRecorder] (main) Bootstrap runtime: org.apache.camel.quarkus.main.CamelMainRuntime
[1] 2022-11-23 11:13:02,000 INFO  [org.apa.cam.mai.MainSupport] (main) Apache Camel (Main) 3.18.3 is starting
[1] 2022-11-23 11:13:02,044 INFO  [org.apa.cam.k.lis.SourcesConfigurer] (main) Loading routes from: SourceDefinition{name='HelloJBang', language='java', type='source', location='file:/etc/camel/sources/HelloJBang.java', }
[1] 2022-11-23 11:13:02,816 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.18.3 (camel-1) is starting
[1] 2022-11-23 11:13:02,824 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Routes startup (started:1)
[1] 2022-11-23 11:13:02,825 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started java (timer://java)
[1] 2022-11-23 11:13:02,825 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.18.3 (camel-1) started in 70ms (build:0ms init:61ms start:9ms)
[1] 2022-11-23 11:13:02,828 INFO  [io.quarkus] (main) camel-k-integration 1.10.3 on JVM (powered by Quarkus 2.13.4.Final) started in 2.234s.
[1] 2022-11-23 11:13:02,829 INFO  [io.quarkus] (main) Profile prod activated.
[1] 2022-11-23 11:13:02,829 INFO  [io.quarkus] (main) Installed features: [camel-bean, camel-core, camel-java-joor-dsl, camel-k-core, camel-k-runtime, camel-kubernetes, camel-timer, cdi, kubernetes-client, security]
[1] 2022-11-23 11:13:03,850 INFO  [java] (Camel (camel-1) thread #1 - timer://java) Hello Camel from java

You will see that the Camel K operator will take care to do the necessary transformation and build the Integration and related resources according to the expected lifecycle. Once this is live, you will be able to follow up with the operations you usually do on a deployed Integration.

The benefit of this process is that you don’t need to worry about the remote cluster until you’re really happy with your Integration tuned locally. Said in other words, you will be able to fine tune your complex Integration while flying back to Europe from last ApacheCon in New Orleans…

Fine tuning for Cloud

Once landed and with your Integration ready to rock, you’ll need to take care about the kind of tuning related to cluster deployment. Having this separation of concerns is however positive, because you don’t need to worry on deployments details at an early stage of the development. Or you can even have a separation of roles in your company where the domain expert may develop the integration locally and the cluster expert may do the deployment at a later stage.

Let’s see some example about how to develop an integration that will later need some fine tuning in the cluster. This one is simple but it has certain aspects that will require attention at deployment stage:

import org.apache.camel.builder.RouteBuilder;

public class MyJBangRoute extends RouteBuilder {

    @Override
    public void configure() throws Exception {
        from("file:/tmp/input")
            .convertBodyTo(String.class)
            .log("Processing file ${headers.CamelFileName} with content: ${body}")
            /*
            .filter(simple("${body} !contains 'checked'"))
                .log("WARN not checked: ${body}")
                .to("file:/tmp/discarded")
            .end()
            .to("file:/tmp/output");
            */
            .choice()
                .when(simple("${body} !contains 'checked'"))
                    .log("WARN not checked!")
                    .to("file:/tmp/discarded")
                .otherwise()
                    .to("file:/tmp/output")
            .end();
    }
}

There is a process that is in charge to write files into a directory. I need to filter those files based on their content. I’ve left the code comments on purpose because it was the way I developed iteratively. I tested something locally with Camel JBang, until I come to the final version of the integration. I had tested the Filter EIP but while testing I realized I needed a Content Based Router EIP instead. It should sound a familiar process as it happens probably every time we develop something.

Now that I’m ready, I run a last round of testing locally via Camel JBang:

$ jbang run -Dcamel.jbang.version=3.18.3 camel@apache/camel run MyJBangRoute.java
2022-11-23 12:19:11.516  INFO 55909 --- [           main] org.apache.camel.main.MainSupport        : Apache Camel (JBang) 3.18.3 is starting
2022-11-23 12:19:11.592  INFO 55909 --- [           main] org.apache.camel.main.MainSupport        : Using Java 11.0.17 with PID 55909. Started by squake in /home/squake/workspace/jbang/camel-blog
2022-11-23 12:19:14.020  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.3 (CamelJBang) is starting
2022-11-23 12:19:14.220  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext : Routes startup (started:1)
2022-11-23 12:19:14.220  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started route1 (file:///tmp/input)
2022-11-23 12:19:14.220  INFO 55909 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.3 (CamelJBang) started in 677ms (build:133ms init:344ms start:200ms JVM-uptime:3s)
2022-11-23 12:19:27.757  INFO 55909 --- [le:///tmp/input] MyJBangRoute.java:11                     : Processing file file_1669202367381 with content: some entry

2022-11-23 12:19:27.758  INFO 55909 --- [le:///tmp/input] MyJBangRoute:21                          : WARN not checked!
2022-11-23 12:19:32.276  INFO 55909 --- [le:///tmp/input] MyJBangRoute.java:11                     : Processing file file_1669202372252 with content: some entry checked

I’ve tested adding files on the input directory and I am pretty fine with it. Ready to promote to my development cluster! As we’ve seen previously, this should be as easy as running the following command:

kamel run MyJBangRoute.java --dev

The Integration started correctly, but, wait a minute: we are using a file system that will be local to the Pod where the Integration is running, is it what we really need?

Kubernetes fine tuning

Probably it’s not what we wanted. So, now it’s turn to configure our application for the cloud. Cloud Native development should take in consideration a series of challenges that are implicit in the way how this new paradigm works (as a reference see the 12 factors).

Kubernetes could be sometimes a bit difficult to fine tune. Many resources to edit and check. Camel K provide a user friendly way to apply most of the tuning your application will need directly in the kamel run command (or in the modeline). Most of the time you will need to get familiar with Camel K Traits.

In this case we want to use certain volumes we had made available in our cluster. We can use the --volume option (syntactic sugar of mount trait) and enable them easily. We can read and write on those volumes from some other Pod: it depends on the architecture of our Integration process.

$ kamel run MyJBangRoute.java --volume my-pv-claim-input:/tmp/input --volume my-pv-claim-output:/tmp/output --volume my-pv-claim-discarded:/tmp/discarded --dev
...
[1] 2022-11-23 11:39:26,281 INFO  [route1] (Camel (camel-1) thread #1 - file:///tmp/input) Processing file file_1669203565971 with content: some entry
[1]
[1] 2022-11-23 11:39:26,303 INFO  [route1] (Camel (camel-1) thread #1 - file:///tmp/input) WARN not checked!
[1] 2022-11-23 11:39:32,322 INFO  [route1] (Camel (camel-1) thread #1 - file:///tmp/input) Processing file file_1669203571981 with content: some entry checked

Probably you will need to iterate this tuning as well, but at least, now that the internals of the route have been polished locally you should be able to focus on deployments aspects only. And, once you’re ready with this, have the benefit of kamel promote to move your Integration through various stages of development.

How to test Kamelet locally

Another great benefit of Camel JBang is the ability to test a Kamelet locally. Until now, the easiest possibility to test a Kamelet was to upload to a Kubernetes cluster and to run some Integration using it via Camel K. Not really a quick way to proceed.

Let’s develop a simple Kamelet for this scope. It’s a Beer source we’ll be using to generate random beers events:

apiVersion: camel.apache.org/v1alpha1
kind: Kamelet
metadata:
  name: beer-source
  labels:
    camel.apache.org/kamelet.type: "source"
spec:
  definition:
    title: "Beer Source"
    description: "Retrieve a random beer from catalog"
    properties:
      period:
        title: Period
        description: The interval between two events
        type: integer
        default: 1000
  types:
    out:
      mediaType: application/json
  flow:
    from:
      uri: timer:tick
      parameters:
        period: "#property:period"
      steps:
      - to: "https://random-data-api.com/api/beer/random_beer"
      - to: "kamelet:sink"

In order to test it, we can use it a very simple Integration to log its content:

- from:
    uri: "kamelet:beer-source?period=5000"
    steps:
      - log: "${body}"

Now we can run:

$ camel run --local-kamelet-dir=</path/to/local/kamelets/dir> beer-integration.yaml
2022-11-24 11:27:29.634  INFO 39527 --- [           main] org.apache.camel.main.MainSupport        : Apache Camel (JBang) 3.18.1 is starting
2022-11-24 11:27:29.706  INFO 39527 --- [           main] org.apache.camel.main.MainSupport        : Using Java 11.0.17 with PID 39527. Started by squake in /home/squake/workspace/jbang/camel-blog
2022-11-24 11:27:31.391  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) is starting
2022-11-24 11:27:31.590  INFO 39527 --- [           main] org.apache.camel.main.BaseMainSupport    : Property-placeholders summary
2022-11-24 11:27:31.590  INFO 39527 --- [           main] org.apache.camel.main.BaseMainSupport    :     [beer-source.kamelet.yaml]     period=5000
2022-11-24 11:27:31.590  INFO 39527 --- [           main] org.apache.camel.main.BaseMainSupport    :     [beer-source.kamelet.yaml]     templateId=beer-source
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext : Routes startup (started:2)
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started route1 (kamelet://beer-source)
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext :     Started beer-source-1 (timer://beer)
2022-11-24 11:27:31.591  INFO 39527 --- [           main] e.camel.impl.engine.AbstractCamelContext : Apache Camel 3.18.1 (CamelJBang) started in 1s143ms (build:125ms init:819ms start:199ms JVM-uptime:2s)
2022-11-24 11:27:33.297  INFO 39527 --- [ - timer://beer] beer-integration.yaml:4                  : {"id":3975,"uid":"2df52cb0-ff5a-4c51-b33e-a2c3a65caac5","brand":"Birra Moretti","name":"St. Bernardus Abt 12","style":"English Pale Ale","hop":"Columbus","yeast":"5526 - Brettanomyces lambicus","malts":"Roasted barley","ibu":"41 IBU","alcohol":"2.1%","blg":"15.0°Blg"}

This is really a boost while you are programming a Kamelet, because you can have a very quick feedback without the need of a cluster. Once ready, you will be able to continue your development as usual uploading the Kamelet to the cluster and using in your Camel K integrations.

Conclusion

With this blog post I have tried to provide some guidelines when it comes to the question: “How to test an Integration for Camel K?”. There is not always a direct answer as it may depends on many factors. However, we see that the usage of Camel JBang is emerging as a way to support Camel K as well. In particular we’ve seen that separating the test of the Integration logic from the test of the “deployment” logic, may result beneficial as we’ll be focusing on a single concern at a time.

Camel K CICD

 , by

In Camel K version 10, we’ve released the CLI `promote feature that provides Camel K an opinionated way of promoting an Integration through the stages of software development. This feature unlock the possibility to combine Camel K with external tooling and let the user develop according to any automated release process. We always ear about CI/CD (Continuous Integration/Continuous Delivery and/or Deployment), and in this blog we’re going to see how to make it for any Camel K integration.

Continue reading ❯

HOWTOSCAMEL KDEVOPS

Camel K 1.10 release

 , by

It’s been a long ride to bring our beloved Camel (K) from 1.9.0 to 1.10.0. We’re particularly happy as we had the time to introduce many features and enhancements that we had planned in our 2022 roadmap back at the beginning of the year. As usual let’s start by acknowledging the tech stack on top of Camel K and the new exciting versions our fellows have baked: Apache Camel K Runtime 1.

Continue reading ❯

RELEASESCAMEL K

Camel K Operations: monitoring

 , by

Camel K offers a wide list of operations you can execute once your Integration has been deployed (likely in a production environment). When we talk about operations, the most typical question we got is “How to monitor a Camel K Integration?”. Fortunately, we have all the ingredients needed to let you manage this operation as smooth as possible. I’ll walk you through the different tools and configuration needed in this blog post.

Continue reading ❯

CAMEL KFEATURES

Camel K 1.9 release

 , by

Attention, attention: time for a new exciting Camel K release! On behalf of Camel K community I am happy to announce the latest 1.9.0. Camel K release. As usual let’s use this blog to share with you the new stack of exciting technology we’ll be using in this version: Apache Camel K Runtime 1.13.0 Apache Camel Quarkus 2.8.0 Apache Camel 3.16.0 Apache Camel Kamelets 0.8.0 Thanks to Apache Camel, Camel Quarkus and Kamelet Catalog contributors for the great efforts they’ve put in those new releases as well.

Continue reading ❯

RELEASESCAMEL K

Camel K 2022 Roadmap

 , by

Signpost png from pngtree.com/ During the last weeks we’ve been asked questions around the direction we’re willing to take on the future development of Camel K. I think it would be good to have some blog in order to let the community understand where our efforts are going. It will be useful for every Camel K user and Camel K developer, as a guide for the future development of the project.

Continue reading ❯

ROADMAPCAMEL K