Telemetry

Since Camel 4.11

This module is a common interface and API for distributed tracing/telemetry. It is not intended to be used directly by end users. This component is implemented concretely in following dependencies:

  • camel-telemetry-dev

Tracing an application in a distributed deployment is an important feature supported by Camel. As there are plenty of potential telemetry implementation out there we have created an abstract component whose goal is to define the structure and lifecycle of a trace from Camel point of view. This component acts as a trace manager, making sure that regardless of the concrete technology used, the traces are consistently treated by Camel applications.

Using any of the concrete implementations above will instrument your Camel application with the tooling required to generate and expose traces to be collected from an external system (typically a trace collector via an agent to be run when executing the application). However, this is hidden to the abstract implementation and must be delegated to the concrete implementation instead.

Configuration

The configuration properties for the Telemetry component are:

Option Default Description

camel.component.telemetry.traceProcessors

false

Trace inner processors.

camel.component.telemetry.excludePatterns

Sets exclude pattern(s) that will disable tracing for Camel messages that matches the pattern. The content is a Set<String> where the key is a pattern. The pattern uses the rules from Intercept.

Tracing structure

There are two important concept we are using in the abstraction. Trace and Span. A Trace is the resource we use to track the execution of a Route. It starts when the Route is fired and is closed when the Route execution ends. A Span is the unit of execution of any event fired by the Route execution. A Trace typically contains one or more Span, depending on the complexity ot the Route.

In order to clarify these concepts, let’s use the following Route as a reference:

from("timer:mytimer")
    .routeId("timer")
    .log("in timer route")
    .to("direct:anotherRoute")
    .to("http://my-service");

For each event triggered, we expect the generation of traces with a Span for the timer component which contains 2 Spans, one for direct and the other for http endpoints. Mind that the log is not considered as a span as indeed is an execution event of the timer endpoint. It’s interesting to notice that the direct endpoint would contain as many Spans as its Route defines. All those spans are wrapped in a Trace and would look like as it follows:

timer
├── direct
│   ├── ...
└── http

The above model is likely to be mapped one to one to the same related standard model (which is, Traces and Spans) in any of the concrete technology adopted. If the technology has a different or richer mapping, then, it will be the implementation that has to take care to do the mapping accordingly.

Tracing lifecycle

The camel-telemetry component takes care to hook into the Camel application lifecycle and capture those meaningful events we want to turn into traces. The component capture the following events:

  • RoutePolicySupport.onExchangeBegin()

  • RoutePolicySupport.onExchangeDone()

  • CamelEvent.ExchangeSendingEvent

  • CamelEvent.ExchangeSentEvent

The first two are in charge to generate a new Trace (or a Span) when an Exchange begins and close it accordingly. The last two are in charge to create and close a Span for each of the events executed by the Route.

Additionally we are also capturing:

  • LogListener.onLog()

The LogListener.onLog() is a special case which we use in order to capture any logging trace and store it as an event into the component endpoint Span.

Main features

The abstract component provides some important features. Here the most important ones.

Span storage

Camel uses a proprietary storage mechanism. The trace is serialized into each generated Exchange and the component is taking care to maintain the hierarchy and the consistency of the Spans, regardless of the threading model which is executing the application logic. Each concrete implementation may provide its additional storing mechanism which should be ignored for the scope of Camel application tracing.

Component exclusions

The component provide the possibility to exclude the trace of any component when using the excludePatterns parameter. This feature is not implementation specific.

Component Span decoration

The component automatically includes certain useful parameter out of the box for the different components you may use within Camel. As an example, if you’re using the Kafka component (camel-kafka), then, it will include in the Kafka endpoint span a few useful information such as partition or offset which you will be able to verify later in the trace collector.

more information about each decorator in the telemetry component code repository.

Distributed Tracing

Distributed tracing are required to be correlated between each other. This is quite important above all when you’re running a microservice oriented architecture. When a Camel application calls another Camel applications, then, there must be in place a mechanism to correlate traces. This is done via context propagation.

The upstream application must inject the context into the event sent (typically a traceparent header in the Exchange). The downstream application must extract the context from the event received (same traceparent header). The result will be a unique distributed tracing with the same Trace ID.

This feature is implementation specific, the abstraction just provide the interface that must be implemented concretely in each of the implementation.

Processor tracing

When this feature is enabled, you will be able to collect a finer grain number of Spans into a Trace. Each of the different endpoint processors will be collected. You can enable the feature using the traceProcessors parameter (default false).

enabling this feature will provide many more Spans for each Trace.

Implementation specific abstraction

the following chapter is dedicate exclusively to developers willing to create a concrete implementation for this component.

In order to simplify the implementation of any tracing technology the abstraction provides the following method to implement:

    /*
     * It has to be provided by the specific implementation
     */
    private SpanLifecycleManager spanLifecycleManager;

    protected abstract void initTracer();

The initTracer() is in charge to inject a concrete implementation of SpanLifecycleManager whose abstraction is:

public interface SpanLifecycleManager {

    Span create(String spanName, Span parent, SpanContextPropagationExtractor extractor);

    void activate(Span span);

    void deactivate(Span span);

    void close(Span span);

    void inject(Span span, SpanContextPropagationInjector injector);

}
Back to top