Camel supports a pluggable shutdown strategy using
org.apache.camel.spi.ShutdownStrategy. Its responsible for shutting down routes in a graceful manner. The other resources will still be handled by CamelContext to shutdown. This leaves the problem at hand with properly shutting down all the routes reliably to the
Camel provides a default strategy in the
org.apache.camel.impl.engine.DefaultShutdownStrategy which is capable of doing that.
Default Shutdown Strategy
The default strategy (
DefaultShutdownStrategy) will gracefully shutdown routes:
in the reverse order they were started
let pending and current in flight exchanges run to completion before shutting down
using a timeout of 45 seconds which then forces a hard shutdown
The settings are configurable, so you can change the timeout, and whether in case of timeout to force a hard shutdown or ignore.
When shutting down the strategy will output to log the progress:
2009-12-20 10:56:53,055 [main ] INFO DefaultCamelContext - Apache Camel (CamelContext:camel-1) is stopping 2009-12-20 10:56:53,056 [main ] INFO DefaultShutdownStrategy - Starting to graceful shutdown routes (timeout 45 seconds) 2009-12-20 10:56:53,059 [1: ShutdownTask] INFO DefaultShutdownStrategy - Waiting as there are still 5 inflight exchanges to complete before we can shutdown 2009-12-20 10:56:54,060 [1: ShutdownTask] INFO DefaultShutdownStrategy - Waiting as there are still 4 inflight exchanges to complete before we can shutdown 2009-12-20 10:56:55,061 [1: ShutdownTask] INFO DefaultShutdownStrategy - Waiting as there are still 3 inflight exchanges to complete before we can shutdown 2009-12-20 10:56:56,065 [1: ShutdownTask] INFO DefaultShutdownStrategy - Waiting as there are still 2 inflight exchanges to complete before we can shutdown 2009-12-20 10:56:57,066 [1: ShutdownTask] INFO DefaultShutdownStrategy - Waiting as there are still 1 inflight exchanges to complete before we can shutdown 2009-12-20 10:56:58,069 [main ] INFO DefaultShutdownStrategy - Graceful shutdown of routes complete in 5 seconds. 2009-12-20 10:56:58,072 [main ] INFO DefaultInflightRepository - Shutting down with no inflight exchanges. 2009-12-20 10:56:58,077 [main ] INFO DefaultCamelContext - Apache Camel (CamelContext:camel-1) stopped
Notice how the strategy waits while there are inflight exchanges still being processed before it shutdown the routes and after that Camel itself.
Suppressing logging due to timeout not allowing all inflight messages to complete
If a graceful shutdown could not shutdown cleanly within the given timeout period, then Camel performs a more aggressive shutdown (hard) by forcing routes and thread pools etc to shut down, and the routing engine will reject continue processing Exchanges. If this happens you may see WARN logs about Exchanges being rejected and other failures due the forced shutdown.
If you do not want to see these logs, you can suppress this by setting the option suppressLoggingOnTimeout to true.
Notice that suppression is a best effort attempt and there may still be some logging from 3rd party libraries, which Camel cannot control.
Logging inflight exchange information on timeout
If a graceful shutdown could not shutdown cleanly within the given timeout period, then Camel performs a more aggressive shutdown by forcing routes and thread pools etc to shut down. When the timeout happens, then Camel logs information about the current inflight exchanges, which shows from which route the exchange origins, and where it currently is being routed. For example the logging below, shows that there is 1 inflight exchange, that origins from route1, and currently is still in route1 at the "delay1" node. The elapsed is time in millis how long at the current node (eg delay1) and duration is total time in mills.
If you enable DEBUG logging level on
org.apache.camel.impl.engine.DefaultShutdownStrategy then it logs the same inflight exchange information during graceful shutdown
2015-01-12 13:23:23,656 [ - ShutdownTask] INFO DefaultShutdownStrategy - There are 1 inflight exchanges: InflightExchange: [exchangeId=ID-davsclaus-air-62213-1421065401253-0-3, fromRouteId=route1, routeId=route1, nodeId=delay1, elapsed=2007, duration=2017]
If you do not want to see these logs, you can turn this off by setting the option logInflightExchangesOnTimeout to false.
Controlling ordering of routes
You can configure the order in which routes should be started, and thus also the same order they are being shutdown. See more at Configuring route startup ordering and autostartup.
Fine grained configuration
You can control two areas that influence graceful shutdown in the Camel routing:
These options can be configured on two levels: context and route. The route level take precedence over context, meaning that if not explicit configured on the route level, then the context level is used.
This option can control how a given route should act during graceful shutdown. It has two values
Default is obviously the default option which lets Camel shutdown the route as early as possible. The
Defer is used to defer shutting down this route to a later stage. This is useful when other routes are dependent upon it. For example an internal route which other routes reuse.
|It is best to only defer shutting down internal routes only. Public routes should shut down as quickly as possible otherwise it will just keep intake new messages which will delay the shutdown processor. Or even have it timeout if a lot of new messages keep coming in.|
This option control how a given route consumer acts during shutdown. Most route consumer will only operate on a single task (message), however the Batch Consumer can operate on many messages (in a batch). This option is for batch consumers.
The default value is
CompleteCurrentTaskOnly which mean that the current in progress task (message) will be completed and then the consumer will shut down. The other option
CompleteAllTasks allows the consumer to complete all the tasks (messages) before shutting down. For example a File consumer will process all the pending files it has picked up before shutting down.
Stop individual routes
It is possible to stop (will do a gracefully shut down) an individual route using
stopRoute(routeId) method as shown:
Routes can also be stopped via JMX.
Stopping and marking routes as failed due to an exception
It is possible to stop and fail (will do a gracefully shut down) an individual route using
stopRoute(routeId, cause) method as shown:
Exception cause = ... camelContext.getRouteController().stopRoute(routeId, cause);
This will stop the route and then mark the route as failed with the caused exception.
|The Camel Health Check detect the route as failed and report it as DOWN. If the route is manually stopped, then the route is not marked as failed, and the Health Check will report the status as UNKNOWN.|
Routes can also be stopped and failed via JMX.
Implementing custom component or ShutdownStrategy
If you develop your own Camel component or want to implement your own shutdown strategy then read this section for details.
You can implement your own strategy to control the shutdown by implementing the
org.apache.camel.spi.ShutdownStrategy and the set it on the
CamelContext using the
org.apache.camel.spi.ShutdownAware is an optional interface consumers can implement to have fine grained control during shutdown. The
ShutdownStrategy must be able to deal with consumers which implement this interface. This interface was introduced to cater for in memory consumers such as SEDA which potentially have a number of pending messages on its internal in memory queues. What this allows is to let it control the shutdown process to let it complete its pending messages.
getPendingExchangesSize should return the number of pending messages which reside on the in memory queues. The method
deferShutdown should return
true to defer the shutdown to a later stage, when there are no more pending and inflight messages.
Batch Consumer should implement
ShutdownAware so they properly support the
ShutdownRunningTask option. See
GenericFileConsumer for an example.