Design Notes for ThreadPool Configuration
CAMEL-1588 is the ticket for a new and improved thread pool configuration for Apache Camel. Its intended for Camel 2.3.
Camel uses thread pool in various places such as EIP patterns, Components, Async API and whatnot. The aim is to improved and allow easier to configure those thread pools in a more streamlined manner. The goal is to offer both a fine grained configuration where you can tweak individual pools and have more coarse grained configuration with fallback to global settings etc.
Some components provides their own thread pool configuration and management which Camel of course cannot and should not try to tailor with. For example Jetty is such an example.
Usages of thread pools in Camel
Currently Camel uses thread pools in camel-core in the following areas:
You can configure the thread pool using the setExecutorService setter methods that usually exists on those places where its in use. Some EIP patterns offer a
We should ensure all EIPs can be configured to use a custom thread pool in a nice and easy way. DONE
Using default ThreadPools
We should use
Only used SingleExecutorService for background tasks, and ScheduledExecutorService for scheduled tasks DONE
It should be possible to configure a thread pool on either per CamelContext level or per Route level, such as you can do with AutoStartup and the likes.
Thread pool configuration by rules
It should be possible to to define a set of rules which matches which thread pool a given source should use.
A ruleset something like this:
Where it will match against route first, so if we got a route3 then it will pick among those
Status: Consider for the future
Default thread pool profile
It should be possible to set a default
If none defined, then Camel should use a sensible default of
And it should validate that only one
The problem with shutdown and restarting pools
The ExecutorService API does not allow to restart a thread pool, which is PITA. So we need to find a better strategy for stopping vs. shutdown.
We may have to only shutdown thread pools if CamelContext is stopping. And then if end user stop a route from JMX we can keep the thread pool around.
We have introduced a
By letting Camel keep track of created thread pool, then Camel knows which pools shutdown when its stopping. Then the need for
The problem with Component, Endpoint
The DefaultComponent and DefaultEndpoint exposes API to get an ExecutorService. We should remove these API as you should use
Managed thread pool
Check whether the thread pools is managed by default and avail in JConsole. If not we should probably at least expose some read-only data about the pools. DONE
Spring Factory for creating custom pools
Create a Spring XML DSL for defining thread pools using custom options such as corePoolSize, maxPoolSize, keepAlive, thread name etc. DONE
Pluggable ExecutorService SPI
We need a
Customizable thread name
We should offer a simple pattern syntax so end users can customize the pattern the thread name is created with: eg Something like:
EIP should mandate an ExecutorService
If the EIPS which leverages a ExecutorService, mandates its being created and passed to it, we can enforce creating/lookup the pool during route creation, which allows us to have the route information as well, so we know which routes creates which pools. By passing in
Let Camel keep track of created pools
We should add configuration about rejection policies for new tasks submitted to a pool. The JDK has options for ABORT, RUN, WAIT, DISCARD etc. DONE