Data Format AppendixData FormatCamel supports a pluggable DataFormat to allow messages to be marshalled to and from binary or text formats to support a kind of Message Translator. The following data formats are currently supported:
And related is the following Type Converters: UnmarshallingIf you receive a message from one of the Camel Components such as File, HTTP or JMS you often want to unmarshal the payload into some bean so that you can process it using some Bean Integration or perform Predicate evaluation and so forth. To do this use the unmarshal word in the DSL in Java or the Xml Configuration. For example DataFormat jaxb = new JaxbDataFormat("com.acme.model"); from("activemq:My.Queue"). unmarshal(jaxb). to("mqseries:Another.Queue"); The above uses a named DataFormat of jaxb which is configured with a number of Java package names. You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. You can also use the DSL itself to define the data format as you use it. For example the following uses Java serialization to unmarshal a binary file then send it as an ObjectMessage to ActiveMQ from("file://foo/bar"). unmarshal().serialization(). to("activemq:Some.Queue"); MarshallingMarshalling is the opposite of unmarshalling, where a bean is marshalled into some binary or textual format for transmission over some transport via a Camel Component. Marshalling is used in the same way as unmarshalling above; in the DSL you can use a DataFormat instance, you can configure the DataFormat dynamically using the DSL or you can refer to a named instance of the format in the Registry. The following example unmarshals via serialization then marshals using a named JAXB data format to perform a kind of Message Translator from("file://foo/bar"). unmarshal().serialization(). marshal("jaxb"). to("activemq:Some.Queue"); Using Spring XMLThis example shows how to configure the data type just once and reuse it on multiple routes <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <jaxb id="myJaxb" prettyPrint="true" contextPath="org.apache.camel.example"/> </dataFormats> <route> <from uri="direct:start"/> <marshal ref="myJaxb"/> <to uri="direct:marshalled"/> </route> <route> <from uri="direct:marshalled"/> <unmarshal ref="myJaxb"/> <to uri="mock:result"/> </route> </camelContext> You can also define reusable data formats as Spring beans <bean id="myJaxb" class="org.apache.camel.model.dataformat.JaxbDataFormat"> <property name="prettyPrint" value="true"/> <property name="contextPath" value="org.apache.camel.example"/> </bean> SerializationSerialization is a Data Format which uses the standard Java Serialization mechanism to unmarshal a binary payload into Java objects or to marshal Java objects into a binary blob. from("file://foo/bar"). unmarshal().serialization(). to("activemq:Some.Queue"); DependenciesThis data format is provided in camel-core so no additional dependencies is needed. JAXBJAXB is a Data Format which uses the JAXB2 XML marshalling standard which is included in Java 6 to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload. Using the Java DSLFor example the following uses a named DataFormat of jaxb which is configured with a number of Java package names to initialize the JAXBContext. DataFormat jaxb = new JaxbDataFormat("com.acme.model"); from("activemq:My.Queue"). unmarshal(jaxb). to("mqseries:Another.Queue"); You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g. from("activemq:My.Queue"). unmarshal("myJaxbDataType"). to("mqseries:Another.Queue"); Using Spring XMLThe following example shows how to use JAXB to unmarshal using Spring configuring the jaxb data type <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <unmarshal> <jaxb prettyPrint="true" contextPath="org.apache.camel.example"/> </unmarshal> <to uri="mock:result"/> </route> </camelContext> This example shows how to configure the data type just once and reuse it on multiple routes. <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <jaxb id="myJaxb" prettyPrint="true" contextPath="org.apache.camel.example"/> </dataFormats> <route> <from uri="direct:start"/> <marshal ref="myJaxb"/> <to uri="direct:marshalled"/> </route> <route> <from uri="direct:marshalled"/> <unmarshal ref="myJaxb"/> <to uri="mock:result"/> </route> </camelContext>
Partial marshalling/unmarshallingThis feature is new to Camel 2.2.0. <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:marshal"/> <marshal> <jaxb prettyPrint="false" contextPath="org.apache.camel.example" partClass="org.apache.camel.example.PurchaseOrder" fragment="true" partNamespace="{http://example.camel.org/apache}po" /> </marshal> <to uri="mock:marshal"/> </route> <route> <from uri="direct:unmarshal"/> <unmarshal> <jaxb prettyPrint="false" contextPath="org.apache.camel.example" partClass="org.apache.camel.example.Partial" /> </unmarshal> <to uri="mock:unmarshal"/> </route> </camelContext> For marshalling you have to add partNamespace attribute with QName of destination namespace. Example of Spring DSL you can find above. FragmentThis feature is new to Camel 2.8.0. Ignoring the NonXML CharacterThis feature is new to Camel 2.2.0.
This feature has been tested with Woodstox 3.2.9 and Sun JDK 1.6 StAX implementation. Working with the ObjectFactoryIf you use XJC to create the java class from the schema, you will get an ObjectFactory for you JAXB context. Since the ObjectFactory uses JAXBElement to hold the reference of the schema and element instance value, jaxbDataformat will ignore the JAXBElement by default and you will get the element instance value instead of the JAXBElement object form the unmarshaled message body. Setting encodingYou can set the encoding option to use when marshalling. Its the Marshaller.JAXB_ENCODING encoding property on the JAXB Marshaller. In this Spring DSL we have defined to use iso-8859-1 as the encoding: <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <marshal> <jaxb prettyPrint="false" encoding="iso-8859-1" contextPath="org.apache.camel.example"/> </marshal> <to uri="mock:result"/> </route> </camelContext> Controlling namespace prefix mappingAvailable as of Camel 2.11 When marshalling using JAXB or SOAP then the JAXB implementation will automatic assign namespace prefixes, such as ns2, ns3, ns4 etc. To control this mapping, Camel allows you to refer to a map which contains the desired mapping. Notice this requires having JAXB-RI 2.1 or better (from SUN) on the classpath, as the mapping functionality is dependent on the implementation of JAXB, whether its supported. For example in Spring XML we can define a Map with the mapping. In the mapping file below, we map SOAP to use soap as prefix. While our custom namespace "http://www.mycompany.com/foo/2" is not using any prefix. <util:map id="myMap"> <entry key="http://www.w3.org/2003/05/soap-envelope" value="soap"/> <!-- we dont want any prefix for our namespace --> <entry key="http://www.mycompany.com/foo/2" value=""/> </util:map> To use this in JAXB or SOAP you refer to this map, using the namespacePrefixRef attribute as shown below. Then Camel will lookup in the Registry a java.util.Map with the id "myMap", which was what we defined above. <marshal> <soapjaxb version="1.2" contextPath="com.mycompany.foo" namespacePrefixRef="myMap"/> </marshal> Schema validationAvailable as of Camel 2.11 The JAXB Data Format supports validation by marshalling and unmarshalling from/to XML. Your can use the prefix classpath:, file:* or *http: to specify how the resource should by resolved. You can separate multiple schema files by using the ',' character. Using the Java DSL, you can configure it in the following way: JaxbDataFormat jaxbDataFormat = new JaxbDataFormat(); jaxbDataFormat.setContextPath(Person.class.getPackage().getName()); jaxbDataFormat.setSchema("classpath:person.xsd,classpath:address.xsd"); You can do the same using the XML DSL: <marshal> <jaxb id="jaxb" schema="classpath:person.xsd,classpath:address.xsd"/> </marshal> DependenciesTo use JAXB in your camel routes you need to add the a dependency on camel-jaxb which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-jaxb</artifactId> <version>x.x.x</version> </dependency> XmlBeansXmlBeans is a Data Format which uses the XmlBeans library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload. from("activemq:My.Queue"). unmarshal().xmlBeans(). to("mqseries:Another.Queue"); DependenciesTo use XmlBeans in your camel routes you need to add the dependency on camel-xmlbeans which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xmlbeans</artifactId> <version>x.x.x</version> <!-- use the same version as your Camel core version --> </dependency> XStreamXStream is a Data Format which uses the XStream library to marshal and unmarshal Java objects to and from XML. // lets turn Object messages into XML then send to MQSeries from("activemq:My.Queue"). marshal().xstream(). to("mqseries:Another.Queue"); XMLInputFactory and XMLOutputFactoryThe XStream library uses the javax.xml.stream.XMLInputFactory and javax.xml.stream.XMLOutputFactory, you can control which implementation of this factory should be used. The Factory is discovered using this algorithm: How to set the XML encoding in Xstream DataFormat?From Camel 2.2.0, you can set the encoding of XML in Xstream DataFormat by setting the Exchange's property with the key Exchange.CHARSET_NAME, or setting the encoding property on Xstream from DSL or Spring config. from("activemq:My.Queue"). marshal().xstream("UTF-8"). to("mqseries:Another.Queue"); <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <!-- we define the json xstream data formats to be used (xstream is default) --> <dataFormats> <xstream id="xstream-utf8" encoding="UTF-8"/> <xstream id="xstream-default"/> </dataFormats> <route> <from uri="direct:in"/> <marshal ref="xstream-default"/> <to uri="mock:result"/> </route> <route> <from uri="direct:in-UTF-8"/> <marshal ref="xstream-utf8"/> <to uri="mock:result"/> </route> </camelContext> DependenciesTo use XStream in your camel routes you need to add the a dependency on camel-xstream which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xstream</artifactId> <version>x.x.x</version> </dependency> CSVThe CSV Data Format uses Apache Commons CSV to handle CSV payloads (Comma Separated Values) such as those exported/imported by Excel. Options
Marshalling a Map to CSVThe component allows you to marshal a Java Map (or any other message type that can be converted in a Map) into a CSV payload. An example: if you send a message with this map... Map<String, Object> body = new HashMap<String, Object>(); body.put("foo", "abc"); body.put("bar", 123); ... through this route ... from("direct:start"). marshal().csv(). to("mock:result"); ... you will end up with a String containing this CSV message abc,123 Sending the Map below through this route will result in a CSV message that looks like foo,bar Unmarshalling a CSV message into a Java ListUnmarshalling will transform a CSV messsage into a Java List with CSV file lines (containing another List with all the field values). An example: we have a CSV file with names of persons, their IQ and their current activity. Jack Dalton, 115, mad at Averell
Joe Dalton, 105, calming Joe
William Dalton, 105, keeping Joe from killing Averell
Averell Dalton, 80, playing with Rantanplan
Lucky Luke, 120, capturing the Daltons
We can now use the CSV component to unmarshal this file: from("file:src/test/resources/?fileName=daltons.csv&noop=true"). unmarshal().csv(). to("mock:daltons"); The resulting message will contain a List<List<String>> like... List<List<String>> data = (List<List<String>>) exchange.getIn().getBody(); for (List<String> line : data) { LOG.debug(String.format("%s has an IQ of %s and is currently %s", line.get(0), line.get(1), line.get(2))); } Marshalling a List<Map> to CSVAvailable as of Camel 2.1 If you have multiple rows of data you want to be marshalled into CSV format you can now store the message payload as a List<Map<String, Object>> object where the list contains a Map for each row. File Poller of CSV, then unmarshalingGiven a bean which can handle the incoming data... MyCsvHandler.java // Some comments here public void doHandleCsvData(List<List<String>> csvData) { // do magic here } ... your route then looks as follows <route> <!-- poll every 10 seconds --> <from uri="file:///some/path/to/pickup/csvfiles?delete=true&consumer.delay=10000" /> <unmarshal><csv /></unmarshal> <to uri="bean:myCsvHandler?method=doHandleCsvData" /> </route> Marshaling with a pipe as delimiterUsing the Spring/XML DSL: <route> <from uri="direct:start" /> <marshal> <csv delimiter="|" /> </marshal> <to uri="bean:myCsvHandler?method=doHandleCsv" /> </route> Or the Java DSL: CsvDataFormat csv = new CsvDataFormat(); CSVConfig config = new CSVConfig(); config.setDelimiter('|'); csv.setConfig(config); from("direct:start") .marshal(csv) .convertBodyTo(String.class) .to("bean:myCsvHandler?method=doHandleCsv"); CsvDataFormat csv = new CsvDataFormat(); csv.setDelimiter("|"); from("direct:start") .marshal(csv) .convertBodyTo(String.class) .to("bean:myCsvHandler?method=doHandleCsv"); Using autogenColumns, configRef and strategyRef attributes inside XML DSLAvailable as of Camel 2.9.2 / 2.10 You can customize the CSV Data Format to make use of your own CSVConfig and/or CSVStrategy. Also note that the default value of the autogenColumns option is true. The following example should illustrate this customization. <route> <from uri="direct:start" /> <marshal> <!-- make use of a strategy other than the default one which is 'org.apache.commons.csv.CSVStrategy.DEFAULT_STRATEGY' --> <csv autogenColumns="false" delimiter="|" configRef="csvConfig" strategyRef="excelStrategy" /> </marshal> <convertBodyTo type="java.lang.String" /> <to uri="mock:result" /> </route> <bean id="csvConfig" class="org.apache.commons.csv.writer.CSVConfig"> <property name="fields"> <list> <bean class="org.apache.commons.csv.writer.CSVField"> <property name="name" value="orderId" /> </bean> <bean class="org.apache.commons.csv.writer.CSVField"> <property name="name" value="amount" /> </bean> </list> </property> </bean> <bean id="excelStrategy" class="org.springframework.beans.factory.config.FieldRetrievingFactoryBean"> <property name="staticField" value="org.apache.commons.csv.CSVStrategy.EXCEL_STRATEGY" /> </bean> Using skipFirstLine option while unmarshalingAvailable as of Camel 2.10 You can instruct the CSV Data Format to skip the first line which contains the CSV headers. Using the Spring/XML DSL: <route> <from uri="direct:start" /> <unmarshal> <csv skipFirstLine="true" /> </unmarshal> <to uri="bean:myCsvHandler?method=doHandleCsv" /> </route> Or the Java DSL: CsvDataFormat csv = new CsvDataFormat(); csv.setSkipFirstLine(true); from("direct:start") .unmarshal(csv) .to("bean:myCsvHandler?method=doHandleCsv"); Unmarshaling with a pipe as delimiterUsing the Spring/XML DSL: <route> <from uri="direct:start" /> <unmarshal> <csv delimiter="|" /> </unmarshal> <to uri="bean:myCsvHandler?method=doHandleCsv" /> </route> Or the Java DSL: CsvDataFormat csv = new CsvDataFormat(); CSVStrategy strategy = CSVStrategy.DEFAULT_STRATEGY; strategy.setDelimiter('|'); csv.setStrategy(strategy); from("direct:start") .unmarshal(csv) .to("bean:myCsvHandler?method=doHandleCsv"); CsvDataFormat csv = new CsvDataFormat(); csv.setDelimiter("|"); from("direct:start") .unmarshal(csv) .to("bean:myCsvHandler?method=doHandleCsv"); CsvDataFormat csv = new CsvDataFormat(); CSVConfig csvConfig = new CSVConfig(); csvConfig.setDelimiter(";"); csv.setConfig(csvConfig); from("direct:start") .unmarshal(csv) .to("bean:myCsvHandler?method=doHandleCsv");
DependenciesTo use CSV in your Camel routes you need to add a dependency on camel-csv, which implements this data format. If you use Maven you can just add the following to your pom.xml, substituting the version number for the latest and greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-csv</artifactId> <version>x.x.x</version> </dependency> The String Data Format is a textual based format that supports encoding. Options
MarshalIn this example we marshal the file content to String object in UTF-8 encoding.
from("file://data.csv").marshal().string("UTF-8").to("jms://myqueue");
UnmarshalIn this example we unmarshal the payload from the JMS queue to a String object using UTF-8 encoding, before its processed by the newOrder processor.
from("jms://queue/order").unmarshal().string("UTF-8").processRef("newOrder");
DependenciesThis data format is provided in camel-core so no additional dependencies is needed. HL7 DataFormatThe HL7 component ships with a HL7 data format that can be used to format between String and HL7 model objects.
To use the data format, simply instantiate an instance and invoke the marshal or unmarshal operation in the route builder: DataFormat hl7 = new HL7DataFormat(); ... from("direct:hl7in").marshal(hl7).to("jms:queue:hl7out"); In the sample above, the HL7 is marshalled from a HAPI Message object to a byte stream and put on a JMS queue. DataFormat hl7 = new HL7DataFormat(); ... from("jms:queue:hl7out").unmarshal(hl7).to("patientLookupService"); Here we unmarshal the byte stream into a HAPI Message object that is passed to our patient lookup service.
Notice there is a shorthand syntax in Camel for well-known data formats that is commonly used. from("direct:hl7in").marshal().hl7().to("jms:queue:hl7out"); from("jms:queue:hl7out").unmarshal().hl7().to("patientLookupService");
EDI DataFormatWe encourage end users to look at the Smooks which supports EDI and Camel natively. Flatpack DataFormatThe Flatpack component ships with the Flatpack data format that can be used to format between fixed width or delimited text messages to a List of rows as Map.
Notice: The Flatpack library does currently not support header and trailers for the marshal operation. OptionsThe data format has the following options:
UsageTo use the data format, simply instantiate an instance and invoke the marshal or unmarshal operation in the route builder: FlatpackDataFormat fp = new FlatpackDataFormat(); fp.setDefinition(new ClassPathResource("INVENTORY-Delimited.pzmap.xml")); ... from("file:order/in").unmarshal(df).to("seda:queue:neworder"); The sample above will read files from the order/in folder and unmarshal the input using the Flatpack configuration file INVENTORY-Delimited.pzmap.xml that configures the structure of the files. The result is a DataSetList object we store on the SEDA queue. FlatpackDataFormat df = new FlatpackDataFormat(); df.setDefinition(new ClassPathResource("PEOPLE-FixedLength.pzmap.xml")); df.setFixed(true); df.setIgnoreFirstRecord(false); from("seda:people").marshal(df).convertBodyTo(String.class).to("jms:queue:people"); In the code above we marshal the data from a Object representation as a List of rows as Maps. The rows as Map contains the column name as the key, and the the corresponding value. This structure can be created in Java code from e.g. a processor. We marshal the data according to the Flatpack format and convert the result as a String object and store it on a JMS queue. DependenciesTo use Flatpack in your camel routes you need to add the a dependency on camel-flatpack which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-flatpack</artifactId> <version>x.x.x</version> </dependency> JSONJSON is a Data Format to marshal and unmarshal Java objects to and from JSON. For JSON to object marshalling, Camel provides integration with three popular JSON libraries:
By default Camel uses the XStream library.
Using JSON data format with the XStream library// lets turn Object messages into json then send to MQSeries from("activemq:My.Queue"). marshal().json(). to("mqseries:Another.Queue"); Using JSON data format with the Jackson library// lets turn Object messages into json then send to MQSeries from("activemq:My.Queue"). marshal().json(JsonLibrary.Jackson). to("mqseries:Another.Queue"); Using JSON data format with the GSON library// lets turn Object messages into json then send to MQSeries from("activemq:My.Queue"). marshal().json(JsonLibrary.Gson). to("mqseries:Another.Queue"); Using JSON in Spring DSLWhen using Data Format in Spring DSL you need to declare the data formats first. This is done in the DataFormats XML tag.
<dataFormats>
<!-- here we define a Json data format with the id jack and that it should use the TestPojo as the class type when
doing unmarshal. The unmarshalTypeName is optional, if not provided Camel will use a Map as the type -->
<json id="jack" library="Jackson" unmarshalTypeName="org.apache.camel.component.jackson.TestPojo"/>
</dataFormats>
And then you can refer to this id in the route:
<route>
<from uri="direct:back"/>
<unmarshal ref="jack"/>
<to uri="mock:reverse"/>
</route>
Excluding POJO fields from marshallingAs of Camel 2.10 public class Views { static class Weight { } static class Age { } } Use the marker classes with the @JsonView annotation to include/exclude certain fields. The annotation also works on getters. @JsonView(Views.Age.class) private int age = 30; private int height = 190; @JsonView(Views.Weight.class) private int weight = 70; Finally use the Camel JacksonDataFormat to marshall the above POJO to JSON. JacksonDataFormat ageViewFormat = new JacksonDataFormat(TestPojoView.class, Views.Age.class); from("direct:inPojoAgeView").marshal(ageViewFormat); Note that the weight field is missing in the resulting JSON:
{"age":30, "height":190}
The GSON library supports a similar feature through the notion of ExclusionStrategies: /**
* Strategy to exclude {@link ExcludeAge} annotated fields
*/
protected static class AgeExclusionStrategy implements ExclusionStrategy {
@Override
public boolean shouldSkipField(FieldAttributes f) {
return f.getAnnotation(ExcludeAge.class) != null;
}
@Override
public boolean shouldSkipClass(Class<?> clazz) {
return false;
}
}
The GsonDataFormat accepts an ExclusionStrategy in its constructor: GsonDataFormat ageExclusionFormat = new GsonDataFormat(TestPojoExclusion.class, new AgeExclusionStrategy()); from("direct:inPojoExcludeAge").marshal(ageExclusionFormat); The line above will exclude fields annotated with @ExcludeAge when marshalling to JSON. Configuring field naming policyAvailable as of Camel 2.11 The GSON library supports specifying policies and strategies for mapping from json to POJO fields. A common naming convention is to map json fields using lower case with underscores. We may have this JSON string
{
"id" : 123,
"first_name" : "Donald"
"last_name" : "Duck"
}
Which we want to map to a POJO that has getter/setters as public class PersonPojo { private int id; private String firstName; private String lastName; public int getId() { return id; } public void setId(int id) { this.id = id; } public String getFirstName() { return firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return lastName; } public void setLastName(String lastName) { this.lastName = lastName; } } Then we can configure the org.apache.camel.component.gson.GsonDataFormat in a Spring XML files as shown below. Notice we use fieldNamingPolicy property to set the field mapping. This property is an enum from GSon com.google.gson.FieldNamingPolicy which has a number of pre defined mappings. If you need full control you can use the property FieldNamingStrategy and implement a custom com.google.gson.FieldNamingStrategy where you can control the mapping.
<!-- define the gson data format, where we configure the data format using the properties -->
<bean id="gson" class="org.apache.camel.component.gson.GsonDataFormat">
<!-- we want to unmarshal to person pojo -->
<property name="unmarshalType" value="org.apache.camel.component.gson.PersonPojo"/>
<!-- we want to map fields to use lower case and underscores -->
<property name="fieldNamingPolicy" value="LOWER_CASE_WITH_UNDERSCORES"/>
</bean>
And use it in Camel routes by referring to its bean id as shown: <camelContext xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:inPojo"/> <marshal ref="gson"/> </route> <route> <from uri="direct:backPojo"/> <unmarshal ref="gson"/> </route> </camelContext> Include/Exclude fields using the jsonView attribute with JacksonDataFormatAvailable as of Camel 2.12 As an example of using this attribute you can instead of: JacksonDataFormat ageViewFormat = new JacksonDataFormat(TestPojoView.class, Views.Age.class); from("direct:inPojoAgeView"). marshal(ageViewFormat); Directly specify your JSON view inside the Java DSL as:
from("direct:inPojoAgeView").
marshal().json(TestPojoView.class, Views.Age.class);
And the same in XML DSL: <from uri="direct:inPojoAgeView"/> <marshal> <json library="Jackson" unmarshalTypeName="org.apache.camel.component.jackson.TestPojoView" jsonView="org.apache.camel.component.jackson.Views$Age"/> </marshal> Dependencies for XStreamTo use JSON in your camel routes you need to add the a dependency on camel-xstream which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-xstream</artifactId> <version>2.9.2</version> </dependency> Dependencies for JacksonTo use JSON in your camel routes you need to add the a dependency on camel-jackson which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-jackson</artifactId> <version>2.9.2</version> </dependency> Dependencies for GSONTo use JSON in your camel routes you need to add the a dependency on camel-gson which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-gson</artifactId> <version>2.10.0</version> </dependency> The Zip Data Format is a message compression and de-compression format. Messages marshalled using Zip compression can be unmarshalled using Zip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads. It facilitates more optimal use of network bandwidth while incurring a small cost in order to compress and decompress payloads at the endpoint.
Options
MarshalIn this example we marshal a regular text/XML payload to a compressed payload employing zip compression Deflater.BEST_COMPRESSION and send it an ActiveMQ queue called MY_QUEUE. from("direct:start").marshal().zip(Deflater.BEST_COMPRESSION).to("activemq:queue:MY_QUEUE"); Alternatively if you would like to use the default setting you could send it as from("direct:start").marshal().zip().to("activemq:queue:MY_QUEUE"); UnmarshalIn this example we unmarshal a zipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnZippedMessageProcessor. Note that the compression Level employed during the marshalling should be identical to the one employed during unmarshalling to avoid errors. from("activemq:queue:MY_QUEUE").unmarshal().zip().process(new UnZippedMessageProcessor()); DependenciesThis data format is provided in camel-core so no additional dependencies is needed. TidyMarkupTidyMarkup is a Data Format that uses the TagSoup to tidy up HTML. It can be used to parse ugly HTML and return it as pretty wellformed HTML.
TidyMarkup only supports the unmarshal operation as we really don't want to turn well formed HTML into ugly HTML Java DSL ExampleAn example where the consumer provides some HTML
from("file://site/inbox").unmarshal().tidyMarkup().to("file://site/blogs");
Spring XML ExampleThe following example shows how to use TidyMarkup to unmarshal using Spring <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="file://site/inbox"/> <unmarshal> <tidyMarkup/> </unmarshal> <to uri="file://site/blogs"/> </route> </camelContext> DependenciesTo use TidyMarkup in your camel routes you need to add the a dependency on camel-tagsoup which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-tagsoup</artifactId> <version>x.x.x</version> </dependency> BindyThe goal of this component is to allow the parsing/binding of non-structured data (or to be more precise non-XML data)
to one or many Plain Old Java Object (POJO). Bindy converts the data according to the type of the java property. POJOs can be linked together with one-to-many relationships available in some cases. Moreover, for data type like Date, Double, Float, Integer, Short, Long and BigDecimal, you can provide the pattern to apply during the formatting of the property. For the BigDecimal numbers, you can also define the precision and the decimal or grouping separators.
Decimal* = Double, Integer, Float, Short, Long
To work with camel-bindy, you must first define your model in a package (e.g. com.acme.model) and for each model class (e.g. Order, Client, Instrument, ...) add the required annotations (described hereafter) to the Class or field.
AnnotationsThe annotations created allow to map different concept of your model to the POJO like :
This section will describe them : 1. CsvRecordThe CsvRecord annotation is used to identified the root class of the model. It represents a record = a line of a CSV file and can be linked to several children model classes.
case 1 : separator = ',' The separator used to segregate the fields in the CSV record is ',' : 10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009 @CsvRecord( separator = "," ) public Class Order { ... } case 2 : separator = ';' Compare to the previous case, the separator here is ';' instead of ',' : 10; J; Pauline; M; XD12345678; Fortis Dynamic 15/15; 2500; USD; 08-01-2009 @CsvRecord( separator = ";" ) public Class Order { ... } case 3 : separator = '|' Compare to the previous case, the separator here is '|' instead of ';' : 10| J| Pauline| M| XD12345678| Fortis Dynamic 15/15| 2500| USD| 08-01-2009 @CsvRecord( separator = "\\|" ) public Class Order { ... } case 4 : separator = '\",\"' When the field to be parsed of the CSV record contains ',' or ';' which is also used as separator, we whould find another strategy "10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009" @CsvRecord( separator = "\",\"" ) public Class Order { ... } From Camel 2.8.3/2.9 or never bindy will automatic detect if the record is enclosed with either single or double quotes and automatic remove those quotes when unmarshalling from CSV to Object. Therefore do not include the quotes in the separator, but simple do as below: "10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009" @CsvRecord( separator = "," ) public Class Order { ... } Notice that if you want to marshal from Object to CSV and use quotes, then you need to specify which quote character to use, using the quote attribute on the @CsvRecord as shown below: @CsvRecord( separator = ",", quote = "\"" ) public Class Order { ... } case 5 : separator & skipfirstline The feature is interesting when the client wants to have in the first line of the file, the name of the data fields : order id, client id, first name, last name, isin code, instrument name, quantity, currency, date To inform bindy that this first line must be skipped during the parsing process, then we use the attribute : @CsvRecord(separator = ",", skipFirstLine = true) public Class Order { ... } case 6 : generateHeaderColumns To add at the first line of the CSV generated, the attribute generateHeaderColumns must be set to true in the annotation like this : @CsvRecord( generateHeaderColumns = true ) public Class Order { ... } As a result, Bindy during the unmarshaling process will generate CSV like this : order id, client id, first name, last name, isin code, instrument name, quantity, currency, date case 7 : carriage return If the platform where camel-bindy will run is not Windows but Macintosh or Unix, than you can change the crlf property like this. Three values are available : WINDOWS, UNIX or MAC @CsvRecord(separator = ",", crlf="MAC") public Class Order { ... } Additionally, if for some reason you need to add a different line ending character, you can opt to specify it using the crlf parameter. In the following example, we can end the line with a comma followed by the newline character: @CsvRecord(separator = ",", crlf=",\n") public Class Order { ... } case 8 : isOrdered Sometimes, the order to follow during the creation of the CSV record from the model is different from the order used during the parsing. Then, in this case, we can use the attribute isOrdered = true to indicate this in combination with attribute 'position' of the DataField annotation. @CsvRecord(isOrdered = true) public Class Order { @DataField(pos = 1, position = 11) private int orderNr; @DataField(pos = 2, position = 10) private String clientNr; ... } Remark : pos is used to parse the file, stream while positions is used to generate the CSV 2. LinkThe link annotation will allow to link objects together.
e.g : If the model Class Client is linked to the Order class, then use annotation Link in the Order class like this : Property Link @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @Link private Client client; ... AND for the class Client : Class Link
@Link
public class Client {
...
}
3. DataFieldThe DataField annotation defines the property of the field. Each datafield is identified by its position in the record, a type (string, int, date, ...) and optionally of a pattern
case 1 : pos This parameter/attribute represents the position of the field in the csv record Position @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 5) private String isinCode; ... } As you can see in this example the position starts at '1' but continues at '5' in the class Order. The numbers from '2' to '4' are defined in the class Client (see here after). Position continues in another model class public class Client { @DataField(pos = 2) private String clientNr; @DataField(pos = 3) private String firstName; @DataField(pos = 4) private String lastName; ... } case 2 : pattern The pattern allows to enrich or validates the format of your data Pattern @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 5) private String isinCode; @DataField(name = "Name", pos = 6) private String instrumentName; @DataField(pos = 7, precision = 2) private BigDecimal amount; @DataField(pos = 8) private String currency; @DataField(pos = 9, pattern = "dd-MM-yyyy") -- pattern used during parsing or when the date is created private Date orderDate; ... } case 3 : precision The precision is helpful when you want to define the decimal part of your number Precision @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @Link private Client client; @DataField(pos = 5) private String isinCode; @DataField(name = "Name", pos = 6) private String instrumentName; @DataField(pos = 7, precision = 2) -- precision private BigDecimal amount; @DataField(pos = 8) private String currency; @DataField(pos = 9, pattern = "dd-MM-yyyy") private Date orderDate; ... } case 4 : Position is different in output The position attribute will inform bindy how to place the field in the CSV record generated. By default, the position used corresponds to the position defined with the attribute 'pos'. If the position is different (that means that we have an asymetric processus comparing marshaling from unmarshaling) than we can use 'position' to indicate this. Here is an example Position is different in output @CsvRecord(separator = ",") public class Order { @CsvRecord(separator = ",", isOrdered = true) public class Order { // Positions of the fields start from 1 and not from 0 @DataField(pos = 1, position = 11) private int orderNr; @DataField(pos = 2, position = 10) private String clientNr; @DataField(pos = 3, position = 9) private String firstName; @DataField(pos = 4, position = 8) private String lastName; @DataField(pos = 5, position = 7) private String instrumentCode; @DataField(pos = 6, position = 6) private String instrumentNumber; ... }
case 5 : required If a field is mandatory, simply use the attribute 'required' setted to true Required @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 2, required = true) private String clientNr; @DataField(pos = 3, required = true) private String firstName; @DataField(pos = 4, required = true) private String lastName; ... } If this field is not present in the record, than an error will be raised by the parser with the following information : Some fields are missing (optional or mandatory), line : case 6 : trim If a field has leading and/or trailing spaces which should be removed before they are processed, simply use the attribute 'trim' setted to true Trim @CsvRecord(separator = ",") public class Order { @DataField(pos = 1, trim = true) private int orderNr; @DataField(pos = 2, trim = true) private Integer clientNr; @DataField(pos = 3, required = true) private String firstName; @DataField(pos = 4) private String lastName; ... } case 7 : defaultValue If a field is not defined then uses the value indicated by the defaultValue attribute Default value @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 2) private Integer clientNr; @DataField(pos = 3, required = true) private String firstName; @DataField(pos = 4, defaultValue = "Barin") private String lastName; ... }
4. FixedLengthRecordThe FixedLengthRecord annotation is used to identified the root class of the model. It represents a record = a line of a file/message containing data fixed length formatted and can be linked to several children model classes. This format is a bit particular beause data of a field can be aligned to the right or to the left.
case 1 : Simple fixed length record This simple example shows how to design the model to parse/format a fixed message 10A9PaulineMISINXD12345678BUYShare2500.45USD01-08-2009 Fixed-simple
@FixedLengthRecord(length=54, paddingChar=' ')
public static class Order {
@DataField(pos = 1, length=2)
private int orderNr;
@DataField(pos = 3, length=2)
private String clientNr;
@DataField(pos = 5, length=7)
private String firstName;
@DataField(pos = 12, length=1, align="L")
private String lastName;
@DataField(pos = 13, length=4)
private String instrumentCode;
@DataField(pos = 17, length=10)
private String instrumentNumber;
@DataField(pos = 27, length=3)
private String orderType;
@DataField(pos = 30, length=5)
private String instrumentType;
@DataField(pos = 35, precision = 2, length=7)
private BigDecimal amount;
@DataField(pos = 42, length=3)
private String currency;
@DataField(pos = 45, length=10, pattern = "dd-MM-yyyy")
private Date orderDate;
...
case 2 : Fixed length record with alignment and padding This more elaborated example show how to define the alignment for a field and how to assign a padding character which is ' ' here'' 10A9 PaulineM ISINXD12345678BUYShare2500.45USD01-08-2009 Fixed-padding-align
@FixedLengthRecord(length=60, paddingChar=' ')
public static class Order {
@DataField(pos = 1, length=2)
private int orderNr;
@DataField(pos = 3, length=2)
private String clientNr;
@DataField(pos = 5, length=9)
private String firstName;
@DataField(pos = 14, length=5, align="L") // align text to the LEFT zone of the block
private String lastName;
@DataField(pos = 19, length=4)
private String instrumentCode;
@DataField(pos = 23, length=10)
private String instrumentNumber;
@DataField(pos = 33, length=3)
private String orderType;
@DataField(pos = 36, length=5)
private String instrumentType;
@DataField(pos = 41, precision = 2, length=7)
private BigDecimal amount;
@DataField(pos = 48, length=3)
private String currency;
@DataField(pos = 51, length=10, pattern = "dd-MM-yyyy")
private Date orderDate;
...
case 3 : Field padding Sometimes, the default padding defined for record cannnot be applied to the field as we have a number format where we would like to padd with '0' instead of ' '. In this case, you can use in the model the attribute paddingField to set this value. 10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009 Fixed-padding-field
@FixedLengthRecord(length = 65, paddingChar = ' ')
public static class Order {
@DataField(pos = 1, length = 2)
private int orderNr;
@DataField(pos = 3, length = 2)
private String clientNr;
@DataField(pos = 5, length = 9)
private String firstName;
@DataField(pos = 14, length = 5, align = "L")
private String lastName;
@DataField(pos = 19, length = 4)
private String instrumentCode;
@DataField(pos = 23, length = 10)
private String instrumentNumber;
@DataField(pos = 33, length = 3)
private String orderType;
@DataField(pos = 36, length = 5)
private String instrumentType;
@DataField(pos = 41, precision = 2, length = 12, paddingChar = '0')
private BigDecimal amount;
@DataField(pos = 53, length = 3)
private String currency;
@DataField(pos = 56, length = 10, pattern = "dd-MM-yyyy")
private Date orderDate;
...
case 4: Fixed length record with delimiter Fixed-length records sometimes have delimited content within the record. The firstName and lastName fields are delimited with the '^' character in the following example: 10A9Pauline^M^ISINXD12345678BUYShare000002500.45USD01-08-2009 Fixed-delimited
@FixedLengthRecord()
public static class Order {
@DataField(pos = 1, length = 2)
private int orderNr;
@DataField(pos = 2, length = 2)
private String clientNr;
@DataField(pos = 3, delimiter = "^")
private String firstName;
@DataField(pos = 4, delimiter = "^")
private String lastName;
@DataField(pos = 5, length = 4)
private String instrumentCode;
@DataField(pos = 6, length = 10)
private String instrumentNumber;
@DataField(pos = 7, length = 3)
private String orderType;
@DataField(pos = 8, length = 5)
private String instrumentType;
@DataField(pos = 9, precision = 2, length = 12, paddingChar = '0')
private BigDecimal amount;
@DataField(pos = 10, length = 3)
private String currency;
@DataField(pos = 11, length = 10, pattern = "dd-MM-yyyy")
private Date orderDate;
case 5 : Fixed length record with record-defined field length Occasionally a fixed-length record may contain a field that define the expected length of another field within the same record. In the following example the length of the instrumentNumber field value is defined by the value of instrumentNumberLen field in the record. 10A9Pauline^M^ISIN10XD12345678BUYShare000002500.45USD01-08-2009 Fixed-delimited
@FixedLengthRecord()
public static class Order {
@DataField(pos = 1, length = 2)
private int orderNr;
@DataField(pos = 2, length = 2)
private String clientNr;
@DataField(pos = 3, delimiter = "^")
private String firstName;
@DataField(pos = 4, delimiter = "^")
private String lastName;
@DataField(pos = 5, length = 4)
private String instrumentCode;
@DataField(pos = 6, length = 2, align = "R", paddingChar = '0')
private int instrumentNumberLen;
@DataField(pos = 7, lengthPos=6)
private String instrumentNumber;
@DataField(pos = 8, length = 3)
private String orderType;
@DataField(pos = 9, length = 5)
private String instrumentType;
@DataField(pos = 10, precision = 2, length = 12, paddingChar = '0')
private BigDecimal amount;
@DataField(pos = 11, length = 3)
private String currency;
@DataField(pos = 12, length = 10, pattern = "dd-MM-yyyy")
private Date orderDate;
case 6 : Fixed length record with header and footer Bindy will discover fixed-length header and footer records that are configured as part of the model – provided that the annotated classes exist either in the same package as the primary @FixedLengthRecord class, or within one of the configured scan packages. The following text illustrates two fixed-length records that are bracketed by a header record and footer record. 101-08-2009 Fixed-header-and-footer-main-class @FixedLengthRecord(hasHeader = true, hasFooter = true) public class Order { @DataField(pos = 1, length = 2) private int orderNr; @DataField(pos = 2, length = 2) private String clientNr; @DataField(pos = 3, length = 9) private String firstName; @DataField(pos = 4, length = 5, align = "L") private String lastName; @DataField(pos = 5, length = 4) private String instrumentCode; @DataField(pos = 6, length = 10) private String instrumentNumber; @DataField(pos = 7, length = 3) private String orderType; @DataField(pos = 8, length = 5) private String instrumentType; @DataField(pos = 9, precision = 2, length = 12, paddingChar = '0') private BigDecimal amount; @DataField(pos = 10, length = 3) private String currency; @DataField(pos = 11, length = 10, pattern = "dd-MM-yyyy") private Date orderDate; ... } @FixedLengthRecord(isHeader = true) public class OrderHeader { @DataField(pos = 1, length = 1) private int recordType = 1; @DataField(pos = 2, length = 10, pattern = "dd-MM-yyyy") private Date recordDate; ... } @FixedLengthRecord(isFooter = true) public class OrderFooter { @DataField(pos = 1, length = 1) private int recordType = 9; @DataField(pos = 2, length = 9, align = "R", paddingChar = '0') private int numberOfRecordsInTheFile; ... } 5. MessageThe Message annotation is used to identified the class of your model who will contain key value pairs fields. This kind of format is used mainly in Financial Exchange Protocol Messages (FIX). Nevertheless, this annotation can be used for any other format where data are identified by keys. The key pair values are separated each other by a separator which can be a special character like a tab delimitor (unicode representation : \u0009) or a start of heading (unicode representation : \u0001)
case 1 : separator = 'u0001' The separator used to segregate the key value pair fields in a FIX message is the ASCII '01' character or in unicode format '\u0001'. This character must be escaped a second time to avoid a java runtime error. Here is an example : 8=FIX.4.1 9=20 34=1 35=0 49=INVMGR 56=BRKR 1=BE.CHM.001 11=CHM0001-01 22=4 ... and how to use the annotation FIX - message @Message(keyValuePairSeparator = "=", pairSeparator = "\u0001", type="FIX", version="4.1") public class Order { ... }
6. KeyValuePairFieldThe KeyValuePairField annotation defines the property of a key value pair field. Each KeyValuePairField is identified by a tag (= key) and its value associated, a type (string, int, date, ...), optionaly a pattern and if the field is required
case 1 : tag This parameter represents the key of the field in the message FIX message - Tag @Message(keyValuePairSeparator = "=", pairSeparator = "\u0001", type="FIX", version="4.1") public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1) // Client reference private String Account; @KeyValuePairField(tag = 11) // Order reference private String ClOrdId; @KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...) private String IDSource; @KeyValuePairField(tag = 48) // Fund code private String SecurityId; @KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell) private String Side; @KeyValuePairField(tag = 58) // Free text private String Text; ... } case 2 : Different position in output If the tags/keys that we will put in the FIX message must be sorted according to a predefine order, then use the attribute 'position' of the annotation @KeyValuePairField FIX message - Tag - sort @Message(keyValuePairSeparator = "=", pairSeparator = "\\u0001", type = "FIX", version = "4.1", isOrdered = true) public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1, position = 1) // Client reference private String account; @KeyValuePairField(tag = 11, position = 3) // Order reference private String clOrdId; ... } 7. SectionIn FIX message of fixed length records, it is common to have different sections in the representation of the information : header, body and section. The purpose of the annotation @Section is to inform bindy about which class of the model represents the header (= section 1), body (= section 2) and footer (= section 3) Only one attribute/parameter exists for this annotation.
case 1 : Section A. Definition of the header section FIX message - Section - Header @Section(number = 1) public class Header { @KeyValuePairField(tag = 8, position = 1) // Message Header private String beginString; @KeyValuePairField(tag = 9, position = 2) // Checksum private int bodyLength; ... } B. Definition of the body section FIX message - Section - Body @Section(number = 2) @Message(keyValuePairSeparator = "=", pairSeparator = "\\u0001", type = "FIX", version = "4.1", isOrdered = true) public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1, position = 1) // Client reference private String account; @KeyValuePairField(tag = 11, position = 3) // Order reference private String clOrdId; C. Definition of the footer section FIX message - Section - Footer @Section(number = 3) public class Trailer { @KeyValuePairField(tag = 10, position = 1) // CheckSum private int checkSum; public int getCheckSum() { return checkSum; } 8. OneToManyThe purpose of the annotation @OneToMany is to allow to work with a List<?> field defined a POJO class or from a record containing repetitive groups.
The relation OneToMany ONLY WORKS in the following cases :
case 1 : Generating CSV with repetitive data Here is the CSV output that we want : Claus,Ibsen,Camel in Action 1,2010,35 Remark : the repetitive data concern the title of the book and its publication date while first, last name and age are common and the classes used to modeling this. The Author class contains a List of Book. Generate CSV with repetitive data @CsvRecord(separator=",") public class Author { @DataField(pos = 1) private String firstName; @DataField(pos = 2) private String lastName; @OneToMany private List<Book> books; @DataField(pos = 5) private String Age; ... public class Book { @DataField(pos = 3) private String title; @DataField(pos = 4) private String year; Very simple isn't it !!! case 2 : Reading FIX message containing group of tags/keys Here is the message that we would like to process in our model : "8=FIX 4.19=2034=135=049=INVMGR56=BRKR" tags 22, 48 and 54 are repeated and the code Reading FIX message containing group of tags/keys public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1) // Client reference private String account; @KeyValuePairField(tag = 11) // Order reference private String clOrdId; @KeyValuePairField(tag = 58) // Free text private String text; @OneToMany(mappedTo = "org.apache.camel.dataformat.bindy.model.fix.complex.onetomany.Security") List<Security> securities; ... public class Security { @KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...) private String idSource; @KeyValuePairField(tag = 48) // Fund code private String securityCode; @KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell) private String side; Using the Java DSLThe next step consists in instantiating the DataFormat bindy class associated with this record type and providing Java package name(s) as parameter. For example the following uses the class BindyCsvDataFormat (who correspond to the class associated with the CSV record type) which is configured with "com.acme.model" DataFormat bindy = new BindyCsvDataFormat("com.acme.model"); Unmarshalingfrom("file://inbox") .unmarshal(bindy) .to("direct:handleOrders"); Alternatively, you can use a named reference to a data format which can then be defined in your Registry e.g. your Spring XML file: from("file://inbox") .unmarshal("myBindyDataFormat") .to("direct:handleOrders"); The Camel route will pick-up files in the inbox directory, unmarshall CSV records into a collection of model objects and send the collection The collection returned is a List of Map objects. Each Map within the list contains the model objects that were marshalled out of each line of the CSV. The reason behind this is that each line can correspond to more than one object. This can be confusing when you simply expect one object to be returned per line. Each object can be retrieve using its class name.
List<Map<String, Object>> unmarshaledModels = (List<Map<String, Object>>) exchange.getIn().getBody();
int modelCount = 0;
for (Map<String, Object> model : unmarshaledModels) {
for (String className : model.keySet()) {
Object obj = model.get(className);
LOG.info("Count : " + modelCount + ", " + obj.toString());
}
modelCount++;
}
LOG.info("Total CSV records received by the csv bean : " + modelCount);
Assuming that you want to extract a single Order object from this map for processing in a route, you could use a combination of a Splitter and a Processor as per the following: from("file://inbox") .unmarshal(bindy) .split(body()) .process(new Processor() { public void process(Exchange exchange) throws Exception { Message in = exchange.getIn(); Map<String, Object> modelMap = (Map<String, Object>) in.getBody(); in.setBody(modelMap.get(Order.class.getCanonicalName())); } }) .to("direct:handleSingleOrder") .end(); MarshalingTo generate CSV records from a collection of model objects, you create the following route : from("direct:handleOrders") .marshal(bindy) .to("file://outbox") Unit testHere is two examples showing how to marshall or unmarshall a CSV file with Camel Marshall package org.apache.camel.dataformat.bindy.csv; import java.math.BigDecimal; import java.util.ArrayList; import java.util.Calendar; import java.util.GregorianCalendar; import java.util.HashMap; import java.util.List; import java.util.Map; import org.apache.camel.EndpointInject; import org.apache.camel.Produce; import org.apache.camel.ProducerTemplate; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink.Client; import org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink.Order; import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration; import org.junit.Test; import org.springframework.config.java.annotation.Bean; import org.springframework.config.java.annotation.Configuration; import org.springframework.config.java.test.JavaConfigContextLoader; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; @ContextConfiguration(locations = "org.apache.camel.dataformat.bindy.csv.BindyComplexCsvMarshallTest$ContextConfig", loader = JavaConfigContextLoader.class) public class BindyComplexCsvMarshallTest extends AbstractJUnit4SpringContextTests { private List<Map<String, Object>> models = new ArrayList<Map<String, Object>>(); private String result = "10,A1,Julia,Roberts,BE123456789,Belgium Ventage 10/12,150,USD,14-01-2009"; @Produce(uri = "direct:start") private ProducerTemplate template; @EndpointInject(uri = "mock:result") private MockEndpoint resultEndpoint; @Test public void testMarshallMessage() throws Exception { resultEndpoint.expectedBodiesReceived(result); template.sendBody(generateModel()); resultEndpoint.assertIsSatisfied(); } private List<Map<String, Object>> generateModel() { Map<String, Object> model = new HashMap<String, Object>(); Order order = new Order(); order.setOrderNr(10); order.setAmount(new BigDecimal("150")); order.setIsinCode("BE123456789"); order.setInstrumentName("Belgium Ventage 10/12"); order.setCurrency("USD"); Calendar calendar = new GregorianCalendar(); calendar.set(2009, 0, 14); order.setOrderDate(calendar.getTime()); Client client = new Client(); client.setClientNr("A1"); client.setFirstName("Julia"); client.setLastName("Roberts"); order.setClient(client); model.put(order.getClass().getName(), order); model.put(client.getClass().getName(), client); models.add(0, model); return models; } @Configuration public static class ContextConfig extends SingleRouteCamelConfiguration { BindyCsvDataFormat camelDataFormat = new BindyCsvDataFormat("org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink"); @Override @Bean public RouteBuilder route() { return new RouteBuilder() { @Override public void configure() { from("direct:start").marshal(camelDataFormat).to("mock:result"); } }; } } } Unmarshall package org.apache.camel.dataformat.bindy.csv; import org.apache.camel.EndpointInject; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration; import org.junit.Test; import org.springframework.config.java.annotation.Bean; import org.springframework.config.java.annotation.Configuration; import org.springframework.config.java.test.JavaConfigContextLoader; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; @ContextConfiguration(locations = "org.apache.camel.dataformat.bindy.csv.BindyComplexCsvUnmarshallTest$ContextConfig", loader = JavaConfigContextLoader.class) public class BindyComplexCsvUnmarshallTest extends AbstractJUnit4SpringContextTests { @EndpointInject(uri = "mock:result") private MockEndpoint resultEndpoint; @Test public void testUnMarshallMessage() throws Exception { resultEndpoint.expectedMessageCount(1); resultEndpoint.assertIsSatisfied(); } @Configuration public static class ContextConfig extends SingleRouteCamelConfiguration { BindyCsvDataFormat csvBindyDataFormat = new BindyCsvDataFormat("org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink"); @Override @Bean public RouteBuilder route() { return new RouteBuilder() { @Override public void configure() { from("file://src/test/data?noop=true").unmarshal(csvBindyDataFormat).to("mock:result"); } }; } } } In this example, BindyCsvDataFormat class has been instantiated in a traditional way but it is also possible to provide information directly to the function (un)marshal like this where BindyType corresponds to the Bindy DataFormat class to instantiate and the parameter contains the list of package names.
public static class ContextConfig extends SingleRouteCamelConfiguration {
@Override
@Bean
public RouteBuilder route() {
return new RouteBuilder() {
@Override
public void configure() {
from("direct:start")
.marshal().bindy(BindyType.Csv, "org.apache.camel.dataformat.bindy.model.simple.oneclass")
.to("mock:result");
}
};
}
}
Using Spring XMLThis is really easy to use Spring as your favorite DSL language to declare the routes to be used for camel-bindy. The following example shows two routes where the first will pick-up records from files, unmarshal the content and bind it to their model. The result is then send to a pojo (doing nothing special) and place them into a queue. The second route will extract the pojos from the queue and marshal the content to generate a file containing the csv record spring dsl <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd"> <bean id="bindyDataformat" class="org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat"> <constructor-arg value="org.apache.camel.bindy.model" /> </bean> <bean id="csv" class="org.apache.camel.bindy.csv.HandleOrderBean" /> <!-- Queuing engine - ActiveMq - work locally in mode virtual memory --> <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent"> <property name="brokerURL" value="vm://localhost:61616"/> </bean> <camelContext xmlns="http://camel.apache.org/schema/spring"> <jmxAgent id="agent" disabled="false" /> <route> <from uri="file://src/data/csv/?noop=true" /> <unmarshal ref="bindyDataformat" /> <to uri="bean:csv" /> <to uri="activemq:queue:in" /> </route> <route> <from uri="activemq:queue:in" /> <marshal ref="bindyDataformat" /> <to uri="file://src/data/csv/out/" /> </route> </camelContext> </beans>
DependenciesTo use Bindy in your camel routes you need to add the a dependency on camel-bindy which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-bindy</artifactId> <version>x.x.x</version> </dependency> XMLSecurity Data FormatThe XMLSecurity Data Format facilitates encryption and decryption of XML payloads at the Document, Element, and Element Content levels (including simultaneous multi-node encryption/decryption using XPath). The encryption capability is based on formats supported using the Apache XML Security (Santaurio) project. Symmetric encryption/decryption is currently supported using Triple-DES and AES (128, 192, and 256) encryption formats. Additional formats can be easily added later as needed. This capability allows Camel users to encrypt/decrypt payloads while being dispatched or received along a route. Available as of Camel 2.9 The XMLSecurity Data Format also has improved support for namespaces when processing the XPath queries that select content for encryption. A namespace definition mapping can be included as part of the data format configuration. This enables true namespace matching, even if the prefix values in the XPath query and the target xml document are not equivalent strings. Basic Options
Asymmetric Encryption OptionsThese options can be applied in addition to relevant the Basic options to use asymmetric key encryption.
MarshalIn order to encrypt the payload, the marshal processor needs to be applied on the route followed by the secureXML() tag. UnmarshalIn order to decrypt the payload, the unmarshal processor needs to be applied on the route followed by the secureXML() tag. ExamplesGiven below are several examples of how marshalling could be performed at the Document, Element, and Content levels. Full Payload encryption/decryptionfrom("direct:start") .marshal().secureXML() .unmarshal().secureXML() .to("direct:end"); Partial Payload Content Only encryption/decryptionString tagXPATH = "//cheesesites/italy/cheese"; boolean secureTagContent = true; ... from("direct:start") .marshal().secureXML(tagXPATH, secureTagContent) .unmarshal().secureXML(tagXPATH, secureTagContent) .to("direct:end"); Partial Multi Node Payload Content Only encryption/decryptionString tagXPATH = "//cheesesites/*/cheese"; boolean secureTagContent = true; ... from("direct:start") .marshal().secureXML(tagXPATH, secureTagContent) .unmarshal().secureXML(tagXPATH, secureTagContent) .to("direct:end"); Partial Payload Content Only encryption/decryption with choice of passPhrase(password)String tagXPATH = "//cheesesites/italy/cheese"; boolean secureTagContent = true; ... String passPhrase = "Just another 24 Byte key"; from("direct:start") .marshal().secureXML(tagXPATH, secureTagContent, passPhrase) .unmarshal().secureXML(tagXPATH, secureTagContent, passPhrase) .to("direct:end"); Partial Payload Content Only encryption/decryption with passPhrase(password) and Algorithmimport org.apache.xml.security.encryption.XMLCipher; .... String tagXPATH = "//cheesesites/italy/cheese"; boolean secureTagContent = true; String passPhrase = "Just another 24 Byte key"; String algorithm= XMLCipher.TRIPLEDES; from("direct:start") .marshal().secureXML(tagXPATH, secureTagContent, passPhrase, algorithm) .unmarshal().secureXML(tagXPATH, secureTagContent, passPhrase, algorithm) .to("direct:end"); Partial Paryload Content with Namespace supportJava DSLfinal Map<String, String> namespaces = new HashMap<String, String>(); namespaces.put("cust", "http://cheese.xmlsecurity.camel.apache.org/"); final KeyStoreParameters tsParameters = new KeyStoreParameters(); tsParameters.setPassword("password"); tsParameters.setResource("sender.ts"); context.addRoutes(new RouteBuilder() { public void configure() { from("direct:start") .marshal().secureXML("//cust:cheesesites/italy", namespaces, true, "recipient", testCypherAlgorithm, XMLCipher.RSA_v1dot5, tsParameters) .to("mock:encrypted"); } } Spring XMLA namespace prefix that is defined as part of the camelContext definition can be re-used in context within the data format secureTag attribute of the secureXML element. <camelContext id="springXmlSecurityDataFormatTestCamelContext" xmlns="http://camel.apache.org/schema/spring" xmlns:cheese="http://cheese.xmlsecurity.camel.apache.org/"> <route> <from uri="direct://start"/> <marshal> <secureXML secureTag="//cheese:cheesesites/italy" secureTagContents="true"/> </marshal> ... Asymmetric Key EncryptionSpring XML Sender<!-- trust store configuration --> <camel:keyStoreParameters id="trustStoreParams" resource="./sender.ts" password="password"/> <camelContext id="springXmlSecurityDataFormatTestCamelContext" xmlns="http://camel.apache.org/schema/spring" xmlns:cheese="http://cheese.xmlsecurity.camel.apache.org/"> <route> <from uri="direct://start"/> <marshal> <secureXML secureTag="//cheese:cheesesites/italy" secureTagContents="true" xmlCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc" keyCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" recipientKeyAlias="recipient" keyOrTrustStoreParametersId="trustStoreParams"/> </marshal> ... Spring XML Recipient<!-- key store configuration --> <camel:keyStoreParameters id="keyStoreParams" resource="./recipient.ks" password="password" /> <camelContext id="springXmlSecurityDataFormatTestCamelContext" xmlns="http://camel.apache.org/schema/spring" xmlns:cheese="http://cheese.xmlsecurity.camel.apache.org/"> <route> <from uri="direct://encrypted"/> <unmarshal> <secureXML secureTag="//cheese:cheesesites/italy" secureTagContents="true" xmlCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc" keyCipherAlgorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" recipientKeyAlias="recipient" keyOrTrustStoreParametersId="keyStoreParams" keyPassword="privateKeyPassword" /> </unmarshal> ... DependenciesThis data format is provided within the camel-xmlsecurity component. The GZip Data Format is a message compression and de-compression format. It uses the same deflate algorithm that is used in Zip DataFormat, although some additional headers are provided. This format is produced by popular gzip/gunzip tool. Messages marshalled using GZip compression can be unmarshalled using GZip decompression just prior to being consumed at the endpoint. The compression capability is quite useful when you deal with large XML and Text based payloads or when you read messages previously comressed using gzip tool. OptionsThere are no options provided for this data format. MarshalIn this example we marshal a regular text/XML payload to a compressed payload employing gzip compression format and send it an ActiveMQ queue called MY_QUEUE. from("direct:start").marshal().gzip().to("activemq:queue:MY_QUEUE"); UnmarshalIn this example we unmarshal a gzipped payload from an ActiveMQ queue called MY_QUEUE to its original format, and forward it for processing to the UnGZippedMessageProcessor. from("activemq:queue:MY_QUEUE").unmarshal().gzip().process(new UnGZippedMessageProcessor()); DependenciesThis data format is provided in camel-core so no additional dependencies is needed. CastorAvailable as of Camel 2.1 Castor is a Data Format which uses the Castor XML library to unmarshal an XML payload into Java objects or to marshal Java objects into an XML payload. As usually you can use either Java DSL or Spring XML to work with Castor Data Format. Using the Java DSLfrom("direct:order"). marshal().castor(). to("activemq:queue:order"); For example the following uses a named DataFormat of Castor which uses default Castor data binding features. CastorDataFormat castor = new CastorDataFormat (); from("activemq:My.Queue"). unmarshal(castor). to("mqseries:Another.Queue"); If you prefer to use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g. from("activemq:My.Queue"). unmarshal("mycastorType"). to("mqseries:Another.Queue"); If you want to override default mapping schema by providing a mapping file you can set it as follows. CastorDataFormat castor = new CastorDataFormat (); castor.setMappingFile("mapping.xml"); Also if you want to have more control on Castor Marshaller and Unmarshaller you can access them as below. castor.getMarshaller(); castor.getUnmarshaller(); Using Spring XMLThe following example shows how to use Castor to unmarshal using Spring configuring the castor data type <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <unmarshal> <castor validation="true" /> </unmarshal> <to uri="mock:result"/> </route> </camelContext> This example shows how to configure the data type just once and reuse it on multiple routes. You have to set the <castor> element directly in <camelContext>. <camelContext> <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <castor id="myCastor"/> </dataFormats> <route> <from uri="direct:start"/> <marshal ref="myCastor"/> <to uri="direct:marshalled"/> </route> <route> <from uri="direct:marshalled"/> <unmarshal ref="myCastor"/> <to uri="mock:result"/> </route> </camelContext> OptionsCastor supports the following options
DependenciesTo use Castor in your camel routes you need to add the a dependency on camel-castor which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-castor</artifactId> <version>x.x.x</version> </dependency> Protobuf - Protocol Buffers"Protocol Buffers - Google's data interchange format"
Camel provides a Data Format to serialse between Java and the Protocol Buffer protocol. The project's site details why you may wish to choose this format over xml. Protocol Buffer is language-neutral and platform-neutral, so messages produced by your Camel routes may be consumed by other language implementations. API Site Protobuf overviewThis quick overview of how to use Protobuf. For more detail see the complete tutorial Defining the proto formatThe first step is to define the format for the body of your exchange. This is defined in a .proto file as so: addressbook.proto package org.apache.camel.component.protobuf; option java_package = "org.apache.camel.component.protobuf"; option java_outer_classname = "AddressBookProtos"; message Person { required string name = 1; required int32 id = 2; optional string email = 3; enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; } message PhoneNumber { required string number = 1; optional PhoneType type = 2 [default = HOME]; } repeated PhoneNumber phone = 4; } message AddressBook { repeated Person person = 1; } Generating Java classesThe Protobuf SDK provides a compiler which will generate the Java classes for the format we defined in our .proto file. You can run the compiler for any additional supported languages you require. protoc --java_out=. ./addressbook.proto This will generate a single Java class named AddressBookProtos which contains inner classes for Person and AddressBook. Builders are also implemented for you. The generated classes implement com.google.protobuf.Message which is required by the serialisation mechanism. For this reason it important that only these classes are used in the body of your exchanges. Camel will throw an exception on route creation if you attempt to tell the Data Format to use a class that does not implement com.google.protobuf.Message. Use the generated builders to translate the data from any of your existing domain classes. Java DSLYou can use create the ProtobufDataFormat instance and pass it to Camel DataFormat marshal and unmarsha API like this. ProtobufDataFormat format = new ProtobufDataFormat(Person.getDefaultInstance()); from("direct:in").marshal(format); from("direct:back").unmarshal(format).to("mock:reverse"); Or use the DSL protobuf() passing the unmarshal default instance or default instance class name like this. // You don't need to specify the default instance for protobuf marshaling from("direct:marshal").marshal().protobuf(); from("direct:unmarshalA").unmarshal(). protobuf("org.apache.camel.dataformat.protobuf.generated.AddressBookProtos$Person"). to ("mock:reverse"); from("direct:unmarshalB").unmarshal().protobuf(Person.getDefaultInstance()).to("mock:reverse"); Spring DSLThe following example shows how to use Castor to unmarshal using Spring configuring the protobuf data type <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <route> <from uri="direct:start"/> <unmarshal> <protobuf instanceClass="org.apache.camel.dataformat.protobuf.generated.AddressBookProtos$Person" /> </unmarshal> <to uri="mock:result"/> </route> </camelContext> DependenciesTo use Protobuf in your camel routes you need to add the a dependency on camel-protobuf which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-protobuf</artifactId> <version>2.2.0</version> </dependency> SOAP DataFormatAvailable as of Camel 2.3 SOAP is a Data Format which uses JAXB2 and JAX-WS annotations to marshal and unmarshal SOAP payloads. It provides the basic features of Apache CXF without need for the CXF Stack.
ElementNameStrategyAn element name strategy is used for two purposes. The first is to find a xml element name for a given object and soap action when marshaling the object into a SOAP message. The second is to find an Exception class for a given soap fault name.
If you have generated the web service stub code with cxf-codegen or a similar tool then you probably will want to use the ServiceInterfaceStrategy. In the case you have no annotated service interface you should use QNameStrategy or TypeNameStrategy. Using the Java DSLThe following example uses a named DataFormat of soap which is configured with the package com.example.customerservice to initialize the JAXBContext. The second parameter is the ElementNameStrategy. The route is able to marshal normal objects as well as exceptions. (Note the below just sends a SOAP Envelope to a queue. A web service provider would actually need to be listening to the queue for a SOAP call to actually occur, in which case it would be a one way SOAP request. If you need request reply then you should look at the next example.) SoapJaxbDataFormat soap = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class)); from("direct:start") .marshal(soap) .to("jms:myQueue");
Using SOAP 1.2Available as of Camel 2.11 SoapJaxbDataFormat soap = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class)); soap.setVersion("1.2"); from("direct:start") .marshal(soap) .to("jms:myQueue"); When using XML DSL there is a version attribute you can set on the <soap> element.
<!-- Defining a ServiceInterfaceStrategy for retrieving the element name when marshalling -->
<bean id="myNameStrategy" class="org.apache.camel.dataformat.soap.name.ServiceInterfaceStrategy">
<constructor-arg value="com.example.customerservice.CustomerService"/>
<constructor-arg value="true"/>
</bean>
And in the Camel route <route> <from uri="direct:start"/> <marshal> <soap contentPath="com.example.customerservice" version="1.2" elementNameStrategyRef="myNameStrategy"/> </marshal> <to uri="jms:myQueue"/> </route> Multi-part MessagesAvailable as of Camel 2.8.1 Multi-part SOAP messages are supported by the ServiceInterfaceStrategy. The ServiceInterfaceStrategy must be initialized with a service interface definition that is annotated in accordance with JAX-WS 2.2 and meets the requirements of the Document Bare style. The target method must meet the following criteria, as per the JAX-WS specification: 1) it must have at most one in or in/out non-header parameter, 2) if it has a return type other than void it must have no in/out or out non-header parameters, 3) if it it has a return type of void it must have at most one in/out or out non-header parameter. The ServiceInterfaceStrategy should be initialized with a boolean parameter that indicates whether the mapping strategy applies to the request parameters or response parameters. ServiceInterfaceStrategy strat = new ServiceInterfaceStrategy(com.example.customerservice.multipart.MultiPartCustomerService.class, true); SoapJaxbDataFormat soapDataFormat = new SoapJaxbDataFormat("com.example.customerservice.multipart", strat); Multi-part RequestThe payload parameters for a multi-part request are initiazlied using a BeanInvocation object that reflects the signature of the target operation. The camel-soap DataFormat maps the content in the BeanInvocation to fields in the SOAP header and body in accordance with the JAX-WS mapping when the marshal() processor is invoked. BeanInvocation beanInvocation = new BeanInvocation(); // Identify the target method beanInvocation.setMethod(MultiPartCustomerService.class.getMethod("getCustomersByName", GetCustomersByName.class, com.example.customerservice.multipart.Product.class)); // Populate the method arguments GetCustomersByName getCustomersByName = new GetCustomersByName(); getCustomersByName.setName("Dr. Multipart"); Product product = new Product(); product.setName("Multiuse Product"); product.setDescription("Useful for lots of things."); Object[] args = new Object[] {getCustomersByName, product}; // Add the arguments to the bean invocation beanInvocation.setArgs(args); // Set the bean invocation object as the message body exchange.getIn().setBody(beanInvocation); Multi-part ResponseA multi-part soap response may include an element in the soap body and will have one or more elements in the soap header. The camel-soap DataFormat will unmarshall the element in the soap body (if it exists) and place it onto the body of the out message in the exchange. Header elements will not be marshaled into their JAXB mapped object types. Instead, these elements are placed into the camel out message header org.apache.camel.dataformat.soap.UNMARSHALLED_HEADER_LIST. The elements will appear either as element instance values, or as JAXBElement values, depending upon the setting for the ignoreJAXBElement property. This property is inherited from camel-jaxb. You can also have the camel-soap DataFormate ignore header content all-together by setting the ignoreUnmarshalledHeaders value to true. Holder Object mappingJAX-WS specifies the use of a type-parameterized javax.xml.ws.Holder object for In/Out and Out parameters. A Holder object may be used when building the BeanInvocation, or you may use an instance of the parameterized-type directly. The camel-soap DataFormat marshals Holder values in accordance with the JAXB mapping for the class of the Holder's value. No mapping is provided for Holder objects in an unmarshalled response. ExamplesWebservice clientThe following route supports marshalling the request and unmarshalling a response or fault. String WS_URI = "cxf://http://myserver/customerservice?serviceClass=com.example.customerservice&dataFormat=MESSAGE"; SoapJaxbDataFormat soapDF = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class)); from("direct:customerServiceClient") .onException(Exception.class) .handled(true) .unmarshal(soapDF) .end() .marshal(soapDF) .to(WS_URI) .unmarshal(soapDF); The below snippet creates a proxy for the service interface and makes a SOAP call to the above route. import org.apache.camel.Endpoint; import org.apache.camel.component.bean.ProxyHelper; ... Endpoint startEndpoint = context.getEndpoint("direct:customerServiceClient"); ClassLoader classLoader = Thread.currentThread().getContextClassLoader(); // CustomerService below is the service endpoint interface, *not* the javax.xml.ws.Service subclass CustomerService proxy = ProxyHelper.createProxy(startEndpoint, classLoader, CustomerService.class); GetCustomersByNameResponse response = proxy.getCustomersByName(new GetCustomersByName()); Webservice ServerUsing the following route sets up a webservice server that listens on jms queue customerServiceQueue and processes requests using the class CustomerServiceImpl. The customerServiceImpl of course should implement the interface CustomerService. Instead of directly instantiating the server class it could be defined in a spring context as a regular bean. SoapJaxbDataFormat soapDF = new SoapJaxbDataFormat("com.example.customerservice", new ServiceInterfaceStrategy(CustomerService.class)); CustomerService serverBean = new CustomerServiceImpl(); from("jms://queue:customerServiceQueue") .onException(Exception.class) .handled(true) .marshal(soapDF) .end() .unmarshal(soapDF) .bean(serverBean) .marshal(soapDF); DependenciesTo use the SOAP dataformat in your camel routes you need to add the following dependency to your pom. <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-soap</artifactId> <version>2.3.0</version> </dependency> CryptoAvailable as of Camel 2.3 The Crypto Data Format integrates the Java Cryptographic Extension into Camel, allowing simple and flexible encryption and decryption of messages using Camel's familiar marshall and unmarshal formatting mechanism. It assumes marshalling to mean encryption to cyphertext and unmarshalling to mean decryption back to the original plaintext. Options
Basic UsageAt its most basic all that is required to encrypt/decrypt an exchange is a shared secret key. If one or more instances of the Crypto data format are configured with this key the format can be used to encrypt the payload in one route (or part of one) and decrypted in another. For example, using the Java DSL as follows: KeyGenerator generator = KeyGenerator.getInstance("DES"); CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES", generator.generateKey()); from("direct:basic-encryption") .marshal(cryptoFormat) .to("mock:encrypted") .unmarshal(cryptoFormat) .to("mock:unencrypted"); In Spring the dataformat is configured first and then used in routes <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <crypto id="basic" algorithm="DES" keyRef="desKey" /> </dataFormats> ... <route> <from uri="direct:basic-encryption" /> <marshal ref="basic" /> <to uri="mock:encrypted" /> <unmarshal ref="basic" /> <to uri="mock:unencrypted" /> </route> </camelContext> Specifying the Encryption AlgorithmChanging the algorithm is a matter of supplying the JCE algorithm name. If you change the algorithm you will need to use a compatible key. KeyGenerator generator = KeyGenerator.getInstance("DES"); CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES", generator.generateKey()); cryptoFormat.setShouldAppendHMAC(true); cryptoFormat.setMacAlgorithm("HmacMD5"); from("direct:hmac-algorithm") .marshal(cryptoFormat) .to("mock:encrypted") .unmarshal(cryptoFormat) .to("mock:unencrypted"); Specifying an Initialization VectorSome crypto algorhithms, particularly block algorithms, require configuration with an initial block of data known as an Initialization Vector. In the JCE this is passed as an AlgorithmParameterSpec when the Cipher is initialized. To use such a vector with the CryptoDataFormat you can configure it with a byte[] contianing the required data e.g. KeyGenerator generator = KeyGenerator.getInstance("DES"); byte[] initializationVector = new byte[] {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07}; CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES/CBC/PKCS5Padding", generator.generateKey()); cryptoFormat.setInitializationVector(initializationVector); from("direct:init-vector") .marshal(cryptoFormat) .to("mock:encrypted") .unmarshal(cryptoFormat) .to("mock:unencrypted"); or with spring, suppling a reference to a byte[] <crypto id="initvector" algorithm="DES/CBC/PKCS5Padding" keyRef="desKey" initVectorRef="initializationVector" />
The same vector is required in both the encryption and decryption phases. As it is not necessary to keep the IV a secret, the DataFormat allows for it to be inlined into the encrypted data and subsequently read out in the decryption phase to initialize the Cipher. To inline the IV set the /oinline flag. KeyGenerator generator = KeyGenerator.getInstance("DES"); byte[] initializationVector = new byte[] {0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07}; SecretKey key = generator.generateKey(); CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES/CBC/PKCS5Padding", key); cryptoFormat.setInitializationVector(initializationVector); cryptoFormat.setShouldInlineInitializationVector(true); CryptoDataFormat decryptFormat = new CryptoDataFormat("DES/CBC/PKCS5Padding", key); decryptFormat.setShouldInlineInitializationVector(true); from("direct:inline") .marshal(cryptoFormat) .to("mock:encrypted") .unmarshal(decryptFormat) .to("mock:unencrypted"); or with spring. <crypto id="inline" algorithm="DES/CBC/PKCS5Padding" keyRef="desKey" initVectorRef="initializationVector" inline="true" /> <crypto id="inline-decrypt" algorithm="DES/CBC/PKCS5Padding" keyRef="desKey" inline="true" /> For more information of the use of Initialization Vectors, consult
Hashed Message Authentication Codes (HMAC)To avoid attacks against the encrypted data while it is in transit the CryptoDataFormat can also calculate a Message Authentication Code forthe encrypted exchange contents based on a configurable MAC algorithm. The calculated HMAC is appended to the stream after encryption. It is separated from the stream in the decryption phase. The MAC is recalculated and verified against the transmitted version to insure nothing was tampered with in transit.For more information on Message Authentication Codes see http://en.wikipedia.org/wiki/HMAC KeyGenerator generator = KeyGenerator.getInstance("DES"); CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES", generator.generateKey()); cryptoFormat.setShouldAppendHMAC(true); from("direct:hmac") .marshal(cryptoFormat) .to("mock:encrypted") .unmarshal(cryptoFormat) .to("mock:unencrypted"); or with spring. <crypto id="hmac" algorithm="DES" keyRef="desKey" shouldAppendHMAC="true" />
By default the HMAC is calculated using the HmacSHA1 mac algorithm though this can be easily changed by supplying a different algorithm name. See [here] for how to check what algorithms are available through the configured security providers KeyGenerator generator = KeyGenerator.getInstance("DES"); CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES", generator.generateKey()); cryptoFormat.setShouldAppendHMAC(true); cryptoFormat.setMacAlgorithm("HmacMD5"); from("direct:hmac-algorithm") .marshal(cryptoFormat) .to("mock:encrypted") .unmarshal(cryptoFormat) .to("mock:unencrypted"); or with spring. <crypto id="hmac-algorithm" algorithm="DES" keyRef="desKey" macAlgorithm="HmacMD5" shouldAppendHMAC="true" />
Supplying Keys DynamicallyWhen using a Recipient list or similar EIP the recipient of an exchange can vary dynamically. Using the same key across all recipients may neither be feasible or desirable. It would be useful to be able to specify keys dynamically on a per exchange basis. The exchange could then be dynamically enriched with the key of its target recipient before being processed by the data format. To facilitate this the DataFormat allow for keys to be supplied dynamically via the message headers below
CryptoDataFormat cryptoFormat = new CryptoDataFormat("DES", null); /** * Note: the header containing the key should be cleared after * marshalling to stop it from leaking by accident and * potentially being compromised. The processor version below is * arguably better as the key is left in the header when you use * the DSL leaks the fact that camel encryption was used. */ from("direct:key-in-header-encrypt") .marshal(cryptoFormat) .removeHeader(CryptoDataFormat.KEY) .to("mock:encrypted"); from("direct:key-in-header-decrypt").unmarshal(cryptoFormat).process(new Processor() { public void process(Exchange exchange) throws Exception { exchange.getIn().getHeaders().remove(CryptoDataFormat.KEY); exchange.getOut().copyFrom(exchange.getIn()); } }).to("mock:unencrypted"); or with spring. <crypto id="nokey" algorithm="DES" />
PGPDataFormat Options
PGPDataFormat Message HeadersYou can override the PGPDataFormat options by applying below headers into message dynamically.
Encrypting with PGPDataFormatThe following sample uses the popular PGP format for encrypting/decrypting files using the Bouncy Castle Java libraries: // Public Key FileName String keyFileName = getKeyFileName(); // Private Key FileName String keyFileNameSec = getKeyFileNameSec(); // Keyring Userid Used to Encrypt String keyUserid = getKeyUserId(); // Private key password String keyPassword = getKeyPassword(); from("direct:inline") .marshal().pgp(keyFileName, keyUserid) .to("mock:encrypted") .unmarshal().pgp(keyFileNameSec, keyUserid, keyPassword) .to("mock:unencrypted"); The following sample performs signing + encryption, and then signature verification + decryption. It uses the same keyring for both signing and encryption, but you can obviously use different keys: PGPDataFormat pgpSignAndEncrypt = new PGPDataFormat(); pgpSignAndEncrypt.setKeyFileName(keyFileName); pgpSignAndEncrypt.setKeyUserid(keyUserid); pgpSignAndEncrypt.setSignatureKeyFileName(keyFileNameSec); pgpSignAndEncrypt.setSignatureKeyUserid(keyUserid); pgpSignAndEncrypt.setSignaturePassword(keyPassword); PGPDataFormat pgpVerifyAndDecrypt = new PGPDataFormat(); pgpVerifyAndDecrypt.setKeyFileName(keyFileNameSec); pgpVerifyAndDecrypt.setKeyUserid(keyUserid); pgpVerifyAndDecrypt.setPassword(keyPassword); pgpVerifyAndDecrypt.setSignatureKeyFileName(keyFileName); pgpVerifyAndDecrypt.setSignatureKeyUserid(keyUserid); from("direct:inline-sign") .marshal(pgpSignAndEncrypt) .to("mock:encrypted") .unmarshal(pgpVerifyAndDecrypt) .to("mock:unencrypted"); Or using Spring: <dataFormats> <!-- will load the file from classpath by default, but you can prefix with file: to load from file system --> <pgp id="encrypt" keyFileName="org/apache/camel/component/crypto/pubring.gpg" keyUserid="sdude@nowhere.net"/> <pgp id="decrypt" keyFileName="org/apache/camel/component/crypto/secring.gpg" keyUserid="sdude@nowhere.net" password="sdude"/> </dataFormats> <route> <from uri="direct:inline"/> <marshal ref="encrypt"/> <to uri="mock:encrypted"/> <unmarshal ref="decrypt"/> <to uri="mock:unencrypted"/> </route> To work with the previous example you need the following
Managing your keyringTo manage the keyring, I use the command line tools, I find this to be the simplest approach in managing the keys. There are also Java libraries available from http://www.bouncycastle.org/java.html if you would prefer to do it that way.
DependenciesTo use the Crypto dataformat in your camel routes you need to add the following dependency to your pom. <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-crypto</artifactId> <version>2.9.0</version> </dependency> See AlsoSyslog DataFormatAvailable as of Camel 2.6 The syslog dataformat is used for working with RFC3164 messages. This component supports the following:
Maven users will need to add the following dependency to their pom.xml for this component: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-syslog</artifactId> <version>x.x.x</version> <!-- use the same version as your Camel core version --> </dependency> RFC3164 Syslog protocolSyslog uses the user datagram protocol (UDP) [1] as its underlying transport layer mechanism. To expose a Syslog listener service we reuse the existing camel-mina component or camel-netty where we just use the Rfc3164SyslogDataFormat to marshal and unmarshal messages Exposing a Syslog listenerIn our Spring XML file, we configure an endpoint to listen for udp messages on port 10514, note that in netty we disable the defaultCodec, this <camelContext id="myCamel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <syslog id="mySyslog"/> </dataFormats> <route> <from uri="netty:udp://localhost:10514?sync=false&allowDefaultCodec=false"/> <unmarshal ref="mySyslog"/> <to uri="mock:stop1"/> </route> </camelContext> The same route using camel-mina <camelContext id="myCamel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <syslog id="mySyslog"/> </dataFormats> <route> <from uri="mina:udp://localhost:10514"/> <unmarshal ref="mySyslog"/> <to uri="mock:stop1"/> </route> </camelContext> Sending syslog messages to a remote destination<camelContext id="myCamel" xmlns="http://camel.apache.org/schema/spring"> <dataFormats> <syslog id="mySyslog"/> </dataFormats> <route> <from uri="direct:syslogMessages"/> <marshal ref="mySyslog"/> <to uri="mina:udp://remotehost:10514"/> </route> </camelContext> See Also |