BindyAvailable as of Camel 2.0 The idea that the developers has followed to design this component was to allow the parsing/binding of non structured data (or to be more precise non-XML data)
to one or many Plain Old Java Object (POJO) and to convert the data according to the type of the java property. POJO can be linked together and relation one to many is available in some cases. Moreover, for data type like Date, Double, Float, Integer, Short, Long and BigDecimal, you can provide the pattern to apply during the formatting of the property. For the BigDecimal number, you can also define the precision and the decimal or grouping separators
Decimal* = Double, Integer, Float, Short, Long
To work with camel-bindy, you must first define your model in a package (e.g. com.acme.model) and for each model class (e.g. Order, Client, Instrument, ...) associate the required annotations (described hereafter) with Class or property name. AnnotationsThe annotations created allow to map different concept of your model to the POJO like :
This section will describe them : 1. CsvRecordThe CsvRecord annotation is used to identified the root class of the model. It represents a record = a line of a CSV file and can be linked to several children model classes.
case 1 : separator = ',' The separator used to segregate the fields in the CSV record is ',' : 10, J, Pauline, M, XD12345678, Fortis Dynamic 15/15, 2500, USD,08-01-2009 @CsvRecord( separator = "," ) public Class Order { ... } case 2 : separator = ';' Compare to the previous case, the separator here is ';' instead of ',' : 10; J; Pauline; M; XD12345678; Fortis Dynamic 15/15; 2500; USD; 08-01-2009 @CsvRecord( separator = ";" ) public Class Order { ... } case 3 : separator = '|' Compare to the previous case, the separator here is '|' instead of ';' : 10| J| Pauline| M| XD12345678| Fortis Dynamic 15/15| 2500| USD| 08-01-2009 @CsvRecord( separator = "\\|" ) public Class Order { ... } case 4 : separator = '\",\"' When the field to be parsed of the CSV record contains ',' or ';' which is also used as separator, we whould find another strategy "10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009" @CsvRecord( separator = "\",\"" ) public Class Order { ... } From Camel 2.8.3/2.9 or never bindy will automatic detect if the record is enclosed with either single or double quotes and automatic remove those quotes when unmarshalling from CSV to Object. Therefore do not include the quotes in the separator, but simple do as below: "10","J","Pauline"," M","XD12345678","Fortis Dynamic 15,15" 2500","USD","08-01-2009" @CsvRecord( separator = "," ) public Class Order { ... } Notice that if you want to marshal from Object to CSV and use quotes, then you need to specify which quote character to use, using the quote attribute on the @CsvRecord as shown below: @CsvRecord( separator = ",", quote = "\"" ) public Class Order { ... } case 5 : separator & skipfirstline The feature is interesting when the client wants to have in the first line of the file, the name of the data fields : order id, client id, first name, last name, isin code, instrument name, quantity, currency, date To inform bindy that this first line must be skipped during the parsing process, then we use the attribute : @CsvRecord(separator = ",", skipFirstLine = true) public Class Order { ... } case 6 : generateHeaderColumns To add at the first line of the CSV generated, the attribute generateHeaderColumns must be set to true in the annotation like this : @CsvRecord( generateHeaderColumns = true ) public Class Order { ... } As a result, Bindy during the unmarshaling process will generate CSV like this : order id, client id, first name, last name, isin code, instrument name, quantity, currency, date case 7 : carriage return If the platform where camel-bindy will run is not Windows but Macintosh or Unix, than you can change the crlf property like this. Three values are available : WINDOWS, UNIX or MAC @CsvRecord(separator = ",", crlf="MAC") public Class Order { ... } case 8 : isOrdered Sometimes, the order to follow during the creation of the CSV record from the model is different from the order used during the parsing. Then, in this case, we can use the attribute isOrdered = true to indicate this in combination with attribute 'position' of the DataField annotation. @CsvRecord(isOrdered = true) public Class Order { @DataField(pos = 1, position = 11) private int orderNr; @DataField(pos = 2, position = 10) private String clientNr; ... } Remark : pos is used to parse the file, stream while positions is used to generate the CSV 2. LinkThe link annotation will allow to link objects together.
e.g : If the model Class Client is linked to the Order class, then use annotation Link in the Order class like this : Property Link @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @Link private Client client; ... AND for the class Client : Class Link
@Link
public class Client {
...
}
3. DataFieldThe DataField annotation defines the property of the field. Each datafield is identified by its position in the record, a type (string, int, date, ...) and optionally of a pattern
case 1 : pos This parameter/attribute represents the position of the field in the csv record Position @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 5) private String isinCode; ... } As you can see in this example the position starts at '1' but continues at '5' in the class Order. The numbers from '2' to '4' are defined in the class Client (see here after). Position continues in another model class public class Client { @DataField(pos = 2) private String clientNr; @DataField(pos = 3) private String firstName; @DataField(pos = 4) private String lastName; ... } case 2 : pattern The pattern allows to enrich or validates the format of your data Pattern @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 5) private String isinCode; @DataField(name = "Name", pos = 6) private String instrumentName; @DataField(pos = 7, precision = 2) private BigDecimal amount; @DataField(pos = 8) private String currency; @DataField(pos = 9, pattern = "dd-MM-yyyy") -- pattern used during parsing or when the date is created private Date orderDate; ... } case 3 : precision The precision is helpful when you want to define the decimal part of your number Precision @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @Link private Client client; @DataField(pos = 5) private String isinCode; @DataField(name = "Name", pos = 6) private String instrumentName; @DataField(pos = 7, precision = 2) -- precision private BigDecimal amount; @DataField(pos = 8) private String currency; @DataField(pos = 9, pattern = "dd-MM-yyyy") private Date orderDate; ... } case 4 : Position is different in output The position attribute will inform bindy how to place the field in the CSV record generated. By default, the position used corresponds to the position defined with the attribute 'pos'. If the position is different (that means that we have an asymetric processus comparing marshaling from unmarshaling) than we can use 'position' to indicate this. Here is an example Position is different in output @CsvRecord(separator = ",") public class Order { @CsvRecord(separator = ",", isOrdered = true) public class Order { // Positions of the fields start from 1 and not from 0 @DataField(pos = 1, position = 11) private int orderNr; @DataField(pos = 2, position = 10) private String clientNr; @DataField(pos = 3, position = 9) private String firstName; @DataField(pos = 4, position = 8) private String lastName; @DataField(pos = 5, position = 7) private String instrumentCode; @DataField(pos = 6, position = 6) private String instrumentNumber; ... }
case 5 : required If a field is mandatory, simply use the attribute 'required' setted to true Required @CsvRecord(separator = ",") public class Order { @DataField(pos = 1) private int orderNr; @DataField(pos = 2, required = true) private String clientNr; @DataField(pos = 3, required = true) private String firstName; @DataField(pos = 4, required = true) private String lastName; ... } If this field is not present in the record, than an error will be raised by the parser with the following information : Some fields are missing (optional or mandatory), line : case 6 : trim If a field has leading and/or trailing spaces which should be removed before they are processed, simply use the attribute 'trim' setted to true Trim @CsvRecord(separator = ",") public class Order { @DataField(pos = 1, trim = true) private int orderNr; @DataField(pos = 2, trim = true) private Integer clientNr; @DataField(pos = 3, required = true) private String firstName; @DataField(pos = 4) private String lastName; ... } 4. FixedLengthRecordThe FixedLengthRecord annotation is used to identified the root class of the model. It represents a record = a line of a file/message containing data fixed length formatted
case 1 : Simple fixed length record This simple example shows how to design the model to parse/format a fixed message 10A9PaulineMISINXD12345678BUYShare2500.45USD01-08-2009 Fixed-simple
@FixedLengthRecord(length=54, paddingChar=' ')
public static class Order {
@DataField(pos = 1, length=2)
private int orderNr;
@DataField(pos = 3, length=2)
private String clientNr;
@DataField(pos = 5, length=7)
private String firstName;
@DataField(pos = 12, length=1, align="L")
private String lastName;
@DataField(pos = 13, length=4)
private String instrumentCode;
@DataField(pos = 17, length=10)
private String instrumentNumber;
@DataField(pos = 27, length=3)
private String orderType;
@DataField(pos = 30, length=5)
private String instrumentType;
@DataField(pos = 35, precision = 2, length=7)
private BigDecimal amount;
@DataField(pos = 42, length=3)
private String currency;
@DataField(pos = 45, length=10, pattern = "dd-MM-yyyy")
private Date orderDate;
...
case 2 : Fixed length record with alignment and padding This more elaborated example show how to define the alignment for a field and how to assign a padding character which is ' ' here'' 10A9 PaulineM ISINXD12345678BUYShare2500.45USD01-08-2009 Fixed-padding-align
@FixedLengthRecord(length=60, paddingChar=' ')
public static class Order {
@DataField(pos = 1, length=2)
private int orderNr;
@DataField(pos = 3, length=2)
private String clientNr;
@DataField(pos = 5, length=9)
private String firstName;
@DataField(pos = 14, length=5, align="L") // align text to the LEFT zone of the block
private String lastName;
@DataField(pos = 19, length=4)
private String instrumentCode;
@DataField(pos = 23, length=10)
private String instrumentNumber;
@DataField(pos = 33, length=3)
private String orderType;
@DataField(pos = 36, length=5)
private String instrumentType;
@DataField(pos = 41, precision = 2, length=7)
private BigDecimal amount;
@DataField(pos = 48, length=3)
private String currency;
@DataField(pos = 51, length=10, pattern = "dd-MM-yyyy")
private Date orderDate;
...
case 3 : Field padding Sometimes, the default padding defined for record cannnot be applied to the field as we have a number format where we would like to padd with '0' instead of ' '. In this case, you can use in the model the attribute paddingField to set this value. 10A9 PaulineM ISINXD12345678BUYShare000002500.45USD01-08-2009 Fixed-padding-field
@FixedLengthRecord(length = 65, paddingChar = ' ')
public static class Order {
@DataField(pos = 1, length = 2)
private int orderNr;
@DataField(pos = 3, length = 2)
private String clientNr;
@DataField(pos = 5, length = 9)
private String firstName;
@DataField(pos = 14, length = 5, align = "L")
private String lastName;
@DataField(pos = 19, length = 4)
private String instrumentCode;
@DataField(pos = 23, length = 10)
private String instrumentNumber;
@DataField(pos = 33, length = 3)
private String orderType;
@DataField(pos = 36, length = 5)
private String instrumentType;
@DataField(pos = 41, precision = 2, length = 12, paddingChar = '0')
private BigDecimal amount;
@DataField(pos = 53, length = 3)
private String currency;
@DataField(pos = 56, length = 10, pattern = "dd-MM-yyyy")
private Date orderDate;
...
5. MessageThe Message annotation is used to identified the class of your model who will contain key value pairs fields. This kind of format is used mainly in Financial Exchange Protocol Messages (FIX). Nevertheless, this annotation can be used for any other format where data are identified by keys. The key pair values are separated each other by a separator which can be a special character like a tab delimitor (unicode representation : \u0009) or a start of heading (unicode representation : \u0001)
case 1 : separator = 'u0001' The separator used to segregate the key value pair fields in a FIX message is the ASCII '01' character or in unicode format '\u0001'. This character must be escaped a second time to avoid a java runtime error. Here is an example : 8=FIX.4.1 9=20 34=1 35=0 49=INVMGR 56=BRKR 1=BE.CHM.001 11=CHM0001-01 22=4 ... and how to use the annotation FIX - message @Message(keyValuePairSeparator = "=", pairSeparator = "\u0001", type="FIX", version="4.1") public class Order { ... }
6. KeyValuePairFieldThe KeyValuePairField annotation defines the property of a key value pair field. Each KeyValuePairField is identified by a tag (= key) and its value associated, a type (string, int, date, ...), optionaly a pattern and if the field is required
case 1 : tag This parameter represents the key of the field in the message FIX message - Tag @Message(keyValuePairSeparator = "=", pairSeparator = "\u0001", type="FIX", version="4.1") public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1) // Client reference private String Account; @KeyValuePairField(tag = 11) // Order reference private String ClOrdId; @KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...) private String IDSource; @KeyValuePairField(tag = 48) // Fund code private String SecurityId; @KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell) private String Side; @KeyValuePairField(tag = 58) // Free text private String Text; ... } case 2 : Different position in output If the tags/keys that we will put in the FIX message must be sorted according to a predefine order, then use the attribute 'position' of the annotation @KeyValuePairField FIX message - Tag - sort @Message(keyValuePairSeparator = "=", pairSeparator = "\\u0001", type = "FIX", version = "4.1", isOrdered = true) public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1, position = 1) // Client reference private String account; @KeyValuePairField(tag = 11, position = 3) // Order reference private String clOrdId; ... } 7. SectionIn FIX message of fixed length records, it is common to have different sections in the representation of the information : header, body and section. The purpose of the annotation @Section is to inform bindy about which class of the model represents the header (= section 1), body (= section 2) and footer (= section 3) Only one attribute/parameter exists for this annotation.
case 1 : Section A. Definition of the header section FIX message - Section - Header @Section(number = 1) public class Header { @KeyValuePairField(tag = 8, position = 1) // Message Header private String beginString; @KeyValuePairField(tag = 9, position = 2) // Checksum private int bodyLength; ... } B. Definition of the body section FIX message - Section - Body @Section(number = 2) @Message(keyValuePairSeparator = "=", pairSeparator = "\\u0001", type = "FIX", version = "4.1", isOrdered = true) public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1, position = 1) // Client reference private String account; @KeyValuePairField(tag = 11, position = 3) // Order reference private String clOrdId; C. Definition of the footer section FIX message - Section - Footer @Section(number = 3) public class Trailer { @KeyValuePairField(tag = 10, position = 1) // CheckSum private int checkSum; public int getCheckSum() { return checkSum; } 8. OneToManyThe purpose of the annotation @OneToMany is to allow to work with a List<?> field defined a POJO class or from a record containing repetitive groups.
The relation OneToMany ONLY WORKS in the following cases :
case 1 : Generating CSV with repetitive data Here is the CSV output that we want : Claus,Ibsen,Camel in Action 1,2010,35 Remark : the repetitive data concern the title of the book and its publication date while first, last name and age are common and the classes used to modeling this. The Author class contains a List of Book. Generate CSV with repetitive data @CsvRecord(separator=",") public class Author { @DataField(pos = 1) private String firstName; @DataField(pos = 2) private String lastName; @OneToMany private List<Book> books; @DataField(pos = 5) private String Age; ... public class Book { @DataField(pos = 3) private String title; @DataField(pos = 4) private String year; Very simple isn't it !!! case 2 : Reading FIX message containing group of tags/keys Here is the message that we would like to process in our model : "8=FIX 4.19=2034=135=049=INVMGR56=BRKR" tags 22, 48 and 54 are repeated and the code Reading FIX message containing group of tags/keys public class Order { @Link Header header; @Link Trailer trailer; @KeyValuePairField(tag = 1) // Client reference private String account; @KeyValuePairField(tag = 11) // Order reference private String clOrdId; @KeyValuePairField(tag = 58) // Free text private String text; @OneToMany(mappedTo = "org.apache.camel.dataformat.bindy.model.fix.complex.onetomany.Security") List<Security> securities; ... public class Security { @KeyValuePairField(tag = 22) // Fund ID type (Sedol, ISIN, ...) private String idSource; @KeyValuePairField(tag = 48) // Fund code private String securityCode; @KeyValuePairField(tag = 54) // Movement type ( 1 = Buy, 2 = sell) private String side; Using the Java DSLThe next step consists in instantiating the DataFormat bindy class associated with this record type and providing Java package name(s) as parameter. For example the following uses the class CsvBindyFormat (who correspond to the class associated with the CSV record type) which is configured with "com.acme.model" DataFormat bindy = new CsvBindyDataFormat("com.acme.model"); from("file://inbox"). unmarshal(bindy). to("bean:handleOrder"); The Camel route will pick-up files in the inbox directory, unmarshall CSV records in a collection of model objects and send the collection The collection is a list of Map. Each Map of the list contains the objects of the model. Each object can be retrieve using its class name. int count = 0; List<Map<String, Object>> models = new ArrayList<Map<String, Object>>(); Map<String, Object> model = new HashMap<String, Object>(); models = (List<Map<String, Object>>) exchange.getIn().getBody(); Iterator<Map<String, Object>> it = models.iterator(); while(it.hasNext()){ model = it.next(); for(String key : model.keySet()) { Object obj = model.get(key); LOG.info("Count : " + count + ", " + obj.toString()); } count++; } LOG.info("Nber of CSV records received by the csv bean : " + count); To generate CSV records from a collection of model objects, you create the following route : from("bean:handleOrder") marshal(bindy) to("file://outbox") You can if you prefer use a named reference to a data format which can then be defined in your Registry such as via your Spring XML file. e.g. from("file://inbox"). unmarshal("myBindyDataFormat"). to("bean:handleOrder"); Unit testHere is two examples showing how to marshall or unmarshall a CSV file with Camel Marshall package org.apache.camel.dataformat.bindy.csv; import java.math.BigDecimal; import java.util.ArrayList; import java.util.Calendar; import java.util.GregorianCalendar; import java.util.HashMap; import java.util.List; import java.util.Map; import org.apache.camel.EndpointInject; import org.apache.camel.Produce; import org.apache.camel.ProducerTemplate; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink.Client; import org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink.Order; import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration; import org.junit.Test; import org.springframework.config.java.annotation.Bean; import org.springframework.config.java.annotation.Configuration; import org.springframework.config.java.test.JavaConfigContextLoader; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; @ContextConfiguration(locations = "org.apache.camel.dataformat.bindy.csv.BindyComplexCsvMarshallTest$ContextConfig", loader = JavaConfigContextLoader.class) public class BindyComplexCsvMarshallTest extends AbstractJUnit4SpringContextTests { private List<Map<String, Object>> models = new ArrayList<Map<String, Object>>(); private String result = "10,A1,Julia,Roberts,BE123456789,Belgium Ventage 10/12,150,USD,14-01-2009"; @Produce(uri = "direct:start") private ProducerTemplate template; @EndpointInject(uri = "mock:result") private MockEndpoint resultEndpoint; @Test public void testMarshallMessage() throws Exception { resultEndpoint.expectedBodiesReceived(result); template.sendBody(generateModel()); resultEndpoint.assertIsSatisfied(); } private List<Map<String, Object>> generateModel() { Map<String, Object> model = new HashMap<String, Object>(); Order order = new Order(); order.setOrderNr(10); order.setAmount(new BigDecimal("150")); order.setIsinCode("BE123456789"); order.setInstrumentName("Belgium Ventage 10/12"); order.setCurrency("USD"); Calendar calendar = new GregorianCalendar(); calendar.set(2009, 0, 14); order.setOrderDate(calendar.getTime()); Client client = new Client(); client.setClientNr("A1"); client.setFirstName("Julia"); client.setLastName("Roberts"); order.setClient(client); model.put(order.getClass().getName(), order); model.put(client.getClass().getName(), client); models.add(0, model); return models; } @Configuration public static class ContextConfig extends SingleRouteCamelConfiguration { BindyCsvDataFormat camelDataFormat = new BindyCsvDataFormat("org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink"); @Override @Bean public RouteBuilder route() { return new RouteBuilder() { @Override public void configure() { from("direct:start").marshal(camelDataFormat).to("mock:result"); } }; } } } Unmarshall package org.apache.camel.dataformat.bindy.csv; import org.apache.camel.EndpointInject; import org.apache.camel.builder.RouteBuilder; import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.spring.javaconfig.SingleRouteCamelConfiguration; import org.junit.Test; import org.springframework.config.java.annotation.Bean; import org.springframework.config.java.annotation.Configuration; import org.springframework.config.java.test.JavaConfigContextLoader; import org.springframework.test.context.ContextConfiguration; import org.springframework.test.context.junit4.AbstractJUnit4SpringContextTests; @ContextConfiguration(locations = "org.apache.camel.dataformat.bindy.csv.BindyComplexCsvUnmarshallTest$ContextConfig", loader = JavaConfigContextLoader.class) public class BindyComplexCsvUnmarshallTest extends AbstractJUnit4SpringContextTests { @EndpointInject(uri = "mock:result") private MockEndpoint resultEndpoint; @Test public void testUnMarshallMessage() throws Exception { resultEndpoint.expectedMessageCount(1); resultEndpoint.assertIsSatisfied(); } @Configuration public static class ContextConfig extends SingleRouteCamelConfiguration { BindyCsvDataFormat csvBindyDataFormat = new BindyCsvDataFormat("org.apache.camel.dataformat.bindy.model.complex.twoclassesandonelink"); @Override @Bean public RouteBuilder route() { return new RouteBuilder() { @Override public void configure() { from("file://src/test/data?noop=true").unmarshal(csvBindyDataFormat).to("mock:result"); } }; } } } In this example, BindyCsvDataFormat class has been instantiated in a traditional way but it is also possible to provide information directly to the function (un)marshal like this where BindyType corresponds to the Bindy DataFormat class to instantiate and the parameter contains the list of package names.
public static class ContextConfig extends SingleRouteCamelConfiguration {
@Override
@Bean
public RouteBuilder route() {
return new RouteBuilder() {
@Override
public void configure() {
from("direct:start")
.marshal().bindy(BindyType.Csv, "org.apache.camel.dataformat.bindy.model.simple.oneclass")
.to("mock:result");
}
};
}
}
Using Spring XMLThis is really easy to use Spring as your favorite DSL language to declare the routes to be used for camel-bindy. The following example shows two routes where the first will pick-up records from files, unmarshal the content and bind it to their model. The result is then send to a pojo (doing nothing special) and place them into a queue. The second route will extract the pojos from the queue and marshal the content to generate a file containing the csv record spring dsl <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd"> <bean id="bindyDataformat" class="org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat"> <constructor-arg value="org.apache.camel.bindy.model" /> </bean> <bean id="csv" class="org.apache.camel.bindy.csv.HandleOrderBean" /> <!-- Queuing engine - ActiveMq - work locally in mode virtual memory --> <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent"> <property name="brokerURL" value="vm://localhost:61616"/> </bean> <camelContext xmlns="http://camel.apache.org/schema/spring"> <jmxAgent id="agent" disabled="false" /> <route> <from uri="file://src/data/csv/?noop=true" /> <unmarshal ref="bindyDataformat" /> <to uri="bean:csv" /> <to uri="activemq:queue:in" /> </route> <route> <from uri="activemq:queue:in" /> <marshal ref="bindyDataformat" /> <to uri="file://src/data/csv/out/" /> </route> </camelContext> </beans>
DependenciesTo use Bindy in your camel routes you need to add the a dependency on camel-bindy which implements this data format. If you use maven you could just add the following to your pom.xml, substituting the version number for the latest & greatest release (see the download page for the latest versions). <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-bindy</artifactId> <version>2.1.0</version> </dependency> |