BeanIO ItemReader and ItemWriter

jberet-support module includes beanIOItemReader and beanIOItemWriter that employs BeanIO to read and write data in various formats that are supported by BeanIO, e.g., fixed length file, CSV file, XML, etc. They also support restart, ranged reading, custom error handler, and dynamic BeanIO mapping properties.

The following BeanIO dependencies are required for beanIOItemReader and beanIOItemWriter:

<dependency>
    <groupId>org.beanio</groupId>
    <artifactId>beanio</artifactId>
    <version>${version.org.beanio}</version>
    <scope>compile</scope>
</dependency>
<dependency>
    <groupId>org.ow2.asm</groupId>
    <artifactId>asm</artifactId>
    <version>${version.org.ow2.asm}</version>
</dependency>

Batch Configuration Properties in Job XML

beanIOItemReader and beanIOItemWriter are configured through <reader> or <writer> batch properties in job xml. All properties are of type String, unless noted otherwise. The following is an example job xml that references beanIOItemReader and beanIOItemWriter:

<job id="BeanIOReaderWriterTest" xmlns="http://xmlns.jcp.org/xml/ns/javaee" version="1.0">
    <step id="BeanIOReaderWriterTest.step1">
        <chunk item-count="1000000">
            <reader ref="beanIOItemReader">
                <properties>
                    <property name="resource" value="#{jobParameters['resource']}"/>
                    <property name="start" value="1"/>
                    <property name="end" value="1000"/>
                    <property name="streamName" value="persons"/>
                    <property name="streamMapping" value="person-beanio-mapping.xml"/>
                    <property name="mappingProperties" value="zipCodeFieldName=zipCode, zipCodeFieldType=string"/>
                    <property name="errorHandler" value="org.beanio.BeanReaderErrorHandlerSupport"/>
                    <!--<property name="locale" value="en_US"/>-->
                    <property name="charset" value="UTF-8"/>
                </properties>
            </reader>

            <writer ref="beanIOItemWriter">
                <properties>
                    <property name="resource" value="#{jobParameters['writeResource']}"/>
                    <property name="streamName" value="persons"/>
                    <property name="streamMapping" value="person-beanio-mapping.xml"/>
                    <property name="mappingProperties" value="zipCodeFieldName=zipCode, zipCodeFieldType=string"/>
                    <property name="charset" value="UTF-8"/>
                </properties>
            </writer>
        </chunk>
    </step>
</job>

The following is a sample fixed length flat file (downloaded from http://star.cde.ca.gov/star2013/research_fixfileformat.aspx), and its BeanIO mapping file, referenced in job xml as streamMapping property:

000000000000000000201304State of California
010000000000000000201305Alameda
011001700000000000201306Alameda                                           Alameda County Office of Education
011001701098350728201309Alameda                                           FAME Public Charter                               FAME Public Charter                               94560
011001701126070811201309Alameda                                           Envision Academy for Arts & T                     Envision Academy for Arts & T                     94612
011001701184891049201309Alameda                                           Aspire California College Prep                    Aspire California College Prep                    94606

<beanio xmlns="http://www.beanio.org/2012/03"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
    <stream name="star-entities" format="fixedlength">
        <record name="star-entity" class="org.jberet.support.io.CaliforniaStarEntity">
            <field name="countyCode" length="2"/>
            <field name="districtCode" length="5"/>
            <field name="schoolCode" length="7"/>
            <field name="charterNumber" length="4"/>
            <field name="testYear" length="4"/>
            <field name="typeId" length="2"/>
            <field name="countyName" length="50" />
            <field name="districtName" length="50"/>
            <field name="schoolName" length="50"/>
            <field name="${zipCodeFieldName}" type="${zipCodeFieldType}" length="5"/>
        </record>
    </stream>
</beanio>

Another sample BeanIO mapping file for person data file with "|" as the delimiter:

Number|Gender|Title|GivenName|MiddleInitial|Surname|StreetAddress|City|State|ZipCode|Country|CountryFull|EmailAddress|Username|Password|TelephoneNumber|MothersMaiden|Birthday|CCType|CCNumber|CVV2|CCExpires|NationalID|UPS|Color|Occupation|Company|Vehicle|Domain|BloodType|Pounds|Kilograms|FeetInches|Centimeters|GUID|Latitude|Longitude
1|female|Mrs.|Susannah|R|Pemberton|1585 Court Street|Maryland Heights|MO|63141|US|United States|[email protected]|Turs1956|eiGufib3th|636-898-6411|Andrews|11/5/1956|MasterCard|5112913274177177|156|9/2015|490-10-6886|1Z 4Y0 424 93 3839 606 7|Purple|Sampler|Forth & Towne|2002 MCC Smart|RentHoldings.com|O+|218.7|99.4|5' 1"|156|c738a629-5108-47b1-a168-8ab6d1438e50|38.728008|-90.37838
2|male|Mr.|Anthony|J|Quirion|4928 Carson Street|San Diego|CA|92103|US|United States|[email protected]|Quoinep|ew2Keewoh|858-856-8403|Ferguson|7/5/1979|MasterCard|5533216847446447|542|1/2015|622-80-6651|1Z W28 086 52 9104 825 3|Orange|Shampooer|JumboSports|1993 Mazda 121|PoliticalPlanners.com|B+|140.8|64.0|5' 11"|180|79b9a58b-ada9-4fab-8093-d0f17c541cf1|32.831002|-117.113794
3|male|Mr.|Shaun|P|Schwartz|1246 Kyle Street|Grand Island|NE|68801|US|United States|[email protected]|Sivend88|saeKooca8r|308-615-8773|Guevara|10/23/1988|MasterCard|5562020815274730|454|10/2019|507-90-6312|1Z 96V V28 41 7598 869 9|White|Lease operator|Grass Roots Yard Services|2000 BMW 528|StationCard.com|O+|191.4|87.0|5' 10"|179|ebaa641c-0344-45aa-9649-b5ae134e9e61|40.897922|-98.182374
4|male|Mr.|Michael|B|Petrey|3875 Jehovah Drive|Spotsylvania|VA|22553|US|United States|[email protected]|Thibust|ooho3Ocoh|540-507-6775|Weaver|12/26/1972|MasterCard|5420696909140416|637|5/2016|230-80-7928|1Z 8V1 528 75 9778 835 3|Red|Telecommunications specialist|Great American Music|1995 Ford Ranger|EmploymentCars.com|B+|165.9|75.4|6' 0"|182|dbc2c6d6-d967-4908-be39-2f4b22ba2c17|38.118885|-77.706684

<beanio xmlns="http://www.beanio.org/2012/03"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">

    <stream name="persons" format="delimited" ignoreUnidentifiedRecords="true">
        <parser>
            <property name="delimiter" value="|"/>
            <property name="comments" value="#"/>
        </parser>
        <record name="person" class="org.jberet.support.io.Person" occurs="0+">
            <field name="number"/>
            <field name="gender" />
            <field name="title" />
            <field name="givenName" />
            <field name="middleInitial" />
            <field name="surname" />
            <field name="streetAddress" />
            <field name="city" />
            <field name="state" />
            <field name="zipCode" />
            <field name="country" />
            <field name="countryFull" />
            <field name="emailAddress" />
            <field name="username" />
            <field name="password" />
            <field name="telephoneNumber" />
            <field name="mothersMaiden" />
            <field name="birthday" format="M/d/YYYY"/>
            <field name="CCType" />
            <field name="CCNumber" />
            <field name="CVV2" />
            <field name="CCExpires" />
            <field name="nationalID" />
            <field name="UPS" />
            <field name="color" />
            <field name="occupation" />
            <field name="company" />
            <field name="vehicle" />
            <field name="domain" />
            <field name="bloodType" />
            <field name="pounds" />
            <field name="kilograms" />
            <field name="feetInches" />
            <field name="centimeters" />
            <field name="GUID" />
            <field name="latitude" />
            <field name="longitude" />
        </record>
    </stream>
</beanio>

Batch Properties for Both `beanIOItemReader` and `beanIOItemWriter`

resource

The resource to read from (for batch readers), or write to (for batch writers). Some reader or writer implementations may choose to ignore this property and instead use other properties that are more appropritate.

streamName

Name of the BeanIO stream defined in BeanIO mapping file. It corresponds to the batch job property.

streamMapping

Location of the BeanIO mapping file, which can be a file path, a URL, or a resource loadable by the current class loader.

streamFactoryLookup

JNDI name for looking up org.beanio.StreamFactory when running in application server. When streamFactoryLookup property is specified in job xml and hence injected into batch reader or writer class, org.beanio.StreamFactory will be looked up with JNDI, and streamMapping and mappingProperties will be ignored.

mappingProperties

java.util.Map

User properties that can be used for property substitution in BeanIO mapping file. When used with batch job JSL properties, they provide dynamic BeanIO mapping attributes. For example,

In batch job client class, set the properties values as comma-separated key-value pairs:

params.setProperty("mappingProperties", "zipCodeFieldName=zipCode, zipCodeFieldType=string");

In job xml file, make the properties available for beanIOItemReader or beanIOItemWriter via @BatchProperty injection:
```
<property name="mappingProperties" value="#{jobParameters['mappingProperties']}"/>
```

In BeanIO mapping file, reference the properties defined above:

<field name="${zipCodeFieldName}" type="${zipCodeFieldType}" length="5"/>

charset

The name of the character set to be used for reading and writing data, e.g., UTF-8. This property is optional, and if not set, the platform default charset is used.

Batch Properties for `beanIOItemReader` Only

In addition to the above common properties, beanIOItemReader also supports the following batch properties in job xml:

skipBeanValidation

boolean

Indicates whether the current batch reader will invoke Bean Validation API to validate the incoming data POJO. Optional property and defaults to false, i.e., the reader will validate data POJO bean where appropriate.

start

int

A positive integer indicating the start position in the input resource. It is optional and defaults to 1 (starting from the 1st data item).

end

int

A positive integer indicating the end position in the input resource. It is optional and defaults to Integer.MAX_VALUE.

errorHandler

java.lang.Class

A class implementing BeanReaderErrorHandler for handling exceptions thrown by a BeanReader.

locale

The locale name for this beanIOItemReader

Batch Properties for `beanIOItemWriter` Only

None

BeanIO ItemReader and ItemWriter

BeanIO ItemReader and ItemWriter

Batch Configuration Properties in Job XML

Batch Properties for Both `beanIOItemReader` and `beanIOItemWriter`

resource

streamName

streamMapping

streamFactoryLookup

mappingProperties

charset

Batch Properties for `beanIOItemReader` Only

skipBeanValidation

start

end

errorHandler

locale

Batch Properties for `beanIOItemWriter` Only

results matching ""

No results matching ""

BeanIO ItemReader and ItemWriter

Batch Configuration Properties in Job XML

Batch Properties for Both beanIOItemReader and beanIOItemWriter

resource

streamName

streamMapping

streamFactoryLookup

mappingProperties

charset

Batch Properties for beanIOItemReader Only

skipBeanValidation

start

end

errorHandler

locale

Batch Properties for beanIOItemWriter Only

results matching ""

No results matching ""

Batch Properties for Both `beanIOItemReader` and `beanIOItemWriter`

Batch Properties for `beanIOItemReader` Only

Batch Properties for `beanIOItemWriter` Only