En batch
From The Real BLU AGE® Wiki
Modeling Batch with Blu Age for Websphere Computer Grid Technology
Document Version: 1.0
Blu Age version: 3.6.3 SP1
Introduction
Object
This document is the user guide for the Blu Age Forward application related to Batch Generation for IBM Websphere Compute Grid technology. It explains how to model Batch with the Blu Age Forward tool.
Before reading this document you may read the " Blu Age forward User Guide" available on wiki.bluage.com.
Audience
This document is particularly addressed to technical teams in charge of software architecture as well as project managers. It is then devoted to development teams focused on batch implementation.
Minimum prerequisites
This document requires expert skills on software engineering techniques and more precisely specific knowledge in the following: MDA, UML, profile UML. Reader must also understand the Batch principles and know at least one application server able to execute batch on modern technology: IBM Websphere Compute Grid or SpringBatch.
Conventions
In the document, following icons mean
A trick or a tip.
A warning or a point of attention.
Table of figures
- Figure 1: Package organization for main batch application media:Bth_001.png
- Figure 2: Profiles used in Blu Age batch models media:Bth_002.png
- Figure 3: Items model using entities and BO media:Bth_003.png
- Figure 4: Discriminator usage sample media:Bth_004.png
- Figure 5: Sample step model media:Bth_005.png
- Figure 6: Sample for step organization media:Bth_006.png
- Figure 7: Step action EBCDIC Reader media:Bth_007.png
- Figure 8: Step action Flat File Reader media:Bth_008.png
- Figure 9: Step action JDBC Reader media:Bth_009.png
- Figure 10: Step action JPA Reader, including fetched elements media:Bth_010.png
- Figure 11: Entity diagram related to the JPA reader in Figure 10 media:Bth_011.png
- Figure 12: Applying GroupReader stereotype on a reader media:Bth_012.png
- Figure 13: Step action EBCDIC writer media:Bth_013.png
- Figure 14: Step action flat file writer media:Bth_014.png
- Figure 15: Step action JDBC writer media:Bth_015.png
- Figure 16: Step action SQL Script Writer media:Bth_016.png
- Figure 17: Condition applied to writer media:Bth_017.png
- Figure 18: Processors and activities media:Bth_018.png
- Figure 19: processor action media:Bth_019.png
- Figure 20: Pre-processor action media:Bth_020.png
- Figure 21: Post-processor action media:Bth_021.png
- Figure 22: Step reports sample media:Bth_022.png
- Figure 23: Sample for command-line usage media:Bth_023.png
- Figure 24: Conditional flow regarding result from <<STEP_EMPTY_FILE_CHECK>> media:Bth_024.png
- Figure 25: Sample job model, and step media:Bth_025.png
- Figure 26: Conditional step flow media:Bth_026.png
- Figure 27: JobContextInterface provided by batch Blu Age profile media:Bth_027.png
- Figure 28: Sample of Job Context usage media:Bth_028.png
- Figure 29: using $job or $step variable media:Bth_029.png
- Figure 30: StepContextInterface provided by batch Blu Age profile media:Bth_030.png
- Figure 31: Generation menu media:Bth_031.png
Batch model: key concept
Step, Items and Jobs
A batch in Blu Age is always composed by the following elements:
- Items
- represent the business data used in a step as an input. An item is always provided by a reader and represents one record: a line from a SQL query, a line from a flat file, a line from an EBCDIC file. Items are modeled using "BusinessObjects" (as it is done in forward PIM model for screens applications).
- Steps
- represent a unitary process of a treatment. This can be a loop. A step is usually composed by three basic actions:
- Read data
- Data processing
- Write data
- Jobs
- represent a sequence of steps. There is only one Job in a Blu Age batch model. Job represents the final batch to be executed.
Package organization
A batch model contains elements stored into packages. Each package is stereotyped according to its content. Below you can see the package organization including profiles provided by Blu Age. Packages may be named according to designer preferences. The modeled application must contain some packages. See below.
Target batch model: package tree-structure
Rules and stereotypes for mandatory packages
- PK_TARGET
- This stands for the batch root model package. Prior packages in the tree are not mandatory; they are used to define namespace. This package is unique.
- PK_STEP
- This stands for the package containing the steps’ model elements. This package can be an external UML module. It must directly or indirectly descend from the <<PK_TARGET>> stereotyped package.
- PK_JOB
- This stands for the package containing the job’s model elements. This package can be an external UML module. It must directly or indirectly descend from the <<PK_TARGET>> stereotyped package.
- PK_BUSINESS
- It stands for the root package containing business elements. Business elements represent the items that will be manipulated by the batch. This package can be an external UML module. It must directly or indirectly descend from the <<PK_TARGET>> stereotyped package and contain a <<PK_BUSINESS_OBJECT>> and <<PK_ENTITY>> sub-package.
- PK_BUSINESS_OBJECT
- It stands for the package where the “business objects” are modeled. This package can be an external UML module. It must descend from the <<PK_BUSINESS>> stereotyped package.
- PK_ENTITY
- It stands for the package where entities are modeled. This package can be an external UML module. It must descend from the <<PK_BUSINESS>> stereotyped package.
- PK_SERVICE
- It stands for the package where services are modeled. Mainly in batch design, services are processors, pre-processors or post-processors. This package can be an external UML module. It must directly or indirectly descend from the <<PK_TARGET>> stereotyped package.
The profiles used by batch models
<<Profile>> Packages description
datatype
- This contains classes and data types for entities, business objects and services modeling.
service
- This contains the stereotypes, the tagged values and enumerations for business services modeling.
batch
- This contains the stereotypes, the tagged values and enumerations for batch specific element modeling.
persistence
- This contains the stereotypes, the tagged values and enumerations for persistence layers modeling.
bo
- This contains the stereotypes, the tagged values and enumerations for business objects modeling.
entity
- This contains the stereotypes, the tagged values and enumerations for entities modeling.
Modeling Items
Main description
Items are represented using business objects (BO) and entities. Entities and BO are modeled as a class. Entities must be stereotyped using <<Entity>> stereotype.
Entities are used to define the mapping to elements retrieved by Readers. Mapping is for a record and an input type (flat file, sql, EBCDIC file). Entities represent the unitary business element to be manipulated by a batch step and for each loop. This business element contains only attributes. Each attribute must be related to a column retrieved from a record provided by a reader. If the reader is other than EBCDIC_Reader, it must be stereotyped by the <<COLUMN>> stereotype. If the Reader is type of EBCDIC_Reader, mapping is done using attribute name that must match with copybook entry. If Reader is type of JPAReader, entity represents the element retrieved from database as a business element.
Business objects (BO) inherit from entities. Business object can contain other attributes that are not provided by reader. BO can contain also getters, setters, operations to manipulate collections and <<formula>> stereotyped operations. The name for a business object must be the same as the name of the related entity including "BO" as suffix.
Consider Items as a way to map rows retrieved by a reader to a business element.
Modeling Rules
- Rules for modeling entities
- An entity must have a unique name,
- The entity name must use the camel upper case convention (CamelUpperCase),
- An entity must be stereotyped using <<Entity>>,
- Entities must be contained in a package stereotyped <<PK_ENTITY>>,
- Rules for modeling business objects
- A "business object" must inherit from one and only one entity,
- A business object name must be the same as the related entity including "BO" as suffix,
- The business object name must use the camel upper case convention (CamelUpperCase),
- Rules for modeling entity Attributes
- Each attribute must be typed (using Blu Age data types),
- Each attribute must be related to a column retrieved from a reader,
- If the reader is type of "FlatFileReader" or "JDBCReader" attribute must be stereotyped using <<COLUMN>> stereotype. The "index" tagged value associated to this stereotype can be used to define the column position. If "index" value is not defined, the attributes must be in the same order as the retrieved columns,
- If the reader is type of "EBCDIC_Reader", the <<COLUMN>> stereotype is not needed. The mapping will be done using the attribute name and the column name defined in the copybook,
- If the reader is type of "JPAReader", there is no need to stereotype attributes. Entity will be retrieved directly from database.
- Rules for modeling BO attributes
- Each attribute must be typed (using Blu Age data types),
- Attributes present only in BO can be used on Processes or on Writers but are not mapped to readers.
- Rule for modeling BO operations
- Operation must be named using camel lower case convention (camelLowerCase)
- Every operation must be stereotyped. the usable stereotypes for batch modeling are:
- <<getter>>
- <<setter>>
- <<attach>>
- <<detach>>
- <<formula>>
- The conventions for operations are as follow:
| Stereotype | Naming convention | Parameters | Operation Type |
| <<getter>> | "get" + attribute name | None | Type of related attribute |
| <<setter>> | "set" + attribute name | Type of related attribute | void |
| <<attach>> | "addTo" + collection name | Type of collection element | void |
| "addAllTo" + collection name | Type of List | void | |
| <<detach>> | "removeFrom" + collection name | Type of collection element | void |
| "removeAllFrom" + collection name | Type of List | void | |
| <<formula>> | N/A | 0 to many arguments | Must be typed |
<<attach>> and <<detach>> are mainly used when conception is using JPA objects.
- Entities modeling: Guide/3-7-1 - UML - Entities - Entity modeling and applicable rules
- Persistence profile: Guide/3-7-3 - UML - Entities - Persistence Profile
Usage of <<Discriminator>> stereotype
If input record is coming from an EBCDIC file, each line can be of different type. Types are defined in the EBCDIC definition from copybook. There is a discriminator column on the EBCDIC file used to recognize the record type. While reading an EBCDIC file, you need to associate to the right entity (item) according to record type. In this case, you need to apply the <<Discriminator>> stereotype to the entity and set the “regex” tagged value with a string representing a regular expression used to recognize the type.
Tagged values to be applied to <<Discriminator>>
| Property | Type | Description |
| regex | String | Matching pattern used to define the type for the retrieved record in case of EBCDIC input. |
Modeling Steps
Main description
A step represents an iterative process based on a list of items provided by a reader. A step is modeled using an activity diagram. While modeling a step we need to focus on the process executed on each loop. This activity diagram can contain the following actions:
- One or zero PreProcessor used to define an execution that will occur one time before the first loop, mainly used to initialize the step
- Represented by a Call Behavior Action and associated to an activity defining a process service representing the action,
- Action is stereotyped <<PreProcessor>>
- One or zero Reader used to define the input Items
- Represented by a Call Operation Action,
- Stereotyped :
- <<EBCDIC_READER>> → reading a EBCDIC file
- <<FlatFileReader>> → reading a CSV or a fixed length file
- <<JDBCReader>> → reading from data base
- <<JPAReader>> → reading data using JPA request and populating business objects
- One or zero Processor used to define the computation to be applied on the item
- Represented by a Call Behavior Action and associated to an activity defining a process service representing the process itself,
- Zero or many writers used to store data
- Represented by a Call Operation Action
- Stereotyped:
- <<EBCDIC_WRITER>> → if data is written to an EBCDIC file,
- <<FlatFileWriter>> → if data is written to a text file (using delimited or fixed-length columns),
- <<JDBC Writer>> → if data is written to a database,
- <<SQLScriptWriter>> → to execute directly a SQL script.
- One or zero PostProcessor used to define an execution that will occur one time after the last loop, mainly used for step finalization
- Represented by a Call Behavior Action and associated to an activity defining a process service,
- Stereotyped <<PostProcessor>>
Step modeling rules
- Step is represented by an activity,
- Activity representing a step must be stereotyped by <<STEP>>,
- Model for a step can be done using activity diagram,
- Step activities must be located inside package stereotyped <<PK_STEP>> or inside a sub-package of the one stereotyped <<PK_STEP>> (see Fig. 6)
Available tagged values associated to <<STEP>>
| Property | Type | Description |
| commitInterval | int | Commit interval for the step. The step will perform a commit every 100 records if set up to 100. (Not used for now, will be implemented in future release) |
| Timeout | int | (Not used for now, will be implemented in future release) |
The readers
The readers are modeled using a call operation action. To define the type of reader, a stereotype must be applied on it. The available stereotypes are:
- <<EBCDIC_READER>>
- <<FlatFileReader>>
- <<JDBCReader>>
- <<JPAReader>>
<<EBCDIC_Reader>>
Define an EBCDIC file reader. The read elements will be type of BO defined in the "file" tagged value. The EBCDIC file to read will be defined into the xJCL file. The tagged value “copybook” must be also defined. It contains the EBCDIC file description.
Available tagged values associated to <<EBCDIC_Reader>>
| Property | Type | Description |
| booleanTrueExpressionValue | String | Definse a regular expression to recognize the EDCDIC column mapped to a boolean attribute in an entity. |
| dateParserClass | String | Defines the class and operation used to parse the field type of date. This must be provided as specific library. This class must implement the interface: com.netfective.batch.date.IDateParser (see #Reading and parsing dates from/to files) If not provided, a default parser will be used. |
| file | Class | Type defined by a Business Object of each record |
| copybook | String | Name of the copybook (including path) defining the EBCDIC file to read. |
<<FlatFileReader>>
Define a flat file reader. The read elements will be type of BO defined in the "file" tagged value. In this case, modeler needs to define at least one of these two tagged values:
- "delimiter" defining the character used as fields separator,
- "position" defining the position of fields in case of fixed length file.
Available tagged values associated to <<FlatFileReader>>
| Property | Type | Description |
| dateParserClass | String | Define the class and operation used to parse the field type of date. This must be provided as specific library. This class must implement the interface: com.netfective.batch.date.IDateParser (see #Reading and parsing dates from/to files) If not provided, a default parser will be used. |
| delimiter | String | Character used to delimit fields |
| position | String | When file is fixed length, define the positions for each fields in comma separated values |
| file | Class | Type of each record defined by a Business Object. |
<<JDBCReader>>
Define a reader based on a SQL query. The read elements will be type of BO defined in the "query" tagged value. The “sql” tagged value must contain the SQL statement to execute.
Available tagged values associated to <<JDBCReader>>
| Property | Type | Description |
| query | Class | Type defined by a Business Object of each record |
| sql | String | SQL request used to select items in database |
<<JPAReader>>
Define a reader based on a "jpql" request (jpql is for Java Persistence Query Language). This is very useful because objects are retrieved from database, including association, and mapped to entity model. The "jpql" tagged value must be defined and the entities and BO must represent database structure. If there are fetched elements retrieved by jpql query, an object flow can be modeled to set up the field to be fetched according to a business object. The control flow must be stereotyped using <<DynamicFetch>>.
- OpenJPA
- Hibernate for JPA
If you are planning to generate using Hibernate, there is no need to model fetched elements. Hibernate will perform an automatic fetch.
Available tagged values associated to <<JDBCReader>>
| Property | Type | Description |
| jpql | String | jpql request used to retrieve objects. Can contain many fetches. |
<<GroupReader>>
<<GroupReader>> stereotype can be applied on readers (but not JPAReader). As a result, the element that will be processed will not be the single BO retrieved by the reader, but a list of BO regarding the built group. The group is defined by one or many fields.
- For <<EBCDIC_Reader>>, the file must be ordered by the fields used for grouping,
- For <<FlatFileReader>>, the file must be ordered by the fields used for grouping,
- For <<JDBCReader>>, a statement “order by” must be applied on the column on which we like to order.
You can provide one or many fields to group on. The tagged value is type of String[1..*].
Available tagged values associated to <<GroupReader>>
| Property | Type | Description |
| field | String [1..*] | Field used to set the "group-by" property. Multiple fields can be provided to group on. |
The Writers
The writers are modeled using a call operation action. To define the type of writer, a stereotype must be applied on it. The available stereotypes are:
- <<EBCDIC_WRITER>>
- <<FlatFileWriter>>
- <<JDBC Writer>>
- <<SQLScriptWriter>>
- <<TextFileWriter>>
<<EBCDIC_WRITER>>
Define a writer creating an EBCDIC file. The mapping is done using the copybook. The reference to copybook must be defined on the xJCL file generated. The copybook can be also added as a reference in the tagged value "copybook".
Available tagged values associated to <<GroupReader>>
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| booleanEncoder | String | Define the class that implements the Boolean encoder (transforming a Boolean attribute value to a string or a big-decimal). This class must be provided as specific library. This class must implement the interface: com.netfective.batch.bool.IBooleanEncoder If not provided, a default parser will be used. |
| dateParserClass | String | Define the class and operation used to parse the field type of date. This must be provided as specific library. This class must implement the interface: com.netfective.batch.date.IDateParser (see #Reading and parsing dates from/to files) If not provided, a default parser will be used. |
| file | Class | Type defined by a Business Object of each record |
| copybook | String | Name of the copybook (including path) defining the EBCDIC file to read. |
<<FlatFileWriter>>
Define a writer creating a flat file. The fields to write must be defined using “fieldsToWrite” tagged value.
Available tagged values associated to <<FlatFilewriter>>
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| booleanEncoder | String | Define the class that implements the Boolean encoder (transforming a Boolean attribute value to a string or a big-decimal). This class must be provided as specific library. This class must implement the interface: com.netfective.batch.bool.IBooleanEncoder If not provided, a default parser will be used. |
| delimiter | String | Character used to delimit fields |
| fieldsToWrite | String | Comma separated name of fields to be written. The fields must be related to the name of BO attribute. |
| format | String | Format defined by Java Formatter class ( http://download.oracle.com/javase/1.5.0/docs/api/java/util/Formatter.html ) |
<<JDBCWriter>>
Define a writer executing a SQL statement on the database. The SQL statement can be INSERT, UPDATE or DELETE and must be defined by the tagged values "sql".
Available tagged values associated to <<JDBCWriter>>
| Property | Type | Description |
| sql | String | SQL request use to write data (DELETE, UPDATE or INSERT) |
<<SQLScriptWriter>>
Define a writer that will execute a SQL script directly in database. The script to be executed must be defined in the xJCL file generated. Usually this writer is used to initialize database using a static SQL script. There is no tagged value associated with this stereotype.
<<TextFileWriter>>
Define a writer that will use direct operations to write a text file. The text file to write to will be defined in the xJCL file ("Substitution.properties" can be used to define the value). You need to define two tagged values:
- headerWriter
- reference to an operation in the model containing the implementation to create the header file. This operation must retrieve a string that will be written on the file,
- recordWriter
- reference to an operation in the model containing the implementation to create the record line into the file. This operation provides a string containing the line to write on file. The BO type of the record must be provided as argument.
Available tagged values associated to <<TextFileWriter>>
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| headerWriter | Operation | Operation used to write header (no parameter, type of string) |
| recordWriter | Operation | Operation used to write the record (record as parameter, type of string) |
Conditional writer
On any writer you can define a condition to check before executing the writer. This condition will be applied as a filter. The condition must be defined as a process operation associated to an interface located in the <<PK_SERVICE>> stereotyped package. The operation must be type of boolean and get as argument a business object representing the item to write.
Next you need to define an interface stereotyped <<ItemCondition>>, with the "service" tagged value set as the name of the operation representing the condition (reference to the operation in the model). To apply the filter defined by the condition, an input pin related to the interface stereotyped <<ItemCondition>> can be added on the writer action. This input pin must be a "Target" input pin (not an "Argument" input pin).
Available tagged values associated to <<TextFileWriter>>
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| headerWriter | Operation | Operation used to write header (no parameter, type of string) |
| recordWriter | Operation | Operation used to write the record (record as parameter, type of string) |
Available tagged values associated to <<ItemCondition>>
| Property | Type | Description |
| service | Operation | Operation implementing the condition. Must be type of Boolean. |
The processors
There are 3 types of processors that can be used on steps:
- pre-processor,
- processor,
- post-processor.
All of them must be represented using a stereotyped UML Call Behavior Action bound to the activity representing the process. This activity is related to an operation defined in the <<PK_SERVICE>> stereotyped package. Activities are modeled the same way as "process operation",
Processors
Processor contains the main computation that can be applied on an item and for a loop. Processor is defined by a Call Behavior Action related to the activity describing the implementation of an operation stereotyped <<process>> and located on the <<PK_SERVICE>> package.
Processor rules:
- No stereotype is applied on processor Call Behavior Action,
- There is one or zero processor in a step,
- Processor must be defined by a process operation,
- The arguments for the process operation must be:
- in: an object type of BO representing the current item (record) retrieved by the reader,
- in: a boolean (optional) indicating if the processed record is the last one or not (true if the record is last one)
- result: object type of BO representing the item to pass to the writers. Can be different to the input item.
Pre-processor
Pre-processor is executed one time and before the first loop on items. Pre-processor is defined by a Call Behavior Action related to the activity describing the implementation of an operation stereotyped <<process>> and located on the <<PK_SERVICE>> package. This Call Behavior Action must be stereotyped <<PreProcessor>>.
Pre-processor rules:
- <<PreProcessor>> stereotype must be applied to the action representing pre-processor on a step,
- There is one or zero pre-processor in a step,
- Pre-processor must be defined by a process operation,
- The arguments for the process operation must be:
- in: none
- result no type (void)
Post-processor
Post-processor is executed one time and after the last loop on items. Post-processor is defined by a Call Behavior Action related to the activity describing the implementation of an operation stereotyped <<process>> and located on the <<PK_SERVICE>> package. This Call Behavior Action must be stereotyped <<PostProcessor>>.
Post-processor rules:
- <<PostProcessor>> stereotype must be applied to the action representing post-processor on a step,
- There is one or zero post-processor in a step,
- Post-processor must be defined by a process operation,
- The arguments for the process operation must be:
- in: none
- result type of int
The "int" result can be used on the Job activity diagram as a condition on a control-flow to set up conditional step execution. See #Conditional step flow in a job.
Reading and parsing dates from/to files
File readers and file writers (flat file and EBCDIC) provide a tagged value named "dateParserClass". This tagged value defines the class that will be used to parse dates (transforming string or number values from file into date or opposite). The implementation must be provided within a specific library. This class must implement the interface "com.netfective.batch.date.IDateParser" and redefine the following operations:
- public abstract java.util.Date decode(java.math.BigDecimal arg0)
- public abstract java.util.Date decode(java.lang.String arg0)
- public abstract java.lang.String encodeString(java.util.Date arg0)
- public abstract java.math.BigDecimal encodeBigDecimal(java.util.Date arg0)
Specials Steps modeling
Report step
To generate reports during a batch process, you need to define a new step stereotyped <<REPORT_BATCH>>. This step is defined by an activity located in the <<PK_STEP>> package. There is no need to describe the content of this activity using activity diagram. This step can be used on a job process. The report definition must be defined using JasperReport. It must be designed to work alone; that means all SQL statement must be defined inside the report. The Jasper report file must be located on the class path and must have the same name as the step. The used data-source will be defined in the xJCL generated. By default the data-source would be "OUT_JOBNAME".
- jasperreports-4.0.2.jar
- commons-digester-1.7.jar
- groovy-all-1.7.5.jar
- iText-2.1.7.jar
Available tagged values associated to <<Report_STEP>>
| Property | Type | Description |
| format | EnumReportType | Type of output:
|
Command-line step
Using Blu Age, it is possible to generate steps that will execute a system command-line. You need to create an activity stereotyped <<STEP_COMMAND_LINE>>. This step must contain a single call operation action stereotyped <<CommandLine>> with the tagged value "strCommandLine" defined and containing the command-line text to be executed. This command-line must be compliant with the batch execution environment. The created step can be used on a job activity diagram.
Available tagged values associated to <<CommandLine>>
| Property | Type | Description |
| strCommandLine | String | Command line to execute (must be compliant to the execution environment, should be redefined in the xJCL generated file) |
Check-empty-file step
This feature enables to find if a file is empty or not. You need to model a step using an activity stereotyped <<STEP_EMPTY_FILE_CHECK>>. This step will contain only one call operation action stereotyped <<EmptyFileCheck>>. The result will be stored in a JobContext variable using the step name as a key name:
- Value is 1: file is empty
- Value is 0: file is not empty
The created step can be used on a job and also associated to a flow condition (see #Conditional step flow in a job).
Modeling Job
Job principles
A job represents the batch itself. It contains many steps to be executed in the batch. A job is modeled using activity diagram. Each step on a job must be added as a call behavior action, where the behavior is defined by the activity representing the step.
Job Modeling Rules
- There is one and only one job in a batch model,
- Job is represented by an activity,
- Activity representing a job must be stereotyped by <<JOB>>,
- Model for a job can be done using activity diagram,
- Job activities must be located inside package stereotyped <<PK_JOB>>
- Activity representing a job contains a sequence of actions (call behavior actions). Each action is related to a step and the behavior is the activity representing the step,
- Action representing a step must be named.
Available tagged values associated to <<Job>>
| Property | Type | Description |
| record | int | Define the commit interval to be applied on each step. If not defined, will be set to 10000. |
Conditional step flow in a job
It is possible to execute a step according to a condition. The condition to be tested must be defined by the return value of a post-processor (type of int, see #Post-processor). To set up a path, a guard condition must be located on the control flow connecting two steps. The guard condition will contain the value of the retrieved integer from the post-processor from previous step.
Job context and Step context
Job context
Presentation
Job context is a way to store variables into a hash-map (key/value). These variables can be used (read and updated) into any processor (including pre-processor and post-processor) of any step in the job. This is a good way to pass variables thru steps.
Blu Age batch profile contains an interface named: "JobContextManager" providing 3 operations:
- putInContext(key:Object, record:Object):void → create a new variable for the "key" in the hash-map and set the value by "record"
- getFromContext(key:Object):Object → retrieve the value stored in the hash-map for the "key"
- empty():void → delete all variable (empty hash-map)
Using JobContext in a model
You can use JobContext in any activity diagram representing a process. You need to initialize a new instance of JobContext, and then use the 3 operations according to your need.
Using job context in a query
If you need to use a value stored in a JobContext or in a StepContext as part of query (SQL query only), you can use the following variables:
- From the job context : $job(var,type),
- From the step context : $step(var,type),
- Where "var" represents the key and "type" is the type of the value element.
Step context
StepContext is works the same way as JobContextJob. The main difference is the scope of the variable. When in JobContext the scope is on the entire batch, in the StepContext, the scope is on the step. So you can use this feature to share variable between processor, pre-processor and post-processor for a single step.
Blu Age batch profile contains an interface named: "StepContextManager" providing 3 operations:
- putInContext(key:Object, record:Object):void
- getFromContext(key:Object):Object
- empty():void
Generation Workflow
com.netfective.bluage.bsp.batch.jet2.xdgrid BSP specifics properties
| Property name | Description | Type | Sample value |
| langageMapping | Provide the mapping file for datatypes to language | string | file:/PROJECT/resources/mappings/JavaMappings.xml |
| wrapperMapping | Provide the mapping file for datatype to wrapper | string | file:/PROJECT/resources/mappings/JavaWrapperMappings.xml |
| jdbc_driver | JDBC driver used with the batch simulator. | string | org.postgresql.Driver |
| userid | User id for database user, to be used with the batch simulator. | string | Myname |
| pswd | Password for database user, to be used with the batch simulator. | string | myPassWord |
| jdbc_url | JDBC url for the database server, to be used with the batch simulator. | string | jdbc:postgresql://localhost/myDatabase |
| fwkPackage | Contains the name of package containing framework. | string | com.netfective.bluage |
| WAS_profile_home_default | Repository for local WAS instance used to execute batches for testing | string | D:\IBM_WebSphere\profiles\AppSrv01 |
| OpenJpaAssociationStyle | Set bidirectional mode in case of 1..* associations for OpenJPA implementation | Boolean | true (must be true if OpenJPA is used) |
| specificCollectionInterfaces | If collection type is not defined in the model, collection will be set to type of List. | Boolean | true |
| AppendJobIdToFileName | The job name will be added to output (written) flat file name. | Boolean | True |
| orderStrategy | Algorithm to define process services path. Must be Hybrid | String | hybrid (must be "hybrid") |
| datasource | Name for used data-source. | String | USERS (this sample will define 2 data-sources in the application server: USER_IN and USER_OUT) |
| timeout | Timeout value for each transaction. | Int | 300 |
| jpaEnabled | Enable or disable JPA generation. | Boolean | True |
| database | In case of JPA generation, type of database used. | String | postgres |
| jpaDatasource | Name of data-source used for JPA connection | String | USERS (datasource USERS_IN will be used in this case) |
| jpaImplementation | Provide the inplementation of JPA to generate | String | "hibernate" or "OpenJPA" |
Generating, Deploying and executing Batch
Generating
Generation is done by right clicking on the workflow and selecting "Run As" then " Blu Age JET2 launcher".
Blu Age will generate 1 project named "..._WCG-service". This project contains the following folders:
src Java sources
lib Used libraries
props.packaging
props.simulator
script.ant Ant scripts used to package deploy and execute batch in test mode.
script.wsadmin
xJCL xJCL generated by Blu Age. There are 2 JCL files, one specific for zOS environment and other one for other environments.
Deploying
Data-sources need to be configured on your application server. For a batch, you must have at least 2 data-sources. The Data sources names are composed by the Job name plus "_IN" or "_OUT" as a suffix, depending on the direction.
- The IN source has to be a non-XA source,
- The OUT source can be XA or not,
- The OUT source has to be XA if a single step uses several outputs. In this case, the XA mode is defined in the implementation class of the jdbc driver.
Before deploying your batch, you need to build an EAR file. This EAR can be deployed on the application server. Other way to build the EAR file is to execute the generated Ant task "1.packageApp.JOB_NAME.xml" located in the folder: \script.ant\JOB_NAME”. Executing this script will create an EAR file named "JOB_NAME.deployed.ear" in the "export" folder. You can deploy this EAR file using the WAS administration console or executing the Ant task "2.installApp.JOB_NAME.xml".
Executing
To execute the batch you can follow one of the 2 methods:
- Deploying the generated xJCL file on WCG using the administration console
- Executing the Ant task file “3.runJob.JOB_NAME.xml” directly on eclipse. The execution result will be located on the eclipse console.
SubstitutionProperties.xml is generated on the “script.ant” folder. A good practice would be to create it using a dir-def directive on the workflow. This file has the following format:
<?xml version="1.0" encoding="UTF-8" ?> <substitution-props> <prop name="PROPERTY.NAME.IN.xJCL.FILE" value="property value" /> </substitution-props>
Annexes: Stereotypes used in batch modeling with Blu Age
Applied on packages
- PK_JOB
- Set the stereotyped package as a JOB container
| Property | Type | Description |
| N/A |
- PK_STEP
- Set the stereotyped package as a STEP container
| Property | Type | Description |
| N/A |
Applied on class
- Discriminator
- Define the type of the input record retrieved from EBCDIC file according to the tagged value "regex".
| Property | Type | Description |
| regex | String | Matching pattern used to define the type for the retrieved record in case of EBCDIC input. |
Applied on class attribute
- COLUMN
- The stereotyped attribute will be considered as an element retrieved from the reader.
| Property | Type | Description |
| index | int | Optional, position in the fields retrieved by the reader |
Applied on interfaces
- ItemCondition
- Applied on an interface that can be used as a type of an input-pin associated to a writer action. This interface will be associated to a process operation type of boolean and defining a condition. The action on which the input-pin is applied will be executed only if the Boolean is true.
| Property | Type | Description |
| service | Operation | Operation implementing the condition. Must be type of Boolean. |
Applied on Activities
- JOB
- Set the stereotyped activity as a JOB
| Property | Type | Description |
| abstract | boolean | Deprecated |
| String | String | Deprecated |
| int | int | Define the commit interval to be applied on each step. If not defined, will be set to 10000. |
| boolean | boolean | Deprecated |
- STEP
- Set the stereotyped activity as a STEP
| Property | Type | Description |
| abstract | boolean | Deprecated |
| allowStartIfComplete | boolean | Deprecated |
| commitInterval | int | Commit interval for the step. The step will perform a commit every 100 records if set up to 100. (Not used for now, will be implemented in future release) |
| isolation | ISOLATION | Deprecated |
| isStandalone | boolean | Deprecated |
| noRollbackExceptionClasses | EnumerationLiteral | Deprecated |
| parent | STEP | Deprecated |
| propagation | PROPAGATION | Deprecated |
| readerTransactionalQueue | boolean | Deprecated |
| restartable | boolean | Deprecated |
| retryableExceptionClasses | EnumerationLiteral | Deprecated |
| retryLimit | int | Deprecated |
| skipLimit | int | Deprecated |
| skippableExceptionClasses | EnumerationLiteral | Deprecated |
| startLimit | int | Deprecated |
| timeout | int | (Not used for now, will be implemented in future release) |
- REPORT_STEP
- Set the stereotyped activity as a STEP that will be used to generate a report
| Property | Type | Description |
| format | EnumReportType | Type of output:
|
- STEP_COMMAND_LINE
- This stereotype can be applied to an activity representing a step executing a command-line. The command-line to execute will be defined as an action of the step activity. This action will be stereotyped <<CommandLine>>. The command line to call will be provided in the generated xJCL.
| Property | Type | Description |
| N/A |
- STEP_EMPTY_FILE_CHECK
- To be applied on a step activity. If the reader retrieves no result (empty file, null result set from a sql query, ...) an int value will be added to the JobContext for a key named as the step.
| Property | Type | Description |
| N/A |
Applied on actions
- CommandLine (on call operation action)
- Action containing a command-line. Must be included in an activity stereotyped <<STEP_COMMAND_LINE>> (see 10.5 sub part "STEP_COMMAND_LINE")
| Property | Type | Description |
| strCommandLine | String | Command line to execute (must be compliant to the execution environment, should be redefined in the xJCL generated file) |
- EBCDIC_READER (on call operation action)
- Set the stereotyped step action as a reader for EBCDIC file.
| Property | Type | Description |
| booleanTrueExpressionValue | String | Define a regular expression to recognize the EDCDIC column mapped to a boolean attribute in an entity. |
| dateParserClass | String | Define the class and operation used to parse the field type of date. This must be provided as specific library. This class must implement the interface: com.netfective.batch.date.IDateParser (see chapter 4.4) If not provided, a default parser will be used. |
| file | Class | Type defined by a Business Object of each record |
| copybook | String | Name of the copybook( including path) defining the EBCDIC file to read. |
- EBCDIC_WRITER (on call operation action)
- Set the stereotyped step action as a writer for EBCDIC files.
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| booleanEncoder | String | Define the class that implements the Boolean encoder (transforming a Boolean attribute value to a string or a big-decimal). This class must be provided as specific library. This class must implement the interface: com.netfective.batch.bool.IBooleanEncoder If not provided, a default parser will be used. |
| dateParserClass | String | Define the class and operation used to parse the field type of date. This must be provided as specific library. This class must implement the interface: com.netfective.batch.date.IDateParser (see chapter 4.4) If not provided, a default parser will be used. |
| items | Class | Deprecated |
| rejectedFields | Property | Deprecated |
| copybook | String | Name of the copybook (including path) defining the EBCDIC file. |
- FlatFileReader (on call operation action)
- Set the stereotyped step action as a reader for flat file.
| Property | Type | Description |
| autoMapping | boolean | Deprecated |
| dateParserClass | String | Define the class and operation used to parse the field type of date. This must be provided as specific library. This class must implement the interface: com.netfective.batch.date.IDateParser (see chapter 4.4) If not provided, a default parser will be used. |
| delimiter | String | Character used to delimit fields |
| position | String | When file is fixed length, define the positions for each fields in comma separated values |
| fields | String | Deprecated |
| file | Class | Type of each record defined by a Business Object. |
- FlatFileWriter (on call operation action)
- Set the stereotyped step action as a writer for a flat file.
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| booleanEncoder | String | Define the class that implements the Boolean encoder (transforming a Boolean attribute value to a string or a big-decimal). This class must be provided as specific library. This class must implement the interface: com.netfective.batch.bool.IBooleanEncoder If not provided, a default parser will be used. |
| delimiter | String | Character used to delimit fields |
| fieldsToWrite | String | Comma separated name of fields to be written. The fields must be related to the name of BO attribute. |
| format | String | Format defined by Java Formatter class (http://download.oracle.com/javase/1.5.0/docs/api/java/util/Formatter.html") |
- JDBCReader (on call operation action)
- Set the stereotyped step action as a a JDBC reader (execute a SQL query to retrieve data, the statement should be SELECT)
| Property | Type | Description |
| driverSupportsAbsolute | boolean | Deprecated |
| fetchSize | int | Deprecated |
| ignoreWarnings | boolean | Deprecated |
| maxRows | int | Deprecated |
| query | Class | Type defined by a Business Object of each record |
| queryTimeout | int | Deprecated |
| saveState | boolean | Deprecated |
| setUseSharedExtendedConnection | boolean | Deprecated |
| sql | String | SQL request used to select items in database |
| verifyCursorPosition | boolean | Deprecated |
- JDBCWriter (on call operation action)
- Set the stereotyped step action as a JDBC writer (execute a SQL query to write data, the statement can be DELETE, UPDATE or INSERT)
| Property | Type | Description |
| assertUpdates | boolean | Deprecated |
| sql | sql | SQL request use to write data (DELETE, UPDATE or INSERT) |
- JPAReader
- Set the stereotyped step action as a JPA reader. It will be used to retrieve values from database as objects and associated objects.
| Property | Type | Description |
| jpql | String | jpql request used to retrieve objects. Can contain many fetches. |
- GroupReader (on call operation action)
- Stereotype to be applied on an action already stereotyped as “reader” (Any type of reader: flat file, EBCDIC, JDBC). Define the group-by property used to define a list of BO that can be used for each loop.
| Property | Type | Description |
| field | String [1..*] | Field used to set the "group-by" property. Many fields can be provided to group on. |
- SQLScriptWriter (on call operation action)
- Create an action that will execute a sql script on database.
| Property | Type | Description |
| N/A |
- EmptyFileCheck (on call operation action)
- Define an action to be used in activity stereotyped <<STEP_EMPTY_FILE_CHECK>> (see #Applied on Activities sub part "STEP_EMPTY_FILE_CHECK")
| Property | Type | Description |
| N/A |
- PreProcessor (on call behavior action)
- Stereotype to be applied on an action defining a post-processor treatment. The action must be defined by a process activity diagram.
| Property | Type | Description |
| N/A |
- PostProcessor (on call behavior action)
- Stereotype to be applied on an action defining a post-processor treatment. The action must be defined by a process activity diagram.
| Property | Type | Description |
| N/A |
- SUB_REPORT (on call behavior action)
- Not yet implemented, will be available on future release.
| Property | Type | Description |
| N/A |
- TextFileWriter (on call operation action)
- Define a step to write directly in a file.
| Property | Type | Description |
| append | boolean | Define if record will be appended to an existing file (true) or if the file will be overwritten (false). |
| headerWriter | Operation | Operation used to write header (no parameter, type of string) |
| recordWriter | Operation | Operation used to write the record (record as parameter, type of string) |
Applied on object flows
- DynamicFetch
- Define that the object node related to this association will be dynamically fetched.
| Property | Type | Description |
| Field | Property | Field retrieved by SQL query and on which the fetch will be done. |
Technical stereotypes used on profile only
- ServiceStop
- Not to be used, only to be applied on interface "ServiceStop" provided on batch profile.
| Property | Type | Description |
| N/A |
- JobContext
- Not to be used, only to be applied on interface "JobContext" provided on batch profile.
| Property | Type | Description |
- StepContext
- Not to be used, only to be applied on interface "StepContext" provided on batch profile.
| Property | Type | Description |
Deprecated stereotypes
- BatchQuery (applied on class)
- EBCDIC_ITEM (applied on class)
- FlatFileLine (applied on class)
- PK_ITEM (applied on packages)
- ItemProcessor (applied on interfaces)
- ItemReader (applied on interfaces)
- ItemWriter (applied on interfaces)
- Conditional_Flow (applied on controlFlow)
- End_Flow (applied on controlFlow)
- Fail_Flow (applied on controlFlow)
- Stop_Flow (applied on controlFlow)
- RecordErrorLog (applied on class)









