Red Hat Job Opening – Software Sustaining Engineer

We are looking to hire someone to help improve the quality of BRMS and BPM Suite platforms. These are the productised versions of the Drools and jBPM open source projects.

The role will involve improving our test coverage, diagnosis problems, creating reproducers for problems as well as helping fix them. You’ll also be responsible for helping to setup and maintain our continuous integration environment to help streamline the various aspects involved in getting timely high quality releases out.

So if you love Drools and jBPM, and want to help make them even better and even more robust – then this is the job for you 🙂

The role is remote, so you can be based almost anywhere.

URL to apply now http://jobs.redhat.com/jobs/descriptions/software-engineer-brno-jihomoravsky-kraj-czech-republic-job-1-4759718

…make JBDS faster for SwitchYard

While working with SwitchYard, JBoss Developer Studio can be a pain in the ass. Red Hat is working to provide some better user experience, but in the meantime, you can try some of these tips.

Increase heap memory

Eclipse needs a lot of memory, and maven and SwitchYard projects even more, so provide with a good amount of it to your eclipse. Modify jbdevstudio.ini in the appropiate section:
-vmargs
-XX:MaxPermSize=256m
-Xms2G
-Xmx2G
-XX:-UseParallelGC
-XX:+AggressiveOpts
-XX:-UseConcMarkSweepGC
Provide a good amount of memory, so if you can give 3 or 4 GBs instead of 2 better.

Disable automatic updates

Faster startup time. You’ll update or check for updates whenever you want.
Preferences --> Automatic updates --> (Disable) Automatically find new updates and notify me

Disable auto build

If you build whenever you want, the project you want, then disable. If you have a big ammount of projects you can skip having eclipse doing background building all the time. If you have few projects, you can keep it.
Project --> Build automatically (Uncheck)

Refresh workspace on startup

If you don’t do things on command line, then your workspace should be refreshed. If you use git (command line) or maven (command line) maybe you want to keep it:
General -> Startup and shutdown -> Refresh workspace on startup (Enable)

Disable validations

If you there is a lot of background task processing gone on validating your project (due to any facet your project has, like JPA, …​)
Validation -> Suspend all validations (check)

Disable startup features not needed (FUSE, …​)

Use the fewer plugins needed for your work.
General -> Startup & shutdown ->  Plugins activated on Startup (Remove FUSE, Fabric8, JBoss Central, Forge UI, JBoss Tools reporting, Equinox autoupdate)

Disable XML Honour schemas

There is a known bug in JBDS and SwitchYard, so avoid it with:
XML -> XML Files -> Validation -> Honour all XML schema locations (uncheck)

Close JPA and EAR projects if not working on them

Every project that you have opened is both, eating resources and having to be check with the background tasks, so close them if not needed (as dependencies or at all)

… using vaulted properties in SwitchYard

Description

Usually we define externalize configuration in properties files to just make the application configuration independent. But one problem we face is that this information is usually endpoints and login credentials, like username and password. For this first type of information, there is usually no need to have them encrypted, but there typically is a requirement for this when storing credentials information.
I’m going to explain how to do this in SwitchYard.
Thanks to Nacim Boukhedimi for this post
The steps to be able to use vaulted configuration is:
  • Create a vault file and store the property there
  • Configure the application to read

Create a vault file

Vault mechanism is the mechanism provided by JBoss EAP 6 to enable you to encrypt sensitive strings and store them in an encrypted keystore. This mechanism relies upon tools that are included in all supported Java Development Kit (JDK) implementations. Below is a step by step description of how to encrypt your properties stored in an external properties file.
  1. Create a directory to hold your keystore and other important information. (for instance ${jboss.server.config.dir})
  2. Enter the following command to create a keystore file named vault.keystore:
    keytool -genkey -alias vault -keyalg RSA -keysize 1024 -keystore vault.keystore
  3. Mask the Keystore Password and Initialize the Password Vault. From EAP_HOME/bin folder, run vault.sh and start a new interactive session
    The salt value, together with the iteration count (below), are used to create the hash value.
    Make a note of generated vault Configuration in a secure location. It will be used in the next step to configure EAP to use vault.
  4. Configure JBoss EAP 6 to Use the Password Vault. Copy and paste the generated vault Configuration in EAP config file:
    <vault>
    <vault-option name="KEYSTORE_URL" value="${jboss.server.config.dir}/vault.keystore"/>
    <vault-option name="KEYSTORE_PASSWORD" value="MASK-18JTA2ZfD4eISrndbFgJRk"/>
    <vault-option name="KEYSTORE_ALIAS" value="vault"/>
    <vault-option name="SALT" value="8675309K"/>
    <vault-option name="ITERATION_COUNT" value="50"/>
    <vault-option name="ENC_FILE_DIR" value="${jboss.server.config.dir}/vault/"/>
    </vault><management>

Store your data in the vault

  1. Store encrypted Sensitive strings in the Java Keystore:
    1. run vault.sh and start a new interactive session
    2. Enter the path to the keystore, the keystore password, vault name, salt, and iteration count to perform a handshake.
    3. Select the option 0 to store a value.
    4. Enter the value, vault block, and attribute name.
    5. As a result, the value is stored and a message is displayed showing the vault block, attribute name, shared key, and advice about using the string in your configuration.
      Vault Block:container
      Attribute Name:file.name
      Configuration should be done as follows:
      VAULT::container::file.name::1
      Please make note of the vaultID to use in your properties file

Use the encrypted property in your properties file

Now it is time to use the property. As we want the property to be externalized, we will define the property in a properties file, and then instruct the server to use that properties file.
  1. In your properties, refer to this property as:
 file.name=${VAULT::container::file.name::1}
You need to start the server with the P option to provide the path to the properties file.
./standalone.sh  -P=file:<path_to_properties>

Use the property in your switchyard.xml

You can refer then to this property from your switchyard.xml
      <file:binding.file>
<file:directory>/tmp/input</file:directory>
<file:fileName>${file.name}</file:fileName>
<file:consume>
<file:initialDelay>10</file:initialDelay>
<file:delay>10</file:delay>
<file:delete>true</file:delete>
</file:consume>
</file:binding.file>
The file endpoint will be scanning for files under /tmp/input/<your secret filename>

…how to read a file programmatically in SwitchYard

I’ve been asked recently how to read a file programmatically, based on a cron, or when a request is made via a call to a Service. I didn’t have an answer, well, really I had an answer and I was wrong. Today, somebody brought the topic back, and I revisited the problem, and now I have an answer, one that works, so a good answer.

Problem description

I have the following use case:
A file should be downloaded by FTP from a remote server. The download of the file can be triggered in two ways:
  • by a scheduler whose polling interval may be changed at runtime
  • by a HTTP GET Request on a specified url. The response of the HTTP call must include transformed data based on the downloaded file.
What is a good way to achieve these requirements with SY/Camel? I would like to use the bindings provided by Camel, but have no idea how they can be triggered (e.g. the FTP binding) from outside the binding.

Solution

I have created a sample project, just to demonstrate how it works, not the full problem is solved. You have to use thecontent ennricher EIP.
We have a service, with a scheduled binding (Every 30 secs) that calls a Camel component. In this component, we use pollEnrich to read a file, and the use the contents of the file (the Message) to write into a File (a Reference with file binding).
This is the route we will be using. Note the pollEnrich to read the file.
public void configure() {
from("switchyard://ProgramaticReload")
.log("Going to read the file")
.pollEnrich("file:///input?fileName=in.txt",1000)
.log("Read from file : ${body}")
.to("switchyard://FileWriterService");
}
The project is available in github

…SwitchYard in docker

Introduction

Now that there is an official docker image for SwitchYard, I’m going to explain some usages of this image to make you productive with docker and SwitchYard. This way, you’ll be able to have reproducible environments for:
  • development
  • demos
  • support/bug hunting
  • integration / testing
Let’s get us started:

SwitchYard docker official image

The official images, are maintained by Red Hat, and are published to the docker hub. You can browse the source code for the images. Currently there is only support for SwitchYard on Wildfly. Of course, on the official Wildfly image, also supported by Red Hat.
All of the JBoss community projects that have a docker image, are published in the official docker community site. From there, you can link to the individual projects. As I’m going to talk about SwitchYard, let’s see what’s there:

Get started

In order to get you started you just need to download the image
docker pull jboss/switchyard-wildfly
and create a container
docker run -it jboss/switchyard-wildfly
or in domain mode
docker run -it jboss/switchyard-wildfly /opt/jboss/wildfly/bin/domain.sh -b 0.0.0.0 -bmanagement 0.0.0.0
You’ll see that the base SwitchYard docker container is not so useful (doesn’t have an admin user), so the best thing to do, is to extend it.

Extending the SwitchYard container

First thing you need to know is what you want the container for, so let’s see some sample uses:

Development

In this use case, what you want is a container where to deploy your applications, those that you are developing, so it can be useful to have:
  • An admin user created
  • Debug port exposed
  • Binding to all interfaces, so you can access without forwarding (if required)
  • Some volumes to access your information in the container
  • Some volumes (to test file services)
Let’s create a container for this:
Dockerfile
FROM jboss/switchyard-wildfly

#
# Add admin user
RUN $JBOSS_HOME/bin/add-user.sh admin admin123! --silent

#
# Enable port for debugging with --debug
EXPOSE 8787

#
# Volumes
VOLUME /input
VOLUME /output

# Enable binding to all network interfaces and debugging inside the server
RUN echo "JAVA_OPTS="$JAVA_OPTS -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0"" >> $JBOSS_HOME/bin/standalone.conf

ENTRYPOINT ["/opt/jboss/wildfly/bin/standalone.sh"]
CMD []
I’m going to explain each and every command:
  1. FROM jboss/switchyard-wildfly, this is the base image, so we are reusing everything in there (jboss user, EAP_HOME set, and more. See doc for reference)
  2. RUN $JBOSS_HOME/bin/add-user.sh admin admin123! –silent, this creates and admin user with admin123! as password.
  3. EXPOSE 8787 adds debug port exposure (8787 is the default one in wildfly.
  4. VOLUME /inputVOLUME /output adds two volumes to the image, so we can easily acces or export this volumes if needed. They are very handy when developing file services, as we will see when we run the container.
  5. RUN echo “JAVA_OPTS=”$JAVA_OPTS -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0″” >> $JBOSS_HOME/bin/standalone.conf this adds default binding to any interface (by default only 127.0.0.1) so we can access the server via the ip or name.
  6. ENTRYPOINT [“/opt/jboss/wildfly/bin/standalone.sh”] defines default command to be executed. I like to set the command here as this image will be single purpose, so no need to be able to switch (although it is possible with –entrypoint on command line).
  7. CMD [] this is the arguments to the entrypoint, none by default, but it allows you to set on command line things like “–debug” to enable debugging mode in the container.
Now it is time to build the image:
docker build -t "jmorales/switchyard-dev" .
You can name the image as you like, as this image will be yours. Just need to think that has to have meaningul name so it is easy to remember when used.
Let’s now go to run the image:
docker run -it jmorales/switchyard-dev
This is the simplest command to run the container, but we are not using many of the power we have provided to it, so, let’s make a better command:
docker run -it -name "switchyard" -p 8080:8080 -p 9990:9990 jmorales/switchyard-dev
Now we can access the console with the admin/admin123! user that we have created.
Still not there yet. We need to be able to use file services (remember that in the container the directories will be /input and /output) and we want to debug in the container from the JBDS.
docker run -it -name "switchyard" -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 -p 8787:8787 jmorales/switchyard-dev --debug
And even maybe use a different profile
docker run -it -name "switchyard" -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 -p 8787:8787 jmorales/switchyard-dev --debug -c standalone-full.xml
Important things to note here:
  • The image is run in foreground, so we will be viewing the console log. Control-c will stop the container. Since we have the container created, if we want to execute it again, it is just a:
docker start switchyard
  • If you want a disposable container, you can add –rm option to the command line, and every time you stop the container it will be removed. (This option is more useful on other use cases).
  • If you always work with some options, you can just add them to your Dockerfile and create your image with these options, so no need to type them when creating the container (like the profile, –debug, ..)
Of course, this line is very long, and difficult to remember, so we can create a simple alias:
alias docker_switchyard_create="docker run -it -name "switchyard" -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 -p 8787:8787 jmorales/switchyard-dev --debug -c standalone-full.xml"
alias docker_switchyard_start="docker start switchyard"
Once you are done, you can just delete this container. If you want a fresh instance, or just not working on it anymore:
docker rm -vf switchyard
It is important to delete the volumes if the container has volumes (with the -v) as otherwise this volumes will remain in your docker internal filesystem.

Demos

The demo use case is somehow different as you’ll probably have some application developed that you want to have in acontainer ready to start, so the container for demos will be an extension of the previous one, adding the application/configruation that you want.
FROM jboss/switchyard-wildfly

#
# Add admin user
RUN $JBOSS_HOME/bin/add-user.sh admin admin123! --silent

#
# Volumes
VOLUME /input
VOLUME /output

# Enable binding to all network interfaces and debugging inside the server
RUN echo "JAVA_OPTS="$JAVA_OPTS -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0"" >> ${JBOSS_HOME}/bin/standalone.conf

ADD myapp.war $JBOSS_HOME/standalone/deployments/
ADD standalone.xml $JBOSS_HOME/standalone/configuration/

ENTRYPOINT ["/opt/jboss/wildfly/bin/standalone.sh"]
CMD []
This example above has 3 main differences with the development one:
  • It removes the exposure of the debug port (not really neccesary)
  • It adds an application to the deployments dir ADD myapp.war $JBOSS_HOME/standalone/deployments/. This app has to be on the same directory as the Dockerfile
  • It adds some configuration to the server ADD standalone.xml $JBOSS_HOME/standalone/configuration/There are many ways to customize the configuration of your wildfly/EAP image, but this is one of the simplest.
And of course, if you had your image from the development use case created, you can just simply extend it.
FROM jmorales/switchyard-dev

#
# Add customizations
ADD myapp.war $JBOSS_HOME/standalone/deployments/
ADD standalone.xml $JBOSS_HOME/standalone/configuration/
Now, time to build your image:
docker build -t "jmorales/myapp-demo" .
And to run it
docker run -it --rm -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 jmorales/myapp-demo
This time I’ve added –rm to the command line, so when I stop the container, it gets deleted. As this is a demo with everything there, if I need to do it again, just run the container with the same run command line.

Support

The support use case is for somebody to make the environment reproducible, so if sombeody is experiencing a problem, he can create a docker container to isolate the app and the problem, and send it back to the support engineers. Also, some times, we want to test one app with different patches, so can be useful to have all the different versions of the server container as images, so we can try an app in them.
Let’s assume we have to try a feature in SwitchYard 1.1.1 and 2.0.0, and that we have an image for both containers:
  • jboss/switchyard:1.1.1
  • jboss/switchyard:2.0.0
We can just spin up both containers and test the app to see if it works
docker run -it --rm -P jboss/switchyard:1.1.1
docker run -it --rm -P jboss/switchyard:2.0.0
Once we have both containers (just note that if run in foreground, need to run in different terminals, and have different ports exported. I have added -P to auto export ports), we can go to the corresponding consoles and deploy the app and test.
If we want to have our tools set to certain ports (to be predictive) it is easier to have the ports defined in the command line, and use one container at a time

Integration / Testing

In the integration test case, what we want is to have our integration tools spinning up the container for us and deploying the app into this container, do all the integration tests and then stop/destroy the container.
This can easily be achievable with Jenkins (or many of the CI servers out there).
This way we can:
  • Have a clean environment for every test / app
  • Test against many different versions of the Application Server container in an automated way
When adopting testing agains containers, the time for detecting problems lowers so much, that the effort it makes to put it in action pays back very soon.

Tips for developers

As we can develop easily with docker as the container for our Application server, we can easily configure our tools to use this container, so we can have a maven profile to use our containers:
    <profile>
<id>sy-1</id>
<properties>
<jboss-as.hostname>localhost</jboss-as.hostname>
<jboss-as.port>9999</jboss-as.port>
<jboss-as.username>admin</jboss-as.username>
<jboss-as.password>admin123!</jboss-as.password>
</properties>
</profile>
<profile>
<id>sy-2</id>
<properties>
<jboss-as.hostname>localhost</jboss-as.hostname>
<jboss-as.port>19999</jboss-as.port>
<jboss-as.username>admin</jboss-as.username>
<jboss-as.password>admin123!</jboss-as.password>
</properties>
</profile>
And we can do a
mvn clean install jboss-as:deploy -P sy-1
to deploy to the SwitchYard 1 container (with port 9999 exported at localhost 9999), or
mvn clean install jboss-as:deploy -P sy-2
to deploy to the SwitchYard 2 container (with port 9999 exported at localhost 19999).

…SwitchYard contracts

Description

When working with SwitchYard we have to define the contraccts for our services as well as for our component implementations. In order to define our contracts there is different ways, and different points, depending on:
  • Component implementation
  • Service binding
As we can see in the picture above, in a simple service we can find, at least, the following contracts:
  • Component contracts
    • Component service
    • Component reference
  • Composite contracts
    • Composite service
    • Composite reference
  • Binding contracts
    • Service Binding
    • Reference Binding
SERVICES CONTRACTS SHOULD BE CORRECTLY TYPED

Component contracts

This is the contract a component exposes or uses, depending on whether it is a service or reference contract. A component contract can be defined in 3 different ways in SwitchYard:
  • Java. Using a Java interface.
  • WSDL. Using a port type in a wsdl file.
  • ESB. Using a virtual interface definition. (No real file will be used).
A component contract in SwitchYard has the following characteristics:
  • 0..1 argument. If used, this will be the message content. It is optional as there can be operations that don’t expect a message (REST GET, Scheduled operations,…​). Used in Exchanges of type IN_ONLY and IN_OUT.
  • 0..1 return type. If used, this will be the message content for the response. Used only in Exchanges of type IN_OUT.
  • 0..1 exceptions. If used, this will be the message content for the response if there is an Exception. Used in Exchanges of type IN_ONLY and IN_OUT.
Contracts in Camel can be defined with empty parameters, and you will still be able to work with the message contant. DO NOT RELY ON THIS

Java contract

A Java contract is defined by a Java Interface.

Java components require Java contracts.

WSDL contract

A WSDL contract is defined by a port type in a wsdl file.

ESB contract

An ESB contract is a virtual contract (no file required) that declares the types of the input, output and exception types.

This contract can only be used in components with one single operation.

Transformations between contracts

When a message flows in SwitchYard, and there is different types in the contracts on both ends of the “channel”, a transformation must be done. There can be implicit or explicit transformations happening, depending on where they have to happen, and there are extension points to define how the transformation between types in contracts must happen.

Composite Service Binding and Composite Service / Composite Reference and Composite Reference Binding

Contract differences are handled in the ServiceHandlers when composing and decomposing the SwitchYard message.
Any difference in contracts must be handled in the Message composer.
Camel and Http MessageComposers will do Camel Implicit Transformations if required. If target type is JAXB, the class needs to have @XMLRootElement annotation.

Composite Service and Component Service / Component Reference and Composite Reference

Contract differences are handled by transformers defined in the composite application, which are applied by theExchangeHandler chain during the execution of the Exchange.
The transformers usually map from an origin type to a destination type.
When the Exchange is between a composite service and a component service:
  • In the IN phase, from is the argument’s type of the composite service and the to is the type in the component service.
  • In the OUT phase, from is the return/exception type of the component service and the to is the return/exception typein the composite service.
When the Exchange is between a component reference and a composite reference:
  • In the IN phase, from is the argument’s type of the component reference and the to is the type in the composite reference.
  • In the OUT phase, from is the return/exception type of the composite reference and the to is the return/exception typein the component reference.
There will be some implicit transformation happening (not need to declare a transformer) if SwitchYard supports automatic transformation between both types, as declared by CamelTransformer
As of SY 1.1.1, there is no implicit transformation for primitives types

Component Service and Component Reference

Contract differences are handled in the component implementation, and has to be explicitly transformed.

Special attention

  • If a Composite Service does not declare an interface, it will use the interface defined by the promoted Component Service
  • Every Component can have one Service
  • Binding name can be null. In this case, a binding name will be automatically generated with “ServiceName+BindingType+i”
  • When the input parameter of a service contract is empty, the message will not be change, it will be in it’s original form (e.g. java.io.Reader for streaming bindings like HTTP, File,…​)

…copy files with SwitchYard

Description

While copying files from one place to another we sometimes face performance problems, and as the file gets bigger more problems, even OutofMemoryExceptions. These problems can be related to how we transfer files in SwitchYard, and if we do conversions of the files.
Here there is an explanation of the different ways to transfer a file, in streaming mode, or as a String.

Project

This project demonstrates how to copy files as stream or as a String.
  • I have used ESB interfaces, an used java.io.InputStream as input param for the copy files as stream.
  • I have used ESB interfaces, an used java.lang.String as input param for the copy files as string.
You can manage where to do conversion by having the left side as InputStream and right side as String, so you can convert at the ServiceHandler, at the Exchange (implicit transformers).
In the attached project drop a file called test.txt in /input. It will get copied in Streaming mode to /output. In the attached project drop a file called string.txt in /input. It will get copied in String mode to /output.
If using big files, copy the file with another name, and then mv it to the correct name, as otherwise copy will be happening not for the
If modifying the source, adapt source and destinations folders and filenames

Results

1 KB message

As Stream:
13:50:23,222 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #15 - file:///input) Total 1 files to consume
13:50:23,222 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #15 - file:///input) About to process file: GenericFile[/input/test.txt] using exchange: Exchange[test.txt]
13:50:23,224 DEBUG [org.apache.camel.component.file.GenericFileConverter] (Camel (camel-15) thread #15 - file:///input) Read file /input/test.txt (no charset)
13:50:23,231 DEBUG [org.apache.camel.component.file.FileOperations] (Camel (camel-15) thread #15 - file:///input) Using InputStream to write file: /output/ID-b64fd2f46bfd-46028-1412943213101-14-60
13:50:23,232 DEBUG [org.apache.camel.component.file.GenericFileProducer] (Camel (camel-15) thread #15 - file:///input) Wrote [/output/ID-b64fd2f46bfd-46028-1412943213101-14-60] to [Endpoint[file:///output]]
13:50:23,234 DEBUG [org.apache.camel.component.file.GenericFileOnCompletion] (Camel (camel-15) thread #15 - file:///input) Done processing file: GenericFile[/input/test.txt] using exchange: Exchange[null]
13:50:23,275 DEBUG [org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy] (Camel (camel-15) thread #15 - file:///input) Renaming file:
GenericFile[/input/test.txt] to: GenericFile[/input/.camel/test.txt]
As String:
13:50:46,976 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #16 - file:///input) Total 1 files to consume
13:50:46,976 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #16 - file:///input) About to process file: GenericFile[/input/string.txt] using exchange: Exchange[string.txt]
13:50:46,977 DEBUG [org.apache.camel.component.file.GenericFileConverter] (Camel (camel-15) thread #16 - file:///input) Read file /input/string.txt (no charset)
13:50:46,983 DEBUG [org.apache.camel.component.file.FileOperations] (Camel (camel-15) thread #16 - file:///input) Using InputStream to write file: /output/ID-b64fd2f46bfd-46028-1412943213101-14-70
13:50:46,984 DEBUG [org.apache.camel.component.file.GenericFileProducer] (Camel (camel-15) thread #16 - file:///input) Wrote [/output/ID-b64fd2f46bfd-46028-1412943213101-14-70] to [Endpoint[file:///output]]
13:50:46,986 DEBUG [org.apache.camel.component.file.GenericFileOnCompletion] (Camel (camel-15) thread #16 - file:///input) Done processing file: GenericFile[/input/string.txt] using exchange: Exchange[null]
13:50:46,987 DEBUG [org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy] (Camel (camel-15) thread #16 - file:///input) Renaming file:
GenericFile[/input/string.txt] to: GenericFile[/input/.camel/string.txt]
There is no significative difference.

400 MB file

As Stream:
13:52:58,301 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #15 - file:///input) Total 1 files to consume
13:52:58,302 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #15 - file:///input) About to process file: GenericFile[/input/test.txt] using exchange: Exchange[test.txt]
13:52:58,304 DEBUG [org.apache.camel.component.file.GenericFileConverter] (Camel (camel-15) thread #15 - file:///input) Read file /input/test.txt (no charset)
13:52:58,311 DEBUG [org.apache.camel.component.file.FileOperations] (Camel (camel-15) thread #15 - file:///input) Using InputStream to write file: /output/ID-b64fd2f46bfd-46028-1412943213101-14-80
13:52:58,526 DEBUG [org.apache.camel.component.file.GenericFileProducer] (Camel (camel-15) thread #15 - file:///input) Wrote [/output/ID-b64fd2f46bfd-46028-1412943213101-14-80] to [Endpoint[file:///output]]
13:52:58,528 DEBUG [org.apache.camel.component.file.GenericFileOnCompletion] (Camel (camel-15) thread #15 - file:///input) Done processing file: GenericFile[/input/test.txt] using exchange: Exchange[null]
13:52:58,528 DEBUG [org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy] (Camel (camel-15) thread #15 - file:///input) Renaming file:
GenericFile[/input/test.txt] to: GenericFile[/input/.camel/test.txt]
As String:
13:53:42,017 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #16 - file:///input) Total 1 files to consume
13:53:42,017 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #16 - file:///input) About to process file: GenericFile[/input/string.txt] using exchange: Exchange[string.txt]
13:53:42,019 DEBUG [org.apache.camel.component.file.GenericFileConverter] (Camel (camel-15) thread #16 - file:///input) Read file /input/string.txt (no charset)
13:53:43,535 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #15 - file:///input) Took 0.000 seconds to poll: /input
13:53:48,535 DEBUG [org.apache.camel.component.file.FileConsumer] (Camel (camel-15) thread #15 - file:///input) Took 0.000 seconds to poll: /input
13:53:51,794 ERROR [org.apache.camel.processor.DefaultErrorHandler] (Camel (camel-15) thread #16 - file:///input) Failed delivery for (MessageId: ID-b64fd2f46bfd-46028-1412943213101-14-83 on ExchangeId: ID-b64fd2f46bfd-46028-1412943213101-14-84). Exhausted after delivery attempt: 1 caught: org.apache.camel.TypeConversionException: Error during type conversion from type: org.apache.camel.component.file.GenericFile to the required type: java.lang.String with value GenericFile[/input/string.txt] due java.lang.OutOfMemoryError: Java heap space: org.apache.camel.TypeConversionException: Error during type conversion from type: org.apache.camel.component.file.GenericFile to the required type: java.lang.String with value GenericFile[/input/string.txt] due java.lang.OutOfMemoryError: Java heap space
at org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:126) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:72) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:47) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.switchyard.component.camel.common.composer.CamelMessageComposer.compose(CamelMessageComposer.java:62) [switchyard-component-common-camel-1.1.1-p7-redhat-1.jar:1.1.1-p7-redhat-1]
at org.switchyard.component.camel.common.composer.CamelMessageComposer.compose(CamelMessageComposer.java:42) [switchyard-component-common-camel-1.1.1-p7-redhat-1.jar:1.1.1-p7-redhat-1]
at org.switchyard.component.camel.SwitchYardProducer.composeForGateway(SwitchYardProducer.java:136) [switchyard-component-camel-switchyard-1.1.1-p7-redhat-1.jar:1.1.1-p7-redhat-1]
at org.switchyard.component.camel.SwitchYardProducer.process(SwitchYardProducer.java:104) [switchyard-component-camel-switchyard-1.1.1-p7-redhat-1.jar:1.1.1-p7-redhat-1]
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:122) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:298) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:117) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.fabric.FabricTraceProcessor.process(FabricTraceProcessor.java:81) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:334) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:46) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:308) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:117) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:46) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.UnitOfWorkProcessor.processAsync(UnitOfWorkProcessor.java:150) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:117) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.RouteInflightRepositoryProcessor.processNext(RouteInflightRepositoryProcessor.java:48) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:336) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:189) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:155) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:142) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [rt.jar:1.7.0_65]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [rt.jar:1.7.0_65]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.7.0_65]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65]
Caused by: org.apache.camel.RuntimeCamelException: java.lang.OutOfMemoryError: Java heap space
at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1326) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:972) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:47) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(BaseTypeConverterRegistry.java:253) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
at org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:111) [camel-core-2.10.0.redhat-60061.jar:2.10.0.redhat-60061]
... 57 more
Caused by: java.lang.OutOfMemoryError: Java heap space

13:53:51,796 DEBUG [org.apache.camel.component.file.GenericFileOnCompletion] (Camel (camel-15) thread #16 - file:///input) Done processing file: GenericFile[/input/string.txt] using exchange: Exchange[null]
13:53:51,796 WARN [org.apache.camel.component.file.GenericFileOnCompletion] (Camel (camel-15) thread #16 - file:///input) Rollback file strategy: org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy@164d2158 for file: GenericFile[/input/string.txt]
As you can see, streaming is recommended for big files. So be careful whatever you do in an Streaming file copy with the message content, as any log, transformation,…​ can convert it to String and give an OOME.

…SwitchYard development guidelines (I)

This document describes guidelines and best practices on how to develop services. Although there are many ways of achieving the same functionality in SwitchYard, these is the recommended approach to ease development effort and to leverage all the capabilities provided by the engine.

Strongly type your contracts

As the input type in an operation’s contract is the Message, you should always use a type if you expect a message. Even if it works without a message it is in no way recommended, and can lead to multiple problems with difficult dianostic.

Wrong

public interface MyWrongContract {
public void operation();
}

Right

public interface MyRightContract {
public void operation(MyType input);
}

Use the appropiate types in your contracts

If you want to leverage the type conversion provided with SwitchYard, and also be able to efficiently work with your information (objects/types) in your components it is strongly adviced to use the appropiate types, and not relying on generic types, that will need conversions.
Sometimes can be convenient to use a generic type, mostly to avoid conversions, but it is not a general rule.

Wrong

public interface MyContract {
public void operation(org.w3c.dom.Element input);
}

Right

public interface MyContract {
public void operation(com.company.type.MyType input);
}

Use declarative transformations

When you have to tranform a type, like when a Composite Service interface is different from the Component Service interface, you can instruct SwitchYard to do the transformation for you. It will be applied automatically while processing the wire.
You can add required transformers or a specific tranformation, and you can use one of the multiple transformations mechanisms. See: https://docs.jboss.org/author/display/SWITCHYARD/Transformation

If using Java transformations, create common functionality, to not repeat yourself

When using Java transformations, and more if transforming to/from XML, you’ll end up adding same code in many transformations. Extract a base class into your utils to ease Java/XML conversion.
public abstract class BaseTransformers {

/**
* Gets an element value for transforming from Element to Java Type
*
* Usage: MyType myTransform(Element e) {
* String s = getElementValue(e, "xxx");
* return new MyType(s);
* }
*/

private String getElementValue(Element parent, String elementName) {
String value = null;
NodeList nodes = parent.getElementsByTagName(elementName);
if (nodes.getLength() > 0) {
value = nodes.item(0).getChildNodes().item(0).getNodeValue();
}
return value;
}

/**
* Creates an Element, for tranforming from Java Type to Element
*
* Usage: public Element myTransform(MyType from) {
* String xml = "<xxx>" + from.toString + "</xxx>";
* return toElement(xml);
* }
*
*/

private Element toElement(String xml) {
DOMResult dom = new DOMResult();
try {
TransformerFactory.newInstance().newTransformer().transform(new StreamSource(new StringReader(xml)), dom);
} catch (Exception e) {
e.printStackTrace();
}
return ((Document)dom.getNode()).getDocumentElement();
}

Use @OperationTypes in your interface if you want to instruct SwitchYard for a different type than that in the interface

Sometimes it can be convenient to have the XML representation of an Object and not require SwitchYard to do a transformation from the XML type to a String. For this cases (and many others) can be convenient to annotate your interface so SwitchYard knows it doesn’t need to transform to the String type.
public interface OrderService {

@OperationTypes(in = "{urn:switchyard-quickstart:bean-service:1.0}submitOrder")
void submitOrder(String orderXML);
}
In this way, the String that will get to the submitOrder method will be:
<urn:submitOrder xmlns:urn="urn:switchyard-quickstart:bean-service:1.0">
<order>
<orderId>1</orderId>
<itemId>1</itemId>
<quantity>100</quantity>
</order>
</urn:submitOrder>

Get access to the Context in a Transformer

@Context annotation is limited to be used inside of components. If you need to get access to the context in a SwitchYard transformer, you can qualify the “from” and use Message as input type in the transformer method.
@Transformer(from = "java:com.example.ClassOne")
public ClassTwo transform(Message message) {
Context context = message.getContext();
....
return classTwo;
}

Use MessageComposers to adapt the message transformation from/to Message from/to Binding

Message composers are used to translate the binding message (SOAPEnvelope, HTTP Body, Rest Body, File contents, …​) into the SwitchYard Message. This is the place to put logic to customize this transformation. And only transformation of the message/body.
Create a custom MessageComposer (best to extend the one for the binding you are using) and override the appropriate method:
compose
Takes the data from the passed in source object and composes a SwithYardMessage based on the specified Exchange.
public Message compose(D source, Exchange exchange) throws Exception;
decompose
Takes the data from the SwitchYardMessage in the specified Exchange and decomposes it into the target object.
public D decompose(Exchange exchange, D target) throws Exception;

Use ContextMappers to customize context mapping from/to Context from/to Binding

Context mappers are used to translate the binding context (SOAP Headers, HTTP Headers, …​) into the SwitchYard Context. This is the place to put logic to customize the creation and assigment of information into the Context.
Create a custom ContextMapper (best to extend the one for the binding you are using) and override the appropriate method:
mapFrom
Maps a source object’s properties to the context.
public void mapFrom(D source, Context context) throws Exception;
mapTo
Maps a context’s properties into a target object.
public void mapTo(Context context, D target) throws Exception;

Use SCA to decompose and decouple your applications

Usually applications start to grow big, and very coupled the information in there. Many wires from services to components to wires,…​ In such an scenario it is very good option to think as SCA binding as the solution, as introduces 2 benefits:
  • Allows for easy decouplig and composition of applications.
  • Application dependency is only honoured at runtime. You can deploy appA that has an sca call to appB before even deploying appB. It will only fail at runtime if you do not deploy appB.

Externalize configuration from the system (but be aware of the limitations)

Applications should not have configuration information that may be different in different environments hard coded. Even component, composite and domain properties are not good for this, as they require rebuilding of the application, so use JBossAS properties or System.properties or environment to externalize your configuration.
Configuration is applied typically at activation of a service, so any Environment Property will be read, and if changed will require, at a minimum, from an application reactivation, or a server restart.
JBossAS7 properties can be changed from the console, and if you read them in your execution path will be hot-reloaded, otherwise, if it is configuration property, will still require an application reactivation.
static property
This property will be read once (when the class is instantiated) and needs an application reactivation/redeployment to change
static String password = System.getProperty("password");

public void process(Exchange exchange) throws Exception {
....
}
dynamic property
This property will be read per every invocation and will be changed in real time
public void process(Exchange exchange) throws Exception {
String password = System.getProperty("password");

....
}

..SwitchYard ExchangeHandler Chains

Communications between components and bindings to/from components is done by a LocalExchangeBus.
This is the wire
There is a chain of ExchangeHandlers acting as interceptors for the request and response pipelines of the wire. The default chain has the following:
  • RequestChain
    • addressing: AddressingHandler resolves service instances based on a service reference.
    • transaction-pre-invoke: TransactionHandler interprets transactional policy specified on an exchange and handles transactional requirements. Suspends current transaction and creates a new one, creates a new one, join an ongoing transaction,…​
    • security-process: SecurityHandler interprets security policy specified on an exchange and handles security requirements. Cretes the SecurityContext and SecurityProvider and attaches to the thread.
    • generic-policy: PolicyHandler
    • validation-before-transform: ValidateHandler executes the org.switchyard.validate.Validator defined for the Exchange`s content type.
    • transformation: TransformHandler applies all the defined transformations (org.switchyard.transform.TRansformer) for the message for consumer.inputType to provider.inputType and set the Exchange’s new content type.
    • validation-after-transform: ValidateHandler executes the org.switchyard.validate.Validator defined for the new Exchange`s content type.
    • provider: ProviderHandler dispatches the message to the provider (component).
    • security-cleanup: SecurityHandler cleanup of context and thread info.
    • transaction-post-invoke: TransactionHandler commits or rollback, or resume suspended transaction,…​
Only IN_OUT Exchanges will execute the ReplyChain, that’s why there are 2 Handlers after the execution of the provider, to cleanup/finish tasks.
  • ReplyChain
    • validation-before-transform: ValidateHandler executes the org.switchyard.validate.Validator defined for the Exchange`s content type.
    • transformation: TransformHandler applies all the defined transformations (org.switchyard.transform.TRansformer) for the message for provider.outputType to consumer.outputType and set the Exchange’s new content type.
    • validation-after-transform: ValidateHandler executes the org.switchyard.validate.Validator defined for the new Exchange`s content type.
    • HandlerChain.CONSUMER_HANDLER: BaseHandler