Create Interactive BPMN Models Using JavaScript

In this tutorial I would like to show how to visualize BPMN files in a web browser.
All we need is jQuery for easy DOM navigation, XMLJS to parse the BPMN file and Raphael to visualize it. Since BPMN version 2.0 the BPMN XML file contains additional diagram interchange information. This information together with the actual process data can be used to draw process diagrams from the XML with little effort.

Step 1: Create the process
First we create a simple BPMN2.0 process like the one below using our tool of choice. In this case it was created with Signavio Modeler.

If we open the bpmn file in a text editor it looks like this:

What we see is that in addition to the actual process information it contains elements named BPMNShape and BPMNEdge. BPMNShape contains the bounds of the shape whereas BPMNEdge contains the path of the respective shape connector. This is the diagram interchange format which specifies the size and location of the graphical elements and paths.

Step 2: Prepare the HTML page
We need a simple HTML page that loads the required scripts for jQuery, XMLJS, Raphael and of course our own script bpmnjs.js.

The textarea element (line 20) contains the BPMN data. It is omitted here for the sake of readability.
The div element (line 23) specifies the drawing area that is needed to output the graphics.
Both are needed to initialize BPMNJS in the document ready event (line 14-15).

Step 3: Parse the DI elements using XMLJS
The following listing shows the parsing loop for the BPMN shapes within the plot function of BPMNJS.

First the XMLJS dom parser is initialized (line 49-51). The bpmndi namespace is determined in the getNamespaces function at line 54. Line 57 obtains all shapes. The rest of the listing iterates over the shapes, gets positions, sizes and bpmnElement-ids of the related BPMN elements and passes them to the paintShape function which performs the painting. The parsing loop for the edges is almost identical. The simplest possible paintShape implementation would look like the following.

The constructor function (lines 2-4) connects Raphael with our div and stores the Raphael instance in a variable called paper. The paper is used in the function paintShape which draws a simple rectangle using the given coordinates (line 8).
The result looks like this:

A few lines of JavaScript and we see the BPMN skeleton already. Not too bad.

Step 4: Find related process elements
In order to draw specific BPMN elements we have to determine the element type. The bpmnElement attribute that we have got in step 3 helps us to do that. We use it to get the process element by issuing an xPath expression on the DOM as you can see in the next listing (line 128).

The localName of the element is the name of the BPMN element an therefore the type such as startEvent. With this information we can implement a dispatcher to draw all BPMN elements individually (lines 130-133).

Step 5: Draw elements using Raphael and CSS
Raphael is a powerful JavaScript library that allows to paint graphical primitives easily.
The next listing shows an implementation to draw a basic BPMN task.

First the rectangle is painted (line 3) then the text is written onto the shape (lines 6-8). We can even apply CSS stylesheets to the shapes that allow very flexible customization without changing the JavaScript code (line 10). The function getCss determines the CSS class name. The simplest implementation just uses the elementType as CSS class name as you can see in the following CSS snippet.

After implementing some more type specific paint functions the result looks like this:

Step 6: Highlight paths
One of the advantages of painting the BPMN model at runtime over a static JPG like the one shown in step 1 is that it allows interactivity. For instance we could highlight the actual token flow in the model. To do that we need information about the executed BPMN steps. This can typically be obtained from a process tracking system which is part of almost every BPM product. To achieve highlighting we extend the getCss function to return a different class if the current shape is in the list of already executed steps.

We add the postfix -high to the CSS class in case the element should be highlighted. By specifying for example the class .task-high we can achieve highlighting for shapes and edges. The result is show below.

Step 7: Add interactivity
Finally we can even add interactivity to the model with a little help from Raphael.

As this can not be demonstrated using static images you can find an interactive version here. In this version you can hover over and click individual shapes to make them react.

Summary
With the help of jQuery, XMLJS, Raphael and less than 300 lines of JavaScript code, this blog post has shown how to create interactive BPMN models in your web browser. The complete demo can be downloaded here.

Integrated Process Management with Open Source

If you ever tried to create an execution environment to automate business- or integration processes based on Open Source products, you know that this is not an easy task. Although Open Source products like Activiti or Apache Camel are of high quality, they do not run with production grade quality out-of-the-box. For serious usage scenarios typically a lot of work is required to integrate those products into a sound platform. This fact hinders companies to use those great products and turn to closed source alternatives from Oracle, Appian or Inubit, just to name a few.

Now there is an interesting alternative called oparo. oparo is an integrated process automation platform based on rock solid Open Source products. oparo is not limited to BPMN processes only. It rather focuses on the entire process spanning business, workflow, mediation and integration.

The platform does all the plumbing required to turn single products such as Activiti, Apache Camel, Apache ActiveMQ, Lucene/Solr, etc. into a platform that can be used out-of-the box. Even better, oparo is entirely ASF2.0 licensed (today and tomorrow) which offers broad usage options and does not involve any hidden costs for enterprise features.
oparo shields the process engineer (the guy who analyses and automates processes) as much as possible from low level technical tasks such as connecting and transforming Camel and Activiti message payloads. It offers a unified development approach for the process engineer to focus on business functionality instead of technical plumbing. Moreover it comprises additional valuable services such as process flow tracking, humantask integration or a registry. Due to oparos service binding approach, those services can be easily integrated in existing IT landscapes using almost any technology (e.g. .NET, JEE, HTML5/JS/CSS). The runtime is scalable (in terms of technology and licenses), the set up is automated and the whole platform is based on proven standards.

If that sounds promising, you can give it a try. You can find more information and a downloadable jumpstart distribution at oparo – the efficient process platform (German only)

Business Process Evolution and Versioning

(Automated) business processes evolve over time! And they usually evolve faster than IT systems do.
So how can business process changes be delivered to the users quickly?

Let’s look at an example:
Assume we have a process for vacation planning for the staff of a large company. Initially the process was automated based on the knowledge of the human resource department. After 2 months new insights require a process change. The process should be optimized to speed up the decison whether vacation is granted or not. The process has evolved and the changes have to be put in place as soon as possible. This is a common situation and actually one of the promises of business process management is: Deliver business value fast.

Sounds simple, but how can we deliver the changed process?

There are serveral options to put the changed process in place:

Option 1: Parallel
The changed process coexists with the initial one for a period of time. Existing process instances must continue with the inital process definition.

Example: Users of the process are gradually trained to use the changed process. Some departments can still use the initial process, some use the new one. The process is triggered by IT systems as well. Those systems should have a smooth upgrade path.

Action: Create a new version of the process and deploy it in parallel to the one already in place.

|--- Startable V1 -------->
|--- Instances V1 -------->
                 |--- Startable V2 --------->
                 |--- Instances V2  -------->

Option 2: Merge
The changed process replaces the initial one. Existing process instances must continue using the changed process definition.

Example: Law changes render invalid the initial process. As of now all processes, including already running instances, must run with the latest process definition.

Action: Create a new version of the process and migrate existing instances to the new process definition.

|--- Startable V1 ------|--- Startable V2 --------->
|--- Instances V1 ------|--- Instances V1 + V2 ---->

Option 3: Phase Out
The changed process replaces the initial one. Existing process instances must continue with the inital process definition.

Example: Process analysis caused the process to be optimized, so that it can be executed in less time. All users should immediately use the changed process.
To keep effort low, already running process instances should continue running with the inital process definition.

Action: Create a new version of the process and deploy it in addition to the one already in place. Prevent the initial process version to be started by disabling the start events.

|--- Startable V1 --------|
|--- Instances V1 --------------------|
                          |--- Startable V2 --------->
                          |--- Instances V2  -------->

Be aware of endpoints:
If process versions are provided in parallel like in scenario 1 and 3 and connected to technical endpoints, for instance filedrops or web services, those endpoints might collide. Changing the structure of an endpoint, for instance the message payload, might cause incompatibility as well. In those cases (which are likely to happen) the endpoints must be versioned. Alternatively a dispatching mechanism can be used to route messages to the appropriate process version.

As you can see versioning is am important concept for process evolution. Which strategy to use depends on the process and the particular business requirements. The options introduced in this blog post might help to take the right decision. Make sure your process platform supports the options you need.

The Future of ServiceMix

In the ServiceMix forum there is an interesting debate going on about the Future of ServiceMix. I’ve collected some quotes from the thread. I know it is sometimes dangerous to cite people without giving the full context, but I think it helps to get a rough idea about the direction in which ServiceMix is heading. If you are interested in more details please read to original post.

Quotes:

“The idea is really to make ServiceMix the best possible container for deploying Camel based integration.”
 
“This also means that there’s really not much to reuse from smx4, so smx5 would have its own new and dedicated svn area.”
 
“Dropping JBI is already a great step in that direction.”
 
“Additional things that could be pushed inside ServiceMix 5 would be a Tomcat based container deployment option (for those that don’t need OSGi), a new manual similar to what we have in Karaf (maybe reusing parts of it).”
 
“For advanced use cases, we should definitely guide users towards Karaf, but for a set of basic things, just using Tomcat and simple WARs may be the simplest thing that works for them!”
 
“To build the added value I’ve been talking about, we don’t really need the NMR imho.  I’m convinced we can do it directly on top of camel without the need for another layer, mostly using Camel interceptors.   That way, OSGi is not really needed anymore, though it’s still the best way to deploy Camel routes  using Karaf and would remain the main and standlone ServiceMix distribution.”
 
“Providing a way to easily deploy Activiti would definitely be a good thing. I don’t think it should part of the default distribution, as I’d like to keep it as light as possible though.”
 
Here is my personal summary: 
  1. ServiceMix 5 will primarily focus on Camel.
  2. JBI is finally dead.
  3. Karaf and OSGi will represent the primary runtime.
  4. Tomcat will act as an alternative container.
  5. Activiti will not be integrated.

Consequences:

If you develop for ServiceMix 4 today, focus on OSGi and Camel instead of  JBI and ServixMix features since those are the stable parts of ServiceMix.
If you have ServiceMix 3 and JBI solutions in place think about migration towards Karaf and Camel. If you plan to integrate Activiti, make sure the required functionality is not already covered by Camel to avoid unnecessary complexity (see blog post Combining Activiti and Camel).

Combining Activiti and Camel?

It seems that that ServiceMix/Camel and Activiti are growing together as you can read in the blog post Supersize Activiti with Mule ESB and Apache Camel and Deploy Activiti as OSGi Bundles

Activiti has its roots in business process management, whereas Camel comes from the area of enterprise integration. Both allow to describe and automate processes. Activiti uses BPMN, a graphical modeling language with an XML representation for persistence and execution. Camel uses a textual DSL which can also be expressed in XML. Grahpical modeling is also possible with the Fuse IDE.

BPMN example:

BPMN process
BPMN process

The BPMN XML representation is not intended to be written directly and can be hard to read.

Camel example:

from("file://start")
.choice()
  .when(header("goldCustomer").isEqualTo("true"))
    .to("bean:processor?method=grantDiscount")
    .to("direct:out")
  .otherwise()
    .to("direct:out");

from("direct:out")
.to("file://end");

Basically both examples are identical. The Camel example is more explicit and shows more technical details which are hidden in the BPMN example.

Especially non technical users tend to think that a graphical process notation is easier to understand because it looks better. It might be true for simple processes like the one above. But for more complex processes textual DSLs are often easier to write and understand.

Activiti and Camel overlap in many areas. Does it make sense to use them side by side anyway? I think yes, because neither Activiti nor Camel is complete in terms of features required by a fully featured SOA/BPM runtime. Actually they complement each other very well in some areas.

Unique features added by Camel:

  • Connectivity / Adaptors to connect to external systems using a variety of different protocols 
  • Predefined EIP patterns for message routing

Unique features added by Activiti:

  • Support for long running stateful processes
  • Human workflow integration

In order to avoid unnecessary complexity it helps to recall the difference between mediation- and business processes which I descibed in my blog post from 2006.

For mediation processes Camel is appropriate in most cases, even long running processes can be executed as long as they are stateless. But if you need long running stateful processes and human interaction Activiti is an interesting option. 

Both Camel and Activiti processes can be modeled using prescriptive and analytical BPMN. The software designer has the responsibility to decide which technology is appropriate for implementing the process. If this is not done right you will end up with unnecessary complexity, limited agility and maintainability.

It would be ideal to have an Open Source SOA/BPM Platform that enables end to end process automation without technology boundaries. As long as this is not available, combining Activiti and Camel is an interesting option. Proper solution design and differentiation of process types is key to create sustainable solutions.

Polyglot Workflows with Activiti and Silverlight

My latest article Polyglot Workflows with Activiti and Silverlight has been published in the current issue of JavaSPEKTRUM. After losts of  theory in form of articles and conference sessions, I thought it would be a good idea to present a real example.  The Activiti engine is an interesting product in the area of BPM. If you go beyond simple demo applications you might want to integrate it with existing UIs, such as corporate portals. Silverlight is an interesting candidate for this task. The article focuses mainly on the interoperability aspects of Activiti and Silverlight and shows how easy it is to bring both together.

Executable BPMN for Business Analysts?

People asking for the differences between BPMN and BPEL they often hear BPMN is graph oriented whereas BPEL is block oriented. But what does that actually mean and why is it important?

Block orientation means that there is no way back in the process like a goto statement. In BPEL a common workaround is to use loops to move backwards in a process. But this approach is cumbersome and creates non intuitive models. Block orientation feels natural for developers as it is how programming languages such as Java or C# work.

For business people this feels odd, cause it is not how they perceive processes in their daily business.
Business people regularly need a way to say “Oh wait, there is a problem in step X, let’s go back to step A and fix it.”.

Graph orientation means that the process can flow between tasks even backwards. In terms of Business/IT aligment graph orientation is a clear advantage as it is closer to how people think and work.

Because BPMN is graph oriented it is way better suited for business analysts than BPEL. Does that mean that with BPMN business people can create executable processes? In theory yes! And in reality?

As long as the BPMN model is purely prescriptive, people can freely connect tasks in all directions (more or less). But if the aim is to create executable process models, the BPMN semantics come into play. Although your modeling tool might allow to create all kind of links, process engines are very strict in what they can do. An executable model is like source code and as such follows strict rules.

Let’s create a simple example:

For an analyst this might look like a perfectly modeled process. The problem is that it ignores the BPMN semantics. Neither the modeling tool (Signavio) nor the process engine (Activity) complains, but at runtime the process runs into a deadlock situation. Why that?
The parallel gateway used to join the process after the execution of Task 1 and Task 2 expects two tokens. On the first run it gets two tokens and everyting is fine. If the completed gateway continues with Task 2 on the second run, the joining gateway receives only one token and the process is on hold cause the second token never arrives.

Conclusion:
If the aim is process automation, it is essential that the process strictly adheres to the BPMN semantics.
For most business oriented people (at least the ones that I know) this is new completely new terrain.

Activiti and the Open Source Reference Platform

Now that Activity published the first release (version 5.0) of their Open Source BPMN 2.0 engine and toolset, it was time to integrate it in our Open Source Reference Platform for SOA (OSRP).

Despite some initial weaknesses, such as BPMN error handling, Actitivi is very promising if you are looking for a lightweight, Open Source BPM engine. Especially the features in the areas IT/Business collaboration and human task  integration are worth a closer look. The modularity makes it easy to integrate the engine with other products.

Using the OSRP one can now compare the BPMN and BPEL approaches to BPM, understand how Activiti can be combined with other Open Source SOA technologies such as Apache ServiceMix ESB, and learn about the strenghts and limitations of the product.