Efficient technology for the enterprise – Page 7

How to Staff an Agile Team

Although there is a greater likeliness of success in Scrum projects than in non-Scum projects, Scrum projects sometimes fail as well. If you ask the people involved in failed Scrum projects, they quickly accuse Scrum of being the cause for the failure. They claim to have done everything that Scrum requires, but failed, so the method is blamed.
In agile software development the most important factor is the team, the team and … … yes, the team. But a common misconception is that you just have to put together a few people to get a team that performs well. This is completely wrong, as a group of people is something very much different than a team. A group is just a bunch of individuals who neither strive for the same aim nor have a deep common understanding of the project. And they often do not trust each other enough to perform well. A team is different. People in a team trust each other, they strive for a common aim and share a deep understanding. And they have fun doing what they do. But how can a group be turned into a team?

There are well known social processes that every group has to undergo to become a team. One of those processes is the Tuckman principle of forming, storming, norming, performing (and adjourning). In their readable article Teamwork: Why teams are more successful than groups. Dr. Eberhard Huber and Sven Lindenhahn describe key factors of successful agile teams. This very much matches my expericence both as team member and coach of Scrum teams. It is essential that the group undergoes a productive storming phase in which an internal hierarchy and decision making structure is cultivated. This can be hard and tiring, but is essential for success. The key is to bring together the right mixture of individuals who have the interpersonal skills to find their place in a larger group of people in a constructive way. The ability to make compromises is important.

And even agile teams need leadership! Not from the outside in form of a project manager, but rather from inside the team. There must be people with the interpersonal and technical skills to take leadership. The authority can’t be given by management, but needs to be earned every day. Personality is key.

If you staff your next agile team, make sure you have the right mixture of skills on board. I am not talking about just technical skills, that’s self-evident. Technical knowledge can be easily shared in an agile context. What I mean are personal skills such as:

Ability to make compromises
Ability to accept constructive criticism
Ability to take responsibility
Ability to take leadership when needed

Don’t expect that the team works in harmony from day one. It is absolutely normal that the team members argue a lot, especially at the beginning. This is not a sign of bad team constellation. It is rather a natural step towards a productive agile team.

Agility through Business Process Automation?

Sometimes business process automation (BPA) is described as the silver bullet to improve agility and time to market. Especially large vendors spend huge amounts of marketing budget to promote their BPM tool suites, “360 Degree”- and “Zero Code”-approaches.

But why does BPM increasy agility? Is it really easier to adapt processes to business changes if a process has been automated using a BPM suite?

Sometimes yes, in the narrow range of allowed options. Often no, cause IT-coded processes are not as flexible as people in an organisation. But that does not mean that business process automation is a bad idea at all. There are areas in which process automation makes perfect sense.

Especially processes for which the following factors apply:

clearly structured and predictable
repetetive
frequently executed

Interestingly, often agility does not come from automated processes itself, but rather from the people who have their hands free for other more sophisticated ad hoc processes. We have experienced that in a large project for an international organisation from the public sector. Provided people have the right skills, BPA can help turning people from “routine workers” to “knowledge workers” (see It is All Taylor’s Fault). BPA allowed them to automate their repetetive tasks. It was a great improvement and productivity gain for the people and the organisation. The key was to give them a tooling that they were able to control, even without much help from IT guys.

Knowledge workers do not need their processes automated. They need other tools mostly to get the right structured information at the right time. IT can help in this regard, but not via BPA. I would call this Business Process Facilitation (BPF) rather than automation.

BPF means giving the people tools to do their job in a efficient manner without imposing predefined processes on them. In other words, it leaves the process and decision power with the people not the machines. User centered dashboards, search engines and adaptive case management tools are examples for BPF. We have experienced this in another project in which we evaluated the value of process automation using a BPM-Suite. In this highly dynamic environment the decision was to not implement BPM as it would have hindered agilty. Instead we implemented BPF to support the knowledge workers. The system mainly focused on efficient data management and decision making.

All in all it is not black and white, not Taylorism against knowledge work. Success comes from a combination of both. The key is focusing on things that are beneficial for the people and organisation. Sometimes it is automation, sometimes not. Process automation is cleary no silver bullet, but if applied wisely and with the right focus it can help organisations to improve efficiency.

Scrum Planning with Enterprise Architect

Although keeping a Scrum team together 100% on-site is the ideal situation, sometimes it is not possible and the team works distributed. In such situations it might be handy to have a tool that can be used instead the whiteboard. A kind of virtual whiteboard that is accessible from everwhere. A popular modeling tool in IT projects is Sparx Enterprise Architect due to its rich modeling capabilities and fair pricing. In order to use it for Scrum planning, I created a UML-Profile which can be used to easily create and maintain Scrum user stories. The profile contains stereotypes for epics, user stories and tasks together with tagged values to track business value, story points and team responsibilities. In order to use it, download the profile and import it using the resource view in Enterprise Architect as show below.

To use the Scrum model elements, open the toolbox and add them to the palette using the More tools … action at the top of the toolbox dialog.

Now create a new diagram for your Sprint planning. You can use swimlanes (Diagram- Swimlanes and Matrix in the menu) to add stage regions to the diagram. Right click on the diagram pane and click properties in context menu. On the elements tab select Tags and Notes. This gives the Scrum elements a card like appearance on the diagram. That’s it. Now you should have something like this:

You can quickly change story points, etc. and the notes using the tagged values and notes view. By dragging the elements between the swimlanes you can change the status of the respective user stories. The package browser (right click on a package in the project browser) can be used to show all stories and tasks in a table view.

A nice gimmick since Enterprise Archtect 9 is the ability to visualize diagrams in whiteboard mode. Just tick the options Whiteboard Mode and Hand Drawn in the diagram options and you will see something like this:

Because the whole model is stored in a repository, you can create story dependencies by connecting the elements, automatically determine backlog sizes and print story cards based on custom templates.

In my opinion using the whiteboard is still the most efficient way for Scrum planning. But especially in cases where teams are distributed, such an approach is hardly possible. As shown in this post, Enterprise Archictect can easily be turned into an effective Scrum planning tool that can be used in those situations.

You can download the Scrum profile here: Scrum.xml
If you want to change the team members, make sure to edit them in the profile before importing it into Enterprise Architect.

BPMN 2.0 Model Interchange with Enterprise Architect, Signavio Modeler and Activiti Designer

I have been using the Enterprise Architect modeling tool for many projects. I like the tool especially due to its quick modeling capabilities and productivity. Therefore is ideal for workshops with participants from IT and business. Performing live modeling together with the domain experts is a very efficient approach to gather accurate information related to processes and IT systems.

So I am very pleased to see that Enterprise Architect 9 now supports the creation of BPMN 2.0 models as well. And even better, models can be easily exported in form of BPMN 2.0 interchange format via Project – Model Publisher in the Enterprise Architect menubar (by the way, import is not possible. :-().
Anyway, I gave it a try and exported a simple model from EA:

To prove that the interchange works, I imported the model in my favorite BPMN tool the Signavio Modeler. This is the result:

That’s not too bad, is it? But it seems that some model details are missing. The dataInputAssociation and dataOutputAssociation, the scriptFormat and the groovy script attached to the ScriptTask were not imported.
It seems that (at least for those two tools) BPMN2.0 model interchange is not yet sufficiently possible.

Update:
I imported the bpmn file into the Activiti Designer Eclipse Plugin version 5.7.1 as you can see below:

The Groovy script was successfully imported, but the dataInputAssociation and dataOutputAssociation was not recognized and therefore removed from the process definition. The original file is solely used for the import. After importing an *.activiti file is created and used as THE model. On saving the model, a fresh BPMN XML file is generated. That is, you loose control over the file at the XML level. I personally don’t like this, as I think executable BPMN files should be treated as source code and need to be fully controlled by developers. A designer tool can help, as long as it does not destroy the orginal source files.

Summary so far:
Sparx Enterprise Architect – Comprehensive modeling, good export, no import
Signavio Modeler – Comprehensive modeling, good export, incomplete import
Activiti Designer – Limited modeling, good export, incomplete import

None of the tools has sound reverse engineering or roundtrip capabilities yet.
For descriptive (level 1) and analytical (level 2) modeling Sparx Enterprise Architect and Signavio Modeler are great.
For executable modeling (level 3) a plain XML editor seems to be the only reasonable option for the time being.

Beautiful Transformations with Groovy

Data transformations are the daily business in ETL and ESB scenarios. If you have a service- or business process boundary it is very likely that data has to be transformed between different representation.
Typical integration scenarios have to deal with a huge amount of different formats (flat file, xml, csv, json, even binary). To make things even worse they tend to change from time to time. Therefore it is important to have tools that are flexible and easy to understand. I found Groovy to be very handy for the transformation job. Let’s have a look at a typical example. Assume we have a flat file in the following format:

Id Product     Amount
 1 Cheese      15
 2 Sausage     2
 3 Bread       3
 4 Butter      4

What we want is an xml representation like this:


  
    
    Cheese
    15
  
  
    
    Sausage
    2
  
  
    
    Bread
    3
  
  
    
    Butter
    4

The groovy script below does the job. The great thing is not only that is works but also that its easy to develop and maintain. A developer can see the source and target structure at the same time.

def reader = new StringReader(flatfile);
def writer = new StringWriter();
def builder = new groovy.xml.MarkupBuilder(writer)

builder.order(){
    reader.eachLine() {
        def line = it
        item(){  
            id(id:line.getAt(0..1).trim())
            name(line.getAt(3..14).trim())
            amount(line.substring(15).trim())
        }
    }
}

println writer.toString();

Thanks to closures an the MarkupBuilder class the transformation can be written in a fluent and intuitive way. Typical other transformations are as simple as that, making Groovy a first choice for data transformations.

The Future of ServiceMix

In the ServiceMix forum there is an interesting debate going on about the Future of ServiceMix. I’ve collected some quotes from the thread. I know it is sometimes dangerous to cite people without giving the full context, but I think it helps to get a rough idea about the direction in which ServiceMix is heading. If you are interested in more details please read to original post.

Quotes:

“The idea is really to make ServiceMix the best possible container for deploying Camel based integration.”

“This also means that there’s really not much to reuse from smx4, so smx5 would have its own new and dedicated svn area.”

“Dropping JBI is already a great step in that direction.”

“Additional things that could be pushed inside ServiceMix 5 would be a Tomcat based container deployment option (for those that don’t need OSGi), a new manual similar to what we have in Karaf (maybe reusing parts of it).”

“For advanced use cases, we should definitely guide users towards Karaf, but for a set of basic things, just using Tomcat and simple WARs may be the simplest thing that works for them!”

“To build the added value I’ve been talking about, we don’t really need the NMR imho. I’m convinced we can do it directly on top of camel without the need for another layer, mostly using Camel interceptors. That way, OSGi is not really needed anymore, though it’s still the best way to deploy Camel routes using Karaf and would remain the main and standlone ServiceMix distribution.”

“Providing a way to easily deploy Activiti would definitely be a good thing. I don’t think it should part of the default distribution, as I’d like to keep it as light as possible though.”

Here is my personal summary:

ServiceMix 5 will primarily focus on Camel.
JBI is finally dead.
Karaf and OSGi will represent the primary runtime.
Tomcat will act as an alternative container.
Activiti will not be integrated.

Consequences:

If you develop for ServiceMix 4 today, focus on OSGi and Camel instead of JBI and ServixMix features since those are the stable parts of ServiceMix.

If you have ServiceMix 3 and JBI solutions in place think about migration towards Karaf and Camel. If you plan to integrate Activiti, make sure the required functionality is not already covered by Camel to avoid unnecessary complexity (see blog post Combining Activiti and Camel).

Combining Activiti and Camel?

It seems that that ServiceMix/Camel and Activiti are growing together as you can read in the blog post Supersize Activiti with Mule ESB and Apache Camel and Deploy Activiti as OSGi Bundles

Activiti has its roots in business process management, whereas Camel comes from the area of enterprise integration. Both allow to describe and automate processes. Activiti uses BPMN, a graphical modeling language with an XML representation for persistence and execution. Camel uses a textual DSL which can also be expressed in XML. Grahpical modeling is also possible with the Fuse IDE.

BPMN example:

The BPMN XML representation is not intended to be written directly and can be hard to read.

Camel example:

from("file://start")
.choice()
  .when(header("goldCustomer").isEqualTo("true"))
    .to("bean:processor?method=grantDiscount")
    .to("direct:out")
  .otherwise()
    .to("direct:out");

from("direct:out")
.to("file://end");

Basically both examples are identical. The Camel example is more explicit and shows more technical details which are hidden in the BPMN example.

Especially non technical users tend to think that a graphical process notation is easier to understand because it looks better. It might be true for simple processes like the one above. But for more complex processes textual DSLs are often easier to write and understand.

Activiti and Camel overlap in many areas. Does it make sense to use them side by side anyway? I think yes, because neither Activiti nor Camel is complete in terms of features required by a fully featured SOA/BPM runtime. Actually they complement each other very well in some areas.

Unique features added by Camel:

Connectivity / Adaptors to connect to external systems using a variety of different protocols
Predefined EIP patterns for message routing

Unique features added by Activiti:

Support for long running stateful processes
Human workflow integration

In order to avoid unnecessary complexity it helps to recall the difference between mediation- and business processes which I descibed in my blog post from 2006.

For mediation processes Camel is appropriate in most cases, even long running processes can be executed as long as they are stateless. But if you need long running stateful processes and human interaction Activiti is an interesting option.

Both Camel and Activiti processes can be modeled using prescriptive and analytical BPMN. The software designer has the responsibility to decide which technology is appropriate for implementing the process. If this is not done right you will end up with unnecessary complexity, limited agility and maintainability.

It would be ideal to have an Open Source SOA/BPM Platform that enables end to end process automation without technology boundaries. As long as this is not available, combining Activiti and Camel is an interesting option. Proper solution design and differentiation of process types is key to create sustainable solutions.

Polyglot Workflows with Activiti and Silverlight

My latest article Polyglot Workflows with Activiti and Silverlight has been published in the current issue of JavaSPEKTRUM. After losts of theory in form of articles and conference sessions, I thought it would be a good idea to present a real example. The Activiti engine is an interesting product in the area of BPM. If you go beyond simple demo applications you might want to integrate it with existing UIs, such as corporate portals. Silverlight is an interesting candidate for this task. The article focuses mainly on the interoperability aspects of Activiti and Silverlight and shows how easy it is to bring both together.

Core Values of Great Software

Over the last decade I have seen many software solutions. Some excellent and some really bad ones. But most solutions were somewhere in the middle. They just did what they were supposed to do.

But is this really good enough? I don’t think so. Especially today where time to market is critical and the budgets are limited, functioning software is the bare minimum. On top of that other core values are getting more and more important.

Functionality – This is obvious. The software must do what people expect it to do in a reliable manner.

Evolvability – This is also known as sustainability. A value that is important in many areas of our daily life. And also in software development. It means that we don’t want to reinvent the wheel over and over again. We don’t want to produce waste. Software must be flexible so that it can be reassembled even if technologies change. Investements have to be saved. Evolvability can be primarily achieved through proper software design. Especially service-oriented approaches are promising here.

Production efficiency – This is the most underestimated value. It means that just work is not enough, rather it must be quick and easy to produce. For a long time IT is has a reputation of being complicated and expensive. As a reaction to that companies are outsourcing to cheaper countries or applying more flexibile development processes auch as Scrum to improve efficiency. Especially the latter has proven to be successful. Choosing the right software is also a good way to come closer to that goal. For example frameworks should not only assessed by its functionality (often done by developers) but also on the ease of use and developing performance.

Although those values are important for every software project, they are especially important in todays SOA/BPM undertakings as they promise to increase agility and time to market. This can only be achieved with software solutions that are flexible and easy to use. Ideally even for business people without a developer background.

Speaking at Agility & User Experience Expert Forum

I am glad to be invited to speak at the Agility & User Experience Expert Forum. The conference organized by Accelsis and the Fraunhofer Institute for Experimental Software Engineering is going to take place on 16.June 2011 in Frankfurt/Germany.

The conference is packed with lots of real world examples showing how innovative approaches in the areas of usability and agility can improve corporate software development.

In my presentations I am going to show how agile techniques can be used in fixed price contexts (Yes, that is possible!) and how to achieve real operational agility beyond just Scrum.

I look forward having inspiring discussions and hope to see you there.
You can register here.