Collocation Is Vital!

What does collocation mean? The concept is very simple. It means bringing together the people who work on a software product in a physical environment. This seems to be natural. But in highly distributed work environments that we have today it is not anymore.

I’ve been working on agile projects for many years and I always hear that collocation is difficult. Especially in larger enterprises it seems that the hurdles are high. There are many reasons stated by different people.

Here are some examples:

  • There are no adequate rooms available (managers).
  • The buisiness guys don’t understand IT. We can’t work in the same office (developers).
  • The IT guys don’t understand our business. We can’t work close to them (business).
  • Our outsourcing partners are from country xyz. It is impossible to work with them in one office (managers).
  • The business people have their work to do. They simply don’t have the time to work closely with IT (business).
  • I have to concentrate. Working close to others would always disrupt my work (developers).
  • We don’t like travelling. (all of them)

All of those are valid arguments, but to me it seems that they are sometimes artificial. In this case reluctancy to change is stronger than the will to increase efficiency.

From my experience collocation is an underestimated factor that can greatly improve effectiveness of teams. The study Collocation Impact on Team Effectiveness goes in the same direction.

Another interesting read is Collocation, Collocation, Collocation. Quote from the blog:
“A big part of the problem is socio-political in nature and would appear to be deeply embedded in the human DNA. We naturally tend to come to trust those that we have regular contact with (our family / village / tribe / region / country / team) and have an equal and opposite tendency to distrust those on the outside of these habitual daily contact networks (all those that the lock on our front door or the defences around the tribal village are designed to keep out).”

If a company is interested in real agility and effectiveness, I think it should try really hard to create collocated work environments and not give up early. It might somtimes be difficult, but the reward will be better results and most likely even better motivated staff.

Scrum in Reinsurance

On 5. July 2012 I am going to give a presentation introducing an agile Project in the area of reinsurance which I accompanied as Scrum master and solution architect. The conference is going to take place in the SAP Service Center in St. Leon-Rot near Walldorf.
The presentation is called Scrum in reinsurance – a success story. It covers factors that made the project a success as well as impediments. The aim is to share our lessons learned on the way to successful agile project management. I hope to see you there.

Scrum and Silverlight in Reinsurance

Today I would like to share a success story of a project I accompanied as Scrum Coach and Solution Architect from analysis to production.

Main success factors were:

  • Scrum (Agile Development)
  • Cross functional Team
  • Service-oriented Design (SOA)
  • Silverlight RIA

You can read more in the case study Modernization of business partner management.

Lightweight Collaboration Tools

When you work with distributed teams it is important to have lightweight tools for efficient communication. Here are some free tools that I would recommend:

Surveymonkey – Online polls
DFN Scheduler – Scheduling
Skype – Video and voice conferencing
Chatzy – Private web based chatrooms (port 80 only, mobile access)
Trello – Scrum and Kanban boards (highly interactive)
Kunagi – Scrum boards (includes tools like planning poker)
Teamviewer – Online meetings and remote control (free for non commercial use only)

Impediment Number One to Agile Adoption

Scrum is in many respects very different from traditional project management approaches, especially waterfall models. It requires a different mindset in which it is ok to say:

“I don’t know yet exactly how long the project is going take, but give me some time to get to know the requirements and the team. After some Sprints we will give you a solid estimation based on empirical knowledge. Trust us, we do the best we can to deliver quality on time”.

For many especially “classic-minded” project managers such a statement is unimaginable. They simply can’t understand the culture of Scrum as it is very different from what they are used to.
After applying Scrum in several projects over the last years and after giving many Scrum workshops I think that the only way of learning Scrum is by doing it, ideally accompanied by a skilled coach. Books and certifications help, but they do not create deep understanding.
And here begins the dilemma. Especially managers in larger organisations never work on projects, they manage. Therefore it is hard for them to leave their classic mindset. This leads to non-supportive behavior which often blocks the way to agile adoption in large enterprises. In a recent interview Scrum in larger organisations Jeff Sutherland describes the challenges to change the management mindset.

He says:
“… major challenges you will have to deal with when you implement Scrum in a large organization is to change the mindset in the organization in general and on management-level in particular …”

For the reasons given above this nut is hard to crack. To me it seems that this is impediment number one on the way to agile adoption in larger enterprises.

RIA goes to Hollywood

Most RIA technologies today use an asynchronous model to communicate with the server, primarily to keep the UI responsive no matter what the server is doing. This principle is also known as the Hollywood principle. Don’t call us, we call you means that instead of continuously polling, the server calls back when the operation finished. In order to be notified when the server finished its job, callbacks are used.

For instance in Silverlight this leads to code like this

void ClickLoad()
{
    LoadCustomer(1,
    (result) =>
    {
        // Process result
    },
    (error) =>
    {
        // Process error
    });
}

void LoadCustomer(int id, Action<Customer> success, Action<Exception> error)
{
    LoadFromServer("select * from customers where id = " + id,
    (result) =>
    {
        success(result[0] as Customer);
    },
    (exp) =>
    {
         error(exp);
    });
}

void LoadFromServer(string query, Action<List<object>> success, Action<Exception> error)
{
    server.LoadCompleted += (s, e) =>
    {
        if (e.Error != null)
            error(e.Error);
        else
            success(e.Data);
    };

    server.LoadAsync(query);
}

Not very nice, is it? In jquery we find a similar pattern.

$.ajax({
  url: 'ajax/load.html',
  data: "query=1"
  success: function(result) {
    // Process result
  }
  error: function(XMLHttpRequest, textStatus, errorThrown) {
    // Process error
  }
});

Usually an application consists of several layers. In that case the callbacks have to be routed back to the original caller, leading to code that is neither easy to read nor easy to maintain.

From what is already visible, it is very likely that Silverlight and WinRT are going to have great overlap. The combination of XAML, C# and WinRT plus the tooling will be very familiar to all Silverlight users.
That means, it is likely that the asynchronous programming model will be the predominant model for client/server communication for Windows Metro style apps. In order to simplify this, C#5.0 is going to include the await/async keywords, that will make callback chaining obsolete.

Example:


void ClickLoad()
{
    try
    {
        Customer result = await LoadCustomer(1);
        // Process result
    }
    catch(Exception e)
    {
        // Process error
    }
}

async Task<Customer> LoadCustomer(int id)
{
    List<object> result = await LoadFromServer("select * from customers where id = " + id);
    return result[0] as Customer;
}

async Task<List<object>> LoadFromServer(string query)
{
    return await server.LoadAsync(query);
}

This is much cleaner than the examples above. Actually it is a synchronous programming model supporting an asynchronous runtime model. As asynchronous calls are very common in RIA applications the async/await keywords are going to change the way we program async calls in the near future, leading to code that is easier to develop and maintain.

I can’t await async in C# 5.0 😉

Combining Groovy and XSLT for Data Transformation

In the blog post Beautiful Transformations with Groovy I described how easy it is to create data transformations with Groovy. But sometimes organisations invested massively in XSLT transformation and want to reuse their existing XSLT templates. Read on for an an example that shows how to do that.

Assume we want to transform the following XML file to HTML:


  
    Germany
    Fast and nice
  
  
    Spain
    Large and handy
  
  
    Italy
    Small and cheap

Lets further assume the result should look like this:

Does it make sense? I don’t know, but that’s not really important. 😉

We have the following XSLT template to perform the transformation:

All you need is a Groovy script like the one below to transform the xml file to html using the given xslt.

// Load xslt
def xslt= new File("template.xsl").getText()

// Create transformer
def transformer = TransformerFactory.newInstance().newTransformer(new StreamSource(new StringReader(xslt)))

// Load xml
def xml= new File("cars.xml").getText()

// Set output file
def html = new FileOutputStream("output.html")

// Perform transformation
transformer.transform(new StreamSource(new StringReader(xml)), new StreamResult(html))

This is self-explanatory, isn’t it?
As XSLT is somewhat limited when it comes to more complex transformations, it can be extended by custom processors which can we implemented in Java or Groovy. A custom processor in Groovy can be implemented like this:

public class AgeProcessor{
    public def process(ExpressionContext context,int n){
        return "Age: " + (2012 - n) + " years";
    }
}

The processor is hooked up to the XSLT using the expressions in line 3 and 28 of the above XSLT file.
The examples above show how to reuse existing XSLT in Groovy. Are you interested to see the same same transformation in pure Groovy? (sorry, I could not resist ;-))
Here is the code:

// Load xml
def cars = new XmlSlurper().parse(new File("cars.xml"))

// Set output file
def writer = new FileWriter("output.html")

// Perform transformation
def builder = new MarkupBuilder (writer);
builder.html(xmlns:"http://www.w3.org/1999/xhtml") {
    head {
        title "Cars collection"
    }
    body {
        h1("Cars")
        ul(){
            cars.car.each{car ->
                li(car.@name.toString() + "," + car.country + "," + car.description + ", Age: " + (2012 - car.@year.toInteger()) + " years")
            }
        }
    }
 }

It is shorter and self-contained. It is also more intuitive and therefore easier to maintain. But if you have the requirement to support XSLT in Groovy you now know how to do that.

Trapped in the Comfort Zone

Many agile techniques such as Kaizen, Sashimi or Kanban correspond to terms and principles found in asian culture. A less known principle is:

“Do not develop an attachment to any one weapon or any one school of fighting”
– Miyamoto Musashi

In the context of agile it means that one should change the process if it helps to achieve the goals. This is something most developers would agree to as processes are often seen as impediments.
The same applies to technology. Translated to the technical world it reads: Do not stick to your favourite technology if there is something better suited to meet the project or customer needs. This is something many developers would not immediately agree to. Developers usually love sticking to their JEE, Spring, .NET, SOAP, REST, [any other technology] with which they grew up. They often argue that learning a new technology is time consuming and therefore hardly possible to change.
I think that is wrong. Provided a developer has a sound background, he or she can become productive in a new technology within a short time. I’ve seen developers switching from JEE to .NET and vice versa without problems. This is possible because technology always evolves. Most new frameworks and programming languages do not reinvent the wheel. The are always based on similar common principles which remain valid and stable over time. It is more a matter of mindset that keeps people trapped in their technology comfort zone.

Is that a problem?

Sometimes yes, especially when paired with Groupthink, it hinders innovation and production efficiency.

How can this prevented?

1. Make sure you have people with long standing experience in different technology domains in your team. People who worked with multiple technologies are usually more willing to reflect technology decisions and align them to the requirements of the business.

2. Don’t start a project with a strong technology committment. Let the team decide which technology is best suited to solve the business problem. Of course in conformance with the corporate standards.

3. Ensure that the team has the freedom to decide which tools they want to use.

Having the option to change weapons (processes, tools, frameworks, etc.) if needed, improves the likeliness of successful project delivery.

Bug or Change – Cause of Conflict in Agile Projects

According to the second rule of the agile manifesto working software is more important than comprehensive documentation. This is definitely true!
To be clear, this does not mean that software developed by agile teams is not documented. If comprehensive documentation brings value to the organisation, agile teams produce this as well. Specifications are written as well in agile projects. Why? Because it is not (and never was) a good idea to start development without knowing what to implement.
But contrary to waterfall projects in which much of the specification is written upfront, in Scrum the specification is written as part of the sprints. And due to the close collaboration in cross functional teams, the specification can be much more lightweight without loss of quality for the final product. This is all great. But there is a challenge to keep in mind.

When it comes to testing (acceptance, performance, etc.), either as part of the sprint (which is definitely the preferred way), or later when the product moves towards production, the testers have to find and classify bugs. Usually they do this based on the specification. In case a functionality is missing or not working as described in the specification it is a bug.
With a lightweight specification that don’t decribe every little detail it is sometimes hard to tell whether someting is a missing feature or rather a change for a later version of the product. This situation can cause conflicts.

But not necessarily. The important factor is that the testers are part of the agile team context from the beginning, so that they share the knowledge and experience with the rest of the team. In a culture of trust, the team can easily negotiate whether a finding is a bug or a change. If the team is commited to deliver quality (the Scrum Master has the responsibility to educate the team to do so), this model works properly.

This strategy correlates with the conflict resolution scenario Use collaboration to resolve the conflict described in the interesting blog post Know These Five Causes of Conflict written by Karen Ruby.

Quote:
“However, if trust is there, this conflict resolution scenario can be the best way to resolve conflicts once and for all. When both parties come together, communicate, and trust each other a definitive resolution to their conflict can occur.”