Moving to an Agile process

published: Thu, 28-Apr-2005   |   updated: Mon, 16-May-2005

Recently at work we've been in discussions about introducing a new development process. We're growing fast enough that it's now time to rethink how we develop it. The product's options and configuration has increased in complexity so much that we're relying more and more on automated testing to increase quality. Although the newer code has unit tests, unfortunately a lot of the more mature (which actually implies more thoroughly tested) code is still unit-test-free and is only shown to work through rigorous acceptance and system tests by the QA team.

So we're at a cross-roads as we prepare to release the new spiffy version. We have some code clean-up to do in our legacy code (if you can call writing unit tests clean-up work, but note that it will probably bring out refactoring opportunities), and a whole bunch of new code to write for the next, even spiffier version. Hence the need for a new process: we want to ensure that the new code gets written properly (as in, a test-driven manner) so that two years down the road we're not having the same thoughts and issues. All in all, we're talking some Agile methodology or other.

There are two paths to take.

The first path is to release the current version then spend N months on clean-up work: writing unit tests for the codebase and maybe refactoring the bits that have the worst code smells. After that, we can start on the next version, secure in the knowledge that the current version is test-armored and hence safely changeable as we implement the new features. In doing this, we won't inadvertently lose current functionality.

The second path is to start on the next version immediately and to add unit tests to the legacy stuff as we implement the new features (the new features are bound to require changes to the existing codebase, after all). So the development reduces to a two-step process (or should that be "two-leap process"?): write tests for the legacy code we're about to change for the new feature and then write the feature (and its unit tests), secure in the knowledge that the new unit tests will cover us and prevent us from losing functionality.

It's a difficult choice to make, I'll give you that. Let's make it simpler by choosing some terminology. The first path could be called the Waterfall method, and the second one the Agile method. Is the choice easier now?

Ron Jeffries, of XP fame, uses the following analogy for this dilemma. Cleaning up legacy code is like cleaning up a very messy kitchen. Certainly you can wake up on Saturday morning, work like stink all day spring-cleaning it (and sending your loved one out for some take-out for lunch), and get it pristine for the evening. And then? Well, it'll just get dirty again, crumb by crumb, dirty plate by dirty plate.

The alternative is to clean the kitchen gradually. Always clean the plates you use straightaway, plus a couple more that have been hanging around. Clean the oven immediately after using it. Clean the floor regularly. As time passes, you'll find that the kitchen becomes clean and then stays clean, all without any heroic ruin-your-Saturday effort. Furthermore, you'll have got into a good habit: you'll always be cleaning up your mess immediately without letting it accumulate. In the heroic blow-your-Saturday-away version, you won't have learnt any good clean-up-as-you-go-along habits at all.

In other words, say you have to fix a bug. As you work through the repro case, you'll be looking at the classes and methods that contribute to the bug. Instead of baldly fixing the immediate bug, fix it and leave those classes and methods slightly "cleaner" than you found them. Or, say you have to implement a priority queuing algorithm for an in-house thread pool (it currently uses normal queues). Clean up the thread pool by writing unit tests for it as you change its queuing algorithm.

Another big problem with the Waterfall method of cleaning up legacy code is a business one. To the outside world, the development group doesn't produce any new functionality for N months. It's as if they've all gone home and are swanning around on loungers by the swimming pool. You can't sell your customers "cleaner code"; they want features. Certainly quality is a selling point, but not in the sense of "Now with better tested code!" Quality has to be a part of the selling proposition, not all of it.

In essence, the development group have to polish the code while they develop a set of improvements: the customer gets better quality as a side-benefit while he/she gets the extra/better/whiter features.

By the way, if you are in the same situation: run, don't walk, to your local bookstore and get Working Effectively with Legacy Code by Michael Feathers. In this book he discusses a whole raft of techniques to help you wrap legacy code with tests as the first step to refactoring and fixing it. His definition of legacy code? No, nothing to do with being written in COBOL. Legacy code is any code that doesn't have unit tests.