In the fall of 1999 I got the biggest productivity boost of my entire career as a software developer. In the October issue of IEEE Computer magazine, there was an article by Kent Beck called “Embracing change with extreme programming”. In it, he outlined Extreme Programming (XP), which includes much of what we now refer to as agile development.

By then, I had been working as a software developer for seven years. The standard development methodology at that time was waterfall: document-heavy year-long projects, frozen requirements, change control boards, manual testing at the end of the project, and so on. Software development succeeded despite the methodology, not because of it. Reading the article was a real eye-opener. I felt it was describing how I naturally worked – in short cycles, with fast feedback, and frequent redesigns.

(And yes, I meant to write this last fall for the 20th anniversary, but I didn’t get around to it then. But hey, better late than never, even if the title has 20.5 in it now)

In the 1990s I worked as a software developer for Ericsson, developing software for mobile phone exchanges. The projects for new releases of the software had names like 91B (for Q2 1991) and 92C (for Q3 1992). A typical project at that time lasted for a year or more. The methodology was typical waterfall: requirement documents were written and handed over to developers. We developers implemented the features, and documented them with elaborate flow charts that nobody ever looked at. Testing was mostly manual, and done at the end of the project. If there were changes to the requirements, they had to be approved by a change control board before they were implemented. By the turn of the millennium at Ericsson, we also had Rational Unified Process (RUP) and an awful tool for generating code from UML diagrams called ROSE.

Even though I only worked at Ericsson (albeit at several different locations), my feeling is that this type of waterfall methodology was the dominant methodology for most software development at the time. It was certainly how I was taught software development at university.

And it seems so reasonable! If you can’t get the complete requirements for what to develop, how can you do a good job developing it? And if they requirements are allowed to change, you jeopardize what you have already done. But I struggled getting the waterfall methodology to work for me. It was always too hard for me to come up with a complete design, and then implementing it. Instead I would make a really tiny version of what I needed, and test that. Then I would add the needed functionality little by little, always testing and running the code along the way. As I developed, I would have new insights into the problem I was trying to solve, and about how the solution should be structured.

When I read the article about XP, I immediately felt that it aligned really well with what I had discovered worked for me. Instead of following a rigid plan and fighting against changes, the article outlined a whole development methodology based on allowing and encouraging changes.

There were of course many other things in the article that I had not discovered for myself, but that sounded like such good ideas when I read about them. For example, I really liked the concept of unit tests. When I wrote my code in tiny increments, I would often test the new code I had written, to gain confidence that it worked. However, these were always one-off tests that I didn’t save. The idea of keeping all those little tests in a test suite that could be run again and again was great.

Another great idea was to design with testing in mind. I still remember the aha-moment when I changed the code to make it easier to unit-test, and saw how much cleaner the design of the program became. Another great idea was refactoring. This is something I did from time to time, but having a name for it, and arguments for why it is needed, was really good.

However, the most important changes XP (and what later became agile) brought were not specific practices like unit testing and refactoring. The most important changes were the themes of iterative development in small steps, with fast feedback and frequent adjustments to what we are developing. These core ideas are almost the opposite of the ideas of waterfall. Today, it seems to me that agile won. Almost all companies try to use agile ways of developing software. Not all succeed perhaps, but the aspiration is there.

Waterfall is good in theory, but it doesn’t work well in practice. 20 years ago I discovered and switched to using agile methodology instead. There the emphasis is on iterative development, frequently delivering small slices of functionality, with fast feedback and many adjustments. Changing to this way of working was the biggest productivity boost of my career, and it is working as well today as it did when I switched. Thank you Kent Beck for your great ideas!