How did we get into this mess?

I’m in the midst of an email conversation with a pretty well-known turnaround CEO adviser, the kind of person who can Make Your Company Not Suck. He’s way beyond me in terms of influence and reach, and he consults enterprise-wide for way bigger outfits, but for some reason he ended up asking me why so many projects still get in trouble even though all kinds of training and certification are available these days.

This is about what I wrote back.

From my observation working with smaller teams, the eight main reasons why software projects get derailed are:

  1. Persisting with a technical approach that doesn’t work.
  2. Wrong developer(s) on the task.
  3. Technical Bankruptcy.
  4. A single technical obstacle.
  5. Code hoarding.
  6. Redoing the same work.
  7. Insufficient time.
  8. Vagueness.

Let’s talk about redoing the same work.

#6 in particular is something I feel really strongly about. Agile has it right with the “DRY” principle: Don’t Repeat Yourself. Don’t have multiple blocks of code doing the same thing in slightly different contexts. Do merge duplicate code while refactoring, even if it already works and appears fragile.

But also… build quality into the entire process, don’t just save it for testing at the end!

I see that mistake made over and over again. It always makes so much more work than techniques that use continuous feedback.

First things first. Last things last.

You will never know less about the implementation than at the beginning of the endeavor, and your knowledge will increase throughout the project, so it makes sense to refrain from overspecifying design and implementation details early on. The shops that insist on locking down all their data structures before any code is written? Those are the ones that end up having to unlock them anyway because their assumptions turn out to be a) directly wrong, or b) overtaken by new and refined customer requirements.

Answering the actual question.

That’s just one of the eight major reasons though. From a higher-level perspective, I’d say software projects go badly when (and because) everyone (everyone!) is unrealistic about how real humans do creative work. And in answer to my correspondent’s main question: Most training and certification programs constitute misguided attempts to mechanize something that’s inherently a little unpredictable, and they’re focused on compliance rather than accomplishment. Thus you get lots of compliance with the process, but not as much accomplishment as you would have liked.

Bottom line, the classical system of training and rewarding developers cultivates the wrong behaviors. We don’t really count lines of code anymore, but we do count change tickets closed. You don’t get credit for outright pairing, nor for taking half an hour to explain the finer points of ADO.NET to your cube neighbor. Some shops pay attention to closed work items but don’t bother tracking if they stay closed through QA and delivery. A really well-placed, intelligent bout of refactoring that erases technical debt and makes all future development less error-prone? That’s not a deliverable, and it’s poor compliance, so it doesn’t usually get done and the whole team’s effectiveness suffers.