But you’re right, it can be an effective technique.

You know, right now I’m thinking of a rhetorical possibility. What if you (or I) said to the project owner something like this:

I’m telling you it takes three months because it takes three months. If I back down now and say it’s six weeks, what will you do when we’re eight weeks in, it’s not done, and the stakeholders want an explanation?

Basically, I’m making the transparency argument. If you lead with balderdash, how can you effectively explain the (inevitable) problems later?

One of my tricks is to define my little part of the world in terms of components, rather than one big rock. On the software side, you might think of this as features or compatibility or speed.

Then, when a pointy-haired boss tells me he wants it done in half the time, I just draw a big red “X” through half of the components and ship it back to him for a confirming signature. (No, I’m not seriously saying I want you to build software that’s only compatible with half the hardware — it’s a negotiating tool.)

This often opens a meaningful dialogue about how many hamsters we have, how fast their little treadmills go, and the fact that decisions must be made. Or, at the worst, I’m able to limit the scope of work and be seen as having met my commitments — even if the product won’t run on the C-64 as they originally wanted.

Not a perfect plan, but it kept my reviews clean and (now that I’m a contractor) it really keeps my client relationships MUCH more professional.

People keep forgetting that software development is always uncertain, because the minute you have certainty you’re no longer doing software development!

I like your idea of getting better estimates after some iterations. That must help a lot.

John (jfbauer), I love the slightly Machiavellian way you calculate your estimates. An incredibly non-cynical interpretation, and a useful one, is that you really do need to plan on those hard dependencies, and there’s nothing wrong with that at all. For example, right at this moment I’m held up a little bit on a software project because there are some issues I can only resolve by testing on certain hardware… that does not exist yet. I didn’t plan on that! And now I’m technically behind, although the client is being cool about it.

But the tactic you describe kind of reminds me of the way escape artists [are said to] flex their muscles and wrists to create a lot of wiggle room after they’re apparently tied up as tightly as possible. Sure, it looks like you’re on the most aggressive schedule that can be imagined, but you’ve bought yourself that wiggle room. Good thinking! Slightly deceptive, but good.

Bull, yes, you’re exactly right. One of the things I like about Scrum is that it’s really explicit about that: You get the best possible software, in a usable state, under the budget they give you. If the client thinks further development is worth the cost, you get more money or other resources. If not, hey, it’s all good.

That avoids the problem that occurs when software is ostensibly 90% done, but useless, and there is no money to finish it. I just realized this is a case of software’s infinite malleability being an advantage in estimation, which it usually isn’t.

Thank you all for your comments. I’d love to discuss this further.

This isn’t a new problem and it will never go away either. I think it is the mettle of the mature and successful CIO/CTO that they constantly “upward and sideways” manage their peers and management in general to get them to understand realities here. I’ve spent a good portion of my time on the job in these roles doing this kind of education and ongoing negotiation on “rules of engagement” etc.

I’ve written about how estimating is, however, a necessary evil. (In other words, the approach that WON’T work is to avoid giving estimates altogether). You can find my post, “Rock and a hard place: Why estimating turns into a squeeze play”, at http://www.peterkretzman.com/2007/11/15/rock-and-a-hard-place-why-estimating-turns-into-a-squeeze-play/

A reference in that post is to Construx’ software’s “Cone of Uncertainty”, which I literally keep framed on my wall, and point to it during meetings with key stakeholders as I evangelize these concepts. You can find that at http://www.construx.com/Page.aspx?hid=1648

Stakeholders are required to take on some of the risk in complex projects. To deliver quality software, the estimate may be exceeded and that is the risk the stakeholder assumes and it should be outlined in the estimate along with many assumptions about the project. The better your estimate the more confidence there will be in your ability to estimate. The better you deliver quality software on time, the less your estimate will matter.

The sales staff must work with the development staff to determine if we must meet the budget constraints or if it is more important to have quality software. Some stakeholders want the Cadillac, and some want the Pinto. Many times this can be determined by the nature of the project. There is no perfect solution and that’s what makes us valuable.

Assumptions and risks to go along with your estimate are key to controlling cost. Along with a sane stakeholder.

This post was mentioned on Twitter by MarkWSchumann: “Programmers are optimists” is a cheap shot, and here’s why. New blog post: http://bit.ly/A7eaa #agile…