hey Paolo,

There could be two things at work here, either a failure to estimate properly, or a failure to do the work in the allotted time, no matter how much time that is.

You should figure out which of the two you're suffering from before you try to remedy the problem.

The first you can combat with keeping detailed logs and adjusting your 'fudge' factor over time, the second is only to be dealt with through discipline.

Don't die from stress please, it's not worth it.

I believe I just suck at estimates, I always find we (I run a 3 people firm) are usually not done within the deadline that seemed really reasonable at the beginning. Also, we're working for multiple clients so it sometimes happen that a project delays 3-4 other projects. We actually get things done, but it seems to me it takes a lot of time.

Ok.

So here is how you could solve that, at least in principle:

Break up your projects in to smaller components and set deadlines for each of those, then track where you go wrong on those much smaller components. Like that you get a feeling for which part of the estimation process fails you.

Some things, such as testing (which you really can't get around) and rework are extremely expensive in terms of time and are easily overlooked. If a customer changes a specification after a contract job has been accepted and the specification isn't sufficiently nailed down then that may be another source of 'creep'.

Good luck with this, it is a difficult thing to tackle. It probably took me a few years before I got good at estimating programming jobs and I still find myself hopelessly wrong from time to time.

The important thing in estimating is to not make the same estimation error twice, eventually you'll get better at it. It really takes practice.

We have bug databases for a reason, we should also have progress databases. Why rely on your gut instinct built from experience, when you can objectively say "it normally takes me one week to implement foo in terms of size and complexity"?

It should be possible to look at your project and see at a glance how many components there are, how long each has taken, and how long it has taken to complete components of similar metrics. I'm surprised that I haven't seen much emphasis on this in Agile, considering the very emphasis on decomposing projects into more-easily estimated components (git plugins to offer nifty visualizations categorized by class, design, cyclical complexity, etc).

There is probably a huge market if you can figure out a reliable, easy way of updating your estimates in a more communicable way than relying on remembered experience

> We have bug databases for a reason, we should also have progress databases.

They're called project management systems, there are plenty of vendors.

The other important piece is milestones so you know as soon as possible when the deadline is at risk. It's one thing to say "Yeah, we have to delay the launch by a month" 6 months out, and another to say "Yeah, we have to delay the launch by a month" 1 week out.

You should work on estimating, or use software to model how much real time it takes for a given amount of estimated time (like fogbugz.)

Yeah! I hate deadlines too !!

Learning to say NO to unrealistic customers is the first step. Took me ages to perfect, but now I'm so much less stressed. For me an internal project has no deadline. And for a one week (estimated) project I quote the client a month.