Delivering projects on a predicable schedule is EASY. It just requires a few things:
* Know what you need to deliver
* Know how to build the things you need to deliver
* Know how long it takes to make, integrate, and test the components you need to deliver.
* Know what resources you have available and how you can distribute the components you need to deliver.
* Work for people who aren't going to panic when you tell them up front how long a project will take and why.
Surprisingly, the last item can be the hardest one...
The most important thing: having done hundreds of very similar things before. Any developer can tell you how long a house will take to build from the prints. Same for normal roads and so on. However if you have a custom bridge that has never been done before nobody can really tell you how long it will take or how much it will cost.
This is the crux of all(?) of it. If the software/system wasn't new/unique, it would've been copied from something that already existed. This doesn't apply in the real world where making the second copy of a bridge is almost as expensive as the first one.
Software is a mechanism that can copy its self infinitely and in multiple forms with limitless layers of abstraction [1]. If you need to make new software, by default you are making something new/unique, otherwise you're reinventing the wheel.
I have been involved in several rewrite from scratch projects. The following projects are always more expensive than the original (probably less than the original counting all the maintenance - but a lot of weird bugs will take years of maintenance to clean up and that was done in the original so it is hard to count that). However once they are done the result is a lot more maintainable, and has a much better UI of a form that would be hard to retrofit to the old. Normally they are written in a better language an the original. (C++ with const correctness is better than C++ without!)
One key takeaway is we as an industry do not enough rewrites to get good at doing them. (I'm not sure that is a bad thing)
Figuring out what you need to deliver is often a subproject all it's own. I used to work at a place that did a lot of fixed-price consulting work and we were small enough that if we blew a bid badly, we were basically dead. We'd often look at the big picture of what the client wanted and offer to break that down into a detailed design and project plan which they could then pay us to implement or hire someone else to do. We used that as an opportunity to figure out what they really wanted/needed.
Sometimes it's enough to "know what you need to deliver in order to make more decisions/do more planning" but your scheduling and planning horizon really doesn't extend beyond that checkpoint. If you're planning/scheduling that way, you're basically 'doing agile'.
* Know what you need to deliver * Know how to build the things you need to deliver * Know how long it takes to make, integrate, and test the components you need to deliver. * Know what resources you have available and how you can distribute the components you need to deliver. * Work for people who aren't going to panic when you tell them up front how long a project will take and why.
Surprisingly, the last item can be the hardest one...