In terms of engineering problems, make sure you aren't making things more complicated than strictly necessary. Most software gets rewritten because constraints/requirements/teams change over time, in unexpected ways. YAGNI.
Also, usually your job isn't to solve an engineering problem, it's to provide business value. It's easier to claim great engineering value (because all the requirements/constraints are in your head), rather than actually solving customer need where someone else gets to decide whether you are providing value.
Oh and of course people who are more visible to higher-ups get promoted more!
I think a huge problem with YAGNI as a principle is that it is - in itself - as subjective as most of the points in this whole discussion.
I've read my fair share of code authored by people who's definition of "done" seems to be "it compiles and doesn't fall over the first time I look at it", ignoring various obvious edge cases that could be triggered randomly. Cleaning up messes like this always costs more than the shoddy implementation saved in the first place. Usually those clean-up session start when some strange edge case occurs on a critical production system.
On the other side, the approach is fine, if the code only runs once or twice an is then discarded. In that case ironing out all possible bugs is clearly a waste of time.
So, YAGNI means different things in different contexts. So, I'd say that being aware of said context is a very crucial skill to have as a developer.
---
When it comes to business problem solving, I couldn't agree more, though. There are a lot of "solutions" that could have gotten so much better if someone tried to figure out the problem first.
People spend a lot of time demonstrating their sexual fitness for reproduction by displays of prowess. It's just what we do. Intellectuals and engineers aren't exceptions, they're often very naive about this tendency and the biological roots of it. I agree there. I've sometimes said that universities are very important because they teach people to meet the irrational demands of cloistered professors in order to succeed; thus preparing them for business. But there are times, you would probably agree, when you have to argue strenuously for engineering problems to be funded for engineering reasons, before they crash the business. Building a bomb because that's what management said they wanted isn't what you're paid for either.
Also, usually your job isn't to solve an engineering problem, it's to provide business value. It's easier to claim great engineering value (because all the requirements/constraints are in your head), rather than actually solving customer need where someone else gets to decide whether you are providing value.
Oh and of course people who are more visible to higher-ups get promoted more!