Not true. You need a deep understanding of the workings of the system and a logical mind to be able to work backward from the symptom to the cause. (Patience helps too.) Lots of programmers who don't have these traits just try fixes at random. Either they find the solution (after a long time), or they give up and ask someone who is better at debugging to help them. I frequently get asked to debug problems that other people have failed to debug.
I'm not sure it's that simple. For example, you push code that breaks something. You have two options (after rolling back the code of course,) you either try to accomplish what you were doing in a sufficiently different way that you think it will avoid the problem (essentially the random guessing strategy you mention, but with purpose), or you try to dig into the problem to understand it enough to fix it at the lowest level of abstraction where the issue is introduced. Both strategies can work. In certain scenarios however, the former can be much, much quicker to enact, but you won't walk away with much reusable knowledge beyond the "the doctor says don't do that" kind.
I tend to attempt to just "go around" problems in the quick manner if I have the intuition that there is some top level fork in the road I can take, and that anything I am to learn in the process of performing proper diagnostics has minimal liklihood of being useful or reusable, and is not just some mindless incidental complexity of the task at hand. It's hard to be 100% correct in this prediction, often things like that in the OP can result in unexpected useful knowledge, but I think you can do better than random at guessing if it will be the case. I think part of the skill of being an engineer is learning what issues are worth tackling head-on and need to be understood deeply to insure against future peril, and what issues should be simply maneuvered around without much understanding.
The risk of not finding the root cause of the bug is that you don't really know for sure whether your fix is good - it may have just changed the state of the system in a way that masks the underlying problem, perhaps by causing it to appear under different conditions that you haven't encountered yet (but which your users may find very quickly).
Also, the code you just pushed may not have been the cause of the bug - it may have just exposed a pre-existing bug. If you just back out your change and try something else, you'd just be sweeping the bug under the rug.
Right, it may have just changed the state of the system to mask the problem, or it may have just swept it under the rug. But you also may have just avoided a complex, long debugging session which would not result in reusable knowledge.
Debugging something that ultimately ends up being a 24-hour one-time glitch in AWS is a waste of time unless you learn something useful in the process or have sufficient reason to believe you altered the speed with which it was fixed, no matter how fun the journey. Sure, if it results in you deciding you need to be resilient under AWS failures more, that's great. But often times you just burned man hours and lost the opportunity to work on something else. I don't think it's reasonable to say that all debugging results in useful learning. (I'd probably argue the contrary, given the number of times I end up grokking incidental implementation details of some third party software or system during debugging that in just a few short months later becomes useless as that code or service is paved over.)
Trying to tilt the odds in your favor so you make educated guesses around when this will be the case is part of the skill. Part of this skill is also being able to recognize when a bug you thought had been routed around was simply swept under the rug, and changing course to understand it properly instead of sweeping it under the rug again. ("Fool me once, shame on you, fool me twice... you can't get fooled again.") I find these are the exception not the rule, and are probably worth the cost. The joys of shipping outweigh the joys of understanding every facet of a misbehaving system, at least for me.
> But you also may have just avoided a complex, long debugging session which would not result in reusable knowledge.
I find this is becoming increasingly rare for me, to the point where I'm having difficulty remembering the last time it occurred. I run into the same "once off" problems more frequently, and getting more value out of debugging them. I'm getting better at using my tools, better at tying symptoms to causes, better at coming up with appropriate defensive coding techniques, and writing more (reusable) code that makes them easier to debug when they (and entire categories of similar bugs) reoccur.
> Part of this skill is also being able to recognize when a bug you thought had been routed around was simply swept under the rug, and changing course to understand it properly instead of sweeping it under the rug again.
Frequently not possible on two fronts:
On recognizing it: I've seen a distressing number of bugs which are hideously context sensitive (meaning "works fine on my machine" is the punchline because it's 100% broken on another, only now you've got a terrible bug report and no good repro case)
On changing course: Any offline distribution channel. Any embedded system. Any onerously regulated environment. Once it's in the wild, it can become very expensive to fix.
> The joys of shipping outweigh the joys of understanding every facet of a misbehaving system, at least for me.
That's fair. I simply hope you avoid the sorrows of burned customers and data loss.
Not true. You need a deep understanding of the workings of the system and a logical mind to be able to work backward from the symptom to the cause. (Patience helps too.) Lots of programmers who don't have these traits just try fixes at random. Either they find the solution (after a long time), or they give up and ask someone who is better at debugging to help them. I frequently get asked to debug problems that other people have failed to debug.