> It strikes me as somehow the compiler is making assumptions that aren't being enforced by the ... OS? Language?
One case of this problem was in a handwritten assembly file. The other was a compiler bug.
This is a case where the ABI requires that if you use a certain register you must save its previous value and restore it afterwords; the two independent bugs were cases of forgetting to look after a certain register.
An ABI is simply an agreement as to how things should work: what registers you are free to clobber, which you must look after when you use, how certain data must be laid out in memory, etc. ABIs are typically language specific, though there may be a lot of commonality at the very high level (i.e. how you use sections in an ELF file) and low (anybody using unboxed integers probably will do the same thing).
You are welcome to violate the ABI as you see fit in your own code. The OS doesn't care; it has its own constraints (how to make a system call, how to pass arguments to each -- though cf above when I talked about ints). So, say, a Lisp compiler can lay out stack frames differently from a C++ compiler because of the languages' different semantics) but if your Lisp program wants to call a library written in C++ it must make sure memory at the call site follows the C++ ABI because that's what the C++ compiler will have assumed.
Both bugs were programming errors in assembly language files. One was inline assembly that was missing entries from a clobber list, the other was an assembly function that lacked invocations of the macros that were supposed to be used to preserve/restore the registers. There was no compiler bug.
It seems to me something that could be found by some kind of valgrind-like tool - it'd be much slower than normal code but "ABI exception detected" or something.
Interesting conclusion given that I found two functions and a (presumed) third-party driver/what-not that were violating the ABI. One of these was causing crashes, and the other one was going to. The crashes went on for over a year and a half, so, ...
One case of this problem was in a handwritten assembly file. The other was a compiler bug.
This is a case where the ABI requires that if you use a certain register you must save its previous value and restore it afterwords; the two independent bugs were cases of forgetting to look after a certain register.
An ABI is simply an agreement as to how things should work: what registers you are free to clobber, which you must look after when you use, how certain data must be laid out in memory, etc. ABIs are typically language specific, though there may be a lot of commonality at the very high level (i.e. how you use sections in an ELF file) and low (anybody using unboxed integers probably will do the same thing).
You are welcome to violate the ABI as you see fit in your own code. The OS doesn't care; it has its own constraints (how to make a system call, how to pass arguments to each -- though cf above when I talked about ints). So, say, a Lisp compiler can lay out stack frames differently from a C++ compiler because of the languages' different semantics) but if your Lisp program wants to call a library written in C++ it must make sure memory at the call site follows the C++ ABI because that's what the C++ compiler will have assumed.