This is embarrassing. We now know that the line "with many eyes, all bugs are shallow" is just wrong. What we do know now is that the open source process does not converge to a no-bugs state.
It's time to start phasing out C/C++. Languages which don't know how big their arrays are have to go. If it can run efficiently in a garbage-collected environment, it should be in Go or some scripting language. If it can't use GC, Rust is almost there. (As I say occasionally, I really hope the Rust guys don't screw up.) C and C++ should not be used for new work.
It has been 1 days since the last buffer overflow vulnerability report.
I wish people would stop tarring C++ with the same brush - this vulnerability is caused by exactly the sort of manual dicking around with memory and buffer sizes that are trivial to avoid and completely unidiomatic in C++ but de rigeur in C. Is it possible to create these sorts of bugs in C++? Of course it is, but that's a far cry from an environment that actively leads you down a dangerous path because it lacks the necessary higher level abstractions.
C++ lacks the necessary higher-level abstractions to write memory-safe code. It is vulnerable to iterator invalidation, use-after-free, dangling references, and so forth. RAII does not provide the necessary guarantees.
> What we do know now is that the open source process does not converge to a no-bugs state.
Nobody to be taken serious has ever thought or said this -- it says about as much as "You know, I'm not perfect...".
Go itself (as of 1.3) is still itself coded in C, and I have no idea if/how C and Rust are related... C isn't going away, and tooling (Coverity, valgrind, nice compilers like clang) are our friends, as well as work like OpenBSDs string-handling ammendments, malloc() guards, etc.
I really am sympathetic to the complaints against C, but "Burn it!! It's a witch!" doesn't grab me.
I don't think Linus ever said it. I'm not sure he even agrees with it. Eric S. Raymond named it after him because of Linux.
That said, Eric S. Raymond didn't say the quoted either. He said that "given enough eyeballs, all bugs are shallow", which is a much less bold claim (imo, at least) than that open source converges towards zero bugs over time.
Well it never meant "there are no bugs in open source code" over any length of time. It just means that if a project has enough eyeballs on it, bugs will be squashed quickly. But how many eyeballs is "enough"? Obviously OpenSSL didn't have enough. Does glibc?
Do we really know that the open source process doesn't converge? No specification for the length of time toward convergence exists, and you could argue that this case is an example of inching ever closer to no-bugs state
Turns out, open source code is written by humans, same as at Microsoft. I remember, back in the day, it was absolutely pathetic that Microsoft Outlook had a buffer overflow in the subject line that could be tripped simply by receiving an email. Well, oops.
Aside from the fact that this bug was seen by eyes and fixed years ago (long enough ago that there's been a new LTS release with it fixed of several if not most distributions), even if they didn't know it was a security fix, this:
> We now know that the line "with many eyes, all bugs are shallow" is just wrong.
Remains approximately similar to saying, while watching the tide go out, that sea levels aren't rising. Anyone who ever thought it meant they were fixed instantaneously, and who uses the fact they get fixed and found as counter-proof of the sentiment, was wrong. It doesn't make the idea fundamentally wrong.
Garbage collection has a performance hit which may not be desirable. You don't really want your network stack to stall from time to time. And OS wouldn't be usable in any mission critical environment.
Array boundaries checking also has a performance hit but I am coming to think it is a necessary evil.
Dealing with strings as char arrays is just absurd. There isn't a significant performance hit to use some string datatype that reduce the opportunity for bugs.
Array bounds checking can often be optimized out. For constructs such as "for x in foo {}" this is trivial. The more general cases require a compiler which can hoist subscript checks out of loops when possible. The compiler has to be allowed to generate checks which fail early; that is, if the loop is going to subscript out of range at iteration 100 and there's no way to exit the loop early, then one check at loop entry is sufficient. This is hard to express to some optimizers.
Well, unless you get rid of C/C++ interfaces in all syscalls or Win32 APIs, hell Microsoft tried that with managed code, and utterly failed to deliver.
Except, "just use another language! All problems of this huge category solved!" is true in this case. You can't have buffer overflows on a memory-safe language. Sure, this is only true assuming the VM and all the stuff it depends on is formally verified not to have buffer overflows either, which is unlikely to happen. But even so, you get the slightly weaker guarantee: "just use another language! All problems of this huge category won't be your fault!" ;)
Now, it is not always practical to use safe languages for everything (specially low level libraries such as say, libc...), and that 'huge category' of problems is not even remotely close to being all the security problems. But using tools that prioritize not shooting yourself in the foot by default is not bad consideration to make, all other things being similar.
Rust provides memory safety without VM to begin with.
Although I do not like the comparison because it's not factually correct, Rust is like the "safe subset of C++". To write actual safe, modern C++ is very close to writing valid Rust. After learning Rust, I became a better C++ programmer.
As I understand it (and keep in mind I have never used Rust, so I might be completely wrong), Rust does have ways to perform unsafe pointer operations and the standard library includes code that does this, no? So, replace VM with libstd. Even if this is not the case, the compiler could produce code that is not memory safe due to bugs in the compiler itself. Turtles all the way down and all that.
Either way, I am all for reducing the number of places where random pointer juggling happens inside a program, be it by using a VM or forbidding certain language features outside of small system libraries (e.g. by banning "unsafe fn" from your own Rust code or by using a static checker to force you to use only the "safe subset of C++"). That way we just need to get a couple thousand lines of code right to solve this particular class of nastiness forever, instead of the hundreds of millions of LOCs that live above the system abstractions.
There are still a few things that rust provides that C++ doesn't. In particular, I think rust's ability to link different object's lifetimes together (think iterators and containers) without forcing you to treat them as the same object will be one of its most significant contributions to language semantic design.
The things is, it's not necessarily true in this case.
While C was intended to run as close to bare metal as possible, and 99% of current implementations do, that doesn't have to be the case. The C specification describes an abstract machine, and uses abstract concepts, with explicit "as-if" rules saying that implementations may, in many cases, do whatever they want, providing that conforming code runs as if it would on a naive implementation.
So there's no reason at all that, for example, pointers have to be implemented as actual bare addresses in a virtual address space which cannot be bounds checked. It should be perfectly possible to create a new implementation, with a new ABI, which defines pointers as a checked type such that all accesses through a pointer are accurately bounds checked, with a guaranteed "virtual" segfault happening whenever an invalid read or write would have occurred.
Sure, it's not ABI-compatible (by defualt) with current bare-metal C ABIs, and would require shims to work with bare-metal C libraries - but that's no different than Rust and similar other new languages. Sure, it's not quite as fast as bare-metal C implementations due to enforced bounds checking, but it's not going to be slower than Rust and similar which do the same job.
And the advantage would be, we wouldn't need to rewrite all our code. We could just recompile all the existing C code we already have to this new safe ABI, rather than having to rewrite everything from scratch in some new language!
Sure, C isn't the nicest language to use. But we already have plenty of existing code that uses it. Why don't we just write a new compiler back-end and take advantage of all that code in a safe manner?
You can't bounds check a pointer begotten from &arr[i]. It simply doesn't carry enough information. Moreover, there might be existing C programs that rely on out of bounds access (where they know that the out of bounds access falls into safe memory). So there is no way to implement a C virtual machine with bounds-checking semantics that is fully compatible with all existing C code.
Sure you can. If, for (bloated, demonstration-only) example, your opaque managed pointer type is actually this under the hood:
struct pointer {
void * real_base;
size_t size;
size_t offset;
};
then
type * p = &arr[i]
translates to
struct pointer p = { arr, sizeof(arr), i * sizeof(arr[0]) }
Any any use of "(p + j)" or "p[j]" can check that (p->offset + j sizeof(p)) is greater than or equal to 0 and less than p->size. (Excuse my confused use of types in the above.)
"there might be existing C programs that rely on out of bounds access (where they know that the out of bounds access falls into safe memory)"
Those programs are completely non-portable. They could break with your next compiler upgrade, let alone moving to a different compiler (clang?) or a different OS (BSD?).
It could happen. Witness the people who complained about their broken programs when memcopy() was sped up by taking advantage of the standard, or those who were surprised when NULL checks started being discarded after a pointer had already been dereferenced.
Even so, if you did have such programs, and were unlucky enough to rely on them, and were unable to fix them to comply with the C language spec, there's no reason you couldn't still compile them for the existing bare-metal ABI. I'm not proposing to ban* the x86-64 ABI. I'm just saying lets create an additional (x86-64-safe?) ABI that we could use to provide a safe execution environment for a subset of our existing code. That subset could range from none of it to all of it, depending on how much you personally valued speed and anti-bloat over safety, how many non-conforming programs you relied upon, and whatever other factors you wanted to take into account.
It's time to start phasing out C/C++. Languages which don't know how big their arrays are have to go. If it can run efficiently in a garbage-collected environment, it should be in Go or some scripting language. If it can't use GC, Rust is almost there. (As I say occasionally, I really hope the Rust guys don't screw up.) C and C++ should not be used for new work.
It has been 1 days since the last buffer overflow vulnerability report.