I'm arguing against "Java performance is as good as it gets" in this thread, but the best Java GCs have sub-millisecond pause times for huge heap sizes and, apparently, such a GC even became the default recent-ish-ly: https://inside.java/2023/11/28/gen-zgc-explainer/
The JVM is amazing technology for what it is, but what it is still limits how fast it can go.
We are really talking about different categories of software. The C++ / Rust world typically operates at a different one than where Java does. It used to be manual memory management was a burden that the biz wasn't willing to invest in for typical software - with LLMs, the trade-off landscape has changed (sounds like buzzword BS, but it's true).
When you can get the benefits and let the LLMs pay for the downsides you'd otherwise have to handle - that's a change of circumstances that we really ought to pay attention to as software devs.
Can you write very fast Java software, yes - LMAX is a great example. Can you write slow Rust/C++ code, absolutely. But the values, community and tech built around the latter really does make it a lot easier when that's your (and your LLM's) goal.
Of the general-purpose methods it does; that's is exactly what these collectors were invented for - not to carry all that overhead of malloc/free. I guess you could argue over the frequency of situations where it yields a significant boost. It's also true that malloc/free have also improved, especially in the very large runtimes (like TCMalloc and jemalloc), but they can't do all the optimisations moving collectors can.
It’s great that they were created for this purpose, but I have a faster allocator in non-moving GC engine in C++. And I don't have to moving gigabytes of data.
Generational moving allocators don't move objects much, and so the total amount of work they need to do is substantially less than malloc/free allocators -- that's the whole idea of these algorithms and the reason why they're used (that's not under dispute; it's just the mathematics of the algorithms [1]). Still, there might be some situations where allocators in C++ could be faster, but if that were the case for my purposes, I would have stuck with C++. We needed very good performance for highly concurrent workloads, and C++ just doesn't come close in many of those (maybe at 10x the budget).
[1]: More precisely, the work they need to do doesn't scale linearly with the allocation rate as is the case with non-moving approaches. Of course, arenas are sort of degenerate moving collectors, so they also display the same effect, but they can't always be used.
The way this works is that you allocate in the young generation with a pointer bump, and to free memory you compact (move) the surviving objects, which make up a small minority, a few times and then promote (move) them to the old generation (where you either manage memory with the same technique or a different one). And so the total amount of moving is not high (objects either never move or rarely move), and what you end up with is essentially two heaps, one where the allocation rate may be high but you can completely negate all the extra work by increasing the size of that heap, and one where the allocation rate is low. This ability to, optionally, not double the amount of work when the allocation rate doubles is what gives moving collectors their efficiency (and that you can bump-allocate in virtually all cases).
Of course, it's possible to come up with programs where objects do move a lot (or there's some other common work), e.g. programs that don't exhibit the behaviour of the generational hypothesis, but the point is that in a very large class of programs, this technique is hard to compete with.
ZGC is effectively pause-free. It uses a STW pause for thread synchronisation of the epoch as an optimisation (we tried synchronising without the pause, but OS scheduling just slowed us down; maybe we'll use new OS techniques for synchronisation in the future). ZGC does zero GC work in these pauses. There is also no strong reason to remove the synchronisation pause because it is no longer than OS jitter (i.e. C programs can experience pauses of similar duration on non-realtime kernels).
BTW, SGCL is a non-moving mark-and-sweep collector and non-moving collectors are no more similar to moving collectors than malloc/free is similar to moving collectors. Garbage collection contains different classes of algorithms with different goals that are so different from one another that it's almost pointless to put them in the same category. Java uses moving collectors because it's the class of algorithms designed to reduce the throughput overhead of all non-moving approaches (from manual malloc/free, through automatic RAII and refcounting GC, through tracing mark-and-sweep collectors like SGCL or Go's GC). In important ways, all non-moving approaches are more similar to each other (regardless of the use of a GC) than any of them are to moving approaches. (Java did have a concurrent mark-and-sweep collector at one time, but we removed it some years ago)
Not everything has to be ARC though and arenas are strictly better than GCs lacking synchronization issues for allocation and not needing any scanning for deallocation.
It depends on the workload. ARC can be more expensive than tracing GC, especially with heavily shared objects.
But my main point was that the presence of a GC has nothing to do with how close a language is to the metal. Memory management strategy and low-level capabilities are two separate things.
Rust could just as well have an optional tracing GC for shared ownership instead of ARC, potentially improving performance rather than reducing it. Having a GC doesn't inherently make a language slower.
Yeah but let's be honest; GC languages (Go, Java, C#) usually are slower than systems languages. Systems languages just give you more, low level control over the computer from within your program. You can use that control to improve performance. Eg, you can control data locality, memory access patterns, the emitted assembler, and way more stuff.
Of course you're right - if you misuse arc, you can make your program slow. So don't misuse arc then. Rust gives you lots of options for structuring memory. If you choose badly, that's on you.
Because that is precisely the trade-off we're discussing.
With immediate reference-counted reclamation, dropping the last reference can put an arbitrarily large amount of destruction and deallocation work on the current thread. If you defer that reclamation to get it off the critical path, you have introduced a collector that reclaims unreachable objects later.
Whether that collector discovers garbage by tracing or receives zero-reference objects from reference counting is an implementation detail. In both cases, memory reclamation is automatic and deferred.
So the supposed distinction "with ARC I know when I pay the reclamation cost, with GC I don't" only holds if you're willing to pay an unbounded reclamation cost synchronously when the last reference is dropped.
In the normal implementation of reference counts, counters can be decremented only at block exits and not at any other program point.
At a block exit some of the local variables that are freed may contain references, so freeing them will decrement some reference counts. Then some counters will reach zero, triggering other deallocations and the decrementing of other counters. This will repeat until no other counters reach zero.
All the memory deallocation happens predictably, only at block exits.
If a variable is not freed immediately when a counter reaches zero, but the deallocation is deferred for a later time, which is not predictable, that is no longer classic memory management with reference counts, but it is a garbage collector, which happens to also use reference counts, probably in combination with some tracing algorithm.
When reference counts are implemented, manual memory deallocation, like with C free() or C++ delete, should be forbidden, but even if it were used that would just introduce other program points besides the block exits, where it is known that memory deallocation will happen.
You've described where deallocation can happen, not when it will. Every block exit is a candidate, but which one drops the last reference depends on runtime state: how many other owners the object has and which of them goes away first. With shared ownership across threads it's a race by construction. The free runs on whichever thread happens to release last.
The cost isn't bounded either. Dropping the last reference to the head of a list or the root of a tree frees the whole structure at that block exit, and its size is a runtime property.
And it isn't only block exits. Every assignment to a variable or field holding a reference decrements the old target, and so does removing an element from a container. Swift's ARC doesn't even promise the scope boundary: the optimizer may release right after the last use, which is why withExtendedLifetime exists.
By the same "where" criterion a non-concurrent tracing GC is predictable too, because it can only run at allocation points. That doesn't tell you which allocation will trigger it, just as knowing the block exits doesn't tell you which one will free.
reply