I can shed some light here! This is going to be longish comment, but hopefully by the end of it you should understand _why_ we decided to avoid using the disk initially, even if you don't agree with that decision.
Historically, the decision to not use disk traces back to this comment https://github.com/rust-lang/rfcs/pull/1317#issuecomment-150..., which is perhaps the single GitHub comment that influenced my life most. Very high impact, thanks dgrunwald! Specifically,
>Don't store anything to disk. It's likely the oracle can be fast enough without doing this; and unnecessary complexity creates bugs. "Have you tried deleting the .ncb file?" (I remember having to do this a couple times per day when using VS, ca. 2005)
>Use lazy evaluation. The IDE is only interested in very specific bits of information, almost always restricted to a couple of lines around the cursor. Avoid calculating stuff that might never get used before it gets invalidated by the next code change.
>At least for C#, laziness saves so much time that incremental compilation is unnecessary for IDE purposes
The other part of historical context was that the motivation for creating rust-analyzer was that I didn't want to write a second Rust compiler (having been doing that for a couple of years at JetBrains). So it was explicitly an experimental project to prototype the right architecture for an IDE, to ultimately change how rustc works internally, so that, down the line, an IDE and a command-line compiler could use the same core. Given that rust-analyzer is now effectively a separate rust compiler, it's safe to say I am not good at achieving my life's goals!
In that context, I believe that avoiding disk was the _right_ decision:
* It's not really germane to the problem space, if all you need is literally a cache, it can always be added later.
* Disk is a can of worms of data consistency problems. They can be overcome with engineering effort to ultimately give better user experience, but user experience wasn't the primary goal. And using disk wouldn't actually illuminate the interesting aspects of the architecture, the intended primary goal.
* Finally, _not_ using disk would be a forcing function to keep analysis fast enough, to not make startup prohibitive.
The last one was a particularly big argument in my mind --- I didn't want to reach out for "easy" solutions prematurely, to avoid avoiding hard problems. And, again, my recollection is probably not 100% correct, but, until we added support for proc macros and build scripts, it was fine-ish from the perspective of startup time (RAM usage is a different story). The problem with proc_macros and build.rs is that they need to run the rust code, so they have to run the real rustc compiler, so all our usual IDE tricks ("information ... restricted to a couple of lines around the cursor") just don't apply.
The reason why we didn't add it later was that it seemed a relatively lower priority task than the work to share the parser between rust-analyzer and rustc. So that's what I was focusing on, though, I didn't deliver that. I still think we should do it! There's no _insurmountable_ technical reasons why the parsers can't be shared! It's just (a lot of) engineering work. And, while the parser is the boring part of compiler, it's the interesting part of an IDE.
Anyway, that explains how we ended up where we are.
That being said, I don't think that "just adding disk cache" is the right approach --- the salsa in-memory data structure is very sparse and pointy. Dumping that to disk would help somewhat, but wouldn't be a great long term solution. What is needed (I also explain this in https://matklad.github.io/2026/08/21/rust-glancer.html) is to design a compact, first class data format for representing analysis information about the crate, and than teaching rust-analyzer to be polymorphic in the source of data. For current workspace, you want to use a lazy incremental in-memory data structure (I do think we sadly need incrementally for Rust, given its compilation unit structure). For dependencies, you want to work off a compact on disk index. And, if the user "goes to definition" and mutates its file in place, we want to transparently switch between the two. The _pre requisite_ for that was to define a backend agnostic analysis API, and that work was always slowly progressing in the background (https://hackmd.io/ytd82QNiT_Ku2XFr1EAtiQ), but it generally took the backseat, while sharing the parser was the main focus.
It would be nice if you could configure a maximum memory amount it's allowed to use. It regularly goes over 13gb in my large workspace so I have to configure it to run in a cgroup so it gets killed when it goes over 8gb or I can't run the other things on my desktop that I need to. I restart nvim after it crashes and it's back to around 6gb and working fine so it doesn't seem to need 13 GB to function. If it could handle that gracefully on it's own without cgroups/restarts that'd be a win.
Regardless, thanks for your work on RA!
Feels like you are missing the point a little bit. LSP is a human facing tool, so providing a good user experience should be the ultimate goal of it. What you described, feels to me, is not quite aligned with this goal.
Of course, since you describe RA as "explicitly an experimental project to prototype the right architecture for an IDE", all that doesn't matter and you can set the project's goal to be whatever you want. However, it just sucks for us, the users, because RA is the only thing we have. I'm just glad to see someone else is trying to do something different. And may the best project win.
The goal always was to refactor rustc to make compiler-based LSP feasible. The idea of original RLS, "we'll just use the compiler", is fundamentally sound. It's just that you'll need to turn the compiler sideways to achieve that, which is a hard work to motivate without having a real example demonstrating the benefits (or having political clout in the project to just mandate that ^^)
Your thoughts are quite cool! but I thought featuring the project itself would make more sense for a frontpage thread, so I'm going to merge the comments (such as they are) from https://news.ycombinator.com/item?id=49392654 and add your link to the toptext above. Thanks for drawing attention to this topic!
Thanks for the coverage and kind words!
The title of the post is a bit more ambitious than what I am confident to guarantee, but I'll try my best to live up to it ^_^"
Some comments on the thoughts post
> I think that part can perhaps be made lazy (but not incremental!) with little overhead?
I am still thinking about making stuff lazy, since with non-incremental approach it can introduce more lags than would be perceived comfortable, but what I do right now is that I prioritize open buffers (so the stuff user needs gets processed faster), and everything else is indexed in background. I have some thoughts about lazy approach, but before I'll try them, I want to work on the quality of analysis first.
> Would be interesting to compare memory usage with Rust Rover. Net of the IDE GUI itself, I would expect RR to be more compact.
I've received a few comments about RR already, and, to be honest, I've never tried it (somehow I never got along with JetBrains IDEs) -- but will look into it.
> One potential approach here is to pull the Sorbet trick, where you don’t run meta programming at all, and instead have a plugin interface to “explain” the effects of what that would have done.
Funnily, that's exactly (well, mostly) the idea I have in mind and want to try out. Tentatively planned for Rust Glancer 0.3.0 (0.2.0 will be mostly about more complete indexing/functionality and editors support). In short, I don't want to have random code execution in the LSP itself (even diagnostics are disabled by default), but it's quite possible that we don't need that for proc macros.
> Try changing this option and see if it helps?
I have tried both editor and server watcher options, didn't really feel the difference, but can't say that I performed a high quality investigation. I certainly noticed that vs code is not very good at properly reporting external changes (it misses a lot of them), and the server watcher was tricky to get right (and yeah, it has quite a bit of platform-specific quirks; which is one of the reasons I don't feel comfortable providing a server for Windows yet -- I have no machine to test it).
> This still seems to me to be the lowest-hanging watermelon here — split the world into arcy-pointy incremental tip of the iceberg, and mostly read-only, on disk, compact, dark, moist breeding ground for supply chain attacks.
This would be awesome! And I'd be really happy to see that change making Rust Glancer redundant; while ability to experiment is cool, I think that unified tooling is ultimately better for the language.
To add more context around lifetime errors and TigerBeetle's particular style guide:
>Many projects opt to answer these kinds of questions through a style guide. TigerBeetle's TigerStyle is an example in Zig and Google's 31,000 word C++ style guide is another. The challenge with style guides is enforcement.
TigerStyle[1] is a bit more than just a style guide. The key rule for this discussion, uplifted straight from of NASA[2], is *static memory allocation*: all memory is allocated in the startup phase, and there's absolutely zero `alloc`s afterwrads . This plus crash only[3] design means that we never call `free`.
This rule is self-enforcing and compositional, in Zig. There's no global memory allocator, so the code after startup simply hasn't the API to allocate. You can't circumvent this by accident. Of course, if the programmer is byzantine, they can stuff allocator in the global, or just directly `mmap` and `unmap` pages of memory, but, at our scale, we don't have problems with that. This is a similar in kind (not degree) to Rust, where untrusted code generally can circumvent safety guarantees, even without literally spelling `unsafe`.
And, naturally, never `free`ing goes a long way towards solving many memory errors by construction. Empirically, they just haven't been a problem for TigerBeetle. It's hard to untangle contribution of static allocation in particular from everything else we are doing, but it would make sense for it to play a leading role.
(As a footnote, we aren't actually do static allocation to avoid memory errors, we use it as a linter to check that every quantity has a known _logical_ static limit, the main property we care about)
By all accounts, TigerBeetle has been a tremendous success, congratulations! My understanding is that it has a deliberately fixed scope, which makes me wonder: how applicable would TigerStyle be in more general-purpose applications? If the system needs to, say, ingest arbitrary JSON documents ranging from 100 bytes to 100 GB, how would TigerStyle fare?
What matklad said as a footnote deserves a little more attention!
A huge benefit is having to think and be explicit about the limits up front. To take your example of arbitrary sized JSON and flip it around: how would you do it normally?
Maybe, you'd allocate a buffer to hold your entire object when it comes in - but now you'll end up crashing when you get a large document that exceeds your available memory.
Maybe you can do things in a fixed amount of memory, using streaming or chunking - which would be pretty simple to turn to static allocation!
Static allocation forces understanding of those limits upfront.
If it helps anyone explain the SkiFire point any better, I like to analogize it to an I-bar cursor vs. a block cursor for text entry. An I-bar is unambiguously "between characters" while a block cursor is not. So, there are questions that arise for block cursors that basically never arise for I-bar cursors. When just looking at an integer like 2 or 3, there is no cursor at all. So, we must instead rely on names/conventions/assumptions with their attendant issues.
To be clear, I liked the SkiFire explanation, but having multiple ways to describe/think about a problem is usually helpful.
I assume you ment to write `assert(subexpression != undefined)`?
This is resilient parsing --- we are parsing source code with syntax errors, but still want to produce a best-effort syntax tree. Although expression is required by the grammar, the `expression` function might still return nothing if the user typed some garbage there instead of a valid expression.
However, even if we return nothing due to garbage, there are two possible behaviors:
* We can consume no tokens, making a guess that what looks like "garbage" from the perspective of expression parser is actually a start of next larger syntax construct:
```
function f() {
let x = foo(1,
let not_garbage = 92;
}
```
In this example, it would be smart to _not_ consume `let` when parsing `foo(`'s arglist.
* Alternatively, we can consume some tokens, guessing that the user _meant_ to write an expression there
```
function f() {
let x = foo(1, /);
}
```
In the above example, it would be smart to skip over `/`.
- Operational predictability --- latencies stay put, the risk of threshing is reduced (_other_ applications on the box can still misbehave, but you are probably using a dedicated box for a key database)
- Forcing function to avoid use-after-free. Zig doesn't have a borrow checker, so you need something else in its place. Static allocation is a large part of TigerBeetle's something else.
- Forcing function to ensure existence of application-level limits. This is tricky to explain, but static allocation is a _consequence_ of everything else being limited. And having everything limited helps ensure smooth operations when the load approaches deployment limit.
- Code simplification. Surprisingly, static allocation is just easier than dynamic. It has the same "anti-soup-of-pointers" property as Rust's borrow checker.
Yes, kind of. In the same sense that Vec<T> in Rust with reused indexes allows it.
Notice that this kind of use-after-free is a ton more benign though. This milder version upholds type-safety and what happens can be reasoned about in terms of the semantics of the source language. Classic use-after-free is simply UB in the source language and leaves you with machine semantics, usually allowing attackers to reach arbitrary code execution in one way or another.
That what happens can be reasoned about in the semantics of the source language as opposed to being UB doesn't necessarily make the problem "a ton more benign". After all, a program written in Assembly has no UB and all of its behaviours can be reasoned about in the source language, but I'd hardly trust Assembly programs to be more secure than C programs [1]. What makes the difference isn't that it's UB but, as you pointed out, the type safety. But while the less deterministic nature of a "malloc-level" UAF does make it more "explosive", it can also make it harder to exploit reliably. It's hard to compare the danger of a less likely RCE with a more likely data leak.
On the other hand, the more empirical, though qualitative, claim made by by matklad in the sibling comment may have something to it.
[1]: In fact, take any C program with UB, compile it, and get a dangerous executable. Now disassemble the executable, and you get an equally dangerous program, yet it doesn't have any UB. UB is problematic, of course, partly because at least in C and C++ it can be hard to spot, but it doesn't, in itself, necessarily make a bug more dangerous. If you look at MITRE's top 25 most dangerous software weaknesses, the top four (in the 2025 list) aren't related to UB in any language (by the way, UAF is #7).
>If you look at MITRE's top 25 most dangerous software weaknesses, the top four (in the 2025 list) aren't related to UB in any language (by the way, UAF is #7).
FWIW, I don't find this argument logically sound, in context. This is data aggregated across programming languages, so it could simultaneously be true that, conditioned on using memory unsafe language, you should worry mostly about UB, while, at the same time, UB doesn't matter much in the grand scheme of things, because hardly anyone is using memory-unsafe programming languages.
There were reports from Apple, Google, Microsoft and Mozilla about vulnerabilities in browsers/OS (so, C++ stuff), and I think there UB hovered at between 50% and 80% of all security issues?
And the present discussion does seem overall conditioned on using a manually-memory-managed language :0)
You're right. My point was that there isn't necessarily a connection between UB-ness and danger, and stuck together two separate arguments:
1. In the context of languages that can have OOB and/or UAF, OOB/UAF are very dangerous, but not necessarily because they're UB; they're dangerous because they cause memory corruption. I expect that OOB/UAF are just as dangerous in Assembly, even though they're not UB in Assembly. Conversely, other C/C++ UBs, like signed overflow, aren't nearly as dangerous.
2. Separately from that, I wanted to point out that there are plenty of super-dangerous weaknesses that aren't UB in any language. So some UBs are more dangerous than others and some are less dangerous than non-UB problems. You're right, though, that if more software were written with the possibility of OOB/UAF (whether they're UB or not in the particular language) they would be higher on the list, so the fact that other issues are higher now is not relevant to my point.
> In fact, take any C program with UB, compile it, and get a dangerous executable. Now disassemble the executable, and you get an equally dangerous program, yet it doesn't have any UB.
I'd put it like this:
Undefined behavior is a property of an abstract machine. When you write any high-level language with an optimizing compiler, you're writing code against that abstract machine.
The goal of an optimizing compiler for a high-level language is to be "semantics-preserving", such that whatever eventual assembly code that gets spit out at the end of the process guarantees certain behaviors about the runtime behavior of the program.
When you write high-level code that exhibits UB for a given abstract machine, what happens is that the compiler can no longer guarantee that the resulting assembly code is semantics-preserving.
There's some reshuffling of bugs for sure, but, from my experience, there's also a very noticeable reduction! It seems there's no law of conservation of bugs.
I would say the main effect here is that global allocator often leads to ad-hoc, "shotgun" resource management all other the place, and that's hard to get right in a manually memory managed language. Most Zig code that deals with allocators has resource management bugs (including TigerBeetle's own code at times! Shoutout to https://github.com/radarroark/xit as the only code base I've seen so far where finding such bug wasn't trivial). E.g., in OP, memory is leaked on allocation failures.
But if you manage resources manually, you just can't do that, you are forced to centralize the codepaths that deal with resource acquisition and release, and that drastically reduces the amount of bug prone code. You _could_ apply the same philosophy to allocating code, but static allocation _forces_ you to do that.
The secondary effect is that you tend to just more explicitly think about resources, and more proactively assert application-level invariants. A good example here would be compaction code, which juggles a bunch of blocks, and each block's lifetime is tracked both externally:
I see a weak connection with proofs here. When you are coding with static resources, you generally have to make informal "proofs" that you actually have the resource you are planning to use, and these proofs are materialized as a web of interlocking asserts, and the web works only when it is correct in whole. With global allocation, you can always materialize fresh resources out of thin air, so nothing forces you to do such web-of-proofs.
To more explicitly set the context here: the fact that this works for TigerBeetle of course doesn't mean that this generalizes, _but_, given that we had a disproportionate amount of bugs in small amount of gpa-using code we have, makes me think that there's something more here than just TB's house style.
That's an interesting observation. BTW, I've noticed that when I write in Assembly I tend to have fewer bugs than when I write in C++ (and they tend to be easier to find). That's partly because I'm more careful, but also because I only write much shorter and simpler things in Assembly.
Hey matklad! Thanks for hanging out here and commenting on the post. I was hoping you guys would see this and give some feedback based on your work in TigerBeetle.
You mentioned, "E.g., in OP, memory is leaked on allocation failures." - Can you clarify a bit more about what you mean there?
Gotcha. Thanks for clarifying! I guess I wasn't super concerned about the 'try' failing here since this code is squarely in the initialization path, and I want the OOM to bubble up to main() and crash. Although to be fair, 1. Not a great experience to be given a stack trace, could definitely have a nice message there. And 2. If the ConnectionPool init() is (re)used elsewhere outside this overall initialization path, we could run into that leak.
Those guidelines are quite clear that they're written specifically in the context of the C programming language, and may not make sense in other contexts:
"For fairly pragmatic reasons, then, our coding rules primarily target C and attempt to optimize our ability to more thoroughly check the reliability of critical applications written in C."
A version of this document targeting, say, Ada would look quite different.
The JPL C rules are quite old, but avoiding dynamic allocation outside initialization is am considered best practice for spaceflight software regardless of language. Here's the recommendation from NASA's language-agnostic cFS:
4.2.4 Consolidate Resource Allocations
It is generally recommended to consolidate resource allocations to the application initialization function(s). Allocations and setup of resources such as memory pools and child tasks should happen once during initialization in order to provide more determinism during run time.
> The JPL C rules are quite old, but avoiding dynamic allocation outside initialization is am considered best practice for spaceflight software regardless of language.
The rules are written with the historical context of C making it too easy to leak heap-allocated memory. In the safety-critical Rust code that I've worked on, we tend not to dynamically allocate due to the usual constraints, and we're well aware of the "thou shalt not allocate" rules in the scripture, but we've already gotten clearance from the relevant certification authorities that Rust is exempt from the restriction against dynamic allocation specifically because of its ownership system.
This is scary, the issue in safe-critical code is not leaks (which Rust also does not necessarily prevent), but accidental resource exhaustion. This is also why JPL forbids recursion.
Even MISRA has rules allowing dynamic allocation these days. It's just a recommendation in most standards to make memory usage patterns easier to reason about.
They do make a lot of sense in other contexts :-) From the actual rules, only #2 (minimize preprocessor) and #10 (compiler warnings) are C specific. Everything else is more-or-less universally applicable.
As a tiny nit, TigerBeetle isn't _file system_ backed database, we intentionally limit ourselves to a single "file", and can work with a raw block device or partition, without file system involvement.
>we intentionally limit ourselves to a single "file", and can work with a raw block device or partition, without file system involvement
those features all go together as one thing. and it's the unix way of accessing block devices (and their interchangeability with streams from the client software perspective)
Historically, the decision to not use disk traces back to this comment https://github.com/rust-lang/rfcs/pull/1317#issuecomment-150..., which is perhaps the single GitHub comment that influenced my life most. Very high impact, thanks dgrunwald! Specifically,
>Don't store anything to disk. It's likely the oracle can be fast enough without doing this; and unnecessary complexity creates bugs. "Have you tried deleting the .ncb file?" (I remember having to do this a couple times per day when using VS, ca. 2005)
>Use lazy evaluation. The IDE is only interested in very specific bits of information, almost always restricted to a couple of lines around the cursor. Avoid calculating stuff that might never get used before it gets invalidated by the next code change.
>At least for C#, laziness saves so much time that incremental compilation is unnecessary for IDE purposes
The other part of historical context was that the motivation for creating rust-analyzer was that I didn't want to write a second Rust compiler (having been doing that for a couple of years at JetBrains). So it was explicitly an experimental project to prototype the right architecture for an IDE, to ultimately change how rustc works internally, so that, down the line, an IDE and a command-line compiler could use the same core. Given that rust-analyzer is now effectively a separate rust compiler, it's safe to say I am not good at achieving my life's goals!
In that context, I believe that avoiding disk was the _right_ decision:
* It's not really germane to the problem space, if all you need is literally a cache, it can always be added later.
* Disk is a can of worms of data consistency problems. They can be overcome with engineering effort to ultimately give better user experience, but user experience wasn't the primary goal. And using disk wouldn't actually illuminate the interesting aspects of the architecture, the intended primary goal.
* Finally, _not_ using disk would be a forcing function to keep analysis fast enough, to not make startup prohibitive.
The last one was a particularly big argument in my mind --- I didn't want to reach out for "easy" solutions prematurely, to avoid avoiding hard problems. And, again, my recollection is probably not 100% correct, but, until we added support for proc macros and build scripts, it was fine-ish from the perspective of startup time (RAM usage is a different story). The problem with proc_macros and build.rs is that they need to run the rust code, so they have to run the real rustc compiler, so all our usual IDE tricks ("information ... restricted to a couple of lines around the cursor") just don't apply.
The reason why we didn't add it later was that it seemed a relatively lower priority task than the work to share the parser between rust-analyzer and rustc. So that's what I was focusing on, though, I didn't deliver that. I still think we should do it! There's no _insurmountable_ technical reasons why the parsers can't be shared! It's just (a lot of) engineering work. And, while the parser is the boring part of compiler, it's the interesting part of an IDE.
Anyway, that explains how we ended up where we are.
That being said, I don't think that "just adding disk cache" is the right approach --- the salsa in-memory data structure is very sparse and pointy. Dumping that to disk would help somewhat, but wouldn't be a great long term solution. What is needed (I also explain this in https://matklad.github.io/2026/08/21/rust-glancer.html) is to design a compact, first class data format for representing analysis information about the crate, and than teaching rust-analyzer to be polymorphic in the source of data. For current workspace, you want to use a lazy incremental in-memory data structure (I do think we sadly need incrementally for Rust, given its compilation unit structure). For dependencies, you want to work off a compact on disk index. And, if the user "goes to definition" and mutates its file in place, we want to transparently switch between the two. The _pre requisite_ for that was to define a backend agnostic analysis API, and that work was always slowly progressing in the background (https://hackmd.io/ytd82QNiT_Ku2XFr1EAtiQ), but it generally took the backseat, while sharing the parser was the main focus.