> Big remaining problems involve deadlock analysis, both in the thread sense and in the Rc/borrow sense. Those are somewhat equivalent. I'd like to have static deadlock analysis in Rust. If you had that, I think you could get safe back-pointers, too. If you can prove that all borrow and upgrade calls can't fail, you can eliminate most reference counts. That gives you free interior mutability when that's possible.
I’m not a formal verification guy, but I don’t think locks in the traditional sense is helpful. No compiler will give you anything useful to work with wrt locks. You just accept and live with the danger.
In fact, locks and ref counts are runtime constructs in Rust. Imo it “goes against” even Rust’s own strong principles of single ownership - these things are shared mutable state. Arc broke Rusts RAII model so much that they had to remove scoped threads (which needs destructors to run before end-of-scope).
Anyway, rust aside. Global refcounts have cyclic and leak issues that again are global. I don’t think we can necessarily get rid of them, but I do believe containing them is better, eg with arena-style scoping.
As for locks my gut feeling is we need an asymmetric data structure, for instance a channel. In eg Go you can break a channel into sender and receiver, and have the sender goroutine do `defer close(ch)` which is guaranteed to run if the rest of thread finishes, even if there’s a panic. Doesn’t necessarily have to be channels but separating the roles (reader-writer / producer-consumer) is much easier to reason about, and that probably helps formal analysis too.
Scoped threads have been reintroduced to Rust since ver 1.63, released in August 2022. There's also significant interest in introducing true linear types to Rust, where destructors would be guaranteed to run - however the changes required for that are large and will probably have to be introduced in a new Rust edition.
Unfortunately rust language development has practically ground to a halt. There is a trickle of work but it’s highly unlikely that major language features will happen any time soon.
I'm not sure what you mean. There are fewer flashy announcements for things that block people, but development continues unabated, with large refactors that enable things that weren't possible before. Things like Return Position Impl Trait In Traits, Derefs in patterns, a new trait system checker, enhancement to const eval, generators and that's just of the top of my head. Not everything is gonna land soon, but development is happening. The major language feature work is happening, they just are large enough or contentious enough that stabilizing them will take time.
Oh please, they’ve landed a few nice things but larger items like generators are likely not coming soon.
Just look at the async roadmap for example, how many years has it been and how much progress has been made?
As someone who is firmly in the Rust community it’s annoying how everyone seems to know this but you can’t fucking talk about it. I can’t tell you how many times I’ve seen passive aggressive comments like “this would be much easier if we had variadics… maybe someday”
> landed a few nice things but larger items like generators are likely not coming soon.
Generators got blocked on settling AsyncIterator/Stream. The work for it is happening. It's taken too long, but not because people are sitting on their thumbs.
> Just look at the async roadmap for example, how many years has it been and how much progress has been made?
Hard things are hard and take time.
> As someone who is firmly in the Rust community it’s annoying how everyone seems to know this but you can’t fucking talk about it.
You can say whatever you want to say, everyone can. But expect pushback from the people pulling the cart when you state that the cart isn't moving just because it isn't moving as fast as you want in the exact direction you want. Just take a damn glance at the review queue, or even the release notes. A ton of work is being done, of different levels of difficulty and the things that aren't stabilized quickly is because they have research-level open questions.
> I can’t tell you how many times I’ve seen passive aggressive comments like “this would be much easier if we had variadics… maybe someday”
I don't know why you read that as passive-aggressive, but I certainly see your reply as dickish.
If the cart is moving slowly I’m going to point out it’s moving slowly. If you want to take that personally that’s your problem, but I’d prefer to be honest about the situation.
The fact of the matter is significant language features have stalled. And comments like the one i quoted are passive aggressive when you consider they’re casting doubt on whether certain language features will ever see the light of day.
It’s especially disheartening when you don’t really hear much from the language team and it doesn’t seem like progress is being made on these language features.
GGP here. I think you’re both right. As a formerly involved contributor I can assure you that there are legitimately hard problems, and that there are very bright people working on it.
However, the reason why it’s so hard, is imo because Rust matured itself so quickly, by basically having a bunch of 10x mega brains lay a foundation of insufficient but unchangeable features and then leave a lot of details to be solved later when (a) there are much more people and stakeholders involved and (b) there are much more promises made around governance, stability etc. And, as it turns out, laying the “remaining pieces of the puzzle” is harder, especially if you’re not allowed to move the initial pieces.
This in turn creates an incongruence for outside observers, where it appears like development is slower (which is true) because people are incompetent/lazy/prioritize badly (which is not true).
The mistake here, in my humble opinion, is moving ahead so quickly many years ago. Async is the perfect example of this, but as I alluded to earlier, I think allowing leaks was a similarly rushed decision, and that was much earlier. If I were to bet, people were excited about maturing the language for production, but, since there was sooo much uncharted territory, there were novel subtle challenges which during the frenzy appeared like they could be solved quickly with brainpower alone to meet the deadlines and keep the excitement going. But even the wrinkliest of brains can’t deduce the implications of 4th order effects down the line - you have to try it out and build real life things. You have to be humble, when everyone else isn’t.
By what benchmark is it moving slowly, and are you qualified to make that judgement? You might be quite correct, but I don't have enough context to know what role you've played and without context you're coming across as "being loud and obnoxious on the Internet" which isn't (I think?) what you're aiming to be.
The reason I think this has low changes of landing has little to do with the rust developers as the complexity of the feature and the invasiveness to the ecosystem.
I mostly agree with your sentiment, but at the same time I remember being quite skeptical of the possibility of Rust getting async/await at all because I saw no way around the lack of immovable types in the language, and I thought it would be way to disruptive to add it. And then Pin came and allowed to work around that issue (don't get me wrong, Pin is a dirty hack and I hate it, but at least it works, even if it makes tge ergonomics questionable for some use-cases…).
Since then reserve my judgment on stuff I think to be impossible.
Right, but those are a mere shadow of what they used to be, which was JoinHandle with a lifetime parameter. You can’t build a thread pool or async runtime with static borrowing across tasks, which makes the use case very narrow.
> There's also significant interest in introducing true linear types to Rust, where destructors would be guaranteed to run
I’m glad to hear that. I’ve always hoped it’s possible to introduce non-breaking through a new auto trait but there are many delicate details around unwinds and such that may be gnarly. As well as an accumulated dependence on Arcs in library code.
> I’m not a formal verification guy, but I don’t think locks in the traditional sense is helpful. No compiler will give you anything useful to work with wrt locks. You just accept and live with the danger.
Deadlock analysis is well understood. You have to prove that locks get locked in the same order regardless of the path taken. That can be easy or hard. In many cases, if you have the call graph annotated with where locks are set and cleared, you can try to prove that the program is deadlock-free. I'd like to see Rust tools to help with that.
Right, but that’s a global property of everything that uses the lock, so if you’re writing a library you'd have to run the analysis against real or synthetic call graphs. Plus it’s very easy to accidentally break that property if you’re not using it in an analysis-friendly way, no? Instead, role-separated data structures like eg channels, run-once-functions and atomic value swaps can - due to their constraints - make them harder to use in ways that break analysis, and can be analyzed locally.
Say eg my library is acting as a producer. Ideally I can check that I am producing 0-N items and that it closes the channel. Analyzing the consumer can be deferred and decoupled, which is good because it may not exist yet.
I’m not a formal verification guy, but I don’t think locks in the traditional sense is helpful. No compiler will give you anything useful to work with wrt locks. You just accept and live with the danger.
In fact, locks and ref counts are runtime constructs in Rust. Imo it “goes against” even Rust’s own strong principles of single ownership - these things are shared mutable state. Arc broke Rusts RAII model so much that they had to remove scoped threads (which needs destructors to run before end-of-scope).
Anyway, rust aside. Global refcounts have cyclic and leak issues that again are global. I don’t think we can necessarily get rid of them, but I do believe containing them is better, eg with arena-style scoping.
As for locks my gut feeling is we need an asymmetric data structure, for instance a channel. In eg Go you can break a channel into sender and receiver, and have the sender goroutine do `defer close(ch)` which is guaranteed to run if the rest of thread finishes, even if there’s a panic. Doesn’t necessarily have to be channels but separating the roles (reader-writer / producer-consumer) is much easier to reason about, and that probably helps formal analysis too.