Does having the worker pool hold as many threads as cores work well alongside the async pool? It is basically oversubscribed by design.
I built a system once which had (this is Rust) a Rayon worker thread pool of 4 threads and a Tokio async pool of 2 (multithreaded runtime). On a system of 6 vCPU. This ended up working fine. Tokio was not starved so handled network requests at low latency.
One difference is DuckDB is a pure network client. If one of its async threads is starved it is not the end of the world (e.g. k8s does not kill your pod for failure of replying to health checks).
I've run quite a few benchmarks on that as well, on a few different machines, and oversubscribing ASYNC threads demonstrated very little performance downside. In the end, the memory governor also keeps these threads "in check" while still allowing full utilization when possible.
There is still something to gain from tuning it further (as you can see in the async I/O tuned benchmark), but having that network saturation by default is still a work in progress.
Network IO is heavily NIC queue bound, if your NIC only has one queue it just makes it slower to do any threading workload against it.
On my ryzen 9 it needs around 8 cores to do the same work in a threaded io loop than you can do single threaded. And the mechanism doesnt matter, you could share an fd, use SO_REUSEPORT or just share memory between threads.
Just doing the sharing makes everything extremely slow. One context switch becomes more expensive than just doing it single threaded.
As long as you're scheduled by the kernel and not something like Kubernetes with a CPU limit, you can generally oversubscribe I/O threads without much of a problem. They're mostly parked waiting for syscalls anyway. Heck, even if they're mostly doing CPU work, the scheduler generally deals with it pretty gracefully.
Yeah, they're excellent. I use a small Qwen3-4B-Instruct as the "actor" agent which goes out to the website, searches for a batch of items, decides which best matches the criteria, and puts them in the cart. Then I have Qwen3.6-35B-A3B as the "orchestrator" agent (it's just what I use for my daily driver), it's responsible for chatting with my wife about the menu, recipes and ingredients, and tells the smaller agent what to put in the cart with our brand preferences, dietary restrictions, etc. in mind.
I could probably drop the smaller Qwen at this point, but when I was first building this I was having an issue with search results and cart data filling up the main agent's context.
Which has been working well. I admit I do not understand what all of these flags do in detail.
This uses ssh agent forwarding, and then sudo via PAM. That allows for passwordless sudo. Building (well, activating) without sudo is pretty involved last I checked, I could not get it to work.
Funnily enough, spawn_blocking is not the right tool here. It is meant for blocking I/O, such as DNS lookups, where your platform might not give you anything better.
For genuine CPU-bound work, submitting to a Rayon worker pool is the way to go. It solved a runtime starvation issue for us at work, spawn_blocking did not work.
The reason for all this is spawn_blocking having a very large underlying thread pool, in the hundreds. That is okay if you assume work will yield those threads and mostly sleep/wait. It is not okay if the work never yields, like pure data crunching. (Go solves this by forcefully preempting loops, no such thing in Rust without a language runtime)
Our solution shape was: multi-threaded Tokio (2 threads), then give the rest of available_concurrency to a Rayon thread pool. If you grant 6 vCPU you should see a thread pool of 4, and a maximum CPU consumption of about 400%, as the Tokio threads sit mostly idle (under low load single-threaded runtime should also suffice).
You inject the thread pool using an Arc.
Then, when work comes in, just spawn Tokio tasks liberally (cheap) and submit to the thread pool. Rayon will internally queue and limit concurrency and parallelism to 4 (this is the important bit compared to spawn_blocking: no way your system can hog all 6 threads with non-yielding work and starve Tokio runtime threads).
We use one-shot channels to submit results back, they are designed for exactly this. The tx aka sender end is sync, as there is never a wait (cannot block). The rx aka receiver side is async and can be awaited normally on the async side. This is a cheap operation, similar to Go.
Optionally you can reach for semaphores to also limit I/O concurrency. You probably want to do this for more control and avoiding resource exhaustion loudly (that is, not silently accidentally peg thousands of FDs, database connections, …).
It ended up working beautifully for our purposes and relatively simply. No lifetime woes, Arc solves those. Oneshot channels just transfer ownership etc.
Perhaps this is what TFA talks about, I have not read it.
One caveat: to reach all the above conclusions and designs, we had help from some genuine Rust experts. As much as I dislike Go, it "just works" there even if one writes naive code.
Exactly. I do not know the specifics, but for example if libraries you call into liberally spawn_blocking under the expectation that it is okay, you will be in trouble.
Says it right there actually:
> It’s recommended to not set this limit too low in order to avoid hanging on operations requiring spawn_blocking.
So a total like 6 - reasonable for a web backend - would be way too low.
I also run self-hosted Wireguard. Initially on a Debian box, nowadays it is integrated into my router (admittedly, this is closed source). For around 6 years at this point.
The whole thing could not be easier and simpler. It has never randomly broken on me. It is fast. It is free. There is no middle man, no vendor.
I never understood the popularity of Tailscale, though that is on me. I'm sure it is a great product, I just never tried it, do not seem the target audience.
What confuses me is the often accompanying, sometimes aggressive anti-selfhosting stance in these sorts of threads. I do not see this in other topics, e.g. someone mentioning they run Jellyfin isn't met with "why not Plex?". Where does that come from? We are on HackerNews, not ProductShillNews, aren't we? I guess self hosting Wireguard is too boring to warrant any further discussion? The VPN equivalent of a Toyota Corolla.
My WireGuard uses (either at home or at work) are very much mobile client to single network
Where Tailscale comes into its own is automatic managing of mesh networking (like an “sdwan” solution). The other thing it excels at is firewall busting - if you have a firewall (with or without address translation) which only allows outgoing traffic to be established (with UDP timeouts for session) then Tailscale also works in a similar way to turn/stun.
If I needed that capability then I’d be looking at Headscale. I don’t need it though.
Remember that this is hackernews, not slashdot. Where the community used to be far smaller and the technology far smaller it was quite normal for everyone to understand basic building blocks of ip addresses, use open source software, wear t-shirts threatening to replace people with a small shell script etc.
It’s not the same community, many people here have no real understanding of computer fundamentals, but instead have expertise in specific narrow areas. They also have little interest in things like free software, but do have an interest in building a new billion dollar company to sell to a behemoth.
Some would consider that an anti-feature. Firewalls are not to be busted. Nothing good lies at the extreme end of working around overly strict policies. Change the policy instead.
I think Tailscale is popular because of how plug and play it is for most people. Although the main reason I use it over self hosting wireguard is the NAT busting it does, which has so far worked flawlessly for me with no setup aside from installing on both devices. There is nothing wrong with self hosting wireguard, but it doesn't actually do the same job as tailscale.
Wireguard by itself also doesn't allow for 2FA or expiring keys. Not as relevant for private use, but some orgs need it for compliance. The idea was always that things like that need to be implemented by an application on top of it, so you end up with something like tailscale eventually.
i have my homelab only reachable via tailscale and can access everything i would ever want on the go that way. it was a matter of 15 min to get it all working.
> I never understood the popularity of Tailscale, though that is on me.
> I guess self hosting Wireguard is too boring to warrant any further discussion?
It's popular because you don't have to deal with NAT punching. It "just works", all the time. And Wireguard is not too boring, it's just not enough on its own.
I'm all for self-hosting and this is exactly why I prefer to use Tailscale and not have to manage jump-hosts and STUN points on some cloud, given that I won't be able to make it as reliable as Tailscale and as cheap as Tailscale (effectively $0). So this is literally the only tradeoff I made while self-hosting everything else.
> I never understood the popularity of Tailscale, though that is on me. I'm sure it is a great product, I just never tried it, do not seem the target audience.
Can you talk an $elderly_relative through a wireguard installation on the phone so they can join your VPN?
it wasn't meant as a rebuttal, I'm genuinely asking. tailscale + headscale was just recommended to me, hence that's what I'm using for self hosting. is wireguard's client roughly equivalent to tailscale's? especially tailscale's always-on nature is very appealing.
Yeah. I don't think PG could even come close. Column-oriented is fundamentally different, and pairs well with all the SIMD acceleration ClickHouse is also doing. There's just no comparison. If a Postgres rewrite came close to that, it must've sacrificed something else.
But do the markets care about a Postgres in Rust? Probably not, or at least not right away. It is a long way towards commercial success.
> I suspect rather than hire less people we will just produce more code changes.
Why? Towards what end? Code changes are output, not outcome. It also needs to be connected to someone willing to pay you hard cash. That is the hard part, a race to the bottom, and the reason I also believe there will be downwards pressure on salaries and even employment.
> People still learn math, despite the calculator existing. Accounts still learn accounting, despite Excel and accounting software existing.
They do, but you need far fewer or none of the original workers whose full-time job this sort of stuff was.
Raw math does not matter, but what you do with it. Similarly, you could earn a (modest) living knowing nothing but raw HTML, JavaScript and a bit of browser tech not too long ago. That is no longer possible.
Programming and software engineering will be devalued. These occupations won't disappear overnight, but you will see compensation and growth stagnate until equilibrium is reached again. Currently, supply outstrips demand, and I do think it is structural, not just hype.
I'm certainly not creative enough, but I currently do not see demand picking up sufficiently; Gen Z is bearish on social media, VR was a bust, blockchain was a bust, software has already penetrated almost all walks of live and lines of work. There is no next big thing (Internet, ...) on the horizon, to unlock the next order of magnitude of demand. There is certainly more work to do still, but it very suddenly does not require the same headcount, but something like 5%-30% less. Lots of the remaining work will be around integrating LLMs into existing software, which does not sound exciting either.
This is the mess a language lands on when it conflates optionality (a semantic concept) with references/pointers (purely a machine concept). In Go, the requirement "need (non-optional) a reference to an object" is simply not expressible. This is a solved problem in other languages, for example `&T` vs. `Option<&T>` in Rust.
It's really difficult to view Go as a serious language when fundamental design decisions such as this one have seemingly been glossed over. It's in a precarious spot, on the one hand cushioning the C it wants to resemble, but on the other hand not yielding any capable tools or abstractions which could otherwise be unlocked via the safe architecture. Go developers seem uninterested in language design.
It's not that it has been glossed over, or was a mistake. It's a tradeoff in favor of simplicity (and compiler / tooling speed).
It is difficult to view Go as a serious language because it fails to acknowledge these decisions, repeatedly. You can't really trust the language in that sense.
This is the most boring argument in computer science. It's like arguing about whether a language should have "goto" or not. There is no new ground to tread here. Most mainstream languages have null references. An entire cinematic universe of languages have been built from the premise that you should not have null references. This is a fundamental rift in programming language theory, and the very best you can do on HN, at least on stories where that rift is not the main point of the article, is to restate it poorly.
Seriously, Tony Hoare dropped the mic on these arguments back in 1965. You have to move forward in these discussions on the premise that everybody already gets this very basic, very old PLT argument.
Just like if you had somehow managed to find a way to do a spaces versus tabs complaint in a story about (I don't know) Typescript, you will reliably generate sprawling threads by bringing this stuff up on any thread about a language with null references. It's easy for everybody to have an opinion here! Everybody knows the issue! Not everybody agrees! But you aren't doing any good for the thread itself; you're just jamming it.
I think you’re arguing against a point that GP didn’t make. Optionality and empty/uninitialized references can both be encoded in a type system, or one, or the other.
I didn’t interpret GP as arguing for or against null or otherwise rehashing what you correctly identify as one of the oldest intractable arguments in programming. The sibling comments not so much.
Yes, my point was not related to null. For all I care you can have `&T` and `Option<&T>` in your language, but allow `&T` to be null. In Rust, that would be `Option<*const T>`. Is that useful? I don't know. But it still separates the two orthogonal concepts. Go conflates them, rolling them into one, permanently removing useful expressivity.
No, nobody agrees about these things, which is why this is the most boring argument in computer science. You can't even get people to agree on typing. Not "which type system", any of it.
If you think your current position on these debates --- nullability, gotos, typing --- is the obviously correct position, you simply haven't talked to enough people. The answer to all these questions --- the real answer --- is "it's more complicated than you think".
We didnt't agree to that, only thet thr over usage of goto hurts readability, but it is perfectly fine where appropriate (as a JMP analog to non-assembly[0]). Any language that supports loop naming implements a subset of goto, and proves why it's sometimes necessary.
> Seriously, Tony Hoare dropped the mic on these arguments back in 1965. You have to move forward in these discussions on the premise that everybody already gets this very basic, very old PLT argument.
I like using Go for many reasons, but exactly this one is making me sad every time.
I can’t accept interface and be sure it’s non-nil at the same time.
I think this is a flaw and it’s just a shame.
There's nothing particularly special about null pointers: you can also have an invalid non-null pointer, e.g. through pointer arithmetic.
When you write `int& ref = *ptr;` you are dereferencing the pointer with `*ptr` therefore you have promised that it's valid. The compiler doesn't need to do anything to validate ptr because it already has your assurance.
It's really no different than if you were to write `printf("%d", *ptr);`. It's only a little weird because in `ref = *ptr;` the compiler doesn't actually emit any instruction for the dereference, but that doesn't mean that the assurance you gave doesn't exist.
It would indeed be problematic were it to be `int& ref = ptr;` without the dereference but it's not.
> When you write `int& ref = ptr;` you are dereferencing the pointer with `ptr` therefore you have promised that it's valid.
Yes, that's exactly the problem. "I promise I didn't make a mistake when reasoning about this code" is tied for worst strategy in the world for preventing bugs, along with every other strategy that doesn't actually prevent bugs.
The danger and utility of pointer are two sides of the same coin. You can reduce the need for pointers but not eliminate them completely if you want to be able to call C functions or build low-level data structures.
The real problem is that references are not good enough in C++, so some C++ developers end up using pointers for everything. Rust's references are good enough that you can avoid using pointers most of the time.
Im not entirely sure this helps with your point but;
The contract is that the reference is still non-null, and that the error is dereferencing the pointer. There’s two big problems with defining the behaviour of the deterrence - 0 is a valid memory address on some (ancient) platforms so for better or worse the behaviour is platform dependent.
The other is that there’s many other ways to have absolute garbage in a pointer that aren’t null.
int& foo() {
int local = 42;
return local;
}
Now, a compiler catches this case, but the point is that null isn’t the only invalid state that needs to be checked. Adding a compiler overhead of checking each pointer to every single pointer dereference wouldn’t work.
Modern codebases ran with static analysis tools will catch these errors (honestly even valgrind will find most if not all of these).
The philosophy of C++ is to not introduce unnecessary overhead, and to trust the programmer. This design choice is prevalent throughout the language. They were never going to make an exception, especially for something as prevalently used as references.
There are countless examples of this "no unnecessary overhead and/or trust the programmer" choice:
- primitive types and standard containers are not thread safe - it's up to the programmer to know this and use them accordingly.
- std::unique_ptr lets you grab the underlying raw pointer, in which case it's no longer a "unique_ptr". But there are cases in which it's useful to do this (e.g. interfacing with C code), so they let you do it, and trust that you do it in a safe way. They could have made unique_ptr not support this, but then it would be less useful (or force you into copying data unnecessarily to call an API that requires a raw pointer).
> But there's no enforcement.
There's no strict enforcement, but it is undefined behaviour, so compilers can randomly choose to act as if it's enforced and simply crash your program or make it act weirdly.
> primitive types and standard containers are not thread safe - it's up to the programmer to know this and use them accordingly.
Which (sort of) makes sense: most types should not be used across threads. Having everything use atomics/mutexes under the hood would have significant overhead. However, the problem is that the language doesn't then protect you against using these across threads by mistake, this is one of the things that I really like about Rust.
Funnily enough, shared_ptr in C++ is thread safe (for the reference count at least), leading to pointless overhead when not used between threads. Rust has both thread safe and non-thread safe versions (Arc and Rc respectively), and it will error if you try to send an Rc to another thread.
For a typical type Goose it's fine if two threads can both look at the Goose (via a reference, pointer or whatever), so long as nobody can mutate the Goose. If thread A finds that the Goose is Happy, thread B weighs the Goose and finds it to be Heavy, and thread A again measures the length of the Goose as 860 millimetres this is all fine, it won't matter if (by the vagaries of hardware) the weight is measured before the length or after, there's no difference.
In Rust this is reflected in &Goose, the immutable reference to a Goose, being Send, ie a thing you can give to other threads. The mutable reference &mut Goose is not Send.
> The philosophy of C++ is to not introduce unnecessary overhead, and to trust the programmer.
The first part of that is reasonable. The second part is just naive; nobody writes bug-free code, so if your strategy for not having memory corruption is "just don't screw it up", you're going to get quite a lot of memory corruption, and that's how we ended up where we are today.
Not really. It's possible to write this mistake but it's pretty obviously a bad idea, I've never seen someone do this and need correcting.
Edited to expand: Sometimes it feels reasonable to have a construction function which returns Option<Goose> rather than Goose because you might be OK with getting back None, for example if you want to make a NonZeroU8 the function to do that will of course give you back Option<NonZeroU8> because you might give it a zero and that's er... not nonzero. But I've never seen people go oh, OK, I guess i'll scatter all my checks throughout the rest of my software and just pass Option<NonZeroU8> everywhere even though I need a NonZeroU8. Rust's shape encourages them to check once during creation like this article suggests.
true, "non-nil pointers"/references will help here to avoid nil checks.
also true, if you have optional you still need to unpack it somwhere, and your nil checks become unpacking statements. delayed conditionals and delegation to callsites far from offending code (what author says) is still present.
and if you also have pointers, then you can do Optional<Pointer>.. and now you have to option unpakcing + nil checks. 2x more problems.
If you have an actual pointer type *mut P then Option<*mut P> might be None or it might be Some(null_pointer) or Some(other_pointer) that's not 2x more problems it's just a representation of a more complicated scenario - we may or may not have a pointer and, if we do have a pointer that might be null. We'd presumably have done this because we need to distinguish those cases.
If you actually mean Option<NonNull<P>> you should write that, now we're saying this is either a non-null pointer or it's nothing. Often though you want Option<&P> either a reference or nothing, or you actually did mean a raw pointer *mut P and you're going to handle scenarios where it is null or whatever.
> with rust you have 2x more ways to shoot yourself in the foot.
The checking isn't how you shoot yourself in the foot, it's the absence of checking. Rust doesn't allow you to forget to check. This entire class of problems just disappears in Rust.
In this if the code needs a non-null redis client to work you take `RedisClient` not `Option<RedisClient>`.
yes that is correct. tbh in Go for service structs that what you would do as well. use value receivers for such things. and inject dependencies as interfaces. so pointers not immediately visible and it is just type RedisClient interface in your field/arg.
No, because RateLimiter is then copied on passing it around (pass by value).
That is problematic for two reasons: it might be a large type, so copying might be expensive. Second, more likely, it might violate invariants in your domain. For a rate limiter, this might mean accidentally copying around some internal state like a mutex, which then exists n times instead of 1 time, which can represent a problem (e.g. if you want to internally limit whole-app concurrency toward Redis).
As a fan and believer of obscurity in support of security, I do not understand why
> that step didn't add any security.
It is a decision that’s part of the entire process. A branch of many in the decision tree. Other branches are deciding which characters to type for the password; ASCII characters can be as little as 1 bit apart. Deciding between left and right is also 1 bit apart.
I think it boils down to what people commonly understand to be publicly knowable information versus understood-to-be-secret information.
One example: I self-host my password manager at pw.example.com/some-secret-path/. That extra path adds as much to security as a randomly picked username in HTTP Basic Auth: arguably none. Yet, it is as impossible for attackers to enumerate and find that path as it is with passwords.
The difference is that the path leaks easier. It’s not generally understood to be a secret. Yet I argue it helps security. (Example: leaking the domain name through certificate transparency logs AND even, say, user credentials means an attack is still unsuccessful; a strictly necessary piece of the puzzle is missing).
I built a system once which had (this is Rust) a Rayon worker thread pool of 4 threads and a Tokio async pool of 2 (multithreaded runtime). On a system of 6 vCPU. This ended up working fine. Tokio was not starved so handled network requests at low latency.
One difference is DuckDB is a pure network client. If one of its async threads is starved it is not the end of the world (e.g. k8s does not kill your pod for failure of replying to health checks).