Great if you have fiber and live down the road from the datacenter.

I routinely use it over a 5G mobile connection, often while moving, far from any datacenter, and it's usually fine as long as my signal is good. They do a fantastic job out of squeezing what they can from your connection.

I'm actually some 200 miles from the data center, but Nvidia has been doing a stellar job in connecting to the backbones of all the major ISPs around. If I actually lived down the road from them, it would be as low as 1 to 2ms, as reported by plenty of users. This isn't uncommon, especially in the EU.

But yeah, you need a fiber.

We aren't talking about insect identification models from 2019.

What do you think are running on the T4 GPUs in AWS? A lot of the use cases I know of for them are mid-level computer vision models that don't need to be frontier level.

I can no longer edit this, but want to expand on my comment.

I've seen those vision researchers want to train on H100s at the time and being told know, wait for the T4s.

I've seen T4s running BERT models for document classification.

When there are enough Blackwells in data centers that H100s are useless for inference by your standards (I don't know if we've arrived there or not yet), there will be people who, say, want to run the Taco Bell ordering chatbot on them. There will be people who have applications that are just fine with Qwen 2.5 who will be happy renting them.

There seems to be this crazy consensus that hyperscalers are going to go into their datacenters and throw away their old GPUs. The reality is they have a ton of paying customers for them.

And there may be insect identification apps from 2019 that say "you know what? H100s have gotten cheap enough I can use a VLLM so the user can describe where they saw the insect too", or the McDonald's website support chatbot developers say "Hey, the bigger cheapers have gotten cheap enough we can upgrade our models to Qwen 2.5".

The frontier level GPUs in e.g. AWS have a huge premium. When the newer generations come out, they will be able to cut prices to a bit of a premium over the operational costs and still make a profit, and there are a ton of down-market customers who will be interested, who aren't willing to try to outbid Anthropic for Blackwells.

> I don't think I average even 2 captchas a day being terminally online, so 10 across every soul in the world sounds way too much for me. (we're ignoring bots it's meant to deter?)

You're mixing up checks, fingerprinting, and PoW with a captcha being triggered because those didn't pass. The less abnormal your setup is, the fewer captchas you'll get.

I agree with the rest of what you said.

Also I think you mean "scraper" and not "scrapper".

N=1 P=0.5

Replacing parts of PE firms with AI is so far from being a useful suggestion here.

I don't think gaming on Linux is comparable here.

Libraries that automatically throw errors for status codes in the 400 and 500 ranges are pretty obnoxious (looking at you, axios). It adds unnecessary overhead, complexity, and bad ergonomics by hijacking control flow from the app.

Responses with status codes in the 400 range are client errors, so the client shouldn't retry the same request. So a 404 is appropriate despite how annoying a library might be at handling it. Depending on which language/ecosystem you are using, there are likely more sane alternatives.

Completely agree on the axios part - one implication of that is you can't statically type the error response shapes (since exceptions can't be typed). Where as with fetch you can have a discriminated union based on the status code (eg: https://github.com/mnahkies/openapi-code-generator/blob/main...)

Although I do feel like I've seen too many instances of a 404 being used for an empty collection where it would make more sense to return `[]` and treat it as an expected (successful) state.

Generally true although 429 is often used for rate limiting so a back off and retry is appropriate. 409, 412, 428 may also be retriable depending on the specific semantics of the given situation. 421 apparently shows up commonly in HTTP/2 connection reuse and is retriable. 423 and 425 too potentially.

It would have been nice if there was an actually grouping of retriable and not retriable but in reality it’s a complete mess.

But at a minimum beware of 429. That’s not a permanent outage and is a frequent one you might get that needs a careful retry.

Yep, the Anthropic acquisition, this petulant Rust rewrite, and bun's increasingly buggy releases (slop) have caused me to migrate my projects (personal and work) to nodejs+pnpm.

The risks of using bun are no longer just those concerns around a newer tech and "drop-in" replacement for nodejs. Now you have to marry Anthropic, Rust, and a founder with conflicting priorities.

Having read the comments from the actual engineer doing this rewrite, the only petulance I have seen is from those reacting so strongly to it.

just wait a year or two.

How exactly will waiting a year or two make this effort appear “characterized by impatience and grumpy annoyance”, as opposed to the people right now who are loudly bemoaning an engineer trying something out as an experiment?

Hey Grok is pretty good for meme videos and pics. For anything serious, not so much.

That is.. inaccurate.

How so? I’m not saying most of work doesn’t go into creating the drafting model or enabling a new head on the primary model, but the point is that however cool it is the result is, more weights. Speculative decoding requires code to be aware of how this works at the inference level.