(disclaimer: i wrote that post)

It is not. We rely on more than User Agents because they are too often faked, so it is not just marketing. There are other signals we see that confirm whether the request came from a "legitimate" AI scraper, or a different scraper with the same user agent.

> There are other signals we see that confirm whether the request came from a "legitimate" AI scraper, or a different scraper with the same user agent.

Great! What are these signals? That seems to be the meat of the post but it's conspicuously absent. How are we supposed to validate the post?

> how are we supposed to validate the post?

Imagine you were a vendor who were trying to trick the author into divulging his methods. Can a stranger on the Internet be trusted?

I imagine if that information is disclosed, you won’t be able to verify it, as it will be bypassed… because it was disclosed.

What a wonderful world we live in where serious people are expected to believe press releases based purely on brand prestige.

There’s a lot of assumptions in that comment.

So user-agent and whois to see if it's coming from a plausible netblock and accept: strings, http header stuff?

More than user-agent, because user-agent cannot be trusted.

Great! Well then... how?

HAProxy Edge is their product, and akin to Cloudflare and other competitors the heuristics to stifle bad actors is likely the secret sauce. Disclosing it would only lend bad actors the advantage in their game of cat and mouse.

Hey, I work at HAProxy. Want to reach out to me in a DM on https://www.linkedin.com/in/jakubsuchyus/ and I'll see if I can connect you with our Edge team to see what we can do about your Asia -> NA needs?

Jetbrains is Czech and most Czech people would get offended by your characterization of them as Russian.

There’s quite enough companies and real estate in Czechia owned by Russians, despite even said Russians hastily changing their citizenship in 2022.

It’s sometimes beneficial to pose as an EU-based business if a purely Russian business was either sanctioned or considered too risky/dirty/shady to deal with.

So while Czechs don’t like to be equated with Russians, not all of them would quite sing „běž domů, Ivane” or share the feeling.

Sales, yes. Development, has been in Saint Petersburg not so long ago.

Reverse HTTP is fascinating!