Right, because it scratches some nostalgia thing that people have obtained osmotically. Nobody actually wants a replaceable battery that makes their phone vastly thicker and harder to waterproof, nobody wants phones that randomly catch fire because their OOPLMXC brand battery from amazon wasn’t up to snuff, nobody wants to be carrying a pocket full of extra batteries to swap all day. Devices with field replaceable batteries just sucked a lot more than people don’t remember. As much as people clamor online about how they want these things, given the option they will shy away from it because it doesn’t match the fantasy that’s been created about “what has been taken from us”.
Yep, on my aging phone it takes quite w noticable amount of time to go through. But on my more recent laptop wnd workstations, it's a quick flash of a manga lolita.
You could just spend 30 seconds googling instead of confidently asserting nonsense. There are plenty of people demonstrating that Anubis reduces bot traffic.
That’s because its weird and mostly irrelevant, it’s nothing to do with the proof of work being any sort of economic impediment. It takes Claude a few minutes to write a near instant solver, that’s the maximum proof of work cost this adds.
This strain of nihilism about anti-bot efforts baffles me. Cloudflare built a gigantic business on stopping DDoS attacks. There will never be good faith self-restraint by AI bros, there is too much money at stake and no regulation, doing nothing means your only alternative is disconnect from the Internet. Anubis introduces enough friction that some percentage of the lunacy is throttled and possibly give your server a fighting chance.
>There are plenty of people demonstrating that Anubis reduces bot traffic.
"Anubis reduces bot traffic" and "[the proof of work] is not stopping anyone" isn't contradictory, especially if anubis isn't being benchmarked against non-POW alternatives like IP/ASN bans, or tls fingerprinting.
IP blocks don't work, because they're using proxy networks so that you see an ip address 1 or 2 times within 10 minutes. They have effectively infinite ip addresses. (actually, looking at my data from today, I think this relationship holds over ~3 hours, where we're seeing ip address cardinality at about 1/2 of the hits.)
* Sometimes there's a pattern to the country. Oftentimes, not.
* User-agent, rotated between common, valid, current web browsers.
* Other headers, sec-*, accept, etc, generally valid and rotating.
* Bots will load the site to saturation in a denial of wallet attack.
The only thing that's specific is:
* urls have a pattern.
* it's obviously invalid traffic.
(non-bot traffic on my sites does not go from 0 to 200r/sec on the search interface in seconds. It does not go away that fast either)
Most devices never test their memory and don’t have ecc, so you generally never know. Random instability can be fixed by restarting and your program not being in an area with bad bits anymore.
People tried this in 2013 or so, there's no point to it. Doing proof of work in javascript in a browser is so crushingly, pointlessly slow that there's no value at all. Some browsers also intentionally detect attempts to do proof of work and attempt to block it entirely.
To help you combat these unwelcome scripts, we’ve partnered with Disconnect and compiled lists of domains that serve cryptomining scripts so you can block them directly through Firefox. When you download Firefox, tracking protection is enabled by default, blocking known cryptominers.
Technically true, but the list is mostly nonfunctional - cryptominers are basically extinct in the wild, and most of the domains on the list have expired or no longer serve mining scripts.
Which is trivially bypassed by an actual implementation of the proof of work in non-javascript, rendering it absolutely useless. The website is approximately 3800x times slower than native code, and hundreds of thousands of times slower than the CUDA kernel claude wrote. The "proof of work" is just non existent at that point, they're solved in milliseconds for what would take the browser version 10 minutes or more, it's security by obscurity being dressed up as something more.
pow_server http://127.0.0.1:8080 backend avx512-x16
──────────────────────────────────────────────────────────
uptime 00:03:12
solver ● BUSY difficulty 9, 0.3s
queue [####################............] 5/8 peak 12
──────────────────────────────────────────────────────────
accepted 1240 solved 1180
503 shed 48 504 timeout 2 4xx/5xx 10
──────────────────────────────────────────────────────────
last difficulty 5 nonce 645376 in 9 ms (101.6MH/s, avx512-x16)
hashes 3.90GH total avg 65.3MH/s
Ctrl-C to stop
Claude even made a nice little API server for it after implementing midstate compression, AVX multi way hashing, and a CUDA kernel. This doesn't stop the literal LLM it's trying to block from solving the challenges, it's really annoying that everybody is using it and claiming that it's something that's usable in the real world as a result of it using proof of work. It's obscure, and obscure is fine so long as nobody is pretending that it is secure.
It is not absolutely useless, empirically, which you'd discover if you had a website getting hammered by bots and experimented with anubis as a countermeasure.
While dedicated scrapers/attackers could work around it, and they could do so much more efficiently than the client-side js, almost none of them do. Unless you like paying additional hosting resource fees to serve bots, it's a worthwhile option, and is less annoying to typical human visitors than cloudflare's interactive captcha/challenge which is what most people use.
The main author is aware that the algorithm is far from ideal for this purpose. See https://news.ycombinator.com/item?id=48869064 . If more bots start to answer the primitive challenge anubis uses now, that'll hasten implementation of a different algorithm.
Don't let the perfect be the enemy of the good enough. For now, the algorithm or challenge scheme almost doesn't matter. Since it's much smaller-scale than cloudflare's challenges, that's probably why very few scrapers and botnets bother to solve anubis's trivial sha2 pow.
Targeted attacks may not be repelled at all. That's not the point.
It does not even require the PoW thing Anubis does. I've setup a simple logic that just:
Checks for existence of a specific static cookie, if it does not exist, output a small page that sets the cookie via JS and reloads. Sadly this kills Noscript, but it would be possible to add a form in <noscript> that when submitted sets the cookie serverside.
Is this trivial to bypass? Yes. It still keeps out 95% of unwanted bots.
Reality is most do not target you specifically they just want to mass-scrape with low effort. Running headless browsers is way more expensive for their op
I've extended this with a FCRDNS checked exclusion for Googlebot.
Another quite effective measure I figured out was checking the existence of Sec-Fetch-Dest header if the User-Agent claims to be a modern browser. If you don't want to close down too much.
Also, I only apply these rules to routes that are not cheap and cached.
That's not far from what anubis does for clients that are determined to have light souls. It doesn't always send a PoW challenge.
For a webapp that sets a long-lived cookie, that cookie could be used to bypass anubis completely, or lower the weight in anubis so that it doesn't send its pow challenge unless there are major red flags. If bots start to abuse that exception, it can be removed.
I once discover you can put escaped XML or json in css content. The purpose was to have static data sets that work cross domain. No headers to configure no letting strangers run all you can eat malicious js on your site.
Define "scale". For any reasonable wait that you're willing to impose on your users, any PoW scheme heavily favors attackers. They have unlimited time and can be scraping even while they're asleep. Your visitors on the other hand don't have that luxury. You might argue that's not the point and it's only to stop dumb scrapers that are effectively ddosing your site, but if it's just dumb scrapers, you could've stopped them less onerous measures like tls or javascript fingerprinting.
It's more or less designed for it, it's SHA256 with a break in the middle for midstate compression to be effective, and the difficulty system is based on a misunderstanding of how bitcoin PoW works ("number of zeros" is never, ever a consideration in bitcoin, it's a match to a floating point target).
sha256(challenge + ascii(nonce)) means that the first compression round of the function can be cached and the second compression round is just the nonce plus the cache. This is the same trick used in Bitcoin mining and would have been avoidable by putting the nonce first, so immediately any non-naive code has to do half the proof of work as the vanilla solver.
write a implementation of the anubis proof of work in native c code, optimized for speed above all else. use every trick available to make the proof of work as efficient and fast as possible, including modern processor tricks on the x86 platform. your code should avoid using external libraries where possible, include tests, and be readable and concise. a reference for what needs to be met is in this repository. https://github.com/TecharoHQ/anubis
Then
let’s develop this more. turn this solver into a local HTTP server that can be given work in the request, and it returns solved work. make an end to end tester that sends test work to the solver and waits for a valid response. add support for solving with a GPU using cuda.
Then it was done more or less, it happily made a local server that supports solving the challenges given to it in bulk with priority based queue and can tolerate potentially tens of thousands of requests a second with no issue. The CPU time spent solving the challenges is less than the SSL setup for the connections. The GPU version does in excess of 20GH/s (but with high latency) though I didn't really test it, I'm not using this for anything but proving a point that the LLM itself can write the bypass tools and run them happily.
I'm almost ready to ship the wasm feature in the next version of Anubis after the one that's about to come out.
The big blocker is that testing against dozens of googles chrome to ensure functionality on abandoned smart TV oses takes a nontrivial amount of time. As an example of the level of debugging and the like required: https://github.com/TecharoHQ/anubis/pull/1684/changes/67621f...
My office gets very warm when chromesweep runs.
This is something that is complicated enough that even though LLM tools can help, it's not a magic bullet. It's just complicated in general.
Claude Opus can make an optimized version of the proof of work solver that’s more than 10000x faster than the javascript one, in about 5 minutes time. Who is this stopping exactly? It’s the wrong tool in the wrong place.
Practically, the people indiscriminately scraping don't bother to do the work to bypass it or implement the POW test, which results in reduced CPU load for all the properties that were having trouble with scrapers before. Until scrapers start implementing it en masse then, it still serves its purpose.
If it becomes widespread (as it has been doing), they will. Anubis' strategy only works while it remains a niche approach only adopted by a small number of sites.
I think this is a pragmatist vs idealist debate. The pragmatic answer is that Anubis solves a problem now, and so it will be used until either it doesn't or a better solution presents itself. The idealist approach is that it's obvious that there's ways to get around Anubis, and so some people argue from there that it shouldn't be used. But the only other alternatives being offered are to either eat up the costs (in server resources or engineering time), or to go behind cloudflare with its own tradeoffs.
Every individual practitioner has a strong incentive to overuse it, because it's extremely effective for them individually. Every additional practitioner that uses it increases the selection pressure on their collective adversary to develop resistance. Eventually, it reaches a tipping point and becomes ineffective. But the ecosystem-level impact of its historical use remains, and makes things worse for everyone.
Recently, the baseline expectations of the Web have shifted, and I need to enable JavaScript to read a non-CloudFlare-proxied simple HTML site like LKML. I do not expect this shift will revert once Anubis outlives its effectiveness.
Viruses do not have motivations, so it’s hard to assign blame to them. The companies running badly behaving AI scrapers are run by people with more of a mind than viruses, so please direct your complaints about the second order effects of their actions that way, rather than on their direct victims.
The barrier is a two sentence prompt to Claude to add a patch to curl that does 500MH/s to the 50kh/s the javascript version does on the same hardware, it only works because it’s obscure and largely irrelevant.
Which is why its more surprising this was first announced last year and there's no Proof of Concept demo yet?
But really, I've given up on ink jet printers, and have gone the cheap B&W laser route for anything I need to print at home (In the past year, 2 times, a backup ticket and some paperwork that needed a real signature sent back).
But when I had them, the thing that went bad 99.999% of the time was the cartridges or a clogged nozzle on the head. So the advantage here, on the repairability side not DRM, is the rails and motors?
Also that cutter is going to be a pain, having worked on Lightjet printers, that cutter was nearly all field service issues until the FEs started leaving the "laser" key so lab managers could reset the blade themselves.
1 petabyte per 60 second scans implies a kind of comical data rate to storage, even at RAM speeds that’s implausible. Imagine we need to write these to hard drives, they happily sustain 150Mb/s on the high end, which would imply you’d need 115,000 hard drives to absorb that amount of writes. Even with top end NVMe drives you’d need a thousand of them writing simultaneously.
That's likely the datarate of the ADC chips. You would downsample them directly on the FPGA board and maybe perform an FFT or similar transform. 16 TB/s across a few dozen FPGA boards is nothing crazy. After some early stages in the signal processing you might transfer 1 or 2 TB/s over ethernet to the servers. Entirely feasible considering we have 800 gigabit/s ethernet.
You’re completely right, this is why currently ultrasound reconstruction happens on FPGAs. They would need a lot of them given the number of transducers.
https://pmc.ncbi.nlm.nih.gov/articles/PMC6057541/
reply