I hope the test data is the same when comparing the different runs. So the big o notation should be the same across different runs.

I find that thinking/agent mode sometimes makes it worse/comes up with the same thing and just takes a long time. But I’m sure it’ll be different with fable for a few months until that hype blows over

Something a lot of folks struggling with these systems don't get is that the instruction and management of them is often quite important - just because they're capable doesn't mean they're mind readers.

Most of the skepticism I encounter on this front is due to lack of proper direction, process involving planning and review before execution, and appropriate attention given to evaluation and feedback loops.

If you asked the smartest person in the world to YOLO a task with the sort of instruction the average denier uses to evaluate an LLM, you'd likely find they wouldn't get back what they were expecting either - and if you're evaluating on subpar models/tools, you shouldn't be surprised to get subpar results.

I asked Qwen 3.7 pro to create a C# project that takes a string and reverses it, with a single file WASM target. It spun wheels for over 30 minutes and got nothing.

I use LLMs all the time to help me diagnose bugs and work through my designs, but again and again, I am super unimpressed by their coding abilities. I can see how in some cases with a proper harness they probably do a decent job at certain tasks, but almost everything I try to do, they flail.

Qwen is a lightweight locally hosted model that's many months behind the SoTA available from the big three - while the crowd here (myself included) is excited for locally hosted models to catch up to the usable baseline, regardless of what benchmarks you based that selection on they aren't there yet.

This seems to be a very generic/common response to any ai critique. It kind of reinforces my point there’s a lot of situations where the appropriate harness isn’t some agent that’s set to ultra high thinking mode. Chat mode gives the better response and answers the question more quickly

That's fair, I do agree that you don't need a harness or ultra-high thinking mode for many problems. Many folks evaluate without those things on a task that would benefit from them leading to the sort of attitudes in this article and its comments section, which is where my comment was coming from.

If you're just saying different tools are best suited for different problems, apologies - that's my take as well.

That’s quite a bit slower to process. At least if you’re converting to integers to do the calculations and the calculations would be quite a bit slower if you kept the big decimal type

True, but this is usually your least concern when you're dealing with monetary amounts/math.

I guess I’m coming at it from an optimizing market data provider perspective once you preallocate memory the next thing to optimize is the string decimal conversion if the feed isnt binary encoded

It might be your least concern, and that's fine but it's not the least concern for many people who need to process large volumes of transactions.

Money, even within fintech, is a concept used across a wide variety of domains, and you can't assume that what concerns you is what concerns everyone else relying on it elsewhere.

They specifically mentioned HFT so I suspect they care a lot about processing speed

There’s quite a few historical cases of mergers acquisitions being poorly valued aol time warner for example. So maybe v the calculus is different from the execs perspective but it’s not necessarily any better.

I wouldn’t call comparing yourself to Fabrice Ballard and not just saying he’s a better programmer modest.

Yeah he phrases it odd. Like with the "almost certainly" and "overall" qualifiers. Not "he is a better programmer than I would dream to be..."

I don’t know like chess engines didn’t kill chess. You could just play with people that don’t use the “engine”

It’s different, unfortunately. I wish you were right. The problem is that creating interesting and fun CTF challenges is a very active, time consuming, creativity-heavy task. A chess board is always the same, and always will be, but every CTF competition is unique. There is little to no incentive anymore to spend time creating the challenges. You might say “well create the challenges and share them with people who care and who want to play honestly” which is probably the right answer here, and might happen at a smaller scale. I picture CTF in the future almost like a tabletop RPG experience, one where a small amount of people will share with close friends who they trust. But the usual “open” CTF scene (as mentioned by op) is probably over for good, if we’re being honest.

Yea, but chess adapted to it and is restricting use of engines. When you play a tournament you are banned from using a phone and will be disqualified if you do so. Online tournaments don't have a prize money for that reason, so there is no real benefit for cheating. Lichess and chess.com additionally add rankings for bots and have a strict anticheat if you use bots for regular games.

For me it feels like this is not really possible for live CTFs. In contrast to chess you can't ban AI, as live CTFs are about breaking things by design, so they'll always try to circumvent an AI ban.

> I don’t know like chess engines didn’t kill chess. You could just play with people that don’t use the “engine”

Impossible to do online, at least. People go beyond things for virtual fame and "owning" others. Even if it is based on cheating.

Not if it drives up energy prices and makes other businesses that employ more people less competitive. Not saying that is the case but it’s certainly not a given

Some of the utilization comparisons are interesting, but the article says 2 trillion was spent on laying fiber but that seems suspicious.

There's an enormous amount of unused, abandoned fiber. All sorts of fiber was run to last mile locations, across most cities in the US, and a shocking amount effectively got abandoned in the frenzy of mergers and acquisitions. 2 trillion seems like a reasonable estimate.

Giant telecoms bought big regional telecoms which came about from local telecoms merging and acquiring other local telecoms. A whole bunch of them were construction companies that rode the wave, put in resources to run dark fiber all over the place. Local energy companies and the like sometimes participated.

There were no standard ways of documenting runs, and it was beneficial to keep things relatively secret, since if you could provide fiber capabilities in a key region, but your competition was rolling out DSL and investing lots of money, you could pounce and make them waste resources, and so on. This led to enormous waste and fraud, and we're now on the outer edge of usability for most of the fiber that was laid - 29-30 years after it was run, most of it will never be used, or ever have been used.

The 90s and early 2000's were nuts.

For infrastructure, central planning and state-run systems make a lot of sense - this after all is how the USA's interstate highway system was built. The important caveat is that system components and necessary tools should be provided by the competitive private sector through transparent bidding processes - eg, you don't have state-run factories for making switches, fiber cable, road graders, steel rebar, etc. There are all kinds of debatable issues, eg should system maintenance be contracted out to specialized providers, or kept in-house, etc.

I so desperately wish it weren't abandoned. I hate that it's almost 2026 and I still can't get a fiber connection to my apartment in a dense part of San Diego. I've moved several times throughout the years and it has never been an option despite the fact that it always seems to be "in the neighborhood".

That has nothing to do with fiber, it’s all about politics and a regulatory environment where nobody is incented to act. Basically, the states can’t fully regulate internet and the Federal government only wants to fund buildouts on a pork barrel basis. Most recently rural.

At the local level, there is generally a cable provider with existing rights of way. To get a fiber provider, there’s 4 possible outcomes: universal service with subsidy (funded by direct subsidy), cherry-picked service (they install where convenient), universal service (capitalized by the telco) and “fuck you”, where they refuse to operate. (ie. Verizon in urban areas)

The private capitalized card was played out by cable operators in the 80s (they were innovators then, and AT&T was just broken up and in chaos). They have franchise agreements whose exclusivity was used as loan collateral.

Forget about San Diego, there are neighborhoods in Manhattan with the highest population density in the country where Verizon claims it’s unprofitable to operate.

I served on a city commission where the mayor and county were very interested in getting our city wired, especially as legacy telco services are on the way out and cable costs are escalating and will accelerate as the merger agreement that formed Spectrum expires. The idea was to capitalize last mile with public funds and create an authority that operated both the urban network and the rural broadband in the county funded by the Federal legislation. With the capital raised with grants and low cost bonding (public authority bonds are cheap and backed by revenue and other assets), it would raise a moderate amount of income in <10 years.

We had the ability to get the financing in place, but we would have needed legislation passed to get access to rights of way. Utilities have lots of ancient rights and laws that make disruption difficult. The politicians behind it turned over before that could be changed.

The worst part is it'd probably cost less than $100 of fiber and labor to splice something into your building, maybe $200-400 of gear to light it up, and you'd have a 10 gbps pipe back to some colo. It's more economical to run new fiber in most places these days, even if the local ISP knows exactly where all the old abandoned legacy lines are run, because of subsidization and basically scamming. The big companies like Lumen keep their knowledge regionally compartmented, legally shielded, and deliberately obfuscated, because if it became known that existing fiber was already run to a place they claim they can't serve, they can't get access to yet more funding for their eternal "service for the underserved" government money grift.

I stumbled on old maps that showed a complete coverage of fiber in my municipality, paperwork from a company that was acquired, and which in turn merged, then was bought out by one of the big 5 ISPs. When local officials requested information regarding existing fiber, this ISP refused and said any such information was proprietary. They later bid on and won contracts to run new fiber (parallel to existing lines which they owned, which still had more than a decade of service life left in them at that point).

I estimated that only around 10-15% of the funding went toward actual labor and materials, the remainder was pure profit. The local government considered it a major victory, money well spent.

The GDP 1995-2000 (inclusive) was about $52T. So that assertion would mean that about %3.8 of the US' economic activity was laying fiber. That seems like a lot, but in my ignorance doesn't sound totally impossible.

> I would guess that at least in the cloud flare instance some of the responsible code was ai generated

Your whole point isn't supported by anything but ... a guess?

If given the chance to work with an AI who hallucinates sometimes or a human who makes logical leaps like this

I think I know what I'd pick.

Seriously, just what even? "I can imagine a scenario where AI was involved, therefore I will treat my imagination as evidence."

Microsoft is saying they're generating 30% of their code now and there's clearly been a lot of stability issues with Windows 11 recently that they've publicly acknowledged. It's not hard to tell a story that involves layoffs, increased pressure to ship more code, AI tools, and software quality issues. You can make subtle jabs about your peers as much as you want but that isn't going to change public perception when you ship garbage.

The whole point is that the outages happened not that the ai code caused them. If ai is so useful/amazing then these outages should be less common not more. It’s obviously not rock solid evidence. Yeah ai could be useful and speed up or even improve a code base but there isn’t any evidence that that’s actually improving anything the only real studies point to imagined productivity improvements

good thing before “ai” when humans coded we had many decades of no outages… phew