Apparently it's 2/3 times faster than the Go version.
Whatever one think, the fact that there's 4 vibe coded compilers, 2 of those with 100% coverage of the official one and all being much faster than the Go version should be a wake up call for the TypeScript team.
Despite the resources and having a huge team of people that are clearly good at programming languages and compilers, something is definitely not working as it should there if vibecoders can get much faster fully compliant compilers.
"Vibe coded" and "supposed a lot faster than the competition" is becoming a smell to me. LLMs have a demonstrated capability to benchmarkmaxx that is even greater than human's already substantial capabilities to do so.
I'm not saying this isn't possible, I'm saying I'd wait a bit before I got too excited. That it seems to have held up to some other commenter's testing which was presumably not in the benchmark is an interesting sign, as a lot of time benchmarkmaxxed code falls over dead the instant you give it anything not in the benchmark. But I've still seen a whole bunch of projects "X times faster than what we told the AI to copy" that were less than met the eye to jump immediately to this must be awesome without more people trying it out seriously.
The vibe coded version probably work and are probably faster.
But probably not more maintainable or easy to write new features for.
But for stuff like that, it is usually not often a problem since features are most often first built as a spec and a test(suite) which by itself makes the thing easier to maintain with vibe coding. Since the implementation in itself is not import, only that it works like the spec.
But that works only for software that can have a well defined specification and a test(suite).
Basically it only works for things that are formally verified through the spec, which is not a lot of things at the moment.
I think a bit differently. The real problem with vibe coding is that it's hard to write a precise specification of the problem. But if the specification is accurate, running the AI dozens of times tends to make it much faster than a human (purely from experience). However, for most problems where vibe coding is actually useful, a human still has to set the initial framing.
It's apparently not fuel but pressurized air being refueled by the car itself.
> "Will it even be maintainable?"
Probably doesn't matter. I don't think it will be permitted on public roads anyway.
The presented variant is invite-only, so I guess it will be made to put in the garage of some (Saudi) investor who commits a billion dollar investment...
The advantage of AI coding is that it's non-deterministic. In fact, as you code, your style tends to harden, but when you repeatedly run AI code toward the same goal, you come to think that the non-determinism is actually better.
But can they even tell the difference? Honestly, setting aside whether AI was used or not, even completely human-written text comes out as MIX, and text that's 100% AI comes out as human-written...[1]
But is Pelican Bench really that helpful? I'm not sure. Why is this even trending?
It seems like a kind of meta joke — is there anyone who can explain in what sense it became a kind of community meme?
I understand it became famous because a well-known person named Simon tried it.
But I feel like there should be more coding benchmarks, honestly...
Why can't the Gemini model code?
I'd guess Gemini probably used the best dataset.
Even Google is the place where the world's best coders are gathered, isn't it?
Yet the GPT code from OpenAI, which collaborated with Microsoft, is beautiful, while the Gemini model's coding is trash.
I have no problem with Gemini Flash 3.8 for coding, though I only use Elixir which is one of the better languages for LLMs. Where does it fall short elsewhere?
The game industry is genuinely terrible, starting with low pay and brutal working hours, but it's interesting and lets you do a lot of fun things.
Because everyone wants to do it, there are plenty of people willing to put up with somewhat lower pay.
reply