A fascinating thing for me after reading this is: how can it be that the "circuit input" is compatible with its output to the point where the performance improves? The training process never saw this particular connection just like it didn't see layer 60 output into layer 3 or whatever.

Great read, makes you wonder what else is encoded in these models that might be useful!

I think the intuition is that the first N layers decode into "thought language" while the last N encode back to desired output language. So if there are well defined points where it transitions between decoding/understanding, thinking, and rendering back to language, those 2 transition points should be in the same vector space of "LLM magic thinking language".

A good example of this is Rust. Rust is by default memory safe when compared to say, C, at the expense of you having to be deliberate in managing memory. With LLMs this equation changes significantly because that harder/more verbose code is being written by the LLM, so it won't slow you down nearly as much. Even better, the LLM can interact with the compiler if something is not exactly as it should.

On a different but related note, it's almost the same as pairing django or rails with an LLM. The framework allows you to trust that things like authentication and a passable code organization are being correctly handled.

I was under the impression from Rust developers that it was one of the languages LLMs struggled with a bit more than others? My view could be (probably is) very outdated.

If it were ever true, it's not anymore.

Rust is a nice choice even just for its amazing sum types and the ability to make impossible states unrepresentable at the type level.

Very small suggestion: Can you make the entries actual links/anchor tags so that it is possible to copy link, middle-click to open in a new tab, and so on?

You can look at it both ways.

Using the database as a queue, you no longer need to setup transaction triggers to fire your tasks, you can have atomic guarantees that the data and the task were created successfully, or nothing was created.

I think the problem starts with the name. I've been coding with LLMs for the past few months but most of it is far from "vibed", I am constantly reviewing the output and guiding it in the right direction, it's more like a turbo charged code editor than a "junior developer", imo.

Have you tried Roo Code in "Orchestrator" mode? I find it generally "chews" the tasks I give it to then spoon feed into sub-tasks in "Code" (or others) mode, leaving less room to stray from very focused "bite-sized" changes.

I do need to steer it sometimes, but since it doesn't change a lot at a time, I can usually guide the agent and stop the disaster before it spreads.

A big caveat is I haven't tried heavy front-end stuff with it, more django stuff, and I'm pretty happy with the output.

And a great way to pick up the guitar every day is to have it ready to go at all times. Keep it outside the bag/case and at arms reach.

And an acoustic rather than an electric (although electrics are great too).

If you're visiting Shinjuku, nearby there's a narrow street called Omoide Yokocho. Just take in the vibe and choose a yakitori spot to grab a bite and drink your poison of choice (tea/beer/sake). I would recommend going at night/dinner time.

Speaking of Shinjuku and videogames, if you've ever played any yakuza/like a dragon game, you owe it to yourself to go to Kabukicho and its big red gate.

In any case, whatever you choose to visit in Tokyo, it will be really nice, and a lot of it will still be waiting when you eventually come back.

Cheers!

The worst so far, and the best when compared to all of your remaining days.

I think Elden Ring (and the souls genre in general) IS designed with the assumption players have access to the internet to search for stuff if they want.

The game will still provide quite a challenge even if you know where you need to go and get weapons/items/etc. The bosses won't defeat themselves even if you know the overall strategy to use.

I think of it as kind of a self-regulated difficulty system, if you want to go in blind you are still free to do so.

Elden Ring and Fallouts are IMO good examples of games that have done great in this environment. There are people wondering at the art and archaeology(!) and cultures and various descriptions and other lore bits scattered around the games while others are speedrunning and stat optimizing things. They're extracting enjoyment whether they're on their first playthrough or have thousands of hours invested in countless builds. Surely just about every technical detail has been discovered, debugged, and charted a long ago.

There are probably far simpler ones as well. People still talk about chess strategies and things like that all across the detail spectrum.

There is also the reality that most games produced just aren't worth that much of people's time. They may still be fun, but in a more limited capacity.