Hacker Newsnew | past | comments | ask | show | jobs | submit | almostuseful's commentslogin

If Astra uses some sort of looped transformer architecture (and if Fable doesn't), then that could explain why Astra can do more computation per token. It is essentially simulating a much deeper model.

But yea, I was also quite surprised by the low token usage.


You have to start somewhere. But looped transformers https://sebastianraschka.com/llm-architecture-gallery/looped... are more similar to your idea.


They still only loop from the last hidden layer, and there is no router it's deterministically to the start


Give it a few months and open-weight self-hosted models will be able to do similar reverse engineering work. It is an extraordinary future we live in.


That doesn't really solve "being unable to do anything yourself", though.

(It's strange to me how often the "Hacker News" commentariat seems not to value knowing things.)


This just plain won't happen. They'll either drive hardware prices through the roof, or heavily regulate open models. Radical changes are coming. It's a bleak, bleak future we live in.


But the first time someone came up with that idea it was an act of creativity.


Yeah I agree. Though it is hard to tell if LLMs are capable of "easy" creativity like that though because anything that easy has already been done many times in its training set.

You've have to invent some new domain I guess and see if it could be creative within that domain. Difficult to think of a good test though.


If at some point a competent senior software engineer can be automated away, I think we are so close to a possible 'AI singularity' in as much as that concept makes sense, that nothing really matters anyway.

I don't know what will be automated first of the competent senior software engineer and say, a carpenter, but once the programmer has been automated away, the carpenter (and everything else) will follow shortly.

The reasoning is that there is such a functional overlap between being a standard software engineer and an AI engineer or researcher, that once you can automate one, you can automate the other. Once you have automated the AI engineers and researchers, you have recursive self-improving AI and all bets are off.

Essentially, software engineering is perhaps the only field where you shouldn't worry about automation, because once that has been automated, everything changes anyways.


Carpenters and other manual jobs might outlast software engineers. It seems that AI is advancing a lot faster than robotics.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: