Hacker Newsnew | past | comments | ask | show | jobs | submit | honr's commentslogin

The set of great ideas is much much larger than the set of viable business opportunities in any given year. So, the point... is missing the point :-)

For people working in electronics, this is absolutely nothing new. The electronics / hardware side of silicon valley (and the counter parts in Taiwan, South Korea, China, and to a lower extent in Japan) is a cut-throat highly volatile environment. Software, in comparison, is much... softer.

NVIDIA can lose to the new Compute-in-Memory startups, if one of them successfully partners with other hardware companies with solid experience in scaling up. CIM has been the main topic of pretty much all electronics conferences I have attended this year, and it is picking up quickly.

Their main moat is in the training side of the hardware, whose market is far more limited than the inference side.


The current compute they are buying is NOT state of the art (check this year's academic research on inference hardware). There is 10x-100x inference-per-watt improvements in existing prototypes (in the industry, not just academia). Not sure what I am talking about? Ask your friendly AI agent of choice, for the details.

Check out mythic.ai for one example.


How long till that "research capability" becomes just 5 people with many AI agents, and no other humans in the main research loop?

I don't know and can't answer or predict that. But that's exactly what it could become (although probably not that extreme), where the agents doing much of the internal work are proprietary, conferring a research advantage.

My point is that it is not a real moat if other players can hire people from core research teams, and get a foothold that easily. And when/if models are good enough, you don't even need many people to form a fully functioning team.

It may actually get harder to replicate with time, not easier, even if the number of people needed to run the operation goes down. As the largest labs move towards internal RSI, the most important IP for doing that will transition into their proprietary internal infrastructure and models, none of which will ever be released and which increasingly won't be fully known or understood by the humans you might hire away.

Current RAM is no moat. Upcoming Compute-in-RAM is. And nobody has that built up at scale yet. So, while a moat might become possible, we only have yesteryear's hardware as "moat".

Can you explain what compute in RAM means?

I thought HBMs are technically the fastest possible thing for inference.


The academic term is now "Compute-in-Memory". That's MOSTLY what has suddenly been hiring SRAM and DRAM designers across the industry this year. It is part of the fast evolving field of doing inference efficiently. This has been all the rage in several Electronics conferences this year.

Please don't rely on my [inferior] explanation and search for yourself (start here for insance: https://www.google.com/search?q=what+is+compute+in+memory+an...). Asking about a new field is much better done with AI these days, because you can get it to explain just the interests and questions you personally have, rather than a generic and boring explanation.

In short, better than HBM is not having to move data in and out of memory at all or at least not as much. The compute part in inference is much simpler than the data hauling part.

Couple of examples: Check out Axelera's Europa board (upcoming) or their Metis board (existing). Or check out Mythic.ai which FINALLY managed to raise funds in Nov / Dec 2025. Ironically, her PhD thesis had been gathering dust before that.


Thank you

Exactly. As for the article itself, I would take with a grain of salt as the author's previous company built some sort of software factory. I think we can give them the benefit of the doubt and move on.

With that out of the way, I think trying out developing software-building automation is a great learning experience, regardless of the final outcome. Beside [re]learning various ropes that might have been modernized recently, you are likely to end up with bonus [possibly tiny] helper tools that you can carry into your other software development projects. Or you might learn new [non technical] disciplines around software development life cycle.


It certainly has compaction (since the public launch I assume) and I HATE it. I have some remedies but nothing perfect yet. It never retains ALL the crucial bits. If a conversation runs into two compactions it is often a sign that I have to abandon it and retain whatever I can, to form a seed prompt for an adjacent conversation.


That is really the biggest beef I have with agy over others, the forced auto compaction at the 250k token threshold (3.8-flash), while the model itself (via API) would be fine with a 1M context window. Even if the model is great, restricting context to 250k tokens (and auto compacting no matter what) limits certain applications and workflows somewhat.


I find this discussion interesting. I've had huge increases in accuracy and huge reductions in token usage by capping my Claude models at 200k instead of 1M. I find 1M unusable and wasteful and feel that 200k should be the default. This also makes sense given that the whole reason people use "Ralph loops" is to keep the context window small for all tasks to get better results. Of course, clearing it yourself and manually managing it is better, but if I have 8 projects going in different terminal tabs I'm not watching any one of them that closely to effectively do that.

What are people using 1M context window for?


The problem for me is that it's a forced compaction at a fixed token number without wiggle room. I would rather run into e.g. 300k and be done than have a compaction at 250k, followed by a compressed context where it has to try and re-read what is needed. There are many times where I had to force it to re-read something in full because of its complexity, and which would get lost in compaction and not fully recovered afterwards.


There is auto compaction at 250k? Having used agy for months with multi day sessions, I have never seen this.


I think up until a few versions ago, you wouldn't even know it happened because it didn't tell you. But you can always check with /context. For many applications the compaction is very good, so you likely never noticed.


Wtf. Can you turn that off? On CC I have compaction completely turned off. I’d rather hit the hard out of context limit at 1M.


- That should nowadays be very obvious. You start any substantial work by doing research. LLMs are [perhaps surprisingly to some] good at that. They are also pretty good at doing the plumbing. That is a godsent in the hands of a good researcher and architect. And I don't have any qualms if tokens are "wasted" on research or on the precursor work. That doesn't replace reading good books on a subject but it DOES replace finding and filling knowledge gaps.

- But that shouldn't be confused with getting the LLMs to make the decisions. I believe that would quickly ruin a good design, unless the decisions are about truly inconsequential aspects, which are very rare in language and API design.

- I don't believe that (sharing) is to the detriment of LLM providers either. Not realistically. We would build faster and the questions / research directed at LLMs would be more sophisticated. Believe it or not, they can't cache questions as easily as websearch queries. If anything, I believe the more people learn to use LLMs effectively (rather than just to generate slop), the more their usage will be ingrained in daily life. Some of that will be redirected towards current LLM providers. But perhaps more of the economic share will increasingly go to hardware providers, as more and more people will be interested to run their own models.


I have never had a problem with kiwi skins (except when the kiwi is unripe; a natural mat with acids and possibly oxalate. NOPE).

But are you claiming you eat the whole fruit, or you throw out the tiny thorn?


I usually throw out the tiny thorn.


I don't get your point. The planet will not realistically run out of either. The water price goes up, then soler + desal becomes more cost effective, then people get to use the material that covers 75% of the planet's surface a bit more. Similarly, when oil gets more limited the price goes up and the planet gets to keep its oil while people stop being able to afford burning more of it. Or perhaps this was exactly your point.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: