It could be interesting to experiment with more complex representations in the memory/file system, i.e. vectors. Seems like the full advantage of this architecture is yet to be explored. Blending neurons with deterministic compute to fit on one GPU is a fascination concept, never considered it before!
First class hardware support makes this really exciting and different. On device processing is cheaper and faster, critical for real time applications of AI. Really excited for the rise of local, smart intelligence!
Whilst general self-reporting of confidence will be hard to achieve (we can't even do it as humans), I'm sceptical of the notion that we can't train models to give some confidence scores accurately. I think over the past year that with the correct RL, models can showcase some pretty insane capabilities. If reporting the correct confidence was a RL training target say for predicting weather events or a game outcome that is supported by real RL environment or simulated data, then I suspect LLMs will develop a strong intuition for confidence similar to how the recent JEV and decision models are trained.
Awesome write up and intro by a fellow Sydney-sider, kept me hooked! As someone who's not a maths wizard, the technical details flew by my head but the prose allowing me to empathise deeply with those in the field was very enlightening. I never realised how big of a deal the maths dump was and how much it has shaken up the field. I guess we are all just trying to figure out this whole "intelligence on tap" thing together.
You don’t need to be a math wizard. Just upload the paper to an AI and ask, “Explain this like I’m five.” Then, with each follow-up prompt, increase the age and level of expertise you want it to assume.
Great question! If you are looking to just store secrets you can just use infisical.
The real value of alter comes in when you want to give your agents access to end users credentials. When agents act on behalf of end user as a developer you have to manage multiple credentials and more importantly manage the mapping relationship between end user identity and the credentials.
Also, as agents take actions the problem of non deterministic actions become more pronounced. With alters policy guardrails any chance of actions taking negative actions is mitigated.
Lastly, the full compliance trails of what the agent has done ends up being quite valuble especially as agents take more autonomus actions!
I often think of this question. Probably things that are enduring. How to be a good person, how do people work, how does society work, how to learn, some hobbies and life skills.
I've been building voice agents for the past year, and honestly the word error rate and how good they sound are "good enough" across countless providers. The real gaps are in latency, handling interruptions, interrupting the speaker if needed (this is surprisingly useful in most contexts) and adapting to the emotion/tone of the context. Would love to see a benchmark or tests that measure these effectively and test the claims of Voxtral 3! These existing benchmark results look very impressive.
reply