Please make this more generic, using the OpenAI API standard with model discovery. That way it works against a bare Llama.cpp instance out of the box, the local inference server that many others wrap around (like Ollama and LM Studio).
This looks and works great, super impressed. This is what Siri/Apple AI should be like.
One concern I have is about sending private info to Anthropic. So it would be great to have the option to use local AI.
Local options should support the OpenAI API that seemingly is the standard for these connections, as provided by Llama.app and Unsloth Studio for example. That way one could run against a local model and keep personal information private.
That of course begs the question if a local model is smart enough for the tasks at hand.
I'm also trying out OOMU, a similar but closed source personal assistant app for MacOS. It emphasizes privacy with local models, and even has an 'Airplane' mode. The app has a built in llama.cpp server and uses Gemma-4-4B, their own router model plus a vision model.
Most daily commands like launching and moving around sites/apps happen without AI, but I agree with the privacy concern and will definitely look into utilizing local AI.
Smaller models, like Haiku, I have found to already struggle during multi-step tasks, so I expect local model capabilities to reduce performance further.
I'll look into how OOMU splits their work up, but what local model would you suggest starting with?
I would probably run qwen 27B or qwen fn but I would run the actual inference on one of my other boxes and not on actual desktop so initially it could be sufficient if you just enabled openai or anthropic compatible endpoint.
The choice of model could be up to the user, depending on the available RAM on their M class Mac. Newer and bigger usually being smarter.
Llama.app/.cpp and Unsloth Studio on Mac already provide guidance to pick the optimal model for the available memory, all model wheights hosted by huggingface. Both apps can use the local huggingface model cache locally.
If you make it configurable, the user could also run a separate machine with an NVidia card and Linux to serve tokens on the LAN. Or use OpenRouter or any other cloud hosted model offering (DeepSeek, Z.ai, etc.).
A lot of the comments on that thread are about how the board is full of military/CIA it smells like a cover company. It had Henry Kissinger on it of all people.
Did we ever find out what all those people were doing at Theranos? I really haven’t kept up to date.
I have a theory! They saw all the other security state people involved, and figured this was something it would be a good career move to support without knowing exactly why.
"Don't obey in advance", Tim Snyder said. But obeying in advance is called showing initiative and is a great career move. In secretive industries it can be hard to know what to obey in advance, but they don't let that stop them!
One of the unspoken secrets of research is that a lot of people are doing it to keep themselves in a job. It is perfectly possible to do research for years which goes nowhere, just goes over old ground etc. As long as the money rolls in, someone's making a living and not necessarily dishonestly if they're at the lower levels.
reply