Have you tried Open WebUI? It doesn't determine which model works best for your hardware, but it otherwise has a very large feature set with a large community, and runs offline by default. Geared more for technical users, however.
It feels like the last mile is so long in this space. I'm a SWE with a ton of experience and I still haven't gotten over the decision paralysis of figuring out which model to run locally, and which inference platform should I use, and will my M1 Macbook Pro actually run anything well enough to be useful, and on and on.
It's kind of embarrassing to admit, but having something Just Work(tm) makes getting started so much easier.
It’s a huge new subject area and it’s completely exhausting to learn. I still feel like a total novice.
I have an M1 Max (more memory bandwidth) so I am not sure how much advice I can give about models, but I would suggest maybe trying the new LFM2.5-24B-A2B. It’s surprisingly good. If you want to mess about with multimodal work, Gemma 4 12B is really impressive. (I mostly use Muse Glimmer but it might be rather slow on an M1 Pro)
If you want to have the space to learn the terminology, tinker with what you can control etc., then you have three easy options: LM Studio, Unsloth Desktop or the llama.cpp native Mac app. But don’t get put off by the llama.cpp command line; most of these tools are running it behind the scenes, and most entries on Hugging Face have recommended parameters.
MLX is an option but so far I haven’t found it to have an edge over simple GGUF on Metal on the M1 at least; it seems it might have more advantages on the later M series chips, but it is usually slower for me.
I have explored running OpenCode inside a VM (you can “remote into it” from the OpenCode GUi desktop app). You might want to look at Paseo, which can do a similar thing but with many more coding agents.
I am quite intrigued by Osaurus; on macOS 26 and up it looks like it will manage a container for you.
You might also want to look at Orchard. Not done this myself yet but I can see the appeal.
I am an experienced web dev in my fifties. Six months ago everything I have written above would have been total gibberish to me. Learning some of the open weights stuff has made me feel much less helpless than I think I might if I just developed a Claude dependency. Nothing matters more to me than grasping the fundamentals.
I still don’t really use LLMs for hands-off coding. But I have found that a local LLM can be a useful sidekick; even at the most basic level it’s a better Google that remembers the context of queries.
Thanks for taking the time to write all this up, I'm checking out the projects you mentioned now.
I feel you on the tinkering-to-feel-less-overwhelmed front. I've been through a few ground shifts in my career so far, and even a little bit of "Oh, so that's how it works, ok..." can make it feel so much less daunting.
Mostly I think my own approach has been to try to combat my anxiety and exasperation about all of this by taking this measure of control.
While no really small local model comes super-close to a frontier model, the shape of the technology is settling and I think one can learn so much by approaching it the conventional nerdy way: what's a practical setup I can run on kit I already have? What can I build that helps me explore the problems?
I am quite cynical about AI still, including for generative coding, but I am not cynical about what I can run locally. There is no doubt that they are already helpful to me even in a minimal form.
And the really cool thing about being able to run them locally is to understand the tradeoffs.
Jailbroken 3DS scene is usually for modding the console OS itself given how easy it is to run custom firmware. And for ports, and emulation. While the GBA has a pretty dedicated community for custom games and romhacks, people often create games for the platform at gamejams because of how great the dev tooling is. It's a bit lackluster on the 3DS, but I did find the Love Brew[0] framework to be really nice. Even has simulator support[1] for testing your builds locally, though fonts will look much different on real hardware, with likely a bunch of other gotchas. Running out of RAM being a big one.
edit: compared it to the GBA homebrew scene since it's a large dedicated community to derive pros and cons between the two from. Switch is still receiving updates from Nintendo to brick your system.
This is just an ad for contextetech. Search engines build their own index, which is hard. Decentralized search engines like Searx have middling results, and it's a project running for over a decade. "Smart" is subjective, and not something I'd trust an LLM to determine.
Recreational programming used to be one of the rubrics to hire a software developer. If you have an intrinsic motivation to code, it would often be more valuable than your credentials. That's why junior devs would push up sites showcasing their resume of random projects they've done outside of education in order to give them an edge over their competition. Some companies would even encourage some time on the clock for working on side projects to prevent burnout.
At the same time, some of the workers in the industry were disgruntled that you were expected to enjoy coding in your spare time to keep up. While some truly just wanted to do it because they were good at it, and would rather turn off the software dev brain after work to do other things they enjoy in life.
With the advent of high-performing LLM's for coding, though, it seems more important to showcase how you can harness AI for your recreational projects than the projects themselves. It's a bit of a catch-22 for junior devs though.
I still think it's good rubric to get into programming if you like to do it recreationally, but these are just a few of my thoughts on how it can be problematic.
> With the advent of high-performing LLM's for coding, though, it seems more important to showcase how you can harness AI for your recreational projects than the projects themselves.
people who ride horses recreationally don't really care that cars exist
I think yours and the rest of the comments to mine are missing the point of the presentation I'm critiquing, and missing the (admittedly cursorily added) escape hatch on why I think it's totally fine to program recreationally. It should be encouraged for everyone to learn. Akin to learning an instrument and/or music theory. It helps you in ways that you might never notice.
something about metrics and targets. When I got my first programming job, one of the reasons they hired me, apparently, was because I knew what components I had in my PC at home. If they had published the question beforehand and everybody knew to study it, it would no longer have made sense to use that question as a filter. You want the people who knew this stuff anyway, with no expectation of reward
That's not the goal of recreational programming though. You don't do it for the ROI. It may have a ROI, but I don't do this for that, I do it because I am genuinely curious.
Last time I visited Miami Beach, I found out they have a forced 20% gratuity at restaurants. And still ask for a tip on top that. My friends and I were likely tipping >40% for subpar service and overpriced drinks for a few days until someone noticed it on their receipt.
No way. I routinely tip 20%, rounded up. If the place has a mandatory tip, I add max(0, 20-[service change]) to make up the difference between that and 20%. If it's 10%, I'll add 10% more. If it's 20%, I'll add 0. If it's 25%, I'll leave a bed Yelp review and never go back.
Don't most things people use day-to-day for both work and entertainment require use of the actual app these days, or at least make it very difficult to do so?
So now we're believing it's great that LLM's can write macros, thus further obfuscating human maintenance of the code-base, because LLM's can understand and implement it than better than humans.
This isn't the future I wanted, and is why I advocate for trade schools these days.
I miss the days I could listen to music and use my adhd/autism to its fullest potential reading documentation and figuring things out myself.
Of course, these are same people you'd want managing these AI agents in the first place, but reviewing code written by others always kind of sucks, especially when it's assumed the language model knows better than you do which isn't often the case!
Was the PRD perfected on the requirements? Only God knows, and I personally want to be there when it's written.
reply