In 5 years the models will probably be so much better and more compact that phones will be running models equivalent at least to Opus 4.6 if not Fable, at least within the areas they are tuned for (which probably won't include coding).
I'm really excited for this future. Both what you said and datadrivenangel.
The fact that Apple shipped a more than capable laptop for most of the population using a last generation iPhone chip is just mind blowing. Silicon advancements are going to allow this, and I think the global majority will catch up and make their own chips that compete or exceed western performance. Especially when the US is scared of science, rapidly divesting and defunding it.
Did you give it a tool to increase temperature, or only one that sets temperature to an absolute value?
Either way, setting temperature to 5° is obviously wrong - even if it knew the current temperature - but models of this size can't reason about relative values very well.
Give it a tool to change temperature by a given amount, and see what happens!
As a child, I lived in Firenze (Florence, Italy) and had a pet cricket. All the kids had them, in little bamboo cages, for the Festival of the Cricket:
Anything you ever do with any non-locally-hosted model always "uploads code" to the inference provider because that's how it works: the model uses tools to inspect the code, the result of the tool use is sent in an API call to provide context (and a prompt for the next turn), and then the response continues the process.
This is true and has to be true for any hosted model that works with existing code: it's not specific to this application.
This got a downvote and I understand why: because I didn't describe the test, which is to ask it "Please recite Jabberwocky".
This is actually difficult because there are so many invented words in the poem which have extremely low frequencies in the training data. So a model that can do it properly is likely to be very good in other ways. Qwen-3.6-27B can do this until it gets overly quantized.
Curious why you find that’s a useful test since it seems to be solely measuring training data memorization, something you’d expect to degrade from quantization.
Looks very cool. I would like to try it, but don't want to use API billing. OpenAI I think would allow it to use account login. Would you support that?
