What is the point of using a outright ancient model like DeepSeek-R1-Distill-1.5B from 2025 based on another ancient model like Qwen2.5-Math-1.5B from 2024, when a much better modern model like LFM3-1.7B exists?
This is a well known tell for using (bad) models to do "research". They always use qwen2.5 and R1, because of the amount of "news" that the two models received. So the models "suggest" those lineages a lot. The local llama sub used to get a few of these a day. That and a zenodo "paper" link are clear tells of badly written stuff.
Although with Anthropic purposefully sandbagging their models for AI R&D, how much of it comes down to model failure, and how much of it comes to deliberate misalignment of the model maker and the customers they sell to?
We have tested it on all kinds of small models including LFM3-1.7B. Our goal is to make a a system that can compress any AI. This is our first version and with your feedback we can improve.
Unless your task happens to be completely contained in the set of your "curated works", that is, trivial and computer-sciency, the resulting model is likely to have for example absolutely no concept of common sense, society, the natural world, what humans consider artistic or of good taste. It will probrably make, say, a good and correct compiler, but fail spetacularly on anything else more complex that a toy crud.
The first comment was kind of iffy when it was also talking about buffers and characters, and focusing on code points is mostly a bad focus. It's worth bringing up so nobody thinks fixed width at a single layer is particularly useful, because other layers will still be variable.
Isn't Fable intentionally trained and system prompted to act maliciously and attempt to sabotage third party attempts to use it to train or improve other LLMs?
The only (still in prototype stage!) "competitor" for those GB10/Ryzen Al Max+ 395 (in my region, borderline unobtainable) systems seems to be the Xiaomi AI Cube.
reply