Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Basically, the most aggressive quants for a larger sized model is always better than the unquantized model of a smaller model: https://github.com/ggerganov/llama.cpp/pull/1684

Tim Dettmers did a bunch of research on this last year as well: https://arxiv.org/abs/2212.09720



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: