Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
lhl
on Aug 16, 2023
|
parent
|
context
|
favorite
| on:
How Is LLaMa.cpp Possible?
Basically, the most aggressive quants for a larger sized model is always better than the unquantized model of a smaller model:
https://github.com/ggerganov/llama.cpp/pull/1684
Tim Dettmers did a bunch of research on this last year as well:
https://arxiv.org/abs/2212.09720
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
Tim Dettmers did a bunch of research on this last year as well: https://arxiv.org/abs/2212.09720