for 3.8? I can get 40 tok/sec on it (M3 Max 64GB), but I don't use it because I ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		seanmcdirmid 18 hours ago \| parent \| context \| favorite \| on: Run Qwen3.8 27B locally: real numbers from my Mac ... for 3.8? I can get 40 tok/sec on it (M3 Max 64GB), but I don't use it because I can get 90 tok/sec with 3.6 MoE (MTP + 6 bit quant), and I don't notice any quality improvements for my tasks using a dense model. Did you ask Gemini or DeepSeek to look at your oMLX server log to see what was going on? This can help a lot if it is just a misconfiguration.
		help

robotresearcher 15 hours ago | [–]

Just did the fun thought experiment of playing back this comment thread in my head ten years in the past and it's amusingly incomprehensible.

dcastm 14 hours ago | [–]

What’s the context size?

seanmcdirmid 13 hours ago | [–]

128k, 256k is also possible, but my tok/s drops off and the performance isn’t good.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact