Apple should get working on a version of the Neural Engine that is useful for th... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

anentropic on April 11, 2023 | parent | context | favorite | on: The Coming of Local LLMs

Apple should get working on a version of the Neural Engine that is useful for these models, and remove the 3GB size limit [1] to take full advantage of the 'unified' memory architecture. Game changer.

Waste of die space currently (on Macbook at least, I'm sure they find uses for it in the iPhone)

[1] https://github.com/smpanaro/more-ane-transformers/blob/main/...

leetharris on April 12, 2023 [–]

It's not a waste on Mac, it will dynamically switch between GPU and NPU whenever CoreML is called. There are a decent amount of applications that use CoreML.

But I do agree it should be improved!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact