Could you please elaborate on the security implications?

wokwokwok · on Nov 13, 2023

It’s as secure as any C++ backend that performs no input validation.

Ie. whatever memory safety or sandbox you had from using wasm or rust is gone when you use it.

jart · on Nov 13, 2023

The llama.cpp author thinks security is "very low priority and almost unnecessary". https://github.com/ggerganov/llama.cpp/pull/651#pullrequestr... So I'm not sure why a sandbox would bundle llama.cpp and claim to be secure. They would need more evidence than this to make such a claim.

halyconWays · on Nov 15, 2023

This user was caught stealing code and banned from llama.cpp by its creator https://news.ycombinator.com/item?id=35411909