Would it be ever possible to run a GPT-{n}, n>3, similar model in a home computer wihtout GPU? I have a "good" laptop with 32GB, good processor, but no GPU (I was never interested in gaming, crypto or ML), but I found GPT very useful and I'd prefer to run a local version instead of keep feeding OpenAI.
OpenHermes-2.5-Mistral-7B is better than GPT-3 (and scores even better than GPT-3.5-Turbo in human evaluations) and can even run on a raspberry pi or in the browser. On a laptop CPU it uses about 5GB of RAM (in 5bit) and runs around 20-30 tokens per second, which is very fast.
I recommend downloading and running OpenHermes inside LM Studio. https://lmstudio.ai/
In LM Studio, search for OpenHermes. Pick the Q5_K_M version (this is the best quality/speed trade off). Then go to the chat tab.
On the chat tab, set the context length to 4096 (or up to 16k if you want longer context) and set the number of CPU cores you have under "Hardware Settings."
Select the model from the drop down and start chatting!
Most laptops these days have a pretty sizable GPU on the same chip. IIRC Triton makes proper use of the Intel graphics while AMDs equivalents work well with OpenCL out of the box. Apple's M1-3 architecture saw some major speedups on llama.cpp etc. as well. Worth noting is that some may need special drivers, my Xeons from 2010 has support for executing OpenCL but needed extra drivers; no comment on modern processors.
