Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Nvidia GPUs over significantly more performance per card than Intel and AMD. In general the limiting factor is total computation, so people will pay a premium for the best. I'm not familiar with Google's hardware, and don't think it's generally available.


What about AMD's MI300X? I see reports all over the web that it runs LLMs at a similar speed as an H100.


It’s available on GCP through Cloud TPU, though Nvidia on GCP is still a far bigger business.


Can you run LLAMA on Google's TPUs?


Yes. You can serve Llama on TPU: https://cloud.google.com/vertex-ai/docs/predictions/serve-ll...

You can also train Llama on TPU: https://cloud.google.com/tpu/docs/v5p-training




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: