How many SaaS / Cloud Native companies are really GPU constrained? The overwhelming majority of SaaS is a relatively trivial CRUD web-app interfacing with a database, performing some niche business automation, all of which would fit comfortably on a couple of physical servers.
… and that situation will persist until other vendors release consumer GPUs with significant VRAM. Nvidia craftily hamstrings the top consumer GPUs by restricting VRAM to 24GB. To get a bit more costs 3-5x. Only competition will fix this.
Even then NVIDIA has a pretty significant technology moat because most of the tools are built around and deeply integrate CUDA so moving off NVIDIA is a costly rewrite.
I think there will be a real shift back on prem with software delivered traditionally due to increased in shit like this (and also due to cost)