Tensor parallelism, so you only need to store a fraction of kv cache per gpu.

		jychang 60 days ago \| parent \| context \| favorite \| on: Alibaba Cloud says it cut Nvidia AI GPU use by 82%... Tensor parallelism, so you only need to store a fraction of kv cache per gpu.