Plus one to this comment so much a scaling laws, test time compute and LLM architecture decisions can be traced back/understood if you view LLMs as efficient compression databases that search based on an a retrieval query. Just turns out that they are finding super novel compression techniques that solve unknown answer