I agree with your basic thesis here, retrospection will view LLMs as a transitional architecture.

However, this paper is evidence that the field is figuring out how to built what's actually needed, which is a good thing.