That's not exactly true. What you describe is a Markov chain, not LLM. The LLMs use neural networks to extract information from the language itself, and make decisions based on this stored model. The model is built from the language, not reality, but the model can integrate information that isn't present in the language, but on the meanings that were conveyed by training data.