Would have to be productionized, the current repo is a PoC, but if there is enough interest we could def commit some resources to hardening this and adding things like MCP.
Super simplistic example, but say i mention my Daughter, who is 9.
Then mention she is 10,
a few years later she is 12 but now i call her by her name.
I have struggled to get any of the RAG approaches to handle this effectively. It is also 3 entries, but 2 of them are no longer useful, they are nothing but noise in the system.
> I have struggled to get any of the RAG approaches to handle this effectively.
You need to annotate your text chunks. For example you can use a LLM to look over the chunks and their dates and generate metadata like summary or entities. When you run embedding the combination data+metadata will work better than data alone.
The problem with RAG is that it only sees the surface level, for example "10+10" will not embed close to "20" because RAG does not execute the meaning of the text, it only represents the surface form. Thus using LLM to extract that meaning prior to embedding is a good move.
Make the implicit explicit. Circulate information across chunks prior to embedding. Treat text like code, embed <text inputs + LLM outputs> not text alone. The LLM is how you "execute" text to get its implicit meaning.
(memory) [2024-05-30] My daughter turned 11 today!
System prompt: Today is 2025-08-21
User prompt: How old is my daughter?
The vector DB does the work of fetching the daughter-age-related memories, your system decides (perhaps with another LLM) if the question needs time-based sorting or something else.
Yeah good call, I missed that. I don't think there's a correct answer here, but it could be another step of the read or write. Either it would do another lookup of "my daughter" -> Name on read, or do a lookup on write if you already have a "my daughter is Name" memory. Whatever's less expensive in the long run. The graph memory someone else mentioned also seems like a good option there.
When the RAG sees retrieved facts, it should see the timestamp for each. It will easily then use the latest fact if there aren't too many conflicting ones. I am assuming that the relevance ordering of the retrieved facts won't help.
Separating a RAG from a memory system, it is important for a memory system to be able to consolidate facts. Any decent memory system will have this feature. In our brain we even have an eight hour sleep window where memory consolidation can happen based on simulated queries via dreams.
That is because basic RAG is not very useful as a long-term knowledge base. You have to actively annotate and transform data for it to become useful knowledge. I have the same problem in the regulation domain, which also constantly evolves.
In your case, you do not want to store the age as fact without context. Better is e.g. to transform the relative fact (age) into an absolute fact (year of birth), or contextualize it enough to transform it into more absolutes (age 10 in 2025.
Keen to do this as well! But couldn't find a good reliable eval for memory creation and retrieval, if you have one you could recommend i'd give it a go asap.
