AI agent memory is the information an AI agent keeps and reuses beyond a single model request. Short-term memory covers the current conversation, and long-term memory carries information from one conversation to the next.
Memory usually lives in the application around the model. The agent stores selected information, then loads the relevant parts into the context window when it needs them.
Short-term and long-term memory
Short-term memory. LangChain's documentation describes it as memory that lets an application "remember previous interactions within a single thread or conversation". On a call, it holds what the customer said, what the agent looked up, and what was agreed.
Long-term memory. The same documentation describes it as information retained "across different conversations or sessions". A customer who calls back on Thursday about Monday's dispute does not have to start over.
Researchers split long-term memory further. The CoALA framework by Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan and Thomas L. Griffiths (2023) describes language agents with modular memory components. LangChain uses three types.
Semantic memory. Facts, such as a preferred language or contact channel.
Episodic memory. Past events or actions, such as the outcome of the last contact.
Procedural memory. The rules for doing a task. For an agent, LangChain places these in the model, the agent's code, and its prompt.
How memory reaches the model
The context window limits how much the model sees in one request. Memory decides what is worth keeping, and which stored items to load back for the current turn. A typical system writes items to a store. It searches the store when a conversation starts or a topic comes up, and adds the matches to the input.
When to write is a design choice. LangChain notes that writing memories "in the hot path", during the conversation, makes them available at once but can add latency. Writing them in the background avoids that delay, at the cost of deciding when memory formation runs.
On a voice call, that latency is time the caller spends waiting. Many teams therefore keep in-call memory light and write longer summaries after the call ends.
What to store and what not to store
Under the GDPR, personal data must be "adequate, relevant and limited to what is necessary" (Article 5(1)(c)). It must be kept in identifiable form no longer than its purpose requires (Article 5(1)(e)). Memory about a customer is personal data like any other.
Worth keeping. A language preference, open case references, a callback agreed for a set time, and the outcome of the last contact.
Better left out. Card numbers, passwords, and answers to security questions. Health details, which are special category data under Article 9, unless there is a lawful basis and a clear purpose. Full transcripts where a short note serves the purpose, and guesses about a person's character or finances.
Read live. Balances, due dates, and claim status change often. Fetching them from the system of record each time avoids acting on a stale memory.
Retention and erasure. Each kind of memory item needs a retention period and a way to delete it. Article 17 gives people a right to erasure. Access controls matter too, covered under AI agent security.
Frequently asked questions
What is the difference between short-term and long-term memory in AI agents?
Short-term memory holds the current conversation: what was said, looked up, and agreed. Long-term memory keeps selected information after the conversation ends and brings it back in a later one. Short-term memory usually disappears or is archived when the session closes.
Does an AI agent remember previous calls?
Only if it is built to. Many agents start each conversation fresh and read the case history from the CRM. Others store a summary of past contacts and load it when the same customer returns. It is worth asking a vendor which applies, what is stored, and for how long.
How is AI agent memory different from a CRM?
The CRM is the company's system of record, with fields that people and systems maintain. Agent memory is what the agent writes for itself to carry context, often as short notes. Many teams prefer the agent to read facts from the CRM and keep its own memory small.
Is AI agent memory allowed under GDPR?
Generally yes, when it follows the same principles as other personal data. That means a lawful basis, a clear purpose, only what is necessary, a retention limit, and support for access and erasure requests. It helps to review the design with your data protection officer before launch.
How we approach this at Zowie
Zowie's AI agents work on voice, chat, and email, with over 100 million conversations a year across our clients. The language model speaks. A separate, deterministic Decision Engine returns every amount, rate, offer, disclosure, and escalation from your rules, with an audit log. To walk through data handling for your own use cases, you are welcome to book a demo.
Explore: Context window, GDPR compliance, PII redaction, AI agent security, Data residency (data sovereignty), AI agent