Agent memory vs RAG: the short answer
If you build with LLMs you have probably heard both terms and maybe used them interchangeably. They are not the same thing. The agent memory vs RAG distinction comes down to one question: who wrote the data, and what it is about. Agent memory is state the agent writes about its own experience as it runs. RAG (retrieval-augmented generation) reads from a knowledge base you curated ahead of time. Both often sit on the same vector store, which is exactly why they get confused.
Here is the one-line version. Agent memory is the agent remembering. RAG is the agent looking something up. A support agent that recalls you prefer email over phone is using memory. The same agent quoting the current refund policy from your docs is using RAG. Most serious systems use both, and telling them apart changes how you build storage, freshness, and cleanup.
What is agent memory?
Agent memory is the agent's own evolving state across time. It is written by the agent, about what the agent did, saw, or was told. It usually splits into a few types:
Short-term (working) memory: the current task context, the recent turns of a conversation, the scratchpad for the job in front of it. This lives in the context window and maybe a small cache.
Long-term episodic memory: a record of past events. "On June 3 the user asked me to deploy the staging branch and it failed on migrations." The agent writes this as it goes and reads it back later.
Long-term semantic memory: durable facts the agent has learned. "This user's production database is Postgres 15." "The CI script lives in scripts/ci.sh."
Personalization: preferences and patterns tied to a user or workspace that shape future behavior.
The defining trait is that memory is generated by the agent at run time. Nobody hand-curates it. It accumulates, it can contradict itself, and it needs maintenance (more on eviction below). Memory only matters if it survives past the current session, which means it needs somewhere durable to live and a runtime that stays up long enough to write it. That is the whole premise of always-on versus session-based agents: a session-based agent forgets by design.
What is RAG (retrieval-augmented generation)?
RAG is a technique for feeding an LLM facts it was not trained on, at query time, without fine-tuning. The flow is fixed:
Ingest: take a corpus (docs, wiki, tickets, PDFs), split it into chunks, embed each chunk into a vector, and store the vectors.
Retrieve: embed the incoming question, find the nearest chunks by vector similarity, optionally re-rank them.
Augment: paste the top chunks into the prompt as context.
Generate: the model answers grounded in those chunks.
The corpus is something you curated and mostly control. It is comparatively static: it changes when you re-ingest, not when the agent has a thought. RAG's job is to reduce hallucination and bring in knowledge the model does not have, like your internal documentation or last night's numbers. If you want the canonical explanation, the original retrieval-augmented generation paper from Lewis et al. is still the clearest source, and vector database docs such as pgvector walk the same pipeline in code.
A small retrieval snippet makes it concrete:
# RAG: read from a curated corpus at query time
q_vec = embed(user_question)
chunks = vector_db.search(q_vec, top_k=5, collection="company_docs")
prompt = build_prompt(context=chunks, question=user_question)
answer = llm.generate(prompt)# RAG: read from a curated corpus at query time
q_vec = embed(user_question)
chunks = vector_db.search(q_vec, top_k=5, collection="company_docs")
prompt = build_prompt(context=chunks, question=user_question)
answer = llm.generate(prompt)Contrast that with memory, where the agent writes:
# Memory: the agent records its own experience
note = summarize(current_task_outcome)
vector_db.upsert(embed(note), collection="agent_memory", user_id=user.id)# Memory: the agent records its own experience
note = summarize(current_task_outcome)
vector_db.upsert(embed(note), collection="agent_memory", user_id=user.id)Same vector database, opposite direction of data flow. That is the crux of agent memory vs RAG.
Agent memory vs RAG: a side-by-side comparison
| Dimension | Agent memory | RAG |
|---|---|---|
| Who writes the data | The agent, as it runs | You, ahead of time |
| What it is about | The agent's own experience | An external knowledge corpus |
| Freshness | Continuously updated | Updated on re-ingest |
| Direction | Mostly writes, then reads | Mostly reads |
| Typical store | Vector DB plus a state DB | Vector DB (or a search index) |
| Main failure mode | Bloat, stale or wrong memories | Missing or outdated chunks |
| Maintenance | Eviction, summarization | Re-embedding, re-indexing |
| Answers the question | "What have I learned or done?" | "What does the knowledge base say?" |
The two rows that matter most are who writes the data and how fresh it stays. Almost everything else follows from those.
When should you use agent memory, RAG, or both?
Use RAG when the value lives in a body of knowledge that exists independently of the agent:
Question answering over your documentation, policies, or codebase.
Support bots that must quote current, correct information.
Anything where a wrong or invented answer is expensive and you have a source of truth to ground against.
Use agent memory when the value lives in continuity across time:
A coding agent that should remember your stack, conventions, and past decisions.
A long-running research or outreach agent that builds on yesterday's work instead of restarting cold.
Personal assistants where remembering preferences is the product.
Use both, which is the common case for real agents:
A support agent uses RAG for the policy and memory for "this customer already tried the reset twice and is frustrated."
A coding agent uses RAG to pull the relevant file and memory to recall that the team banned a certain dependency last month.
Neither pattern is a competitor to the other. RAG gives the agent knowledge; memory gives it a history. The framework you reach for affects how you wire both up, and the LangChain vs CrewAI comparison covers how each handles retrieval and state.
A quick gut check when you cannot decide: ask whether a brand-new agent instance, with no history, could answer the question correctly given only your documents. If yes, it is a RAG problem. If the right answer depends on what this specific agent has done or been told before, that is memory. Most product requirements turn out to need a bit of each, which is why the two so often ship together in the same codebase.
Storage, staleness, and eviction
Both patterns lean on the same primitives, so it helps to see where they diverge in practice.
Storage. Both usually use a vector database with embeddings: pgvector if you want it inside Postgres, Qdrant or Weaviate if you want a purpose-built engine. RAG collections tend to be large and read-heavy. Memory collections are smaller, per-user, and write-heavy. Keep them in separate collections even on the same engine, because their lifecycles differ.
Freshness and staleness. RAG goes stale when your corpus changes and you have not re-ingested: the docs updated, the vectors did not. The fix is a re-ingest pipeline on a schedule or a trigger. Memory goes stale in a different way. A fact the agent recorded months ago may now be wrong, because the user who preferred Slack has since moved to email. Memory needs a notion of recency, and sometimes confidence, not just similarity.
Eviction and summarization. This part is memory-specific and often skipped. If an agent writes a note every run, its memory grows without bound, retrieval gets noisier, and token costs climb. You need a policy:
Summarize: periodically compress many episodic notes into one semantic fact.
Evict: drop low-value or superseded memories by age, access frequency, or relevance score.
Deduplicate: merge near-identical notes so the same fact is not stored twenty times.
RAG has an equivalent hygiene task (re-embedding when you change embedding models, pruning dead documents), but it does not accumulate on its own the way memory does. Memory is a garden you have to weed; a RAG corpus is a library you restock on a schedule.
Where the line blurs
Being honest, the boundary is fuzzy and getting fuzzier. A few cases where the two blend:
Agent memory is often retrieved with the exact same nearest-neighbor search RAG uses. Mechanically, reading memory is a retrieval step. The difference is provenance, not the query.
Some teams treat their "learned facts" memory as a corpus that later queries read from. Today's memory becomes tomorrow's knowledge base.
Long conversations use retrieval over past turns to fit the context window, which is memory implemented with RAG machinery.
So the useful test is not the database or the query. It is the direction of authorship. If the agent produced the data by living through something, that is memory. If you assembled the data in advance for the agent to consult, that is RAG. Hold that distinction and you will pick the right storage, the right freshness strategy, and the right cleanup, whether you run one pattern or both.
And because memory only exists if the agent stays alive long enough to keep it, a persistent runtime is the quiet prerequisite under all of this. That is one of the layers covered in the AI agent infrastructure stack, and it is what separates an agent that learns from one that resets every time it starts. On Sokko, that layer is the Memory add-on.