Long-Term Memory for AI Agents: What It Is and How to Add It

Why the context window is not memory

Picture an agent that spent last Tuesday learning your codebase. It read forty files, worked out that your auth logic hides in an odd piece of middleware, and helped you ship a fix. Wednesday morning you send a follow-up, and it stares back blankly. It re-reads the same forty files from scratch. That gap is exactly what long term memory for ai agents is meant to close, and the first instinct most builders have is the wrong one: make the context window bigger.

The context window is working memory, not long-term memory. It is the text the model can see on this one turn: the system prompt, the recent messages, whatever documents you stuffed in. When the run ends, or the conversation gets long enough that old turns fall off the front, that content is gone. A 200,000-token window feels huge until you notice it resets to empty every time the process restarts. It is RAM, not a hard drive.

There is a second reason a bigger window does not equal memory. Even inside one session, everything in context competes for the model's attention and costs tokens on every single call. Carrying a month of history in the prompt is slow, expensive, and actively harmful to answer quality, because the signal you need drowns in noise. Real memory works the other way around: store a lot durably, and pull in only the few things that matter for the task in front of you.

So memory for an agent is not one thing. It is a small stack of systems that each answer a different question. What am I doing right now? What happened before? What do I know? What files do I own?

What long term memory for ai agents actually is

Break "memory" into the parts an agent needs and it stops being fuzzy. Most production agents end up with some mix of four layers.

Short-term memory is the context window we just covered. The other three are what people usually mean by long term memory for ai agents.

Episodic memory is the agent's diary: an ordered record of what happened. "On July 2 the user asked me to refactor the billing module, I opened a PR, they said the tests were flaky." You store these as rows or log entries and replay or summarize them later. Episodic memory is how an agent answers "what did we decide last week."

Semantic memory is recall by meaning rather than by exact match. You embed text into vectors, store the vectors, and at query time you embed the new question and fetch the nearest neighbors. This is what lets an agent surface "the user prefers TypeScript and hates default exports" three weeks after it was mentioned, without that sentence ever sitting in the current prompt. Vector search is the engine under most "the agent remembers me" features.

Durable file and workspace state is the least glamorous and the most underrated. It is the actual files on disk: a notes.md the agent maintains, a SQLite database it writes rows into, cached artifacts, a checkpoint of a long job. No embeddings, no similarity search, just a filesystem that is still there tomorrow. If your agent runs in a container that gets recreated nightly, this layer is where "why did all of my agent's work vanish" comes from.

How to add memory: the tools that actually do this

You do not build a vector database from scratch. Every practical approach to long term memory for ai agents comes down to picking a layer and wiring in a tool that already solves it. Here are the ones worth knowing by name.

Mem0 is a memory layer you drop in front of your LLM calls. It handles extraction (deciding what is worth remembering from a conversation), storage, and retrieval, and it exposes a plain add and search API. Good default when you want "give my agent memory" without designing a schema first. The Mem0 docs cover both the hosted and open-source options.

Zep is a memory server focused on long-running chat agents. It keeps a temporal record of conversations, auto-summarizes older history, and builds a knowledge graph of facts about each user, so you get both episodic and semantic recall behind one service.

Letta (formerly MemGPT) comes at it from the agent-architecture side. It treats the LLM like an operating system managing its own memory, paging facts in and out of context and deciding what to keep. If you want memory management to be a first-class part of the agent rather than a bolt-on, Letta is the project to read.

A raw vector database sits under a lot of the above, and sometimes you want it directly. pgvector turns Postgres into a vector store, so your embeddings live in the same database as the rest of your app: one backup, one connection string. Pinecone is the managed, serverless option when you would rather not run the database yourself. Reach for a raw vector DB when you need full control over chunking, metadata filters, and how retrieval is scored.

Plain files on disk cover the durable-state layer, and you should not skip them because they sound too simple. A markdown scratchpad the agent reads at startup and appends to as it works is a legitimate, debuggable form of long-term memory. So is a local SQLite file. The one hard requirement is that the disk survives a restart, which is exactly where a lot of setups quietly fall over.

A rough rule of thumb for choosing:

• Want "remember the user across sessions" quickly: Mem0 or Zep.

• Building a chat agent with lots of history to summarize: Zep.

• Want memory baked into the agent's control loop: Letta.

• Already run Postgres and want control: pgvector.

• Need to persist files, checkpoints, or a local DB: a durable disk volume (see below).

A tiny Mem0 example you can run today

The fastest way to feel the difference is to add a memory layer to an existing agent. With Mem0 that is a few lines.

pip install mem0ai

pip install mem0ai

from mem0 import Memory

mem = Memory()

# after a conversation, store what matters, tagged to a user
mem.add("Prefers TypeScript, dislikes default exports", user_id="salim")

# on a later run, pull back only the relevant memories
hits = mem.search("what are this user's coding preferences?", user_id="salim")
for h in hits["results"]:
    print(h["memory"], h["score"])

from mem0 import Memory

mem = Memory()

# after a conversation, store what matters, tagged to a user
mem.add("Prefers TypeScript, dislikes default exports", user_id="salim")

# on a later run, pull back only the relevant memories
hits = mem.search("what are this user's coding preferences?", user_id="salim")
for h in hits["results"]:
    print(h["memory"], h["score"])

The add call extracts and embeds the fact. The search call embeds the query and returns the closest memories with a relevance score. You take those hits and paste them into your prompt for the current turn. That is the whole loop: store durably, retrieve by meaning, inject only what is relevant. Everything fancier (graph memory, auto-summarization, decay) is a refinement of those two calls.

Where durable file and workspace state fits (and where Sokko fits)

Here is the honest part, because it is where a lot of "add memory to your agent" advice gets vague. A vector database and a memory framework give your agent semantic recall. They do not, on their own, keep your agent's files alive. Those are two different problems.

If your agent writes a progress.json checkpoint, maintains a SQLite database, or keeps a working directory of cloned repos, that state lives on a disk. When the agent runs on your laptop, the disk is your laptop, and it disappears the moment you close the lid or the process dies. Run the agent in a container on a generic host and you hit the same wall: unless the volume is explicitly persistent, a redeploy wipes it. That is the failure behind an agent whose work vanishes whenever the machine goes away, and it is really a hosting problem, covered in how to run an AI agent 24/7.

This is the specific, narrow thing Sokko's persistent workspace does, and it is worth being precise about what it is not. That persistent workspace is not a memory framework and not a vector database; it does no embeddings and no semantic search. What it gives you is the durable-state layer underneath all of that: Sokko runs your agent (OpenClaw, Hermes, Paperclip, or OpenSRE, deployed from the dashboard in a couple of clicks) as an always-on service on infrastructure we manage, and anything the agent saves lands on durable, Sokko-managed storage that persists across restarts. Your SQLite file, your checkpoints, your scratchpad markdown, even a local pgvector database file, all survive a restart or redeploy, because the storage outlives any single run of the agent.

So the clean division of labor looks like this:

• Semantic recall (remember facts by meaning): Mem0, Zep, Letta, or pgvector/Pinecone.

• Episodic history (an ordered log of events): a database, often the same Postgres.

• Durable workspace state (files, checkpoints, local DB on disk): a persistent workspace, which is what Sokko gives you without renting and babysitting a VPS.

You pair them, you do not pick one. A common self-hosted stack is Postgres with pgvector for both episodic rows and semantic vectors, plus a persistent disk for the agent's working files. If you run that yourself, self-hosted AI agents walks through the tradeoffs. If you would rather not manage the box, a persistent-workspace runtime keeps the same files alive without the server maintenance, the way you would keep a Claude Code agent running without a Mac mini.

If you want your durable memory in Postgres itself, a pgvector table for semantic recall is small.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE agent_memories (
    id         bigserial PRIMARY KEY,
    user_id    text         NOT NULL,
    content    text         NOT NULL,
    embedding  vector(1536) NOT NULL,
    created_at timestamptz DEFAULT now()
);

-- nearest-neighbor search: closest memories to a query embedding
SELECT content
FROM agent_memories
WHERE user_id = 'salim'
ORDER BY embedding <-> '[query_vector_here]'
LIMIT 5;

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE agent_memories (
    id         bigserial PRIMARY KEY,
    user_id    text         NOT NULL,
    content    text         NOT NULL,
    embedding  vector(1536) NOT NULL,
    created_at timestamptz DEFAULT now()
);

-- nearest-neighbor search: closest memories to a query embedding
SELECT content
FROM agent_memories
WHERE user_id = 'salim'
ORDER BY embedding <-> '[query_vector_here]'
LIMIT 5;

Put that database file on a persistent volume and you have episodic rows, semantic vectors, and durable state in one place that survives restarts. The pgvector project on GitHub documents the index types (HNSW and IVFFlat) you will want once the table grows past a few thousand rows.

A memory stack for a real agent

Pulling the layers together, a practical setup for a working agent looks like this:

1. Keep the context window lean. Put only the system prompt, the current task, and retrieved memories in it. Do not dump raw history.

2. Log every meaningful event to an episodic store. Postgres rows work fine. This is your audit trail and your "what did we do last week" answer.

3. Embed the facts worth recalling and store the vectors (Mem0, Zep, or pgvector). At each turn, search and inject the top few hits.

4. Write working files, checkpoints, and any local DB to a disk that persists. Read the checkpoint at startup so a restart resumes work instead of restarting it.

5. Make sure that disk actually survives. This is the step people skip, and it is the difference between an agent with memory and an agent that convincingly pretends to have one until the next redeploy.

None of these layers is optional if you want an agent that behaves like it remembers you. But they are separate concerns with separate tools, and the biggest mistake is assuming one product covers all four. A vector DB will not save your files. A persistent volume will not do semantic search. You wire the layers together on purpose.

Where to go next

Start small. Add Mem0 or a single pgvector table to one agent and watch it recall a fact across two separate runs. Once that works, the harder question is keeping the whole thing alive between runs so the memory keeps accumulating, which is a hosting question more than a memory one. Pairing a persistent workspace (files and memory that survive restarts) with the tools above is how long term memory for ai agents holds up in production: the agent both remembers what it learned and stays awake long enough to use it.