How agents remember

How does a chatbot forget your name? Let's try to understand. In a simple way. You type your name.

It answers, using it. Ten messages later, it has no idea who you are. That is not a bug. That is the default.

Here is why. Every time you press send, the model runs once, produces an answer, and stops. Nothing is kept afterwards. The word for that is stateless.

Between two runs the model holds nothing. No notebook, no file, no trace of the last reply. Its weights, the numbers it learned during training, are frozen. Chatting with you never changes them.

So your name is never written down anywhere. Then why does it seem to remember? Because the app pastes the whole conversation back in front of it, every single turn. The past is re-read, not recalled.

And that pasted text has a hard limit. The context window. A fixed number of tokens the model can look at in one go. Grow past it and the oldest lines fall off the front.

Your name was in the first message, so it is the first thing to go. Open a new chat and even that is gone. Nothing carries over, because nothing was ever stored to carry. So memory is not something the model has.

It is something you build around it, and hand back at the right moment. So, it forgets your name because it never stored it. It only ever sees the text in front of it, right now. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is an agent's short term memory buffer? Let's try to understand. In a simple way. The buffer is not inside the model.

It is a plain list, sitting in your code, beside it. Every turn, two things get pushed onto that list. What you said, and what the agent replied. Nothing clever.

Just append. Before the next call, your code glues that whole list together and sends it again, from the top. The model reads it fresh, every time. That is why the agent seems to hold a thread.

The buffer is its working memory. What happened two minutes ago, still in front of it. But this list cannot grow forever. So you give it a cap.

The last ten turns, say. Or a budget, like four thousand tokens of history. When a new turn pushes it past that cap, your code drops the oldest turn off the front. Newest in, oldest out.

A sliding window. Some agents soften the loss. Before the old turns are dropped, one extra call squeezes them into a few lines, and that summary rides at the top. And the buffer lives in running memory only.

Close the session, and it is gone. Nothing in here reaches tomorrow. So the cap is a dial. Too small, and the agent loses the thread.

Too big, and every call gets slower and costs more. So, the short term memory buffer is a capped list of recent turns, kept by your code, sent again in full, and trimmed from the oldest end. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is semantic and episodic memory? Let's try to understand. In a simple way. An agent's long term memory is not one bucket.

It holds two kinds of thing, borrowed from how we describe human memory. Facts, and events. Semantic memory is the facts. Maya is vegetarian.

She works in Berlin. The team ships on Fridays. Things that are simply true, with no moment attached. Episodic memory is the events.

On the third of March, Maya asked for a dinner table, and the agent booked the wrong night. Something that happened, once. That is the giveaway. Every episode carries a time stamp.

A fact does not. Take the when away, and the episode is gone. But the real split is how they get written. A fact has one current value.

Tell the agent you eat fish now, and the old line is overwritten. An episode is never overwritten. It happened, and that cannot stop being true. So the event store is append only.

So they answer different questions. What is true about this user? That is the facts. What happened last time?

That is the episodes. Their shapes differ too. The fact store stays small, one line per thing you know. The event store only grows, one line per thing that occurred.

When the same episode keeps repeating, the agent writes it up as a fact. Three window seat requests become one line. Prefers window seats. So, semantic is what is true.

Episodic is what happened. One gets overwritten, the other gets added to. Quick check now. One question is coming up.

Let's see if it clicked.

How does a simple memory log work? Let's try to understand. In a simple way. The short term buffer vanishes when the session closes.

So for the agent to know you tomorrow, something has to be written down. The simplest long term memory is exactly that. A plain text file on disk, next to your code. One line per thing worth keeping.

At the end of a turn, one extra model call reads what was said and answers one question. Anything worth remembering? It replies with a short line. Your code appends that line to the bottom of the file.

Nothing in the middle is touched. The log only grows downward. A line is small and plain. March third.

Maya prefers a window seat. A date, and a fact, in ordinary words. Now tomorrow. Before the first message goes out, your code opens that file and reads the whole thing into a string.

That string is pasted into the system prompt, under a heading. What you already know about this user. The model reads it as plain text, like any other input. Notice what is missing.

Nothing picks which lines matter. Every line goes in, on every call. There is no search step here. And because it is ordinary text, you can open the file yourself, read it, fix a line, delete one.

Nothing is hidden. So, a simple memory log is a file. Append a line after a turn, read the whole file back at start up, paste it in. That is all.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is wrong with a simple text log? Let's try to understand. In a simple way. The log has one rule.

Append. Every turn can add a line, and nothing ever takes one away. Months later, it is two thousand lines. And remember how it is used.

Your code pastes the whole file into the prompt. All two thousand lines, before every message. The context window does not grow with it. It is fixed.

Eventually the memory alone fills it, and the conversation is pushed out. You pay for it too. Two thousand lines sent again every turn, billed every turn, for a question that needed one line. One line matters right now.

The rest is noise. Bury a fact in a wall of text, and the model skims past it. Append only has a second cost. You said vegetarian in March, and eat fish in July.

Both lines sit there, and nothing says which is current. So the fix looks obvious. Stop sending everything. Send the three lines that matter for this message.

Which turns memory into a search problem. The first idea is to search the file for the words in the question. Plain text matching, like control F. But watch it break.

The user asks, where should we eat tonight. The line says, Maya is vegetarian. No shared words. The search finds nothing.

So a text log fails twice. Too big to send whole, and words alone cannot find meaning. Memory has to be searchable by meaning. Quick check now.

One question is coming up. Let's see if it clicked.

How does turning a memory into a coordinate point work? Let's try to understand. In a simple way. Searching memories by words fails.

So give every memory a position instead. A place on a map, decided by what the line means. Here is how a line gets its position. You send the text to a small model called an embedding model.

Out comes a list of numbers. That list is the position. Two numbers would place a dot on a page. The list does the same thing.

They are coordinates. Real ones are longer. Often seven hundred sixty eight numbers. You cannot draw that many directions, but nothing changes.

It is still one point. What decides the numbers? Training. The embedding model read huge amounts of text, and learned to place lines with similar meaning in similar spots.

So, Maya is vegetarian, and, she avoids meat, land almost on top of each other. Not one word is shared. The meaning put them there. It cuts the other way too.

Maya is vegetarian, and, Maya loves steak, share almost every word, and still sit far apart. One rule matters. Every line must go through the same embedding model. Two models draw two different maps, and their numbers cannot be compared.

Your code does this once, as the memory is written, and stores the numbers beside the original line. A point cannot be turned back into a sentence, so the words stay. So an embedding turns a memory into a point. Same meaning, nearby points.

The words stop mattering. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a vector database? Let's try to understand. In a simple way. Every memory is now a long list of numbers.

Those lists need somewhere to live. That somewhere is a vector database. Picture one row inside it. The vector, the sentence it came from, and a little extra, like a date.

All kept together. A normal database answers exact questions. Find the row where the name equals Maya. This one answers, which stored points sit nearest to mine.

With ten memories you could just compare against all ten. With two million, checking every point on every turn is far too slow. So the store builds an index over the vectors, up front. It groups nearby points together, so a lookup visits a few neighborhoods, not everything.

That index is approximate. Now and then it misses a true nearest point. You trade a sliver of accuracy for an answer in milliseconds. The ordinary fields still work.

You can ask for the nearest points, but only this user's, and only from last month. Filters, plus nearness. And it stays live. Write a new memory and it is inserted.

Change one, the vector is replaced. Forget one, the row is deleted. One warning. The database does not understand meaning.

The embedding model chose the positions. The store only keeps them, and finds neighbors fast. So, a vector database is a store for points, with the words beside them, and an index that makes nearby fast. Quick check now.

One question is coming up. Let's see if it clicked.

How does finding a similar memory work? Let's try to understand. In a simple way. You type a new line.

First, that line goes through the same embedding model your memories went through. Your question becomes a point too. So the search is not word matching. It is geometry.

Which stored points sit closest to the point your question just made? The store compares the two lists of numbers and gives every memory a score. Near one means almost the same meaning. Near zero means unrelated.

It sorts by that score and hands back only the top few. Usually three to five. That count is called k, and you pick it. Why not take everything?

Because every memory scores something. Take too many and the useful line is buried in weak ones. So you also set a floor. If even the best score sits under it, the agent takes nothing.

An empty result beats a wrong memory. One catch. A question and a stored statement are worded differently, so the closest point is not always the right one. Some agents rewrite the question first.

And similar is not true. If two memories contradict each other, both can come back. Nearness ranks meaning, never correctness or freshness. What comes back is not the numbers.

Every row kept its original sentence, and that sentence is what the agent receives, with its score. So, finding a memory means turning the question into a point, ranking stored points by nearness, and keeping the best few. Quick check now. One question is coming up.

Let's see if it clicked.

How does thinking with a retrieved memory work? Let's try to understand. In a simple way. The search hands back three old sentences.

The model has not seen them yet. They are just text sitting in your program. So your code pastes those lines into the prompt, above the new message, before the model runs. That paste is the whole act of remembering.

Do not drop them in bare. Put a header above them, like, notes from earlier chats. Unlabeled, an old line reads like a fresh command. Then add one instruction under the block.

Use these notes if they help. Without that line, the model often just repeats them back at the user. Now the reply lands. The agent says, booking the window seat again.

It never learned that. It read it, one moment ago. And nothing sticks. The weights never changed.

Next turn you search again, and paste again, building a fresh block for that question. Two memories can disagree. Window seat in March. Aisle seat last week.

So stamp every line with a date, and tell the model to trust the newest. Retrieval hands back its best few even when none of them fit. So say it plainly. If none of these apply, ignore them.

And each pasted line costs tokens on every turn. That is why agents paste three to five short memories, not thirty long ones. So, using a memory means pasting the retrieved lines into the prompt, labeling them, and telling the model how to lean on them. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is an importance score? Let's try to understand. In a simple way. Your agent saves everything.

I am allergic to peanuts. Nice weather today. To a nearest point search, those two lines look equal. So when a line is saved, the agent attaches a number beside it.

One to ten. It sits in the row, like the date does. Who picks it? Usually a second model call.

A tiny prompt asks how important this line is for remembering this person. Your code reads the digit back. A ten is something still true next month. An allergy.

A deadline. A one is small talk. The weather, or thanks, that was great. Now the search changes.

Nearness alone no longer picks. Three numbers do. How similar, how important, and how recent. They are added up, each with a weight, and the top few by that total go into the prompt.

A slightly less similar line can win, if it matters more. Those weights live in your code, not in the model. Turn importance up and the allergy stays near. Turn it down and the agent drifts back to chit chat.

One catch. Ask a model to rate, and almost everything comes back an eight. So show it two examples, a ten and a one, inside the prompt. Importance is judged once, at writing time.

Recency keeps moving on its own, but that number never gets a second look. So, importance is a number stored beside a memory, and retrieval ranks by similarity, importance and recency together. Quick check now. One question is coming up.

Let's see if it clicked.

How does creating a new memory work? Let's try to understand. In a simple way. A conversation is not a memory.

It is hellos, thanks, and one useful line buried in the middle. Saving the whole thing is the log that failed. So your code makes a second model call. It hands over the last few turns and asks, what here is worth keeping?

Back comes a short list of facts, each rewritten to stand alone. Not she moved there, but Maya moved to Berlin. Next Friday becomes a date. Why?

This line gets read months later with no chat around it. And one fact per line, so a search can match just one piece. When does it run? Not on every word.

At the end of a turn, or when the session closes. It is a paid call. Before writing, the agent searches its own store with the new fact. If something very close is already there, no second copy is added.

Three outcomes then. Add, if it is new. Update, if it replaces an older line. Skip, if the store already says it.

Only a kept line becomes a point and a row, with its date beside it. From the next question on, it can be found. And often the honest answer is nothing worth keeping. The extractor must be allowed to return an empty list, or it invents facts.

So, a new memory is a second call that pulls standalone facts from the chat, checks the store, and writes only what is new. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is forgetting in an agent? Let's try to understand. In a simple way. Here is the quiet problem.

A memory store only grows. Every session adds rows, and nothing ever takes one out. But retrieval only pastes the top few. Five seats, whether the store holds one hundred rows or fifty thousand.

The seats never grow. So every row saved is a competitor. Save one fact five different ways, and those five near copies can take all five seats. And an old line does not look old.

I live in Delhi sits right beside I live in Berlin. Nearness cannot tell which one is still true. It also costs. A bigger index searches slower, every row takes storage, and junk that wins a seat is pasted into a prompt you pay for.

Now notice. Nothing removes a row on its own. A vector store keeps whatever you insert, forever, until your code calls delete. So forgetting is not something the model does.

It is a job you schedule. Code that walks the rows and takes some out. Two moves do most of the work. Drop a row that is dead.

Or merge a cluster of near copies into one clean line. Careful though. Delete the allergy line and it is gone for good. So mark the row hidden instead, and keep the text somewhere cold.

So, agents forget because retrieval has a fixed number of seats, and a store that only grows fills those seats with noise. Quick check now. One question is coming up. Let's see if it clicked.

How does memory decay work? Let's try to understand. In a simple way. Every memory row already carries a timestamp.

The text, the point, and the moment it was written. At search time your code subtracts. Now, minus that stamp. Every memory has an age, counted in hours.

That age turns into one number between zero and one. A minute old scores nearly one. A year old scores nearly zero. The turning is a multiplication.

Take zero point nine nine five, and raise it to the number of hours. So the drop is a curve, steep at first, then flattening out. The one knob is the half life. How long until a memory is worth half of what it was.

Hours for a support bot, months for a companion. Now the important part. Nothing is deleted. Fading only shrinks the recency number, and that number is added into the ranking.

The row sits in the store, untouched. So a faded memory still comes back, if nothing else is close. It drops out only when a fresher line outscores it for one of the few seats. And when a memory is retrieved, your code writes a fresh timestamp on it.

Used memories stay young. The ones nobody needs keep sinking. One more thing. Importance never decays.

A nine stays a nine, so the allergy line holds its place long after the small talk has faded. So, fading is a recency score that shrinks with the clock, ranking old memories down instead of removing them. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a memory that no longer applies? Let's try to understand. In a simple way. Old is not the same as wrong.

A line written this morning can already be false. A line from last year can still be true. The clock cannot tell them apart. Three things make a memory stop applying.

It got replaced. It ran out of time. Or the job it belonged to is over. So the check happens at write time.

Before saving a new fact, your code searches for lines about the same thing. If the new line answers the same question differently, that is a supersede. Write the new row, and mark the old one replaced. For facts that die on a date, write that date when you save.

An extra column. Valid until. Your code reads it. For facts that belong to one job, tag the row with that job.

When the job closes, one sweep retires every row with the tag. None of this deletes anything. Each row carries a status, active or retired. Retrieval just adds a plain filter, status equals active.

Dates and tags your code handles alone. Messy pairs need a second model call, holding both lines, answering one question. Does the new line replace the old. One trap.

Home city is a slot, so a new answer replaces the old. But allergies are a list. Replacing there deletes something still true. So, relevance forgetting retires a row when a newer fact, a date, or a finished job says it no longer holds.

Quick check now. One question is coming up. Let's see if it clicked.

How does an agent's personality work? Let's try to understand. In a simple way. After training, the weights are frozen.

Every user talks to the exact same model. So a personality cannot be sitting inside it. What differs is the text in front of the model. Your retrieved memories are pasted above your message, every single turn.

And the model treats those lines like instructions. Keeps answers short. Writes in Python. Never calls you buddy.

It just follows what it reads. A hand written persona is fixed. A memory built personality is written by your conversations, and it keeps growing. Why does it feel like the same someone every time?

Because the same lines come back. Personality here is really just repeatability. But retrieval only pulls a handful of lines. Ask about code, and the coding lines come back.

So the character you meet depends on the question. Same weights, two stores, two different agents. One clipped and technical, one chatty. Nothing inside the model changed.

Here is the trap. One passing remark can be saved as a permanent trait. You say no emojis once, and it never uses one again. And a trait lives only as long as its line survives.

Retire that row, and the habit disappears. The character is stored, not learned. So, an agent's personality is not in the weights. It is the set of lines your memory keeps handing it.

Quick check now. One question is coming up. Let's see if it clicked.