EXPLAINER

Why your AI forgets — and how each one fakes remembering.

Claude chat, Claude Code and ChatGPT all sit on a model that remembers nothing. Six short animations show how each one works around that — no maths required.

September 2026 · 7 minute read · Simplified from a longer technical reference

1The model has no memory. None.

Here is the single fact that everything else follows from. A language model is a pure function: you hand it text, it hands you back the next likely word. Nothing about you survives between two requests. There is no session, no notebook, no "as we discussed".

So when an assistant says "as you mentioned earlier", it isn't remembering. It is re-reading — because your whole conversation was just sent to it again, from the first message onwards.

The file gets thicker every turn. The model reads it fresh every time.

In plain terms. Picture a brilliant consultant with zero memory. Every question, you slide the entire case file under the door. He reads it from page one, writes one paragraph back, and forgets everything the moment you take the file away. The "relationship" is an illusion produced by the file getting thicker. The context window is simply the thickest file he can read in one sitting.

2The window is a budget, not a container

Text is counted in tokens — roughly three-quarters of a word each. But your messages are only one line-item. Everything the product quietly packs into the request counts against the same total, and the reply being written comes out of it too.

Everything in the request, plus the answer, shares one number.

3Every turn re-sends everything — caching makes that affordable

Nothing stays open on the server between turns. The app rebuilds the whole conversation and sends it again. Turn 3 contains turns 1 and 2 verbatim. That sounds wasteful, and it would be, except for one trick.

Prompt caching. If the start of this request is byte-for-byte identical to the start of the last one, the model reuses its earlier work on that part. The catch: it is a prefix match. Change one character near the top and everything below it is recomputed.

Only the new lines cost full price — until one edit near the top resets it all.

In plain terms. A courtroom where the stenographer must read the entire transcript aloud before each new question. Caching is the judge saying "we all remember pages 1 to 400 from this morning — start at 401." But only if pages 1 to 400 are word for word what they were. Fix a typo on page 3 and the reading starts from the top.

4What happens when the file gets too thick

Eventually the conversation outgrows the window. Every product has to do something. There are really only three moves, and the products differ mainly in which they pick and whether they tell you.

Evict, compact, or keep an index and fetch. Every product is a mix of these.

The one pattern worth stealing. Your desk holds six open folders. Behind you is a cabinet with four hundred. You don't try to fit the cabinet on the desk — you tape a one-page index to it and pull a folder when a meeting needs it. The desk is the window and it is exactly as big today as yesterday, no matter how full the cabinet gets. When a client says "the assistant must remember our whole account history," they think they need a bigger desk. They need a cabinet and an index.

5Sub-agents: let someone else read the thick file

Research is expensive in tokens and cheap in conclusions. You might read 6,000 tokens of source to produce a 400-token finding. Keeping the 6,000 in your window afterwards is pure cost.

So Claude Code hands the reading to a sub-agent with its own fresh window. It reads whatever it needs, and only its final answer comes back to you. The general rule: isolate any step whose inputs are far bigger than its outputs.

Big reading happens elsewhere. Only the conclusion comes home.

6Same physics, three different contracts

Strip away the branding and all three products face one constraint: a fixed window over a function with no memory. They differ on one axis that matters to you — whether they tell you when something was dropped.

	Claude chat	Claude Code	ChatGPT
When the window fills	Oldest turns quietly leave	History replaced by a summary; project files reloaded from disk	Oldest turns dropped or condensed
Are you told?	No	Yes — visible compaction, /context breakdown	No
Across sessions	Memory files, past-chat search, projects	CLAUDE.md, MEMORY.md, rules, skills — re-read from disk	Saved memories, chat-history insights, project memory
Biggest token hog	Uploaded docs and long generated artifacts	File reads and tool output	Pasted content and long threads
Effective window	The model's full window	200K by default, 1M on supported models	Tiered by plan and mode — well below the API window

ChatGPT's consumer design favours uninterrupted flow: the thread never refuses to continue, it quietly truncates and carries on. Claude Code favours legibility: you are interrupted, but you know exactly what happened. Neither is wrong. Which is right depends on whether the user wants a conversation or a controllable system.

One governance point for regulated teams: in ChatGPT, deleting a chat does not delete memories derived from it. A fact extracted from a conversation can outlive it and surface in an unrelated future chat. That is a data-flow decision to make before rollout, not after an incident.

Bigger is not better

A one-million-token window is not the end of the problem. As the window fills, recall degrades — Anthropic calls this context rot. Material at the very start and very end of a long prompt is remembered best; things buried in the middle are weakest. A big window full of noise underperforms a small one kept clean. That is why curating what goes in matters at least as much as how much room there is.

The playbook

The chat is getting slow, vague or forgetful
Start a new one and paste a short brief. You are swapping 80,000 tokens of noise for 400 of signal.
Working through a long document
Put it at the top, above your question. Ask for the relevant quotes before the analysis.
Agentic work with heavy exploration
Delegate the reading to a sub-agent. Isolate any step whose inputs dwarf its outputs.
Switching to an unrelated task
Clear, don't compact. A summary of finished work is still tokens you pay for every turn.
Your own app's cache hit rate is poor
Look for a timestamp, session id or rotating instruction above the breakpoint. Stable content first.
A client asks for "memory of everything"
Sell them a working set, not a bigger window: durable store, index in the prompt, retrieval on demand.

Simplified from a longer technical reference verified against Anthropic's context-window, caching, compaction and sub-agent documentation in September 2026. ChatGPT's internal truncation behaviour is not published by OpenAI and is described here from observed behaviour; its window sizes change by plan and mode.

xploreai builds WhatsApp bots, booking automation and lead-gen AI for clinics, real-estate firms, schools and hotels in Mumbai — most of it on exactly the working-set pattern above.

Talk to us