--- title: "Budget an AI context window so critical information stays in view" description: "See what fills an AI context window and why the oldest content drops out, then budget the window so the facts you need stay in view." canonical: "https://scalewithsearch.com/articles/ai-context-window-how-ai-loses-information" date: "2026-01-27" modified: "2026-09-25" --- ## Site navigation - [Scale With Search](https://scalewithsearch.com/) - Real estate - Real estate - [Real estate](https://scalewithsearch.com/for/real-estate) - Work - Start here - [Send your brief](https://scalewithsearch.com/work#send-your-brief) - [Prepare your six-question brief](https://scalewithsearch.com/work#prepare-your-six-question-brief) - Build - [Site build, content library with SEO, signal desk](https://scalewithsearch.com/work) - For your business - Trades and home services - [Auto body and collision shops](https://scalewithsearch.com/for/auto-body-and-collision-shops) - [Foundation and home repair contractors](https://scalewithsearch.com/for/foundation-and-home-repair) - [Garage door and fencing contractors](https://scalewithsearch.com/for/garage-door-and-fencing-contractors) - [HVAC contractors](https://scalewithsearch.com/for/hvac-contractors) - [Janitorial and commercial cleaning companies](https://scalewithsearch.com/for/janitorial-and-commercial-cleaning) - [Locksmiths](https://scalewithsearch.com/for/locksmiths) - [Moving companies](https://scalewithsearch.com/for/moving-companies) - [Pest control companies](https://scalewithsearch.com/for/pest-control-companies) - [Plumbing and electrical contractors](https://scalewithsearch.com/for/plumbing-and-electrical-contractors) - [Restoration and water or fire damage companies](https://scalewithsearch.com/for/restoration-and-water-fire-damage) - [Roofing companies](https://scalewithsearch.com/for/roofing-companies) - [Towing companies](https://scalewithsearch.com/for/towing-companies) - [Tree services and landscaping companies](https://scalewithsearch.com/for/tree-services-and-landscaping) - [Solar installers](https://scalewithsearch.com/for/solar-installation) - [General contractors](https://scalewithsearch.com/for/general-contractors-and-construction) - [Paving, concrete, and flooring contractors](https://scalewithsearch.com/for/paving) - Practices and professional services - [Bookkeeping and tax practices](https://scalewithsearch.com/for/bookkeeping-and-tax-practices) - [Dental practices](https://scalewithsearch.com/for/dental-practices) - [Family and criminal defense law firms](https://scalewithsearch.com/for/family-and-criminal-defense-law-firms) - [Med spas and aesthetics practices](https://scalewithsearch.com/for/med-spas-and-aesthetics) - [Personal injury law firms](https://scalewithsearch.com/for/personal-injury-law-firms) - [Veterinary clinics](https://scalewithsearch.com/for/veterinary-clinics) - [Gyms and fitness studios](https://scalewithsearch.com/for/fitness) - [Therapy and outpatient health practices](https://scalewithsearch.com/for/therapy-and-outpatient-health) - [Medical billing companies](https://scalewithsearch.com/for/medical-billing) - [Insurance agencies](https://scalewithsearch.com/for/insurance-agencies) - [Financial advisors](https://scalewithsearch.com/for/financial-advisors) - [Property management companies](https://scalewithsearch.com/for/property-management) - [Recruiting and staffing agencies](https://scalewithsearch.com/for/recruiting-and-staffing) - [Architects and interior designers](https://scalewithsearch.com/for/architects-and-interior-designers) - [Logistics and supply chain companies](https://scalewithsearch.com/for/logistics-and-supply-chain) - Agencies, MSPs, and manufacturing - [IT and managed service providers](https://scalewithsearch.com/for/it-and-managed-service-providers) - [Machine shops and precision manufacturers](https://scalewithsearch.com/for/machine-shops-and-precision-manufacturing) - [Marketing agencies and freelancers](https://scalewithsearch.com/for/marketing-agencies-and-freelancers) - [SEO agencies and consultants](https://scalewithsearch.com/for/seo-agencies-and-consultants) - [Small manufacturers and fabricators](https://scalewithsearch.com/for/small-manufacturers-and-fabricators) - Restaurants, shops, studios, and nonprofits - [Restaurants and hospitality businesses](https://scalewithsearch.com/for/restaurants-and-hospitality) - [Retail stores and ecommerce sellers](https://scalewithsearch.com/for/retail-and-ecommerce) - [Photographers, event planners, and travel agents](https://scalewithsearch.com/for/photographers) - [Churches and nonprofits](https://scalewithsearch.com/for/churches-and-nonprofits) - [All industries](https://scalewithsearch.com/for/) - Learn - For your office - [Office job guides](https://scalewithsearch.com/guides/) - [Browser calculators](https://scalewithsearch.com/tools/) - Start here - [How it works](https://scalewithsearch.com/how-it-works) - [Free Starter Kit](https://scalewithsearch.com/kit/business-memory-starter-kit.zip) - [Synthetic specimen](https://scalewithsearch.com/specimen/working-session-specimen.zip) - Guides - [The Complete Guide to Business Memory for AI Agents](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide) - [The Complete Small-Business Guide to AI Agent Governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business) - [The Complete Guide to Leaving Vendor AI Memory](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide) - Articles by cluster - [Business memory](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide) - [Agent governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business) - [Migration and ownership](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide) - For machines - [llms.txt](https://scalewithsearch.com/llms.txt) - [llms-full.txt](https://scalewithsearch.com/llms-full.txt) - [Machine view](https://scalewithsearch.com/?view=machine) - Company - Evidence - [Proof](https://scalewithsearch.com/proof) - Company - [About](https://scalewithsearch.com/about) # Budget an AI context window so critical information stays in view. Thirty messages into a project conversation, the assistant asks you to explain a decision you made in the first ten minutes. The earlier message is still on your screen. The model behaves as if it never existed. The context window explains that failure. It also explains inconsistent output between chats and the constant need to re-explain your business. When you know how the window fills and empties, you can plan around it instead of fighting it. ## What a context window is A context window is the maximum amount of text a model can use for one response. Think of it as the model's working memory: everything it can reference while it writes the next answer. The model has nothing before the window, and it keeps nothing after it. For any single response, the window is the entire set of information available. The contents add up like this: ```text system prompt + custom instructions + conversation history + uploaded files + tool results + space for the response = context window If the total exceeds the window size: the product drops or compacts the oldest content ``` Anthropic's [context windows guide](https://platform.claude.com/docs/en/build-with-claude/context-windows) defines the window the same way. The window is all the text a model can reference when it generates a response. That text includes system instructions, messages, documents, tool definitions, tool results, and output. ## Count in tokens, not words Context windows are measured in tokens. A token is the basic unit of text the model processes. In common English text, a token is often a whole short word or a piece of a longer one. The ratio varies. A short common word is often one token. A long or rare word splits into several. URLs, code, technical terms, and non-English text use more tokens per word than plain prose. A 1,000-word technical document can need noticeably more tokens than 1,000 words of simple text. Token counts also differ between vendors, because each vendor uses its own tokenizer. When the number matters, count with the vendor's own token counter. ## List what consumes the window Your messages are not the only content in the window. A typical session holds five kinds of content. The system prompt comes first. The provider's instructions define behavior, safety rules, and tool use. You do not see this text in a consumer product, and most vendors do not publish its size. Custom instructions come next. As checked on 2026.09.19, OpenAI allowed 1,500 characters of custom instructions for Free and Go accounts and 5,000 for paid plans. Source: [ChatGPT Custom Instructions](https://help.openai.com/en/articles/8096356-chatgpt-custom-instructions). Conversation history is the third part. Every message in the session counts, yours and the model's. Responses often use more tokens than prompts. Uploaded files and tool results are the fourth. A long PDF can consume tens of thousands of tokens. Search results and tool output also count. The response buffer is the fifth. The model needs room to write. If the window is full, the response can be cut short. A worked example shows the effect. Suppose a window holds 128,000 tokens. The system prompt, instructions, and two uploaded reports take 48,000. About 80,000 tokens remain for the conversation and the answer. The limit arrives sooner than the headline number suggests. ## Know what happens at the limit When a conversation grows past the window, something has to go. Most products drop the oldest content or compress it into a summary. The exact method depends on the product. The model does not know what it lost. It has no record that earlier content existed. From its view, the conversation starts wherever the window now begins. That is why an assistant can ask you to explain something you covered thirty messages ago. Those messages are no longer in its input. The interface still shows them to you, because the chat history and the context window are separate. The history is for you. The window is what the model processes. Degradation is often sudden. A long conversation works well until it does not. Watch for the first sign that the model has lost an early decision, and act then. ## Why windows have limits The limit is not arbitrary. Transformer models, the architecture behind GPT, Claude, and similar models, use self-attention. In the standard form, each token is compared with every other token, so compute grows with the square of the input length. Source: Vaswani and colleagues, [Attention Is All You Need](https://arxiv.org/abs/1706.03762). | Context length | Attention compute relative to 8,000 tokens | |---|---| | 8,000 tokens | 1 times | | 32,000 tokens | 16 times | | 128,000 tokens | 256 times | | 1,000,000 tokens | about 15,600 times | The table covers the attention step of a standard transformer only. Vendors use optimizations, so real costs differ. The direction holds: long inputs cost more compute, memory, energy, and hardware. That sets a practical ceiling on what a vendor can offer at scale. ## Bigger is not automatically better A larger window costs more to run. It also does not guarantee that the model uses all of it well. Liu and colleagues, in [Lost in the Middle](https://arxiv.org/abs/2307.03172), found that performance is often highest when relevant information sits at the start or end of the input. It degrades when the model must use information in the middle of a long context. Hsieh and colleagues tested long-context models with the [RULER benchmark](https://arxiv.org/abs/2404.06654) in 2024. Every model they tested claimed a context size of 32,000 tokens or greater. Their abstract reports that only half kept satisfactory performance at 32,000 tokens. Treat an advertised window as a ceiling, not as a promise of recall. ## The session boundary is a second limit The window explains loss inside one session. A second loss happens between sessions. When you close a conversation and start a new one, the working set starts again. The new session does not contain the old conversation. Some products carry selected facts across chats through a memory feature. That feature is a separate layer with its own rules. The comparison of [context windows and persistent AI memory](/articles/context-window-vs-persistent-ai-memory) shows how to test which layer keeps which fact. ## Three approaches to context management The mechanism points to three approaches. Summarization compresses older content to save tokens. It works, but it loses nuance. A model's summary of what happened is less useful than what happened. Retrieval-augmented generation (RAG) stores information outside the model and retrieves only the passages that match the current question. You load what matters for this request instead of everything. Retrieval still needs a rule for which source is current. The article [RAG is retrieval; business memory also needs authority](/articles/rag-vs-business-memory) covers that gap. Persistent context files load baseline context at the start of every session. The model reads your preferences, business facts, and relevant history without you retyping them. The files stay small, so they leave most of the window for the work. ## Plan the session around the window Use five practices. 1. Put critical context where the product keeps it: project instructions or a loaded context file. An early chat message is the first thing to drop. 2. In a long thread, restate the key facts near your request. The end of the input is a strong position. 3. Watch for the first lost decision. When it appears, write a checkpoint and continue in a new session. The guide to [preserving working state when ChatGPT forgets in the same chat](/articles/why-chatgpt-forgets-same-chat) shows the checkpoint format. 4. Keep unrelated work in separate sessions. Mixed topics waste window space. 5. Build the context system outside the chat. A [small-business memory architecture](/articles/ai-agent-memory-architecture-small-business) keeps sources, corrections, and approvals in records you control. ## Test your own tool You can see the limit in your own product in about twenty minutes. 1. Start a new conversation and state one harmless, specific fact, such as "The test code is ORCHID-7." 2. Paste several long, unrelated documents into the conversation. 3. Ask for the test code. 4. Repeat with more text until the answer fails or changes. 5. Note the product, the model, the date, and the approximate amount of text at failure. The result is specific to that product and date. Rerun it after a product update, and keep the facts that must survive in files the next session reads. ## Related: AI memory - [Compare eight AI context limit workarounds and keep the ones that last](/articles/ai-context-limit-workarounds) - [Restart a long AI conversation before its answers get worse](/articles/why-does-ai-get-worse-in-long-conversations) - [Find out why AI lost a detail mid-conversation and restore it](/articles/why-does-ai-lose-context-mid-conversation) ---- ```text .|########||. .|########||. .|########||. |##||. .||##|. |##||. .||##|. |##||. .||##|. |#|. .|#|. |#|. .|#|. |#|. .|#|. |#| |#| |#| |#| |#| |#| .#| |#. .#| |#. .#| |#. .#. .#| .#. .#| .#. .#| |#. .#| |#. .#| |#. .#| |# ...... #| |# ...... #| |# ...... #| .# ||#########| #| .# ||#########| #| .# ||#########| #| .#. |######||######|. .#. .#. |######||######|. .#. .#. |######||######|. .#. #. .##|###|##|#|######| .# #. .##|###|##|#|######| .# #. .##|###|##|#|######| .# || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# # .||####|###########################|. # # .||####|###########################|. # # .||####|###########################|. # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. #.|||||||#############################| ||| .# #.|||||||#############################| ||| .# #.|||||||#############################| ||| .# || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. || |#||||||||||||||||||#| || || |#||||||||||||||||||#| || || |#||||||||||||||||||#| || #. .######|#|#########| .# #. .######|#|#########| .# #. .######|#|#########| .# .#. |#####||||#####| .#. .#. |#####||||#####| .#. .#. |#####||||#####| .#. |# ||########|| #. |# ||########|| #. |# ||########|| #. |# ...... #| |# ...... #| |# ...... #| |#. .#| |#. .#| |#. .#| |#. .#. |#. .#. |#. .#. .#| |#. .#| |#. .#| |#. |#| |#| |#| |#| |#| |#| .|#|. .|#|. .|#|. .|#|. .|#|. .|#|. |##||. .||##| |##||. .||##| |##||. .||##| .||########||. .||########||. .||########||. Scale With Search 2026 [scalewithsearch.com](https://scalewithsearch.com) ```