--- title: "Decide whether your business needs retrieval-augmented generation" description: "RAG lets an AI search your documents before it answers. Learn how the retrieval pipeline works, what it costs to run, and when a context file is enough." canonical: "https://scalewithsearch.com/articles/what-is-retrieval-augmented-generation" date: "2026-01-28" modified: "2026-09-25" --- ## Site navigation - [Scale With Search](https://scalewithsearch.com/) - Real estate - Real estate - [Real estate](https://scalewithsearch.com/for/real-estate) - Work - Start here - [Send your brief](https://scalewithsearch.com/work#send-your-brief) - [Prepare your six-question brief](https://scalewithsearch.com/work#prepare-your-six-question-brief) - Build - [Site build, content library with SEO, signal desk](https://scalewithsearch.com/work) - For your business - Trades and home services - [Auto body and collision shops](https://scalewithsearch.com/for/auto-body-and-collision-shops) - [Foundation and home repair contractors](https://scalewithsearch.com/for/foundation-and-home-repair) - [Garage door and fencing contractors](https://scalewithsearch.com/for/garage-door-and-fencing-contractors) - [HVAC contractors](https://scalewithsearch.com/for/hvac-contractors) - [Janitorial and commercial cleaning companies](https://scalewithsearch.com/for/janitorial-and-commercial-cleaning) - [Locksmiths](https://scalewithsearch.com/for/locksmiths) - [Moving companies](https://scalewithsearch.com/for/moving-companies) - [Pest control companies](https://scalewithsearch.com/for/pest-control-companies) - [Plumbing and electrical contractors](https://scalewithsearch.com/for/plumbing-and-electrical-contractors) - [Restoration and water or fire damage companies](https://scalewithsearch.com/for/restoration-and-water-fire-damage) - [Roofing companies](https://scalewithsearch.com/for/roofing-companies) - [Towing companies](https://scalewithsearch.com/for/towing-companies) - [Tree services and landscaping companies](https://scalewithsearch.com/for/tree-services-and-landscaping) - [Solar installers](https://scalewithsearch.com/for/solar-installation) - [General contractors](https://scalewithsearch.com/for/general-contractors-and-construction) - [Paving, concrete, and flooring contractors](https://scalewithsearch.com/for/paving) - Practices and professional services - [Bookkeeping and tax practices](https://scalewithsearch.com/for/bookkeeping-and-tax-practices) - [Dental practices](https://scalewithsearch.com/for/dental-practices) - [Family and criminal defense law firms](https://scalewithsearch.com/for/family-and-criminal-defense-law-firms) - [Med spas and aesthetics practices](https://scalewithsearch.com/for/med-spas-and-aesthetics) - [Personal injury law firms](https://scalewithsearch.com/for/personal-injury-law-firms) - [Veterinary clinics](https://scalewithsearch.com/for/veterinary-clinics) - [Gyms and fitness studios](https://scalewithsearch.com/for/fitness) - [Therapy and outpatient health practices](https://scalewithsearch.com/for/therapy-and-outpatient-health) - [Medical billing companies](https://scalewithsearch.com/for/medical-billing) - [Insurance agencies](https://scalewithsearch.com/for/insurance-agencies) - [Financial advisors](https://scalewithsearch.com/for/financial-advisors) - [Property management companies](https://scalewithsearch.com/for/property-management) - [Recruiting and staffing agencies](https://scalewithsearch.com/for/recruiting-and-staffing) - [Architects and interior designers](https://scalewithsearch.com/for/architects-and-interior-designers) - [Logistics and supply chain companies](https://scalewithsearch.com/for/logistics-and-supply-chain) - Agencies, MSPs, and manufacturing - [IT and managed service providers](https://scalewithsearch.com/for/it-and-managed-service-providers) - [Machine shops and precision manufacturers](https://scalewithsearch.com/for/machine-shops-and-precision-manufacturing) - [Marketing agencies and freelancers](https://scalewithsearch.com/for/marketing-agencies-and-freelancers) - [SEO agencies and consultants](https://scalewithsearch.com/for/seo-agencies-and-consultants) - [Small manufacturers and fabricators](https://scalewithsearch.com/for/small-manufacturers-and-fabricators) - Restaurants, shops, studios, and nonprofits - [Restaurants and hospitality businesses](https://scalewithsearch.com/for/restaurants-and-hospitality) - [Retail stores and ecommerce sellers](https://scalewithsearch.com/for/retail-and-ecommerce) - [Photographers, event planners, and travel agents](https://scalewithsearch.com/for/photographers) - [Churches and nonprofits](https://scalewithsearch.com/for/churches-and-nonprofits) - [All industries](https://scalewithsearch.com/for/) - Learn - For your office - [Office job guides](https://scalewithsearch.com/guides/) - [Browser calculators](https://scalewithsearch.com/tools/) - Start here - [How it works](https://scalewithsearch.com/how-it-works) - [Free Starter Kit](https://scalewithsearch.com/kit/business-memory-starter-kit.zip) - [Synthetic specimen](https://scalewithsearch.com/specimen/working-session-specimen.zip) - Guides - [The Complete Guide to Business Memory for AI Agents](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide) - [The Complete Small-Business Guide to AI Agent Governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business) - [The Complete Guide to Leaving Vendor AI Memory](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide) - Articles by cluster - [Business memory](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide) - [Agent governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business) - [Migration and ownership](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide) - For machines - [llms.txt](https://scalewithsearch.com/llms.txt) - [llms-full.txt](https://scalewithsearch.com/llms-full.txt) - [Machine view](https://scalewithsearch.com/?view=machine) - Company - Evidence - [Proof](https://scalewithsearch.com/proof) - Company - [About](https://scalewithsearch.com/about) # Decide whether your business needs retrieval-augmented generation. A customer asks your support assistant about the refund window for annual plans. The model answers with a confident 30 days. Your policy says 14. The model never saw your policy. It answered from general patterns in its training data. AI models do not know your business by default. They know what was in their training data, and that data stops at a fixed date. Your documents, processes, and customer records are not in it. Retrieval-augmented generation (RAG) is one way to close that gap. Before the model answers, a search step finds the relevant parts of your documents and adds them to the prompt. The model then answers from that material. This page explains how RAG works, what it costs to run, and how to decide whether you need it or a simpler context file. ## Follow the three steps of RAG RAG has three steps: retrieval, augmentation, and generation. **Retrieval.** You ask a question. Before the model answers, the system searches your knowledge base for relevant passages. A question about the refund policy finds the policy document. A question about a client finds that client's files. **Augmentation.** The system adds the retrieved passages to your question. The prompt becomes, in effect: "Here are the relevant policy passages. Answer this question from them." **Generation.** The model reads the passages and the question, then writes an answer based on both. It works from your source material instead of from general patterns. RAG reduces guessing. It does not remove it. The model can still misread a passage, merge two passages, or answer when the search returned the wrong document. Ask the system to cite the passages it used, so a person can check the answer. ## Know the parts of a RAG system A typical RAG system has five parts. | Part | Job | Examples | |---|---|---| | Chunker | Splits each document into passages of a set size | A script that cuts documents at headings or every few hundred words | | Embedding model | Converts each passage and each question into a list of numbers (a vector) that represents its meaning | OpenAI's text-embedding models; open-source Sentence Transformers | | Vector database | Stores the vectors and finds the ones closest to a question | Pinecone, Weaviate, Qdrant | | Retriever | Returns the top matches for a question, usually 3 to 10 passages, ranked by a relevance score | Vector search, keyword search, or both combined | | Language model | Writes the answer from the question and the retrieved passages | Any chat model with an API | The embedding model does its work twice. It converts your documents into vectors when you build the index. It converts each question into the same kind of vector when someone asks. Passages with similar meaning sit close together, so a question about "cancellations" can find a policy that says "refunds." ## Know where RAG helps **Large document sets.** Thousands of support tickets, hundreds of policy documents, or a long product catalog cannot all go into one prompt. RAG finds the few passages that matter for each question. **Changing information.** Product specs change each month. Policies change each quarter. The model's training data stays frozen, but a RAG index reads your current documents. Update a document and refresh the index, and the next answer uses the new text. **Many users with different access.** Each customer can get answers from their own account records. Each employee can get answers from their department's documents. One system serves everyone, and each person sees only the passages their permissions allow. That permission filter must run in the retriever, not in the prompt. ## Know where RAG adds work without benefit **Small knowledge bases.** If the whole knowledge base fits in the model's context window, you can load all of it. The search step then adds complexity and a chance to miss the right passage. **Stable information.** Brand voice, standard procedures, and your service list change rarely. A Markdown file that loads at the start of each session gives the model all of it without a search system. **One user.** A solo operator who wants the AI to know the business does not search thousands of documents. A well-organized context file is simpler to build and simpler to check than a vector database. ## Count what RAG costs to run RAG has three running costs. The exact figures change often, so check each vendor's pricing page before you plan. **Storage and queries.** Hosted vector databases such as Pinecone, Weaviate, and Qdrant charge for stored data and query volume. A small deployment costs little. The bill grows with the number of documents and questions. **Latency.** Each question now needs an embedding call and a search before the model starts to write. That adds delay to every request. Measure it in your own setup before you promise response times to customers. **Maintenance.** Someone must chunk new documents, refresh the index when documents change, and remove retired documents. Someone must also check retrieval quality over time and re-embed the collection when you change embedding models. A RAG system is a small software product, not a one-time setup. ## Size your documents against a context window Before you build anything, estimate whether your material fits in one prompt. A token is a piece of a word. One token averages about three quarters of an English word. **Worked example 1: a policy handbook.** Fifty pages at about 500 words per page is 25,000 words, or about 33,000 tokens. Claude Sonnet 4.5, released in September 2025, accepts 200,000 tokens. The handbook fits with room for the conversation. Load it as a file. You do not need RAG. **Worked example 2: a support archive.** Four thousand tickets at about 300 words each is 1.2 million words, or about 1.6 million tokens. That is larger than most context windows, and it grows every week. This archive needs a search step. RAG fits. A window that can hold your material is not a guarantee that the model uses all of it well. Recall of details in the middle of a long prompt is weaker than recall at the start and end. Keep the loaded material as small as the job allows. The [context window and persistent memory comparison](/articles/context-window-vs-persistent-ai-memory) explains the two limits. ## Use a context file first A context file is one Markdown document with your key information. It loads at the start of a session, so there is no index, no embedding step, and no search delay. You can read it, edit it, and compare its versions like any other document. A context file works when the content is stable and fits in the context window. Your business overview, writing rules, service list, and client roster belong in a context file, not in a RAG system. The guide to [what context an agent should read](/articles/what-context-should-an-agent-read) shows how to pick the smallest set of files for a job. You can combine the two approaches. Use a context file for stable information that every session needs. Use RAG for lookups in a large, changing document set. Most small businesses need the file. Fewer need RAG. ## Test retrieval before you trust it If you build RAG, test the search step on its own before you judge the answers. 1. Write 10 questions that your documents answer. 2. For each question, record the document that holds the correct answer. 3. Run each question through the retriever only. 4. Check whether the correct document appears in the top five results. 5. Record each miss and the document that ranked above it. 6. Fix the chunk size, the metadata, or the search method. 7. Run the same 10 questions again. A retriever that misses the right document cannot produce a correct answer, however good the model is. The [context-aware search guide](/articles/ai-knowledge-base-context-aware-search) covers further tests. Relevance is also not authority. A search can rank an old, detailed pricing page above a short current decision because the old page matches the question's words. The [RAG and business memory guide](/articles/rag-vs-business-memory) shows how to record which document governs and how to test that the current record wins. ## Decide with four questions Answer these four questions for the job in front of you: - Does the material exceed the context window, or will it soon? - Does the material change every week or month? - Do different users need different subsets of the material? - Can someone maintain an index, refresh it, and test it? If you answer no to the first three, start with a context file. If you answer yes to the first two and yes to the fourth, build RAG for that document set. Keep a context file for the stable rules. If you answer yes to the first three but no to the fourth, the system will drift after launch. Name an owner before you build. ## Related: AI memory - [RAG Is Retrieval. Business Memory Also Needs Authority.](/articles/rag-vs-business-memory) - [Best AI for Long Document Analysis: Run a Retrieval Test Before You Trust the Context Window](/articles/best-ai-long-document-analysis-business) - [Seed a Company List from Entity Search, Then Research Each One](/articles/entity-search-company-research-list) ---- ```text .|########||. .|########||. .|########||. |##||. .||##|. |##||. .||##|. |##||. .||##|. |#|. .|#|. |#|. .|#|. |#|. .|#|. |#| |#| |#| |#| |#| |#| .#| |#. .#| |#. .#| |#. .#. .#| .#. .#| .#. .#| |#. .#| |#. .#| |#. .#| |# ...... #| |# ...... #| |# ...... #| .# ||#########| #| .# ||#########| #| .# ||#########| #| .#. |######||######|. .#. .#. |######||######|. .#. .#. |######||######|. .#. #. .##|###|##|#|######| .# #. .##|###|##|#|######| .# #. .##|###|##|#|######| .# || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# # .||####|###########################|. # # .||####|###########################|. # # .||####|###########################|. # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. #.|||||||#############################| ||| .# #.|||||||#############################| ||| .# #.|||||||#############################| ||| .# || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. || |#||||||||||||||||||#| || || |#||||||||||||||||||#| || || |#||||||||||||||||||#| || #. .######|#|#########| .# #. .######|#|#########| .# #. .######|#|#########| .# .#. |#####||||#####| .#. .#. |#####||||#####| .#. .#. |#####||||#####| .#. |# ||########|| #. |# ||########|| #. |# ||########|| #. |# ...... #| |# ...... #| |# ...... #| |#. .#| |#. .#| |#. .#| |#. .#. |#. .#. |#. .#. .#| |#. .#| |#. .#| |#. |#| |#| |#| |#| |#| |#| .|#|. .|#|. .|#|. .|#|. .|#|. .|#|. |##||. .||##| |##||. .||##| |##||. .||##| .||########||. .||########||. .||########||. Scale With Search 2026 [scalewithsearch.com](https://scalewithsearch.com) ```