
RAG: retrieval-augmented generation explained
In short: RAG retrieves relevant documents at query time and gives them to an LLM so answers draw on your data: how it works, an example and why it fails.
Retrieval-augmented generation (RAG) is a pattern where a system first searches a collection of documents for passages relevant to a question, then gives those passages to a language model to answer from. It lets models use private or recent information they were never trained on, and lets answers cite sources. The term comes from a 2020 Facebook AI research paper.
RAG in plain words
An open-book exam. The model does not need to have memorised your company handbook; it looks up the right pages, then writes the answer.
How it works
- Index: split documents into chunks, create embeddings, store them in a vector index (often alongside a keyword index).
- Retrieve: turn the question into a query, find the most relevant chunks.
- Rerank (optional): reorder candidates with a more precise model.
- Generate: put the top chunks in the prompt and ask the model to answer only from them, with citations.
The original design, as Meta's September 2020 write-up describes it, had the same shape: retrieve documents for the input, then pass both to a generator.
Example
As an illustration, "What is our parental leave policy in India?" retrieves the two HR policy sections tagged India, and the model answers in three sentences citing section numbers. Without retrieval, it would guess from general knowledge.
Where you'll meet it
- Internal knowledge assistants and support bots
- Documentation search with AI answers
- Enterprise search products and AI features in SaaS tools
- AI search engines that cite web pages
What it changes for you
For developers, answer quality depends far more on retrieval (chunking, search, reranking) than on the model. For data teams, document hygiene, permissions and freshness become AI problems. For leaders, RAG is usually the first enterprise AI system worth building, and the first to fail without anyone noticing when content goes stale.
RAG vs fine-tuning
RAG gives the model the right facts at question time. Fine-tuning changes how the model behaves. Use RAG for knowledge, fine-tuning for style or skills.
Common mistake
My view: blaming the model when RAG answers are wrong. Check whether the right passage was retrieved at all. A wrong RAG answer usually starts as a retrieval miss, so debug the search before the prompt.
Related terms
Embeddings · Vector database · Hybrid search · Reranking · Chunking · Agentic RAG
From the archive: Why your RAG pipeline returns irrelevant results · Fine-tune to change behaviour, use RAG to change knowledge
When a RAG answer is wrong, check what was retrieved before you touch the prompt.


