
Agentic RAG: an agent that decides what to search for
In short: Agentic RAG lets an AI agent plan searches, choose sources, judge results and search again before answering, at the cost of more model calls and latency.
Agentic RAG is retrieval-augmented generation where an AI agent controls retrieval instead of a fixed pipeline. The agent decides whether to search, which source or tool to use, how to phrase queries, whether the results are good enough and when to search again before it answers. An early research form is Self-RAG, submitted in October 2023, which retrieves passages on demand and critiques them.
Agentic RAG in plain words
Standard RAG is one trip to the library with one request slip. Agentic RAG is a researcher who goes back for more books when the first ones do not answer the question.
How it works
Search becomes a tool in an agent loop. The agent breaks the question into sub-questions, searches each (possibly across a vector index, SQL, the web or APIs), checks whether results actually answer it, rewrites queries if not, then synthesises with citations.
Anthropic's December 2024 guide to building agents describes the base unit as an LLM augmented with retrieval and tools that generates its own search queries. The same guide recommends stopping conditions, such as a maximum number of iterations, which is what keeps the agent from searching forever.
Example
As an illustration, "How did our Q3 cloud spend compare with budget, and which team drove the overrun?" The agent queries the finance database for totals, searches the budget document, then queries cost-allocation data by team, and only then answers. One-shot RAG would retrieve a few loosely related chunks.
Where you'll meet it
- Deep-research features in assistants
- Enterprise assistants spanning documents, databases and APIs
- Coding agents searching codebases iteratively
- Research on self-reflective retrieval such as Self-RAG
What it changes for you
For developers, retrieval tools need clear descriptions and good error messages, because the agent chooses among them. For product teams, multi-part and ambiguous questions are where the extra searches earn their keep, and every one of them adds latency, so route simple questions to plain RAG. For cost owners, track searches per answer as a metric.
Agentic RAG vs standard RAG
Standard RAG retrieves once with the user's question and answers. Agentic RAG plans, retrieves repeatedly and checks its evidence.
Common mistake
My view: making every question agentic. Most questions are simple lookups where an extra planning step only adds delay and cost. Use a router: simple questions go to standard RAG, complex ones to the agent.
Related terms
Retrieval-augmented generation · AI agent · Tool calling · Model routing · GraphRAG
From the archive: Why your RAG pipeline returns irrelevant results
Route simple lookups to standard RAG and keep the agent for questions that need several searches.


