Agentic RAG vs agentic search: why coding agents dropped the vector index, and where retrieval still wins
Claude Code started on a vector database and moved to grep. Cursor measured a gain from embeddings, then built an exact index for regex and removed semantic search from its SDK. What an agent needs from its search tools is an empty result it can act on, and exact search can be built to give one.
Contents
A common picture of agentic RAG has four boxes: a router that picks a store, a vector database that returns chunks, a model that drafts an answer, and a loop that checks the answer and goes round again. Singh et al., in their survey of agentic RAG on arXiv, define the idea more loosely and more usefully. A fixed retrieval pipeline becomes a set of decisions the agent makes for itself:
These limitations motivate autonomous control mechanisms that dynamically decide when to retrieve, how to reformulate queries, and when sufficient evidence has been collected.
Agentic search is the narrower half of that: the agent chooses a query, reads what comes back, and decides what to search for next. Coding agents do it all day, and the two whose search I can document from their own sources both started with a vector database and left it. Claude Code’s first internal release handed the model snippets from a pre-built vector index; it now greps. Cursor published numbers in November 2025 showing semantic search helped its agent, shipped a fast exact index for grep two weeks later, and its documentation now says it stores no embeddings of your codebase for search. Cursor kept an index. The one it kept is exact.
My reading of why: an agent that runs its own retrieval loop needs at least one tool whose empty result it can act on. Exact search can be built to give that. Nearest-neighbor ranking, as it usually ships, cannot, and Cursor’s own engineering post says as much in passing.
How Claude Code moved from a vector database to grep
Thariq Shihipar of the Claude Code team wrote in April that the first internal version used RAG: a vector database pre-indexed the codebase, and the harness fetched relevant snippets and handed them to Claude before each response. Then the team gave Claude a Grep tool instead. His reasons:
While RAG was powerful and fast, it required indexing and setup and could be fragile across a host of different environments. Most importantly, Claude was given this context instead of finding the context itself.
That last sentence is the difference between RAG and agentic search in one line. Anthropic’s September 2025 post on context engineering describes the design that replaced it:
Claude Code is an agent that employs this hybrid model: CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time, effectively bypassing the issues of stale indexing and complex syntax trees.
The up-front half is the part I took apart in a context window inventory. The just-in-time half is plain text search. The current tools reference says the dedicated Grep tool is built on ripgrep, and that on macOS, Linux and WSL Claude Code now leaves Glob and Grep out of the default tool set and runs find and grep through Bash instead. An optional LSP tool asks a language server for definitions, references and workspace symbols, and stays inactive until you install a code intelligence plugin and its server.
The context engineering post is candid about the cost. Exploring at runtime is slower than reading precomputed results, and an agent without good tools and heuristics can burn its context on dead ends. In exchange it searches the files as they are saved at that moment, with no index to keep in sync.
Why Cursor added semantic search to its agent, then built a grep index
Cursor’s November 2025 post made the case for embeddings, backed by a model trained on its own agent sessions:
While you could rely exclusively on grep and similar command-line tools for search, we've found that semantic search significantly improves agent performance, especially over large codebases:
The figures come from two different measurements. On Cursor’s own benchmark, answer accuracy rose 12.5% on average, between 6.5% and 23.5% depending on the model. In an A/B test, the share of agent-written code that stayed in users’ codebases rose 0.3% overall and 2.6% on codebases of 1,000 files or more. The post does not include the benchmark questions or the raw test data.
Fifteen days later, Cursor 2.1 shipped Instant Grep in beta: every grep the agent runs, served from an index. In March 2026 Vicent Marti explained how it works. Cursor’s agents lean on regex search heavily, ripgrep has to read every eligible file in scope, and in large monorepos Cursor saw rg calls take more than 15 seconds. The fix was an n-gram index for regular expressions, built and queried on the user’s machine. The post also explains why that index must be held to a stricter standard than the semantic one, which was still running at the time:
Unlike with semantic indexes, an index for regular expression search also needs to be very fresh, particularly when it comes to the model reading its own writes. We don't have to continuously update our semantic index because re-computing the embeddings for a file after it is modified does not cause the new embedding to significantly displace itself in the multi-dimensional space. The nearest-neighbor search we perform will still send the Agent in the right direction. However, if the agent is searching for specific text and it does not find it, it'll often go into a wild goose chase, waste tokens, and defeat the purpose of our performance optimization in the first place.
For months both paths ran side by side. Today Cursor’s search documentation describes Instant Grep and states that Cursor keeps no embeddings of your codebase for search, and the Agent SDK’s troubleshooting page is blunter:
Semantic search has been removed. Use Grep.
The docs and changelog entries I read do not say when the IDE dropped it or why, so I will not supply a reason on Cursor’s behalf. Anthropic’s stated reasons for Claude Code were setup, fragility, and context that was handed over rather than found.
What an empty grep result tells a coding agent
Cursor’s freshness paragraph holds the argument I care about in two sentences. A stale semantic index is tolerable because it only has to point the agent somewhere nearby. A stale regex index is not, because when exact search comes back empty, the agent believes it.
What an empty exact result establishes is narrow. If rg parseInvoice completes and finds nothing, no searched line contains that string. That is weaker than “there is no invoice parser”: the function may be called parse_invoice, sit in a directory the search ignores, or be assembled at runtime. But every one of those gaps suggests the next search, and an agent can work through them. A language server is similar: an empty “find references” means the server reported no references for that symbol, which is not the same as the symbol being safe to delete.
Even that narrow promise has to be engineered. GNU grep separates the cases by exit status, 1 when nothing matched and 2 when something went wrong. Claude Code’s dedicated Grep tool blurred them for a while, and the tools reference records the fix:
Before v2.1.208, Claude Code reported a rejected input as No files found instead of an error, even when the searched-for text existed in the target files.
A malformed regex looked exactly like an honest miss, so the agent could conclude that code did not exist when it did. The same page describes a second guard: when the agent pages past the last match in content mode, the tool returns a distinct message, so the agent resets the offset instead of deciding the pattern has no matches. No match, error and past the end are different outputs, and an agent deciding whether it has enough evidence needs to know which one it got.
Why vector search rarely gives an agent a usable “not found”
An unthresholded vector query returns up to k chunks ranked by similarity, and it returns them whether or not any of them answers the question. Vector databases let you set a minimum score, and with one a search can come back empty. Then the empty result means nothing cleared a number someone tuned, on scores that are not calibrated across queries, and with approximate search a qualifying chunk can still be missed. Unless someone sets that bar, the tool ranks and never declines.
Kevin Liao’s line on agent memory, which I quoted when I wrote about agent memory as Markdown documents, puts it bluntly: similarity ranks how close two snippets are and tells you nothing about which is correct, current or missing. Cursor’s freshness paragraph is the same observation from the builder’s side. A semantic index can afford to lag because its job is to send the agent in the right direction. That is a tool built to propose, and proposing is a different job from ruling something out.
My argument has a limit, and I have written it down before. The exact route has a silent miss too. In the memory essay I noted that an agent choosing which page to open by reading one-line descriptions can leave the right page closed, and nothing records that it stayed closed. Grep with the wrong vocabulary fails the same way: the empty result is true about the string and useless about the thing. The difference is that the agent can see the miss and name the next query, while an unthresholded ranking hands back five plausible chunks and no miss to see.
Where a vector index still wins in agentic retrieval
The empty result is only as good as the agent’s guess at the words. Code is the friendly case: identifiers are exact strings, the reader and the author share most of the vocabulary, and the directory tree and naming conventions are an index the authors already maintain. Turn those conditions around and similarity search earns its place again. A support archive written by hundreds of people, contracts that say “termination for convenience” when the question says “can we get out early”, a knowledge base in another language than the query: there the agent cannot guess the author’s words, and grep’s honest empty result is honest about the wrong thing.
Churn is the other condition. Code changes during the session, often because the agent just changed it, which is why Cursor’s regex index has to track the agent’s own writes. A corpus that changes slowly needs few embeddings recomputed after the initial build.
Cursor’s 2025 retention gain, larger on big codebases, fits this, though only as my inference: in a large unfamiliar repository the agent cannot guess names, so the vocabulary gap that code usually avoids opens anyway. Cursor did not publish a breakdown by whether the agent knew what to search for.
So where I would still use an index, I would give it the job it does well: a tool whose results the agent treats as candidates and confirms by opening the file and grepping for the name it found. For what it is worth, agent-recall, the memory server I wrote for agents, sits on the exact side of this line: it uses SQLite full-text search and no vectors.
What the agentic RAG diagram leaves out
The four-box diagram has a loop that checks whether the answer is good enough, and no account of what that check runs on. The survey names the decision, when the agent has enough evidence to stop, and the coding agents show what it needs: at least one tool that can tell the agent it looked and found nothing, and keep that apart from looking badly. Cursor’s numbers show an embedding index helped its agent, most on large codebases; I read that as the case where the agent could not have guessed the name. Its later engineering says the search the agent leans on all day has to be exact and fresh, because a missed match sends the agent on a chase. Claude Code’s v2.1.208 fix to its dedicated Grep tool says even grep does not meet that bar until the harness separates an empty result from a broken one. A vector database added to an agent should be held to the same contract, and without a tuned threshold it has no empty result to hold.
01 / Discussion
No comment section here — all discussions happen on X.
02 / More articles
Agentic workflows: one word for two opposite designs, and the difference is who owns the loop
Anthropic's taxonomy and GitHub's product disagree on what the term covers, so the label settles nothing about a design. One test does: hand an LLM the steering only when an outside check, not its own claim, can confirm the job is finished.
Anthropic's always-on agents guide: what the inbox, timers and outbox do when a worker dies, tested in SQLite
Most of the failure handling the guide leaves out comes down to one move: make every wake-up an inbox event with a key the database refuses to store twice. Retried webhooks, timer replays, approval answers and their expiries all fit through that one constraint. Leases, retries and the outbox need a little more.
How to build an MCP server in 2026: one TypeScript factory, both protocol eras, and what the SDK's HTTP handler leaves to you
One factory serves both transports and both generations of clients. In a test run the SDK handler answered a forged Origin with 200 and gave clients no freshness window to cache by, and the Inspector checked only the old handshake until asked for the new one. The two guards to add, the cache hints to set, and what a first server can leave out.