RAG in one page: retrieve, rank, answer
RAG finds relevant text, ranks it, and gives it to a model to answer from. The generation step is rarely where quality is lost — chunking, retrieval and ranking determine almost everything, which is why evaluating retrieval separately from answering is the highest-value thing a team can do.
RAG gets described as though it were an architecture. It is closer to a habit: before answering, go and find the relevant text, and answer from that rather than from memory.
The three steps
Embed the user's question. Use it to retrieve candidate passages from an index. Rank those candidates, keep the best few, and send them to the model with an instruction to answer from the supplied text.
That is the whole pipeline. Everything else — chunking strategy, hybrid search, rerankers, citation formats — is refinement of those three steps.
Why it works
It converts a recall problem into a reading problem. Recall from weights is unreliable and unverifiable; reading supplied text is something models do well, and the source can be shown to the user.
Retrieval is where quality is decided
When a RAG system gives a bad answer, the instinct is to blame the model or rewrite the prompt. Usually the right passage was never retrieved, and no prompt fixes that.
So evaluate retrieval on its own. Take fifty real questions, note which document should answer each, and measure how often it appears in the top five. That number, not the answer quality, tells you where to spend the next week.
Use both kinds of search
Vector search handles paraphrase well and exact strings badly. A user pasting an order number or an error code gets poor results from embeddings and excellent results from keyword matching.
Running both and merging — reciprocal rank fusion is the standard approach and is a few lines — is reliably better than either alone, and it is the cheapest quality improvement in the whole pipeline.
Rank harder, send less
Retrieve generously, then narrow aggressively. A reranking step that scores each candidate against the question specifically is slower per candidate and much more accurate than the initial retrieval.
Then send three to five passages, not twenty. Context you did not need is context competing with the context you did.
Show the source
Requiring the answer to cite which passage it came from, and rendering that link, does two things at once: it constrains the model towards the supplied text, and it gives the user a way to check that costs you nothing.
Frequently asked questions
How many chunks should I pass to the model?
Fewer than instinct suggests. Three to five good ones usually beats twenty mediocre ones — extra passages dilute attention and add cost. Retrieve widely, rank hard, pass little.
Do I still need keyword search?
Almost always. Vector search is weak on exact identifiers, product codes and rare proper nouns — precisely the things users paste in. Running both and merging the results is the reliable default.
