RAG pitfalls

RAG looks like the safe, source-backed version of a chatbot. It is better than a bare model, but it has its own failure modes, and two of them are serious.

The two serious ones

Sourced hallucination. The model answers confidently, with citations, and the citations do not actually support the claim. This is worse than a plain hallucination because the sources make it look trustworthy. A wrong answer with a plausible footnote is believed.
Retrieval failure. The right chunk exists but was not retrieved, so the model answers from the wrong chunks, or says it does not know when it should. Most "RAG is useless" complaints are actually retrieval failures, not model failures.

The other traps

The model ignores the context. You fed the right chunk and the model still answered from its training data. This happens when the instruction to "answer from the provided text" is weak or the model is too strong-willed.
Sensitive data leaks. If the retrieved chunks contain PII or confidential text, the model can echo it. Local keeps it on your machine, but it does not keep it out of the answer.
Stale index. You changed the documents but did not re-index, so retrieval returns old facts.

The safeguards

Force grounding. Tell the model explicitly: answer only from the provided excerpts, and quote or cite each claim.
Inspect retrieval, not just answers. During testing, look at what was retrieved before you look at what was written.
Test with questions whose answer you know. A RAG you have not tested on known answers is a RAG you cannot trust.
Filter sensitive fields before they go into the model.
Re-index on a schedule, and version your index alongside your data.

The deeper point

RAG moves the failure from "the model does not know" to "the system retrieved badly." That is a better failure, because it is testable. The method track is exactly this: build the retrieval, measure it on real questions, and fix the weakest step, not the model.

Treat RAG as a system to engineer and verify, not a feature to switch on.