This resume is also available in English.

9. RAG and tools3 / 5 · All columns

Columns › 9. RAG and tools › Part 40

Evaluating RAG, Hallucinations and Common Failures: How to Know It Answers Right

RAG 的評估、幻覺與常見失敗:怎麼知道它答得對

Evaluate retrieval and answers separately with a question set: recall@k, MRR, faithfulness and refusals. Break hallucinations into four sources and block them layer by layer, from data, retrieval and prompt to checks in code and the interface. Then the most common failures and fixes, and monitoring and regression tests once it’s live.

  • RAG
  • AI
  • Go
  • 評估
In this article
  1. What this article solves
  2. Step 1: A wrong answer isn’t one kind of failure
  3. Step 2: A golden question set
  4. Step 3: Retrieval metrics: recall@k and MRR
  5. Step 4: Answer metrics: faithfulness, correctness and refusals
  6. Step 5: Where hallucinations come from, and how to block them layer by layer
  7. Step 6: Common failures and fixes
  8. Step 7: Once it’s live: logging, monitoring and regression tests
  9. Common pitfalls
  10. Key takeaways
  11. Next

The previous article, “Where RAG Quality Comes From: Chunking, Embeddings, Vector Databases and Hybrid Search”, explained what each stage can tune. This one is the advanced level: after a change, did things get better or worse? When the model answers wrong, which stage failed? And the thing everyone cares about most: how do you avoid hallucinations?

What this article solves

You change the chunk size from 800 characters to 500, try five questions, the answers look better, and you ship it. A week later support reports that refund questions have started coming back wrong. This is the most common way RAG gets built, and the most common way it breaks: tuning by feel.

This article sets up a repeatable method:

  1. Break “a wrong answer” into kinds of failure, so you know which stage to fix.
  2. Build a golden question set, including questions that should be answered with “I don’t know”.
  3. Measure retrieval (recall@k, MRR) separately from answers (faithfulness, correctness, refusals).
  4. Where hallucinations come from, and five layers of defence from data to interface.
  5. A table of common failures and their fixes.
  6. Logging, monitoring and regression tests once it’s live.

Step 1: A wrong answer isn’t one kind of failure

When an answer is wrong, ask four questions in order: fix the step that failed 1. Data Is the answer written? missing, wrong, stale what to fix fix the docs not the RAG 2. Retrieval Was the chunk found? the usual culprit what to fix chunking, hybrid filters, recall@k 3. Ranking, prompt Found, but was it used? cut off or buried what to fix rerank, budget number of chunks 4. Generation Used, yet misstated? hallucination proper what to fix prompt rules, threshold citation checks Make sure retrieval works before blaming generation: most wrong answers were never retrieved.

Figure: a “wrong answer” may come from wrong data, a chunk never retrieved, a chunk retrieved but not used, or a chunk used and misstated; each needs a different fix.

Given a wrong answer, ask four questions in order and you’ll know what to fix:

  1. Is the correct answer in the documents? If not, or it’s wrong or stale, that’s a data problem. Fix the document, not the RAG.
  2. Was the right chunk retrieved? If not, that’s a retrieval problem: look at chunking, embeddings, hybrid search, filters.
  3. It was retrieved; did it reach the prompt and did the model use it? Cut off for ranking too low, or buried in unrelated chunks: that’s a ranking and prompt problem.
  4. It was used, yet the answer misstates it or adds to it? Only this is a generation problem, a hallucination in the narrow sense.

This split matters, because most people who see a wrong answer go and edit the prompt or switch models, while in practice the failure is most often at step 2. Make sure retrieval works before you blame generation.

Step 2: A golden question set

Evaluation needs fixed questions, run in full after every change. For each question in this golden set, record:

  • The question: worded the way real users ask, not copied from the documents (or keyword search will look unrealistically good).
  • The right sources: which chunks (file and line, or chunk ids) contain the answer. This is what retrieval is measured against.
  • A reference answer or key points: what the answer must mention, for example “don’t retry directly”, “confirm the idempotency key first”.
  • The kind: a plain fact, needs two documents together, an error code or id, or should be refused.

The last kind is the one people forget: ask “Does our refund policy give VIPs a discount?” when the documents say nothing about it, and the right answer is “The documents don’t say.” Without such questions, you’ll never measure hallucination.

Where do questions come from? The best source is real records: support tickets, questions asked on call, users’ questions once it’s live (with personal data removed). Thirty to fifty questions are already useful at the start; after that, every wrong answer you find becomes a new question.

// internal/rag/eval/golden.go: one golden question
package eval

type Case struct {
	Question string   `json:"question"`
	Relevant []string `json:"relevant"` // ids of the right chunks; empty means it should be refused
	MustSay  []string `json:"must_say"` // points the answer has to mention
	Kind     string   `json:"kind"`     // fact, multi_doc, exact_id, refuse
}

Step 3: Retrieval metrics: recall@k and MRR

Measure retrieval on its own: it calls no generation model, so it’s fast and cheap enough to run on every chunking or parameter change. The two most common metrics:

  • recall@k: what share of the right chunks appear in the top k. k is the number of chunks you actually put in the prompt. A low recall@5 means the model never gets the answer, and no amount of prompt tuning will help.
  • MRR (Mean Reciprocal Rank): the rank of the first right chunk, inverted (first place scores 1, second 0.5, not found 0), averaged over all questions. It tells you whether the right thing is near enough the top.
// internal/rag/eval/metrics.go
package eval

// RecallAtK: the share of relevant that appears in the first k of got; questions with no right
// chunk (ones to refuse) don't count
func RecallAtK(got, relevant []string, k int) (float64, bool) {
	if len(relevant) == 0 {
		return 0, false
	}
	if k > len(got) {
		k = len(got)
	}
	top := map[string]bool{}
	for _, id := range got[:k] {
		top[id] = true
	}
	hit := 0
	for _, id := range relevant {
		if top[id] {
			hit++
		}
	}
	return float64(hit) / float64(len(relevant)), true
}

// ReciprocalRank: 1 over the rank of the first right chunk; 0 if none was found
func ReciprocalRank(got, relevant []string) float64 {
	want := map[string]bool{}
	for _, id := range relevant {
		want[id] = true
	}
	for i, id := range got {
		if want[id] {
			return 1 / float64(i+1)
		}
	}
	return 0
}

Average recall@k and reciprocal rank over all questions and you have the score for this configuration. Change one thing at a time, compare the scores before and after, and look by kind: if overall recall rises but the “error code” kind drops, you’ve probably turned down keyword search.

Step 4: Answer metrics: faithfulness, correctness and refusals

Evaluate generation only once retrieval is sound. An answer is judged on three things:

  • Faithfulness: every sentence cites a chunk, and the chunk really supports the sentence. This is the direct measure of hallucination.
  • Correctness: it mentions the reference points (MustSay) and says nothing that contradicts them.
  • Refusals: questions that should be refused are refused, and questions that shouldn’t aren’t. Measure both directions, or a system tuned to say “I don’t know” to everything will score full marks on faithfulness.

How to judge? Mix three methods, from cheapest to most expensive:

  1. Rules in code: every sentence cites, every cited id is among this retrieval’s results, the key points’ keywords appear, refused questions get the fixed refusal sentence. Fast and reproducible; put them in CI.
  2. An LLM judge: ask another model whether “this chunk supports this sentence”. Smarter than rules, but it makes mistakes too: first check its verdicts agree with human labels, and keep its questions, prompt and version fixed and recorded.
  3. Human spot checks: a person reads 20 to 30 questions after every big change. The most accurate, and what calibrates the other two.

Step 5: Where hallucinations come from, and how to block them layer by layer

In RAG, “hallucination” usually has four sources:

  • Nothing retrieved, so the model fills in: retrieval found no answer, and the model answers from general knowledge anyway.
  • Unrelated chunks, forced to fit: the chunk is about order timeouts, the model uses it to answer about payment timeouts.
  • Read correctly, then embellished: the chunk says “don’t retry directly”; the answer becomes “don’t retry; wait 5 minutes and try again”. The second half isn’t in any document.
  • Fake citations: it cites [c3], but c3 says nothing of the kind, or c3 doesn’t exist.
Five layers against hallucination (top to bottom, each blocks some) 1. Data only maintained, current documents; resolve contradictions 2. Retrieval hybrid + filters; below the threshold, do not ask the model 3. Prompt chunks only, cite ids, say when missing, flag conflicts 4. Checks in code every sentence cited, ids retrieved, numbers in the source 5. Interface show sources, open the original, report mistakes The two highlighted layers work best and cost least: they do not rely on the model obeying.

Figure: five layers from data to interface, each blocking some; don’t rely on the prompt layer alone.

No single setting brings hallucinations to zero, but you can block them layer by layer until they’re rare enough to accept, and visible when they happen:

Layer one: data. Only documents someone maintains and that are currently valid go into the corpus; outdated pages and drafts stay out of the index; contradictory documents get resolved first. An answer that follows a wrong document is wrong however well it cites.

Layer two: retrieval. Hybrid search and filters raise the odds of retrieving the right thing; on top, set a relevance threshold: if even the best chunk scores below it, treat it as nothing found, don’t call the model, answer “the documents don’t say”. This is the most effective and cheapest layer, because the first source of hallucination (nothing retrieved, model fills in) is stopped right here. Tune the threshold with the refusal questions in your set.

Layer three: the prompt. Answer only from the chunks, cite an id in every sentence, say you don’t know when the chunks don’t say, point out contradictions; asking it to list the chunks it uses before answering also helps. Keep the temperature low to reduce improvisation. But remember: a prompt is a request, not a guarantee, which is why the next layer exists.

Layer four: checks in code. After the model answers, verify it in code, and don’t show users an answer that fails:

  • Every sentence (or paragraph) has a citation.
  • Every cited id is among this retrieval’s results; one that isn’t is a fabricated citation.
  • Every number, amount, error code and version in the answer appears as a literal string in the cited chunks. This rule is simple, and it stops a great deal of embellishment.

When a check fails, retry once (telling the model why it failed); if it still fails, return a refusal or just the retrieved passages themselves.

Layer five: the interface. Show the source beside every sentence, openable to the original; say clearly “according to such-and-such document”; provide a “this answer is wrong” button. When users can see the sources they catch problems themselves, and the reported questions go straight into the next version of the question set.

// internal/rag/check.go: checking an answer's citations (layer four)
package rag

import (
	"fmt"
	"regexp"
	"strings"
)

var cite = regexp.MustCompile(`\[(c\d+)\]`)
var exact = regexp.MustCompile(`[A-Z][A-Z0-9_]{3,}|\d+(\.\d+)?`) // error codes, numbers, versions

// CheckAnswer verifies every sentence cites, every citation was retrieved this time, and every
// exact string comes from the cited chunks
func CheckAnswer(answer string, retrieved map[string]string) error {
	for _, s := range sentences(answer) {
		ids := cite.FindAllStringSubmatch(s, -1)
		if len(ids) == 0 {
			return fmt.Errorf("no citation: %q", s)
		}
		var sources strings.Builder
		for _, m := range ids {
			text, ok := retrieved[m[1]]
			if !ok {
				return fmt.Errorf("cites %s, which was not retrieved", m[1])
			}
			sources.WriteString(text)
		}
		plain := cite.ReplaceAllString(s, "")
		for _, token := range exact.FindAllString(plain, -1) {
			if !strings.Contains(sources.String(), token) {
				return fmt.Errorf("%q is not in the cited chunks", token)
			}
		}
	}
	return nil
}

// sentences splits at full stops, question and exclamation marks (a Latin one only before a
// space, so 1.5 stays whole) and newlines; good enough, not perfect sentence splitting
func sentences(text string) []string {
	var out []string
	for _, s := range regexp.MustCompile(`[。!?\n]+|[.!?](\s+|$)`).Split(text, -1) {
		if strings.TrimSpace(s) != "" {
			out = append(out, s)
		}
	}
	return out
}

For example, with c1 retrieved as “Do not retry a charge until its idempotency key is confirmed”, the answer “Don’t retry it directly [c1]. Wait 5 minutes first [c1].” is stopped: 5 isn’t in c1. A rule like this doesn’t understand meaning and can’t stop every hallucination, but it’s reproducible, free, and it stops the most dangerous kind: specific numbers and codes that were made up. Whether a chunk supports a sentence in meaning is left to the LLM judge and human checks of step 4.

Step 6: Common failures and fixes

SymptomUsual causeFix
Error codes and ids not foundVector search onlyAdd keyword search, merge with RRF
Rephrase the question and it’s not foundKeywords only, or chunks without headingsAdd vector search; keep heading paths in chunks
The right chunk ranks low and is cut offRanking not precise enoughFetch more and rerank; check recall@k
The answer cites an old ruleStale documents still indexed, updates not syncedIncremental updates and deletes by content hash; filter by date or version
Two documents disagree and the model picks oneCorpus not curated, no precedenceTidy the documents; mark source precedence; have the prompt point out conflicts
The answer shows what the user mustn’t seePermissions left to the modelFilter by identity at retrieval
Off-topic but confidentUnrelated chunks retrieved, no thresholdRelevance threshold; refuse when nothing is found
Cites chunks that don’t existNo checks in codeCitations must come from this retrieval
A sentence in a document changes the model’s behaviourPrompt injection: chunks read as instructionsChunks inside clear markers; least-privilege tools; a person confirms risky actions
Says “I don’t know” to everythingThreshold too high, refusal rule too strictTune the threshold on answerable and refusal questions together

Step 7: Once it’s live: logging, monitoring and regression tests

Evaluation doesn’t end with one run before launch. Once it’s live you need to answer “did it get better or worse this week?”:

  • Log every question: the question (personal data removed), the retrieved chunk ids and scores, the chunks put in the prompt, the answer, the check results, user feedback, and the embedding and generation model versions. Only then can you trace a wrong answer to a stage.
  • Watch a few numbers: the refusal rate (a sudden rise often means the index broke or the filters got too strict), the citation check failure rate, the distribution of top relevance scores, the share of reported mistakes, latency and cost.
  • Regression tests: put the retrieval part of the golden set and the code-rule checks in CI; run them before changing chunking, embeddings, prompts or models, and don’t merge if the scores drop.
  • Keep adding real questions: especially the ones reported wrong, and the refused ones. A question set that grows with real use keeps the evaluation meaningful.

Common pitfalls

Shipping after trying a few questions by hand. The sample is too small to see improvement or regression. Compare scores on a fixed set.

Evaluating retrieval and answers together. When an answer is wrong, you can’t tell whether nothing was retrieved or the model misstated it. Measure retrieval first, then generation.

No refusal questions in the set. You can’t measure hallucination, and a system tuned to say “I don’t know” to everything goes unnoticed.

Believing a prompt can eliminate hallucination. The prompt is one layer; the relevance threshold, checks in code and sources in the interface matter just as much.

Trusting an LLM judge completely. The judge makes mistakes too; validate it against human labels and fix its questions and version.

Going live without logs. When something goes wrong you can’t tell what was retrieved or why it answered that way.

Key takeaways

  • Wrong answers come in four kinds: wrong data, not retrieved, retrieved but unused, used and misstated; make sure retrieval works before blaming generation.
  • A golden question set: questions in real users’ words, the right chunks, points the answer must make, a kind, and always some questions to refuse; keep adding the ones answered wrong.
  • Measure retrieval separately with recall@k and MRR, cheaply and on every change, and look by question kind.
  • Judge answers on faithfulness, correctness and refusals in both directions; let rules in code, an LLM judge and human checks calibrate each other.
  • Hallucination has four sources; block it in five layers: data, retrieval (a relevance threshold, and no model call when nothing is found), the prompt, checks in code (citations from this retrieval, numbers and codes in the source) and the interface (sources shown, mistakes reportable).
  • Once live, log every retrieval and answer, watch the refusal and check failure rates, and run the question set in CI as a regression test.

Next

The ideas, the quality and the evaluation are all covered; now build it. The next article, “Building a RAG Pipeline in Go: Chunking, Vector Search and Tool Calling”, uses Go, PostgreSQL + pgvector and Claude’s tool use to implement chunking, hybrid search, citation checks, tests and evaluation.