檢索與 RAGRetrieval and RAG

RAG 的品質上限由檢索決定：沒被找出來的文件，模型不可能引用。這一章談怎麼把檢索做對、怎麼在權限之內做，以及怎麼量測它到底做得好不好。

Retrieval sets the ceiling on RAG quality: a document that isn't found can't be cited. This chapter covers getting retrieval right, doing it within permissions, and measuring whether it actually works.

先問：需要檢索嗎？First: do you need retrieval?

現在的模型 context window 很大，如果資料只有幾十份文件、不常更新、所有使用者都能看，直接放進 prompt（搭配 prompt caching 降低成本）往往更簡單、品質也更好。以下情況才需要檢索：

Context windows are now large. If the data is a few dozen documents, rarely updated and visible to every user, putting it straight into the prompt (with prompt caching to cut cost) is often simpler and better. You need retrieval when:

• 資料量超過 context 能放的範圍，或每次都放進去成本太高。
• 權限：不同使用者能看的文件不同。
• 新鮮度：資料持續更新，需要增量索引。
• 可追溯：需要精確指出答案出自哪一段。

• Volume exceeds what fits in context, or including it every time costs too much.
• Permissions differ between users.
• Freshness: data keeps changing and needs incremental indexing.
• Traceability: you must point to the exact passage behind an answer.

整體流程The pipeline

切段Chunking

• 依文件結構切（標題、條款、段落），不要只按固定字數切。條款被切成兩半，兩半都沒用。
• 每段帶上下文：文件標題、章節標題。單獨拿出來看，要知道它在講什麼。lab 的 runbook 切段就是這樣做的。
• 保留來源：文件 ID、段落編號、頁碼或位移，引用與除錯都需要。
• 表格與 PDF 要特別處理。解析品質常常比 embedding 模型的選擇更影響結果。先人工檢查幾份解析後的文字。

• Split by document structure (headings, clauses, paragraphs), not just fixed character counts. A clause split in half makes both halves useless.
• Give each chunk context: document title and section heading. Read on its own, a chunk should make sense. The lab's runbook chunker does this.
• Keep provenance: document ID, section number, page or offset. Citations and debugging need it.
• Tables and PDFs need care. Parsing quality often matters more than the choice of embedding model. Inspect some parsed output by hand first.

詞彙檢索與向量檢索Lexical vs vector search

兩者的盲點剛好互補，所以實務上常同時跑兩種，再把結果融合。

Their blind spots complement each other, so in practice you often run both and fuse the results.

Reciprocal Rank FusionReciprocal Rank Fusion

兩種分數的尺度不同，直接相加沒有意義。RRF（Cormack 等人，2009）只看名次：

The two scores live on different scales, so adding them is meaningless. RRF (Cormack et al., 2009) uses only ranks:

RRF(d) = Σ  1 / (k + rank_r(d))      rank 從 1 開始 / ranks start at 1
         r

k 控制「前幾名比後幾名重要多少」。k 越小，第一名的優勢越大；k 越大，各名次的貢獻越平均。60 是原論文的經驗值，通常不用調。在兩份清單都出現的文件，分數會比只在一份清單拿第一名的文件高，這正是融合想要的效果。

k controls how much more top ranks count than lower ones. Smaller k gives first place a bigger edge; larger k flattens the contributions. 60 comes from the original paper and rarely needs tuning. A document appearing in both lists usually outscores one that tops only a single list, which is exactly the point of fusion.

動手試：混合檢索Try it: hybrid retrieval

可以觀察到：PAY-4102 由 BM25 精準命中；「migrate a big postgres table」BM25 完全找不到（migrate ≠ migration、postgres ≠ postgresql），3-gram 找得到；勾選「忽略權限」再查 personal data，會看到無權限的 CLAUSE-002 出現在結果中。

Things to notice: BM25 nails PAY-4102; for "migrate a big postgres table" BM25 finds nothing (migrate ≠ migration, postgres ≠ postgresql) while 3-grams do; tick "ignore ACL" and search for personal data to see the restricted CLAUSE-002 appear.

權限：在查詢裡過濾Permissions: filter inside the query

常見的錯誤是先取前 k 筆，再把使用者無權看的刪掉。這有兩個問題：結果可能被刪到剩零筆；更糟的是，只要中間有任何一層（快取、log、重排序服務）看到未過濾的結果，就已經外洩了。權限條件要放在查詢本身的 WHERE 裡，而且要用發出請求的使用者身分，不是服務帳號的身分。

A common mistake is to take the top k and then drop what the user can't see. Two problems: the result can shrink to zero, and worse, any layer that sees the unfiltered list (a cache, a log, a reranking service) has already leaked it. Put the permission condition in the query's WHERE clause, using the requesting user's identity, not a service account's.

PostgreSQL + pgvector 範例PostgreSQL + pgvector example

如果客戶已經在用 PostgreSQL，混合檢索可以在同一個資料庫裡完成，權限規則和資料放在一起。注意內層子查詢先 ORDER BY ... LIMIT，才能用到 HNSW 索引。

If the customer already runs PostgreSQL, hybrid retrieval can live in the same database, with access rules next to the data. Note the inner subqueries do ORDER BY ... LIMIT first so the HNSW index can be used.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id             bigserial PRIMARY KEY,
    doc_id         text   NOT NULL,
    title          text   NOT NULL,
    body           text   NOT NULL,
    allowed_groups text[] NOT NULL DEFAULT '{everyone}',
    embedding      vector(1024) NOT NULL,     -- 維度依 embedding 模型 / depends on the model
    tsv            tsvector GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || body)) STORED
);
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);   -- 預設 m=16, ef_construction=64
CREATE INDEX ON chunks USING gin (tsv);
CREATE INDEX ON chunks USING gin (allowed_groups);

-- $1 = query embedding, $2 = query text, $3 = requesting user's groups (text[])
WITH lexical AS (
    SELECT id, row_number() OVER (ORDER BY rank DESC) AS rnk
    FROM (
        SELECT id, ts_rank_cd(tsv, q) AS rank
        FROM chunks, websearch_to_tsquery('english', $2) AS q
        WHERE tsv @@ q AND allowed_groups && $3::text[]
        ORDER BY rank DESC
        LIMIT 30
    ) s
),
semantic AS (
    SELECT id, row_number() OVER (ORDER BY dist) AS rnk
    FROM (
        SELECT id, embedding <=> $1::vector AS dist
        FROM chunks
        WHERE allowed_groups && $3::text[]
        ORDER BY embedding <=> $1::vector
        LIMIT 30
    ) s
)
SELECT c.id, c.doc_id, c.title, c.body, sum(1.0 / (60 + r.rnk)) AS rrf
FROM (SELECT * FROM lexical UNION ALL SELECT * FROM semantic) r
JOIN chunks c USING (id)
GROUP BY c.id
ORDER BY rrf DESC
LIMIT 8;

重排序Reranking

cross-encoder 重排序模型會同時讀查詢與候選段落，判斷相關性，通常比單純的向量相似度準，但每個候選都要算一次，會增加延遲與成本。做法是先用便宜的方法取 20 到 50 個候選，再重排序取前幾名。要不要加，用測試集量過再決定：如果 RRF 的 recall@5 已經很高，重排序的效益可能很小。

A cross-encoder reranker reads the query and each candidate together to judge relevance. It is usually more accurate than vector similarity alone, but it runs once per candidate, adding latency and cost. Take 20–50 candidates cheaply, then rerank to the top few. Decide whether to add it by measuring on your test set: if RRF's recall@5 is already high, the gain may be small.

量測檢索品質Measuring retrieval

把檢索和生成分開評估。答案錯了，你需要知道是「沒找到對的文件」還是「找到了但模型用錯」，兩者的修法完全不同。

Evaluate retrieval separately from generation. When an answer is wrong you need to know whether the right document wasn't found or was found and misused; the fixes are completely different.

• recall@k：相關文件中，有多少出現在前 k 筆。RAG 最重要的指標，因為沒進前 k 筆的文件模型看不到。
• MRR：第一個相關文件名次的倒數平均。衡量「對的東西排得夠不夠前面」。
• 權限外洩數：權限測試案例中，回傳了無權限文件的次數。這個必須是 0。

• recall@k: the share of relevant documents appearing in the top k. The key RAG metric, because the model never sees anything outside the top k.
• MRR: mean reciprocal rank of the first relevant document; how high the right thing ranks.
• Permission leaks: how often permission test cases return unauthorised documents. Must be zero.

def recall_at_k(retrieved: list[str], relevant: set[str], k: int) -> float:
    if not relevant:
        return 1.0
    return len(set(retrieved[:k]) & relevant) / len(relevant)


def reciprocal_rank(retrieved: list[str], relevant: set[str]) -> float:
    for rank, doc_id in enumerate(retrieved, start=1):
        if doc_id in relevant:
            return 1.0 / rank
    return 0.0

第 7 章會把這些指標放進一個可以在 CI 執行的評測流程。

Chapter 7 puts these metrics into an evaluation run you can execute in CI.

常見錯誤Common mistakes

• 換了 embedding 模型卻沒重建索引。不同模型的向量不能混用，要整批重算。
• 索引過期。文件更新或刪除後，舊段落仍被檢索到。建立增量更新與刪除的流程，並監控索引延遲。
• 只用向量檢索。精確代號類的查詢會大量失敗，而這類查詢在企業環境非常多。
• 只看最終答案評分。分不出是檢索還是生成出問題。
• 相信 demo 的語料。真實文件有掃描檔、表格、重複版本、過時內容，先看資料再設計。

• Changing the embedding model without re-indexing. Vectors from different models don't mix; recompute everything.
• Stale indexes. Updated or deleted documents still get retrieved. Build incremental update and delete paths and monitor index lag.
• Vector-only search. Exact-identifier queries fail en masse, and enterprises are full of them.
• Scoring only final answers. You can't tell whether retrieval or generation broke.
• Trusting a demo corpus. Real documents include scans, tables, duplicate versions and stale content; look at the data before designing.

小測驗Quiz