RAG Series (11): Rerank — Putting the Right Documents First

Part 11 of the RAG series. How bad is vector search ranking quality? What's the real difference between Bi-Encoder and Cross-Encoder? How do you wire up a Reranker with ContextualCompressionRetriever? We measure with RAGAS: context_precision jumps from 0.552 to 0.792. Full LangChain implementation included.

·8 min read·AI Engineering
RAGRerankCross-EncoderBi-EncoderLangChainRAGAS

The Ranking Problem in Vector Search

Last article we solved the "recall" problem with hybrid search — combining BM25 and vector retrieval to cast a wider net.

But retrieval is just the first half. Once you have the documents, a second question kicks in: in what order do you feed them to the LLM?

Vector search ranks documents by the cosine similarity between the query embedding and the document embedding. This is fast to compute, but rough — it measures a coarse "overall similarity," not "how well does this specific document answer this specific question."

A common failure pattern: you retrieve top-4, the first result is a generic background paragraph, and the document that actually answers the question lands at rank 3. The LLM reads context that opens with noise and buries the key information. Generation quality suffers.

The Rerank idea is simple: retrieve more, then re-sort.

Retrieve top-10 first (cast a wide net, don't miss anything). Then use a more precise model to rescore all 10 documents and rerank them. Finally, hand the top-4 to the LLM.

Bi-Encoder vs Cross-Encoder

Understanding Rerank starts with understanding these two encoding architectures.

Bi-Encoder (used in vector search)

query  → Encoder → query vector
doc    → Encoder → doc vector
score  = cosine(query_vec, doc_vec)

Query and document are encoded independently. Similarity is computed via vector comparison. The big advantage: doc vectors can be precomputed offline; at query time you only need one embedding call and then a fast vector search. The downside: query and doc never "see each other" during encoding — relevance judgment is necessarily coarse.

Cross-Encoder (used in Rerankers)

[query, doc] → Encoder → relevance score

Query and document are concatenated and fed together into the model, which directly outputs a relevance score. Every token in the query can attend to every token in the document — relevance judgment is far more precise. The downside: slow — every (query, doc) pair requires a full forward pass with no precomputation.

The right relationship between the two is serial, not competitive:

Vector retrieval (Bi-Encoder, fast recall) → Reranker (Cross-Encoder, precise ordering)

The Key Metric: context_precision

Among RAGAS's four metrics, context_precision is the one that directly measures ranking quality:

context_precision = how many "right" documents are ranked above "wrong" ones
  • context_precision = 1.0: All relevant documents are ranked above irrelevant ones
  • context_precision = 0.5: Relevant document ordering is essentially random
  • context_precision = 0.0: All relevant documents sank to the bottom

This is exactly what a Reranker targets. context_recall asks "did we find it?", context_precision asks "did we rank it correctly?"

Experiment Design

We reuse the existing knowledge base (8 RAG technical documents) and test set (8 questions), comparing two strategies:

StrategyRetrievalFinal output
BaselineVector search, direct top-44 docs
RerankVector search top-10 → Cross-Encoder re-sort → top-44 docs

Both strategies deliver exactly 4 documents to the LLM. The only difference is ranking quality.

The Reranker uses BAAI/bge-reranker-v2-m3, called via the SiliconFlow API.

Implementation: Custom SiliconFlowReranker

LangChain doesn't have a built-in SiliconFlow Reranker wrapper. We implement one by subclassing BaseDocumentCompressor and overriding compress_documents:

import requests
from langchain_core.documents import Document
from langchain_core.documents.compressor import BaseDocumentCompressor
from typing import Sequence
 
class SiliconFlowReranker(BaseDocumentCompressor):
    model: str = "BAAI/bge-reranker-v2-m3"
    api_key: str = ""
    api_base: str = "https://api.siliconflow.cn/v1"
    top_n: int = 4
 
    def compress_documents(
        self,
        documents: Sequence[Document],
        query: str,
        callbacks=None,
    ) -> Sequence[Document]:
        if not documents:
            return []
 
        doc_texts = [d.page_content for d in documents]
 
        resp = requests.post(
            f"{self.api_base}/rerank",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={
                "model": self.model,
                "query": query,
                "documents": doc_texts,
                "top_n": self.top_n,
                "return_documents": True,
            },
            timeout=30,
        )
        resp.raise_for_status()
 
        reranked = []
        for item in resp.json().get("results", []):
            doc = documents[item["index"]]
            doc.metadata["rerank_score"] = item["relevance_score"]
            reranked.append(doc)
 
        return reranked

API format:

SiliconFlow's /v1/rerank endpoint:

// Request
{
  "model": "BAAI/bge-reranker-v2-m3",
  "query": "which embedding model for Chinese text",
  "documents": ["doc 1 content", "doc 2 content", ...],
  "top_n": 4,
  "return_documents": true
}
 
// Response
{
  "results": [
    {"index": 2, "relevance_score": 0.952, "document": {"text": "..."}},
    {"index": 0, "relevance_score": 0.834, "document": {"text": "..."}},
    ...
  ]
}

index is the position in the original document list. relevance_score is the Cross-Encoder's relevance score — higher is better.

Wiring It Up: ContextualCompressionRetriever

With the Reranker in hand, use ContextualCompressionRetriever to chain it after the vector retriever:

from langchain_classic.retrievers import ContextualCompressionRetriever
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
 
embeddings = OpenAIEmbeddings(
    model="BAAI/bge-large-zh-v1.5",
    api_key=os.getenv("EMBEDDING_API_KEY"),
    base_url="https://api.siliconflow.cn/v1",
)
vectorstore = Chroma.from_documents(docs, embedding=embeddings)
 
# Recall stage: retrieve more
recall_retriever = vectorstore.as_retriever(search_kwargs={"k": 10})
 
# Rerank stage
reranker = SiliconFlowReranker(
    api_key=os.getenv("EMBEDDING_API_KEY"),
    top_n=4,
)
 
# Chain them together
rerank_retriever = ContextualCompressionRetriever(
    base_compressor=reranker,
    base_retriever=recall_retriever,
)

Usage is identical to any other retriever:

docs = rerank_retriever.invoke("Which embedding model for Chinese text?")
# Internally: vector search top-10 → reranker rescores → returns top-4

Experimental Results

======================================================================
  RAGAS Metrics Comparison (Vector Search vs Vector + Rerank)
======================================================================
 
  Metric               Baseline (top-4)   Rerank (10→4)    Delta
  ────────────────────────────────────────────────────────────────────
  context_precision          0.552              0.792     ↑+0.240  ◀ key metric
  context_recall             0.500              0.688     ↑+0.188
  faithfulness               0.688              0.854     ↑+0.167
  answer_relevancy           0.429              0.381     ↓-0.049
======================================================================
 
  Conclusion:
  ✓ Rerank lifts context_precision from 0.552 to 0.792
    → More relevant documents rank first; LLM sees higher-quality context

Reading the numbers:

  • context_precision +0.240: The biggest win. Baseline at 0.552 means plenty of relevant documents are buried behind irrelevant ones. After Rerank, 0.792 — ranking quality is dramatically better.
  • context_recall +0.188: An unexpected bonus. By expanding recall from top-4 to top-10, we actually retrieve relevant documents that vector search was missing entirely. The Reranker then sorts them to the front. Two improvements for the price of one.
  • faithfulness +0.167: Better context quality leads to fewer hallucinations. The LLM stays grounded because the relevant content is now at the top. This is the chain reaction that good ranking unlocks.
  • answer_relevancy -0.049: A slight dip, likely within LLM scoring noise. Doesn't change the conclusion.

How Reranking Works in Practice

A concrete example to make this tangible.

Query = "Which embedding model should I use for Chinese text?"

Vector search returns these 4 docs (sorted by cosine similarity):

Vector search order (cosine similarity):
1. doc-001  "Introduction to RAG" — generic intro, mentions "embedding" once
2. doc-002  "Vector Database Selection" — relevant, but focused on DBs
3. doc-003  "Embedding Model Recommendations" — directly answers the question ✓
4. doc-005  "RAG Evaluation Methods" — unrelated

After the Reranker rescores:

Reranker order (Cross-Encoder relevance score):
1. doc-003  score=0.952  "Embedding Model Recommendations" ✓
2. doc-002  score=0.621  "Vector Database Selection"
3. doc-001  score=0.234  "Introduction to RAG"
4. doc-005  score=0.089  "RAG Evaluation Methods"

The correct document moves from rank 3 to rank 1. The LLM's context now opens with the actual answer. Generation quality improves directly.

When to Use Rerank

Use Rerank when:

  • Your knowledge base is large (> 100 documents) and vector ranking is unreliable
  • Queries involve precise terminology or specific concepts where relevance is subtle
  • You're optimizing for answer quality and can absorb slightly higher API cost and latency
  • You're already using hybrid search and want to keep improving downstream

Skip Rerank when:

  • Knowledge base is small (< 20 documents); vector ranking is already good enough
  • Latency is critical (Rerank adds an extra API round-trip)
  • Queries are all broad, conceptual questions where ranking order barely matters

Cost note: SiliconFlow's bge-reranker-v2-m3 is token-based. Reranking 10 documents costs roughly 3–5× a vector search call. Typically used for high-value queries or quality-sensitive applications.

Full Code

Complete code is open-sourced at:

https://github.com/chendongqi/llm-in-action/tree/main/11-rerank

Key file:

  • rerank.py — Full comparison experiment: baseline retrieval vs Rerank pipeline

How to run:

git clone https://github.com/chendongqi/llm-in-action
cd 11-rerank
cp .env.example .env   # Fill in Embedding API key and LLM API key
pip install -r requirements.txt
python rerank.py

Summary

This article demonstrated the value of Reranking through a controlled experiment:

  1. Bi-Encoder (vector search) — Fast recall, but relevance is judged in isolation; ranking is imprecise
  2. Cross-Encoder (Reranker) — Slower but precise; query and document are evaluated together, outputting a direct relevance score
  3. Chaining both (recall + re-sort) — In this experiment, context_precision jumped from 0.552 to 0.792, with context_recall and faithfulness also improving as a knock-on effect

In production RAG systems, Reranking is often the highest-ROI optimization available — no data changes, no prompt rewrites, just putting documents in the right order, and the LLM receives dramatically better context.

References

← Back to Blog