← All resources

Reranking for RAG: Cross-Encoders, LLM Rerankers, and Hosted APIs

Guide · updated 2026-06-25 · Markdown variant

Second-stage retrieval step that re-scores bi-encoder candidates with full query-document attention, boosting precision without sacrificing recall; covers cross-encoder, LLM, and late-interaction reranking, hosted APIs, sizing heuristics, and evaluation.

Bi-encoder retrieval (ANN over embeddings) maximises recall cheaply but scores each document independently, missing subtle query-document interactions. A reranker runs a second pass over the top-N candidates and reorders them with full joint attention, recovering precision without rescanning the full index.

Why rerank? Recall vs. precision

Bi-encoder ANN retrieval optimises recall@k: find everything plausibly relevant fast. Reranking optimises precision@k: put the most relevant chunks first so your LLM context is not diluted. The two-stage split lets each component do what it does best.

Standard sizing heuristic: retrieve top-50 to top-100 with ANN (cheap), rerank to top-5 to top-20 (expensive). Retrieving too few limits recall; reranking too many exceeds latency budgets. Tune against your query latency SLA and your recall@k evaluation scores.

What architectures exist?

Cross-encoder rerankers — encode the query and each candidate together in a single forward pass (e.g., BERT-style model with a [CLS] → relevance score head). Full token-level cross-attention captures interactions that separate embeddings miss. Accuracy is high; inference is O(N × model_cost) over N candidates. Cross-encoders are the dominant deployed approach.

LLM rerankers — prompt a large language model to score or sort candidates (pointwise: "rate relevance 1-10"; listwise: "rank these passages"). Highest semantic understanding; also highest latency and cost. Practical mainly when candidate N is small (≤10) or quality trumps cost.

Late-interaction (ColBERT-style) — encode query and document independently to per-token vectors; score via MaxSim (maximum inner product across token pairs). Decompresses to near-cross-encoder quality while precomputing document token embeddings offline. The PLAID engine (arXiv:2205.09707) reduces late-interaction search latency substantially versus naïve ColBERTv2 via centroid pruning. Acts more as a first-stage dense retriever than a pure reranker, but fills the gap between bi-encoders and cross-encoders.

Comparison table

Latency is workload-dependent — benchmark on your own data and hardware before committing.

Hosted rerank APIs (verified)

• Cohere Rerank — rerank-v4.0-pro and rerank-v4.0-fast (Rerank 4, with a 32K-token context, ~4× the v3.5 window). Single multilingual model; available direct and on Azure AI Foundry. Docs: https://docs.cohere.com/docs/rerank-overview
• Voyage AI — rerank-2.5 and rerank-2.5-lite (instruction-following rerankers; multilingual; among the lowest-latency hosted rerankers). Docs: https://docs.voyageai.com/docs/reranker
• Jina AI — jina-reranker-v3 via hosted API; earlier versions are also open-weight on Hugging Face. Docs: https://jina.ai/reranker/

Open-weight rerankers (self-hosted)

• BAAI/bge-reranker-v2-m3 — multilingual cross-encoder on the BGE M3 backbone; use via the FlagEmbedding library.
• colbert-ir/colbertv2.0 — ColBERT late-interaction model; use with the PLAID engine for large-scale retrieval.

How should agents call a reranker?

Expose retrieve-then-rerank as one tool so the agent does not orchestrate two separate calls: the tool takes a query and returns ranked chunks with scores. Set a score threshold or a hard top-k limit before injecting into context to prevent overflow. See /resources/rag-retrieval-for-agents for the full pipeline and /resources/embeddings-vector-search for the ANN retrieval stage that feeds the reranker.

Evaluation

Measure reranker impact in isolation: fix the first-stage retrieval, vary the reranker, and compare NDCG@k (normalised discounted cumulative gain) or MRR (mean reciprocal rank) on a labelled query set. BEIR (arXiv:2104.08663) is the standard zero-shot retrieval benchmark; many reranker papers report BEIR scores for comparability. For end-to-end RAG evaluation, use RAGAS (github.com/explodinggradients/ragas, arXiv:2309.15217).

Verified sources

• Cohere Rerank overview: https://docs.cohere.com/docs/rerank-overview
• Voyage AI reranker docs: https://docs.voyageai.com/docs/reranker
• Jina Reranker: https://jina.ai/reranker/
• BGE reranker-v2-m3 (Hugging Face): https://huggingface.co/BAAI/bge-reranker-v2-m3
• ColBERTv2 (Hugging Face): https://huggingface.co/colbert-ir/colbertv2.0
• ColBERT paper (arXiv:2004.12832, SIGIR 2020): https://arxiv.org/abs/2004.12832
• PLAID efficient late-interaction retrieval (arXiv:2205.09707): https://arxiv.org/abs/2205.09707
• BEIR retrieval benchmark (arXiv:2104.08663): https://arxiv.org/abs/2104.08663
• RAGAS paper (arXiv:2309.15217): https://arxiv.org/abs/2309.15217

#rag #retrieval #reranking #cross-encoder #embeddings #vector-search #evaluation #agents

Category: Guide