← All resources

Hybrid Search for RAG: BM25 + Dense Retrieval and Fusion

Guide · updated 2026-06-25 · Markdown variant

How to combine lexical (BM25/SPLADE) and dense vector retrieval with Reciprocal Rank Fusion for higher first-stage recall in RAG pipelines — with the RRF formula, a sparse-method comparison table, and verified DB support.

Hybrid search is a first-stage retrieval strategy that runs lexical (sparse) and dense vector search in parallel, then merges the ranked result lists before any reranking step. It consistently outperforms either method alone because each covers the other's failure modes.

Key facts

• Hybrid search runs lexical and dense retrieval simultaneously and merges the outputs before reranking, consistently beating either method alone since each compensates for the other's weak spots.
• Dense embeddings struggle on rare or domain-specific tokens like SKUs, error codes, and version strings that BM25 catches via exact term matching, while dense retrieval instead handles paraphrased or differently worded queries that trip up BM25.
• Reciprocal Rank Fusion is the go-to way to merge ranked lists because it works purely on rank position rather than raw scores, sidestepping the cross-scale scoring problem that makes weighted blending fragile — a k value of 60 is the usual default, with a lower k favoring smaller corpora and a higher k smoothing larger ones.
• Hybrid retrieval targets first-stage recall, always running before an optional, more expensive reranking pass that improves precision on the surviving candidates.
• Native hybrid support differs by database — e.g. Weaviate offers RRF plus a score-normalizing fusion, Qdrant prefetches sparse and dense in one query, Elasticsearch/OpenSearch expose a dedicated RRF retriever, Pinecone uses an alpha-weighted blend, and Milvus supports both weighted and RRF fusion.
• The recommended baseline setup indexes both BM25 and dense vectors at ingest, defaults to RRF fusion, retrieves a wide top-k before reranking, and is evaluated against a pure-dense baseline using ranking metrics.

Why does hybrid beat pure-dense retrieval?

Dense embedding models learn distributional similarity. They fail on tokens that are rare, domain-specific, or absent from their training corpus: product SKUs, error codes, proper nouns, version strings. BM25 treats these as exact term-frequency signals and ranks them correctly.

Conversely, BM25 fails whenever the query and the relevant document use different vocabulary for the same concept ("heart attack" vs "myocardial infarction"). Dense retrieval handles paraphrase well.

Hybrid runs both paths in parallel and fuses the results, so the recall gain is nearly additive with negligible extra latency (the bottleneck is the ANN sweep, not the BM25 sweep).

What sparse retrieval methods exist?

BM25 is the safe default: zero extra model cost, built into every major search and vector DB. Add SPLADE when domain recall tests show consistent BM25 failures.

How do you combine result lists?

Reciprocal Rank Fusion (RRF) is the standard algorithm (Cormack, Clarke, Büttcher, SIGIR 2009). The fused score for document d across m ranked lists is:

RRF(d) = Σ_m 1 / (k + rank_m(d))

where rank_m(d) is the 1-based position of d in list m (documents absent from a list are unscored for it). k = 60 is the conventional default. At rank 1 a document contributes 1/61 ≈ 0.0164 per list; at rank 100, 1/160 ≈ 0.006 — a ratio of only ~2.6×, which makes RRF robust to one method producing a cluster of mediocre hits.

Why RRF and not weighted score combination? BM25 scores and cosine similarities live on incompatible scales; adding or linearly blending them needs per-corpus normalisation that breaks when the index changes. RRF operates only on rank order, so no normalisation is needed — the main reason it dominates production deployments.

Tuning k: lower k (10–20) concentrates weight on top ranks (small corpora); raise k toward 60–100 for large corpora where you want a gentler curve.

Alternative — weighted convex combination: normalise both score distributions (e.g. min-max per query) and blend with a tunable alpha (Pinecone uses alpha = 1.0 for pure dense, 0.0 for pure sparse). Finer control, but needs re-tuning when the distribution shifts. Default to RRF unless you have a labelled eval set.

Where does hybrid fit relative to reranking?

Hybrid is first-stage retrieval — its job is recall@k (get the answer into the candidate set). Reranking is a second-stage precision step that re-scores those candidates with a more expensive model. Pipeline order is always: hybrid retrieve → (optional) rerank → assemble context. See /resources/reranking-for-rag.

Which vector databases support hybrid search natively?

Verified as of 2026-06; hybrid APIs evolve quickly, so check each vendor's current release notes.

Practical checklist for agents

• Build both a dense ANN index and a BM25 index at ingest (most vector DBs do this in one call).
• Use RRF (k=60) as the default fusion function unless you have a labelled eval set to tune alpha.
• Retrieve top-50 to top-100 from hybrid, then pass to a reranker if latency allows.
• Expose the combined retrieve-fuse step as one agent tool call so the agent sees a single ranked list.
• Evaluate with NDCG@10 or Recall@10 on a held-out query set; track the gain over a pure-dense baseline.

Cross-links

• Full RAG pipeline: /resources/rag-retrieval-for-agents
• Embedding model selection and ANN index types: /resources/embeddings-vector-search
• Second-stage reranking: /resources/reranking-for-rag

Verified sources

• RRF paper — Cormack, Clarke, Büttcher, SIGIR 2009: https://research.google/pubs/reciprocal-rank-fusion-outperforms-condorcet-and-individual-rank-learning-methods/
• OpenSearch native RRF: https://opensearch.org/blog/introducing-reciprocal-rank-fusion-hybrid-search/
• Weaviate hybrid search docs: https://weaviate.io/developers/weaviate/search/hybrid
• Qdrant sparse vectors docs: https://qdrant.tech/documentation/concepts/sparse-vectors/
• Pinecone hybrid search docs: https://docs.pinecone.io/guides/search/understanding-hybrid-search

#rag #retrieval #hybrid-search #bm25 #embeddings #vector-search #reciprocal-rank-fusion #agents

Category: Guide