Hybrid Search and Reranking: BM25, Vectors, RRF and Cross-Encoders

Hybrid search for RAG and search: BM25 keyword search plus vector search, reciprocal rank fusion, cross-encoder reranking and the latency each step costs.

Intermediate 3 min read · Updated 29 October 2026

Vector search finds passages with similar meaning; keyword search finds passages with the same words. Each misses things the other catches, so production retrieval for RAG and search usually runs both (hybrid search), merges the results, and then reranks the top candidates with a slower, more accurate model. This guide explains each piece, how to fuse rankings, and the latency each step costs.

Why one retriever isn't enough

Query	Keyword (BM25)	Vector (embeddings)
"error E4012 on checkout"	Finds the exact code	May return generic checkout errors
"how do I get my money back"	Misses a page titled "Refund policy"	Finds it by meaning
"SKU 88-1932 dimensions"	Exact match on the SKU	Numbers embed poorly
"laptop for a kid starting school"	Weak: few literal matches	Strong
Rare names and acronyms	Strong	Often weak, especially for new terms

The hybrid retrieval pipeline

Hybrid retrieval: run keyword and vector search in parallel, fuse their rankings, rerank the top candidates with a cross-encoder, and pass the best few to the model.

Keyword search: BM25

BM25 scores a document by how often query terms appear in it (with diminishing returns), how rare those terms are across the corpus, and the document's length. It runs on an inverted index, handles exact identifiers perfectly, needs no training, and is fast. Its weakness is vocabulary mismatch: synonyms and paraphrases score zero.

Vector search: bi-encoders

A bi-encoder embeds the query and each passage separately, so passage vectors are computed once at ingestion, and query time is one embedding plus an ANN search. Quality depends heavily on the embedding model and domain; fine-tuning on your own query-passage pairs often helps. See embeddings.

Fusing two rankings

BM25 scores and cosine similarities live on different scales, so adding them directly is fragile. Reciprocal rank fusion (RRF) uses only ranks:

RRF score(d) = Σ over retrievers r of  1 / (k + rank_r(d))      (k ≈ 60 is common)

doc   BM25 rank   vector rank   RRF
A        1            8         1/61 + 1/68 = 0.0311
B        5            1         1/65 + 1/61 = 0.0318   ← top: strong in both
C        2            —         1/62         = 0.0161
D        —            2         1/62         = 0.0161

Alternatives: normalise each score to [0, 1] and take a weighted sum (tunable, but sensitive to score distributions), or learn the combination with a small ranking model if you have labelled data.

Reranking with cross-encoders

A cross-encoder reads the query and a passage together and outputs a relevance score. Because it sees both at once, it's much more accurate than comparing two separate embeddings, but it can't precompute anything, so it only runs on the top tens of candidates.

	Bi-encoder	Cross-encoder	LLM as reranker
Input	Query and passage separately	Query + passage together	Query + several passages in a prompt
Precompute passages?	Yes	No	No
Speed	Milliseconds over millions	Tens of ms for ~50 passages on a GPU	Slowest and most expensive
Accuracy	Good	Better	Often best on nuanced relevance
Role	First-stage retrieval	Second-stage reranking	Final selection in high-value flows

Latency budget

An illustrative retrieval latency budget for a RAG request. Reranking is usually the most expensive retrieval step; it's worth it when precision matters.

Keyword and vector searches run in parallel, so retrieval costs roughly the slower of the two, plus reranking. The LLM's own generation (seconds) usually dwarfs all of it.

Filters and permissions

Apply access-control filters inside both searches, not after retrieval, or users can be shown (or the model can quote) documents they shouldn't see.
Pre-filtering in ANN indexes can hurt recall when the filter is very selective; use indexes with filter-aware search or partition by tenant.
Metadata filters (product, language, date range) narrow results cheaply and improve precision.

Evaluating retrieval

Measure recall@k for each retriever alone and for the hybrid, then NDCG or MRR after reranking, on a labelled set of queries with known relevant passages. Retrieval quality should be tracked separately from answer quality; see LLM evaluation.

Where it’s used

Enterprise RAG

Hybrid search with permission filters and cross-encoder reranking over internal documents.

E-commerce search

Lexical matching for brands and SKUs plus semantic retrieval for descriptive queries.

Support bots

Error codes found by keyword, how-to questions by meaning, merged with RRF.

In an interview

Explain what keyword and vector search each miss, with examples.
Fuse with reciprocal rank fusion and say why raw scores don't add.
Rerank the top candidates with a cross-encoder, and give its latency.
Apply permission filters inside retrieval, not afterwards.

Frequently asked questions

What is hybrid search?

Running keyword search such as BM25 and vector search in parallel and merging their results, so queries with exact terms like error codes and queries phrased differently from the documents are both handled.

What is reciprocal rank fusion?

A way to merge rankings using only positions: each document scores the sum of 1 divided by a constant plus its rank in each list, so documents ranked well by both retrievers rise to the top.

Why use a cross-encoder reranker?

A cross-encoder reads the query and passage together, making it more accurate than comparing separate embeddings. It is too slow for the whole corpus, so it reranks only the top few dozen candidates.