How to Intermediate · 3 min read

How to combine BM25 with reranking

Q: How to combine BM25 with reranking

Combine BM25 for initial document retrieval with an AI model for reranking by first retrieving top candidates using BM25, then scoring and sorting them with a chat completion model like gpt-4o-mini or claude-3-5-sonnet-20241022. This two-step approach improves relevance by leveraging lexical matching and semantic understanding.

Quick answer

Combine BM25 for initial document retrieval with an AI model for reranking by first retrieving top candidates using BM25, then scoring and sorting them with a chat completion model like gpt-4o-mini or claude-3-5-sonnet-20241022. This two-step approach improves relevance by leveraging lexical matching and semantic understanding.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works) or Anthropic API key
pip install rank_bm25 openai anthropic

Setup

Install the necessary Python packages for BM25 retrieval and AI API clients. Set your API key as an environment variable.

Install BM25 and OpenAI/Anthropic SDKs:

bash

pip install rank_bm25 openai anthropic

Step by step

This example shows how to retrieve documents with rank_bm25 and rerank them using OpenAI's gpt-4o-mini model. Replace with Anthropic similarly.

python

import os
from rank_bm25 import BM25Okapi
from openai import OpenAI

# Sample documents
corpus = [
    "Python is a programming language.",
    "OpenAI provides powerful AI models.",
    "BM25 is a ranking function used by search engines.",
    "Reranking improves search results by rescoring candidates."
]

# Tokenize corpus
tokenized_corpus = [doc.lower().split() for doc in corpus]

# Initialize BM25
bm25 = BM25Okapi(tokenized_corpus)

# Query
query = "How to improve search results with AI?"
tokenized_query = query.lower().split()

# Retrieve top 3 documents with BM25
top_n = 3
bm25_scores = bm25.get_scores(tokenized_query)
top_indices = sorted(range(len(bm25_scores)), key=lambda i: bm25_scores[i], reverse=True)[:top_n]
top_docs = [corpus[i] for i in top_indices]

# Initialize OpenAI client
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Prepare prompt for reranking
rerank_prompt = "You are a search relevance model. Rank these documents by relevance to the query.\nQuery: " + query + "\nDocuments:\n"
for i, doc in enumerate(top_docs, 1):
    rerank_prompt += f"{i}. {doc}\n"
rerank_prompt += "\nReturn the document numbers sorted by relevance, most relevant first, as a comma-separated list."

# Call OpenAI chat completion for reranking
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": rerank_prompt}]
)
rerank_order = response.choices[0].message.content.strip()

# Parse rerank order
order = [int(x.strip()) - 1 for x in rerank_order.split(",") if x.strip().isdigit()]

# Output reranked documents
print("Reranked documents:")
for idx in order:
    print(f"- {top_docs[idx]}")

output

Reranked documents:
- Reranking improves search results by rescoring candidates.
- OpenAI provides powerful AI models.
- BM25 is a ranking function used by search engines.

Common variations

You can use Anthropic's claude-3-5-sonnet-20241022 model for reranking by replacing the OpenAI client with Anthropic's SDK. For async workflows, use async clients if supported. Adjust top_n to retrieve more or fewer candidates. You can also embed documents and rerank by cosine similarity using OpenAI embeddings.

python

import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

rerank_prompt = "You are a search relevance model. Rank these documents by relevance to the query.\nQuery: How to improve search results with AI?\nDocuments:\n1. Python is a programming language.\n2. OpenAI provides powerful AI models.\n3. BM25 is a ranking function used by search engines.\n4. Reranking improves search results by rescoring candidates.\n\nReturn the document numbers sorted by relevance, most relevant first, as a comma-separated list."

message = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=100,
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": rerank_prompt}]
)
rerank_order = message.content.strip()
print("Rerank order:", rerank_order)

output

Rerank order: 4, 2, 3, 1

Troubleshooting

If the reranking output is not parsable, ensure the prompt clearly asks for a comma-separated list of document numbers.
If BM25 returns poor candidates, check tokenization and corpus quality.
API errors usually relate to missing or invalid API keys; verify environment variables.

✅

Key Takeaways

Use BM25 for fast initial retrieval of candidate documents based on lexical matching.
Rerank top BM25 candidates with an AI chat model to leverage semantic understanding.
Clear prompt instructions improve reranking output parsing and accuracy.

Verified 2026-04 · gpt-4o-mini, claude-3-5-sonnet-20241022

Verify ↗