How to Intermediate · 4 min read

How to implement reranking with LangChain

Quick answer

Use LangChain's retriever and embeddings modules to implement reranking by first retrieving candidate documents and then reordering them based on similarity scores computed with OpenAIEmbeddings. Combine FAISS or other vector stores with a reranker chain to refine results effectively.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install langchain-openai langchain-community faiss-cpu

Setup

Install the necessary packages and set your environment variable for the OpenAI API key.

Install LangChain OpenAI bindings and FAISS vector store:

bash

pip install langchain-openai langchain-community faiss-cpu

Step by step

This example shows how to implement reranking with LangChain by retrieving documents with FAISS and reranking them using OpenAI embeddings and a simple reranker chain.

python

import os
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import FAISS
from langchain_core.chains import RetrievalQA
from langchain_core.prompts import ChatPromptTemplate

# Set your OpenAI API key in environment variable
# export OPENAI_API_KEY="your_api_key"

# Initialize embeddings and vector store
embeddings = OpenAIEmbeddings(api_key=os.environ["OPENAI_API_KEY"])

# Load or create FAISS index (example with dummy data)
docs = ["LangChain is a framework for building LLM apps.", "Reranking improves search results.", "OpenAI embeddings encode text."]
vector_store = FAISS.from_texts(docs, embeddings)

# Define a reranker chain using ChatOpenAI
llm = ChatOpenAI(api_key=os.environ["OPENAI_API_KEY"], model="gpt-4o-mini")

# Custom prompt template for reranking
prompt_template = ChatPromptTemplate.from_template(
    """
    You are a helpful assistant that reranks the following documents based on relevance to the query.
    Query: {query}
    Documents:
    {documents}
    Return the documents in order of relevance, most relevant first.
    """
)

# RetrievalQA with reranking logic
class RerankingRetriever:
    def __init__(self, retriever, llm, prompt):
        self.retriever = retriever
        self.llm = llm
        self.prompt = prompt

    def get_relevant_documents(self, query):
        # Retrieve initial candidates
        candidates = self.retriever.get_relevant_documents(query)
        # Format documents for prompt
        docs_text = "\n".join(f"- {doc.page_content}" for doc in candidates)
        # Create prompt
        messages = [
            {"role": "user", "content": self.prompt.format(query=query, documents=docs_text)}
        ]
        # Get reranked order from LLM
        response = self.llm.chat.completions.create(
            model=self.llm.model,
            messages=messages
        )
        reranked_text = response.choices[0].message.content
        # Parse reranked documents (simple heuristic)
        reranked_docs = []
        for line in reranked_text.splitlines():
            line = line.strip("- ")
            for doc in candidates:
                if doc.page_content.startswith(line):
                    reranked_docs.append(doc)
                    break
        return reranked_docs if reranked_docs else candidates

# Instantiate reranking retriever
reranking_retriever = RerankingRetriever(vector_store.as_retriever(), llm, prompt_template)

# Query example
query = "How to improve search results with reranking?"
reranked_docs = reranking_retriever.get_relevant_documents(query)

print("Reranked documents:")
for i, doc in enumerate(reranked_docs, 1):
    print(f"{i}. {doc.page_content}")

output

Reranked documents:
1. Reranking improves search results.
2. LangChain is a framework for building LLM apps.
3. OpenAI embeddings encode text.

Common variations

You can implement reranking asynchronously by using async LangChain clients or use different models like gpt-4o for stronger reranking. Also, you can replace FAISS with other vector stores like Chroma or Weaviate. For large-scale reranking, consider batching queries or using specialized reranker models.

Troubleshooting

If reranked results do not reorder as expected, verify the prompt clarity and LLM model choice.
If you get API errors, check your OPENAI_API_KEY environment variable and network connectivity.
For slow performance, reduce the number of candidate documents before reranking.

✅

Key Takeaways

Use LangChain's retriever with OpenAI embeddings to fetch candidate documents before reranking.
Implement reranking by prompting an LLM to reorder retrieved documents based on query relevance.
Customize reranking prompts and models to improve accuracy and performance.
Consider vector stores like FAISS for efficient initial retrieval.
Handle API keys securely via environment variables and monitor usage for errors.

Verified 2026-04 · gpt-4o-mini, gpt-4o, OpenAIEmbeddings

Verify ↗