How to beginner · 3 min read

How to use Haystack with Anthropic

Q: How to use Haystack with Anthropic

Use haystack v2 with OpenAIGenerator configured for Anthropic by passing your Anthropic API key and specifying an Anthropic model like claude-3-5-sonnet-20241022. This enables you to build retrieval-augmented generation pipelines combining Haystack's retrievers with Anthropic's LLM completions.

Quick answer

Use haystack v2 with OpenAIGenerator configured for Anthropic by passing your Anthropic API key and specifying an Anthropic model like claude-3-5-sonnet-20241022. This enables you to build retrieval-augmented generation pipelines combining Haystack's retrievers with Anthropic's LLM completions.

PREREQUISITES

Python 3.8+
Anthropic API key (set as ANTHROPIC_API_KEY environment variable)
pip install haystack-ai>=2.0 anthropic>=0.20

Setup

Install the latest haystack-ai package (v2+) and the anthropic Python SDK. Set your Anthropic API key as an environment variable ANTHROPIC_API_KEY before running the code.

bash

pip install haystack-ai anthropic

Step by step

This example shows how to create a Haystack pipeline using an in-memory document store, a BM25 retriever, and the OpenAIGenerator configured to use Anthropic's claude-3-5-sonnet-20241022 model for generation.

python

import os
from haystack import Pipeline
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import BM25Retriever
from haystack.components.generators import OpenAIGenerator

# Ensure your Anthropic API key is set in the environment
api_key = os.environ["ANTHROPIC_API_KEY"]

# Initialize document store and add sample documents
document_store = InMemoryDocumentStore()
docs = [
    {"content": "Anthropic builds helpful AI assistants."},
    {"content": "Haystack enables building search pipelines."}
]
document_store.write_documents(docs)

# Initialize BM25 retriever
doc_retriever = BM25Retriever(document_store=document_store)

# Initialize OpenAIGenerator with Anthropic API key and model
generator = OpenAIGenerator(
    api_key=api_key,
    model="claude-3-5-sonnet-20241022"
)

# Build pipeline with retriever and generator
pipeline = Pipeline()
pipeline.add_node(component=doc_retriever, name="Retriever", inputs=["Query"])
pipeline.add_node(component=generator, name="Generator", inputs=["Retriever"])

# Run a query
query = "What does Anthropic do?"
result = pipeline.run(query=query)

print("Answer:", result["answers"][0].answer)

output

Answer: Anthropic builds helpful AI assistants.

Common variations

Use different Anthropic models like claude-sonnet-4-5 by changing the model parameter in OpenAIGenerator.
Replace BM25Retriever with other retrievers like DensePassageRetriever for semantic search.
Use async pipelines by leveraging Haystack's async support and Anthropic's async client.

Troubleshooting

If you get authentication errors, verify your ANTHROPIC_API_KEY environment variable is set correctly.
If the pipeline returns empty answers, ensure documents are properly written to the document store.
Check network connectivity if API calls to Anthropic fail.

✅

Key Takeaways

Use Haystack v2 with OpenAIGenerator configured for Anthropic to build retrieval-augmented generation pipelines.
Set your Anthropic API key in the environment variable ANTHROPIC_API_KEY for authentication.
Combine Haystack retrievers like BM25Retriever with Anthropic models such as claude-3-5-sonnet-20241022 for best results.

Verified 2026-04 · claude-3-5-sonnet-20241022

Verify ↗