How to Intermediate · 4 min read

How to use Azure Cognitive Search for RAG

Q: How to use Azure Cognitive Search for RAG

Use Azure Cognitive Search to index and retrieve relevant documents, then pass those documents as context to Azure OpenAI chat completions for Retrieval-Augmented Generation (RAG). This involves querying the search index for relevant content and combining it with your prompt to generate informed AI responses.

Quick answer

Use Azure Cognitive Search to index and retrieve relevant documents, then pass those documents as context to Azure OpenAI chat completions for Retrieval-Augmented Generation (RAG). This involves querying the search index for relevant content and combining it with your prompt to generate informed AI responses.

PREREQUISITES

Python 3.8+
Azure subscription with Cognitive Search and Azure OpenAI deployed
pip install azure-search-documents azure-identity openai>=1.0

Setup

Install the required Python packages and set environment variables for authentication.

Install Azure SDKs and OpenAI client:

bash

pip install azure-search-documents azure-identity openai>=1.0

Step by step

This example demonstrates querying Azure Cognitive Search for relevant documents and using the results as context in an Azure OpenAI chat completion for RAG.

python

import os
from azure.search.documents import SearchClient
from azure.identity import DefaultAzureCredential
from openai import AzureOpenAI

# Set environment variables before running:
# AZURE_SEARCH_ENDPOINT, AZURE_SEARCH_INDEX_NAME, AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT

# Initialize Azure Cognitive Search client
search_endpoint = os.environ["AZURE_SEARCH_ENDPOINT"]
index_name = os.environ["AZURE_SEARCH_INDEX_NAME"]
credential = DefaultAzureCredential()
search_client = SearchClient(endpoint=search_endpoint, index_name=index_name, credential=credential)

# Initialize Azure OpenAI client
client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version="2024-02-01"
)

# User query
query = "Explain the benefits of renewable energy"

# Step 1: Search for relevant documents
search_results = search_client.search(query, top=3)

# Extract content from search results
context_docs = []
for doc in search_results:
    # Assuming documents have a 'content' field
    context_docs.append(doc["content"])

# Combine retrieved documents as context
context_text = "\n---\n".join(context_docs)

# Step 2: Create prompt with context for RAG
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": f"Use the following documents to answer the question:\n{context_text}\nQuestion: {query}"}
]

# Step 3: Call Azure OpenAI chat completion
response = client.chat.completions.create(
    model=os.environ["AZURE_OPENAI_DEPLOYMENT"],
    messages=messages
)

print("AI response:")
print(response.choices[0].message.content)

output

AI response:
Renewable energy offers numerous benefits including reducing greenhouse gas emissions, decreasing dependence on fossil fuels, and promoting sustainable development...

Common variations

Use async versions of Azure SDK and OpenAI client for non-blocking calls.
Adjust top parameter in search to retrieve more or fewer documents.
Use different Azure OpenAI models or deployments by changing model parameter.
Preprocess or filter search results to improve context quality.

Troubleshooting

If you get authentication errors, verify your Azure credentials and environment variables.
If search returns no results, check your index data and query syntax.
For Azure OpenAI errors, confirm your deployment name and API version match your Azure portal configuration.
Ensure network access and firewall rules allow your app to reach Azure services.

✅

Key Takeaways

Use Azure Cognitive Search to retrieve relevant documents as context for RAG.
Pass retrieved documents in the prompt to Azure OpenAI chat completions for informed answers.
Configure environment variables for Azure credentials and endpoints to authenticate securely.

Verified 2026-04 · gpt-4o, azure_openai

Verify ↗