How to Intermediate · 4 min read

How to use AWS Bedrock with LlamaIndex

Q: How to use AWS Bedrock with LlamaIndex

Use AWS Bedrock's boto3 client to call Bedrock LLMs and connect the responses to LlamaIndex by implementing a custom LLM wrapper. This enables you to leverage Bedrock models like anthropic.claude-3-5-sonnet-20241022-v2:0 within LlamaIndex for document indexing and querying.

Quick answer

Use AWS Bedrock's boto3 client to call Bedrock LLMs and connect the responses to LlamaIndex by implementing a custom LLM wrapper. This enables you to leverage Bedrock models like anthropic.claude-3-5-sonnet-20241022-v2:0 within LlamaIndex for document indexing and querying.

PREREQUISITES

Python 3.8+
AWS account with Bedrock access
AWS CLI configured with credentials
pip install boto3 llama-index

Setup

Install the required Python packages and configure AWS credentials for Bedrock access.

Install boto3 and llama-index:

bash

pip install boto3 llama-index

Step by step

This example shows how to create a custom LLM wrapper for AWS Bedrock using boto3 and integrate it with LlamaIndex to index and query documents.

python

import os
import boto3
from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, LLMPredictor, ServiceContext
from llama_index.llms.base import LLM

class BedrockLLM(LLM):
    def __init__(self, model_id: str, region_name: str = "us-east-1"):
        self.model_id = model_id
        self.client = boto3.client("bedrock-runtime", region_name=region_name)

    def _call(self, prompt: str, stop: list = None) -> str:
        response = self.client.invoke_model(
            modelId=self.model_id,
            body=bytes(prompt, "utf-8"),
            contentType="application/json"
        )
        output = response["body"].read().decode("utf-8")
        # Bedrock returns raw text, parse if JSON-wrapped
        return output

    @property
    def _identifying_params(self):
        return {"model_id": self.model_id}

    @property
    def _llm_type(self):
        return "bedrock"

# Initialize Bedrock LLM with Anthropic Claude model
bedrock_llm = BedrockLLM(model_id="anthropic.claude-3-5-sonnet-20241022-v2:0")

# Create LLM predictor and service context for LlamaIndex
llm_predictor = LLMPredictor(llm=bedrock_llm)
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)

# Load documents from local directory
documents = SimpleDirectoryReader("./docs").load_data()

# Build the index
index = GPTVectorStoreIndex.from_documents(documents, service_context=service_context)

# Query the index
query = "What is the main topic of the documents?"
response = index.query(query)
print("Response:", response.response)

output

Response: The main topic of the documents is ...

Common variations

Use different Bedrock models by changing model_id, e.g., amazon.titan-text-express-v1.
Implement async calls with aiobotocore for concurrency.
Customize prompt formatting before sending to Bedrock.

Troubleshooting

If you get AccessDeniedException, verify your AWS Bedrock permissions and credentials.
Ensure your AWS region supports Bedrock.
Check that boto3 is up to date to support Bedrock APIs.
For JSON parsing errors, inspect the raw response["body"] content.

✅

Key Takeaways

Use boto3 with bedrock-runtime client to call AWS Bedrock LLMs.
Wrap Bedrock calls in a custom LLM class to integrate with LlamaIndex seamlessly.
Configure AWS credentials and region properly to avoid access errors.
You can switch Bedrock models by changing the model_id parameter.
Test with local documents and verify responses before production deployment.

Verified 2026-04 · anthropic.claude-3-5-sonnet-20241022-v2:0, amazon.titan-text-express-v1

Verify ↗