How to Intermediate · 4 min read

How to use MCP with LlamaIndex

Q: How to use MCP with LlamaIndex

Use the mcp Python SDK to create an MCP server that connects to LlamaIndex for managing AI agent context and tool access. Implement the stdio_server from mcp.server.stdio to handle communication, and use LlamaIndex APIs within the MCP server handlers to query or update your index.

Quick answer

Use the mcp Python SDK to create an MCP server that connects to LlamaIndex for managing AI agent context and tool access. Implement the stdio_server from mcp.server.stdio to handle communication, and use LlamaIndex APIs within the MCP server handlers to query or update your index.

PREREQUISITES

Python 3.8+
pip install mcp llama-index
OpenAI API key (for LlamaIndex usage)
Set environment variable OPENAI_API_KEY

Setup

Install the required packages mcp and llama-index via pip, and set your OpenAI API key in the environment for LlamaIndex to function.

bash

pip install mcp llama-index

Step by step

This example shows how to create an MCP stdio server that integrates LlamaIndex to respond to AI agent queries by searching an index.

python

import os
from mcp.server import Server
from mcp.server.stdio import stdio_server
from llama_index import SimpleDirectoryReader, GPTVectorStoreIndex

# Load documents and build LlamaIndex
documents = SimpleDirectoryReader('data').load_data()
index = GPTVectorStoreIndex(documents)

# Define MCP server handler
class LlamaIndexMCPServer(Server):
    def handle(self, request):
        # Extract query from MCP request
        query = request.get('query', '')
        # Query LlamaIndex
        response = index.query(query)
        # Return response text
        return {'result': str(response)}

if __name__ == '__main__':
    # Run MCP stdio server with LlamaIndex handler
    stdio_server(LlamaIndexMCPServer())

Common variations

Use async MCP server by implementing async handle method and running an async stdio server.
Integrate other LlamaIndex index types like GPTListIndex or GPTTreeIndex depending on your data structure.
Use different AI models by configuring LlamaIndex with alternative OpenAI or Anthropic models.

Troubleshooting

If the MCP server does not start, ensure no other process is using stdio and that Python environment is correct.
If LlamaIndex queries fail, verify your OpenAI API key is set in os.environ['OPENAI_API_KEY'].
For unexpected MCP request formats, log incoming requests to debug the handle method.

✅

Key Takeaways

Use the official mcp Python SDK to build MCP servers that integrate with LlamaIndex for AI agent tool access.
Implement the handle method in your MCP server class to process queries and return results from LlamaIndex.
Run the MCP server with stdio_server for standard input/output communication with AI agents.
Configure LlamaIndex with your preferred document loaders and AI models for flexible context management.
Always set your OpenAI API key in environment variables to enable LlamaIndex functionality.

Verified 2026-04 · gpt-4o, claude-3-5-sonnet-20241022

Verify ↗