How to beginner · 3 min read

How to test Pydantic AI agents

Q: How to test Pydantic AI agents

To test Pydantic AI agents, instantiate an Agent with your model and run synchronous or asynchronous calls using agent.run_sync() or agent.run(). Validate the returned BaseModel data for correctness and expected fields.

Quick answer

To test Pydantic AI agents, instantiate an Agent with your model and run synchronous or asynchronous calls using agent.run_sync() or agent.run(). Validate the returned BaseModel data for correctness and expected fields.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install openai>=1.0 pydantic-ai pydantic

Setup

Install the required packages and set your OpenAI API key as an environment variable.

Install packages: openai, pydantic-ai, and pydantic.
Set OPENAI_API_KEY in your environment.

bash

pip install openai pydantic-ai pydantic

output

Collecting openai
Collecting pydantic-ai
Collecting pydantic
Successfully installed openai pydantic-ai pydantic-2.x.x

Step by step

Define a Pydantic BaseModel for the expected output, create an Agent with the OpenAI model, then run a test prompt and assert the output fields.

python

import os
from pydantic import BaseModel
from pydantic_ai import Agent

# Define the expected response model
class QAResponse(BaseModel):
    answer: str
    confidence: float

# Initialize the agent with OpenAI GPT-4o-mini
agent = Agent(
    "openai:gpt-4o-mini",
    system_prompt="You are a helpful assistant.",
    result_type=QAResponse,
    api_key=os.environ["OPENAI_API_KEY"]
)

# Run a synchronous test
result = agent.run_sync("What is Retrieval Augmented Generation?")

# Access the typed response
print(f"Answer: {result.data.answer}")
print(f"Confidence: {result.data.confidence}")

output

Answer: Retrieval Augmented Generation (RAG) is a technique that combines retrieval of relevant documents with generation of answers.
Confidence: 0.95

Common variations

You can test Pydantic AI agents asynchronously using await agent.run(), switch models by changing the model string, or customize the system_prompt for different behaviors.

python

import asyncio

async def async_test():
    result = await agent.run("Explain RAG in simple terms.")
    print(f"Async answer: {result.data.answer}")

asyncio.run(async_test())

output

Async answer: RAG helps AI answer questions by looking up information and then writing a response based on it.

Troubleshooting

If you get validation errors, ensure your BaseModel matches the expected response structure.
If the agent returns unexpected data, check your system_prompt and model choice.
Verify your OPENAI_API_KEY is set correctly to avoid authentication errors.

✅

Key Takeaways

Use agent.run_sync() or agent.run() to test Pydantic AI agents with typed outputs.
Define clear BaseModel schemas to validate AI responses automatically.
Switch between synchronous and asynchronous testing easily depending on your app needs.

Verified 2026-04 · gpt-4o-mini

Verify ↗