How to beginner · 3 min read

How to validate LLM request inputs with Pydantic in FastAPI

Q: How to validate LLM request inputs with Pydantic in FastAPI

Use Pydantic models to define and validate the schema of LLM request inputs in FastAPI. Declare the input data as a BaseModel subclass and use it as a request body parameter to ensure type safety and automatic validation.

Quick answer

Use Pydantic models to define and validate the schema of LLM request inputs in FastAPI. Declare the input data as a BaseModel subclass and use it as a request body parameter to ensure type safety and automatic validation.

PREREQUISITES

Python 3.8+
FastAPI
Uvicorn
pip install fastapi pydantic uvicorn openai>=1.0
OpenAI API key (free tier works)

Setup

Install FastAPI, Uvicorn for the server, and openai SDK for LLM calls. Set your OpenAI API key as an environment variable.

bash

pip install fastapi uvicorn openai>=1.0

Step by step

Define a Pydantic model for the LLM request input, then use it as a parameter in a FastAPI POST endpoint. Validate inputs automatically and call the LLM with the validated data.

python

import os
from fastapi import FastAPI
from pydantic import BaseModel, Field
from openai import OpenAI

app = FastAPI()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

class LLMRequest(BaseModel):
    model: str = Field(..., example="gpt-4o")
    prompt: str = Field(..., min_length=1, example="Say hello")
    max_tokens: int = Field(100, ge=1, le=2048, example=100)

@app.post("/generate")
async def generate_text(request: LLMRequest):
    response = client.chat.completions.create(
        model=request.model,
        messages=[{"role": "user", "content": request.prompt}],
        max_tokens=request.max_tokens
    )
    return {"text": response.choices[0].message.content}

# To run: uvicorn filename:app --reload

output

{
  "text": "Hello"
}

Common variations

Use async openai client calls if supported.
Validate additional fields like temperature or stop sequences with Pydantic.
Switch models by changing the model field value.
Use Field(...) to enforce required fields and constraints.

python

class LLMRequestExtended(BaseModel):
    model: str = Field(..., example="gpt-4o")
    prompt: str = Field(..., min_length=1)
    max_tokens: int = Field(100, ge=1, le=2048)
    temperature: float = Field(0.7, ge=0.0, le=1.0)
    stop: list[str] | None = None

@app.post("/generate-extended")
async def generate_text_extended(request: LLMRequestExtended):
    response = client.chat.completions.create(
        model=request.model,
        messages=[{"role": "user", "content": request.prompt}],
        max_tokens=request.max_tokens,
        temperature=request.temperature,
        stop=request.stop
    )
    return {"text": response.choices[0].message.content}

Troubleshooting

If you get validation errors, check the input JSON matches the Pydantic model schema.
Ensure environment variable OPENAI_API_KEY is set correctly.
Use uvicorn --reload during development to auto-reload changes.
Check API limits and model availability if requests fail.

✅

Key Takeaways

Use Pydantic models to enforce input schema and constraints for LLM requests in FastAPI.
FastAPI automatically validates and documents request bodies defined with Pydantic.
Leverage Pydantic's Field for detailed validation like min_length, ge, le, and examples.
Always source API keys from environment variables for security and flexibility.
Extend models easily to support additional LLM parameters like temperature and stop sequences.

Verified 2026-04 · gpt-4o

Verify ↗