API Intermediate medium · 6 min

JSON mode vs structured outputs: the difference

What you will learn
JSON mode forces valid JSON output syntax, while structured outputs guarantee schema-validated responses with guaranteed function calling.

Why this matters

You need structured data from LLMs in production. JSON mode is cheaper but can fail validation; structured outputs cost more but are production-safe with guaranteed schema compliance and type safety.

Skip if: Use plain text responses when you don't need to parse the output. Use JSON mode only for cost-critical applications where occasional invalid JSON is acceptable. Use structured outputs when correctness is mandatory (financial transactions, database inserts, API integrations).

Explanation

JSON mode (response_format={'type': 'json_object'}) tells the model to output valid JSON syntax, but does not validate the structure. The model can return valid JSON that doesn't match your schema. Structured outputs (response_format={'type': 'json_schema', 'json_schema': {...}}) provide a JSON Schema definition to the model, and OpenAI guarantees the response will conform to that schema: the model sees your schema during generation and adjusts its output accordingly. The second approach is newer and more reliable. Under the hood, JSON mode uses constraints on the token generation, while structured outputs feed the schema into the model's context to steer generation. JSON mode costs the same as regular responses. Structured outputs have a 25% token usage increase. Use JSON mode for quick prototyping or low-stakes parsing where you can handle occasional malformed output. Use structured outputs for production systems, financial data, database writes, or any context where a schema mismatch causes cascading failures.

Request code

python
import os
from openai import OpenAI
import json

client = OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))

user_text = "I bought 3 apples at $2.50 each and 2 oranges at $1.75 each"

print("=== JSON MODE ===")
response_json_mode = client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {
            "role": "user",
            "content": f"Extract the items and prices from this receipt: {user_text}. Return valid JSON only."
        }
    ],
    response_format={"type": "json_object"}
)
print(response_json_mode.choices[0].message.content)
print()

print("=== STRUCTURED OUTPUTS ===")
response_structured = client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {
            "role": "user",
            "content": f"Extract the items and prices from this receipt: {user_text}"
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "receipt_items",
            "schema": {
                "type": "object",
                "properties": {
                    "items": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {"type": "string"},
                                "quantity": {"type": "integer"},
                                "unit_price": {"type": "number"},
                                "total": {"type": "number"}
                            },
                            "required": ["name", "quantity", "unit_price", "total"]
                        }
                    },
                    "grand_total": {"type": "number"}
                },
                "required": ["items", "grand_total"]
            },
            "strict": True
        }
    }
)
data = json.loads(response_structured.choices[0].message.content)
print(json.dumps(data, indent=2))
print()
print(f"Grand total parsed as number: {data['grand_total']}")
print(f"First item: {data['items'][0]['name']} x {data['items'][0]['quantity']}")

Authentication

Ensure OPENAI_API_KEY is set in your environment. Both features require API key authentication: export OPENAI_API_KEY='sk-...' No additional auth headers are needed beyond the standard OpenAI client initialization.

Response shape

FieldDescription
choices List of completion choices
choices[0].message.content String containing the JSON response (either valid JSON or schema-compliant JSON object)
choices[0].message.refusal If present, the model refused to complete the request
usage.prompt_tokens Tokens used in the input (same for both modes)
usage.completion_tokens Tokens used in output (25% higher for structured outputs)
model The model used (e.g., 'gpt-4.1')

Field guide

choices[0].message.content

For JSON mode: valid JSON string (may not match your intended schema). For structured outputs: JSON string guaranteed to match the schema you provided. You must parse this string with json.loads().

usage.completion_tokens

Structured outputs are more expensive. Track this separately if you're monitoring cost per request. The 25% increase is consistent across API calls.

choices[0].message.refusal

In structured outputs with strict:true, refusal indicates the model couldn't generate valid output. This is rare but signals a schema that's too restrictive or a user prompt that contradicts the schema.

Setup trap

If your OpenAI() client is initialized before OPENAI_API_KEY is set in the environment, both JSON mode and structured outputs will fail silently at request time with authentication errors. The SDK reads the API key at instantiation time, not at request time. Set the environment variable or pass api_key explicitly to OpenAI(api_key='sk-...') before creating the client. Trying to set os.environ['OPENAI_API_KEY'] after OpenAI() is called will not work.

Cost

JSON mode: same token cost as regular completions. Structured outputs: 25% increase in completion tokens. For a 1000-token response, expect 250 extra tokens. At $0.06 per 1M input tokens and $0.18 per 1M completion tokens, structured outputs cost ~$0.0000045 more per 1000-token response. For high-volume applications (100k requests/day), this adds up to ~$1.62/day.

Rate limits

No special rate limit behavior. Standard tier limits apply equally to both JSON mode and structured outputs.

Common gotcha

JSON mode returns a string that passes JSON syntax validation but can have fields missing or types wrong. Developers assume the schema is enforced, then .get('field') returns None in production. Always validate the parsed JSON against your expected schema even with JSON mode. With structured outputs, developers sometimes forget that strict:true is optional: omitting it allows the model to return null fields or missing keys; set strict:true to guarantee exact schema adherence.

Error recovery

json.JSONDecodeError
The API returned non-JSON text. Rare with JSON mode but can happen if the model hallucinates or hits edge cases. With structured outputs set to strict:true, if this happens, re-raise: it indicates a model failure. Fallback: try again with a simpler schema or fewer required fields.
jsonschema.ValidationError
You're manually validating parsed JSON against your schema and it fails. With JSON mode only. Solution: check the parsed JSON for missing fields, wrong types, or extra keys. Log the response and the validation error to understand the gap.
ValueError (strict:true with unmatched schema)
You set strict:true but the model couldn't conform. The response will be a refusal in choices[0].message.refusal. Solution: simplify the schema, add examples in the prompt, or reduce constraints.

Experienced dev note

Structured outputs with strict:true eliminate an entire class of production bugs: invalid JSON, missing required fields, type mismatches. The 25% token cost is worth it the moment you have >10k API calls/month. Use structured outputs for anything touching a database, payment processor, or downstream system. Save JSON mode for exploratory work and user-facing text that happens to be JSON. Also: structured outputs prevent model refusals more reliably because the schema is explicit: if the model sees it can't generate compliant output, it fails fast rather than returning garbage you have to debug downstream.

Check your understanding

You're building a financial transaction validator. Your schema has strict:true with required fields for amount (number), currency (enum of 3 options), and account_id. The user sends an edge case prompt: 'I want to send $500 but I'm not sure which account.' Should you use JSON mode or structured outputs, and why would choosing wrong cause a production issue?

Show answer hint

Structured outputs with strict:true will refuse rather than return a partial or hallucinated account_id. JSON mode might return a made-up account_id that passes JSON syntax but fails your database foreign key constraint. The question is whether you want predictable failure (refusal) vs. unpredictable silent data corruption.

VERSION Structured outputs (json_schema with strict:true) were introduced in November 2024 and require gpt-4-turbo or gpt-4o. JSON mode has been available since gpt-3.5-turbo. Always pin model version to gpt-4.1 or later for production use. older versions may not support strict:true.

Community Notes

No notes yetBe the first to share a version-specific fix or tip.