Debug Fix medium · 3 min read

Fix JSON parse error from LLM response

Quick answer

JSON parse errors from LLM responses occur when the model outputs malformed or incomplete JSON strings. Use prompt engineering to enforce strict JSON formatting and parse the response safely with error handling in Python to fix this.

ERROR TYPE model_behavior

⚡ QUICK FIX

Add explicit instructions in your prompt to output valid JSON and wrap JSON parsing in try-except blocks to handle malformed responses gracefully.

Why this happens

LLMs often generate text that looks like JSON but may include trailing commas, missing quotes, or extra explanations, causing json.JSONDecodeError in Python. For example, a prompt asking for JSON output without strict formatting instructions can produce invalid JSON:

{
  "name": "Alice",
  "age": 30,
}

This trailing comma breaks JSON parsing. The error message typically is:

json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 45)

python

from openai import OpenAI
import os
import json

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

response = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Give me a JSON object with name and age."}]
)

text = response.choices[0].message.content

# This will raise JSONDecodeError if text is malformed
parsed = json.loads(text)

output

json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 45)

The fix

Fix this by instructing the model explicitly to output only valid JSON without extra text and by parsing with error handling. Use a system prompt like "Respond only with valid JSON." and strip whitespace before parsing. Example code:

python

from openai import OpenAI
import os
import json

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

system_prompt = "You are a helpful assistant. Respond only with valid JSON, no explanations."
user_prompt = "Give me a JSON object with name and age."

response = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt}
    ]
)

text = response.choices[0].message.content.strip()

try:
    parsed = json.loads(text)
except json.JSONDecodeError as e:
    print(f"JSON parse error: {e}")
    # Optionally retry or clean text here
else:
    print(parsed)

output

{'name': 'Alice', 'age': 30}

Preventing it in production

Implement retries with exponential backoff on parse failures, validate JSON schema after parsing, and consider using JSON schema validation libraries. Also, sanitize or extract JSON from noisy responses using regex or specialized parsers if needed. Logging malformed outputs helps diagnose prompt issues.

python

import time

max_retries = 3
for attempt in range(max_retries):
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt}
        ]
    )
    text = response.choices[0].message.content.strip()
    try:
        parsed = json.loads(text)
        break
    except json.JSONDecodeError:
        wait = 2 ** attempt
        time.sleep(wait)
else:
    raise ValueError("Failed to parse JSON after retries")

print(parsed)

output

{'name': 'Alice', 'age': 30}

Related errors

Error	Cause	Quick fix
json.JSONDecodeError	Malformed JSON output from LLM	Add strict JSON instructions and parse with try-except
KeyError	Missing expected JSON keys	Validate keys after parsing and handle missing keys
RateLimitError	Too many API requests	Add exponential backoff retry logic

✅

Key Takeaways

Always instruct LLMs explicitly to output valid JSON only to reduce parse errors.
Wrap JSON parsing in try-except blocks to handle malformed responses gracefully.
Implement retries with backoff and validate JSON schema in production AI apps.

Verified 2026-04 · gpt-4.1-mini

Verify ↗