JSONDecodeError
json.decoder.JSONDecodeError
Stack trace
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
File "/usr/local/lib/python3.9/site-packages/langchain/schema/output_parser.py", line 123, in parse
return json.loads(text)
File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None Why it happens
LangChain expects the LLM to return a strictly valid JSON string matching the parser schema. However, many LLMs return JSON wrapped in markdown fences, extra text, or with formatting errors. This causes the JSON parser to fail with JSONDecodeError.
Detection
Log the raw LLM output before parsing and assert it starts and ends with valid JSON braces/brackets. Use try/except around parsing to catch JSONDecodeError and inspect the raw response.
Causes & fixes
LLM response includes markdown fences (```json) around the JSON output
Modify the prompt to instruct the model to return raw JSON only, or use LangChain's JsonOutputParser which strips markdown fences automatically.
LLM returns extra explanatory text or preamble before or after the JSON
Add explicit prompt instructions to return only the JSON object without any additional text.
Using a base model that is not instruction-tuned and ignores output format constraints
Switch to an instruction-tuned model like gpt-4o-mini or claude-3-5-haiku-20241022 that reliably follows output format instructions.
Code: broken vs fixed
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage
llm = ChatOpenAI(model_name="gpt-4o")
response = llm([HumanMessage(content="Generate JSON output")])
json_data = json.loads(response.content) # JSONDecodeError here if response has markdown fencesimport os
import json
from langchain_openai import ChatOpenAI
from langchain.output_parsers import JsonOutputParser
from langchain.schema import HumanMessage
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY") # Use env var for API key
llm = ChatOpenAI(model_name="gpt-4o-mini")
parser = JsonOutputParser() # Handles markdown fences and retries
response = llm([HumanMessage(content="Generate JSON output only, no markdown fences")])
json_data = parser.parse(response.content)
print(json_data) # Works without JSONDecodeErrorWorkaround
Wrap the JSON parsing in try/except JSONDecodeError, then extract JSON substring using regex from the raw LLM output and parse it manually with json.loads() as a fallback.
Prevention
Use structured output features like OpenAI's response_format or LangChain's JsonOutputParser to enforce schema-valid JSON responses at the API level, avoiding fragile manual parsing.