OutputParserException
langchain.schema.output_parser.OutputParserException
Stack trace
langchain_core.exceptions.OutputParserException: Could not parse LLM output: ```json
{"name": "John", "age": 30}
```
During handling of the above exception, the program crashed. Why it happens
Mistral models in JSON mode sometimes return output wrapped in markdown fences or include extra text, causing strict JSON parsers to fail. The parser expects raw JSON without any formatting or preamble, so any deviation triggers this exception.
Detection
Catch OutputParserException around the parser call and log the raw LLM output to detect unexpected markdown fences or formatting before retrying or cleaning the response.
Causes & fixes
Mistral returns JSON output wrapped in markdown fences (```json ... ```)
Modify the prompt to instruct Mistral to return raw JSON only, or use a parser like JsonOutputParser that automatically strips markdown fences.
Extra explanatory text or preamble is included before or after the JSON output
Refine the prompt to explicitly request only the JSON object without any additional text or commentary.
Mismatch between expected JSON schema and actual keys or structure returned by Mistral
Align the expected Pydantic or JSON schema fields exactly with the keys Mistral outputs, including case sensitivity.
Using a base Mistral model that does not reliably follow output format instructions
Switch to an instruction-tuned Mistral variant or add stronger prompt instructions emphasizing strict JSON output.
Code: broken vs fixed
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage
client = ChatOpenAI(model_name='mistral-7b', temperature=0)
prompt = 'Return a JSON object with name and age.'
response = client([HumanMessage(content=prompt)])
# This line triggers OutputParserException due to markdown fences
parsed = json.loads(response.content)import os
import json
from langchain_openai import ChatOpenAI
from langchain.schema import HumanMessage
from langchain.output_parsers import JsonOutputParser
client = ChatOpenAI(model_name='mistral-7b', temperature=0)
prompt = 'Return ONLY raw JSON object with name and age, no markdown fences or extra text.'
response = client([HumanMessage(content=prompt)])
parser = JsonOutputParser() # Handles markdown fences automatically
parsed = parser.parse(response.content)
print(parsed) # Should print the parsed JSON dictWorkaround
Wrap the parsing call in try/except OutputParserException, then use regex to extract JSON substring from the raw response and parse it with json.loads() as a fallback.
Prevention
Use structured output modes or response_format parameters at the API level to enforce strict JSON output, avoiding parser errors from formatting inconsistencies.