OutputParserException
dspy.parsers.chainofthought.OutputParserException
Stack trace
dspy.parsers.chainofthought.OutputParserException: Could not parse ChainOfThought output: '{"reasoning": "Step 1: ...", "answer": 42}'
File "/usr/local/lib/python3.9/site-packages/dspy/parsers/chainofthought.py", line 87, in parse
raise OutputParserException(f"Could not parse ChainOfThought output: {output}") Why it happens
DSPy's ChainOfThought parser expects the LLM to return a structured JSON object with specific keys like 'reasoning' and 'answer'. If the LLM returns extra text, markdown fences, or deviates from the schema, the parser raises this exception. This often happens when the prompt does not enforce strict output formatting or when using base models that ignore format instructions.
Detection
Wrap calls to the ChainOfThought parser in try/except OutputParserException and log the raw LLM output to detect format mismatches before the app crashes.
Causes & fixes
LLM output includes markdown fences or extra commentary around the JSON response
Modify the prompt to instruct the model to return only raw JSON without markdown fences, or use DSPy's JsonOutputParser which automatically strips fences.
The expected JSON keys in the parser schema do not match the keys returned by the LLM
Ensure the parser's expected keys exactly match the keys in the prompt's output format, including case sensitivity.
Using a base LLM model that does not reliably follow output format instructions
Switch to an instruction-tuned model supported by DSPy, such as gpt-4o-mini or claude-3-5-haiku-20241022, which better respects output formatting.
Code: broken vs fixed
from dspy import ChainOfThoughtParser
parser = ChainOfThoughtParser()
output = llm.generate(prompt)
result = parser.parse(output) # Raises OutputParserException hereimport os
from dspy import ChainOfThoughtParser, JsonOutputParser
os.environ['DSPY_API_KEY'] = os.getenv('DSPY_API_KEY') # Use env var for API key
parser = JsonOutputParser().with_retry(n=3) # Added JsonOutputParser with retry
output = llm.generate(prompt)
result = parser.parse(output) # Now parses robustly
print(result)Workaround
Catch OutputParserException, extract JSON substring from the raw LLM output using regex, then parse it manually with json.loads() as a fallback.
Prevention
Use structured output enforcement at the API level by specifying response_format options or using instruction-tuned models to guarantee schema-valid responses and avoid parser errors.