How to retry on parse error in LangChain
Quick answer
In
LangChain, parse errors occur when the output does not match the expected structured format. To retry on parse errors, wrap your chain invocation in a try-except block catching OutputParserException and re-invoke the chain, optionally with a retry limit or backoff. ERROR TYPE
code_error ⚡ QUICK FIX
Wrap your LangChain call in a try-except catching
OutputParserException and retry the call with a limit to handle parse errors gracefully.Why this happens
Parse errors in LangChain occur when the AI model's output does not conform to the expected structured format defined by your OutputParser. For example, if you use StructuredOutputParser to parse JSON but the model returns malformed JSON or unexpected text, a OutputParserException is raised.
This typically happens when the model output is ambiguous or the prompt does not sufficiently constrain the format. The error looks like:
OutputParserException: Could not parse output: ...
Example broken code:
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StructuredOutputParser
model = ChatOpenAI(model="gpt-4o", temperature=0)
parser = StructuredOutputParser.from_format_string('{"answer": string}')
response = model.chat.completions.create(
messages=[{"role": "user", "content": "Give me a JSON answer."}]
)
# This will raise OutputParserException if response is not valid JSON
parsed = parser.parse(response.choices[0].message.content) output
OutputParserException: Could not parse output: ...
The fix
To fix parse errors, catch OutputParserException and retry the call. Implement a retry loop with a maximum retry count to avoid infinite loops. This works because you give the model multiple chances to produce valid structured output.
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StructuredOutputParser, OutputParserException
import os
model = ChatOpenAI(model="gpt-4o", temperature=0)
parser = StructuredOutputParser.from_format_string('{"answer": string}')
max_retries = 3
for attempt in range(max_retries):
response = model.chat.completions.create(
messages=[{"role": "user", "content": "Give me a JSON answer."}]
)
output = response.choices[0].message.content
try:
parsed = parser.parse(output)
print("Parsed output:", parsed)
break
except OutputParserException as e:
print(f"Parse error on attempt {attempt + 1}: {e}")
if attempt == max_retries - 1:
raise output
Parsed output: {'answer': '...'} Preventing it in production
- Use robust
OutputParserdefinitions that tolerate minor format variations. - Implement retry logic with exponential backoff to handle transient parse errors.
- Validate model outputs before parsing to catch obvious format issues early.
- Log parse errors and fallback gracefully, e.g., return a default response or ask the user to rephrase.
- Test prompts extensively to ensure consistent structured output from the model.
Key Takeaways
- Always catch
OutputParserExceptionwhen parsing structured outputs in LangChain. - Implement retry loops with limits to handle transient parse errors gracefully.
- Design robust parsers and validate outputs to reduce parse errors in production.