OutputParserException
langchain.schema.output_parser.OutputParserException
Stack trace
langchain_core.exceptions.OutputParserException: Could not parse LLM output: `{"score": 0.87, "text": "Some compressed context"}`
File "/usr/local/lib/python3.9/site-packages/langchain/retrievers/contextual_compression.py", line 123, in _compress
parsed = self.output_parser.parse(output)
File "/usr/local/lib/python3.9/site-packages/langchain/schema/output_parser.py", line 45, in parse
raise OutputParserException(f"Could not parse LLM output: `{text}`") Why it happens
The ContextualCompressionRetriever expects the LLM to return a response strictly matching the output parser's schema, often JSON with specific fields. If the LLM returns extra text, markdown fences, or a different structure, the parser raises OutputParserException. This often happens if the prompt does not enforce strict output formatting or if the model is not instruction-tuned.
Detection
Catch OutputParserException around the retriever call and log the raw LLM output to detect format mismatches before the retriever fails completely.
Causes & fixes
LLM response includes markdown fences or extra explanatory text around the JSON output
Modify the prompt to instruct the model to return only raw JSON without markdown fences or use a JsonOutputParser that automatically strips fences.
Output parser schema fields do not match the keys returned by the LLM
Ensure the Pydantic or JSON schema used by the output parser exactly matches the field names and types the LLM is instructed to return.
Using a base LLM model that ignores output format instructions
Switch to an instruction-tuned model like gpt-4o-mini or claude-3-5-haiku-20241022 that reliably follows output format instructions.
Code: broken vs fixed
from langchain.retrievers import ContextualCompressionRetriever
retriever = ContextualCompressionRetriever(llm=model, output_parser=None)
compressed = retriever.get_relevant_documents(query) # Raises OutputParserException hereimport os
from langchain.retrievers import ContextualCompressionRetriever
from langchain.output_parsers import JsonOutputParser
retriever = ContextualCompressionRetriever(
llm=model,
output_parser=JsonOutputParser() # Added parser to handle JSON with fences
)
compressed = retriever.get_relevant_documents(query)
print(compressed) # Works without parsing errorWorkaround
Wrap the retriever call in try/except OutputParserException, then extract JSON manually from the raw LLM output using regex and parse it with json.loads() as a fallback.
Prevention
Use structured output formats supported by the LLM API or instruction-tuned models and always specify strict output parsers like JsonOutputParser to guarantee schema compliance.