GrammarParseError
llamacpp.GrammarParseError
Stack trace
llamacpp.GrammarParseError: Failed to parse grammar at line 1, column 5: unexpected token '}'
File "llamacpp.py", line 123, in generate
self.model.parse_grammar(prompt)
File "llamacpp.py", line 98, in parse_grammar
raise GrammarParseError(f"Failed to parse grammar at {pos}: {msg}") Why it happens
llama.cpp expects prompts and grammar definitions to follow a strict syntax. If the input contains unmatched braces, invalid tokens, or malformed grammar rules, the parser throws a GrammarParseError. This often happens when dynamically generating prompts or incorrectly formatting grammar files.
Detection
Catch GrammarParseError exceptions during prompt parsing and log the exact input causing the failure to identify syntax issues before model inference.
Causes & fixes
Unmatched or extra braces in the grammar or prompt string
Carefully check and balance all braces '{}' in your prompt or grammar definitions before passing them to llama.cpp.
Invalid tokens or characters not supported by llama.cpp grammar syntax
Remove or escape unsupported characters and ensure your grammar follows llama.cpp's documented syntax rules.
Incorrectly formatted grammar rules or missing required elements
Validate your grammar definitions against llama.cpp examples and documentation to ensure all required fields and syntax are correct.
Code: broken vs fixed
from llamacpp import Llama
model = Llama(model_path="./model.bin")
prompt = "{Hello world" # Missing closing brace causes parse error
output = model(prompt) # This line triggers GrammarParseError
print(output)import os
from llamacpp import Llama
os.environ["LLAMA_CPP_MODEL_PATH"] = "./model.bin" # Use env var for model path
model = Llama(model_path=os.environ["LLAMA_CPP_MODEL_PATH"])
prompt = "{Hello world}" # Fixed: balanced braces
output = model(prompt) # No GrammarParseError now
print(output)Workaround
Wrap the llama.cpp call in try/except GrammarParseError, then sanitize or simplify the prompt string by removing suspicious characters or unmatched braces before retrying.
Prevention
Implement prompt validation and sanitization routines that check grammar syntax before sending input to llama.cpp, and use unit tests to catch malformed prompts early.