AnthropicAPIError
anthropic.AnthropicAPIError
Stack trace
anthropic.AnthropicAPIError: Request rejected: prompt too long, exceeds token limit of 9000 tokens
Why it happens
Anthropic models have strict token limits for the combined prompt and completion. If the system prompt plus user input exceed this limit, the API returns an error rejecting the request. This often happens with very long system prompts or accumulated conversation history.
Detection
Monitor API error responses for AnthropicAPIError with messages indicating prompt length exceeded token limits before the request fails.
Causes & fixes
System prompt text is too long and exceeds the model's maximum token limit when combined with user input.
Shorten the system prompt by removing unnecessary instructions or splitting the prompt into smaller parts.
Accumulated conversation history or context is too large, causing the total tokens to exceed the limit.
Implement context window management by truncating or summarizing earlier messages to keep total tokens under the limit.
Using a model with a smaller token limit than required for your prompt and completion length.
Switch to an Anthropic model variant with a larger token limit, such as claude-3-5-haiku-20241022 or claude-3-5-sonnet-20241022.
Code: broken vs fixed
import os
from anthropic import Anthropic
client = Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])
system_prompt = """Very long system prompt text that exceeds token limits..."""
user_input = "User question here"
response = client.completions.create(
model="claude-3-5-sonnet-20241022",
prompt=system_prompt + "\n" + user_input,
max_tokens_to_sample=100
) # This line triggers the prompt too long errorimport os
from anthropic import Anthropic
client = Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])
# Shortened system prompt to fit token limits
system_prompt = """Concise system prompt with essential instructions only."""
user_input = "User question here"
response = client.messages.create(
model="claude-3-5-sonnet-20241022",
system=system_prompt,
messages=[{"role": "user", "content": user_input}],
max_tokens=100
) # Fixed: shortened prompt to avoid token limit error
print(response.text)Workaround
Catch AnthropicAPIError exceptions, then truncate or summarize the system prompt and retry the request automatically.
Prevention
Implement prompt length checks and context window management in your application to ensure combined prompt and input tokens never exceed the model's token limit.