OpenAIError
openai.OpenAIError (token limit exceeded)
Stack trace
openai.OpenAIError: The request was rejected because it exceeded the maximum allowed tokens for the model. Details: 'This model's maximum context length is 8192 tokens, but your request contains 9000 tokens.'
Why it happens
OpenAI models have a fixed maximum token context length that includes both the prompt and the generated completion. When the total tokens exceed this limit, the API rejects the request with a token limit exceeded error. This often happens with long prompts, large conversation histories, or when requesting very long completions.
Detection
Monitor token usage by calculating prompt tokens plus expected completion tokens before sending requests. Use SDK utilities or tokenizers to estimate token counts and log warnings when approaching limits.
Causes & fixes
Prompt or conversation history is too long, exceeding the model's max token context window.
Truncate or summarize conversation history, or split input into smaller chunks to stay within the token limit.
Requesting a completion max_tokens value that combined with prompt tokens exceeds the model limit.
Reduce the max_tokens parameter in the completion request to ensure total tokens fit within the model's context length.
Using a model with a smaller token limit than required for your use case.
Switch to a model with a larger context window, such as gpt-4o or gpt-4o-mini, which support up to 8192 tokens or more.
Code: broken vs fixed
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])
response = client.chat.completions.create(
model='gpt-4o',
messages=[{'role': 'user', 'content': 'A very long prompt that exceeds token limit...'}],
max_tokens=5000 # This combined with prompt tokens exceeds limit
) # This line triggers the token limit exceeded error
print(response.choices[0].message.content)import os
from openai import OpenAI
client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])
# Reduced max_tokens and truncated prompt to fit token limit
truncated_prompt = 'A shorter prompt that fits within token limits'
response = client.chat.completions.create(
model='gpt-4o',
messages=[{'role': 'user', 'content': truncated_prompt}],
max_tokens=1000 # Reduced to fit within model token limit
)
print(response.choices[0].message.content) # Fixed: no token limit errorWorkaround
Catch the OpenAIError exception, then split the input prompt into smaller segments and send multiple requests sequentially to avoid exceeding token limits.
Prevention
Implement token counting before requests using tokenizer libraries and design prompts or conversation histories to stay well below the model's max token context window. Prefer models with larger context windows for long inputs.