How to handle OpenAI timeout error
Quick answer
A
timeout error occurs when the OpenAI API request takes too long to respond. Handle it by wrapping your API call in a try-except block catching TimeoutError and implement retry logic with exponential backoff to automatically retry the request. ERROR TYPE
api_error ⚡ QUICK FIX
Add exponential backoff retry logic around your API call to handle
TimeoutError automatically.Why this happens
A timeout error occurs when the OpenAI API does not respond within the client’s configured timeout period. This can happen due to network latency, server load, or large requests that take longer to process.
Example of code triggering a timeout error:
from openai import OpenAI
import os
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
timeout=1 # very short timeout to simulate error
)
print(response.choices[0].message.content)This will raise a TimeoutError if the API call exceeds 1 second.
from openai import OpenAI
import os
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
timeout=1 # very short timeout to simulate error
)
print(response.choices[0].message.content) output
TimeoutError: The request timed out after 1 second
The fix
Wrap the API call in a try-except block catching TimeoutError and implement exponential backoff retries. This allows your app to recover from transient timeouts by retrying after increasing delays.
Example fixed code:
from openai import OpenAI
import os
import time
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
max_retries = 5
retry_delay = 1 # initial delay in seconds
for attempt in range(max_retries):
try:
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
timeout=10 # reasonable timeout
)
print(response.choices[0].message.content)
break # success, exit loop
except TimeoutError:
if attempt == max_retries - 1:
print("Request failed after retries due to timeout.")
raise
else:
print(f"Timeout occurred, retrying in {retry_delay} seconds...")
time.sleep(retry_delay)
retry_delay *= 2 # exponential backoff output
Hello # or Timeout occurred, retrying in 1 seconds... Hello
Preventing it in production
- Set a reasonable
timeoutparameter on API calls to avoid hanging indefinitely. - Implement retries with exponential backoff to handle transient network issues gracefully.
- Use circuit breakers or fallback logic to degrade functionality if the API is persistently unavailable.
- Monitor API latency and error rates to adjust retry policies dynamically.
Key Takeaways
- Always catch
TimeoutErrorwhen calling OpenAI API to handle slow responses. - Use exponential backoff retry logic to recover from transient timeout errors automatically.
- Set sensible timeout values to balance responsiveness and robustness in your app.