asyncio.TimeoutError
asyncio.exceptions.TimeoutError
Stack trace
Traceback (most recent call last):
File "app.py", line 42, in <module>
response = await client.chat.completions.create(model="mistral-7b", messages=messages)
File "/usr/local/lib/python3.10/site-packages/mistral/client.py", line 128, in create
await asyncio.wait_for(self._send_request(payload), timeout=10)
File "/usr/lib/python3.10/asyncio/tasks.py", line 481, in wait_for
raise exceptions.TimeoutError()
asyncio.exceptions.TimeoutError Why it happens
The Mistral async client uses asyncio to manage network requests. If the server does not respond within the configured timeout window, asyncio raises a TimeoutError. This can happen due to network latency, server overload, or incorrect timeout settings.
Detection
Monitor for asyncio.TimeoutError exceptions in your async calls to Mistral client and log the duration of requests to detect slow or stalled connections before failure.
Causes & fixes
The default timeout for the async client is too low for the current network conditions or server response time.
Increase the timeout parameter in the client call, e.g., pass a higher timeout value to asyncio.wait_for or the client configuration.
Network connectivity issues or firewall blocking the connection to the Mistral API endpoint.
Verify network connectivity, firewall rules, and proxy settings to ensure the client can reach the Mistral server.
The Mistral server is overloaded or temporarily down, causing delayed or no responses.
Implement retry logic with exponential backoff and monitor Mistral service status to handle transient server issues gracefully.
Code: broken vs fixed
import asyncio
from mistral import Mistral
client = Mistral()
async def main():
# This line triggers asyncio.TimeoutError if server is slow
response = await client.chat.completions.create(model="mistral-7b", messages=[{"role": "user", "content": "Hello"}])
print(response)
asyncio.run(main())import os
import asyncio
from mistral import Mistral
client = Mistral()
async def main():
# Increased timeout to 30 seconds to prevent connection timeout
response = await asyncio.wait_for(
client.chat.completions.create(model="mistral-7b", messages=[{"role": "user", "content": "Hello"}]),
timeout=30
)
print(response)
if __name__ == "__main__":
asyncio.run(main())Workaround
Catch asyncio.TimeoutError exceptions around the client call and retry the request after a short delay or fallback to a cached response if available.
Prevention
Configure appropriate timeout values based on network conditions and implement robust retry and circuit breaker patterns to handle transient network or server delays.