LiteLLMFallbackError
litellm.errors.LiteLLMFallbackError
Stack trace
Traceback (most recent call last):
File "app.py", line 42, in <module>
response = litellm_client.generate(prompt)
File "/usr/local/lib/python3.9/site-packages/litellm/client.py", line 128, in generate
raise LiteLLMFallbackError("Fallback model failed all providers")
litellm.errors.LiteLLMFallbackError: Fallback model failed all providers Why it happens
LiteLLM attempts to use a fallback model when the primary model or provider fails. This error occurs when all fallback providers are either unreachable, misconfigured, or return errors, leaving no available model to serve the request.
Detection
Monitor LiteLLM client calls for LiteLLMFallbackError exceptions and log provider statuses to detect when fallback models fail before the application crashes.
Causes & fixes
All fallback providers are misconfigured or missing API keys.
Ensure all fallback providers have valid API keys set in environment variables and are correctly configured in LiteLLM.
Network connectivity issues prevent contacting fallback providers.
Check network access and firewall rules to allow outbound connections to all fallback provider endpoints.
Fallback providers are rate limited or temporarily down.
Implement retry logic with exponential backoff and monitor provider status pages to handle temporary outages.
Fallback model names or versions are incorrect or deprecated.
Verify fallback model identifiers against provider documentation and update to supported versions.
Code: broken vs fixed
import os
from litellm import LiteLLMClient
client = LiteLLMClient()
response = client.generate("Hello world") # Raises LiteLLMFallbackError here
print(response)import os
from litellm import LiteLLMClient
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "your_openai_api_key")
os.environ["ANOTHER_PROVIDER_API_KEY"] = os.environ.get("ANOTHER_PROVIDER_API_KEY", "your_other_provider_key")
client = LiteLLMClient()
response = client.generate("Hello world") # Fixed: fallback providers configured
print(response)Workaround
Catch LiteLLMFallbackError in your code and provide a static canned response or degrade gracefully while investigating provider issues.
Prevention
Configure multiple fallback providers with valid credentials and implement health checks and retries to ensure fallback availability under all conditions.