ConnectionError
requests.exceptions.ConnectionError
Stack trace
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/chat/completions (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8c1a2d1d60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) Why it happens
This error occurs when the Airflow LLM operator cannot establish a network connection to the LLM API service. Common causes include incorrect API endpoint URLs, missing or invalid proxy settings, DNS resolution failures, or network outages.
Detection
Monitor Airflow task logs for ConnectionError exceptions and implement retries with exponential backoff to detect transient network failures before task failure.
Causes & fixes
Incorrect or missing API base URL in the LLM operator configuration
Verify and set the correct API base URL in the Airflow LLM operator parameters or environment variables.
Network firewall or proxy blocking outbound requests to the LLM API endpoint
Configure firewall rules or proxy settings to allow outbound HTTPS traffic to the LLM API host.
DNS resolution failure due to misconfigured DNS servers or network issues
Ensure the Airflow worker nodes have proper DNS configuration and can resolve the LLM API hostname.
Missing or invalid environment variables for API keys or network configuration
Set required environment variables such as OPENAI_API_KEY and any proxy environment variables correctly in the Airflow environment.
Code: broken vs fixed
from airflow.providers.http.operators.http import SimpleHttpOperator
llm_task = SimpleHttpOperator(
task_id='call_llm',
http_conn_id='openai_default',
endpoint='/v1/chat/completions',
method='POST',
data='{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}',
headers={'Content-Type': 'application/json'},
dag=dag
) # This triggers ConnectionError if connection is misconfiguredimport os
from airflow.providers.http.operators.http import SimpleHttpOperator
llm_task = SimpleHttpOperator(
task_id='call_llm',
http_conn_id='openai_default', # Ensure this connection is configured with correct host
endpoint='/v1/chat/completions',
method='POST',
data='{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}',
headers={'Content-Type': 'application/json', 'Authorization': f'Bearer {os.environ["OPENAI_API_KEY"]}'},
dag=dag
) # Fixed: Confirmed connection and API key setupWorkaround
Wrap the LLM operator call in a try/except block catching ConnectionError, then retry the task after a delay or fallback to a cached response if available.
Prevention
Configure Airflow HTTP connections with correct API hosts and credentials, validate network connectivity during deployment, and use Airflow retries with exponential backoff for transient network errors.