OSError
builtins.OSError: [Errno 98] Address already in use
Stack trace
Traceback (most recent call last):
File "server.py", line 42, in <module>
llm.serve(port=1234)
File "/usr/local/lib/python3.9/site-packages/vllm/llm.py", line 210, in serve
sock.bind((host, port))
OSError: [Errno 98] Address already in use Why it happens
This error occurs because the TCP port specified for the vLLM server is already bound by another process or a previous instance of the server that did not shut down cleanly. The operating system prevents multiple processes from listening on the same port simultaneously.
Detection
Before starting the vLLM server, check if the port is free using system tools like 'lsof -i :<port>' or 'netstat -tuln'. Catch OSError exceptions on bind calls to log and handle port conflicts gracefully.
Causes & fixes
Another process is already using the configured port.
Identify and stop the conflicting process using system commands like 'lsof -i :<port>' or 'kill <pid>', then restart the vLLM server.
A previous vLLM server instance did not shut down properly, leaving the port in a TIME_WAIT or CLOSE_WAIT state.
Wait a few minutes for the OS to release the port or restart the machine to clear stale socket states.
The vLLM server is configured to use a hardcoded port that conflicts with other services.
Change the vLLM server port to an unused port by passing a different port number to the serve() method.
Code: broken vs fixed
from vllm import LLM
llm = LLM(model="llama-3b")
llm.serve(port=1234) # This line raises OSError: Address already in useimport os
from vllm import LLM
# Use environment variable for port to avoid conflicts
port = int(os.environ.get("VLLM_PORT", "1235"))
llm = LLM(model="llama-3b")
llm.serve(port=port) # Changed port to avoid conflict
print(f"vLLM server started on port {port}")Workaround
Wrap the serve call in try/except OSError, and if the port is in use, automatically increment the port number and retry binding a few times before failing.
Prevention
Use dynamic port assignment or configuration management to ensure the vLLM server uses a free port. Monitor server processes and cleanly shut down to release ports properly.