RuntimeError
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeError
Stack trace
RuntimeError: [ONNXRuntimeError] : 1 : FAIL : CUDA execution provider is not available, please check your environment and installation.
Why it happens
This error occurs because ONNXRuntime cannot find a compatible CUDA GPU environment. Either the CUDA toolkit or drivers are missing, the ONNXRuntime GPU package is not installed, or the CUDA provider is not enabled correctly in the session options.
Detection
Check for RuntimeError messages mentioning 'CUDA execution provider is not available' during ONNXRuntime session creation or model inference initialization.
Causes & fixes
ONNXRuntime GPU package (onnxruntime-gpu) is not installed, only CPU version is present
Install the onnxruntime-gpu package via pip to enable CUDA support: pip install onnxruntime-gpu
CUDA drivers or toolkit are missing or incompatible with the installed ONNXRuntime GPU version
Install or update the NVIDIA CUDA drivers and toolkit to versions compatible with your onnxruntime-gpu version, following official NVIDIA and ONNXRuntime docs
CUDA execution provider is not enabled in the ONNXRuntime session options
Explicitly add the CUDAExecutionProvider when creating the InferenceSession, e.g., providers=['CUDAExecutionProvider']
Running on a machine without a CUDA-capable GPU or with GPU disabled in BIOS/VM environment
Verify the presence of a CUDA-capable GPU and ensure it is enabled and accessible to the runtime environment
Code: broken vs fixed
import onnxruntime as ort
# This will raise RuntimeError if CUDA provider is unavailable
session = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider']) # triggers errorimport os
import onnxruntime as ort
# Ensure environment is set for CUDA and GPU package is installed
os.environ['CUDA_VISIBLE_DEVICES'] = '0' # optional: specify GPU device
session = ort.InferenceSession('model.onnx', providers=['CUDAExecutionProvider']) # fixed by installing onnxruntime-gpu and enabling provider
print('ONNXRuntime session created with CUDA provider')Workaround
Fallback to CPU execution by removing 'CUDAExecutionProvider' from providers list or using providers=['CPUExecutionProvider'] to avoid the error temporarily.
Prevention
Always install onnxruntime-gpu on CUDA-capable systems, keep CUDA drivers/toolkit updated, and explicitly specify CUDAExecutionProvider in session options to guarantee GPU usage.