FileNotFoundError
autogptq.exceptions.ModelNotFoundError
Stack trace
autogptq.exceptions.ModelNotFoundError: Could not find quantized model files at path '/models/quantized-model' FileNotFoundError: [Errno 2] No such file or directory: '/models/quantized-model/config.json'
Why it happens
AutoGPTQ requires the quantized model files to be present at the exact path specified during model loading. If the path is incorrect, missing, or the files were not properly saved after quantization, the library raises this error indicating it cannot find the necessary model files.
Detection
Check for FileNotFoundError or ModelNotFoundError exceptions when calling AutoGPTQ's model loading functions, and verify the model path exists and contains the expected quantized files before runtime.
Causes & fixes
Incorrect or misspelled model path provided to AutoGPTQ load function
Verify and correct the model path string to point exactly to the directory containing the quantized model files.
Quantized model files were not saved or exported properly after quantization
Ensure the quantization process completed successfully and the output directory contains all required files like config.json and model.bin.
Model files were deleted, moved, or corrupted after quantization
Restore or re-quantize the model to regenerate the missing files at the expected location.
Code: broken vs fixed
from autogptq import AutoGPTQForCausalLM
model = AutoGPTQForCausalLM.from_quantized('/wrong/path/to/model') # triggers FileNotFoundErrorimport os
from autogptq import AutoGPTQForCausalLM
model_path = os.environ.get('QUANT_MODEL_PATH') # Use env var for path
model = AutoGPTQForCausalLM.from_quantized(model_path) # fixed path usage
print('Model loaded successfully')Workaround
Wrap the model loading call in try/except to catch FileNotFoundError, then log the missing path and fallback to a default or unquantized model to keep the app running.
Prevention
Use environment variables or configuration management to centrally control quantized model paths and validate file existence during deployment to avoid missing files at runtime.