RuntimeError
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeError
Stack trace
Traceback (most recent call last):
File "quantize_model.py", line 45, in <module>
quantize_static(model_input_path, model_output_path, calibration_data_reader)
File "/usr/local/lib/python3.9/site-packages/onnxruntime/quantization/quantize.py", line 120, in quantize_static
calibrate(model_fp32, calibration_data_reader, quant_format, activation_type, weight_type, optimize_model)
File "/usr/local/lib/python3.9/site-packages/onnxruntime/quantization/calibrate.py", line 75, in calibrate
raise RuntimeError("Calibration failed due to invalid calibration data or model input mismatch.")
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeError: Calibration failed due to invalid calibration data or model input mismatch. Why it happens
This error happens because the calibration data provided to the ONNX quantization process does not match the model's expected input format or contains invalid values. Calibration requires representative input data to compute quantization parameters accurately. If the data reader yields incorrect shapes, types, or corrupted data, calibration fails.
Detection
Monitor the calibration data reader outputs and validate input shapes and types before passing them to the quantization API. Log calibration steps and catch RuntimeError exceptions to detect calibration failures early.
Causes & fixes
Calibration data reader yields inputs with incorrect shapes or data types not matching the model input.
Ensure the calibration data reader returns inputs with the exact shape and data type expected by the ONNX model's input nodes.
Calibration dataset contains corrupted or invalid data samples causing runtime failures during calibration.
Validate and clean the calibration dataset to remove corrupted or invalid samples before calibration.
Mismatch between the model input preprocessing and the calibration data format.
Apply the same preprocessing steps to calibration data as used during model training and inference to maintain consistency.
Using an unsupported or incompatible ONNXRuntime quantization API version.
Upgrade ONNXRuntime to a stable version that supports your quantization workflow, preferably >=1.12.0.
Code: broken vs fixed
from onnxruntime.quantization import quantize_static
calibration_data_reader = MyCalibrationDataReader()
quantize_static("model.onnx", "model_quant.onnx", calibration_data_reader) # Raises RuntimeError hereimport os
from onnxruntime.quantization import quantize_static
class ValidCalibrationDataReader:
def __init__(self, data):
self.data = data
self.iterator = iter(self.data)
def get_next(self):
try:
input_data = next(self.iterator)
# Ensure input_data matches model input shape and dtype
return {"input": input_data.astype('float32')}
except StopIteration:
return None
calibration_data = load_and_preprocess_calibration_data() # Preprocess to correct shape and dtype
calibration_data_reader = ValidCalibrationDataReader(calibration_data)
quantize_static(os.environ["MODEL_FP32_PATH"], os.environ["MODEL_QUANT_PATH"], calibration_data_reader) # Fixed: valid calibration data
print("Quantization completed successfully.")Workaround
Wrap the quantize_static call in try/except RuntimeError, and on failure, log the calibration data samples to identify and fix invalid inputs manually.
Prevention
Always preprocess and validate calibration datasets to match the model input specifications exactly, and use stable ONNXRuntime versions with tested quantization support.