RuntimeError
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeError
Stack trace
RuntimeError: [ONNXRuntimeError] : 1 : FAIL : Shape inference failed during model optimization: input tensor shape is missing or incompatible.
Why it happens
ONNX shape inference requires complete and consistent shape information for all tensors in the model graph. If the model has dynamic shapes without proper annotations or incompatible operators, the optimizer cannot infer shapes, causing this error during optimization.
Detection
Monitor logs for RuntimeError messages mentioning shape inference failure during ONNX model optimization, and validate model input/output shapes before optimization.
Causes & fixes
Model contains dynamic or missing shape information for inputs or intermediate tensors.
Explicitly specify input shapes using shape annotations or provide shape information before optimization.
Model uses unsupported or custom operators without proper shape inference functions.
Replace unsupported operators with standard ONNX ops or register custom shape inference functions.
ONNX model version or opset is incompatible with the optimization tools used.
Upgrade ONNX and ONNX Runtime to compatible versions and ensure the model opset matches supported versions.
Code: broken vs fixed
import onnx
from onnxruntime import InferenceSession
model = onnx.load('model.onnx')
# This will raise RuntimeError due to missing shape info
optimized_model = onnx.optimizer.optimize(model, passes=['eliminate_deadend']) # triggers shape inference errorimport os
import onnx
from onnxruntime import InferenceSession
# Set environment variable for ONNX Runtime logging
os.environ['ORT_LOG_LEVEL'] = 'VERBOSE'
model = onnx.load('model.onnx')
# Add shape info explicitly before optimization
for input_tensor in model.graph.input:
if not input_tensor.type.tensor_type.shape.dim:
# Example: set batch size 1 and fixed dimensions
input_tensor.type.tensor_type.shape.dim.extend([
onnx.helper.make_tensor_dimension_value(1),
onnx.helper.make_tensor_dimension_value(3),
onnx.helper.make_tensor_dimension_value(224),
onnx.helper.make_tensor_dimension_value(224)
])
optimized_model = onnx.optimizer.optimize(model, passes=['eliminate_deadend']) # fixed
print('Model optimized successfully')Workaround
Catch the RuntimeError during optimization, then manually add or fix shape information in the model graph using onnx.helper functions before retrying optimization.
Prevention
Always validate and annotate all input and intermediate tensor shapes in your ONNX model before optimization, and keep ONNX and ONNX Runtime versions compatible to prevent shape inference failures.