RuntimeError
dspy.errors.PredictForwardPassError
Stack trace
Traceback (most recent call last):
File "app.py", line 42, in <module>
output = model.predict(input_tensor) # <-- triggers error
File "/usr/local/lib/python3.9/site-packages/dspy/model.py", line 128, in predict
raise dspy.errors.PredictForwardPassError("Forward pass failed due to invalid input shape")
dspy.errors.PredictForwardPassError: Forward pass failed due to invalid input shape Why it happens
This error happens when the input tensor shape does not match the model's expected input dimensions, or when model weights are not properly loaded before prediction. It can also occur if the device (CPU/GPU) is not correctly set, causing runtime failures during the forward pass.
Detection
Validate input tensor shapes against the model's expected input shape before calling predict, and ensure model weights are loaded and device is configured properly to catch errors early.
Causes & fixes
Input tensor shape does not match model's expected input dimensions
Check and reshape your input tensor to match the model's required input shape before calling predict.
Model weights are not loaded or corrupted before prediction
Ensure the model weights are correctly loaded using model.load_weights() or equivalent before running predict.
Device (CPU/GPU) is not properly configured or incompatible
Set the device explicitly using model.to(device) and verify compatibility of the input tensor device with the model.
Code: broken vs fixed
import os
from dspy import Model
model = Model()
input_tensor = ... # input with wrong shape
output = model.predict(input_tensor) # triggers PredictForwardPassErrorimport os
from dspy import Model
model = Model()
model.load_weights(os.environ['DSPY_MODEL_WEIGHTS']) # ensure weights loaded
input_tensor = input_tensor.reshape(model.input_shape) # fix input shape
output = model.predict(input_tensor) # fixed forward pass
print(output)Workaround
Wrap the predict call in try/except PredictForwardPassError, log the input tensor shape and model state, then attempt to reshape input or reload weights before retrying.
Prevention
Implement input validation and device configuration checks before prediction calls, and automate model weight loading during initialization to avoid forward pass failures.