BootstrapFewShotMetricError
dspy.metrics.BootstrapFewShotMetricError
Stack trace
Traceback (most recent call last):
File "app.py", line 42, in <module>
score = metric.evaluate(predictions, references)
File "/usr/local/lib/python3.9/site-packages/dspy/metrics/bootstrap_fewshot.py", line 88, in evaluate
raise BootstrapFewShotMetricError("Invalid input data format for BootstrapFewShot metric.")
dspy.metrics.BootstrapFewShotMetricError: Invalid input data format for BootstrapFewShot metric. Why it happens
The BootstrapFewShot metric in DSPy expects input data in a specific format with properly structured predictions and references. If the input data is missing required fields, or the metric configuration lacks necessary parameters, this error is raised. It often happens when the few-shot examples or prompt templates are misconfigured or incomplete.
Detection
Validate input data format and metric configuration before calling evaluate; add logging to capture input shapes and keys to detect mismatches early.
Causes & fixes
Input predictions or references lack required keys or are not in the expected list/dict format.
Ensure predictions and references are lists of dicts with all required keys as specified in the DSPy BootstrapFewShot metric documentation.
Metric configuration missing required parameters such as 'few_shot_examples' or 'sampling_strategy'.
Provide all mandatory configuration parameters when initializing the BootstrapFewShot metric, including valid few-shot examples and sampling strategy.
Using incompatible or outdated DSPy version that does not support current metric API.
Upgrade DSPy to the latest stable version that supports BootstrapFewShot metric with your codebase.
Code: broken vs fixed
from dspy.metrics import BootstrapFewShotMetric
metric = BootstrapFewShotMetric()
predictions = ['answer1', 'answer2']
references = ['answer1', 'answer2']
# This line raises BootstrapFewShotMetricError due to wrong input format
score = metric.evaluate(predictions, references)import os
from dspy.metrics import BootstrapFewShotMetric
metric = BootstrapFewShotMetric(few_shot_examples=[{'input': 'Q1', 'output': 'A1'}], sampling_strategy='random')
predictions = [{'output': 'answer1'}, {'output': 'answer2'}]
references = [{'output': 'answer1'}, {'output': 'answer2'}]
# Fixed: input data formatted as list of dicts with required keys
score = metric.evaluate(predictions, references)
print(f"BootstrapFewShot metric score: {score}")Workaround
Wrap the evaluate call in try/except BootstrapFewShotMetricError, then manually validate and transform input data format before retrying evaluation.
Prevention
Use DSPy's official input data validation utilities and always initialize BootstrapFewShot metric with complete configuration to avoid format errors.