Cost-benefit analysis framework
Why this matters
Fine-tuning costs real money: compute, storage, data prep: and you need to know if the accuracy or latency gains justify the expense before committing to a training run.
Explanation
A cost-benefit framework compares the total cost of fine-tuning against the financial value gained from improved model performance. Mechanically: You estimate (1) fine-tuning cost: compute hours × hourly rate + storage, (2) inference cost reduction: baseline API calls × current cost per 1K tokens minus fine-tuned calls × new cost per 1K tokens, and (3) accuracy improvement value: reduction in human review/rework time. When to use it: Before you queue up a training job, especially if you're managing a budget or serving high-volume inference: this calculation prevents costly experiments that don't move the business needle.
Analogy
Like A/B testing a marketing campaign: you spend money upfront to test, then measure if the lift justifies the ad spend. Fine-tuning is the same: upfront training cost, measured against downstream savings.
Code
import json
from datetime import datetime
def cost_benefit_finetuning(
model_size_b: float,
training_hours: float,
gpu_hourly_rate: float,
storage_gb: float,
storage_monthly_rate: float,
baseline_monthly_api_calls: int,
baseline_cost_per_1k_tokens: float,
finetuned_cost_per_1k_tokens: float,
accuracy_improvement_percent: float,
monthly_review_cost: float,
months_to_amortize: int
) -> dict:
"""
Calculate ROI for fine-tuning vs. staying on base model.
Args:
model_size_b: Model size in billions of parameters (e.g., 7.0 for 7B)
training_hours: Estimated GPU hours needed
gpu_hourly_rate: Cost per GPU hour (e.g., $1.50 for A100)
storage_gb: Total storage for model + checkpoints
storage_monthly_rate: Monthly storage cost per GB
baseline_monthly_api_calls: Current monthly inference calls
baseline_cost_per_1k_tokens: Current API cost per 1K tokens
finetuned_cost_per_1k_tokens: Self-hosted inference cost per 1K tokens
accuracy_improvement_percent: Expected accuracy gain (0.0-1.0)
monthly_review_cost: Current cost of human review on wrong predictions
months_to_amortize: Planning horizon (e.g., 12 months)
Returns:
Dictionary with cost breakdown and ROI metrics
"""
training_cost = training_hours * gpu_hourly_rate
storage_cost = storage_gb * storage_monthly_rate * months_to_amortize
total_upfront_cost = training_cost + storage_cost
monthly_api_cost_baseline = (baseline_monthly_api_calls / 1000) * baseline_cost_per_1k_tokens
monthly_api_cost_finetuned = (baseline_monthly_api_calls / 1000) * finetuned_cost_per_1k_tokens
monthly_inference_savings = monthly_api_cost_baseline - monthly_api_cost_finetuned
total_inference_savings = monthly_inference_savings * months_to_amortize
monthly_review_savings = monthly_review_cost * accuracy_improvement_percent
total_review_savings = monthly_review_savings * months_to_amortize
total_benefit = total_inference_savings + total_review_savings
net_roi = total_benefit - total_upfront_cost
roi_percent = (net_roi / total_upfront_cost * 100) if total_upfront_cost > 0 else 0
payback_months = (
total_upfront_cost / (monthly_inference_savings + monthly_review_savings)
if (monthly_inference_savings + monthly_review_savings) > 0
else float('inf')
)
return {
"timestamp": datetime.now().isoformat(),
"model_params_b": model_size_b,
"upfront_costs": {
"training_gpu_hours": training_hours,
"training_cost_usd": round(training_cost, 2),
"storage_cost_usd": round(storage_cost, 2),
"total_upfront_usd": round(total_upfront_cost, 2)
},
"monthly_benefits": {
"inference_savings_usd": round(monthly_inference_savings, 2),
"review_cost_savings_usd": round(monthly_review_savings, 2),
"total_monthly_benefit_usd": round(monthly_inference_savings + monthly_review_savings, 2)
},
"amortized_benefits": {
"months": months_to_amortize,
"inference_savings_usd": round(total_inference_savings, 2),
"review_savings_usd": round(total_review_savings, 2),
"total_benefit_usd": round(total_benefit, 2)
},
"roi_metrics": {
"net_roi_usd": round(net_roi, 2),
"roi_percent": round(roi_percent, 1),
"payback_months": round(payback_months, 1) if payback_months != float('inf') else "Never",
"worthwhile": net_roi > 0
}
}
scenario = cost_benefit_finetuning(
model_size_b=7.0,
training_hours=24,
gpu_hourly_rate=1.50,
storage_gb=28,
storage_monthly_rate=0.02,
baseline_monthly_api_calls=5_000_000,
baseline_cost_per_1k_tokens=0.30,
finetuned_cost_per_1k_tokens=0.05,
accuracy_improvement_percent=0.12,
monthly_review_cost=2_000,
months_to_amortize=12
)
print(json.dumps(scenario, indent=2)){
"timestamp": "2026-04-15T10:23:45.123456",
"model_params_b": 7.0,
"upfront_costs": {
"training_gpu_hours": 24,
"training_cost_usd": 36.0,
"storage_cost_usd": 16.8,
"total_upfront_usd": 52.8
},
"monthly_benefits": {
"inference_savings_usd": 1250.0,
"review_cost_savings_usd": 240.0,
"total_monthly_benefit_usd": 1490.0
},
"amortized_benefits": {
"months": 12,
"inference_savings_usd": 15000.0,
"review_savings_usd": 2880.0,
"total_benefit_usd": 17880.0
},
"roi_metrics": {
"net_roi_usd": 17827.2,
"roi_percent": 33789.8,
"payback_months": 0.0,
"worthwhile": true
}
} What just happened?
The code built a complete cost model with five sections: upfront training and storage expenses, monthly cost reductions from using a self-hosted fine-tuned model instead of API calls, review cost savings from better accuracy, total 12-month amortized benefit, and ROI metrics including payback period. In this scenario, fine-tuning costs $52.80 upfront but saves $17,827.20 over 12 months: an obvious win. The payback period is 0.0 months because monthly savings ($1,490) exceed upfront costs immediately.
Common gotcha
Developers often forget to account for storage costs during training: checkpoints, intermediate states, and the final model add up fast. A 7B model fine-tuned with LoRA on 4 GPUs for 24 hours can easily consume 100+ GB of disk, and cloud storage compounds the cost. Always include the storage line item or you'll be shocked by the final bill.
Error recovery
ZeroDivisionError on payback_monthsNegative net_roiroi_percent showing absurdly high valuesExperienced dev note
Most teams underestimate inference cost and overestimate training time. Here's the reality: fine-tuning a 7B model on a single A100 is 12–36 hours depending on dataset size, not 2 hours. And if you're serving 10M monthly API calls, you're already spending $3K–$6K per month on inference: fine-tuning becomes a no-brainer at that scale. The payback happens in weeks, not months. The real decision is not 'should we fine-tune?' but 'should we self-host or stay on API?' This framework makes that transparent.
Check your understanding
In the example above, the payback period shows 0.0 months. Why doesn't this mean you recoup costs instantly on day one? What does a 0.0 payback period actually tell you about the timing of costs vs. benefits?
Show answer hint
A correct answer recognizes that upfront costs (training + storage) occur at time zero, while monthly benefits accrue starting month 1. A payback period of 0.0 means the first month's benefit alone exceeds the total upfront cost: you break even before the end of month 1. This is different from 'instant payback' (which would mean no upfront cost). The insight is understanding the timing mismatch between when you spend money (now) and when you save it (monthly, over time).