OpenAI fine-tuning cost calculation
Quick answer
Calculate
OpenAI fine-tuning costs by summing the training file upload size and the number of tokens processed during fine-tuning, multiplied by the model's specific pricing rates. Use the client.files.create to upload data and client.fine_tuning.jobs.create to start fine-tuning, then estimate costs based on documented token prices.PREREQUISITES
Python 3.8+OpenAI API key (free tier works)pip install openai>=1.0
Setup
Install the openai Python package and set your API key as an environment variable for secure authentication.
pip install openai output
Collecting openai Downloading openai-1.x.x-py3-none-any.whl (xx kB) Installing collected packages: openai Successfully installed openai-1.x.x
Step by step
This example uploads a training file, creates a fine-tuning job, and prints the job ID. You can then estimate costs based on file size and tokens used.
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
# Upload training file (JSONL format)
with open("training_data.jsonl", "rb") as f:
training_file = client.files.create(file=f, purpose="fine-tune")
print(f"Uploaded file ID: {training_file.id}")
# Create fine-tuning job
job = client.fine_tuning.jobs.create(
training_file=training_file.id,
model="gpt-4o-mini-2024-07-18"
)
print(f"Fine-tuning job ID: {job.id}") output
Uploaded file ID: file-abc123xyz Fine-tuning job ID: ftjob-xyz789abc
Common variations
You can estimate costs by checking the file size in bytes and multiplying by the upload price per MB, then add the cost of tokens processed during training based on the model's pricing. Use asynchronous calls or different models by adjusting the model parameter.
import os
import math
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
# Example: Calculate upload cost
file_path = "training_data.jsonl"
file_size_bytes = os.path.getsize(file_path)
file_size_mb = file_size_bytes / (1024 * 1024)
upload_price_per_mb = 0.03 # Example price in USD, check official pricing
upload_cost = file_size_mb * upload_price_per_mb
print(f"Training file size: {file_size_mb:.2f} MB")
print(f"Estimated upload cost: ${upload_cost:.4f}")
# Token cost example (estimate tokens processed during fine-tuning)
tokens_processed = 1000000 # Example token count
price_per_1k_tokens = 0.12 # Example price in USD, check official pricing
training_cost = (tokens_processed / 1000) * price_per_1k_tokens
print(f"Estimated training token cost: ${training_cost:.2f}")
# Total estimated cost
print(f"Total estimated fine-tuning cost: ${upload_cost + training_cost:.2f}") output
Training file size: 5.00 MB Estimated upload cost: $0.1500 Estimated training token cost: $120.00 Total estimated fine-tuning cost: $120.15
Troubleshooting
- If you get a
403 Forbiddenerror, verify your API key and permissions. - If the file upload fails, ensure your file is in valid JSONL format and under size limits.
- Check the official OpenAI pricing page regularly as rates may change.
Key Takeaways
- Use
client.files.createto upload training data and note the file size for cost estimation. - Fine-tuning costs include upload fees plus token processing fees based on the model's pricing.
- Always check the latest OpenAI pricing to ensure accurate cost calculations.