RuntimeError
torch.cuda.OutOfMemoryError
Stack trace
RuntimeError: CUDA out of memory. Tried to allocate 512.00 MiB (GPU 0; 8.00 GiB total capacity; 6.50 GiB already allocated; 256.00 MiB free; 6.75 GiB reserved in total by PyTorch)
Why it happens
This error happens because the model, batch size, or sequence length requires more GPU memory than available. HuggingFace's SFTTrainer attempts to allocate memory for gradients and activations during fine-tuning, which can exceed the GPU capacity if not properly configured.
Detection
Monitor GPU memory usage with tools like nvidia-smi during training; catching RuntimeError with 'cuda out of memory' message indicates this issue before a crash.
Causes & fixes
Batch size is too large for the available GPU memory
Reduce the batch size in your training arguments to fit within GPU memory limits.
Sequence length or model size is too large for the GPU
Lower the max sequence length or switch to a smaller model architecture to reduce memory usage.
No gradient checkpointing enabled, causing high memory consumption
Enable gradient checkpointing in the Trainer to trade compute for memory and reduce peak usage.
Multiple processes or other applications occupying GPU memory
Ensure no other GPU processes are running and clear GPU memory before starting training.
Code: broken vs fixed
from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
output_dir='./results',
per_device_train_batch_size=16, # Too large, causes OOM
num_train_epochs=3
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset
)
trainer.train() # This line triggers CUDA OOM errorimport os
from transformers import Trainer, TrainingArguments
os.environ['CUDA_VISIBLE_DEVICES'] = '0' # Use single GPU
training_args = TrainingArguments(
output_dir='./results',
per_device_train_batch_size=4, # Reduced batch size to fix OOM
num_train_epochs=3,
gradient_checkpointing=True # Enable gradient checkpointing to save memory
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset
)
trainer.train() # Fixed: runs without CUDA OOM error
print('Training completed successfully')Workaround
Catch RuntimeError exceptions during training, then reduce batch size or sequence length dynamically and retry training to avoid OOM crashes.
Prevention
Plan GPU memory usage by profiling model size and batch parameters; use gradient checkpointing and mixed precision training to minimize memory footprint before fine-tuning.