How to Intermediate · 4 min read

How to fine-tune with LoRA using PEFT

Q: How to fine-tune with LoRA using PEFT

Use the peft library to apply Low-Rank Adaptation (LoRA) for efficient fine-tuning of large language models. Load a pretrained model with transformers, configure LoraConfig, wrap the model with get_peft_model, then fine-tune using standard training loops or transformers.Trainer.

Quick answer

Use the peft library to apply Low-Rank Adaptation (LoRA) for efficient fine-tuning of large language models. Load a pretrained model with transformers, configure LoraConfig, wrap the model with get_peft_model, then fine-tune using standard training loops or transformers.Trainer.

PREREQUISITES

Python 3.8+
pip install transformers>=4.30.0
pip install peft>=0.4.0
pip install torch>=2.0.0

Setup

Install the required libraries transformers, peft, and torch. Ensure you have a compatible Python environment (3.8+).

bash

pip install transformers peft torch

Step by step

This example fine-tunes a causal language model with LoRA using PEFT. It loads a pretrained model, applies LoRA, and runs a simple training loop.

python

import os
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType

# Load pretrained model and tokenizer
model_name = "meta-llama/Llama-3.1-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)

# Configure LoRA
lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    task_type=TaskType.CAUSAL_LM
)

# Apply LoRA to the model
model = get_peft_model(model, lora_config)

# Prepare dummy dataset
texts = ["Hello, how are you?", "Fine-tuning with LoRA is efficient."]
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")

class DummyDataset(torch.utils.data.Dataset):
    def __init__(self, encodings):
        self.encodings = encodings
    def __len__(self):
        return len(self.encodings.input_ids)
    def __getitem__(self, idx):
        return {key: val[idx] for key, val in self.encodings.items()}

dataset = DummyDataset(inputs)

# Training arguments
training_args = TrainingArguments(
    output_dir="./lora-finetuned",
    per_device_train_batch_size=1,
    num_train_epochs=1,
    logging_steps=1,
    save_strategy="no",
    fp16=True
)

# Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset
)

# Train
trainer.train()

# Save LoRA adapters
model.save_pretrained("./lora-finetuned")

print("LoRA fine-tuning complete.")

output

***** Running training *****
  Num examples = 2
  Num Epochs = 1
  Instantaneous batch size per device = 1
  Total train batch size (w. parallel, distributed & accumulation) = 1
  Gradient Accumulation steps = 1
  Total optimization steps = 2

[...training logs...]
LoRA fine-tuning complete.

Common variations

Use BitsAndBytesConfig with load_in_4bit=True for 4-bit quantized models combined with LoRA (QLoRA).
Apply LoRA to different model architectures by adjusting target_modules.
Use async training loops or Hugging Face Accelerate for distributed training.

Troubleshooting

If you get CUDA out-of-memory errors, reduce batch size or use 4-bit quantization with QLoRA.
Ensure target_modules matches your model's architecture; inspect model layers if unsure.
Verify peft and transformers versions are compatible.

✅

Key Takeaways

Use peft to efficiently fine-tune large models with LoRA by injecting trainable low-rank adapters.
Configure LoraConfig with appropriate rank, alpha, and target modules for your model architecture.
Combine LoRA with quantization (QLoRA) to reduce memory usage and enable fine-tuning on smaller GPUs.
Use Hugging Face Trainer or custom training loops to fine-tune the LoRA-wrapped model.
Always save and load only the LoRA adapters for efficient storage and deployment.

Verified 2026-04 · meta-llama/Llama-3.1-8B-Instruct

Verify ↗