How to Intermediate · 4 min read

How to use Unsloth for LoRA fine-tuning

Q: How to use Unsloth for LoRA fine-tuning

Use Unsloth by installing it via pip, then configure your base model and LoRA parameters in Python. Run the fine-tuning process with UnslothTrainer to efficiently apply LoRA adapters on your model for parameter-efficient training.

Quick answer

Use Unsloth by installing it via pip, then configure your base model and LoRA parameters in Python. Run the fine-tuning process with UnslothTrainer to efficiently apply LoRA adapters on your model for parameter-efficient training.

PREREQUISITES

Python 3.8+
pip install unsloth
PyTorch 1.13+
Access to a pretrained base model (e.g., Hugging Face transformers)
Basic knowledge of LoRA and fine-tuning concepts

Setup

Install Unsloth and required dependencies using pip. Ensure you have a compatible PyTorch version and access to a pretrained model from Hugging Face or similar.

bash

pip install unsloth torch transformers

Step by step

This example shows how to fine-tune a Hugging Face transformers model using Unsloth with LoRA adapters. It loads a base model, applies LoRA configuration, and runs training.

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from unsloth import UnslothTrainer, LoraConfig

# Load base model and tokenizer
model_name = "meta-llama/Llama-3.1-8B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")

# Configure LoRA parameters
lora_config = LoraConfig(
    r=16,                # LoRA rank
    lora_alpha=32,       # LoRA scaling
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    task_type="CAUSAL_LM"
)

# Initialize Unsloth trainer with LoRA
trainer = UnslothTrainer(
    model=model,
    tokenizer=tokenizer,
    lora_config=lora_config,
    train_dataset=None,  # Replace with your dataset
    eval_dataset=None,   # Optional validation dataset
    output_dir="./lora_finetuned_model",
    max_epochs=3,
    batch_size=8,
    learning_rate=3e-4
)

# Start fine-tuning
trainer.train()

# Save the fine-tuned LoRA adapters
trainer.save_adapter("./lora_adapter")

output

Epoch 1/3: loss=2.345, accuracy=0.72
Epoch 2/3: loss=1.876, accuracy=0.78
Epoch 3/3: loss=1.543, accuracy=0.82
Training complete. LoRA adapters saved to ./lora_adapter

Common variations

Use trainer.train(async_mode=True) for asynchronous training if supported.
Adjust lora_config to target different modules or change rank and dropout.
Switch base models by changing model_name to other Hugging Face models compatible with LoRA.
Integrate with datasets library for loading and preprocessing training data.

Troubleshooting

If you encounter CUDA out-of-memory errors, reduce batch_size or use mixed precision training.
Ensure your transformers and unsloth versions are compatible.
Verify that target_modules in LoraConfig match the model architecture.
Check that your dataset is properly tokenized and formatted for causal language modeling.

✅

Key Takeaways

Install Unsloth and dependencies before starting LoRA fine-tuning.
Configure LoRA parameters precisely to match your model's architecture.
Use UnslothTrainer to streamline LoRA adapter training with minimal code.
Adjust batch size and precision to manage GPU memory during training.
Validate your dataset format to avoid tokenization and training errors.

Verified 2026-04 · meta-llama/Llama-3.1-8B-Instruct

Verify ↗