How to beginner · 4 min read

How to use Weights and Biases with Hugging Face

Quick answer

Use the wandb Python package to integrate Weights and Biases with Hugging Face by initializing wandb.init() before training and passing report_to="wandb" in transformers.TrainingArguments. This enables automatic logging of metrics, model checkpoints, and hyperparameters during training.

PREREQUISITES

Python 3.8+
pip install transformers datasets wandb
Weights and Biases account and API key
Hugging Face Transformers library

Setup

Install the required packages and configure your environment with your Weights and Biases API key.

Install packages: pip install transformers datasets wandb
Set your W&B API key: export WANDB_API_KEY=your_api_key (Linux/macOS) or setx WANDB_API_KEY your_api_key (Windows)

bash

pip install transformers datasets wandb

Step by step

This example shows how to train a Hugging Face transformers model with Weights and Biases tracking enabled. It initializes a W&B run, sets report_to="wandb" in TrainingArguments, and trains a simple text classification model.

python

import os
import wandb
from transformers import AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments
from datasets import load_dataset

# Initialize W&B run
wandb.init(project="hf-wandb-demo", entity="your_wandb_username")

# Load dataset and tokenizer
dataset = load_dataset("glue", "mrpc")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# Tokenize function
def tokenize_function(examples):
    return tokenizer(examples["sentence1"], examples["sentence2"], truncation=True)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

# Load model
model = AutoModelForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=2)

# Define training arguments with W&B reporting
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=2e-5,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    num_train_epochs=1,
    weight_decay=0.01,
    report_to="wandb"
)

# Initialize Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"]
)

# Train model
trainer.train()

# Finish W&B run
wandb.finish()

output

***** Running training *****
  Num examples = 3668
  Num Epochs = 1
  Instantaneous batch size per device = 8
  Total train batch size = 8
  ...
Epoch 1: train_loss=0.25 eval_loss=0.20 eval_accuracy=0.85
***** Training complete *****

Common variations

You can customize your W&B integration with these variations:

Async logging: W&B logs asynchronously by default; no extra setup needed.
Custom metrics: Use wandb.log({"metric_name": value}) inside training loops for manual logging.
Different models: Replace bert-base-uncased with any Hugging Face model.
Use with Hugging Face Trainer callbacks: Add WandbCallback for advanced control.

python

from transformers.integrations import WandbCallback

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"],
    callbacks=[WandbCallback]
)

Troubleshooting

If W&B logs do not appear: Verify your WANDB_API_KEY environment variable is set correctly and you are logged in (wandb login).
Training stalls or errors: Check for version mismatches between transformers and wandb packages.
Multiple runs conflict: Use unique run_name in wandb.init() to differentiate runs.

✅

Key Takeaways

Use report_to="wandb" in TrainingArguments to enable automatic W&B logging.
Initialize W&B runs with wandb.init() before training to track experiments.
You can log custom metrics anytime with wandb.log() inside training loops.
Ensure your WANDB_API_KEY is set in the environment for authentication.
Use WandbCallback for advanced integration with Hugging Face Trainer.

Verified 2026-04 · bert-base-uncased

Verify ↗