How to beginner · 4 min read

How to use BERT for text classification

Q: How to use BERT for text classification

Use the transformers library to load a pretrained BERT model fine-tuned for text classification, such as bert-base-uncased. Tokenize your input text with BertTokenizer, then pass tokens to BertForSequenceClassification to get classification logits and predicted labels.

Quick answer

Use the transformers library to load a pretrained BERT model fine-tuned for text classification, such as bert-base-uncased. Tokenize your input text with BertTokenizer, then pass tokens to BertForSequenceClassification to get classification logits and predicted labels.

PREREQUISITES

Python 3.8+
pip install transformers torch
Basic knowledge of PyTorch or TensorFlow

Setup

Install the transformers and torch libraries from Hugging Face to use BERT for text classification.

bash

pip install transformers torch

output

Collecting transformers
Collecting torch
Successfully installed torch-2.0.1 transformers-4.35.0

Step by step

This example shows how to classify text using BertForSequenceClassification with a pretrained BERT model fine-tuned on a classification task.

python

from transformers import BertTokenizer, BertForSequenceClassification
import torch

# Load pretrained tokenizer and model
model_name = "bert-base-uncased"
tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertForSequenceClassification.from_pretrained(model_name, num_labels=2)  # binary classification

# Prepare input text
text = "This is a great movie!"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)

# Forward pass
outputs = model(**inputs)
logits = outputs.logits

# Get predicted class
predicted_class = torch.argmax(logits, dim=1).item()

print(f"Input text: {text}")
print(f"Predicted class: {predicted_class}")

output

Input text: This is a great movie!
Predicted class: 1

Common variations

Use DistilBERT or RoBERTa models for faster or more accurate classification.
Fine-tune BERT on your own labeled dataset using Trainer API from transformers.
Use TensorFlow instead of PyTorch by loading TFBertForSequenceClassification.

Troubleshooting

If you get CUDA out of memory, reduce batch size or run on CPU by setting device='cpu'.
If tokenizer or model download fails, check internet connection or use local cache.
Ensure num_labels matches your classification task labels.

✅

Key Takeaways

Use Hugging Face's transformers library to easily load pretrained BERT models for classification.
Tokenize input text with BertTokenizer before passing to BertForSequenceClassification.
Adjust num_labels to match your classification categories when loading the model.
Fine-tune BERT on your dataset for best accuracy using the Trainer API.
Handle GPU memory issues by reducing batch size or switching to CPU.

Verified 2026-04 · bert-base-uncased, DistilBERT, RoBERTa, BertForSequenceClassification

Verify ↗