How to beginner · 3 min read

How to freeze layers in PyTorch

Q: How to freeze layers in PyTorch

To freeze layers in PyTorch, set the requires_grad attribute of the layer's parameters to False. This prevents gradients from being computed and updated during training, effectively freezing those layers.

Quick answer

To freeze layers in PyTorch, set the requires_grad attribute of the layer's parameters to False. This prevents gradients from being computed and updated during training, effectively freezing those layers.

PREREQUISITES

Python 3.8+
pip install torch>=2.0

Setup

Install PyTorch if not already installed. Use the following command to install the latest stable version:

bash

pip install torch torchvision

Step by step

This example shows how to freeze all layers of a pretrained ResNet18 model except the final fully connected layer for transfer learning.

python

import torch
import torchvision.models as models

# Load pretrained ResNet18
model = models.resnet18(pretrained=True)

# Freeze all layers
for param in model.parameters():
    param.requires_grad = False

# Replace the final fully connected layer (fc) to match new task
num_features = model.fc.in_features
model.fc = torch.nn.Linear(num_features, 10)  # e.g., 10 classes

# Only the new fc layer's parameters require gradients
for param in model.fc.parameters():
    param.requires_grad = True

# Verify which parameters are frozen
frozen = [not p.requires_grad for p in model.parameters()]
print(f"Frozen layers count: {sum(frozen)}")
print(f"Trainable layers count: {len(frozen) - sum(frozen)}")

output

Frozen layers count: 63
Trainable layers count: 1

Common variations

You can freeze specific layers by targeting them by name or index instead of freezing all. For example, freeze only the first two convolutional blocks:

python

import torch
import torchvision.models as models

model = models.resnet18(pretrained=True)

# Freeze first two layers (conv1 and layer1)
for name, param in model.named_parameters():
    if name.startswith('conv1') or name.startswith('layer1'):
        param.requires_grad = False
    else:
        param.requires_grad = True

# Check which layers are frozen
for name, param in model.named_parameters():
    print(f"{name}: requires_grad={param.requires_grad}")

output

conv1.weight: requires_grad=False
bn1.weight: requires_grad=False
bn1.bias: requires_grad=False
layer1.0.conv1.weight: requires_grad=False
layer1.0.bn1.weight: requires_grad=False
layer1.0.bn1.bias: requires_grad=False
layer1.0.conv2.weight: requires_grad=False
layer1.0.bn2.weight: requires_grad=False
layer1.0.bn2.bias: requires_grad=False
layer1.1.conv1.weight: requires_grad=False
layer1.1.bn1.weight: requires_grad=False
layer1.1.bn1.bias: requires_grad=False
layer1.1.conv2.weight: requires_grad=False
layer1.1.bn2.weight: requires_grad=False
layer1.1.bn2.bias: requires_grad=False
layer2.0.conv1.weight: requires_grad=True
layer2.0.bn1.weight: requires_grad=True
layer2.0.bn1.bias: requires_grad=True
... (continues for other layers)

Troubleshooting

If your model still updates frozen layers, verify you set requires_grad = False before the optimizer is created.
Ensure you pass only parameters with requires_grad=True to the optimizer.
Use print statements to confirm which parameters are frozen.

✅

Key Takeaways

Set param.requires_grad = False to freeze layers in PyTorch models.
Freeze layers before creating the optimizer to avoid updating frozen parameters.
You can selectively freeze layers by targeting parameters by name or module.
Only parameters with requires_grad=True are updated during training.
Freezing layers is essential for efficient transfer learning and fine-tuning.

Verified 2026-04 · resnet18

Verify ↗