How to beginner · 3 min read

How to use dropout in PyTorch

Q: How to use dropout in PyTorch

Use torch.nn.Dropout in PyTorch to randomly zero some elements of the input tensor during training, which helps prevent overfitting. Instantiate Dropout(p) with dropout probability p, apply it in your model's forward method, and ensure the model is in training mode to activate dropout.

Quick answer

Use torch.nn.Dropout in PyTorch to randomly zero some elements of the input tensor during training, which helps prevent overfitting. Instantiate Dropout(p) with dropout probability p, apply it in your model's forward method, and ensure the model is in training mode to activate dropout.

PREREQUISITES

Python 3.8+
pip install torch>=2.0

Setup

Install PyTorch if you haven't already. Use the official command from PyTorch installation guide. For CPU-only:

bash

pip install torch torchvision torchaudio

Step by step

Here is a complete example showing how to use torch.nn.Dropout in a simple neural network. Dropout is active only during training mode.

python

import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 20)
        self.dropout = nn.Dropout(p=0.5)  # 50% dropout
        self.fc2 = nn.Linear(20, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.dropout(x)  # Apply dropout here
        x = self.fc2(x)
        return x

# Instantiate model
model = SimpleNet()

# Input tensor
input_tensor = torch.randn(5, 10)

# Training mode activates dropout
model.train()
output_train = model(input_tensor)
print("Output in training mode (dropout active):", output_train)

# Evaluation mode disables dropout
model.eval()
output_eval = model(input_tensor)
print("Output in eval mode (dropout inactive):", output_eval)

output

Output in training mode (dropout active): tensor([[...], [...], [...], [...], [...]])
Output in eval mode (dropout inactive): tensor([[...], [...], [...], [...], [...]])

Common variations

You can adjust the dropout probability p to control the fraction of activations dropped. Use nn.Dropout2d or nn.Dropout3d for spatial dropout in convolutional layers. Dropout is automatically disabled in model.eval() mode.

python

import torch.nn as nn

# Spatial dropout for 2D feature maps
spatial_dropout = nn.Dropout2d(p=0.3)

# Example usage in a conv net forward pass
x = torch.randn(16, 32, 28, 28)  # batch, channels, height, width
output = spatial_dropout(x)
print(output.shape)  # same shape as input

output

(16, 32, 28, 28)

Troubleshooting

If dropout seems ineffective, ensure your model is in training mode with model.train().
Remember dropout is disabled in evaluation mode model.eval().
Check that dropout layers are correctly placed after activation functions.

✅

Key Takeaways

Use torch.nn.Dropout with a probability p to randomly zero activations during training.
Activate dropout by setting model.train() and disable it with model.eval().
For convolutional layers, use Dropout2d or Dropout3d for spatial dropout.
Place dropout after activation functions like ReLU for best regularization effect.

Verified 2026-04

Verify ↗