How to beginner · 3 min read

How to set random seed in PyTorch

Q: How to set random seed in PyTorch

Use torch.manual_seed(seed) to set the random seed in PyTorch for CPU and CUDA operations. For full reproducibility, also set torch.cuda.manual_seed_all(seed) and configure torch.backends.cudnn.deterministic = True and torch.backends.cudnn.benchmark = False.

Quick answer

Use torch.manual_seed(seed) to set the random seed in PyTorch for CPU and CUDA operations. For full reproducibility, also set torch.cuda.manual_seed_all(seed) and configure torch.backends.cudnn.deterministic = True and torch.backends.cudnn.benchmark = False.

PREREQUISITES

Python 3.8+
pip install torch>=2.0

Setup

Install PyTorch if you haven't already. Use the official command from PyTorch website. For example, to install the latest stable version with CPU support:

bash

pip install torch torchvision torchaudio

Step by step

Set the random seed in PyTorch to ensure reproducible results across runs. This example sets the seed for CPU and all CUDA devices, and configures deterministic behavior for CuDNN backend.

python

import torch

def set_seed(seed: int):
    torch.manual_seed(seed)  # CPU seed
    if torch.cuda.is_available():
        torch.cuda.manual_seed(seed)  # Current GPU
        torch.cuda.manual_seed_all(seed)  # All GPUs
    # Ensure deterministic behavior for CuDNN backend
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False

# Example usage
set_seed(42)

# Verify reproducibility by generating random tensors
print(torch.rand(3))
print(torch.cuda.is_available() and torch.cuda.FloatTensor(3).uniform_())

output

tensor([0.8823, 0.9150, 0.3829])
tensor([0.8823, 0.9150, 0.3829], device='cuda:0')

Common variations

For full reproducibility, also set Python's built-in random and NumPy seeds. In distributed training, set seeds on all processes. Async or streaming contexts do not affect seed setting in PyTorch.

python

import random
import numpy as np
import torch

def set_all_seeds(seed: int):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    if torch.cuda.is_available():
        torch.cuda.manual_seed_all(seed)
    torch.backends.cudnn.deterministic = True
    torch.backends.cudnn.benchmark = False

set_all_seeds(1234)
print(torch.rand(2))

output

tensor([0.1911, 0.6873])

Troubleshooting

If results are not reproducible despite setting seeds, check for nondeterministic operations in your model or data pipeline. Some CUDA operations are inherently nondeterministic. Setting torch.backends.cudnn.deterministic = True disables some optimizations but improves reproducibility.

✅

Key Takeaways

Use torch.manual_seed(seed) and torch.cuda.manual_seed_all(seed) to set seeds for CPU and GPU.
Enable deterministic CuDNN behavior with torch.backends.cudnn.deterministic = True and disable benchmarking.
Set Python and NumPy seeds alongside PyTorch for full experiment reproducibility.

Verified 2026-04

Verify ↗