How to Beginner · 3 min read

How to move tensor to GPU in PyTorch

Q: How to move tensor to GPU in PyTorch

In PyTorch, move a tensor to GPU by calling tensor.to('cuda') or tensor.cuda(). This transfers the tensor from CPU memory to GPU memory, enabling faster computation on CUDA-enabled devices.

Quick answer

In PyTorch, move a tensor to GPU by calling tensor.to('cuda') or tensor.cuda(). This transfers the tensor from CPU memory to GPU memory, enabling faster computation on CUDA-enabled devices.

PREREQUISITES

Python 3.8+
pip install torch>=2.0
CUDA-enabled GPU with proper drivers installed

Setup

Ensure you have PyTorch installed with CUDA support. You can install it via pip with the appropriate CUDA version. Verify your GPU is detected by PyTorch.

python

import torch

print(torch.__version__)
print(torch.cuda.is_available())
print(torch.cuda.current_device())
print(torch.cuda.get_device_name(torch.cuda.current_device()))

output

2.0.1
True
0
NVIDIA GeForce RTX 3080

Step by step

Create a tensor on CPU and move it to GPU using .to('cuda') or .cuda(). Confirm the device of the tensor before and after moving.

python

import torch

# Create a tensor on CPU
cpu_tensor = torch.randn(3, 3)
print(f"Original tensor device: {cpu_tensor.device}")

# Move tensor to GPU
if torch.cuda.is_available():
    gpu_tensor = cpu_tensor.to('cuda')  # or cpu_tensor.cuda()
    print(f"Tensor device after moving to GPU: {gpu_tensor.device}")
else:
    print("CUDA is not available on this machine.")

output

Original tensor device: cpu
Tensor device after moving to GPU: cuda:0

Common variations

Use .to(device) with a device variable for flexible code.
Move model parameters to GPU similarly with model.to('cuda').
Use torch.device('cuda:0') to specify GPU index.

python

import torch

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

# Create tensor directly on device
tensor = torch.randn(2, 2, device=device)
print(f"Tensor device: {tensor.device}")

# Move tensor back to CPU
tensor_cpu = tensor.to('cpu')
print(f"Tensor device after moving back to CPU: {tensor_cpu.device}")

output

Tensor device: cuda:0
Tensor device after moving back to CPU: cpu

Troubleshooting

If you get RuntimeError: CUDA error, check your GPU drivers and CUDA installation.
Ensure your tensor is on the same device as your model before operations.
Use torch.cuda.empty_cache() to clear GPU memory if you run out of memory.

✅

Key Takeaways

Use tensor.to('cuda') or tensor.cuda() to move tensors to GPU in PyTorch.
Check torch.cuda.is_available() before moving tensors to avoid runtime errors.
Use torch.device for flexible device management in your code.

Verified 2026-04

Verify ↗