RuntimeError
torch.autograd.runtime.RuntimeError
Stack trace
RuntimeError: Detected a cycle in the autograd graph, which is not supported.
Traceback (most recent call last):
File "script.py", line 15, in <module>
loss.backward() # triggers RuntimeError: autograd graph cyclic dependency
File "/usr/local/lib/python3.9/site-packages/torch/tensor.py", line 245, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/local/lib/python3.9/site-packages/torch/autograd/__init__.py", line 147, in backward
Variable._execution_engine.run_backward(*args)
RuntimeError: Detected a cycle in the autograd graph, which is not supported. Why it happens
PyTorch's autograd engine builds a directed acyclic graph (DAG) to track operations for gradient computation. If your model or loss computation creates a cycle in this graph, autograd cannot compute gradients properly and raises this error. This often happens when tensors are reused or assigned in a way that creates circular references in the computation graph.
Detection
Monitor your model's forward pass for in-place operations or tensor assignments that reuse outputs as inputs. Use torch.autograd.set_detect_anomaly(True) to get detailed traceback on where the cycle is introduced.
Causes & fixes
In-place operations modifying tensors that are needed for gradient computation
Avoid in-place operations (e.g., use x = x + y instead of x += y) on tensors involved in autograd.
Reusing the same tensor as input and output in a computation creating a cycle
Ensure that outputs are new tensors and do not overwrite inputs used in the computation graph.
Manually constructing computation graphs with circular references (e.g., custom autograd Functions)
Review custom autograd Functions to ensure no cycles are introduced; break cycles by detaching tensors where appropriate.
Code: broken vs fixed
import torch
x = torch.tensor([1.0], requires_grad=True)
y = x + 1
x = y # overwriting input tensor with output creates cycle
loss = x * 2
loss.backward() # triggers RuntimeError: autograd graph cyclic dependencyimport os
import torch
os.environ['PYTORCH_ENABLE_ANOMALY_DETECTION'] = '1' # Enable anomaly detection for debugging
x = torch.tensor([1.0], requires_grad=True)
y = x + 1
z = y # assign output to a new variable, avoid overwriting input
loss = z * 2
loss.backward() # fixed: no cyclic dependency
print(x.grad) # prints tensor([2.])Workaround
Wrap your backward call in try/except RuntimeError and use torch.autograd.set_detect_anomaly(True) to identify the operation causing the cycle, then refactor to remove in-place modifications or tensor reuse.
Prevention
Design your model and computations to avoid in-place operations on tensors required for gradient computation and never overwrite inputs with outputs in the same computation step to keep the autograd graph acyclic.