wandb.errors.CommError
wandb.errors.CommError: run already finished log error
Stack trace
wandb.errors.CommError: run already finished, cannot log new data
File "/usr/local/lib/python3.9/site-packages/wandb/sdk/wandb_run.py", line 1234, in log
raise CommError("run already finished, cannot log new data")
File "/app/train.py", line 56, in train_loop
wandb.log({'loss': loss}) # triggers error Why it happens
Weights & Biases runs have a lifecycle state; once a run is finished or closed, the backend disallows further logging to maintain data integrity. Attempting to log after calling wandb.finish() or after an unexpected run termination triggers this error.
Detection
Monitor your code to ensure wandb.log() calls occur only before wandb.finish() is called, and add checks or flags to prevent logging after run completion.
Causes & fixes
Calling wandb.log() after wandb.finish() has been invoked
Ensure all logging calls happen before calling wandb.finish(), and avoid any asynchronous logging after finish.
The run was terminated unexpectedly or timed out, marking it finished on the server side
Add error handling to detect run state and restart or create a new run if the previous one is finished.
Multiple processes or threads trying to log to the same run after it finished
Synchronize logging calls and ensure only one process controls the run lifecycle, or use separate runs per process.
Code: broken vs fixed
import wandb
wandb.init(project="my-project")
wandb.finish()
wandb.log({"accuracy": 0.95}) # triggers run already finished log errorimport os
import wandb
os.environ["WANDB_API_KEY"] = os.getenv("WANDB_API_KEY") # Use env var for API key
run = wandb.init(project="my-project")
wandb.log({"accuracy": 0.95}) # log before finish
wandb.finish() # finish after all logging
print("Logged successfully before finishing run.")Workaround
Wrap wandb.log() calls in try/except wandb.errors.CommError, and on catching the error, skip logging or restart a new run to continue logging.
Prevention
Design your training or evaluation loops to complete all logging before calling wandb.finish(), and avoid asynchronous or delayed logging after run closure.