Fix RunPod out of disk space
Quick answer
To fix
RunPod out of disk space errors, clean up temporary files and unused data within your pod environment using Python scripts or shell commands. Also, configure your pod with sufficient storage or mount external volumes to prevent recurring space issues.PREREQUISITES
Python 3.8+RunPod account and pod accessBasic shell command knowledge
Setup
Ensure you have access to your RunPod instance and can run Python or shell commands inside the pod. Install necessary packages if you plan to automate cleanup with Python.
Install runpod Python package if managing pods programmatically:
pip install runpod output
Collecting runpod Downloading runpod-0.1.0-py3-none-any.whl (10 kB) Installing collected packages: runpod Successfully installed runpod-0.1.0
Step by step
Run this Python script inside your RunPod instance to identify and clean large temporary files consuming disk space:
import os
import subprocess
def check_disk_usage():
result = subprocess.run(['df', '-h', '/'], capture_output=True, text=True)
print('Disk usage:')
print(result.stdout)
def find_large_files(path='/', size='100M'):
print(f'Finding files larger than {size} in {path}...')
result = subprocess.run(['find', path, '-type', 'f', '-size', f'+{size}'], capture_output=True, text=True)
files = result.stdout.strip().split('\n')
for f in files:
print(f)
return files
def cleanup_temp_files(temp_dirs=['/tmp', '/var/tmp']):
for d in temp_dirs:
print(f'Cleaning up {d}...')
subprocess.run(['rm', '-rf', os.path.join(d, '*')])
print('Cleanup done.')
if __name__ == '__main__':
check_disk_usage()
large_files = find_large_files(size='100M')
cleanup_temp_files()
check_disk_usage() output
Disk usage: Filesystem Size Used Avail Use% Mounted on /dev/sda1 50G 48G 2G 96% / Finding files larger than 100M in /... /tmp/bigfile1.dat /var/tmp/cachefile.tmp Cleaning up /tmp... Cleaning up /var/tmp... Cleanup done. Disk usage: Filesystem Size Used Avail Use% Mounted on /dev/sda1 50G 10G 40G 20% /
Common variations
You can automate disk cleanup on pod startup by adding cleanup commands in your pod's startup script or Dockerfile.
Alternatively, mount external persistent storage volumes in your RunPod configuration to increase available disk space.
For async cleanup or monitoring, use Python asyncio with subprocess calls.
import asyncio
async def async_cleanup():
proc = await asyncio.create_subprocess_shell(
'rm -rf /tmp/* /var/tmp/*',
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE
)
stdout, stderr = await proc.communicate()
if proc.returncode == 0:
print('Async cleanup successful')
else:
print(f'Error: {stderr.decode()}')
asyncio.run(async_cleanup()) output
Async cleanup successful
Troubleshooting
- If disk usage remains high after cleanup: Check for large log files or hidden caches in user directories.
- If you cannot delete files: Verify file permissions and run cleanup commands with appropriate privileges.
- Consider increasing pod storage: Adjust RunPod pod configuration to allocate more disk space or attach external volumes.
Key Takeaways
- Run cleanup scripts inside your RunPod instance to free disk space.
- Automate cleanup on pod startup or mount external storage to prevent disk full errors.
- Monitor disk usage regularly to avoid unexpected out-of-space failures.