How to persist LlamaIndex index to disk
Quick answer
Use LlamaIndex's
save_to_disk method to persist an index to a file and load_from_disk to reload it later. This allows you to save your index state and reuse it without rebuilding, improving efficiency.PREREQUISITES
Python 3.8+pip install llama-indexBasic familiarity with LlamaIndex usage
Setup
Install the llama-index package via pip and ensure you have Python 3.8 or newer. No API keys are required for index persistence.
pip install llama-indexStep by step
This example shows how to create a simple LlamaIndex index, save it to disk, and then load it back for querying.
from llama_index import SimpleDirectoryReader, GPTSimpleVectorIndex
# Load documents from a directory
documents = SimpleDirectoryReader('data').load_data()
# Create the index
index = GPTSimpleVectorIndex(documents)
# Save the index to disk
index.save_to_disk('index.json')
# Later or in another script, load the index
loaded_index = GPTSimpleVectorIndex.load_from_disk('index.json')
# Query the loaded index
response = loaded_index.query('What is LlamaIndex?')
print(response.response) output
LlamaIndex is a library that helps you build and query indices over your documents efficiently.
Common variations
- You can persist other index types similarly using their
save_to_diskandload_from_diskmethods. - Use absolute or relative paths for saving/loading.
- For async workflows, wrap calls in async functions but persistence methods are synchronous.
Troubleshooting
- If
load_from_diskfails, verify the file path and that the file was saved correctly. - Ensure consistent LlamaIndex versions between saving and loading to avoid compatibility issues.
- Check file permissions if you get access errors.
Key Takeaways
- Use
save_to_diskandload_from_diskto persist LlamaIndex indices efficiently. - Persisted indices speed up reuse by avoiding re-indexing documents.
- Always verify file paths and LlamaIndex versions to prevent loading errors.