How to load documents in LlamaIndex
Quick answer
Use LlamaIndex's document loaders like
SimpleDirectoryReader or TextLoader to load documents from files or directories into Document objects. Then, pass these documents to LlamaIndex's index builders for querying and retrieval.PREREQUISITES
Python 3.8+pip install llama-index>=0.6.0Basic knowledge of Python file handling
Setup
Install the llama-index package via pip and prepare your environment.
pip install llama-index>=0.6.0 Step by step
Load documents from a directory using SimpleDirectoryReader and print their content.
from llama_index import SimpleDirectoryReader
# Load documents from a local directory
loader = SimpleDirectoryReader('data')
documents = loader.load_data()
# Print the first document's text
print(documents[0].text) output
This is the content of the first document loaded from the 'data' directory.
Common variations
You can also load single files using TextLoader or load documents asynchronously with async loaders if supported.
from llama_index import TextLoader
# Load a single text file
loader = TextLoader('data/example.txt')
documents = loader.load_data()
print(documents[0].text) output
Contents of example.txt file.
Troubleshooting
- If you get a
FileNotFoundError, verify the file path and directory exist. - Ensure your documents are in supported formats like .txt or .md.
- Check that
llama-indexis up to date to avoid deprecated loader issues.
Key Takeaways
- Use
SimpleDirectoryReaderto bulk load documents from folders. - Use
TextLoaderfor loading individual files. - Always verify file paths and supported formats to avoid loading errors.