How to Intermediate · 4 min read

How to use local embeddings in LlamaIndex

Q: How to use local embeddings in LlamaIndex

Use local embeddings in LlamaIndex by integrating an open-source embedding model like SentenceTransformers or HuggingFaceEmbedding as the embedding function. Pass this embedding instance to LlamaIndex during index creation to enable offline vector search without relying on external APIs.

Quick answer

Use local embeddings in LlamaIndex by integrating an open-source embedding model like SentenceTransformers or HuggingFaceEmbedding as the embedding function. Pass this embedding instance to LlamaIndex during index creation to enable offline vector search without relying on external APIs.

PREREQUISITES

Python 3.8+
pip install llama-index sentence-transformers
Basic knowledge of vector embeddings and Python

Setup

Install llama-index and sentence-transformers to use local embedding models. Set up your Python environment and import necessary libraries.

bash

pip install llama-index sentence-transformers

Step by step

This example shows how to create a LlamaIndex using local embeddings from sentence-transformers. It builds a simple vector index from documents and queries it.

python

from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader
from llama_index.embeddings import HuggingFaceEmbedding

# Load documents from local directory
documents = SimpleDirectoryReader('data').load_data()

# Initialize local embedding model
embedding_model = HuggingFaceEmbedding(model_name='sentence-transformers/all-MiniLM-L6-v2')

# Create vector index with local embeddings
index = GPTVectorStoreIndex(documents, embed_model=embedding_model)

# Query the index
query = "What is LlamaIndex?"
response = index.query(query)
print(response.response)

output

LlamaIndex is a data framework for building AI applications with local embeddings and vector search.

Common variations

Use other local embedding models by changing model_name in HuggingFaceEmbedding.
Use SentenceTransformer directly for custom embedding pipelines.
Integrate with other LlamaIndex index types like GPTListIndex or GPTTreeIndex.

python

from sentence_transformers import SentenceTransformer
from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader

# Custom embedding function
model = SentenceTransformer('all-MiniLM-L6-v2')

def embed(texts):
    return model.encode(texts).tolist()

# Use embed function in LlamaIndex
# Note: LlamaIndex expects an embedding class, so wrap accordingly or use HuggingFaceEmbedding for simplicity.

Troubleshooting

If you get errors loading the model, ensure sentence-transformers is installed and the model name is correct.
For slow embedding generation, consider using a smaller model or batch processing.
If queries return empty results, verify documents are loaded correctly and embeddings are generated.

✅

Key Takeaways

Use HuggingFaceEmbedding in LlamaIndex to leverage local embedding models easily.
Local embeddings enable offline vector search without API calls, improving privacy and latency.
Choose embedding models based on your accuracy and performance needs, e.g., 'all-MiniLM-L6-v2' for balance.
Ensure documents are properly loaded and embeddings generated to get meaningful query results.

Verified 2026-04 · sentence-transformers/all-MiniLM-L6-v2

Verify ↗