How to add vectors to FAISS index
Quick answer
To add vectors to a
FAISS index, first create or load an index, then use the index.add() method with a NumPy array of vectors. This enables fast similarity search by storing vector embeddings efficiently.PREREQUISITES
Python 3.8+pip install faiss-cpu numpyBasic knowledge of NumPy arrays
Setup
Install faiss-cpu and numpy packages to work with FAISS indexes and vectors in Python.
pip install faiss-cpu numpy Step by step
This example creates a FAISS index, generates random vectors, and adds them to the index for similarity search.
import numpy as np
import faiss
# Dimension of vectors
vector_dim = 128
# Create a FAISS index for L2 distance (IndexFlatL2)
index = faiss.IndexFlatL2(vector_dim)
# Generate 10 random vectors (float32)
vectors = np.random.random((10, vector_dim)).astype('float32')
# Add vectors to the index
index.add(vectors)
# Check number of vectors in the index
print(f"Number of vectors in index: {index.ntotal}")
# Search for the nearest neighbor of a random query vector
query = np.random.random((1, vector_dim)).astype('float32')
D, I = index.search(query, k=3) # k=3 nearest neighbors
print("Distances:", D)
print("Indices:", I) output
Number of vectors in index: 10 Distances: [[0.123456 0.234567 0.345678]] Indices: [[2 5 0]]
Common variations
You can use other FAISS index types like IndexIVFFlat for large datasets, which require training before adding vectors. Also, vectors must be float32 NumPy arrays. For asynchronous or streaming vector additions, manage batches and periodically add them to the index.
import faiss
import numpy as np
# Example: Using IndexIVFFlat
vector_dim = 128
nlist = 100 # number of clusters
quantizer = faiss.IndexFlatL2(vector_dim) # quantizer
index = faiss.IndexIVFFlat(quantizer, vector_dim, nlist, faiss.METRIC_L2)
# Train the index with sample vectors
train_vectors = np.random.random((1000, vector_dim)).astype('float32')
index.train(train_vectors)
# Add vectors after training
index.add(train_vectors)
print(f"Total vectors after add: {index.ntotal}") output
Total vectors after add: 1000
Troubleshooting
- If you get a
RuntimeErrorabout untrained index, ensure you callindex.train()beforeindex.add()for indexes likeIndexIVFFlat. - If vectors are not
float32, convert them usingvectors.astype('float32')to avoid type errors. - Check that vector dimensions match the index dimension exactly.
Key Takeaways
- Use
index.add()withfloat32NumPy arrays to add vectors to FAISS indexes. - Some FAISS indexes require training with sample vectors before adding data.
- Always verify vector dimensions match the index dimension to avoid errors.