ValueError
builtins.ValueError
Stack trace
ValueError: Embedding dimension mismatch: expected 1536 but got 512
File "app.py", line 42, in add_to_index
index.upsert(vectors=[(id, embedding)])
File "pinecone/index.py", line 123, in upsert
raise ValueError(f"Embedding dimension mismatch: expected {self.dimension} but got {len(embedding)}") Why it happens
Vector stores require all embeddings to have the same fixed dimension as defined at index creation. If you generate embeddings from a model with a different dimension than the vector store expects, the store rejects them with a dimension mismatch error. This often happens when switching embedding models or reusing an index created with a different embedding size.
Detection
Check the embedding vector length before upserting or querying the vector store. Log the expected dimension from the index metadata and compare it to the generated embedding length to catch mismatches early.
Causes & fixes
Using an embedding model that produces vectors of a different dimension than the vector store index expects
Ensure the embedding model matches the vector store index dimension or recreate the index with the correct dimension matching your embedding model.
Reusing an existing vector store index created with a different embedding dimension
Delete and recreate the vector store index with the embedding dimension matching your current embedding model.
Passing incomplete or corrupted embedding vectors that have fewer dimensions than expected
Validate embedding vectors after generation to confirm their length matches the expected dimension before inserting into the vector store.
Code: broken vs fixed
from pinecone import Pinecone
import os
pinecone = Pinecone(api_key=os.environ['PINECONE_API_KEY'])
index = pinecone.Index('my-index')
embedding = [0.1] * 512 # Embedding vector with dimension 512
index.upsert(vectors=[('vec1', embedding)]) # This line raises ValueError due to dimension mismatchfrom pinecone import Pinecone
import os
from openai import OpenAI
pinecone = Pinecone(api_key=os.environ['PINECONE_API_KEY'])
index = pinecone.Index('my-index')
client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])
response = client.embeddings.create(input='test text', model='text-embedding-3-large')
embedding = response.data[0].embedding # Embedding vector with correct dimension
index.upsert(vectors=[('vec1', embedding)]) # Fixed: embedding dimension matches index
print('Upsert succeeded with correct embedding dimension')Workaround
Catch the ValueError on upsert, log the embedding vector length and index dimension, then skip or transform embeddings that do not match the expected dimension.
Prevention
Always create vector store indexes with the embedding dimension of your chosen model and enforce embedding dimension validation before insertion to avoid mismatches.