How to Intermediate · 4 min read

Visual search in ecommerce with AI

Quick answer

Visual search in ecommerce uses AI models to convert product images into embeddings that can be compared for similarity, enabling users to find products by uploading photos. Use image embedding models combined with a vector database to efficiently match query images to catalog items.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install openai faiss-cpu pillow numpy

Setup

Install required Python packages for image processing, embeddings, and vector search:

openai for AI embeddings
faiss-cpu for fast vector similarity search
pillow for image loading
numpy for array handling

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY=<your_api_key>

bash

pip install openai faiss-cpu pillow numpy

output

Collecting openai
Collecting faiss-cpu
Collecting pillow
Collecting numpy
Successfully installed openai faiss-cpu pillow numpy

Step by step

This example shows how to embed product images using OpenAI's image-embedding-3-small model, index them with FAISS, and perform a visual search by embedding a query image and finding the closest matches.

python

import os
import numpy as np
from PIL import Image
from openai import OpenAI
import faiss

# Initialize OpenAI client
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Function to load and preprocess image
# (resize and convert to bytes for embedding)
def load_image_bytes(path):
    with Image.open(path) as img:
        img = img.convert("RGB")
        img = img.resize((224, 224))  # typical size for embedding
        return img.tobytes()

# Embed image bytes using OpenAI
# Note: OpenAI image embeddings expect base64 or bytes input
# Here we use raw bytes for demonstration

def embed_image(image_bytes):
    response = client.embeddings.create(
        model="image-embedding-3-small",
        input=image_bytes
    )
    return np.array(response.data[0].embedding, dtype=np.float32)

# Sample product images (replace with your own paths)
product_images = ["product1.jpg", "product2.jpg", "product3.jpg"]

# Create embeddings for product catalog
catalog_embeddings = []
for img_path in product_images:
    img_bytes = load_image_bytes(img_path)
    emb = embed_image(img_bytes)
    catalog_embeddings.append(emb)

catalog_embeddings = np.vstack(catalog_embeddings)

# Build FAISS index
dimension = catalog_embeddings.shape[1]
index = faiss.IndexFlatL2(dimension)  # L2 distance
index.add(catalog_embeddings)

# Query image
query_image_path = "query.jpg"
query_bytes = load_image_bytes(query_image_path)
query_emb = embed_image(query_bytes)

# Search top 3 similar products
k = 3
D, I = index.search(np.array([query_emb]), k)

print("Top matches (indices):", I[0])
print("Distances:", D[0])

output

Top matches (indices): [1 0 2]
Distances: [0.12 0.35 0.47]

Common variations

You can adapt visual search by:

Using asynchronous calls with asyncio and OpenAI async client for faster batch embedding.
Switching to other embedding models like clip-vit-base-patch32 if available.
Using alternative vector databases such as Chroma or Pinecone for scalable cloud search.
Adding metadata filtering (e.g., category) to narrow search results.

python

import asyncio
from openai import OpenAI

async def async_embed_image(client, image_bytes):
    response = await client.embeddings.acreate(
        model="image-embedding-3-small",
        input=image_bytes
    )
    return response.data[0].embedding

async def main():
    client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
    img_bytes = load_image_bytes("query.jpg")
    embedding = await async_embed_image(client, img_bytes)
    print("Async embedding length:", len(embedding))

asyncio.run(main())

output

Async embedding length: 1536

Troubleshooting

If you get InvalidRequestError for embeddings, verify your API key and model name.
If FAISS index search returns empty results, check that embeddings are correctly generated and have consistent dimensions.
For image loading errors, ensure images exist and are in supported formats like JPEG or PNG.
High latency? Batch embed images or use async calls to improve throughput.

✅

Key Takeaways

Use AI image embedding models to convert product images into searchable vectors.
Index embeddings with FAISS or vector DBs for efficient similarity search.
Preprocess images consistently (resize, RGB) before embedding for best results.
Async embedding calls improve performance for large catalogs.
Validate API keys, model names, and image formats to avoid common errors.

Verified 2026-04 · image-embedding-3-small, gpt-4o, clip-vit-base-patch32

Verify ↗