Cosine, dot product or Euclidean: choosing a vector distance metric without the math headache

#The goal

You're creating a vector index, and it asks for a metric: cosine, dot product or Euclidean. Which should you choose? In most cases the answer is simple, and you'll see why by running a few lines of Python.

If embeddings are new to you, read embeddings and vector search basics first.

#The short answer

1. Use the metric your embedding model's documentation recommends.
2. If it doesn't say, use cosine.

#See them in Python

pip install numpy

import numpy as np

a = np.array([1.0, 2.0, 3.0])
b = np.array([2.0, 4.0, 6.0])     # same direction as a, twice as long
c = np.array([3.0, 2.0, 1.0])     # different direction

def dot(x, y):
    return float(np.dot(x, y))

def cosine(x, y):
    return float(np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y)))

def euclidean(x, y):
    return float(np.linalg.norm(x - y))

for name, other in [("b", b), ("c", c)]:
    print(name,
          "cosine", round(cosine(a, other), 3),
          "dot", round(dot(a, other), 3),
          "euclid", round(euclidean(a, other), 3))

Output:

b cosine 1.0 dot 28.0 euclid 3.742
c cosine 0.714 dot 10.0 euclid 2.828

Look closely at the results. Cosine says b is the perfect match, because it points in the same direction. Euclidean says that c is closer, because its point in space is nearer. So the metric can change which result comes first.

#What each one measures

In text search, you usually care about direction, meaning what the text is about. That's why cosine is the common default.

#Why it often doesn't matter

Most modern text embedding models return normalized vectors, each with a length of 1. When vectors are normalized, all three metrics give the same ranking:

def normalize(v):
    return v / np.linalg.norm(v)

rng = np.random.default_rng(42)
query = normalize(rng.normal(size=8))
docs = [normalize(rng.normal(size=8)) for _ in range(5)]

by_cos = sorted(range(5), key=lambda i: -cosine(query, docs[i]))
by_dot = sorted(range(5), key=lambda i: -dot(query, docs[i]))
by_l2 = sorted(range(5), key=lambda i: euclidean(query, docs[i]))

print(by_cos, by_dot, by_l2)   # all three lists match

On normalized vectors, dot product equals cosine, and it's cheaper to compute, since there's no division. Some vector databases use this as an optimization.

Check whether your model's vectors are normalized:

import numpy as np

# embeddings: a list of vectors from your model
norms = np.linalg.norm(np.array(embeddings), axis=1)
print(norms.min(), norms.max())

If they're all very close to 1.0, your vectors are normalized.

#When it does matter

Dot product on non-normalized vectors tends to favor long vectors, whatever they contain. That's a problem unless the model was designed for it.

#Set it in your database

Names differ between products:

For example, in pgvector, the index and the query must use the same metric:

CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

SELECT id, title, 1 - (embedding <=> $1) AS cosine_similarity
FROM documents
ORDER BY embedding <=> $1
LIMIT 5;

Warning

In pgvector, if your query uses <-> while the index was built with vector_cosine_ops, the index can't be used. You'll get a slow sequential scan. See query not using the HNSW index.

In Chroma, set the metric when you create the collection, because you can't change it afterward. The default isn't cosine. See low recall and fixed settings.

import chromadb

client = chromadb.Client()
collection = client.create_collection(
    name="docs",
    configuration={"hnsw": {"space": "cosine"}},
)

The way to set it depends on your Chroma version, and older versions use metadata={"hnsw:space": "cosine"}.

#Read the score correctly

A frequent mistake is treating every score as "higher is better".

Check what your product returns. If you set a threshold like score > 0.7 on a distance, you'll throw out your best results and keep the worst ones.

Tip

Don't reuse a score threshold after changing the model or metric. Score ranges vary widely. A good match on one model may score 0.85, and 0.45 on another. Set thresholds by testing your own data.

#Common mistakes

Changing the metric on an existing index usually means rebuilding it. The vectors stay the same, but the index structure depends on the metric.

#Test your choice

Don't argue about it. Measure it. Take 20 to 50 real questions, with known good documents, and compare the recall@k for each option. For normalized vectors, you should see no meaningful difference, and then you can stop thinking about it.

#Prevent it next time

• Write the metric and the model name into your index's documentation or metadata.
• Set the metric explicitly when creating an index, and never rely on a default.
• Check normalization when you evaluate a new embedding model.
• Next, read HNSW tuning and choosing a vector store.