Applies toHosted embedding models (Azure OpenAI / Foundry, Bedrock, Cohere, Google, Voyage-style providers) and open-weight models (e.g. BGE, E5, GTE families) served yourself
Criteria
| Criterion | Questions |
|---|---|
| Retrieval quality | Recall@k and ranking on your queries and documents |
| Languages | All languages your users and documents use, including mixed-language queries |
| Domain | Technical, legal, medical, code? |
| Input length | Max tokens per input vs your chunk size |
| Dimensions | Storage and memory cost; flexible dimensions supported? |
| Cost | Per token (hosted) or GPU/CPU (self-hosted); re-embedding cost for the whole corpus |
| Hosting | Data residency, private networking, offline needs |
| Stability | Version pinning, deprecation policy |
Public benchmarks (such as MTEB) help shortlist, but your data decides.
Evaluate on your data
- Collect 200–500 real queries with the documents/chunks that answer them.
- Embed the corpus sample with each candidate.
- Measure recall@5/@10 and MRR; also check failure cases manually.
- Include hybrid search (keyword + vector) in the comparison; it often closes gaps between models.
| Model | Recall@10 | MRR | Dims | Cost to embed corpus | Notes |
|---|---|---|---|---|---|
| Candidate A | |||||
| Candidate B |
Trade-offs
- Bigger models/dimensions aren't always better on your data.
- Flexible-dimension models let you cut storage later without changing models.
- Open-weight models give control and no per-token cost, but you run the serving.
- Some models need different prefixes for queries and documents; follow the model card.
Plan for change
- Store model name and version with each vector.
- Keep chunk text durable so re-embedding is possible.
- Use a blue/green index approach for model changes (see the embedding migration post and the related Sev1).
Tip
A rerank step can make a mid-size embedding model perform close to a much larger one. Evaluate embedding + reranker combinations, not embedding models alone.
Checklist
- Criteria and constraints listed
- Shortlist from public benchmarks
- Evaluated on your queries (with hybrid and reranking)
- Dimensions and cost considered
- Model/version stored with vectors
- Migration approach planned