Compare embedding models

Introduction

Engineers often reach for the first available embedding model without knowing how dimension count, token limits, cost-per-call, or domain tuning will affect their system down the line. Picking the wrong model can mean bloated vector storage, slow similarity search, or retrieval quality that never quite fits your data. By the end of this lesson, you'll be able to compare models across quality, cost, latency, input length, and domain specificity—and write code that embeds the same text with two different providers so you can measure the differences directly.

Key Terminology

• Embedding dimensions — the length of the vector a model outputs for a given input; text-embedding-3-small produces 1536-dimensional vectors while text-embedding-3-large produces 3072, directly trading off retrieval quality against storage and compute cost.
• MTEB (Massive Text Embedding Benchmark) — a standardized evaluation suite that scores embedding models across retrieval, clustering, and semantic similarity tasks, giving a principled basis for quality comparisons rather than relying on vendor claims alone.
• Cosine similarity — a metric that measures the angle between two embedding vectors, computed as np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)); values closer to 1 indicate semantically related texts, which the all-MiniLM-L6-v2 code block uses to confirm the model captures meaning beyond surface-level word overlap.
• Max token limit — the maximum input length a model will encode before truncating; voyage-3 supports 32,000 tokens while all-MiniLM-L6-v2 caps at 256, making this a binding constraint that forces a chunking strategy for long documents.
• Domain-specialized model — an embedding model fine-tuned on a focused corpus such as code or legal text to outperform generalist models on in-domain retrieval benchmarks; Voyage AI's code and legal models are the lesson's primary examples.
• Local embedding model — a model such as all-MiniLM-L6-v2 loaded via SentenceTransformer that runs entirely on local hardware, eliminating per-token API costs and enabling sub-5 ms inference on GPU, at the expense of fewer dimensions and a shorter token limit.

Concepts

Model Choice Is a Systems Decision

Selecting an embedding model feels like a configuration detail, but the decision propagates through every downstream layer of your system. The model's output dimension determines how much storage each vector occupies — moving from a 384-dimensional local model to a 3072-dimensional API model multiplies storage per document roughly eightfold before you write a single line of retrieval code. At million-document scale, that gap is a real infrastructure cost. The token limit shapes whether long documents can be encoded whole or must be chunked first, which in turn determines ingestion pipeline architecture and retrieval granularity — sentence-level, paragraph-level, or document-level. Changing models after building these layers means re-embedding the entire corpus and potentially re-architecting downstream indexes. The case for reasoning through trade-offs upfront is structural, not just economic.

Reading the Five-Factor Framework

No single model wins on every axis. Comparisons in this lesson run across five dimensions: quality, cost, latency, input length, and domain fit. Quality and cost often move in opposite directions — text-embedding-3-large scores higher on MTEB retrieval benchmarks than text-embedding-3-small but costs roughly ten times more per token at scale. Latency and dimension count are similarly linked: smaller models embed faster, which matters for latency-sensitive features like real-time autocomplete where sub-100 ms round trips are required. Input length and domain fit surface only when your data exposes them — a 32,000-token context window is irrelevant if your documents are short, and a code-specialized model only wins on code retrieval tasks. The practical skill is treating each factor as a constraint derived from system requirements, eliminating models that violate any binding constraint, then comparing survivors on quality and cost (see Code Walkthrough).

Quality as an Observable Property

MTEB gives models a task-specific, comparable score rather than requiring you to trust vendor benchmarks. It separates retrieval performance from clustering and semantic similarity performance — a model that ranks well for semantic similarity may rank lower for document retrieval, and those reflect genuinely different use cases. Running the same input through text-embedding-3-small and text-embedding-3-large and printing output dimensions makes the model table concrete rather than abstract. Pairing that with the cosine similarity test on all-MiniLM-L6-v2 — where a related sentence pair scores noticeably higher than an unrelated pair — confirms that even a compact 384-dimensional local model captures semantic structure. Together, the two code blocks establish a repeatable measurement pattern: pick two models, embed the same input, and observe the difference directly rather than reasoning from spec sheets alone.

Code Walkthrough

Now that you've seen how models differ in dimensions, token limits, and intended use cases, it's time to observe those differences in running code.

When selecting an embedding model, weigh five factors against your system's constraints:

1. Quality: Higher dimensions generally improve retrieval quality. Check MTEB (Massive Text Embedding Benchmark) scores for retrieval, clustering, and semantic similarity tasks.
2. Cost: API models charge per token. text-embedding-3-small costs roughly 10× less than text-embedding-3-large—significant at million-document scale. Local models have no per-call cost but require GPU hardware.
3. Latency: Smaller models embed faster. all-MiniLM-L6-v2 can produce vectors in under 5 ms on a GPU; real-time features like autocomplete depend on sub-100 ms round trips.
4. Input length: Check max token limits. voyage-3 supports 32 K tokens; all-MiniLM-L6-v2 caps at 256. Long documents need either a large-context model or a chunking strategy.
5. Domain fit: General models work for most tasks. Voyage AI offers specialized models for code and legal text that outperform generalist models on domain benchmarks.

The code below calls two OpenAI models on the same text and prints each output dimension, making the dimension difference from the model table concrete:

Code snippetpython
1import openai
2
3client = openai.OpenAI()  # reads OPENAI_API_KEY from environment
4
5text = "Semantic search retrieves documents by meaning, not keywords."
6
7small = client.embeddings.create(model="text-embedding-3-small", input=text)
8large = client.embeddings.create(model="text-embedding-3-large", input=text)
9
10print(f"text-embedding-3-small  → {len(small.data[0].embedding)} dims")
11print(f"text-embedding-3-large  → {len(large.data[0].embedding)} dims")

For local deployment without API costs, all-MiniLM-L6-v2 from Sentence Transformers is a common baseline that demonstrates how a smaller model still captures meaning through cosine similarity:

Code snippetpython
1from sentence_transformers import SentenceTransformer
2import numpy as np
3
4model = SentenceTransformer("all-MiniLM-L6-v2")
5texts = [
6    "Semantic search finds documents by meaning.",
7    "Vector similarity measures embedding closeness.",
8    "SQL stores tabular data in rows and columns.",
9]
10vecs = model.encode(texts)
11
12def cosine(a, b):
13    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
14
15print(f"Dimensions: {vecs.shape[1]}")
16print(f"Related pair similarity:   {cosine(vecs[0], vecs[1]):.4f}")
17print(f"Unrelated pair similarity: {cosine(vecs[0], vecs[2]):.4f}")

The related pair should score noticeably higher than the unrelated pair, confirming the model captures meaning rather than surface overlap.

Verify by running both snippets and confirming that text-embedding-3-small produces 1536-dimensional vectors, text-embedding-3-large produces 3072-dimensional vectors, and the related-pair cosine similarity in the local-model block exceeds the unrelated-pair score by a clear margin.

Do's and Don'ts

Do's

1. ✓Do benchmark candidate models against MTEB retrieval scores before committing to one — dimension count alone is not a quality proxy; a model with fewer dimensions can outperform a larger one on your specific task (retrieval, clustering, or semantic similarity), and MTEB subtask scores expose that gap before it becomes a production problem.
2. ✓Do verify that your documents fit within a model's token limit — all-MiniLM-L6-v2 caps at 256 tokens and silently truncates anything longer, while voyage-3 supports up to 32K; choosing the wrong limit corrupts vectors for long documents without raising an error, and the fix (re-embedding everything) is expensive at scale.
3. ✓Do embed the same text with both candidate models and print the output dimensions, then run a cosine similarity check on related versus unrelated pairs — confirming that text-embedding-3-small yields 1536 dims, text-embedding-3-large yields 3072, and the related pair scores noticeably above the unrelated pair makes the quality and cost tradeoff concrete rather than theoretical.

Don'ts

1. ✗Don't reach for text-embedding-3-large by default without projecting million-document cost — at roughly 10× the per-token price of text-embedding-3-small, the extra 1536 dimensions may not justify the expense; run both on a representative sample and measure retrieval quality before locking in the larger model.
2. ✗Don't use a general-purpose model on code or legal corpora without evaluating domain-specialized alternatives — Voyage AI's domain-tuned models are specifically benchmarked to outperform generalists like text-embedding-3-small on those text types, so defaulting to a generalist silently leaves retrieval quality on the table even if the model's MTEB overall score looks competitive.
3. ✗Don't skip the cosine similarity sanity check when switching embedding models — if the related-pair score ("Semantic search finds documents by meaning" vs "Vector similarity measures embedding closeness") does not clearly exceed the unrelated-pair score, the model is not capturing meaning and your similarity search will return noise regardless of how many dimensions it produces.