Embedding Models

The embedding model is the most consequential choice in a retrieval system and the one teams give the least thought to. It defines the geometry your whole pipeline searches in: if two relevant pieces of text don't land near each other in this model's vector space, no amount of clever reranking or chunking can recover them. Retrieval quality is capped by your embedding model.

What Makes Embedding Models Differ

• Dimensionality — 384 to 4096+. Bigger isn't strictly better; it's a storage and speed tradeoff. A 3072-dim vector costs 8x the memory and search time of a 384-dim one. Many modern models support Matryoshka embeddings, where you can truncate to a shorter prefix and keep most of the quality — pick the dimension at deploy time.
• Max sequence length — how much text fits in one embedding. Short-context models (512 tokens) force aggressive chunking; long-context embedders (8k+) let you embed whole sections.
• Symmetric vs asymmetric — symmetric models embed query and document the same way (good for similarity/clustering); asymmetric models are trained for short-query-to-long-document retrieval (what most RAG needs). Many require a prefix ("query:" / "passage:") — forgetting it quietly tanks recall.
• Domain — a model trained on web text may be mediocre on legal, medical, or code. Code and multilingual have specialized models for a reason.

Choosing One

1. Start from a benchmark, then distrust it. MTEB is the standard leaderboard, but it's heavily gamed and may not reflect your domain. Use it to build a shortlist, not to decide.
2. Evaluate on your own data. Build a small set of query→relevant-doc pairs from real usage and measure recall@k and NDCG for each candidate. This is the only number that matters, and it routinely disagrees with the leaderboard.
3. API vs self-hosted. API models (OpenAI, Cohere, Voyage) are zero-ops and strong but cost per call and send data out. Open models (BGE, E5, GTE, Nomic, Jina) run on your hardware, are free at the margin, and keep data in — at the cost of running infrastructure.
4. Mind the lock-in. Switching embedding models means re-embedding your entire corpus and rebuilding the index. Treat the choice as semi-permanent and budget for migrations.

Fine-Tuning Embeddings

When an off-the-shelf model underperforms on your domain, fine-tuning the embedder often beats every downstream fix. It's cheaper and faster than fine-tuning an LLM and can move recall more than a reranker.

• Data — pairs of (query, relevant passage), ideally with hard negatives: passages that look similar but are wrong. Hard negatives are what teach the model your domain's fine distinctions; random negatives barely help.
• Method — contrastive learning (pull positives together, push negatives apart) via a loss like InfoNCE/MultipleNegativesRanking. Frameworks like Sentence-Transformers make this a short script.
• Mining negatives — use the base model to retrieve top-k for each query; the wrong ones near the top are your hard negatives. This bootstrap is most of the value.
• Distilling from a reranker / LLM — let a cross-encoder or LLM label relevance, then train the embedder on those labels. You get reranker-level signal at bi-encoder cost.

What to Watch For

Whatever model you pick, you must use the same model and the same prefixes to embed both your corpus and your live queries — a mismatch silently destroys recall and is a classic production bug. And before fine-tuning, exhaust the cheaper wins: correct prefixes, right model family for your domain, decent chunking, and a reranker. Fine-tune when you've confirmed the base model's geometry is the ceiling, not your pipeline around it.