{
  "slug": "embeddings-vector-search",
  "title": "Embeddings and Vector Search for Agents",
  "description": "How to pick an embedding model, understand distance metrics, choose an ANN index type, and operate a vector store reliably in agent retrieval pipelines.",
  "category": "Reference",
  "tags": [
    "embeddings",
    "vector-search",
    "rag",
    "retrieval",
    "ann",
    "agents"
  ],
  "updated": "2026-07-16",
  "premium": false,
  "canonical": "https://changegamer.ai/resources/embeddings-vector-search",
  "markdown": "https://changegamer.ai/resources/embeddings-vector-search.md",
  "outline": [
    {
      "depth": 2,
      "text": "Embedding model dimensions and selection criteria",
      "anchor": "embedding-model-dimensions-and-selection-criteria"
    },
    {
      "depth": 2,
      "text": "Standard benchmark: MTEB / MMTEB",
      "anchor": "standard-benchmark-mteb-mmteb"
    },
    {
      "depth": 2,
      "text": "Notable embedding model families (as of mid-2026)",
      "anchor": "notable-embedding-model-families-as-of-mid-2026"
    },
    {
      "depth": 2,
      "text": "Distance metrics",
      "anchor": "distance-metrics"
    },
    {
      "depth": 2,
      "text": "ANN index types",
      "anchor": "ann-index-types"
    },
    {
      "depth": 2,
      "text": "Practical guidance",
      "anchor": "practical-guidance"
    },
    {
      "depth": 2,
      "text": "Verified sources",
      "anchor": "verified-sources"
    }
  ],
  "related": [
    {
      "slug": "hybrid-search-for-rag",
      "title": "Hybrid Search for RAG: BM25 + Dense Retrieval and Fusion",
      "description": "How to combine lexical (BM25/SPLADE) and dense vector retrieval with Reciprocal Rank Fusion for higher first-stage recall in RAG pipelines — with the RRF formula, a sparse-method comparison table, and verified DB support.",
      "url": "https://changegamer.ai/resources/hybrid-search-for-rag"
    },
    {
      "slug": "reranking-for-rag",
      "title": "Reranking for RAG: Cross-Encoders, LLM Rerankers, and Hosted APIs",
      "description": "Second-stage retrieval step that re-scores bi-encoder candidates with full query-document attention, boosting precision without sacrificing recall; covers cross-encoder, LLM, and late-interaction reranking, hosted APIs, sizing heuristics, and evaluation.",
      "url": "https://changegamer.ai/resources/reranking-for-rag"
    },
    {
      "slug": "chunking-strategies-for-rag",
      "title": "Chunking Strategies for RAG",
      "description": "Practitioner reference for chunking documents before embedding: fixed-size, recursive, semantic, late chunking, and contextual retrieval — with a strategy comparison table, chunk-size and overlap tradeoffs, code/table/Markdown handling, embedding model context limits, and evaluation methods.",
      "url": "https://changegamer.ai/resources/chunking-strategies-for-rag"
    },
    {
      "slug": "rag-retrieval-for-agents",
      "title": "RAG and Retrieval for Agents",
      "description": "End-to-end practitioner reference for Retrieval-Augmented Generation: pipeline stages, chunking strategies, dense/sparse/hybrid retrieval, reranking, agentic retrieval patterns, quality failure modes, and evaluation — with verified sources for every named technique.",
      "url": "https://changegamer.ai/resources/rag-retrieval-for-agents"
    }
  ],
  "furtherReading": [
    {
      "slug": "acp-vs-ap2-vs-x402",
      "title": "ACP vs. AP2 vs. x402: Which Agent Payment Rail Should You Implement?",
      "description": "A decision framework for choosing between ACP, AP2, and x402 (plus the self-hosted 402 gate) — sorted by who your buyer actually is, what you are selling, and what is live versus waitlisted today.",
      "url": "https://changegamer.ai/articles/acp-vs-ap2-vs-x402"
    },
    {
      "slug": "agent-checkout-vs-human-checkout",
      "title": "Agent Checkout vs. Human Checkout: Why Your Payment Flow Fails Machine Buyers",
      "description": "Why checkout built for a person watching a screen is unusable by an AI agent, and what a checkout flow that actually completes for a machine buyer looks like — 402 + API key versus native x402.",
      "url": "https://changegamer.ai/articles/agent-checkout-vs-human-checkout"
    }
  ],
  "body": "Embeddings map text (or other content) to dense numeric vectors so that semantic similarity can be measured by vector distance. This reference covers the layer between raw text and the retrieval results your agent acts on. For the full retrieval pipeline see /resources/rag-retrieval-for-agents; for memory store architecture see /resources/agent-memory-context.\n\n## Embedding model dimensions and selection criteria\n\nKey parameters to evaluate before choosing a model:\n\n- **Dimensionality** — typical range 256–3072. Higher dimensions can capture more nuance but cost more storage and compute. Matryoshka Representation Learning (MRL, Kusupati et al., NeurIPS 2022, arXiv:2205.13147) trains embeddings so that truncating to fewer dimensions still yields useful representations — use MRL-trained models to right-size vectors at query time without re-embedding.\n- **Max input tokens** — models differ widely (256–128k tokens). A model that silently truncates long documents will produce misleading embeddings.\n- **Language coverage** — English-only models underperform on multilingual corpora. Verify coverage for your target languages.\n- **Domain fit** — general-purpose MTEB scores may not reflect performance on code, legal, medical, or financial text. Benchmark on your own data.\n- **Cost / quality / latency tradeoff** — API-hosted models charge per token; open-weight models traded compute for zero marginal cost. Quantized open-weight models reduce memory at the cost of some recall.\n\n## Standard benchmark: MTEB / MMTEB\n\nThe **Massive Text Embedding Benchmark (MTEB)** — maintained by the embeddings-benchmark organization on HuggingFace (huggingface.co/spaces/mteb/leaderboard) — is the standard leaderboard for comparing embedding models across retrieval, classification, clustering, semantic similarity, and other tasks. It covers 112+ languages and 5,000+ submissions.\n\nMMTEB (Massive Multilingual Text Embedding Benchmark, arXiv:2502.13595, ICLR 2025) is a community-driven expansion covering 500+ tasks across 250+ languages, hosted on HuggingFace alongside MTEB. Use MMTEB scores when multilingual recall matters.\n\nDo not treat leaderboard rankings as permanent. Models update frequently; always check the current leaderboard and run domain-specific recall tests on your data.\n\n## Notable embedding model families (as of mid-2026)\n\nListed as available/notable — rankings shift; verify current MTEB position before choosing:\n\n**API / hosted:**\n- **OpenAI text-embedding-3-small / text-embedding-3-large** — 1536 and 3072 default dimensions; both support MRL-style dimension reduction via the `dimensions` parameter (openai.com).\n- **Cohere Embed v4** — multimodal (text + images), 128k token context, supports MRL dimensions (256/512/1024/1536), 100+ languages (cohere.com).\n- **Voyage AI** — the **voyage-4 family** (voyage-4-large, a MoE architecture; voyage-4; voyage-4-lite; open-weight voyage-4-nano) reportedly launched January 2026, superseding voyage-3.5 as the flagship: all four sizes share one embedding space (index with a larger tier, query with a smaller one, no re-embedding needed), MRL dimensions (256/512/1024/2048), 32K-token context (WebSearch-corroborated via blog.voyageai.com; not independently re-fetched this cycle, see Verified sources). voyage-3.5 / voyage-3.5-lite remain available as the prior generation (voyageai.com).\n- **Google Gemini Embedding (gemini-embedding-001)** — up to 3072 dimensions, MRL-truncatable, 100+ languages; text-embedding-004 deprecated as of Jan 2026 (ai.google.dev). Reportedly joined in public preview (March 2026) by **Gemini Embedding 2**, Google's first natively multimodal embedding model — text, images, video, audio, and PDF pages into one shared 3,072-dimensional space (WebSearch-corroborated via blog.google; not independently re-fetched this cycle, see Verified sources).\n\n**Open-weight:**\n- **BGE / BAAI (e.g., BGE-M3)** — multi-functionality (dense, sparse, multi-vector), multilingual, widely deployed, top open-weight download counts on HuggingFace (huggingface.co/BAAI).\n- **E5 / multilingual-E5** (Microsoft) — strong multilingual retrieval; multiple sizes available (huggingface.co/intfloat).\n- **Nomic Embed** — long-context, Apache 2.0, reproducibly trained (nomic.ai; arxiv:2402.01613).\n- **Jina Embeddings** — extended context; self-hostable (jina.ai).\n- **Qwen3 Embedding** (Alibaba/QwenLM) — 0.6B/4B/8B sizes; Apache 2.0; ranked highly on MTEB multilingual as of June 2025 (github.com/QwenLM).\n\nCross-reference open-weight model infrastructure at /resources/open-weight-models-for-agents.\n\n## Distance metrics\n\n| Metric | Formula basis | Best for |\n|--------|--------------|---------|\n| **Cosine similarity** | Angle between vectors | Default for most retrieval; normalizes magnitude |\n| **Dot product** | Magnitude × angle | Equivalent to cosine when vectors are unit-normalized; faster |\n| **Euclidean (L2)** | Absolute distance | Useful when magnitude carries information (e.g., sparse embeddings) |\n\n**Normalization caveat:** most retrieval models produce unit-normalized vectors by default, making cosine and dot product equivalent. If you normalize, dot product is cheaper to compute. Verify your model's output normalization before choosing.\n\n## ANN index types\n\nApproximate Nearest Neighbor (ANN) indexes trade exact recall for speed. The main types:\n\n- **Flat (brute-force)** — exact exhaustive scan; 100% recall; scales linearly with corpus size. Use only for small corpora (<100k vectors) or as a recall baseline.\n- **IVF (Inverted File Index)** — k-means clusters vectors at build time; at query time only nearby clusters are searched (multi-probe tunable). Faster build than HNSW; lower memory; requires a training step. Implemented in FAISS (github.com/facebookresearch/faiss) and pgvector (IVFFlat).\n- **HNSW (Hierarchical Navigable Small World graphs)** — multi-layer proximity graph; logarithmic search complexity; superior speed-recall tradeoff at query time; slower to build and more memory than IVF; no training step required. See Malkov & Yashunin, arXiv:1603.09320. Implemented in FAISS and pgvector.\n- **Product Quantization (PQ) / Scalar Quantization (SQ)** — compression schemes that reduce vector storage by encoding sub-vectors with a codebook. Typically combined with IVF (IVFPQ) or HNSW. Reduces memory 4–32× at the cost of some recall. Use when corpus exceeds available RAM.\n\n## Practical guidance\n\n- **Match query and document models.** Embeddings from different models occupy different vector spaces — they are not comparable. Always embed queries with the same model used to index documents.\n- **Re-embed on model change.** Embeddings are not portable across model versions. Upgrading an embedding model requires re-indexing the full corpus.\n- **Normalize before cosine search.** If your vector store does not auto-normalize, do it at index and query time to avoid misleading distance scores.\n- **Use quantization at scale.** PQ or SQ cuts memory 4–32× with modest recall loss. Benchmark recall on your own data before and after.\n- **Benchmark recall on your own data.** Leaderboard scores are averages. Your domain, query style, and document length distribution all affect real-world recall.\n- **Start with HNSW for online search; use IVFPQ for large memory-constrained corpora.**\n\n## Verified sources\n\n- MTEB leaderboard (HuggingFace): https://huggingface.co/spaces/mteb/leaderboard\n- MMTEB paper (arXiv:2502.13595): https://arxiv.org/abs/2502.13595\n- Matryoshka Representation Learning (Kusupati et al., NeurIPS 2022, arXiv:2205.13147): https://arxiv.org/abs/2205.13147\n- HNSW paper (Malkov & Yashunin, arXiv:1603.09320): https://arxiv.org/abs/1603.09320\n- OpenAI embedding models (openai.com): https://openai.com/index/new-embedding-models-and-api-updates/\n- Cohere Embed v4 announcement: https://cohere.com/blog/embed-4\n- Voyage AI embedding docs: https://docs.voyageai.com/docs/embeddings\n- Google Gemini Embedding GA (Google Developers Blog): https://developers.googleblog.com/gemini-embedding-available-gemini-api/\n- Voyage AI — announcing the voyage-4 model family (Jan 2026): https://blog.voyageai.com/2026/01/15/voyage-4/\n- Google — Gemini Embedding 2, first natively multimodal embedding model (Mar 2026): https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-embedding-2/\n- Qwen3 Embedding (QwenLM GitHub): https://github.com/QwenLM/Qwen3-Embedding\n- Nomic Embed paper (arXiv:2402.01613): https://arxiv.org/abs/2402.01613\n- FAISS (Meta AI): https://github.com/facebookresearch/faiss\n- pgvector (PostgreSQL vector extension): https://github.com/pgvector/pgvector",
  "sources": [
    "https://huggingface.co/spaces/mteb/leaderboard",
    "https://arxiv.org/abs/2502.13595",
    "https://arxiv.org/abs/2205.13147",
    "https://arxiv.org/abs/1603.09320",
    "https://openai.com/index/new-embedding-models-and-api-updates/",
    "https://cohere.com/blog/embed-4",
    "https://docs.voyageai.com/docs/embeddings",
    "https://developers.googleblog.com/gemini-embedding-available-gemini-api/",
    "https://blog.voyageai.com/2026/01/15/voyage-4/",
    "https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-embedding-2/",
    "https://github.com/QwenLM/Qwen3-Embedding",
    "https://arxiv.org/abs/2402.01613",
    "https://github.com/facebookresearch/faiss",
    "https://github.com/pgvector/pgvector"
  ]
}