{
  "slug": "chunking-strategies-for-rag",
  "title": "Chunking Strategies for RAG",
  "description": "Practitioner reference for chunking documents before embedding: fixed-size, recursive, semantic, late chunking, and contextual retrieval — with a strategy comparison table, chunk-size and overlap tradeoffs, code/table/Markdown handling, embedding model context limits, and evaluation methods.",
  "category": "Guide",
  "tags": [
    "rag",
    "chunking",
    "embeddings",
    "retrieval",
    "langchain",
    "llamaindex",
    "agents",
    "documents"
  ],
  "updated": "2026-07-15",
  "canonical": "https://changegamer.ai/resources/chunking-strategies-for-rag",
  "markdown": "https://changegamer.ai/resources/chunking-strategies-for-rag.md",
  "outline": [
    {
      "depth": 2,
      "text": "What strategy should I use?",
      "anchor": "what-strategy-should-i-use"
    },
    {
      "depth": 2,
      "text": "Strategy comparison",
      "anchor": "strategy-comparison"
    },
    {
      "depth": 2,
      "text": "Chunk size and overlap tradeoffs",
      "anchor": "chunk-size-and-overlap-tradeoffs"
    },
    {
      "depth": 2,
      "text": "Chunking for code, tables, and Markdown",
      "anchor": "chunking-for-code-tables-and-markdown"
    },
    {
      "depth": 2,
      "text": "Embedding model context limits and chunking",
      "anchor": "embedding-model-context-limits-and-chunking"
    },
    {
      "depth": 2,
      "text": "Tooling reference",
      "anchor": "tooling-reference"
    },
    {
      "depth": 2,
      "text": "How to evaluate chunking choices",
      "anchor": "how-to-evaluate-chunking-choices"
    },
    {
      "depth": 2,
      "text": "Verified sources",
      "anchor": "verified-sources"
    }
  ],
  "related": [
    {
      "slug": "rag-retrieval-for-agents",
      "title": "RAG and Retrieval for Agents",
      "description": "End-to-end practitioner reference for Retrieval-Augmented Generation: pipeline stages, chunking strategies, dense/sparse/hybrid retrieval, reranking, agentic retrieval patterns, quality failure modes, and evaluation — with verified sources for every named technique.",
      "url": "https://changegamer.ai/resources/rag-retrieval-for-agents"
    },
    {
      "slug": "hybrid-search-for-rag",
      "title": "Hybrid Search for RAG: BM25 + Dense Retrieval and Fusion",
      "description": "How to combine lexical (BM25/SPLADE) and dense vector retrieval with Reciprocal Rank Fusion for higher first-stage recall in RAG pipelines — with the RRF formula, a sparse-method comparison table, and verified DB support.",
      "url": "https://changegamer.ai/resources/hybrid-search-for-rag"
    },
    {
      "slug": "reranking-for-rag",
      "title": "Reranking for RAG: Cross-Encoders, LLM Rerankers, and Hosted APIs",
      "description": "Second-stage retrieval step that re-scores bi-encoder candidates with full query-document attention, boosting precision without sacrificing recall; covers cross-encoder, LLM, and late-interaction reranking, hosted APIs, sizing heuristics, and evaluation.",
      "url": "https://changegamer.ai/resources/reranking-for-rag"
    },
    {
      "slug": "text-to-sql-agents",
      "title": "Text-to-SQL and Database Agents",
      "description": "How agents answer questions over structured data by generating and executing SQL: schema context, few-shot prompting, self-correction, safety constraints, benchmarks (Spider, BIRD-SQL), and tooling (LangChain SQLDatabaseToolkit, LlamaIndex NLSQLTableQueryEngine, Vanna, MCP Postgres server).",
      "url": "https://changegamer.ai/resources/text-to-sql-agents"
    }
  ],
  "body": "Chunking is the decision that determines what the retriever can find. Every other RAG optimisation is downstream of it. This guide covers the five main strategies, the tooling that implements them, and how to choose. For the full retrieval pipeline see /resources/rag-retrieval-for-agents; for embedding model selection see /resources/embeddings-vector-search; for upstream parsing of PDFs and scans see /resources/document-extraction-for-agents.\n\n## What strategy should I use?\n\nStart with **recursive splitting** (LangChain `RecursiveCharacterTextSplitter`, LlamaIndex `SentenceSplitter`). It respects document structure, is fast at ingest time, and matches or beats semantic chunking in many production settings at far lower cost. Switch strategies only when evaluation shows a real gap.\n\n## Strategy comparison\n\n| Strategy | How it works | Best for | Key tradeoff |\n|---|---|---|---|\n| **Fixed-size + overlap** | Split every N tokens; slide an M-token window between adjacent chunks | Uniform, unstructured prose; prototyping | Fast and simple; cuts mid-sentence, mid-concept |\n| **Recursive / structural** | Split on a hierarchy of separators; stay within a size limit | Markdown, HTML, code, structured docs | Needs reliable structure; still character-based, not semantic |\n| **Semantic** | Embed adjacent sentences; split where similarity drops below a percentile threshold | Dense topic-switching text; academic content | High ingest cost; threshold needs dataset-specific tuning |\n| **Document-structure-aware** | Respect element types (Title, Table, ListItem) as boundaries | PDFs, DOCX, HTML with layout metadata | Requires a parsing step to produce element types first |\n| **Late chunking** | Embed the full document with a long-context model; pool token embeddings after splitting | Long documents where local chunks lose context | Needs a long-context embedding model; total tokens must fit its window |\n| **Contextual retrieval** | Prepend a short LLM-generated summary to each chunk before embedding and BM25 indexing | High-value corpora where recall is critical | An LLM call per chunk at ingest; use prompt caching to lower cost |\n\n## Chunk size and overlap tradeoffs\n\nThere is no universal optimal chunk size — it depends on your query distribution and embedding model. Practical guidance:\n\n- **Too small** (< ~100 tokens): high precision on narrow facts, but chunks lose the surrounding context that makes them meaningful; expect low recall on broader questions.\n- **Too large** (> ~600 tokens): more context per chunk, but the relevant sentence is diluted by noise and the embedding drifts toward topic rather than specific fact.\n- **Typical starting points**: 256–512 tokens with 10–20% overlap. Calibrate by measuring precision@k and recall@k on a held-out query set.\n- **Overlap** prevents facts that straddle chunk boundaries from disappearing, but it does not rescue poor boundary placement — it is not a substitute for a better splitting strategy.\n\n## Chunking for code, tables, and Markdown\n\n**Code** — split on syntactic boundaries (functions, classes), not character count. LangChain `Language` splitters use tree-sitter grammars; LlamaIndex `CodeSplitter` exposes `chunk_lines` and `chunk_lines_overlap`.\n\n**Tables** — never split a table row across chunks; a partial row has no retrievable meaning. unstructured.io isolates `Table` elements as atomic chunks. For retrieval, convert tables to Markdown text or generate a natural-language summary alongside the raw table.\n\n**Markdown / HTML** — use structural splitters that split on headings first. LangChain `MarkdownHeaderTextSplitter` splits on heading levels and preserves header metadata per chunk; LlamaIndex `MarkdownNodeParser` does the same. Header-as-metadata improves retrieval when queries reference section names.\n\n## Embedding model context limits and chunking\n\nAn embedding model silently truncates text that exceeds its context window, producing misleading embeddings. Known limits: OpenAI `text-embedding-3-large`/`-small` accept 8,191 tokens; `jina-embeddings-v4` (current generation, released June 2025) natively supports up to 32,768 tokens, though Jina's hosted Embedding API currently caps input at 8,192 tokens regardless of model — check the live API docs before assuming the full native window is usable; the prior `jina-embeddings-v3` accepts 8,192 tokens. Rule of thumb: keep chunks at or below ~75% of the embedding model's context limit to leave headroom and avoid edge-case truncation.\n\n## Tooling reference\n\n**LangChain** (`langchain-text-splitters`): `RecursiveCharacterTextSplitter` (configure `chunk_size`, `chunk_overlap`), `MarkdownHeaderTextSplitter` (heading-aware, returns metadata), and `Language` splitters (tree-sitter, code-aware).\n\n**LlamaIndex** (`llama-index-core`): `SentenceSplitter` (sentence-boundary-aware; `chunk_size`/`chunk_overlap`), `SemanticSplitterNodeParser` (`buffer_size`, `breakpoint_percentile_threshold`), and `CodeSplitter` (tree-sitter; `language`, `chunk_lines`).\n\n**unstructured.io** (`unstructured` OSS): `partition_*` produces typed elements; `chunk_by_title` closes a chunk at each new section title (never merges across sections; `max_characters`, `new_after_n_chars`); `Table` elements stay atomic. Token-based chunking via `max_tokens`.\n\n## How to evaluate chunking choices\n\nChunking quality is best measured indirectly through retrieval metrics on your actual query distribution, not a generic benchmark.\n\n1. **Build a question set** — for each source document, generate 5–10 questions whose answers live in specific passages (LLM-generated or real user queries).\n2. **Measure retrieval precision@k and recall@k** — how many top-k retrieved chunks are relevant, and how many relevant chunks are in the top-k? Instrument with RAGAS (github.com/explodinggradients/ragas, arXiv:2309.15217).\n3. **A/B compare strategies** — hold retrieval and generation constant; vary only the chunking strategy and chunk size to isolate the effect.\n4. **Check for boundary failures** — sample retrieved chunks; chunks starting/ending mid-sentence, mid-table-row, or mid-code-block indicate a poor splitter.\n\n## Verified sources\n\n- LangChain RecursiveCharacterTextSplitter docs: https://python.langchain.com/docs/how_to/recursive_text_splitter/\n- LangChain MarkdownHeaderTextSplitter docs: https://python.langchain.com/docs/how_to/markdown_header_metadata_splitter/\n- LlamaIndex SentenceSplitter API reference: https://docs.llamaindex.ai/en/stable/api_reference/node_parsers/sentence_splitter/\n- LlamaIndex semantic chunking example: https://developers.llamaindex.ai/python/examples/node_parsers/semantic_chunking/\n- unstructured.io chunking docs: https://docs.unstructured.io/open-source/core-functionality/chunking\n- Late Chunking paper (Jina AI / Weaviate, arXiv:2409.04701): https://arxiv.org/abs/2409.04701\n- Contextual Retrieval (Anthropic, 2024): https://www.anthropic.com/engineering/contextual-retrieval\n- OpenAI text-embedding-3-large model page: https://platform.openai.com/docs/models/text-embedding-3-large\n- Jina Embeddings v4 announcement: https://jina.ai/news/jina-embeddings-v4-universal-embeddings-for-multimodal-multilingual-retrieval/\n- jina-embeddings-v4 model page: https://jina.ai/models/jina-embeddings-v4/\n- RAGAS framework (GitHub): https://github.com/explodinggradients/ragas\n- RAGAS paper (arXiv:2309.15217): https://arxiv.org/abs/2309.15217",
  "sources": [
    "https://python.langchain.com/docs/how_to/recursive_text_splitter/",
    "https://python.langchain.com/docs/how_to/markdown_header_metadata_splitter/",
    "https://docs.llamaindex.ai/en/stable/api_reference/node_parsers/sentence_splitter/",
    "https://developers.llamaindex.ai/python/examples/node_parsers/semantic_chunking/",
    "https://docs.unstructured.io/open-source/core-functionality/chunking",
    "https://arxiv.org/abs/2409.04701",
    "https://www.anthropic.com/engineering/contextual-retrieval",
    "https://platform.openai.com/docs/models/text-embedding-3-large",
    "https://jina.ai/news/jina-embeddings-v4-universal-embeddings-for-multimodal-multilingual-retrieval/",
    "https://jina.ai/models/jina-embeddings-v4/",
    "https://github.com/explodinggradients/ragas",
    "https://arxiv.org/abs/2309.15217"
  ]
}