# OCR vs Vision Language Model for Document Parsing: How to Choose Stage A

> OCR vs vision language model for document parsing: a decision order keyed on residency, layout variance, volume and known schema, how a hybrid arbitrates disagreements, a per-field bake-off, failure-mode mapping, page batching and privacy.

Guide: Document extraction pipelines in production — part 1
Published: 2026-10-25 · Updated: 2026-10-25 · 1611 words · ~2143 tokens (estimate)
Canonical: https://changegamer.ai/articles/ocr-vs-vision-model-vs-hybrid-document-parsing
JSON: https://changegamer.ai/api/articles/ocr-vs-vision-model-vs-hybrid-document-parsing.json
Access: free — editorial guide, not part of the licensed corpus
Pillar: https://changegamer.ai/articles/ai-document-extraction-pipeline-in-production.md

## In short

- Choose the Stage A document parser by working through data residency first, then layout variance, then volume, then whether the schema is known, because residency can rule out an option before accuracy matters (the author's inference).
- A hybrid document parser should let OCR win on exact characters such as digits and identifiers and let the vision model win on structure such as which column a value belongs to (the author's inference).
- A parser bake-off should score each field separately across candidates, splitting correct, missing and wrong values, because a page-level score hides one parser failing on the single field that moves money.
- Sending page images to a hosted vision model transfers whole pages, not selected fields, so data minimization is harder than with text; the Data Privacy reference lists provider retention as 30 days at OpenAI (undated in the reference) and 7 days at Anthropic (as of September 2025).
- As of October 2026 ChangeGamer runs no document-extraction pipeline and has no accuracy, throughput or cost figures, so every threshold in this article is a method to measure on your own documents.

---

Choose the Stage A parser by constraint order: data residency first, layout variance second, volume third and a known schema fourth, then confirm the choice with a per-field bake-off on your own documents. The [AI document extraction pipeline](/articles/ai-document-extraction-pipeline-in-production) pillar explains why parsing and extraction are separate stages; this article is the procedure for picking and running the parser. Where a sentence cites a reference, the claim comes from it; every other rule is the author's inference.

## Which Stage A parser should you start with?

Start with the first constraint that removes an option, and work down this order. The ordering is inference; the facts behind each step come from the [Document Extraction and Parsing for Agents](/resources/document-extraction-for-agents) reference (updated 2026-09-28).

1. **Can the documents leave your infrastructure?** If not, hosted vision models are out and the choice is among self-hosted OCR, layout tooling or an open-weight vision model. The Multimodal Agents reference (updated 2026-10-04) lists open-weight families such as Qwen3-VL and InternVL3.
2. **Is layout variance high?** Unpredictable layouts favour a vision model or a hybrid, because the extraction reference says vision models handle layout ambiguity that rule-based pipelines miss. Stable templates favour OCR.
3. **How large is the volume?** The reference puts OCR plus layout analysis on high-volume work and vision models on low to medium volume, because vision input costs more per page.
4. **Is the schema known up front?** The reference lists a known schema as a case where a vision model fits. Without one, produce Markdown first and decide later.

If two options survive, test both; the corpus publishes no accuracy ranking for any parser.

## What does a hybrid parser look like in operation?

A hybrid runs OCR and a vision model on the same page and compares what each returns for every field. The Multimodal Agents reference recommends feeding the OCR text alongside the page image for financial, medical or legal documents, so the model reasons over both with the OCR text as an anchor. The comparison step below is inference.

| Situation | Winner | Route |
|---|---|---|
| Both readers return the same value | Either | Accept, then run validation |
| Values differ in characters (digits, IDs, dates) | OCR text | Accept only if validation passes, else review |
| Vision value does not appear in the OCR text | Neither | Flag as ungrounded and send to review |
| Readers disagree on which cell or column owns a value | Vision model | Verify against the table source |
| OCR returns nothing for a region the vision model reads | Vision model | Review, since the region may be a stamp or handwriting |

### Why should OCR win on characters?

OCR should win on characters because the multimodal reference lists hallucinated visual details as a known vision-model failure, naming an invented dollar amount or part number as a real production failure. A character an OCR engine never produced is evidence of that. As inference, the reverse also holds: OCR is weaker at deciding what a number belongs to, which is why layout questions go to the vision model.

### How do you route a disagreement?

Log each disagreement as a reason code and send the field, not the document, to the review path in the pillar. The reviewer needs both candidate values, the page image and the Stage A text around the value.

## How do you run a parser bake-off with per-field metrics?

Run the bake-off by hand-labelling target fields once, running every candidate over the sample, and scoring each field on its own. Page-level scores hide the failure that matters: a parser can read 95 percent of a page perfectly and still miss the total.

- **Three outcomes per field.** Correct, missing and wrong. A missing field can be routed to review cheaply; a wrong one is silent.
- **Slice by document type and hard case.** One average across templates and rotated scans tells you nothing about either.
- **Weight by consequence.** Treat fields that move money or identity as separate pass criteria.
- **Cost per accepted field.** Managed OCR is billed per page and vision input in tokens; the multimodal reference says token formulas differ by provider and model version. The only per-page prices in the corpus are Mistral's, reported via secondary sources ($4 per 1,000 pages for OCR 4 standard, $2 batch, and $5 for Mistral Document AI, not Google's), so verify vendor pages before budgeting; [cost per page for document extraction](/articles/cost-per-page-for-document-extraction) gives a blank sensitivity table to fill from your pilot.

The script below scores two invented candidates on two invented records; replace the dictionaries with your labelled sample.

```bash
python3 - <<'EOF'
from collections import defaultdict
gold = [{"total": "120.50", "iban": "DE89 3704"}, {"total": "88.00", "iban": "FR14 2004"}]
runs = {
  "ocr": [{"total": "120.50", "iban": "DE89 3704"}, {"total": "88.00", "iban": None}],
  "vlm": [{"total": "120.50", "iban": "DE89 3704"}, {"total": "80.00", "iban": "FR14 2004"}],
}
for name, preds in runs.items():
    s = defaultdict(lambda: {"correct": 0, "missing": 0, "wrong": 0})
    for g, p in zip(gold, preds):
        for field, value in g.items():
            got = p[field]
            s[field]["correct" if got == value else "missing" if got is None else "wrong"] += 1
    print(name, dict(s))
EOF
```

It prints one line per candidate: OCR leaves one IBAN missing, while the vision model gets one total wrong.

## Which failure mode points to which parser?

Map each observed failure to the parser change most likely to fix it, then confirm on the bake-off sample. The failure descriptions come from the extraction and multimodal references; the "Seen in" and "Try" columns, and the blank-region row, are inference.

| Observed failure | Seen in | Try |
|---|---|---|
| Wrong reading order on multi-column pages | OCR pipelines | Vision model, or a layout-aware library |
| Rotated or skewed scans misread | OCR pipelines | Deskew first, or a vision model |
| Merged or spanning table cells misaligned | Both | Check against source; hybrid with row-count checks |
| Small or low-contrast text mis-read | Vision models | OCR pass at full resolution, hybrid |
| Invented values | Vision models | OCR anchor plus a grounding check |
| Blank output on a region (inference) | OCR pipelines | Vision model on that region |

Changing the parser fixes only Stage A failures. If values are present in the Stage A text but missing from the record, the cause is the schema or extractor; see [structured outputs and JSON mode](/resources/structured-outputs-and-json-mode).

## How many pages should one vision-model call take?

Size the call to the smallest span that contains all the evidence for a field, usually one page, and add the neighbouring page when a table or a total can cross a page break. This is extraction-unit sizing, not retrieval chunking; split strategy for search is in [chunking documents for RAG](/articles/chunking-documents-for-rag).

- **Single-page documents:** one call per document.
- **Short multi-page documents:** one call per page, plus a reconciliation step for fields that can appear on any page. The reconciliation rule is inference.
- **Tables across pages:** send the last rows of one page with the next page, then deduplicate.
- **Hard ceilings:** per the multimodal reference, Anthropic accepts up to 100 images per request for 200k-context models and 600 for others, and applies a 2000 px long-edge limit above 20 images.
- **Resolution:** downscale to the lowest resolution that still reads the smallest text, the reference's main cost lever. Test it on small-print fields, where the reference says vision models fail.

## What changes when page images go to a provider?

Sending a page image to a hosted model transfers everything printed on the page, so you cannot send only the fields the task needs. The [Data Privacy and PII for Agents](/resources/data-privacy-for-agents) reference (updated 2026-09-28) says to send only the data a task requires and to redact before the model call; with a raw image there is nothing to redact yet. The compensating pattern, as inference, is to run self-hosted OCR first, redact the text, and send the redacted text or cropped regions.

| Question | Hosted vision model | Self-hosted OCR |
|---|---|---|
| Where do the bytes go? | To the provider | Stay in your infrastructure |
| Retention | OpenAI up to 30 days (undated in the reference), Anthropic 7 days (as of September 2025 per the reference), so re-check before relying on them; OpenAI zero retention needs approval for eligible endpoints | Yours to set |
| Contract | A signed DPA is required under GDPR Article 28, per the reference; take legal advice | No processor for the parse step |
| Operations | Provider's problem | You run and update the models |

For self-hosting, the extraction reference lists Docling (MIT licence) among open-source parsers re-confirmed reachable on 2026-09-28. Check model-weight licences separately: Marker and Surya weights carry revenue thresholds. Mistral reportedly offers a self-hosted OCR 4 container, but that is unverified in the reference, so confirm it with the vendor. This is not legal advice.

## What does ChangeGamer know first-hand about parsing?

As of October 2026 ChangeGamer runs no document-extraction pipeline and has no accuracy, throughput or cost figures, so nothing above is measured on its own documents. One verifiable fact is adjacent: its `.md` article variants are not produced by a parser. In `src/pages/articles/[slug].md.ts` the comment states the body is "emitted verbatim from the same source the HTML page renders", and `/llms.txt` links those `.md` URLs. The site avoids Stage A by authoring Markdown at the source. Teams that control the origin format can do the same, so parse only the documents you do not author.

## Frequently asked questions

### Is OCR or a vision language model better for document parsing?

No parser wins in general. The Document Extraction reference puts OCR plus layout analysis at high volume and vision models at low to medium volume with complex layouts, and publishes no cross-parser accuracy ranking, so run both on a labelled sample of your own documents and compare field by field.

### What is a hybrid OCR and vision model parser?

A hybrid parser runs a dedicated OCR pass and gives that text to a vision model alongside the page image. The Multimodal Agents reference recommends this for precision-critical fields, with the OCR text acting as a ground-truth anchor and the image supplying layout context.

### How do you compare document parsers fairly?

Hand-label the target fields once on a representative sample that includes your worst scans, run every candidate over it, and score each field separately as correct, missing or wrong. Compare per-field results by document type rather than a single page-level accuracy number.

### Is it safe to send document page images to an AI provider?

Sending page images to a provider is safe only if the data and contract allow it. A page image carries everything printed on the page, so apply data minimization, sign a data processing addendum, and check the provider's retention terms. Where documents cannot leave your infrastructure, parse with self-hosted OCR instead.


---

## The rest of this guide

- [AI Document Extraction in Production: Parse, Extract, Validate, Review](https://changegamer.ai/articles/ai-document-extraction-pipeline-in-production.md): How to run AI document extraction as a production pipeline: separate parsing from schema-constrained field extraction, choose OCR, a vision model or a hybrid, validate values, route doubtful ones to a human, version the schema, and work out cost per page.
- [Schema-Constrained Field Extraction From Documents: Settings and Failure Branches](https://changegamer.ai/articles/schema-constrained-field-extraction-from-documents.md): Schema constrained field extraction from documents: nullable fields, enums with an escape value, evidence fields, a branch table for refusal, truncation, parse failure and empty output, a one-retry policy, and field confidence as inference.
- [Table and Form Validation for Document Extraction: Row Counts, Totals and Required Fields](https://changegamer.ai/articles/table-and-form-validation-for-document-extraction.md): Validating table and form extraction from documents: rows split across pages, subtotal rows, exact money reconciliation, column sums, required-field maps per form type, cross-field checks and a failed-check-to-route table.
- [Human Review Fallback for Document Extraction: Triggers, Capacity and the Correction Loop](https://changegamer.ai/articles/human-review-fallback-for-document-extraction.md): Designing the human review fallback for document extraction: a routing-trigger table, cross-checking high-stakes fields, review capacity and backlog policy, the review item record and a correction loop that feeds regression sets.
- [Versioning a Document Extraction Schema: Migrations, Manifests and Rollback](https://changegamer.ai/articles/extraction-schema-versioning-for-document-pipelines.md): How to version a document extraction schema: which edits break stored records, absent versus null fields, lazy upcast versus backfill, reprocessing from stored parse output, a version manifest, dual-read cutover, correction migration and rollback.
- [Document Extraction Cost per Page: Tokens, Retries, Review and Budget Alarms](https://changegamer.ai/articles/cost-per-page-for-document-extraction.md): How to model document extraction cost per page beyond the parse fee: page-count mix versus per-page pricing, image tokens versus per-call overhead, retries by failure class, review share, batch versus sync, and a budget kill switch on cost per accepted document.
- [Open Source vs Managed Document Parsing: A Decision Order, Operating Burden and Exit Plan](https://changegamer.ai/articles/open-source-vs-managed-document-parsing.md): How to decide between self-hosted open-source parsers and a managed parsing service: residency and licence first, volume and operations capacity second, reversibility third, plus the cost of operating models, lock-in, a single-interface fallback design and a trial protocol on your own documents.
- [Writing Extracted Document Data to a System of Record: Matching, Keys, States and Replay](https://changegamer.ai/articles/feeding-extraction-output-into-a-system-of-record.md): How to write extracted document data to a system of record: matching an extracted record to an existing one, retry keys versus business keys, a staged-to-committed state machine, per-record versus per-document atomicity, dead-letter queues, replay from the saved parse artifact and the write-boundary log.
- [Multimodal Document Understanding for Extraction: Charts, Diagrams, Photos, Stamps and Video Frames](https://changegamer.ai/articles/multimodal-document-understanding-for-extraction-pipelines.md): How to handle non-text visuals in a document extraction pipeline: when OCR is not enough, extracting a value versus a description versus a class, region-level routing, figure-aware page packing, poor scans, video frames, image-borne injection and per-visual evaluation.
- [Monitoring Document Extraction Accuracy in Production: Audit Samples, Field Metrics and Drift](https://changegamer.ai/articles/monitoring-document-extraction-accuracy-in-production.md): How to monitor document extraction accuracy in production: a stratified audit sample with independent labels, per-field metrics with intervals, extraction-specific drift triggers, thresholds from your own variance, and regression intake.
- [How to Split Long Documents for Extraction: Batches, Overlap and Reassembly](https://changegamer.ai/articles/splitting-long-documents-for-document-extraction.md): How to split long documents for extraction: size batches from measured tokens per page, choose section or page windows, overlap pages, carry table headers, reconcile conflicts across batches and reassemble with provenance.

## Reference resources

- https://changegamer.ai/resources/document-extraction-for-agents.md
- https://changegamer.ai/resources/multimodal-agents.md
- https://changegamer.ai/resources/structured-outputs-and-json-mode.md
- https://changegamer.ai/resources/data-privacy-for-agents.md

All guides: https://changegamer.ai/api/articles.json · Reference corpus: https://changegamer.ai/llms.txt
Licensing: https://changegamer.ai/api/pricing.json (offer catalog) · https://changegamer.ai/api/payment.json (payment methods, HTTP 402 flow) · access guide: https://changegamer.ai/resources/access-and-pricing.md
