Home / AI & GenAI

AI & GenAICheat sheetIntermediate

Advisory: choosing document parsing for RAG - text extraction, layout models, OCR services or vision LLMs

Poor parsing silently caps RAG quality: tables flattened, headings lost, scanned pages empty. Classify your documents, pick the cheapest parser that preserves the structure each type needs, keep structure as Markdown or JSON for chunking, and measure parsing quality on a sample before indexing everything.

Deepak Bhardwaj2 min read

Applies toRAG ingestion pipelines; Azure AI Document Intelligence, Amazon Textract, Google Document AI, open-source parsers (e.g. PDF text extractors, Unstructured-style libraries, Docling-style converters), vision-capable LLMs

Document types and needs

Document type Needs to preserve Parser approach
Born-digital PDFs, Word, HTML Headings, lists, paragraphs Text extraction with structure (to Markdown)
Documents with important tables (price lists, specs) Table rows/columns Layout model with table extraction
Scanned PDFs, photos Text at all OCR (cloud service or open-source)
Forms Key-value pairs Form/layout models
Slides Slide titles, bullet hierarchy, speaker notes Office parsers
Diagrams, charts, screenshots Meaning of visuals Vision LLM descriptions (costly; selective)

Options

Option Strengths Watch
Open-source text extraction Free, fast Weak on tables, no OCR
Open-source layout/conversion tools Structure to Markdown/JSON, tables, runs locally Needs compute; quality varies by layout
Cloud document AI (layout/OCR) Strong OCR and tables, many languages, managed Per-page cost; data residency
Vision LLMs Understand complex pages and images Highest cost; possible hallucinated content

Output format

  • Convert to Markdown (headings, lists, tables) or structured JSON with element types.
  • Keep page numbers and section paths for citations.
  • Chunk by structure (sections/headings) rather than fixed characters (see the chunk size post).

Evaluate on a sample

  1. Pick 30–50 documents across types.
  2. Parse with 2–3 candidate approaches.
  3. Check: missing text, broken tables, wrong reading order, lost headings.
  4. Run retrieval tests on questions that depend on tables and headings.

Cost control

  • Route by type: OCR/layout only for scanned or table-heavy documents (see the ingestion cost post).
  • Cache parsed output by content hash.
  • Use vision LLMs only for pages where visuals carry the information.

Tip

Look at the parsed Markdown of your ten most important documents with your own eyes. Many "the model is wrong" complaints trace back to a table that was turned into a jumble of numbers during parsing.

Checklist

  • Documents classified by type
  • Parser chosen per type
  • Structured output with page/section metadata
  • Sample evaluated, including tables
  • Expensive parsing only where needed
  • Parsed output cached

Related fixes