LAB-003: Layout JSON schema and PDF glyph extraction

Context

Comparisons are symmetric only if Word’s output and BayanDocs’ output are expressed in the same format. Glyph-level positions from Word come from its exported PDFs. Candidate pure-Rust PDF parsers as of 2026-10-03 include lopdf (use 0.42 or later for a fixed denial-of-service advisory) and the parsing layer of hayro (experimental); choose with justification.

Objective

A documented layout JSON schema shared by the Word harness and bayan-cli, and a robust extractor that turns PDFs into that schema with glyph-accurate positions.

Scope

In scope

• Schema v1 (JSON Schema plus prose in lab/schema/): document (page count, page sizes), pages (lines with baseline, start and end text offsets, glyphs with character offset, origin, advance, font family and size), other positioned elements where available (images, rules, table cell boxes), and text normalization rules (how numbering labels, field results and special characters are represented) so both sides produce comparable text. All lengths in points with fixed precision, plus BLU where produced by our engine.
• Extractor bayan-lab pdf-extract: parse PDF objects and content streams; track graphics state, text matrices and the current transformation matrix; handle TJ adjustments, character and word spacing, horizontal scaling; map glyphs to Unicode through ToUnicode maps and encodings; group glyphs into lines by baseline and reading order (including right-to-left lines).
• Limits and fuzzing: PDFs are untrusted input; add a fuzz target.
• Tests with PDFs generated by known tools (for example our own spike output, Typst, LibreOffice), plus Word PDFs once LAB-002 produces them (or as a follow-up if not yet available).

Out of scope

Comparison metrics (LAB-004).

Deliverables

Schema, extractor, tests, fuzz target, documentation.

Acceptance criteria

• AC-1 Extracted glyph origins match known positions in synthetic PDFs within 0.01 pt.
• AC-2 Right-to-left and mixed-direction lines are grouped and ordered correctly in tests.
• AC-3 The schema is documented and validated in tests against sample files from both sides.
• AC-4 The fuzz target runs for at least one CPU-hour without crashes.
• AC-5 Parser choice justified; no unsafe, no C dependencies.

Verification

cargo xtask verify; extraction comparisons in tests.

Escalate if

Word’s PDFs encode text in a way that prevents reliable glyph-to-character mapping; propose using the object-model skeleton for those cases.