Bayan Document Model (BDM) — v0

• Status: Draft v0. Binding in its principles (ADR-0007); details are validated and finalized by CORE-004, then promoted to v1 by CORE-101. CORE-004 validated this draft on Loro (reported on 2026-10-07; revised after review, its last convergence runs completed on 2026-10-09); its amendments are included (§5, §6, §14, §16) and its answers and report are in §18.
• Owner stream: MODEL
• Related: ADR-0005, ADR-0007, ADR-0008, ADR-0018, coverage-matrix.md

1. Purpose and design goals

The BDM is the in-memory representation of a document inside bayan-core. It must:

1. Represent all of WordprocessingML losslessly, including content BayanDocs does not understand (Tier B).
2. Make Word’s editing semantics natural, by mirroring Word’s own internal model: a stream of characters in which structural marks are characters.
3. Merge concurrent edits sensibly when stored in a CRDT (ADR-0008).
4. Support fast incremental layout through stable identities and precise change notifications.
5. Be deterministic: the same operations produce the same model on every platform.

2. Overview

flowchart TB
  doc["Document"]
  doc --> stories["Stories<br/>(main, headers, footers, notes, comments, text boxes, table cells, separators)"]
  doc --> entities["Entities<br/>(paragraph props, tables, rows, cells, objects, notes, comments, fields, ranges)"]
  doc --> globals["Global parts<br/>(styles, numbering, settings, theme, font table, media, custom XML, glossary, VBA, signatures, preserved parts)"]
  stories --> atoms["Atoms: text characters + special atoms"]
  atoms --> marks["Marks: character formatting, hyperlinks, comments, revisions"]
  atoms -.->|"special atoms reference"| entities

3. Stories

A story is an independent flow of content. Every story is a sequence of atoms (§4) that ends with a paragraph end.

Every story has a stable StoryId.

4. Atoms

An atom is one position in a story: a Unicode text character, or a special atom. Special atoms are stored as placeholder code points from the C0 control range, which can never occur in document text because the importer and every input path sanitize text: XML 1.0 forbids most C0 characters, and the remaining ones (tab, line feed, carriage return) are converted to atoms on import. Where Word’s binary format (MS-DOC) has a convention, the placeholder mirrors it.

Rules:

• Tables and block-level ranges are block-level atoms: they may appear only at the start of a story or immediately after a paragraph end or another block-level atom. Inline atoms between a paragraph end and a table, such as a bookmark start, belong to a paragraph: the table is then not at a block position, and N4 gives the inline atoms a paragraph of their own.
• Field code text is stored in the stream between FieldBegin and FieldSeparator, as Word does, so field codes can be shown and edited inline; the field result is the content between FieldSeparator and FieldEnd. w:fldSimple is normalized to this form and flagged so it can be written back as a simple field if untouched.
• How literal tab, carriage-return and line-feed characters inside w:t are interpreted follows a Word Behavior Note (default until measured: tab → Tab atom, CR/LF → LineBreak).
• w:lastRenderedPageBreak is not an atom. Its positions are recorded in an import diagnostics table used by the Fidelity Lab and discarded on regeneration.
• w:proofErr is not kept; proofing state is regenerated.

5. Marks

Character-level information is stored as marks: independent key/value annotations over ranges of atoms, merged per key (Peritext semantics in the CRDT).

A mark key starts with its family (r, link, cmt, rev), followed, in families with several keys, by a colon and the rest of the key (r:b, cmt:<CommentId>). The family decides how the mark expands, because Loro configures expansion by the part of a key before its first colon (CORE-004). The colon therefore appears in a key only as the family separator: element paths use dots, and an XML namespace prefix is written with a dot too (r:w14.ligatures, not r.w14:ligatures).

No mark applies to a structural atom (ParagraphEnd, RangeStart and RangeEnd, TableBlock): the view drops every mark there, whatever its family, because these atoms have no glyph of their own, and the paragraph mark’s formatting, its revisions included, lives in the paragraph’s properties (§6). A comment over nothing but an empty paragraph therefore highlights no character in the view; whether Word shows such a comment on the paragraph mark is a question for a Word Behavior Note in CORE-101 (CORE-004).

Effective formatting is not stored in marks: marks hold only direct formatting. bayan-styles computes effective properties from defaults, styles and direct formatting.

6. Paragraphs and sections

Each ParagraphEnd atom has a ParagraphId with a property set mirroring w:pPr: style, justification, indentation, spacing, numbering reference, tabs, borders, shading, keep rules, widow control, outline level, frame properties, text direction, East Asian options, paragraph-level revision (w:pPrChange), preserved unknown children, and Word’s own identifiers (w14:paraId, w14:textId) and revision-save IDs (w:rsid*), which are preserved. The run properties of the paragraph mark (w:pPr/w:rPr) belong to this set too, stored as rPr.<property> (§5).

• When a paragraph ends a section, its property set carries the section properties (w:sectPr), exactly as OOXML stores them. The final section’s properties belong to the story (body.sectPr).
• Splitting a paragraph inserts a ParagraphEnd with a new ParagraphId whose properties are derived per Word’s rules (for example a heading followed by its “next” style).
• Merging paragraphs deletes a ParagraphEnd; which paragraph’s properties survive follows a Word Behavior Note.

7. Tables

Tables are entities referenced by a TableBlock atom.

Rows are kept in a movable list so concurrent row insertions, deletions and reorders merge cleanly; cells in a list per row. Ragged rows (different cell counts) are valid, as they are in Word.

8. Objects

An ObjectAnchor atom references an object entity:

• Placement: inline (extent, effect extent, properties) or anchored (simple position, horizontal and vertical positioning with relativeFrom and alignment or offset, extent, wrapping mode and polygon, distances, behind text, lock, layout in table cell, allow overlap, z-order).
• Graphic: picture (media reference, crop, fill mode, effects, alternative text and title); shape (preset or custom geometry, fill, line, effects, style references, text box story, body properties); group; drawing canvas; chart (chart part reference plus cached data); diagram/SmartArt (parts plus the drawing cache Word stores); ink; OLE object (program identifier, embedded part, preview image); math is a separate atom (§4).
• Legacy VML (w:pict) is mapped to the same object kinds where possible, with the original VML preserved for verbatim write-back while untouched.
• Alternate content: for objects inside mc:AlternateContent, the understood choice is modeled; the fallback is preserved and dropped only when the object is edited (ADR-0018).

9. Notes, comments and fields

• Note: kind (footnote or endnote), story, custom mark flag, preserved children. Numbering and placement come from section and settings properties.
• Comment: author, initials, date (and UTC date), durable and paragraph identifiers from the comments extension parts, parent comment (threads), done state, story, preserved children. Anchored by cmt:<id> marks and a CommentReference atom.
• Field: parsed instruction (field type, arguments, switches) cached from the code text, lock flag, dirty flag, legacy form-field data (w:ffData), binary field data (w:fldData), simple-field flag, preserved children.

10. Ranges

A range entity is delimited by RangeStart/RangeEnd (inline) or BlockRangeStart/BlockRangeEnd (block) atoms:

11. Global parts

12. Identity

• Every entity (paragraph, table, row, cell, object, note, comment, field, range, story, equation, ruby) has a 128-bit identifier generated by BayanDocs, independent of OOXML identifiers and of CRDT-internal identifiers. Tests use a seeded generator so results are deterministic.
• OOXML identifiers (w:id of bookmarks, comments, revisions; w14:paraId; relationship IDs) are preserved as attributes and regenerated only where uniqueness requires.

13. Preservation and source spans

• Unknown content is preserved at the nearest model position: as a PreservedInline atom, a preserved child list on a property set or entity, or a preserved package part with its relationships.
• Source spans: for each paragraph, table and part, the importer records the byte range of its original XML and a content hash. The exporter writes untouched elements verbatim from the original package (ADR-0018). Any edit to an element marks it (and only it) dirty.
• Namespace declarations needed by verbatim fragments are preserved on the part’s root element.

14. Invariants and normalization

Invariants a well-formed model satisfies:

Concurrent edits can violate these. Normalization is a deterministic projection applied when building the view used by layout, export and accessibility. It never writes to the shared state by itself; the next local edit that touches an affected region writes the normalized structure explicitly (materialization), so all replicas converge. Materialization uses the identifiers and properties the view already shows, so it does not change the view, and it is committed outside undo, so undoing the edit returns the view to its state before the edit.

Four more details complete the projection. A C0 control character that is neither a tab (whose atom names no entity) nor a placeholder bound to an existing entity of its kind is dropped, because C0 characters are reserved for placeholders (§4). A cell or comment whose story is missing, or was already reached as another cell’s or comment’s story (N7 for stories), gets an empty story whose identifier is derived from its owner’s, so that I1 and I6 hold. A row or cell that the stored lists of rows and cells hold twice, which happens when two replicas delete it and both undo (each undo inserts it again), is shown once, where it first appears (N7 for rows and cells); operations on columns count columns as the view shows them, so they change such a row once and count such a cell once; a new row gets as many cells as the view shows in the row next to it, and a new column gives no cell to a row or a table that the view omits (N6). The identifier of a virtual paragraph end (N1, N4) is derived deterministically and avoids every identifier that a stored paragraph end binds, and every identifier whose entry in the paragraph registry materialization could not write into: a value, another kind of container, or a map created otherwise than as the mergeable map that materialization creates. A mergeable map that an earlier materialization of the same paragraph end left there, when that end was deleted, is reused, and the view shows its properties, so that every replica derives the same identifier whether or not it received that map, and concurrent materializations merge into one paragraph end.

CORE-004 showed with property-based tests over randomly broken documents, and in 1,000 randomized three-replica runs of 10,000 operations before its review and again after it, with 100 more on its final code, that normalization is deterministic, idempotent and always yields a model satisfying I1–I7, as judged by an invariant checker that has a test for each invariant (§18).

15. Operations and transactions

Semantic operations (implemented in bayan-edit on top of bayan-model) include: insert text; delete range; set or clear run properties on a range; set paragraph properties; split and merge paragraphs; insert and delete table, rows and columns; merge and split cells; set table, row and cell properties; insert, move and delete objects; create, modify and delete styles and numbering definitions; add, reply to, resolve and delete comments; accept and reject revisions; insert and update fields; set settings.

Operations are grouped into transactions. Each transaction is one undo step, carries its origin (local or remote) and metadata (user, time, command identifier), and produces a change set (stories and ranges touched, entities changed) that drives incremental layout.

16. CRDT mapping (validated by CORE-004)

CORE-004 implemented and validated this mapping on Loro 1.16.2 (bayan-crdt and bayan-model in bayan-core).

17. Derived views

bayan-model provides read-only views for consumers: a block tree (sections → blocks → inline runs) for layout and accessibility; run iterators with direct properties; position mapping between atoms, CRDT positions and OOXML source spans; and change-set subscriptions.

18. Questions answered by CORE-004

The CORE-004 report (spikes/crdt-model/REPORT.md in bayan-core) gives the evidence for each answer.

1. Does Loro support per-key expansion for dynamically named mark keys (cmt:<id>)? Yes: Loro configures expansion by the part of a key before its first colon, so the family cmt governs every cmt:<id> key, however many comments overlap (§5).
2. Is the placeholder-plus-mark binding robust under concurrent deletion and re-insertion? Yes. Typing next to an atom never extends its binding, undo restores atoms with their bindings, and cut and paste racing with formatting and deletion converges. Races that bind two atoms to one entity are resolved by N7; no mark applies to structural atoms, which is why paragraph-mark formatting moved into the paragraph’s properties (§5).
3. Memory and load time of many small maps for a 500-page document in WebAssembly? 142 MB of WebAssembly memory after loading and reading a 500-page document with 21,000 paragraph maps, 200 tables and 2,000 comments; 0.76 s to load and read once a quadratic decoding bug in Loro 1.16.2 is fixed (7.0 s without the fix). Reading one map per entity is 38% of the first read natively (§16).
4. Movable lists or a movable tree for table rows and nested structures? Movable lists: nesting is expressed by stories, not by a container tree. Cycles from concurrent moves of table atoms cannot make normalization loop (N7), and nesting depth is limited (N9).
5. How does undo interact with materialization? Materialization is committed outside undo and does not change the view, so undoing an edit returns the view to its state before the edit (§14).
6. Snapshot and update sizes for typical editing sessions? About 105 bytes per edit sent alone and 18.6 KB for 1,000 edits sent together; 9.2 MB for the 500-page document with its whole history and 5.4 MB as a shallow snapshot (which is slower to work with in Loro 1.16.2, §16).
7. Resource limits needed when importing untrusted updates? Size, change, operation and value-depth limits, but limits are not enough: Loro checks the change and operation limits only by decoding the whole blob (only the size limit is checked before), and its decoding panics on crafted updates, aborts the process on crafted headers that request absurd allocations, and overflows the stack when it frees deeply nested values, before the depth limit can look at them; and an update with many overlapping marks costs time and memory cubic in their number in a replica with a subscriber. The adapter contains panics (the poisoned replica refuses everything afterwards); the engine’s hosts must isolate imports from untrusted peers, with a time and memory budget, and restart an engine that aborts (§16, ADR-0008).