Document Logical Units

Search engines for archives think in pages. The page is what the HTR engine produced, what the index is built over, and what the bounding boxes anchor to. That granularity is the right one for retrieval. It is the wrong one for everything that comes after retrieval.

A historian looking for the council’s decision about the irrigation channel is not looking for one page. They are looking for the four pages of the council meeting where the decision was discussed, written down, and signed. A genealogist reading a testamento needs all five pages of it together. A cataloguer writing a scope-and-content note for a parish entry needs the baptism record and the marriage record on the next folio together, because the priest filed them as one act.

archAIc represents that object as a Document Logical Unit (DLU): the curatorial wrapper above the page that bundles one or more pages (or fragments of pages) into a single communication act. Pages stay the unit of search. DLUs are the unit of meaning, citation, and curation.

What a DLU is, concretely

A DLU is typed (acta, testamento, matrimonio, bautismo, padrón, escritura, and so on, drawn from a controlled vocabulary the deployment defines), carries structured metadata (title, primary date, parties, scope note, custom fields), and points at the ink through an ordered list of extents. Each extent is a (page, line range) slice: a whole page, or just lines 3 to 45 of a page, or just the last eight lines of one page and the first six of the next.

A typical DLU looks like this in the data model:

• Identity: a stable id (deployment-local integer), a type from the controlled vocabulary, the collection and libro it lives in.
• Extent: [(p142, lines 3-45), (p143, whole page), (p144, lines 0-12)]. The first page opens with the formula, the middle page is fully inside, the last page closes with signatures before the next document starts.
• Attributes: a primary ArchivalDate (which handles historical messiness: uncertainty intervals, Julian versus Gregorian, regnal dates preserved in the original form), optionally several role-qualified secondary dates, lists of persons, places, and organisations with surface forms and roles, and a typed bag of deployment-specific custom fields.
• Extracted graph: the persons, places, events, and relations the system has lifted out of the text (see Entities, events, relations).
• Provenance: version, created and updated timestamps, the curator who touched it last, and the source (manual, heuristic, model, imported).

A DLU never replaces a page. It is layered on top: a page that belongs to no DLU still indexes, still opens, still highlights matches. A DLU that is deleted leaves its pages intact. The page layer is the foundation; the DLU layer adds the curatorial overlay the field actually works in.

Nested DLUs: the corpus is also a DLU

In archAIc the same DLU shape is used at every level of the archival hierarchy. A libro is a DLU. Its children are the documents inside it, which are themselves DLUs (an acta, a testamento). Those documents may have children too (a testamento with its codicil and execution record). At the top, a collection is a DLU containing libros.

The benefit of using the same shape everywhere is that the rest of the system stops needing special cases. Roll-up logic that proposes “these five anchored persons in this segment DLU are the same Don Emilio that appears in the libro’s other segments” is the same shape as logic that proposes “these libros are part of the same fonds.” Cross-DLU linking, change-log review states, IIIF manifests, and exports all read off one model. There is no separate “fonds object” and “document object” with different APIs.

Anchored to ink

Every DLU points at its source ink through (page, line) coordinates, and every extracted entity, event, and relation inside the DLU carries at least one (page, line) mention with a confidence score. This is the rule that makes the model cataloguer-trustworthy: a DLU is not an abstract record about the past, it is a wrapper around specific lines on specific pages, and so is every claim it contains. See No anchor, no entry for why this is enforced at write time and at read time.

The line ranges in an extent also survive re-HTR. When a page is re-transcribed by a better model and the line indices shift, the DLU’s anchors carry an HTR generation tag and a fuzzy re-anchoring job either updates the indices or flags the DLU for human review. The system does not silently corrupt the curated layer when the foundation is improved.

Drawing the boundaries

Somebody has to say where one document ends and the next begins. archAIc supports three boundary methods, in increasing order of automation.

By hand. The cataloguer selects a range of pages in the editor, picks a type, fills in attributes. Every deployment supports this. For small collections, or for libros that are too irregular for any automation to help, this is just what the work looks like.

By rule. Some archives have conventions. A council book where every acta starts with “En la villa de…”. A protocol where every escritura opens with a notarial formula. A pattern on the first few lines of each page catches most boundaries. The system proposes; the cataloguer reviews.

By meaning. This is the interesting one. archAIc reuses the same passage embeddings it computes for semantic search. Consecutive passages of the same document tend to be close in embedding space (they share vocabulary, names, topic). When one document ends and another begins, the embedding shifts sharply. The system computes that shift across the whole libro as a boundary signal, smooths it, calibrates the threshold against the libro’s expected segment length, and renders the proposed cuts as a histogram the cataloguer can accept, reject, or drag.

Both dense embeddings and sparse TF-IDF vectors exist per passage in archAIc. Segmentation today reads the dense signal; combining both is a near-term improvement that strengthens boundary detection on libros where vocabulary changes more sharply than topic. Either way the segmenter proposes and the cataloguer decides. The cost of a wrong boundary is high; the cost of a confirming click is low.

How DLUs appear in the surfaces

A DLU is not a separate destination in the UI. A page result inside a DLU picks up a badge with the type, date, and primary party, plus a link to read the full document. A collection view groups pages into visual DLU cards, with an Unassigned section that surfaces the pages still needing curation. The MCP server exposes the DLU as a first-class object: get_dlu, list_pages, dlu_children, dlu_ancestors, dlu_rollup. The IIIF manifest exposes the DLU as a range above the canvases. Exports (EAD, EAC-CPF, Linked Art, AtoM CSV) emit one row per DLU at whichever level the consumer asks for.

The page is what the engine indexes. The DLU is what the institution catalogues. Both layers stay in sync, both stay anchored to the ink, and both are visible end to end to anything that reads from archAIc.