Skip to content
archAIc
Blog

AtoM sync and bulk export

archAIc pushes descriptions, authority records, and page images directly into the customer’s existing AtoM instance via REST. No CSV exports to shuffle, no manual symfony csv:import, no nightly cron. An SME confirms a DLU in archAIc; seconds later it is live in the public catalog. Every push is tracked in a sync-state ledger; failures surface with full payload and retry history.

The same export renderers drive the file-based path for environments that prefer bulk CSV / EAD / EAC-CPF over REST.

Parallel deploy, not via the DIP chain

Section titled “Parallel deploy, not via the DIP chain”

archAIc-to-AtoM sync runs as a parallel deploy alongside the Archivematica DIP chain, not on top of it. archAIc is the source of truth; AtoM is a derived view. Nothing flows back from AtoM to archAIc.

The Artefactual pilot (in flight, 2026) extends this model to AM as a peer target through the new AtoM import API, with archAIc as the first external consumer.

Sync modes

Section titled “Sync modes”

Each registered target has a sync mode. A customer can run one target in continuous mirror and another in webhook mode.

ModeTriggerLatency
ManualOperator clicks “Sync to AtoM” in the archAIc admin UISynchronous, 5-30 s for a libro
Webhook-drivenarchAIc emits dlu.confirmed events; a customer-side daemon consumes them and calls archAIc’s /sync endpointsSeconds to minutes
Continuous mirrorarchAIc-internal worker subscribes to its own event stream, calls the AtoM REST API directlySub-minute

Mode is a per-target setting. Manual + continuous shipped in v1; webhook-driven is the same orchestrator wired to a different trigger.

Smart bulk sync (dual-CSV)

Section titled “Smart bulk sync (dual-CSV)”

When the deployment uses the bulk-CSV path (engine writes a .ready marker, the AtoM-side watcher runs csv:import), archAIc routes each DLU into one of two CSVs based on whether it has moved since the last successful sync:

  • descriptions_update.csv — rows where archAIc’s parent matches the parent archAIc last sent. Imported with --update=match-and-update. AtoM-side curator edits on the row survive.
  • descriptions_replace.csv — rows where archAIc has moved the DLU since the last sync. Imported with --update=delete-and-replace. The row is recreated with the new parent.

This means curator enrichments in AtoM (corrected titles, expanded scope-and-content, attached physical-object descriptions) survive every re-sync that doesn’t actually move the DLU. The only edits lost are on rows whose hierarchy changed, which is the inherent cost of overriding hierarchy.

Sync state lives on two dlu columns: last_synced_at_atom and last_synced_parent_atom_legacyid. The engine reads them together; never-synced rows are routed to update.csv on first contact.

Delete propagation

Section titled “Delete propagation”

After every successful bulk sync, archAIc queries for DLUs that were previously synced and are now soft-deleted, then issues DELETE /api/informationobjects/{slug} against each one. Sync-state row is cleared on success so retries are idempotent.

Hard-deletes are out of scope for v1 (archAIc has no record left to identify them). Soft-delete is the default in archAIc; v1 covers the curator re-segmentation workflow (splitting one act into two, merging two into one) that produces orphan AtoM rows.

REST push (continuous mirror)

Section titled “REST push (continuous mirror)”
POST /v1/dlus/{id}/sync/{target_id}
POST /v1/collections/{coll}/libros/{libro}/sync/{target_id}
GET  /v1/dlus/{id}/sync-status

The libro-level POST is the bulk handle: it walks every descendant DLU of the libro and pushes them in one run. Per-DLU sync-status is the only status endpoint that ships today; a cross-target rollup view is not exposed.

A sync run returns a structured report:

{
  "target": "customer-prod-atom",
  "started_at": "2026-05-01T14:23:11Z",
  "finished_at": "2026-05-01T14:23:14Z",
  "synced":   { "descriptions": 39, "authorities": 2788, "digital_objects": 39, "links": 4521 },
  "skipped_unchanged": { "descriptions": 12, "authorities": 200 },
  "failed": []
}

Idempotency is enforced by a payload-hash short-circuit: if the new payload hashes to the value already on file, no API call is made. Saves roughly 80% of re-sync traffic in steady state.

Authority deduplication is keyed on (kind, canonical_name). The same person mentioned in twelve DLUs gets one AtoM actor record across every re-sync.

Bulk export endpoints (file path)

Section titled “Bulk export endpoints (file path)”

For deployments that prefer file delivery, the same data is reachable as static artefacts:

  • GET /v1/dlus/{id}/export?format=isad-json — ISAD(G) JSON, single DLU.
  • GET /v1/dlus/{id}/export?format=atom-csv — AtoM CSV row (descriptions).
  • GET /v1/dlus/{id}/export?format=eac-cpf — EAC-CPF XML, authority records.
  • GET /v1/dlus/{id}/export?format=ead-xml — EAD 2002 XML, single DLU and descendants.
  • GET /v1/dlus/{id}/export?format=mets — METS XML, file groups for the DLU’s pages.
  • GET /v1/dlus/{id}/export?format=dip-zip — OAIS Dissemination Information Package, METS-wrapped, PREMIS-provenanced, with optional ALTO / PAGE / hOCR / TSDO file groups for every page.
  • GET /v1/collections/{coll}/libros/{libro}/export?format=… — libro-level variants of all of the above.

ISAD(G) JSON is editable (PATCH against the same URL); EAC-CPF is editable for SME-confirmed authority records. EAD and METS are read-only renderings.

Pilot status

Section titled “Pilot status”

REST push and the libro-level bulk sync ship against the dipuCoru AtoM deployment today. The Archivematica + AtoM new-import-API pilot with Artefactual is forward-looking; archAIc is the first external consumer scheduled to exercise the new API.

References

Section titled “References”