Skip to content

Crate map

archAIc’s Rust source lives in a single Cargo workspace under rust/crates/. The workspace splits into three binaries and a small handful of library crates. Everything below the line “binary crates” is a library; binaries depend on libraries, never the other way around.

Source-repo binary name: gd (gd-cli/Cargo.toml). The customer-facing rename to archaic is a planned wrapper that does not yet ship; today, operators invoke the engine as gd …. The other two binaries are HTTP servers and run as their own Docker containers.

CrateBinaryResponsibility
gd-cligdThe CLI. Subcommands for build, ingest, semantic build, serve, plus query, inspect, and validate for ops work. Wraps every other crate in the workspace through one command-line surface.
gd-servergd-serverThe REST API. axum 0.8, utoipa for OpenAPI, tower-http for CORS and tracing. Owns the C++ compat endpoint, the v1 API, the IIIF endpoints, every export format, and the natural-language Q&A surface.
gd-mcpgd-mcpThe Model Context Protocol server. Wraps gd-server over HTTP and exposes roughly 30 tools to AI agents with OAuth 2.1 + PKCE + JWT, RFC 8414 discovery, RFC 7591 dynamic client registration. Streamable HTTP transport.
CrateResponsibility
gd-engineIndex core. LMDB storage layer, the FST vocabulary, RoaringBitmap postings, the three-level read-cache, the scoring pipeline, the query execution engine. Everything that touches the lexical index goes through here.
gd-parserQuery parser. winnow-based grammar for Boolean, fuzzy (word~), wildcard (pre*), phrase ("a b"), proximity (a NEAR/3 b), and the legacy C++-compat query string.
gd-buildTwo-pass index builder. Reads PrIx .txt files, writes the LMDB store and the FST. File-backed hit buffer keeps memory bounded on multi-million-page corpora. Optional ratatui TUI for build progress.
gd-ingestTSDO ingestion. Reads page JSON, copies image files, generates thumbnails, writes the corpus SQLite database. Includes the gd ingest add and gd ingest remove flows.
gd-semanticSemantic layer. ONNX-runtime embedding inference (CPU and CUDA), the sliding-window passage builder, dense and sparse vector construction, and the Qdrant client wrapper. Computes per-passage μ-vectors and TF-IDF; fused with RRF at query time.
gd-configConfiguration schema and migrations. SQLite-backed config (cfg_* tables), the migration runner, the encrypted-secret primitives (libsodium crypto_secretbox), and the translation seed. The single source of truth for runtime config.
gd-iiifIIIF emitters. Presentation 3.0 manifests, Image API responses, AnnotationPage, navPlace, Content Search 1.0/2.0, Content State 1.0. Builder API surfaced over the v1 REST endpoints.
gd-linked-artLinked Art and CIDOC-CRM JSON-LD. The DLU graph projected into Linked Art shape; the canonical export format for the DLU layer.
gd-benchThe benchmark harness. compare, perf, and versus modes; drives load against gd-server (or the legacy C++ engine) and produces JSON + ratatui dashboards. Not deployed; engineering-only.

A future-state gd-ric-o crate hosts the RiC-O 1.0 emitter and the proposed archaic:Mention extension. Today it lives inside gd-linked-art; the split is queued.

The gd-text crate names the per-page export pipeline (TSDO importer + ALTO / hOCR / PAGE / CSV emitters). It does not yet exist as a separate crate; the responsibilities live in gd-ingest (for read) and gd-server (for write) until the surface stabilises.

Crate dependency graph — gd-cli at the top depends on every other crate; gd-server, gd-build, gd-semantic, gd-ingest, gd-bench in the middle tier; gd-engine and gd-parser at the bottom; gd-mcp on the side connected to gd-server with a dashed HTTP edge.

gd-iiif, gd-linked-art, and gd-config are library crates that live inside gd-server at the engine tier; the diagram above shows the eight workspace top-level crates.

gd-cli depends on every library crate so a single binary covers build, ingest, semantic build, and serve. gd-mcp is the one process boundary that talks to gd-server over HTTP rather than linking against it at compile time, because the MCP surface is deliberately a thin wrapper that can be deployed on its own host with its own OAuth config.

Every library crate inherits the workspace lint policy: clippy correctness is deny, suspicious and complexity and perf are warn, and CI runs with RUSTFLAGS='-D warnings'. Adding a new #[allow(...)] should come with a one-line comment explaining why.

  • Around 1,500 unit + integration tests across the workspace.
  • Property tests (proptest) on the IIIF manifest builder, the translit and filter-list logic in gd-ingest, and the parser.
  • Five fuzz targets under rust/fuzz/ (excluded from the workspace; run on demand).
  • Live integration smoke tests for the LLM client run against a real Ollama under the live-llm feature flag.

See the deployment overview for how these binaries are composed at runtime.