Command/backend/contract architecture; unify commands under commands/

[Seth Ockerman (OckermanSethGVSU)]

Summary

Reorganizes supernova from a language-oriented layout (crates/, python/) to a command-oriented one, and introduces a language-neutral backend contract system so future nova-load / nova-storm backends (Milvus, Vespa, …) can be added and conformance-checked without touching the user-facing CLI. No new VDB backend is added here — current Qdrant behavior is unchanged and fully preserved.

New layout

commands/            every user-facing nova-* command (Rust + Python together)
  nova-load/         SHIM (Rust): reads vectorstore.type, execs the backend
  nova-storm/        SHIM (Rust): reads target.type, execs the backend
  nova-shim/         shared shim front-controller LIB (not a command)
  nova-contract/     generic language-neutral conformance checker
  nova-inspect/      moved from crates/
  nova-embed, nova-bf, nova-opt, nova-sweep, nova-dist   moved from python/
backends/
  nova-load/qdrant   -> nova-load-qdrant   (moved from crates/nova-load)
  nova-storm/qdrant  -> nova-storm-qdrant  (moved from crates/nova-storm)
  nova-{load,storm}/contracts/rust   shared Rust trait each backend implements
contracts/
  nova-{load,storm}/{v1.yaml,README.md}   canonical language-neutral specs
tests/contracts/     conformance-fixture dirs per command

Two-layer contract enforcement

1. Compile-time (per language): a Rust backend depends on backends/<cmd>/contracts/rust and implements the shared trait (VectorStore for load, QueryTarget for storm, plus neutral types). A missing or mistyped method won't compile.
2. Runtime (cross-language): each backend advertises capabilities --json; nova contract check <backend-exe> --contract <yaml> validates it against the neutral spec (--level shape | dry-run | live). This works for a backend in any language — it only ever talks to the executable. Checks target the backend, never the shim.

Keep the three in lockstep when changing a contract: the Rust trait, the v1.yaml, and each backend's capabilities_json().

Dispatch — UX preserved

nova load ... / nova storm ... / nova sweep ... are unchanged. The shim reads only the backend selector (vectorstore.type / target.type), maps qdrant → nova-<cmd>-qdrant, and execs it (process replacement — stdin/stdout/stderr and exit code/signals pass straight through, including storm's single --json summary line). Unknown type → exit 127 with an install hint. The two shims share all logic via nova-shim, so they can't drift.

Adding a future backend = new backends/<cmd>/<name>/ building nova-<cmd>-<name>, implement the trait, advertise capabilities, pass nova contract check. A config with type: <name> then dispatches automatically — no shim change.

Other changes

• capabilities --json added to both Qdrant backends (kept in lockstep with the specs).
• Makefile: make load/make storm install both shim and backend; new make contract; make test now also runs contract checks at dry-run level.
• CI (rust-binaries.yml) + nova-dist updated to build/install both the shim and the backend on fleet workers (a worker needs both).
• Backend package/binary renamed to nova-<cmd>-qdrant; library crate names kept (nova_load / nova_storm) so internal module paths are unchanged. The neutral StoreError/TargetError drop their qdrant variant; qdrant errors convert at the trait boundary via .map_err(…::backend).
• Docs updated: README.md, AGENTS.md, and new contracts/*/README.md + commands/README.md.

Testing

• cargo test: 87 pass (16 suites) — behavior preserved.
• nova contract check against both backends: PASS (shape/dry-run/live); a negative case (load backend vs storm contract) correctly FAILs with exit 1.
• Shim dispatch verified: default type, unknown-type → 127, and shim output byte-identical to calling the backend directly (stderr + exit code).
• nova-bf pytest 352 passed, 1 skipped; nova-sweep pytest 36 passed; nova-embed has no suite (pre-existing).
• Live Qdrant end-to-end through the shims: load 300 vectors → storm exact search → recall 1.0/1.0/1.0, 0 errors → delete; nova-sweep full load → reindex → storm --json → delete orchestration → 2 rows, 0 errors.
• nova-embed (MiniLM-L6, CPU) and nova-bf (compute+merge, 54/54 self-match) exercised end-to-end locally; nova-inspect runs from its new path.

Notes / risks

• Python packages (nova-embed/bf/opt/sweep/dist) and nova-inspect were moved into commands/ too, so the layout is uniformly command-oriented. nova-opt is a WIP tracked on another branch (only git-ignored artifacts in this checkout).
• nova-dist and the CI release workflow were updated to produce/install both binaries, but the fleet path was not live-tested (AWS/SkyPilot) — the change is mechanical.
• The nova dispatcher (src/cli/) intentionally stays at the repo root (it's the zero-dep root package); it now also lists backends as nova load-qdrant/storm-qdrant, which is harmless and consistent with its zero-opinion design.

🤖 Generated with Claude Code