harmony/docs/adr/023-deploy-architecture.md

# Architecture Decision Record: Deploy Architecture — Scores, Deploy Crates, and the E2E Contract

Initial Author: Jean-Gabriel Gill-Couture

Initial Date: 2026-05-18

Last Updated Date: 2026-05-20

## Status

Accepted. Extends the Score-Topology-Interpret pattern documented
in `CLAUDE.md` (ADR-002, ADR-003) with the *deploy* side of the
contract: what a deploy crate is, how e2e harnesses relate to
production deploys, how the CLI surface is shaped, and the
smoke-test-on-deploy semantics.

## Context

Three failure modes recur in tooling that ships infrastructure as
code, and Harmony exists in part to defeat them. This ADR locks
the deploy-time discipline that keeps them out.

1. **Manifests outside the type system.** YAML/HCL configurations
   are validated at runtime, not compile time — the original
   "YAML mud pit" that ADR-005 names. A Rust framework that
   re-introduces raw `Deployment` / `Service` / `ConfigMap`
   structs in test harnesses, examples, or CLI helpers has only
   dressed up the same anti-pattern in Rust syntax: the typed
   Scores get a clean sample size of one (production), and
   everything else can silently diverge.

2. **Deploy logic with no canonical home.** A "how to apply
   component X end-to-end" routine that lives in three places
   (an example crate, a CLI subcommand, ad-hoc orchestration in
   a test harness) will drift. The framework needs one address
   per deployable component, and every consumer of that
   component composes from there.

3. **"Applied" is not "working."** `helm install` returns
   success the moment the API server accepts the manifest, and
   leaves the operator to debug downstream. Harmony's whole
   reason for existing is to shorten that feedback loop — a
   deploy primitive that doesn't itself verify the result keeps
   the loop open.

## Decision

Nine principles, grouped.

### Deployment as Scores

1. **Deploy with Scores, not handrolled manifests.** Capability
   traits + compile-time bounds are the contract. No
   `k8s_openapi::api::*` structs outside of `Score::interpret`
   bodies. Test harnesses, examples, and CLI helpers compose
   `*Score` types — they never reimplement deploys.

2. **E2E uses the same Scores as production.** Only the
   `Topology` instance changes (local k3d, remote OKD,
   bare-metal HA, …). A test harness is a `Score`-composer
   running against a test Topology. If e2e needs something prod
   doesn't, add the knob to the Score — don't fork the manifest
   in the harness.

3. **One Score per deployable component.** Composition is the
   user-facing primitive: a `MyAppScore` pulls in
   `PostgresScore`, `HttpServerScore`, etc. Don't build
   monolithic "deploy everything" Scores. Each primitive Score
   must be independently testable and substitutable.

4. **Deploy returns only after smoke-test success.** Every Score
   owns a readiness + smoke-test contract that the framework
   runs and blocks on. Convergence errors must be actionable, in
   the style of `rustc`'s error messages, not "exit code 1 from
   helm". The implementation shape of the smoke-test contract is
   deferred (see §Out of scope); the principle is locked in.

### Where deploy logic lives

5. **Deploy logic lives in a `*-deploy` crate** that depends on
   both `harmony` and the runtime crate it deploys. Runtime
   binaries (the artefacts that ship to constrained devices and
   to in-cluster pods) stay free of the `harmony` dep. One
   deploy crate per app area, holding every component-Score for
   that app plus the `main.rs` that drives them via
   `harmony_cli`. The same crate is the single import for any
   consumer — CLI, e2e harness, future control planes.

   `harmony` core stays focused on framework primitives and
   reusable provider modules (DNS, K8s resources, Helm charts,
   NATS, PostgreSQL, …). It is not a parking lot for
   application-specific deploy Scores.

### Topology selection

6. **Topologies are compile-time, selected at runtime.** A
   deploy binary statically lists its supported topologies; the
   operator picks one at deploy time. Adding a brand-new
   topology backend (AWS, GCP, …) is a rebuild — acceptable
   cost, because dynamic-discovery topologies like
   `K8sAnywhereTopology` already cover "any physical place that
   runs k8s". No `Box<dyn Topology>` plugin loaders.

### Framework evolution

7. **Extend Scores with companions, not API changes.** New
   capabilities the framework wants to attach to Scores
   (planning, dry-run, observability, eventually smoke-test)
   default to a *companion* type or trait that wraps a Score
   rather than a new method on `Score` / `Interpret`. The base
   public API stays simple. The exception is principles every
   Score must honor (which may force a required method) — but
   only after the principle has been validated in practice via
   the companion-first iteration.

### CLI

8. **CLI: hybrid, staged.** Today (B): first-party tools ship as
   separate `harmony-*` binaries built on the existing
   `harmony_cli` crate. Tomorrow (C): a top-level `harmony`
   binary discovers `harmony-*` plugin binaries on `$PATH`
   (`kubectl`-style) so a third-party `MyAppScore` author gets
   `harmony deploy my-app` for free. The plugin protocol is
   deferred (see §Out of scope).

### Error handling

9. **`thiserror` almost everywhere; `anyhow` only at binary
   glue.** Library code, public crate boundaries, anything a
   caller might want to match on — typed errors via `thiserror`.
   `anyhow` is reserved for `main.rs`-level glue where the error
   is just printed.

## Out of scope (deferred, not rejected)

- **Score derive macro / deployment DSL.** Strategic intent from
  day one; the framework's value-add concentrates here. Separate
  design effort.
- **Score registry** (Crichton-style:
  <https://willcrichton.net/rust-api-type-patterns/registries.html>).
  Real itch — examples and Scores are hard to discover today.
  Research + ADR first.
- **Inventory as capability-defined physical assets.** Inventory
  is under-engineered today; the original idea is to represent
  physical infrastructure (building → cable → switch port → MAC)
  but most use cases ignore it. Decomposing inventory into a
  capability set is a deep redesign.
- **Plug-in CLI discovery layer (C in principle 8).** The fix
  for the "too many disconnected CLIs" cohesion problem.
  Roadmap item, dedicated future effort.
- **`Application features` ↔ `capabilities` relationship.** An
  in-progress concept the project lead is personally unsure
  about. Not resolved in this ADR.
- **Concrete smoke-test contract shape (principle 4).** Whether
  smoke-test lives as a separate trait, a required method on
  `Score`, a companion struct, or a typestate is open. Until
  it's locked, deploy crates implement per-Score readiness
  checks inside `interpret` bodies — the principle is what
  travels with the Score, not yet the trait shape.

## Consequences

- New deployable components are authored as `*Score` types in a
  `*-deploy` crate, not in `harmony` core. `harmony` core is
  framework primitives plus reusable provider modules; it does
  not accumulate application-specific deploy logic.
- Test harnesses are Score-composers. A harness that finds
  itself building `Deployment` / `Service` / `ConfigMap` structs
  is the signal that a Score is missing, not that the harness
  needs a special path.
- Every Score owns its readiness story. Whatever shape the
  smoke-test contract eventually takes, the Score is the home
  for the logic — not a parallel test fixture.
- Adding a new deploy backend (a new topology) is a deploy-
  binary rebuild. Dynamic loading of topologies is rejected by
  this ADR, and that posture is load-bearing for the
  compile-time-safety guarantees in CLAUDE.md.
- New framework-level capabilities (dry-run, observability,
  smoke-test) ride in on companion types first. Only after a
  companion proves out does it earn a place in the `Score` /
  `Interpret` public API.

## References

- `CLAUDE.md` — Score-Topology-Interpret pattern, capability
  design rules.
- `docs/adr/002-hexagonal-architecture.md` — domain/adapter split
  this builds on.
- `docs/adr/005-interactive-project.md` — the original "no
  YAML-mud-pit" call (Rust DSL over YAML/HCL).
- `docs/adr/009-helm-and-kustomize-handling.md` — established
  pattern: external charts inflate into the same Score pipeline.
- `harmony_agent/deploy` — `*-deploy` crate exemplar.