harmony/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Build & Test Commands

# Full CI check (check + fmt + clippy + test)
./build/check.sh

# Individual commands
cargo check --all-targets --all-features --keep-going
cargo fmt --check          # Check formatting
cargo clippy               # Lint
cargo test                 # Run all tests

# Run a single test
cargo test -p <crate_name> <test_name>

# Run a specific example
cargo run -p <example_crate_name>

# Build the mdbook documentation
mdbook build

What Harmony Is

Harmony is the orchestration framework powering NationTech's vision of decentralized micro datacenters — small computing clusters deployed in homes, offices, and community spaces instead of hyperscaler facilities. The goal: make computing cleaner, more resilient, locally beneficial, and resistant to centralized points of failure (including geopolitical threats).

Harmony exists because existing IaC tools (Terraform, Ansible, Helm) are trapped in a YAML mud pit: static configuration files validated only at runtime, fragmented across tools, with errors surfacing at 3 AM instead of at compile time. Harmony replaces this entire class of tools with a single Rust codebase where the compiler catches infrastructure misconfigurations before anything is deployed.

This is not a wrapper around existing tools. It is a paradigm shift: infrastructure-as-real-code with compile-time safety guarantees that no YAML/HCL/DSL-based tool can provide.

The Score-Topology-Interpret Pattern

This is the core design pattern. Understand it before touching the codebase.

Score — declarative desired state. A Rust struct generic over T: Topology that describes what you want (e.g., "a PostgreSQL cluster", "DNS records for these hosts"). Scores are serializable, cloneable, idempotent.

Topology — infrastructure capabilities. Represents where things run and what the environment can do. Exposes capabilities as traits (DnsServer, K8sclient, HelmCommand, LoadBalancer, Firewall, etc.). Examples: K8sAnywhereTopology (local K3D or any K8s cluster), HAClusterTopology (bare-metal HA with redundant firewalls/switches).

Interpret — execution glue. Translates a Score into concrete operations against a Topology's capabilities. Returns an Outcome (SUCCESS, NOOP, FAILURE, RUNNING, QUEUED, BLOCKED).

The key insight — compile-time safety through trait bounds:

impl<T: Topology + DnsServer + DhcpServer> Score<T> for DnsScore { ... }

The compiler rejects any attempt to use DnsScore with a Topology that doesn't implement DnsServer and DhcpServer. Invalid infrastructure configurations become compilation errors, not runtime surprises.

Higher-order topologies compose transparently:

• FailoverTopology<T> — primary/replica orchestration
• DecentralizedTopology<T> — multi-site coordination

If T: PostgreSQL, then FailoverTopology<T>: PostgreSQL automatically via blanket impls. Zero boilerplate.

Architecture (Hexagonal)

harmony/src/
├── domain/           # Core domain — the heart of the framework
│   ├── score.rs      # Score trait (desired state)
│   ├── topology/     # Topology trait + implementations
│   ├── interpret/    # Interpret trait + InterpretName enum (25+ variants)
│   ├── inventory/    # Physical infrastructure metadata (hosts, switches, mgmt interfaces)
│   ├── executors/    # Executor trait definitions
│   └── maestro/      # Orchestration engine (registers scores, manages topology state, executes)
├── infra/            # Infrastructure adapters (driven ports)
│   ├── opnsense/     # OPNsense firewall adapter
│   ├── brocade.rs    # Brocade switch adapter
│   ├── kube.rs       # Kubernetes executor
│   └── sqlx.rs       # Database executor
└── modules/          # Concrete deployment modules (23+)
    ├── k8s/          # Kubernetes (namespaces, deployments, ingress)
    ├── postgresql/   # CloudNativePG clusters + multi-site failover
    ├── okd/          # OpenShift bare-metal from scratch
    ├── helm/         # Helm chart inflation → vanilla K8s YAML
    ├── opnsense/     # OPNsense (DHCP, DNS, etc.)
    ├── monitoring/   # Prometheus, Alertmanager, Grafana
    ├── kvm/          # KVM virtual machine management
    ├── network/      # Network services (iPXE, TFTP, bonds)
    └── ...

Domain types to know: Inventory (read-only physical infra context), Maestro<T> (orchestrator — calls topology.ensure_ready() then executes scores), Outcome / InterpretError (execution results).

Key Crates

Crate	Purpose
harmony	Core framework: domain, infra adapters, deployment modules
harmony_cli	CLI + optional TUI (--features tui)
harmony_config	Unified config+secret management (env → SQLite → OpenBao → interactive prompt)
harmony_secret / harmony_secret_derive	Secret backends (LocalFile, OpenBao, Infisical)
harmony_execution	Execution engine
harmony_agent / harmony_inventory_agent	Persistent agent framework (NATS JetStream mesh), hardware discovery
harmony_assets	Asset management (URLs, local cache, S3)
harmony_composer	Infrastructure composition tool
harmony-k8s	Kubernetes utilities
k3d	Local K3D cluster management
brocade	Brocade network switch integration

OPNsense Crates

The opnsense-codegen and opnsense-api crates exist because OPNsense's automation ecosystem is poor — no typed API client exists. These are support crates, not the core of Harmony.

• opnsense-codegen: XML model files → IR → Rust structs with serde helpers for OPNsense wire format quirks (opn_bool for "0"/"1" strings, opn_u16/opn_u32 for string-encoded numbers). Vendor sources are git submodules under opnsense-codegen/vendor/.
• opnsense-api: Hand-written OpnsenseClient + generated model types in src/generated/.

Key Design Decisions (ADRs in docs/adr/)

• ADR-001: Rust chosen for type system, refactoring safety, and performance
• ADR-002: Hexagonal architecture — domain isolated from adapters
• ADR-003: Infrastructure abstractions at domain level, not provider level (no vendor lock-in)
• ADR-005: Custom Rust DSL over YAML/Score-spec — real language, Cargo deps, composable
• ADR-007: K3D as default runtime (K8s-certified, lightweight, cross-platform)
• ADR-009: Helm charts inflated to vanilla K8s YAML, then deployed via existing code paths
• ADR-015: Higher-order topologies via blanket trait impls (zero-cost composition)
• ADR-016: Agent-based architecture with NATS JetStream for real-time failover and distributed consensus
• ADR-020: Unified config+secret management — Rust struct is the schema, resolution chain: env → store → prompt
• ADR-023: Deploy architecture — Scores everywhere (incl. tests), per-component *-deploy crates, deploy blocks on smoke-test, topologies are compile-time

Capability and Score Design Rules

Capabilities are industry concepts, not tools. A capability trait represents a standard infrastructure need (e.g., DnsServer, LoadBalancer, Router, CertificateManagement) that can be fulfilled by different products. OPNsense provides DnsServer today; CoreDNS or Route53 could provide it tomorrow. Scores must not break when the backend changes.

Exception: When the developer fundamentally needs to know the implementation. PostgreSQL is a capability (not Database) because the developer writes PostgreSQL-specific SQL and replication configs. Swapping to MariaDB would break the application, not just the infrastructure.

Test: If you could swap the underlying tool without rewriting any Score that uses the capability, the boundary is correct.

Don't name capabilities after tools. SecretVault not OpenbaoStore. IdentityProvider not ZitadelAuth. Think: what is the core developer need that leads to using this tool?

Scores encapsulate operational complexity. Move procedural knowledge (init sequences, retry logic, distribution-specific config) into Scores. A high-level example should be ~15 lines, not ~400 lines of imperative orchestration.

Scores must be idempotent. Running twice = same result as once. Use create-or-update, handle "already exists" gracefully.

Scores must not depend on execution order. Declare capability requirements via trait bounds, don't assume another Score ran first. If Score B needs what Score A provides, Score B should declare that capability as a trait bound.

See docs/guides/writing-a-score.md for the full guide.

Deploy Architecture (ADR-023)

The Score-Topology-Interpret pattern above tells you how to describe a deployment. The rules below tell you how to ship one. These are non-negotiable.

Deploy with Scores, not handrolled manifests. No k8s_openapi::api::* structs outside of Score::interpret bodies. CLIs, examples, and test harnesses all compose *Score types — they never reimplement deploys. If you find yourself building Deployment / Service / ConfigMap structs in a test harness, stop: that's the YAML-mud-pit anti-pattern in Rust clothing. Reach for the existing Score, or write a missing Score in the right deploy crate.

E2E uses the same Scores as production. Only the Topology instance changes (local k3d, remote OKD, bare-metal HA). A test harness is a Score-composer running against a test Topology. If e2e needs something prod doesn't, add the knob to the Score — don't fork the manifest in the harness.

One Score per deployable component. Composition is the user-facing primitive: MyAppScore pulls in PostgresScore, HttpServerScore, etc. Don't build monolithic "deploy everything" Scores; build small testable ones and compose upward.

Deploy returns only after smoke-test success. Every Score owns a readiness + smoke-test contract that the framework runs and blocks on. helm install && hope is the anti-pattern harmony exists to fix. Convergence errors must be actionable in the style of rustc's error messages, not "exit code 1 from helm". (The implementation shape of the smoke-test contract is open; the principle is locked in.)

Deploy logic lives in a *-deploy crate that depends on both harmony and the runtime crate. Runtime binaries (the thing that ships to constrained devices and to in-cluster pods) stay free of the harmony dep. Pattern: harmony_agent/deploy, fleet/harmony-fleet-deploy. Each app area gets one deploy crate that holds every component's Score plus a main.rs driven by harmony_cli that selects which component to deploy.

Topologies are compile-time, selected at runtime. A deploy binary statically lists its supported topologies; the user picks one at deploy time. Adding a new topology backend is a rebuild — that's an acceptable cost because dynamic-discovery topologies like K8sAnywhere already cover "any physical place that runs k8s". No Box<dyn Topology> plugin loaders.

Extend Scores with companions, not API changes. New capabilities the framework wants to attach to Scores (planning, dry-run, observability, eventually smoke-test) default to a companion type or trait that wraps a Score rather than a new method on Score/Interpret. The base public API stays simple.

CLI: hybrid, staged. Today (B): first-party tools ship as separate harmony-* binaries built on the existing harmony_cli crate. Tomorrow (C): a top-level harmony binary discovers harmony-* plugin binaries on $PATH (kubectl-style). The plugin protocol is not in scope for any current PR — dedicated future effort.

Use thiserror almost everywhere; anyhow only at binary glue. Library code, public crate boundaries, anything callers might want to match on — typed errors via thiserror. anyhow is reserved for main.rs-level glue where the error is just printed.

See docs/adr/023-deploy-architecture.md for the full rationale, including what's explicitly deferred (Score derive macro, Score registry, plugin CLI discovery, inventory redesign, smoke-test contract shape).

Conventions

• Rust edition 2024, resolver v2
• Conventional commits: feat:, fix:, chore:, docs:, refactor:
• Small PRs: max ~200 lines (excluding generated code), single-purpose
• License: GNU AGPL v3
• Quality bar: This framework demands high-quality engineering. The type system is a feature, not a burden. Leverage it. Prefer compile-time guarantees over runtime checks. Abstractions should be domain-level, not provider-specific.