Files
harmony/examples
Jean-Gabriel Gill-Couture bc2edf4530 feat(podman): init containers with k8s-style run-to-completion semantics
Customer apps frequently need a one-shot setup step (DB migration,
config render, cache warm-up) to succeed before the long-running
service starts. Without init containers each customer either inlines
the step into the service entrypoint (slow, racy, no failure surface)
or bolts on a sidecar that the platform can't introspect. This change
adds k8s-style init containers at the score layer so the contract is
the same one the customer already knows.

Score:
- New `InitContainer { name, image, args, env, volumes, timeout }`
  in `harmony::modules::podman`.
- `PodmanV0Score.init_containers: Vec<InitContainer>` with
  `#[serde(default)]` — pre-init-container wire payloads parse as an
  empty vec and behave unchanged.
- `DEFAULT_INIT_CONTAINER_TIMEOUT = 300s`; timeout serializes as
  whole seconds for operator readability.
- Idempotency is the customer's contract — documented at module
  level: init containers re-run on every reconcile that needs a
  fresh main container set.

Runtime contract:
- `ContainerRuntime::run_to_completion(spec, timeout) -> RunOutcome`
  added to the domain trait. `RunOutcome::Exited { exit_code }`
  vs `TimedOut { waited }` — distinct arms because the caller's
  failure path is different (operator gets the exit code for
  actionable diagnosis).
- Init containers are NOT surfaced via `list_managed_services`;
  they're removed after they exit so the host's managed-container
  surface stays bounded to long-running services.

PodmanTopology implementation:
- Pre-remove any prior container with the same name (retry-safe).
- Restart policy forced to `No` — a retrying init defeats the
  run-to-completion contract.
- `tokio::time::timeout` around `podman wait`; force-remove + return
  `TimedOut` on deadline.
- Single 200ms retry on inspect for the libpod race where state can
  briefly read `running` between `wait` returning and conmon writing
  the exit code.
- `INIT_CONTAINER_LABEL` on every init container so operators can
  `podman ps -a --filter label=...` to spot init failures.

Interpret:
- Init containers run sequentially before any service. Non-zero exit
  or timeout fails the deployment with a typed `InterpretError`
  carrying the container name + cause.
- Success message reports both counts.

Tests (in tree):
- 3 new wire-format tests in `podman::score`: roundtrip, default
  timeout hydration, ordering preservation.
- All 10 existing podman::score tests still pass; legacy roundtrip
  test now also asserts `init_containers.is_empty()` as a wire-compat
  canary.

Call-site updates (5 sites) — all existing constructors of
`PodmanV0Score` add `init_containers: vec![]`: harmony_apply_deployment
example, fleet_load_test example, operator e2e, vm_deploy_lifecycle
e2e, vm_isolation e2e.

Deferred: per-version "run-once" semantics (customer can build with a
marker file today); the agent-side handler for surfacing init logs to
the operator dashboard (covered by the logs companion PR's deferred
work).
2026-05-24 21:56:39 -04:00
..
2026-05-11 16:48:52 -04:00
2026-05-04 09:03:35 -04:00
2026-05-11 16:48:52 -04:00
2026-05-11 16:48:52 -04:00
2026-01-09 17:30:51 -05:00
2026-05-11 16:48:52 -04:00
2026-04-21 16:03:42 -04:00
2026-05-11 16:48:52 -04:00

Examples

This directory contains runnable examples demonstrating Harmony's capabilities. Each example is a self-contained program that can be run with cargo run -p example-<name>.

Quick Reference

Example Description Local K3D Existing Cluster Hardware Needed
postgresql Deploy a PostgreSQL cluster
ntfy Deploy ntfy notification server
tenant Create a multi-tenant namespace
cert_manager Provision TLS certificates
node_health Check Kubernetes node health
monitoring Deploy Prometheus alerting
monitoring_with_tenant Monitoring + tenant isolation
operatorhub_catalog Install OperatorHub catalog
validate_ceph_cluster_health Verify Ceph cluster health Rook/Ceph
remove_rook_osd Remove a Rook OSD Rook/Ceph
brocade_snmp_server Configure Brocade switch SNMP Brocade switch
opnsense_node_exporter Node exporter on OPNsense OPNsense firewall
opnsense_vm_integration Full OPNsense firewall automation (11 Scores) KVM/libvirt
opnsense_pair_integration OPNsense HA pair with CARP failover KVM/libvirt
okd_pxe PXE boot configuration for OKD
okd_installation Full OKD bare-metal install
okd_cluster_alerts OKD cluster monitoring alerts OKD cluster
multisite_postgres Multi-site PostgreSQL failover Multi-cluster
nats Deploy NATS messaging Multi-cluster
nats-supercluster NATS supercluster across sites Multi-cluster
lamp LAMP stack deployment
openbao Deploy OpenBao vault
zitadel Deploy Zitadel identity provider
try_rust_webapp Rust webapp with packaging Submodule
rust Rust webapp with full monitoring
rhob_application_monitoring RHOB monitoring setup
sttest Full OKD stack test
application_monitoring_with_tenant App monitoring + tenant OKD cluster
kube-rs Direct kube-rs client usage
k8s_drain_node Drain a Kubernetes node
k8s_write_file_on_node Write files to K8s nodes
harmony_inventory_builder Discover hosts via subnet scan
cli CLI tool with inventory discovery
tui Terminal UI demonstration

Status Legend

Symbol Meaning
Works out-of-the-box
Not applicable or requires specific setup

By Category

Data Services

  • postgresql — Deploy a PostgreSQL cluster via CloudNativePG
  • multisite_postgres — Multi-site PostgreSQL with failover
  • public_postgres — Public-facing PostgreSQL (⚠️ uses NationTech DNS)

Kubernetes Utilities

  • node_health — Check node health in a cluster
  • k8s_drain_node — Drain and reboot a node
  • k8s_write_file_on_node — Write files to nodes
  • validate_ceph_cluster_health — Verify Ceph/Rook cluster health
  • remove_rook_osd — Remove an OSD from Rook/Ceph
  • kube-rs — Direct Kubernetes client usage demo

Monitoring & Alerting

  • monitoring — Deploy Prometheus alerting with Discord webhooks
  • monitoring_with_tenant — Monitoring with tenant isolation
  • ntfy — Deploy ntfy notification server
  • okd_cluster_alerts — OKD-specific cluster alerts

Application Deployment

  • try_rust_webapp — Deploy a Rust webapp with packaging (⚠️ requires tryrust.org submodule)
  • rust — Rust webapp with full monitoring features
  • rhob_application_monitoring — Red Hat Observability Stack monitoring
  • lamp — LAMP stack deployment (⚠️ uses NationTech DNS)
  • application_monitoring_with_tenant — App monitoring with tenant isolation

Infrastructure & Bare Metal

  • opnsense_vm_integrationRecommended demo. Boot an OPNsense VM and configure it with 11 Scores (load balancer, DHCP, TFTP, VLANs, firewall rules, NAT, VIPs, LAGG). Fully automated, requires only KVM. See the detailed guide.
  • opnsense_pair_integration — Boot two OPNsense VMs and configure a CARP HA firewall pair with FirewallPairTopology and CarpVipScore. Demonstrates NIC link control for sequential bootstrap.
  • okd_installation — Full OKD cluster from scratch
  • okd_pxe — PXE boot configuration for OKD
  • sttest — Full OKD stack test with specific hardware
  • brocade_snmp_server — Configure Brocade switch via SNMP
  • opnsense_node_exporter — Node exporter on OPNsense firewall

Multi-Cluster

  • nats — NATS deployment on a cluster
  • nats-supercluster — NATS supercluster across multiple sites
  • multisite_postgres — PostgreSQL with multi-site failover

Identity & Secrets

  • openbao — Deploy OpenBao vault (⚠️ uses NationTech DNS)
  • zitadel — Deploy Zitadel identity provider (⚠️ uses NationTech DNS)

Cluster Services

  • cert_manager — Provision TLS certificates
  • tenant — Create a multi-tenant namespace
  • operatorhub_catalog — Install OperatorHub catalog sources

Development & Testing

  • cli — CLI tool with inventory discovery
  • tui — Terminal UI demonstration
  • harmony_inventory_builder — Host discovery via subnet scan

Running Examples

# Build first
cargo build --release

# Run any example
cargo run -p example-postgresql
cargo run -p example-ntfy
cargo run -p example-tenant

For examples that need an existing Kubernetes cluster:

export KUBECONFIG=/path/to/your/kubeconfig
export HARMONY_USE_LOCAL_K3D=false
export HARMONY_AUTOINSTALL=false

cargo run -p example-monitoring

Notes on Private Infrastructure

Some examples use NationTech-hosted infrastructure by default (DNS domains like *.nationtech.io, *.harmony.mcd). These are not suitable for public use without modification. See the Getting Started Guide for the recommended public examples.