diff --git a/examples/harmony_sso/README.md b/examples/harmony_sso/README.md index a6b60900..99767930 100644 --- a/examples/harmony_sso/README.md +++ b/examples/harmony_sso/README.md @@ -1,12 +1,12 @@ # Harmony SSO Example -Deploys Zitadel (identity provider) and OpenBao (secrets management) on a local k3d cluster, then demonstrates using them as `harmony_config` backends for shared config and secret management. +Deploys Zitadel (identity provider) and OpenBao (secrets management) on a local k3d cluster, then round-trips two `harmony_config` structs (Secret + Standard class) through OpenBao via SSO-authenticated access. ## Prerequisites - Docker running - Ports 8080 and 8200 free -- `/etc/hosts` entries (or use a local DNS resolver): +- `/etc/hosts` entries (or a local DNS resolver): ``` 127.0.0.1 sso.harmony.local 127.0.0.1 bao.harmony.local @@ -14,41 +14,19 @@ Deploys Zitadel (identity provider) and OpenBao (secrets management) on a local ## Usage -### Full deployment - ```bash -# Deploy everything (OpenBao + Zitadel) +# Deploy everything (OpenBao + Zitadel) and round-trip the example configs cargo run -p example-harmony-sso # OpenBao only (faster, skip Zitadel) cargo run -p example-harmony-sso -- --skip-zitadel -``` -### Config storage demo (token auth) - -After deployment, run the config demo to verify `harmony_config` works with OpenBao: - -```bash -cargo run -p example-harmony-sso -- --demo -``` - -This writes and reads a `SsoExampleConfig` through the `ConfigManager` chain (`EnvSource -> StoreSource`), demonstrating environment variable overrides and persistent storage in OpenBao KV v2. - -### SSO device flow demo - -Requires a Zitadel application configured for device code grant: - -```bash -HARMONY_SSO_CLIENT_ID= \ - cargo run -p example-harmony-sso -- --sso-demo -``` - -### Cleanup - -```bash +# Teardown cargo run -p example-harmony-sso -- --cleanup ``` +The example sets `HARMONY_SECRET_STORE=file` and `HARMONY_SECRET_NAMESPACE=harmony-sso-example` at startup so the global `harmony_secret` manager uses the local-file backend (`~/.local/share/harmony/secrets/`). Override either in your shell to change behaviour. + ## What gets deployed | Component | Namespace | Access | @@ -62,15 +40,60 @@ cargo run -p example-harmony-sso -- --cleanup - **Secrets engine:** KV v2 at `secret/` - **Policy:** `harmony-dev` grants CRUD on `secret/data/harmony/*` - **Userpass credentials:** `harmony` / `harmony-dev-password` -- **JWT auth:** configured with Zitadel as OIDC provider, role `harmony-developer` -- **Unseal keys:** saved to `~/.local/share/harmony/openbao/unseal-keys.json` +- **JWT auth:** Zitadel as OIDC provider, role `harmony-developer` +- **Unseal keys:** `~/.local/share/harmony/openbao/unseal-keys.json` +- **Audit device:** file audit at `/openbao/audit/audit.log` (see below) + +## Audit log: who changed what + +OpenBao's KV v2 version history shows *when* a value changed and which version, but never *who* — the caller identity isn't in KV metadata. It lives in the **audit log**. + +A file audit device is enabled at `/openbao/audit/audit.log` (`auditStorage` PVC), logging every request/response with the authenticated identity. SSO mints the OpenBao token from the Zitadel id_token with `user_claim=email`, so each operation is attributed to the user. + +**Applying audit config changes:** the device loads at OpenBao startup, and the helm chart's StatefulSet uses `updateStrategy: OnDelete` (a Vault/OpenBao restart needs a manual unseal — auto-rolling is intentionally disabled). So `helm upgrade` updates the server config but does *not* recreate the pod; `kubectl rollout restart` reports success but does nothing here. Delete the pod yourself and re-run to unseal: + +```bash +kubectl delete pod openbao-0 -n openbao +cargo run -p example-harmony-sso # unseals the fresh pod +``` + +Reading the audit needs the **root token** (the `harmony-dev` policy can't touch `sys/audit`): + +```bash +ROOT=$(jq -r .root_token ~/.local/share/harmony/openbao/unseal-keys.json) + +# Confirm device is on: +kubectl exec -n openbao openbao-0 -- sh -c "export VAULT_TOKEN=$ROOT && bao audit list" +# Path Type Description +# file/ file n/a +``` + +(`bao audit list` with no token returns `403 permission denied` — that means "you didn't authenticate", not "audit is off".) + +See who performed each operation. Paths and values are HMAC-hashed by default, so filter on the *identity* fields: + +```bash +kubectl exec -n openbao openbao-0 -- cat /openbao/audit/audit.log | \ + jq -c 'select(.type == "response" and .auth.display_name != null) + | {time, op: .request.operation, + who: .auth.display_name, role: .auth.metadata.role, + entity: .auth.entity_id}' | tail -n 15 +``` + +SSO-authenticated operations are attributed to the user: + +``` +{"time":"...","op":"update","who":"jwt-admin@zitadel.sso.harmony.local","role":"harmony-developer","entity":"81ddf1d7-..."} +``` + +The `jwt-` prefix is the JWT auth-mount name; the rest is the `user_claim=email` value from the id_token. Setup/unseal steps run with the root token and show as `"who":"root"`. For raw paths/values, tune the audit device (`log_raw`, `hmac_accessor`) — out of scope here. ## Architecture ``` Developer CLI | - |-- harmony_config::ConfigManager + |-- harmony_config::ConfigClient | |-- EnvSource (HARMONY_CONFIG_* env vars) | |-- StoreSource | |-- Token auth (OPENBAO_TOKEN) diff --git a/examples/harmony_sso/harmony_sso_plan.md b/examples/harmony_sso/harmony_sso_plan.md index 34f6057f..27db201e 100644 --- a/examples/harmony_sso/harmony_sso_plan.md +++ b/examples/harmony_sso/harmony_sso_plan.md @@ -11,7 +11,7 @@ Deploy Zitadel and OpenBao on a local k3d cluster, use them as `harmony_config` - [x] A.1 -- CLI argument parsing (`--demo`, `--sso-demo`, `--skip-zitadel`, `--cleanup`) - [x] A.2 -- Zitadel deployment via `ZitadelScore` (`external_secure: false` for k3d) - [x] A.3 -- OpenBao JWT auth method + `harmony-dev` policy configuration -- [x] A.4 -- `--demo` flag: config storage demo with token auth via `ConfigManager` +- [x] A.4 -- `--demo` flag: config storage demo with token auth via `ConfigClient` - [x] A.5 -- Hardening: retry loops for pod readiness, HTTP readiness checks, `--cleanup` - [x] A.6 -- README with prerequisites, usage, and architecture @@ -35,9 +35,11 @@ The Zitadel OIDC device flow code exists (`harmony_secret/src/store/zitadel.rs`) - `OPENBAO_JWT_AUTH_MOUNT` (default: `jwt`) - `OPENBAO_JWT_ROLE` (default: `harmony-developer`) -**B.4 -- Silent refresh:** -- Add `refresh_token()` method to `ZitadelOidcAuth` -- Update auth chain in `openbao.rs`: cached session -> silent refresh -> device flow +**B.4 -- Silent refresh -- DONE:** +- `ZitadelOidcAuth::silent_refresh()` + `refresh_id_token()` added in `harmony_secret/src/store/zitadel.rs` +- `authenticate()` chain is now: cached session (valid) -> silent refresh (when expired + refresh_token present) -> device flow +- Refresh response that omits a new `refresh_token` preserves the previous one (OAuth allows rotation to be optional) +- Refreshed session is persisted to `oidc_session_*` so subsequent runs pick up the new tokens **B.5 -- `--sso-demo` flag:** - Already stubbed in `examples/harmony_sso/src/main.rs` @@ -56,7 +58,7 @@ The Zitadel OIDC device flow code exists (`harmony_secret/src/store/zitadel.rs`) - `test_openbao_health` -- health endpoint - `test_zitadel_openid_config` -- OIDC discovery - `test_openbao_userpass_auth` -- write/read secret -- `test_config_manager_openbao_backend` -- full ConfigManager chain +- `test_config_manager_openbao_backend` -- full ConfigClient chain - `test_openbao_jwt_auth_configured` -- verify JWT auth method + role exist **C.2 -- Zitadel application automation** (`examples/harmony_sso/src/zitadel_setup.rs`): @@ -145,7 +147,7 @@ The user flagged several areas that should use `harmony-k8s` instead of raw `kub - `cargo run -p example-harmony-sso` -> deploys k3d + OpenBao + Zitadel (with retry for CNPG CRD + PG readiness) - `curl -H "Host: bao.harmony.local" http://127.0.0.1:8080/v1/sys/health` -> OpenBao healthy (initialized, unsealed) - `curl -H "Host: sso.harmony.local" http://127.0.0.1:8080/.well-known/openid-configuration` -> Zitadel OIDC config with device_authorization_endpoint -- `cargo run -p example-harmony-sso -- --demo` -> writes/reads config via ConfigManager + OpenbaoSecretStore, env override works +- `cargo run -p example-harmony-sso -- --demo` -> writes/reads config via ConfigClient + OpenbaoSecretStore, env override works **Phase B:** - `HARMONY_SSO_URL=http://sso.harmony.local HARMONY_SSO_CLIENT_ID= cargo run -p example-harmony-sso -- --sso-demo` diff --git a/examples/harmony_sso/src/main.rs b/examples/harmony_sso/src/main.rs index 1fe895b3..2a4aa9c8 100644 --- a/examples/harmony_sso/src/main.rs +++ b/examples/harmony_sso/src/main.rs @@ -10,7 +10,7 @@ use harmony::modules::zitadel::{ }; use harmony::score::Score; use harmony::topology::{K8sclient, Topology}; -use harmony_config::{Config, ConfigClient, EnvSource, StoreSource}; +use harmony_config::{Config, ConfigClass, ConfigClient, EnvSource, StoreSource}; use harmony_k8s::K8sClient; use harmony_secret::OpenbaoSecretStore; use k3d_rs::{K3d, PortMapping}; @@ -44,15 +44,13 @@ struct Args { cleanup: bool, } -// --------------------------------------------------------------------------- -// Config type stored via SSO-authenticated OpenBao -// --------------------------------------------------------------------------- - -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq)] +#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] struct SsoExampleConfig { team_name: String, environment: String, max_replicas: u16, + #[config(secret)] + deploy_token: String, } impl Default for SsoExampleConfig { @@ -61,17 +59,69 @@ impl Default for SsoExampleConfig { team_name: "platform-team".to_string(), environment: "staging".to_string(), max_replicas: 3, + deploy_token: "dev-token-replace-me".to_string(), } } } -impl Config for SsoExampleConfig { - const KEY: &'static str = "SsoExampleConfig"; +#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] +struct TeamMetadataConfig { + display_name: String, + slack_channel: String, + oncall_schedule: String, } -// --------------------------------------------------------------------------- -// Helpers -// --------------------------------------------------------------------------- +impl Default for TeamMetadataConfig { + fn default() -> Self { + Self { + display_name: "Platform Team".to_string(), + slack_channel: "#platform".to_string(), + oncall_schedule: "platform-primary".to_string(), + } + } +} + +async fn round_trip(manager: &ConfigClient, default_value: T) -> anyhow::Result<()> +where + T: Config + std::fmt::Debug + Clone + PartialEq, +{ + match manager.get::().await { + Ok(found) => { + info!( + "[{key}] loaded existing value (class={class:?}): {found:?}", + key = T::KEY, + class = T::CLASS, + ); + Ok(()) + } + Err(harmony_config::ConfigError::NotFound { .. }) => { + info!( + "[{key}] not found, storing default (class={class:?})", + key = T::KEY, + class = T::CLASS, + ); + manager.set(&default_value).await?; + let retrieved: T = manager.get().await?; + anyhow::ensure!( + retrieved == default_value, + "Round-trip mismatch for {}: stored != retrieved", + T::KEY + ); + info!("[{key}] round-trip verified", key = T::KEY); + Ok(()) + } + Err(e) => Err(e.into()), + } +} + +fn set_env_default(key: &str, value: &str) { + if std::env::var_os(key).is_none() { + // SAFETY: called from main() before any tokio task is spawned. + unsafe { + std::env::set_var(key, value); + } + } +} fn harmony_data_dir() -> PathBuf { directories::BaseDirs::new() @@ -90,6 +140,8 @@ fn create_topology(k3d: &K3d) -> harmony::topology::K8sAnywhereTopology { let context = k3d .context_name() .unwrap_or_else(|| format!("k3d-{}", CLUSTER_NAME)); + // SAFETY: called from main() before any tokio task is spawned; the topology + // reads these vars immediately below via from_env(). unsafe { std::env::set_var("HARMONY_USE_LOCAL_K3D", "false"); std::env::set_var("HARMONY_AUTOINSTALL", "false"); @@ -98,19 +150,24 @@ fn create_topology(k3d: &K3d) -> harmony::topology::K8sAnywhereTopology { harmony::topology::K8sAnywhereTopology::from_env() } -fn harmony_dev_policy() -> OpenbaoPolicy { - OpenbaoPolicy { - name: "harmony-dev".to_string(), - hcl: r#"path "secret/data/harmony/*" { capabilities = ["create","read","update","delete","list"] } +fn openbao_setup(jwt: Option) -> OpenbaoSetupScore { + OpenbaoSetupScore { + policies: vec![OpenbaoPolicy { + name: "harmony-dev".to_string(), + hcl: r#"path "secret/data/harmony/*" { capabilities = ["create","read","update","delete","list"] } path "secret/metadata/harmony/*" { capabilities = ["list","read"] }"# - .to_string(), + .to_string(), + }], + users: vec![OpenbaoUser { + username: "harmony".to_string(), + password: "harmony-dev-password".to_string(), + policies: vec!["harmony-dev".to_string()], + }], + jwt_auth: jwt, + ..Default::default() } } -// --------------------------------------------------------------------------- -// Zitadel deployment (with CNPG retry) -// --------------------------------------------------------------------------- - async fn deploy_zitadel(k3d: &K3d) -> anyhow::Result<()> { info!("Deploying Zitadel (this may take several minutes)..."); @@ -171,10 +228,6 @@ async fn wait_for_zitadel_ready() -> anyhow::Result<()> { anyhow::bail!("Timed out waiting for Zitadel") } -// --------------------------------------------------------------------------- -// Cluster lifecycle -// --------------------------------------------------------------------------- - async fn ensure_k3d_cluster(k3d: &K3d) -> anyhow::Result<()> { info!("Ensuring k3d cluster '{}' is running...", CLUSTER_NAME); k3d.ensure_installed() @@ -209,13 +262,16 @@ async fn cleanup_openbao_webhook(k8s: &K8sClient) -> anyhow::Result<()> { Ok(()) } -// --------------------------------------------------------------------------- -// Main -// --------------------------------------------------------------------------- - #[tokio::main] async fn main() -> anyhow::Result<()> { env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init(); + + // harmony_secret defaults to Infisical and panics if its env vars are missing. + // Set the file-backend defaults BEFORE the lazy SECRET_MANAGER touches them; + // an operator override from the shell still wins. + set_env_default("HARMONY_SECRET_STORE", "file"); + set_env_default("HARMONY_SECRET_NAMESPACE", "harmony-sso-example"); + let args = Args::parse(); let k3d = create_k3d(); @@ -223,11 +279,7 @@ async fn main() -> anyhow::Result<()> { return cleanup_cluster(&k3d); } - info!("==========================================="); - info!("Harmony SSO Example"); - info!("==========================================="); - - // --- Phase 1: Infrastructure --- + info!("=== Harmony SSO Example ==="); ensure_k3d_cluster(&k3d).await?; @@ -242,7 +294,6 @@ async fn main() -> anyhow::Result<()> { .await .map_err(|e| anyhow::anyhow!("K8s client: {}", e))?; - // Deploy + configure OpenBao (no JWT auth yet -- Zitadel isn't up) cleanup_openbao_webhook(&k8s).await?; OpenbaoScore { host: OPENBAO_HOST.to_string(), @@ -252,28 +303,16 @@ async fn main() -> anyhow::Result<()> { .await .context("OpenBao deploy failed")?; - OpenbaoSetupScore { - policies: vec![harmony_dev_policy()], - users: vec![OpenbaoUser { - username: "harmony".to_string(), - password: "harmony-dev-password".to_string(), - policies: vec!["harmony-dev".to_string()], - }], - jwt_auth: None, // Phase 2 adds JWT after Zitadel is ready - ..Default::default() - } - .interpret(&Inventory::autoload(), &topology) - .await - .context("OpenBao setup failed")?; + openbao_setup(None) + .interpret(&Inventory::autoload(), &topology) + .await + .context("OpenBao setup failed")?; if args.skip_zitadel { - info!("=== Skipping Zitadel (--skip-zitadel) ==="); - info!("OpenBao: http://{}:{}", OPENBAO_HOST, HTTP_PORT); + info!("Skipping Zitadel (--skip-zitadel). OpenBao: http://{OPENBAO_HOST}:{HTTP_PORT}"); return Ok(()); } - // --- Phase 2: Identity + SSO Wiring --- - CoreDNSRewriteScore { rewrites: vec![ CoreDNSRewrite { @@ -293,7 +332,6 @@ async fn main() -> anyhow::Result<()> { deploy_zitadel(&k3d).await?; wait_for_zitadel_ready().await?; - // Provision Zitadel project + device-code application ZitadelSetupScore { host: ZITADEL_HOST.to_string(), scheme: Default::default(), @@ -315,7 +353,6 @@ async fn main() -> anyhow::Result<()> { .await .context("Zitadel setup failed")?; - // Read the client_id from the cache written by ZitadelSetupScore let zitadel_config = ZitadelClientConfig::load().context("ZitadelSetupScore did not produce a client config")?; let client_id = zitadel_config @@ -323,37 +360,23 @@ async fn main() -> anyhow::Result<()> { .context("No client_id for harmony-cli app")? .clone(); - info!("Zitadel app '{}' client_id: {}", APP_NAME, client_id); + info!("Zitadel app '{APP_NAME}' client_id: {client_id}"); - // Now configure OpenBao JWT auth with the real client_id - OpenbaoSetupScore { - policies: vec![harmony_dev_policy()], - users: vec![OpenbaoUser { - username: "harmony".to_string(), - password: "harmony-dev-password".to_string(), - policies: vec!["harmony-dev".to_string()], - }], - jwt_auth: Some(OpenbaoJwtAuth { - oidc_discovery_url: format!("http://{}:{}", ZITADEL_HOST, HTTP_PORT), - bound_issuer: format!("http://{}:{}", ZITADEL_HOST, HTTP_PORT), - role_name: "harmony-developer".to_string(), - bound_audiences: client_id.clone(), - user_claim: "email".to_string(), - policies: vec!["harmony-dev".to_string()], - ttl: "4h".to_string(), - max_ttl: "24h".to_string(), - }), - ..Default::default() - } + openbao_setup(Some(OpenbaoJwtAuth { + oidc_discovery_url: format!("http://{ZITADEL_HOST}:{HTTP_PORT}"), + bound_issuer: format!("http://{ZITADEL_HOST}:{HTTP_PORT}"), + role_name: "harmony-developer".to_string(), + bound_audiences: client_id.clone(), + user_claim: "email".to_string(), + policies: vec!["harmony-dev".to_string()], + ttl: "4h".to_string(), + max_ttl: "24h".to_string(), + })) .interpret(&Inventory::autoload(), &topology) .await .context("OpenBao JWT auth setup failed")?; - // --- Phase 3: Config via SSO --- - - info!("==========================================="); - info!("Storing config via SSO-authenticated OpenBao"); - info!("==========================================="); + info!("=== Storing config via SSO-authenticated OpenBao ==="); let _pf = k8s .port_forward(OPENBAO_POD, OPENBAO_NAMESPACE, 8200, 8200) @@ -362,7 +385,7 @@ async fn main() -> anyhow::Result<()> { tokio::time::sleep(tokio::time::Duration::from_secs(1)).await; let openbao_url = format!("http://127.0.0.1:{}", _pf.port()); - let sso_url = format!("http://{}:{}", ZITADEL_HOST, HTTP_PORT); + let sso_url = format!("http://{ZITADEL_HOST}:{HTTP_PORT}"); let store = OpenbaoSecretStore::new( openbao_url, @@ -385,31 +408,16 @@ async fn main() -> anyhow::Result<()> { Arc::new(StoreSource::new("harmony".to_string(), store)), ]); - // Try to load existing config (succeeds on re-run) - match manager.get::().await { - Ok(config) => { - info!("Config loaded from OpenBao: {:?}", config); - } - Err(harmony_config::ConfigError::NotFound { .. }) => { - info!("No config found, storing default..."); - let config = SsoExampleConfig::default(); - manager.set(&config).await?; - info!("Config stored: {:?}", config); + assert_eq!(SsoExampleConfig::CLASS, ConfigClass::Secret); + assert_eq!(TeamMetadataConfig::CLASS, ConfigClass::Standard); - let retrieved: SsoExampleConfig = manager.get().await?; - info!("Config verified: {:?}", retrieved); - assert_eq!(config, retrieved); - } - Err(e) => return Err(e.into()), - } + round_trip::(&manager, SsoExampleConfig::default()).await?; + round_trip::(&manager, TeamMetadataConfig::default()).await?; - info!("==========================================="); - info!("Success! Config managed via Zitadel SSO + OpenBao"); - info!("==========================================="); - info!("OpenBao: http://{}:{}", OPENBAO_HOST, HTTP_PORT); - info!("Zitadel: http://{}:{}", ZITADEL_HOST, HTTP_PORT); - info!("Run again to verify cached session works."); - info!("cargo run -p example-harmony-sso -- --cleanup # teardown"); + info!("=== Success — config managed via Zitadel SSO + OpenBao ==="); + info!("OpenBao: http://{OPENBAO_HOST}:{HTTP_PORT}"); + info!("Zitadel: http://{ZITADEL_HOST}:{HTTP_PORT}"); + info!("Run again to verify cached session; --cleanup to teardown."); Ok(()) }