diff --git a/ROADMAP/01-config-crate.md b/ROADMAP/01-config-crate.md index 9829cd16..fbbcf945 100644 --- a/ROADMAP/01-config-crate.md +++ b/ROADMAP/01-config-crate.md @@ -9,7 +9,7 @@ Make `harmony_config` production-ready with a seamless first-run experience: clo `harmony_config` now has: - `Config` trait + `#[derive(Config)]` macro -- `ConfigManager` with ordered source chain +- `ConfigClient` with ordered source chain - Five `ConfigSource` implementations: - `EnvSource` — reads `HARMONY_CONFIG_{KEY}` env vars - `LocalFileSource` — reads/writes `{key}.json` files from a directory @@ -140,7 +140,7 @@ for source in &self.sources { ┌─────────────────────────────────────────────────────────────────────┐ │ Harmony CLI / App │ │ │ -│ ConfigManager: │ +│ ConfigClient: │ │ 1. EnvSource ← HARMONY_CONFIG_* env vars (highest priority) │ │ 2. SqliteSource ← ~/.local/share/harmony/config/config.db │ │ 3. StoreSource ← OpenBao (team-scale, via Zitadel OIDC) │ @@ -448,11 +448,11 @@ The example uses `StoreSource` with token auth to avoid the | `OPENBAO_KV_MOUNT` | No | `"secret"` | KV v2 engine mount path. **Also used as userpass auth mount -- this is a bug.** | | `OPENBAO_SKIP_TLS` | No | `false` | Set `"true"` to disable TLS verification | -**Note**: `OpenbaoSecretStore::new()` is `async` and **requires a running OpenBao** at construction time (it validates the token if using cached auth). If OpenBao is unreachable during construction, the call will fail. The graceful fallback only applies to `StoreSource::get()` calls after construction -- the `ConfigManager` must be built with a live store, or the store must be wrapped in a lazy initialization pattern. +**Note**: `OpenbaoSecretStore::new()` is `async` and **requires a running OpenBao** at construction time (it validates the token if using cached auth). If OpenBao is unreachable during construction, the call will fail. The graceful fallback only applies to `StoreSource::get()` calls after construction -- the `ConfigClient` must be built with a live store, or the store must be wrapped in a lazy initialization pattern. ```rust // harmony_config/examples/openbao_chain.rs -use harmony_config::{ConfigManager, EnvSource, SqliteSource, StoreSource}; +use harmony_config::{ConfigClient, EnvSource, SqliteSource, StoreSource}; use harmony_secret::OpenbaoSecretStore; use serde::{Deserialize, Serialize}; use std::sync::Arc; @@ -517,7 +517,7 @@ async fn main() -> anyhow::Result<()> { vec![env_source, sqlite] }; - let manager = ConfigManager::new(sources); + let manager = ConfigClient::new(sources); // Scenario 1: get() with nothing stored -- returns NotFound let result = manager.get::().await; @@ -620,4 +620,4 @@ match self.store.get_raw(&self.namespace, key).await { 6. **Graceful fallback**: `StoreSource::get()` returns `Ok(None)` on any error (connection refused, timeout, etc.), allowing the chain to fall through to the next source. This ensures OpenBao unavailability doesn't break the config chain. -7. **StoreSource errors don't block chain**: When OpenBao is unreachable, `StoreSource::get()` returns `Ok(None)` and the `ConfigManager` continues to the next source (typically `SqliteSource`). This is validated by `test_store_source_error_falls_through_to_sqlite` and `test_store_source_not_found_falls_through_to_sqlite`. +7. **StoreSource errors don't block chain**: When OpenBao is unreachable, `StoreSource::get()` returns `Ok(None)` and the `ConfigClient` continues to the next source (typically `SqliteSource`). This is validated by `test_store_source_error_falls_through_to_sqlite` and `test_store_source_not_found_falls_through_to_sqlite`. diff --git a/docs/adr/020-1-zitadel-openbao-secure-config-store.md b/docs/adr/020-1-zitadel-openbao-secure-config-store.md index cac11576..51ef401c 100644 --- a/docs/adr/020-1-zitadel-openbao-secure-config-store.md +++ b/docs/adr/020-1-zitadel-openbao-secure-config-store.md @@ -211,11 +211,13 @@ The `bound_audiences` claim ties the role to the specific Harmony Zitadel applic For organizations running their own infrastructure, the same architecture applies. The operator deploys Zitadel and OpenBao using Harmony's existing `ZitadelScore` and `OpenbaoScore`. The only configuration needed is three environment variables (or their equivalents in the bootstrap config): - `HARMONY_SSO_URL` — the Zitadel instance URL. -- `HARMONY_SECRETS_URL` — the OpenBao instance URL. +- `OPENBAO_URL` (or `VAULT_ADDR`) — the OpenBao instance URL. - `HARMONY_SSO_CLIENT_ID` — the Zitadel application client ID. None of these are secrets. They can be committed to an infrastructure repository or distributed via any convenient channel. +`OPENBAO_URL`/`VAULT_ADDR` is named after the backend on purpose: it is read only by the OpenBao adapter, never by domain or Score code, and reusing the standard Vault/OpenBao variable names gives operators interop with existing tooling. The tool-agnostic seam lives one layer up — `HARMONY_SECRET_STORE` selects which backend is active, and each backend then reads its own connection params (e.g. `HARMONY_SECRET_INFISICAL_*`). An earlier draft of this ADR proposed a single agnostic `HARMONY_SECRETS_URL`, but it was never implemented: different stores don't share a connection shape (Infisical needs a project and client credentials; AWS Secrets Manager has no URL at all), so one "secrets URL" would force OpenBao's model onto every backend. + ## Consequences ### Positive diff --git a/examples/harmony_sso/src/main.rs b/examples/harmony_sso/src/main.rs index 8bd09a59..1fe895b3 100644 --- a/examples/harmony_sso/src/main.rs +++ b/examples/harmony_sso/src/main.rs @@ -10,7 +10,7 @@ use harmony::modules::zitadel::{ }; use harmony::score::Score; use harmony::topology::{K8sclient, Topology}; -use harmony_config::{Config, ConfigManager, EnvSource, StoreSource}; +use harmony_config::{Config, ConfigClient, EnvSource, StoreSource}; use harmony_k8s::K8sClient; use harmony_secret::OpenbaoSecretStore; use k3d_rs::{K3d, PortMapping}; @@ -380,7 +380,7 @@ async fn main() -> anyhow::Result<()> { .await .context("SSO authentication failed")?; - let manager = ConfigManager::new(vec![ + let manager = ConfigClient::new(vec![ Arc::new(EnvSource) as Arc, Arc::new(StoreSource::new("harmony".to_string(), store)), ]); diff --git a/harmony/src/domain/topology/firewall_pair.rs b/harmony/src/domain/topology/firewall_pair.rs index 09cecc5a..76bbb53f 100644 --- a/harmony/src/domain/topology/firewall_pair.rs +++ b/harmony/src/domain/topology/firewall_pair.rs @@ -62,7 +62,7 @@ impl FirewallPairTopology { pub async fn opnsense_from_config() -> Self { // TODO: both firewalls share the same credentials. Once named config // instances are available (ROADMAP/11), use per-device credentials: - // ConfigManager::get_named::("fw-primary") + // ConfigClient::get_named::("fw-primary") let ssh_creds = SecretManager::get_or_prompt::() .await .expect("Failed to get SSH credentials"); diff --git a/harmony_config/examples/basic.rs b/harmony_config/examples/basic.rs index 2b192492..f4560612 100644 --- a/harmony_config/examples/basic.rs +++ b/harmony_config/examples/basic.rs @@ -12,7 +12,7 @@ use std::sync::Arc; -use harmony_config::{Config, ConfigManager, EnvSource, SqliteSource}; +use harmony_config::{Config, ConfigClient, EnvSource, SqliteSource}; use log::info; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; @@ -36,8 +36,10 @@ impl Default for TestConfig { async fn main() -> anyhow::Result<()> { env_logger::init(); - let sqlite = SqliteSource::default().await?; - let manager = ConfigManager::new(vec![Arc::new(EnvSource), Arc::new(sqlite)]); + // Namespace the SQLite file so this example's state doesn't + // collide with other harmony binaries that also use SqliteSource. + let sqlite = SqliteSource::for_namespace("harmony_config-basic-example").await?; + let manager = ConfigClient::new(vec![Arc::new(EnvSource), Arc::new(sqlite)]); info!("1. Attempting to get TestConfig (expect NotFound on first run)..."); match manager.get::().await { diff --git a/harmony_config/examples/openbao_chain.rs b/harmony_config/examples/openbao_chain.rs index 84fd8870..23f8dccc 100644 --- a/harmony_config/examples/openbao_chain.rs +++ b/harmony_config/examples/openbao_chain.rs @@ -1,60 +1,24 @@ -//! End-to-end example: harmony_config with OpenBao as a ConfigSource +//! Dev-binary template: `ConfigClient` against an OpenBao-backed chain. //! -//! This example demonstrates the full config resolution chain: -//! EnvSource → SqliteSource → StoreSource +//! Bring up the OpenBao + Zitadel stack first (`cargo run -p +//! example-harmony-sso`), then export the connection env vars and run this: //! -//! When OpenBao is unreachable, the chain gracefully falls through to SQLite. -//! -//! **Prerequisites**: -//! - OpenBao must be initialized and unsealed -//! - KV v2 engine must be enabled at the `OPENBAO_KV_MOUNT` path (default: `secret`) -//! - Auth method must be enabled at the `OPENBAO_AUTH_MOUNT` path (default: `userpass`) -//! -//! **Environment variables**: -//! - `OPENBAO_URL` (required for OpenBao): URL of the OpenBao server -//! - `OPENBAO_TOKEN` (optional): Use token auth instead of userpass -//! - `OPENBAO_USERNAME` + `OPENBAO_PASSWORD` (optional): Userpass auth -//! - `OPENBAO_KV_MOUNT` (default: `secret`): KV v2 engine mount path -//! - `OPENBAO_AUTH_MOUNT` (default: `userpass`): Auth method mount path -//! - `OPENBAO_SKIP_TLS` (default: `false`): Skip TLS verification -//! - `HARMONY_SSO_URL` + `HARMONY_SSO_CLIENT_ID` (optional): Zitadel OIDC device flow (RFC 8628) -//! -//! **Run**: //! ```bash -//! # Without OpenBao (SqliteSource only): -//! cargo run --example openbao_chain -//! -//! # With OpenBao (full chain): -//! export OPENBAO_URL="http://127.0.0.1:8200" -//! export OPENBAO_TOKEN="" -//! cargo run --example openbao_chain +//! export OPENBAO_URL=http://bao.harmony.local:8080 +//! export HARMONY_SSO_URL=http://sso.harmony.local:8080 +//! export HARMONY_SSO_CLIENT_ID= # or OPENBAO_TOKEN= +//! cargo run -p harmony_config --example openbao_chain //! ``` //! -//! **Setup OpenBao** (if needed): -//! ```bash -//! # Port-forward to local OpenBao -//! kubectl port-forward svc/openbao -n openbao 8200:8200 & -//! -//! # Initialize (one-time) -//! kubectl exec -n openbao openbao-0 -- bao operator init -//! -//! # Enable KV and userpass (one-time) -//! kubectl exec -n openbao openbao-0 -- bao secrets enable -path=secret kv-v2 -//! kubectl exec -n openbao openbao-0 -- bao auth enable userpass -//! -//! # Create test user -//! kubectl exec -n openbao openbao-0 -- bao write auth/userpass/users/testuser \ -//! password="testpass" policies="default" -//! ``` +//! If OpenBao is unreachable the chain degrades to env → prompt, so the +//! round-trip steps below need a reachable OpenBao to persist. -use std::sync::Arc; - -use harmony_config::{Config, ConfigManager, ConfigSource, EnvSource, SqliteSource, StoreSource}; -use harmony_secret::OpenbaoSecretStore; +use harmony_config::{Config, ConfigClient, ConfigExt}; +use log::info; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; -#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq)] +#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] struct AppConfig { host: String, port: u16, @@ -63,130 +27,70 @@ struct AppConfig { impl Default for AppConfig { fn default() -> Self { Self { - host: "localhost".to_string(), - port: 8080, + host: "production.example.com".to_string(), + port: 443, } } } -impl Config for AppConfig { - const KEY: &'static str = "AppConfig"; +// `password` is `#[config(secret)]`, so the whole struct is Secret-class: +// masked in logs and prompted via `inquire::Password`. +#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] +struct DatabaseCredentials { + host: String, + username: String, + #[config(secret)] + password: String, } -async fn build_manager() -> ConfigManager { - let sqlite = Arc::new( - SqliteSource::default() - .await - .expect("Failed to open SQLite database"), - ); - - let env_source: Arc = Arc::new(EnvSource); - - let openbao_url = std::env::var("OPENBAO_URL") - .or_else(|_| std::env::var("VAULT_ADDR")) - .ok(); - - match openbao_url { - Some(url) => { - let kv_mount = - std::env::var("OPENBAO_KV_MOUNT").unwrap_or_else(|_| "secret".to_string()); - let auth_mount = - std::env::var("OPENBAO_AUTH_MOUNT").unwrap_or_else(|_| "userpass".to_string()); - let skip_tls = std::env::var("OPENBAO_SKIP_TLS") - .map(|v| v == "true") - .unwrap_or(false); - - match OpenbaoSecretStore::new( - url, - kv_mount, - auth_mount, - skip_tls, - std::env::var("OPENBAO_TOKEN").ok(), - std::env::var("OPENBAO_USERNAME").ok(), - std::env::var("OPENBAO_PASSWORD").ok(), - std::env::var("HARMONY_SSO_URL").ok(), - std::env::var("HARMONY_SSO_CLIENT_ID").ok(), - None, - None, - ) - .await - { - Ok(store) => { - let store_source: Arc = - Arc::new(StoreSource::new("harmony".to_string(), store)); - println!("OpenBao connected. Full chain: env → sqlite → openbao"); - ConfigManager::new(vec![env_source, Arc::clone(&sqlite) as _, store_source]) - } - Err(e) => { - eprintln!( - "Warning: OpenBao unavailable ({e}), using local chain: env → sqlite" - ); - ConfigManager::new(vec![env_source, sqlite]) - } - } - } - None => { - println!("OPENBAO_URL not set. Using local chain: env → sqlite"); - ConfigManager::new(vec![env_source, sqlite]) +impl Default for DatabaseCredentials { + fn default() -> Self { + Self { + host: "db.example.com".to_string(), + username: "app_rw".to_string(), + password: "rotate-me-please".to_string(), } } } +// No `Default`, so `get_or_prompt` falls through to an interactive prompt. +#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] +struct ApiCredentials { + client_id: String, + #[config(secret)] + client_secret: String, +} + +// Read `T`, or store `default` and read it back to prove the round-trip. +async fn round_trip(client: &ConfigClient, default: T) -> anyhow::Result<()> +where + T: Config + std::fmt::Debug + Clone + PartialEq, +{ + match client.get::().await { + Ok(found) => info!("[{}] read existing: {:?}", T::KEY, found.masked()), + Err(harmony_config::ConfigError::NotFound { .. }) => { + client.set(&default).await?; + let back: T = client.get().await?; + anyhow::ensure!(back == default, "round-trip mismatch for {}", T::KEY); + info!("[{}] stored + verified: {:?}", T::KEY, back.masked()); + } + Err(e) => return Err(e.into()), + } + Ok(()) +} + #[tokio::main] async fn main() -> anyhow::Result<()> { - env_logger::init(); + env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init(); - let manager = build_manager().await; + let client = ConfigClient::for_namespace("harmony").await; - println!("\n=== harmony_config OpenBao Chain Demo ===\n"); + round_trip(&client, AppConfig::default()).await?; + round_trip(&client, DatabaseCredentials::default()).await?; - println!("1. Attempting to get AppConfig (expect NotFound on first run)..."); - match manager.get::().await { - Ok(config) => { - println!(" Found: {:?}", config); - } - Err(harmony_config::ConfigError::NotFound { .. }) => { - println!(" NotFound - no config stored yet"); - } - Err(e) => { - println!(" Error: {:?}", e); - } - } - - println!("\n2. Setting AppConfig via set()..."); - let config = AppConfig { - host: "production.example.com".to_string(), - port: 443, - }; - manager.set(&config).await?; - println!(" Set: {:?}", config); - - println!("\n3. Getting AppConfig back..."); - let retrieved: AppConfig = manager.get().await?; - println!(" Retrieved: {:?}", retrieved); - assert_eq!(config, retrieved); - - println!("\n4. Demonstrating env override..."); - println!(" HARMONY_CONFIG_AppConfig env var overrides all backends"); - let env_config = AppConfig { - host: "env-override.example.com".to_string(), - port: 9090, - }; - unsafe { - std::env::set_var( - "HARMONY_CONFIG_AppConfig", - serde_json::to_string(&env_config)?, - ); - } - let from_env: AppConfig = manager.get().await?; - println!(" Got from env: {:?}", from_env); - assert_eq!(env_config.host, "env-override.example.com"); - unsafe { - std::env::remove_var("HARMONY_CONFIG_AppConfig"); - } - - println!("\n=== Done! ==="); - println!("Config persisted at ~/.local/share/harmony/config/config.db"); + // No default + Secret class: prompts, with `client_secret` read via Password. + let api: ApiCredentials = client.get_or_prompt().await?; + info!("[{}] resolved: {:?}", ApiCredentials::KEY, api.masked()); Ok(()) } diff --git a/harmony_config/examples/prompting.rs b/harmony_config/examples/prompting.rs index e7342584..1018e211 100644 --- a/harmony_config/examples/prompting.rs +++ b/harmony_config/examples/prompting.rs @@ -12,7 +12,7 @@ use std::sync::Arc; -use harmony_config::{Config, ConfigManager, EnvSource, PromptSource, SqliteSource}; +use harmony_config::{Config, ConfigClient, EnvSource, PromptSource, SqliteSource}; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; @@ -27,8 +27,10 @@ struct UserConfig { async fn main() -> anyhow::Result<()> { env_logger::init(); - let sqlite = SqliteSource::default().await?; - let manager = ConfigManager::new(vec![ + // Namespace the SQLite file so this example's state doesn't + // collide with other harmony binaries that also use SqliteSource. + let sqlite = SqliteSource::for_namespace("harmony_config-prompting-example").await?; + let manager = ConfigClient::new(vec![ Arc::new(EnvSource), Arc::new(sqlite), Arc::new(PromptSource::new()), diff --git a/harmony_config/src/lib.rs b/harmony_config/src/lib.rs index 64086c39..dde30471 100644 --- a/harmony_config/src/lib.rs +++ b/harmony_config/src/lib.rs @@ -1,9 +1,13 @@ +// Lets the derive macro emit `::harmony_config::…` paths that resolve both +// inside this crate and in downstream consumers. +extern crate self as harmony_config; + mod source; use async_trait::async_trait; use directories::ProjectDirs; use interactive_parse::InteractiveParseObj; -use log::debug; +use log::warn; use schemars::JsonSchema; use serde::{Serialize, de::DeserializeOwned}; use std::path::PathBuf; @@ -57,40 +61,107 @@ pub enum ConfigError { SqliteError(String), } -pub trait Config: Serialize + DeserializeOwned + JsonSchema + InteractiveParseObj + Sized { - const KEY: &'static str; +/// Hint passed through the `ConfigSource` chain; backends that care +/// (masking, prompting) inspect it, others ignore it. Elevated to `Secret` +/// automatically when any field — or the struct — is `#[config(secret)]`. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum ConfigClass { + Standard, + Secret, } -#[async_trait] -pub trait ConfigSource: Send + Sync { - async fn get(&self, key: &str) -> Result, ConfigError>; +pub trait Config: Serialize + DeserializeOwned + JsonSchema + InteractiveParseObj + Sized { + const KEY: &'static str; + const CLASS: ConfigClass = ConfigClass::Standard; + // Serialized names of `#[config(secret)]` fields, used for masking and + // password prompts. Matching is on the serialized name, so a secret + // field must not be `#[serde(rename)]`d — the derive records the Rust + // ident, and a rename would leak the value in cleartext. + const SECRET_FIELDS: &'static [&'static str] = &[]; +} - async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError>; +/// Safe-display wrapper: `{:?}` renders `T::SECRET_FIELDS` as `"****"`. +/// Use it on any `Config` value headed for a log line or error message. +pub struct Masked<'a, T: Config>(&'a T); - fn should_persist(&self) -> bool { - true +impl std::fmt::Debug for Masked<'_, T> { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let mut value = serde_json::to_value(self.0).map_err(|_| std::fmt::Error)?; + if let serde_json::Value::Object(map) = &mut value { + for name in T::SECRET_FIELDS { + if let Some(v) = map.get_mut(*name) { + *v = serde_json::Value::String("****".to_string()); + } + } + } + write!(f, "{value:#}") } } -pub struct ConfigManager { +pub trait ConfigExt: Config { + fn masked(&self) -> Masked<'_, Self> { + Masked(self) + } +} + +impl ConfigExt for T {} + +#[async_trait] +pub trait ConfigSource: Send + Sync { + // `Ok(None)` means "absent here, try the next source"; `Err` + // short-circuits the chain. `class` is a hint most sources ignore. + async fn get( + &self, + class: ConfigClass, + key: &str, + ) -> Result, ConfigError>; + + async fn set( + &self, + class: ConfigClass, + key: &str, + value: &serde_json::Value, + ) -> Result<(), ConfigError>; +} + +pub struct ConfigClient { sources: Vec>, } -impl ConfigManager { +impl ConfigClient { pub fn new(sources: Vec>) -> Self { Self { sources } } + /// Canonical chain `EnvSource → OpenBao → PromptSource`, scoped to + /// `namespace`. OpenBao is configured from env vars (see + /// [`openbao_from_env`]) and dropped from the chain if unset or + /// unreachable. SQLite is excluded on purpose: it stores cleartext on + /// disk and can't safely hold `Secret`-class config. For other chains, + /// build the sources yourself and pass them to [`ConfigClient::new`]. + pub async fn for_namespace(namespace: &str) -> Self { + let mut sources: Vec> = vec![Arc::new(EnvSource)]; + if let Some(store) = openbao_from_env(namespace).await { + sources.push(store); + } + sources.push(Arc::new(PromptSource::new())); + Self::new(sources) + } + pub async fn get(&self) -> Result { for source in &self.sources { - if let Some(value) = source.get(T::KEY).await? { - let config: T = - serde_json::from_value(value).map_err(|e| ConfigError::Deserialization { - key: T::KEY.to_string(), - source: e, - })?; - debug!("Retrieved config for key {} from source", T::KEY); - return Ok(config); + if let Some(value) = source.get(T::CLASS, T::KEY).await? { + // A deser failure means the stored value is shaped for a + // different version of the struct (branch switch, rename); + // treat it as a miss for this source and fall through so a + // later source — or a re-prompt — overwrites the stale entry. + match serde_json::from_value::(value) { + Ok(config) => return Ok(config), + Err(e) => warn!( + "Stale value for key {} in source; falling through ({e})", + T::KEY + ), + } } } Err(ConfigError::NotFound { @@ -102,24 +173,8 @@ impl ConfigManager { match self.get::().await { Ok(config) => Ok(config), Err(ConfigError::NotFound { .. }) => { - let config = - T::parse_to_obj().map_err(|e| ConfigError::PromptError(e.to_string()))?; - - let value = - serde_json::to_value(&config).map_err(|e| ConfigError::Serialization { - key: T::KEY.to_string(), - source: e, - })?; - - for source in &self.sources { - if !source.should_persist() { - continue; - } - if source.set(T::KEY, &value).await.is_ok() { - break; - } - } - + let config = PromptSource::new().prompt_for::().await?; + self.set(&config).await?; Ok(config) } Err(e) => Err(e), @@ -133,22 +188,61 @@ impl ConfigManager { })?; for source in &self.sources { - source.set(T::KEY, &value).await?; + source.set(T::CLASS, T::KEY, &value).await?; } Ok(()) } } -static CONFIG_MANAGER: Mutex>> = Mutex::const_new(None); +/// Build an OpenBao-backed `StoreSource` from env vars, or `None` (with a +/// `warn!`) when `OPENBAO_URL`/`VAULT_ADDR` is unset or the connection +/// fails — so a missing OpenBao degrades the chain instead of breaking +/// startup. `jwt_role`/`jwt_auth_mount` are left to `OpenbaoSecretStore`'s +/// defaults, matching what `OpenbaoSetupScore` configures. +async fn openbao_from_env(namespace: &str) -> Option> { + let Some(url) = std::env::var("OPENBAO_URL") + .or_else(|_| std::env::var("VAULT_ADDR")) + .ok() + else { + warn!("OpenBao URL not set; OpenBao source omitted from chain"); + return None; + }; + + let env = |k: &str| std::env::var(k).ok(); + let store = harmony_secret::OpenbaoSecretStore::new( + url, + env("OPENBAO_KV_MOUNT").unwrap_or_else(|| "secret".to_string()), + env("OPENBAO_AUTH_MOUNT").unwrap_or_else(|| "jwt".to_string()), + env("OPENBAO_SKIP_TLS").as_deref() == Some("true"), + env("OPENBAO_TOKEN"), + env("OPENBAO_USERNAME"), + env("OPENBAO_PASSWORD"), + env("HARMONY_SSO_URL"), + env("HARMONY_SSO_CLIENT_ID"), + None, + None, + ) + .await; + + match store { + Ok(store) => Some(Arc::new(StoreSource::new(namespace.to_string(), store))), + Err(e) => { + warn!("OpenBao unreachable ({e}); source omitted from chain"); + None + } + } +} + +static CONFIG_CLIENT: Mutex>> = Mutex::const_new(None); pub async fn init(sources: Vec>) { - let mut manager = CONFIG_MANAGER.lock().await; - *manager = Some(Arc::new(ConfigManager::new(sources))); + let mut manager = CONFIG_CLIENT.lock().await; + *manager = Some(Arc::new(ConfigClient::new(sources))); } pub async fn get() -> Result { - let manager = CONFIG_MANAGER.lock().await; + let manager = CONFIG_CLIENT.lock().await; manager .as_ref() .ok_or(ConfigError::NoSources)? @@ -157,7 +251,7 @@ pub async fn get() -> Result { } pub async fn get_or_prompt() -> Result { - let manager = CONFIG_MANAGER.lock().await; + let manager = CONFIG_CLIENT.lock().await; manager .as_ref() .ok_or(ConfigError::NoSources)? @@ -166,7 +260,7 @@ pub async fn get_or_prompt() -> Result { } pub async fn set(config: &T) -> Result<(), ConfigError> { - let manager = CONFIG_MANAGER.lock().await; + let manager = CONFIG_CLIENT.lock().await; manager .as_ref() .ok_or(ConfigError::NoSources)? @@ -229,6 +323,10 @@ mod tests { data: std::sync::Mutex>, get_count: AtomicUsize, set_count: AtomicUsize, + /// Records each `(class, key)` pair observed on `get` / `set` + /// so tests can assert that `ConfigClient` forwards `T::CLASS` + /// to sources unchanged. + observed: std::sync::Mutex>, } impl MockSource { @@ -237,6 +335,7 @@ mod tests { data: std::sync::Mutex::new(std::collections::HashMap::new()), get_count: AtomicUsize::new(0), set_count: AtomicUsize::new(0), + observed: std::sync::Mutex::new(Vec::new()), } } @@ -245,6 +344,7 @@ mod tests { data: std::sync::Mutex::new(data), get_count: AtomicUsize::new(0), set_count: AtomicUsize::new(0), + observed: std::sync::Mutex::new(Vec::new()), } } @@ -255,18 +355,39 @@ mod tests { fn set_call_count(&self) -> usize { self.set_count.load(Ordering::SeqCst) } + + fn observed(&self) -> Vec<(ConfigClass, String, &'static str)> { + self.observed.lock().unwrap().clone() + } } #[async_trait] impl ConfigSource for MockSource { - async fn get(&self, key: &str) -> Result, ConfigError> { + async fn get( + &self, + class: ConfigClass, + key: &str, + ) -> Result, ConfigError> { self.get_count.fetch_add(1, Ordering::SeqCst); + self.observed + .lock() + .unwrap() + .push((class, key.to_string(), "get")); let data = self.data.lock().unwrap(); Ok(data.get(key).cloned()) } - async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> { + async fn set( + &self, + class: ConfigClass, + key: &str, + value: &serde_json::Value, + ) -> Result<(), ConfigError> { self.set_count.fetch_add(1, Ordering::SeqCst); + self.observed + .lock() + .unwrap() + .push((class, key.to_string(), "set")); let mut data = self.data.lock().unwrap(); data.insert(key.to_string(), value.clone()); Ok(()) @@ -286,7 +407,7 @@ mod tests { ); let source = Arc::new(MockSource::with_data(data)); - let manager = ConfigManager::new(vec![source.clone()]); + let manager = ConfigClient::new(vec![source.clone()]); let result: TestConfig = manager.get().await.unwrap(); assert_eq!(result, config); @@ -307,7 +428,7 @@ mod tests { let source1 = Arc::new(MockSource::new()); let source2 = Arc::new(MockSource::with_data(data2)); - let manager = ConfigManager::new(vec![source1.clone(), source2.clone()]); + let manager = ConfigClient::new(vec![source1.clone(), source2.clone()]); let result: TestConfig = manager.get().await.unwrap(); assert_eq!(result, config); @@ -318,7 +439,7 @@ mod tests { #[tokio::test] async fn test_get_returns_not_found_when_no_source_has_key() { let source = Arc::new(MockSource::new()); - let manager = ConfigManager::new(vec![source.clone()]); + let manager = ConfigClient::new(vec![source.clone()]); let result: Result = manager.get().await; assert!(matches!(result, Err(ConfigError::NotFound { .. }))); @@ -326,7 +447,7 @@ mod tests { #[tokio::test] async fn test_get_returns_error_with_no_sources() { - let manager = ConfigManager::new(vec![]); + let manager = ConfigClient::new(vec![]); let result: Result = manager.get().await; assert!(matches!(result, Err(ConfigError::NotFound { .. }))); @@ -341,7 +462,7 @@ mod tests { let source1 = Arc::new(MockSource::new()); let source2 = Arc::new(MockSource::new()); - let manager = ConfigManager::new(vec![source1.clone(), source2.clone()]); + let manager = ConfigClient::new(vec![source1.clone(), source2.clone()]); manager.set(&config).await.unwrap(); @@ -352,6 +473,228 @@ mod tests { assert_eq!(result1, config); } + #[tokio::test] + async fn test_derive_macro_emits_standard_class_with_no_secret_fields() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct PlainConfig { + host: String, + port: u16, + } + + assert_eq!(PlainConfig::KEY, "PlainConfig"); + assert_eq!(PlainConfig::CLASS, ConfigClass::Standard); + } + + #[tokio::test] + async fn test_derive_macro_emits_secret_class_when_field_is_tagged() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct WithSecretField { + host: String, + #[config(secret)] + password: String, + } + + assert_eq!(WithSecretField::CLASS, ConfigClass::Secret); + } + + #[tokio::test] + async fn test_derive_macro_emits_secret_class_when_struct_is_tagged() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + #[config(secret)] + struct AllSecret { + token: String, + } + + assert_eq!(AllSecret::CLASS, ConfigClass::Secret); + } + + // -- SECRET_FIELDS derivation ----------------------------------------- + + #[tokio::test] + async fn test_derive_emits_empty_secret_fields_for_standard_struct() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct StandardPlain { + host: String, + port: u16, + } + + assert_eq!(StandardPlain::CLASS, ConfigClass::Standard); + assert_eq!(StandardPlain::SECRET_FIELDS, &[] as &[&str]); + } + + #[tokio::test] + async fn test_derive_emits_tagged_fields_for_per_field_secret() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct MixedSecret { + host: String, + #[config(secret)] + password: String, + #[config(secret)] + api_key: String, + non_secret: u16, + } + + assert_eq!(MixedSecret::CLASS, ConfigClass::Secret); + // Names emitted in struct-field order; only the tagged ones. + assert_eq!(MixedSecret::SECRET_FIELDS, &["password", "api_key"]); + } + + #[tokio::test] + async fn test_derive_emits_all_fields_for_struct_level_secret() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + #[config(secret)] + struct WholeStructSecret { + token: String, + refresh: String, + count: u32, + } + + assert_eq!(WholeStructSecret::CLASS, ConfigClass::Secret); + assert_eq!( + WholeStructSecret::SECRET_FIELDS, + &["token", "refresh", "count"] + ); + } + + // -- Masked wrapper / ConfigExt::masked() ----------------------------- + + #[tokio::test] + async fn test_masked_replaces_secret_fields_with_stars() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct Mixed { + host: String, + #[config(secret)] + password: String, + } + + let v = Mixed { + host: "db.example.com".to_string(), + password: "hunter2".to_string(), + }; + let s = format!("{:?}", v.masked()); + assert!(s.contains("db.example.com"), "host should be visible: {s}"); + assert!( + s.contains("\"****\""), + "password should render as ****: {s}" + ); + assert!(!s.contains("hunter2"), "raw password must not appear: {s}"); + } + + #[tokio::test] + async fn test_masked_passes_standard_struct_through_unmodified() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct PlainStandard { + name: String, + count: u32, + } + + let v = PlainStandard { + name: "alice".to_string(), + count: 7, + }; + let s = format!("{:?}", v.masked()); + assert!(s.contains("alice")); + assert!(s.contains('7')); + assert!(!s.contains("****")); + } + + #[tokio::test] + async fn test_masked_masks_every_field_for_struct_level_secret() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + #[config(secret)] + struct AllSecret { + token: String, + refresh: String, + } + + let v = AllSecret { + token: "tok123".to_string(), + refresh: "ref456".to_string(), + }; + let s = format!("{:?}", v.masked()); + assert!(!s.contains("tok123"), "token must be masked: {s}"); + assert!(!s.contains("ref456"), "refresh must be masked: {s}"); + // Both fields should render as "****". + assert_eq!(s.matches("\"****\"").count(), 2, "got: {s}"); + } + + #[tokio::test] + async fn test_manual_impl_defaults_to_standard_class() { + // The trait gives CLASS a default of Standard so existing manual + // impls (like TestConfig above) compile without explicit updates. + assert_eq!(TestConfig::CLASS, ConfigClass::Standard); + } + + #[tokio::test] + async fn test_config_manager_forwards_standard_class_to_sources_on_set() { + let source = Arc::new(MockSource::new()); + let manager = ConfigClient::new(vec![source.clone()]); + let config = TestConfig { + name: "x".into(), + count: 1, + }; + + manager.set(&config).await.unwrap(); + + let observed = source.observed(); + assert_eq!(observed.len(), 1); + let (class, key, op) = &observed[0]; + assert_eq!(*class, ConfigClass::Standard); + assert_eq!(key, "TestConfig"); + assert_eq!(*op, "set"); + } + + #[tokio::test] + async fn test_config_manager_forwards_secret_class_to_sources_on_set() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + struct SecretConfig { + #[config(secret)] + password: String, + } + + assert_eq!(SecretConfig::CLASS, ConfigClass::Secret); + + let source = Arc::new(MockSource::new()); + let manager = ConfigClient::new(vec![source.clone()]); + manager + .set(&SecretConfig { + password: "hunter2".into(), + }) + .await + .unwrap(); + + let (class, key, op) = source.observed().pop().unwrap(); + assert_eq!(class, ConfigClass::Secret); + assert_eq!(key, "SecretConfig"); + assert_eq!(op, "set"); + } + + #[tokio::test] + async fn test_config_manager_forwards_class_to_every_source_on_get() { + #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)] + #[config(secret)] + struct StructLevelSecret { + token: String, + } + + // Two empty sources so the chain probes both before returning NotFound. + let s1 = Arc::new(MockSource::new()); + let s2 = Arc::new(MockSource::new()); + let manager = ConfigClient::new(vec![s1.clone(), s2.clone()]); + + let res: Result = manager.get().await; + assert!(matches!(res, Err(ConfigError::NotFound { .. }))); + + for src in [&s1, &s2] { + let observed = src.observed(); + assert_eq!(observed.len(), 1); + let (class, key, op) = &observed[0]; + assert_eq!(*class, ConfigClass::Secret); + assert_eq!(key, "StructLevelSecret"); + assert_eq!(*op, "get"); + } + } + #[tokio::test] async fn test_derive_macro_generates_correct_key() { #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq)] @@ -372,7 +715,12 @@ mod tests { let env_var = setup_env_vars("TestConfig", Some(r#"{"name":"from_env","count":7}"#)); let source = EnvSource; - let result = source.get(&env_var.replace("HARMONY_CONFIG_", "")).await; + let result = source + .get( + ConfigClass::Standard, + &env_var.replace("HARMONY_CONFIG_", ""), + ) + .await; unsafe { std::env::remove_var(&env_var); @@ -396,7 +744,12 @@ mod tests { .unwrap(); rt.block_on(async { let source = EnvSource; - let result = source.get(&env_var.replace("HARMONY_CONFIG_", "")).await; + let result = source + .get( + ConfigClass::Standard, + &env_var.replace("HARMONY_CONFIG_", ""), + ) + .await; assert!(result.unwrap().is_none()); }); }); @@ -408,7 +761,12 @@ mod tests { let env_var = setup_env_vars("TestConfig", Some("not valid json")); let source = EnvSource; - let result = source.get(&env_var.replace("HARMONY_CONFIG_", "")).await; + let result = source + .get( + ConfigClass::Standard, + &env_var.replace("HARMONY_CONFIG_", ""), + ) + .await; unsafe { std::env::remove_var(&env_var); @@ -430,7 +788,11 @@ mod tests { std::fs::write(&config_path, serde_json::to_string(&config).unwrap()).unwrap(); let source = LocalFileSource::new(dir.path().to_path_buf()); - let result = source.get("TestConfig").await.unwrap().unwrap(); + let result = source + .get(ConfigClass::Standard, "TestConfig") + .await + .unwrap() + .unwrap(); let parsed: TestConfig = serde_json::from_value(result).unwrap(); assert_eq!(parsed, config); @@ -442,7 +804,10 @@ mod tests { let dir = tempdir().unwrap(); let source = LocalFileSource::new(dir.path().to_path_buf()); - let result = source.get("NonExistentConfig").await.unwrap(); + let result = source + .get(ConfigClass::Standard, "NonExistentConfig") + .await + .unwrap(); assert!(result.is_none()); } @@ -459,7 +824,11 @@ mod tests { }; source - .set("TestConfig", &serde_json::to_value(&config).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config).unwrap(), + ) .await .unwrap(); @@ -484,11 +853,19 @@ mod tests { }; source - .set("TestConfig", &serde_json::to_value(&config).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config).unwrap(), + ) .await .unwrap(); - let result = source.get("TestConfig").await.unwrap().unwrap(); + let result = source + .get(ConfigClass::Standard, "TestConfig") + .await + .unwrap() + .unwrap(); let parsed: TestConfig = serde_json::from_value(result).unwrap(); assert_eq!(parsed, config); @@ -502,7 +879,10 @@ mod tests { let path = temp_file.path().to_path_buf(); let source = SqliteSource::open(path).await.unwrap(); - let result = source.get("NonExistentConfig").await.unwrap(); + let result = source + .get(ConfigClass::Standard, "NonExistentConfig") + .await + .unwrap(); assert!(result.is_none()); } @@ -525,15 +905,27 @@ mod tests { }; source - .set("TestConfig", &serde_json::to_value(&config1).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config1).unwrap(), + ) .await .unwrap(); source - .set("TestConfig", &serde_json::to_value(&config2).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config2).unwrap(), + ) .await .unwrap(); - let result = source.get("TestConfig").await.unwrap().unwrap(); + let result = source + .get(ConfigClass::Standard, "TestConfig") + .await + .unwrap() + .unwrap(); let parsed: TestConfig = serde_json::from_value(result).unwrap(); assert_eq!(parsed, config2); @@ -561,17 +953,33 @@ mod tests { let (r1, r2) = tokio::join!( async { source - .set("key1", &serde_json::to_value(&config1).unwrap()) + .set( + ConfigClass::Standard, + "key1", + &serde_json::to_value(&config1).unwrap(), + ) .await .unwrap(); - source.get("key1").await.unwrap().unwrap() + source + .get(ConfigClass::Standard, "key1") + .await + .unwrap() + .unwrap() }, async { source - .set("key2", &serde_json::to_value(&config2).unwrap()) + .set( + ConfigClass::Standard, + "key2", + &serde_json::to_value(&config2).unwrap(), + ) .await .unwrap(); - source.get("key2").await.unwrap().unwrap() + source + .get(ConfigClass::Standard, "key2") + .await + .unwrap() + .unwrap() } ); @@ -608,7 +1016,7 @@ mod tests { let source1 = Arc::new(MockSource::with_data(data)); let source2 = Arc::new(MockSource::new()); - let manager = ConfigManager::new(vec![source1.clone(), source2.clone()]); + let manager = ConfigClient::new(vec![source1.clone(), source2.clone()]); let result: TestConfig = manager.get_or_prompt().await.unwrap(); assert_eq!(result, config); @@ -626,7 +1034,7 @@ mod tests { .unwrap(); let sqlite = Arc::new(sqlite); - let manager = ConfigManager::new(vec![sqlite.clone()]); + let manager = ConfigClient::new(vec![sqlite.clone()]); let config = TestConfig { name: "from_sqlite".to_string(), @@ -634,7 +1042,11 @@ mod tests { }; sqlite - .set("TestConfig", &serde_json::to_value(&config).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config).unwrap(), + ) .await .unwrap(); @@ -653,7 +1065,7 @@ mod tests { let sqlite = Arc::new(sqlite); let env_source = Arc::new(EnvSource); - let manager = ConfigManager::new(vec![env_source.clone(), sqlite.clone()]); + let manager = ConfigClient::new(vec![env_source.clone(), sqlite.clone()]); let sqlite_config = TestConfig { name: "from_sqlite".to_string(), @@ -665,7 +1077,11 @@ mod tests { }; sqlite - .set("TestConfig", &serde_json::to_value(&sqlite_config).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&sqlite_config).unwrap(), + ) .await .unwrap(); @@ -684,7 +1100,13 @@ mod tests { } #[tokio::test] - async fn test_branch_switching_scenario_deserialization_error() { + async fn test_branch_switching_scenario_deserialization_falls_through() { + // ADR 020 contract: a stale value (wrong shape for the current + // struct, e.g. after a branch switch that renamed or retyped a + // field) should be treated as a cache miss for that source. The + // chain falls through; if no source has a valid value, the result + // is `NotFound`, which `get_or_prompt` turns into a re-prompt + // that overwrites the stale entry. use tempfile::NamedTempFile; let temp_file = NamedTempFile::new().unwrap(); @@ -693,22 +1115,61 @@ mod tests { .unwrap(); let sqlite = Arc::new(sqlite); - let manager = ConfigManager::new(vec![sqlite.clone()]); + let manager = ConfigClient::new(vec![sqlite.clone()]); let old_config = serde_json::json!({ "name": "old_config", "count": "not_a_number" }); - sqlite.set("TestConfig", &old_config).await.unwrap(); + sqlite + .set(ConfigClass::Standard, "TestConfig", &old_config) + .await + .unwrap(); let result: Result = manager.get().await; - assert!(matches!(result, Err(ConfigError::Deserialization { .. }))); + assert!(matches!(result, Err(ConfigError::NotFound { .. }))); + } + + #[tokio::test] + async fn test_deserialization_failure_falls_through_to_next_source() { + // First source has a stale value; second source has a fresh one. + // The chain should skip past the stale entry and resolve from + // the second source. + use tempfile::NamedTempFile; + + let temp_file = NamedTempFile::new().unwrap(); + let stale_sqlite = SqliteSource::open(temp_file.path().to_path_buf()) + .await + .unwrap(); + let stale_sqlite = Arc::new(stale_sqlite); + + let stale = serde_json::json!({"name": "stale", "count": "not_a_number"}); + stale_sqlite + .set(ConfigClass::Standard, "TestConfig", &stale) + .await + .unwrap(); + + let fresh_config = TestConfig { + name: "fresh".to_string(), + count: 7, + }; + let mut fresh_data = std::collections::HashMap::new(); + fresh_data.insert( + "TestConfig".to_string(), + serde_json::to_value(&fresh_config).unwrap(), + ); + let fresh = Arc::new(MockSource::with_data(fresh_data)); + + let manager = ConfigClient::new(vec![stale_sqlite.clone(), fresh.clone()]); + + let result: TestConfig = manager.get().await.unwrap(); + assert_eq!(result, fresh_config); } #[tokio::test] async fn test_prompt_source_always_returns_none() { let source = PromptSource::new(); - let result = source.get("AnyKey").await.unwrap(); + let result = source.get(ConfigClass::Standard, "AnyKey").await.unwrap(); assert!(result.is_none()); } @@ -716,7 +1177,11 @@ mod tests { async fn test_prompt_source_set_is_noop() { let source = PromptSource::new(); let result = source - .set("AnyKey", &serde_json::json!({"test": "value"})) + .set( + ConfigClass::Standard, + "AnyKey", + &serde_json::json!({"test": "value"}), + ) .await; assert!(result.is_ok()); } @@ -726,13 +1191,17 @@ mod tests { let source = PromptSource::new(); source .set( + ConfigClass::Standard, "TestConfig", &serde_json::json!({"name": "test", "count": 42}), ) .await .unwrap(); - let result = source.get("TestConfig").await.unwrap(); + let result = source + .get(ConfigClass::Standard, "TestConfig") + .await + .unwrap(); assert!(result.is_none()); } @@ -750,7 +1219,7 @@ mod tests { let prompt_source = Arc::new(PromptSource::new()); let manager = - ConfigManager::new(vec![source1.clone(), sqlite.clone(), prompt_source.clone()]); + ConfigClient::new(vec![source1.clone(), sqlite.clone(), prompt_source.clone()]); let result: Result = manager.get().await; assert!(matches!(result, Err(ConfigError::NotFound { .. }))); @@ -781,14 +1250,18 @@ mod tests { let store_source = Arc::new(StoreSource::new("test".to_string(), AlwaysErrorStore)); - let manager = ConfigManager::new(vec![store_source.clone(), sqlite.clone()]); + let manager = ConfigClient::new(vec![store_source.clone(), sqlite.clone()]); let config = TestConfig { name: "from_sqlite".to_string(), count: 42, }; sqlite - .set("TestConfig", &serde_json::to_value(&config).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config).unwrap(), + ) .await .unwrap(); @@ -825,14 +1298,18 @@ mod tests { let store_source = Arc::new(StoreSource::new("test".to_string(), NeverFindsStore)); - let manager = ConfigManager::new(vec![store_source.clone(), sqlite.clone()]); + let manager = ConfigClient::new(vec![store_source.clone(), sqlite.clone()]); let config = TestConfig { name: "from_sqlite".to_string(), count: 99, }; sqlite - .set("TestConfig", &serde_json::to_value(&config).unwrap()) + .set( + ConfigClass::Standard, + "TestConfig", + &serde_json::to_value(&config).unwrap(), + ) .await .unwrap(); diff --git a/harmony_config/src/source/env.rs b/harmony_config/src/source/env.rs index 5e201598..e976bbf3 100644 --- a/harmony_config/src/source/env.rs +++ b/harmony_config/src/source/env.rs @@ -1,4 +1,4 @@ -use crate::{ConfigError, ConfigSource}; +use crate::{ConfigClass, ConfigError, ConfigSource}; use async_trait::async_trait; pub struct EnvSource; @@ -9,7 +9,11 @@ fn env_key_for(config_key: &str) -> String { #[async_trait] impl ConfigSource for EnvSource { - async fn get(&self, key: &str) -> Result, ConfigError> { + async fn get( + &self, + _class: ConfigClass, + key: &str, + ) -> Result, ConfigError> { let env_key = env_key_for(key); match std::env::var(&env_key) { @@ -27,23 +31,23 @@ impl ConfigSource for EnvSource { } } - async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> { + async fn set( + &self, + _class: ConfigClass, + key: &str, + value: &serde_json::Value, + ) -> Result<(), ConfigError> { let env_key = env_key_for(key); let json_string = serde_json::to_string(value).map_err(|e| ConfigError::Serialization { key: key.to_string(), source: e, })?; - // SAFETY: Setting environment variables is generally safe in single-threaded contexts. - // In multi-threaded contexts, this could cause races, but is acceptable for this use case - // as config is typically set once at startup. + // Rust 2024 makes `set_var` unsafe (data race if another thread reads + // env concurrently); config is set once at startup, so this is safe. unsafe { std::env::set_var(&env_key, &json_string); } Ok(()) } - - fn should_persist(&self) -> bool { - false - } } diff --git a/harmony_config/src/source/local_file.rs b/harmony_config/src/source/local_file.rs index f803b2cc..c435cab1 100644 --- a/harmony_config/src/source/local_file.rs +++ b/harmony_config/src/source/local_file.rs @@ -2,8 +2,13 @@ use async_trait::async_trait; use std::path::PathBuf; use tokio::fs; -use crate::{ConfigError, ConfigSource}; +use crate::{ConfigClass, ConfigError, ConfigSource}; +/// Local file-backed config source (`/.json`). +/// +/// ⚠️ Cleartext on disk and ignores `ConfigClass`, like +/// [`SqliteSource`](crate::SqliteSource) — non-secret config only, via an +/// explicit [`ConfigClient::new`](crate::ConfigClient::new). pub struct LocalFileSource { base_path: PathBuf, } @@ -24,7 +29,11 @@ impl LocalFileSource { #[async_trait] impl ConfigSource for LocalFileSource { - async fn get(&self, key: &str) -> Result, ConfigError> { + async fn get( + &self, + _class: ConfigClass, + key: &str, + ) -> Result, ConfigError> { let path = self.file_path_for(key); match fs::read(&path).await { @@ -45,7 +54,12 @@ impl ConfigSource for LocalFileSource { } } - async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> { + async fn set( + &self, + _class: ConfigClass, + key: &str, + value: &serde_json::Value, + ) -> Result<(), ConfigError> { fs::create_dir_all(&self.base_path).await?; let path = self.file_path_for(key); diff --git a/harmony_config/src/source/prompt.rs b/harmony_config/src/source/prompt.rs index 79a5ea2b..f25ea13b 100644 --- a/harmony_config/src/source/prompt.rs +++ b/harmony_config/src/source/prompt.rs @@ -1,22 +1,41 @@ use async_trait::async_trait; -use std::sync::Arc; +use inquire::{CustomType, Password, PasswordDisplayMode, Text}; +use schemars::schema::{InstanceType, RootSchema, Schema, SchemaObject, SingleOrVec}; +use tokio::sync::Mutex; -use crate::{ConfigError, ConfigSource}; +use crate::{Config, ConfigClass, ConfigError, ConfigSource}; -pub struct PromptSource { - #[allow(dead_code)] - writer: Option>, -} +// Serialises interactive prompts process-wide: `inquire` assumes exclusive +// terminal ownership, so concurrent prompts would corrupt the terminal. +static PROMPT_MUTEX: Mutex<()> = Mutex::const_new(()); + +pub struct PromptSource; impl PromptSource { pub fn new() -> Self { - Self { writer: None } + Self } - #[allow(dead_code)] - pub fn with_writer(writer: Arc) -> Self { - Self { - writer: Some(writer), + /// Prompt for a `T`. `Standard` structs go through `interactive_parse` + /// (full nested support); `Secret` structs go through a flat-field + /// walker that reads `#[config(secret)]` fields via `inquire::Password`. + pub async fn prompt_for(&self) -> Result { + let _guard = PROMPT_MUTEX.lock().await; + let banner = format!( + "── Configuring `{}` ({:?}) — please fill the fields below ──", + T::KEY, + T::CLASS, + ); + // inquire renders on stderr; match that channel and pad with blank + // lines so the banner stays separate from preceding log output. + eprintln!(); + eprintln!("{banner}"); + eprintln!(); + match T::CLASS { + ConfigClass::Standard => { + T::parse_to_obj().map_err(|e| ConfigError::PromptError(e.to_string())) + } + ConfigClass::Secret => prompt_secret_struct::(), } } } @@ -29,15 +48,153 @@ impl Default for PromptSource { #[async_trait] impl ConfigSource for PromptSource { - async fn get(&self, _key: &str) -> Result, ConfigError> { + async fn get( + &self, + _class: ConfigClass, + _key: &str, + ) -> Result, ConfigError> { Ok(None) } - async fn set(&self, _key: &str, _value: &serde_json::Value) -> Result<(), ConfigError> { + async fn set( + &self, + _class: ConfigClass, + _key: &str, + _value: &serde_json::Value, + ) -> Result<(), ConfigError> { Ok(()) } +} - fn should_persist(&self) -> bool { - false +// Walks a Secret struct's schema, reading `T::SECRET_FIELDS` via Password +// and other fields via type-appropriate prompts. Only flat primitive +// fields are supported; anything else returns a clear PromptError. +fn prompt_secret_struct() -> Result { + let root: RootSchema = schemars::schema_for!(T); + let object = root.schema.object.as_deref().ok_or_else(|| { + ConfigError::PromptError(format!( + "Secret struct `{}` has no JSON-object schema; the walker only \ + supports plain structs", + T::KEY + )) + })?; + + let mut json = serde_json::Map::with_capacity(object.properties.len()); + for (field_name, field_schema) in &object.properties { + let field_object = schema_object(field_schema, T::KEY, field_name)?; + let is_secret = T::SECRET_FIELDS.contains(&field_name.as_str()); + let value = prompt_field::(field_name, field_object, is_secret)?; + json.insert(field_name.clone(), value); + } + + serde_json::from_value(serde_json::Value::Object(json)).map_err(|e| { + ConfigError::PromptError(format!( + "Failed to deserialize prompted values into `{}`: {e}", + T::KEY + )) + }) +} + +fn schema_object<'a>( + schema: &'a Schema, + struct_name: &str, + field_name: &str, +) -> Result<&'a SchemaObject, ConfigError> { + match schema { + Schema::Object(obj) => Ok(obj), + Schema::Bool(_) => Err(ConfigError::PromptError(format!( + "Secret struct `{struct_name}` field `{field_name}` resolved to a \ + boolean JSON-schema; use a concrete type." + ))), + } +} + +// Picks the inquire widget for one field from its JsonSchema. Secret +// strings use Password; other primitives use type-appropriate input. +// Returns Err for unsupported shapes — never a silent unmasked fallback. +fn prompt_field( + field_name: &str, + schema: &SchemaObject, + is_secret: bool, +) -> Result { + let instance_type = schema.instance_type.as_ref().ok_or_else(|| { + ConfigError::PromptError(format!( + "Secret struct `{}` field `{field_name}` is not a flat primitive \ + (string/integer/number/bool); flatten it or extend the walker.", + T::KEY + )) + })?; + + let single = match instance_type { + SingleOrVec::Single(boxed) => **boxed, + SingleOrVec::Vec(_) => { + return Err(ConfigError::PromptError(format!( + "Secret struct `{}` field `{field_name}` has a multi-type schema \ + (e.g. nullable); the walker only handles single primitives.", + T::KEY + ))); + } + }; + + let label = format!("{field_name}:"); + match single { + InstanceType::String => { + if is_secret { + // Masked mode echoes `*` per keystroke (inquire's default + // Hidden mode shows nothing until enter). + let raw = Password::new(&label) + .with_display_mode(PasswordDisplayMode::Masked) + .without_confirmation() + .prompt() + .map_err(|e| { + ConfigError::PromptError(format!( + "Password prompt for `{}::{field_name}` failed: {e}", + T::KEY + )) + })?; + Ok(serde_json::Value::String(raw)) + } else { + let raw = Text::new(&label).prompt().map_err(|e| { + ConfigError::PromptError(format!( + "Text prompt for `{}::{field_name}` failed: {e}", + T::KEY + )) + })?; + Ok(serde_json::Value::String(raw)) + } + } + InstanceType::Integer => { + // i64 covers Config integer fields; serde narrows on deserialize. + let n = CustomType::::new(&label).prompt().map_err(|e| { + ConfigError::PromptError(format!( + "Integer prompt for `{}::{field_name}` failed: {e}", + T::KEY + )) + })?; + Ok(serde_json::json!(n)) + } + InstanceType::Number => { + let n = CustomType::::new(&label).prompt().map_err(|e| { + ConfigError::PromptError(format!( + "Number prompt for `{}::{field_name}` failed: {e}", + T::KEY + )) + })?; + Ok(serde_json::json!(n)) + } + InstanceType::Boolean => { + let b = CustomType::::new(&label).prompt().map_err(|e| { + ConfigError::PromptError(format!( + "Boolean prompt for `{}::{field_name}` failed: {e}", + T::KEY + )) + })?; + Ok(serde_json::Value::Bool(b)) + } + other => Err(ConfigError::PromptError(format!( + "Secret struct `{}` field `{field_name}` has unsupported type \ + `{other:?}`; the walker only handles flat primitives.", + T::KEY + ))), } } diff --git a/harmony_config/src/source/sqlite.rs b/harmony_config/src/source/sqlite.rs index c39568ef..eae3c39a 100644 --- a/harmony_config/src/source/sqlite.rs +++ b/harmony_config/src/source/sqlite.rs @@ -3,8 +3,15 @@ use sqlx::{SqlitePool, sqlite::SqlitePoolOptions}; use std::path::PathBuf; use tokio::fs; -use crate::{ConfigError, ConfigSource}; +use crate::{ConfigClass, ConfigError, ConfigSource}; +/// Local SQLite-backed config cache at `//config.db`. +/// +/// ⚠️ Stores values as cleartext JSON and ignores `ConfigClass`, so it must +/// never hold `Secret`-class config — it is excluded from the canonical +/// chain ([`ConfigClient::for_namespace`](crate::ConfigClient::for_namespace)) +/// and only usable via an explicit [`ConfigClient::new`](crate::ConfigClient::new) +/// for non-secret offline caching. pub struct SqliteSource { pool: SqlitePool, } @@ -37,11 +44,14 @@ impl SqliteSource { Ok(Self { pool }) } - pub async fn default() -> Result { + /// Scope the database to `namespace` so harmony binaries on one machine + /// don't share a `config.db`. Use the same namespace as `StoreSource`. + pub async fn for_namespace(namespace: &str) -> Result { let path = crate::default_config_dir() .ok_or_else(|| { ConfigError::SqliteError("Could not determine default config directory".into()) })? + .join(namespace) .join("config.db"); Self::open(path).await } @@ -49,7 +59,11 @@ impl SqliteSource { #[async_trait] impl ConfigSource for SqliteSource { - async fn get(&self, key: &str) -> Result, ConfigError> { + async fn get( + &self, + _class: ConfigClass, + key: &str, + ) -> Result, ConfigError> { let row: Option<(String,)> = sqlx::query_as("SELECT value FROM config WHERE key = ?") .bind(key) .fetch_optional(&self.pool) @@ -69,7 +83,12 @@ impl ConfigSource for SqliteSource { } } - async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> { + async fn set( + &self, + _class: ConfigClass, + key: &str, + value: &serde_json::Value, + ) -> Result<(), ConfigError> { let json_string = serde_json::to_string(value).map_err(|e| ConfigError::Serialization { key: key.to_string(), source: e, diff --git a/harmony_config/src/source/store.rs b/harmony_config/src/source/store.rs index a8bd8ad5..a4ba4e48 100644 --- a/harmony_config/src/source/store.rs +++ b/harmony_config/src/source/store.rs @@ -1,7 +1,8 @@ use async_trait::async_trait; use harmony_secret::SecretStore; +use log::warn; -use crate::{ConfigError, ConfigSource}; +use crate::{ConfigClass, ConfigError, ConfigSource}; pub struct StoreSource { namespace: String, @@ -15,8 +16,14 @@ impl StoreSource { } #[async_trait] +// TODO(ADR 020-1): forward `_class` to `SecretStore` once that trait accepts +// a class hint; kept here so the ConfigSource boundary already matches ADR 020. impl ConfigSource for StoreSource { - async fn get(&self, key: &str) -> Result, ConfigError> { + async fn get( + &self, + _class: ConfigClass, + key: &str, + ) -> Result, ConfigError> { match self.store.get_raw(&self.namespace, key).await { Ok(bytes) => { let value: serde_json::Value = @@ -27,11 +34,23 @@ impl ConfigSource for StoreSource { Ok(Some(value)) } Err(harmony_secret::SecretStoreError::NotFound { .. }) => Ok(None), - Err(_) => Ok(None), + // Log before swallowing: a down/misconfigured OpenBao must not look identical to "key absent". + Err(e) => { + warn!( + "StoreSource: get for key '{key}' failed ({e}); treating as \ + absent and falling through to the next source" + ); + Ok(None) + } } } - async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> { + async fn set( + &self, + _class: ConfigClass, + key: &str, + value: &serde_json::Value, + ) -> Result<(), ConfigError> { let bytes = serde_json::to_vec(value).map_err(|e| ConfigError::Serialization { key: key.to_string(), source: e, diff --git a/harmony_config_derive/src/lib.rs b/harmony_config_derive/src/lib.rs index b5a0681d..ecdd4958 100644 --- a/harmony_config_derive/src/lib.rs +++ b/harmony_config_derive/src/lib.rs @@ -1,17 +1,23 @@ use proc_macro::TokenStream; use proc_macro_crate::{FoundCrate, crate_name}; use quote::quote; -use syn::{DeriveInput, Ident, parse_macro_input}; +use syn::{Attribute, Data, DeriveInput, Fields, Ident, parse_macro_input}; -#[proc_macro_derive(Config)] +/// Derives `Config`, emitting `KEY`, `CLASS`, and `SECRET_FIELDS`. Mark a +/// field — or the whole struct — `#[config(secret)]` to elevate it to +/// `ConfigClass::Secret`. See `Config::SECRET_FIELDS` for the no-rename rule. +#[proc_macro_derive(Config, attributes(config))] pub fn derive_config(input: TokenStream) -> TokenStream { let input = parse_macro_input!(input as DeriveInput); let struct_ident = &input.ident; let key = struct_ident.to_string(); + // Always emit a fully-qualified path: examples are separate binary + // crates whose `crate::` root lacks harmony_config's items, so the + // `extern crate self` alias in the lib makes `::harmony_config` resolve. let config_crate_path = match crate_name("harmony_config") { - Ok(FoundCrate::Itself) => quote!(crate), + Ok(FoundCrate::Itself) => quote!(::harmony_config), Ok(FoundCrate::Name(name)) => { let ident = Ident::new(&name, proc_macro2::Span::call_site()); quote!(::#ident) @@ -23,11 +29,63 @@ pub fn derive_config(input: TokenStream) -> TokenStream { } }; + let struct_level_secret = has_secret_attr(&input.attrs); + + // A struct-level `#[config(secret)]` makes every named field secret. + let mut secret_field_names: Vec = Vec::new(); + if let Data::Struct(ref data) = input.data + && let Fields::Named(named) = &data.fields + { + for field in &named.named { + let Some(ident) = field.ident.as_ref() else { + continue; + }; + if struct_level_secret || has_secret_attr(&field.attrs) { + secret_field_names.push(ident.to_string()); + } + } + } + + let is_secret = struct_level_secret || !secret_field_names.is_empty(); + + let class = if is_secret { + quote!(#config_crate_path::ConfigClass::Secret) + } else { + quote!(#config_crate_path::ConfigClass::Standard) + }; + + let secret_fields_const = if secret_field_names.is_empty() { + quote!() // fall back to the trait default `&[]` + } else { + let names = secret_field_names.iter().map(|n| quote!(#n)); + quote! { + const SECRET_FIELDS: &'static [&'static str] = &[ #( #names ),* ]; + } + }; + let expanded = quote! { impl #config_crate_path::Config for #struct_ident { const KEY: &'static str = #key; + const CLASS: #config_crate_path::ConfigClass = #class; + #secret_fields_const } }; TokenStream::from(expanded) } + +fn has_secret_attr(attrs: &[Attribute]) -> bool { + attrs.iter().any(|attr| { + if !attr.path().is_ident("config") { + return false; + } + let mut found = false; + let _ = attr.parse_nested_meta(|meta| { + if meta.path.is_ident("secret") { + found = true; + } + Ok(()) + }); + found + }) +}