NationTech/harmony
3
0
Fork 2
Code Issues 38 Pull Requests 29 Actions Packages Projects Releases 1 Wiki Activity

feat(harmony_config): unified config layer (ADR-020) — ConfigClient, ConfigClass, masking #304

Merged
johnride merged 4 commits from pr/harmony-config-layer into master 2026-05-29 16:15:56 +00:00
Conversation 0 Commits 4 Files Changed 14 +954 -296
14 changed files with 954 additions and 296 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
ROADMAP/01-config-crate.md
View File

@@ -9,7 +9,7 @@ Make `harmony_config` production-ready with a seamless first-run experience: clo
`harmony_config` now has:
- `Config` trait + `#[derive(Config)]` macro
- `ConfigManager` with ordered source chain
- `ConfigClient` with ordered source chain
- Five `ConfigSource` implementations:
- `EnvSource` — reads `HARMONY_CONFIG_{KEY}` env vars
- `LocalFileSource` — reads/writes `{key}.json` files from a directory
@@ -140,7 +140,7 @@ for source in &self.sources {
┌─────────────────────────────────────────────────────────────────────┐
│ Harmony CLI / App │
│ │
│ ConfigManager: │
│ ConfigClient: │
│ 1. EnvSource ← HARMONY_CONFIG_* env vars (highest priority) │
│ 2. SqliteSource ← ~/.local/share/harmony/config/config.db │
│ 3. StoreSource ← OpenBao (team-scale, via Zitadel OIDC) │
@@ -448,11 +448,11 @@ The example uses `StoreSource<OpenbaoSecretStore>` with token auth to avoid the
| `OPENBAO_KV_MOUNT` | No | `"secret"` | KV v2 engine mount path. **Also used as userpass auth mount -- this is a bug.** |
| `OPENBAO_SKIP_TLS` | No | `false` | Set `"true"` to disable TLS verification |
**Note**: `OpenbaoSecretStore::new()` is `async` and **requires a running OpenBao** at construction time (it validates the token if using cached auth). If OpenBao is unreachable during construction, the call will fail. The graceful fallback only applies to `StoreSource::get()` calls after construction -- the `ConfigManager` must be built with a live store, or the store must be wrapped in a lazy initialization pattern.
**Note**: `OpenbaoSecretStore::new()` is `async` and **requires a running OpenBao** at construction time (it validates the token if using cached auth). If OpenBao is unreachable during construction, the call will fail. The graceful fallback only applies to `StoreSource::get()` calls after construction -- the `ConfigClient` must be built with a live store, or the store must be wrapped in a lazy initialization pattern.
```rust
// harmony_config/examples/openbao_chain.rs
use harmony_config::{ConfigManager, EnvSource, SqliteSource, StoreSource};
use harmony_config::{ConfigClient, EnvSource, SqliteSource, StoreSource};
use harmony_secret::OpenbaoSecretStore;
use serde::{Deserialize, Serialize};
use std::sync::Arc;
@@ -517,7 +517,7 @@ async fn main() -> anyhow::Result<()> {
vec![env_source, sqlite]
};
let manager = ConfigManager::new(sources);
let manager = ConfigClient::new(sources);
// Scenario 1: get() with nothing stored -- returns NotFound
let result = manager.get::<AppConfig>().await;
@@ -620,4 +620,4 @@ match self.store.get_raw(&self.namespace, key).await {
6. **Graceful fallback**: `StoreSource::get()` returns `Ok(None)` on any error (connection refused, timeout, etc.), allowing the chain to fall through to the next source. This ensures OpenBao unavailability doesn't break the config chain.
7. **StoreSource errors don't block chain**: When OpenBao is unreachable, `StoreSource::get()` returns `Ok(None)` and the `ConfigManager` continues to the next source (typically `SqliteSource`). This is validated by `test_store_source_error_falls_through_to_sqlite` and `test_store_source_not_found_falls_through_to_sqlite`.
7. **StoreSource errors don't block chain**: When OpenBao is unreachable, `StoreSource::get()` returns `Ok(None)` and the `ConfigClient` continues to the next source (typically `SqliteSource`). This is validated by `test_store_source_error_falls_through_to_sqlite` and `test_store_source_not_found_falls_through_to_sqlite`.

4
docs/adr/020-1-zitadel-openbao-secure-config-store.md
View File

@@ -211,11 +211,13 @@ The `bound_audiences` claim ties the role to the specific Harmony Zitadel applic
For organizations running their own infrastructure, the same architecture applies. The operator deploys Zitadel and OpenBao using Harmony's existing `ZitadelScore` and `OpenbaoScore`. The only configuration needed is three environment variables (or their equivalents in the bootstrap config):
- `HARMONY_SSO_URL` — the Zitadel instance URL.
- `HARMONY_SECRETS_URL` — the OpenBao instance URL.
- `OPENBAO_URL` (or `VAULT_ADDR`) — the OpenBao instance URL.
- `HARMONY_SSO_CLIENT_ID` — the Zitadel application client ID.
None of these are secrets. They can be committed to an infrastructure repository or distributed via any convenient channel.
`OPENBAO_URL`/`VAULT_ADDR` is named after the backend on purpose: it is read only by the OpenBao adapter, never by domain or Score code, and reusing the standard Vault/OpenBao variable names gives operators interop with existing tooling. The tool-agnostic seam lives one layer up — `HARMONY_SECRET_STORE` selects which backend is active, and each backend then reads its own connection params (e.g. `HARMONY_SECRET_INFISICAL_*`). An earlier draft of this ADR proposed a single agnostic `HARMONY_SECRETS_URL`, but it was never implemented: different stores don't share a connection shape (Infisical needs a project and client credentials; AWS Secrets Manager has no URL at all), so one "secrets URL" would force OpenBao's model onto every backend.
## Consequences
### Positive

4
examples/harmony_sso/src/main.rs
View File

@@ -10,7 +10,7 @@ use harmony::modules::zitadel::{
};
use harmony::score::Score;
use harmony::topology::{K8sclient, Topology};
use harmony_config::{Config, ConfigManager, EnvSource, StoreSource};
use harmony_config::{Config, ConfigClient, EnvSource, StoreSource};
use harmony_k8s::K8sClient;
use harmony_secret::OpenbaoSecretStore;
use k3d_rs::{K3d, PortMapping};
@@ -380,7 +380,7 @@ async fn main() -> anyhow::Result<()> {
.await
.context("SSO authentication failed")?;
let manager = ConfigManager::new(vec![
let manager = ConfigClient::new(vec![
Arc::new(EnvSource) as Arc<dyn harmony_config::ConfigSource>,
Arc::new(StoreSource::new("harmony".to_string(), store)),
]);

2
harmony/src/domain/topology/firewall_pair.rs
View File

@@ -62,7 +62,7 @@ impl FirewallPairTopology {
pub async fn opnsense_from_config() -> Self {
// TODO: both firewalls share the same credentials. Once named config
// instances are available (ROADMAP/11), use per-device credentials:
// ConfigManager::get_named::<OPNSenseApiCredentials>("fw-primary")
// ConfigClient::get_named::<OPNSenseApiCredentials>("fw-primary")
let ssh_creds = SecretManager::get_or_prompt::<OPNSenseFirewallCredentials>()
.await
.expect("Failed to get SSH credentials");

8
harmony_config/examples/basic.rs
View File

@@ -12,7 +12,7 @@
use std::sync::Arc;
use harmony_config::{Config, ConfigManager, EnvSource, SqliteSource};
use harmony_config::{Config, ConfigClient, EnvSource, SqliteSource};
use log::info;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
@@ -36,8 +36,10 @@ impl Default for TestConfig {
async fn main() -> anyhow::Result<()> {
env_logger::init();
let sqlite = SqliteSource::default().await?;
let manager = ConfigManager::new(vec![Arc::new(EnvSource), Arc::new(sqlite)]);
// Namespace the SQLite file so this example's state doesn't
// collide with other harmony binaries that also use SqliteSource.
let sqlite = SqliteSource::for_namespace("harmony_config-basic-example").await?;
let manager = ConfigClient::new(vec![Arc::new(EnvSource), Arc::new(sqlite)]);
info!("1. Attempting to get TestConfig (expect NotFound on first run)...");
match manager.get::<TestConfig>().await {

214
harmony_config/examples/openbao_chain.rs
View File

@@ -1,60 +1,24 @@
//! End-to-end example: harmony_config with OpenBao as a ConfigSource
//! Dev-binary template: `ConfigClient` against an OpenBao-backed chain.
//!
//! This example demonstrates the full config resolution chain:
//! EnvSource → SqliteSource → StoreSource<OpenbaoSecretStore>
//! Bring up the OpenBao + Zitadel stack first (`cargo run -p
//! example-harmony-sso`), then export the connection env vars and run this:
//!
//! When OpenBao is unreachable, the chain gracefully falls through to SQLite.
//!
//! **Prerequisites**:
//! - OpenBao must be initialized and unsealed
//! - KV v2 engine must be enabled at the `OPENBAO_KV_MOUNT` path (default: `secret`)
//! - Auth method must be enabled at the `OPENBAO_AUTH_MOUNT` path (default: `userpass`)
//!
//! **Environment variables**:
//! - `OPENBAO_URL` (required for OpenBao): URL of the OpenBao server
//! - `OPENBAO_TOKEN` (optional): Use token auth instead of userpass
//! - `OPENBAO_USERNAME` + `OPENBAO_PASSWORD` (optional): Userpass auth
//! - `OPENBAO_KV_MOUNT` (default: `secret`): KV v2 engine mount path
//! - `OPENBAO_AUTH_MOUNT` (default: `userpass`): Auth method mount path
//! - `OPENBAO_SKIP_TLS` (default: `false`): Skip TLS verification
//! - `HARMONY_SSO_URL` + `HARMONY_SSO_CLIENT_ID` (optional): Zitadel OIDC device flow (RFC 8628)
//!
//! **Run**:
//! ```bash
//! # Without OpenBao (SqliteSource only):
//! cargo run --example openbao_chain
//!
//! # With OpenBao (full chain):
//! export OPENBAO_URL="http://127.0.0.1:8200"
//! export OPENBAO_TOKEN="<your-token>"
//! cargo run --example openbao_chain
//! export OPENBAO_URL=http://bao.harmony.local:8080
//! export HARMONY_SSO_URL=http://sso.harmony.local:8080
//! export HARMONY_SSO_CLIENT_ID=<the harmony-cli client id> # or OPENBAO_TOKEN=<token>
//! cargo run -p harmony_config --example openbao_chain
//! ```
//!
//! **Setup OpenBao** (if needed):
//! ```bash
//! # Port-forward to local OpenBao
//! kubectl port-forward svc/openbao -n openbao 8200:8200 &
//!
//! # Initialize (one-time)
//! kubectl exec -n openbao openbao-0 -- bao operator init
//!
//! # Enable KV and userpass (one-time)
//! kubectl exec -n openbao openbao-0 -- bao secrets enable -path=secret kv-v2
//! kubectl exec -n openbao openbao-0 -- bao auth enable userpass
//!
//! # Create test user
//! kubectl exec -n openbao openbao-0 -- bao write auth/userpass/users/testuser \
//! password="testpass" policies="default"
//! ```
//! If OpenBao is unreachable the chain degrades to env → prompt, so the
//! round-trip steps below need a reachable OpenBao to persist.
use std::sync::Arc;
use harmony_config::{Config, ConfigManager, ConfigSource, EnvSource, SqliteSource, StoreSource};
use harmony_secret::OpenbaoSecretStore;
use harmony_config::{Config, ConfigClient, ConfigExt};
use log::info;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq)]
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct AppConfig {
host: String,
port: u16,
@@ -63,130 +27,70 @@ struct AppConfig {
impl Default for AppConfig {
fn default() -> Self {
Self {
host: "localhost".to_string(),
port: 8080,
host: "production.example.com".to_string(),
port: 443,
}
}
}
impl Config for AppConfig {
const KEY: &'static str = "AppConfig";
// `password` is `#[config(secret)]`, so the whole struct is Secret-class:
// masked in logs and prompted via `inquire::Password`.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct DatabaseCredentials {
host: String,
username: String,
#[config(secret)]
password: String,
}
async fn build_manager() -> ConfigManager {
let sqlite = Arc::new(
SqliteSource::default()
.await
.expect("Failed to open SQLite database"),
);
impl Default for DatabaseCredentials {
fn default() -> Self {
Self {
host: "db.example.com".to_string(),
username: "app_rw".to_string(),
password: "rotate-me-please".to_string(),
}
}
}
let env_source: Arc<dyn ConfigSource> = Arc::new(EnvSource);
// No `Default`, so `get_or_prompt` falls through to an interactive prompt.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct ApiCredentials {
client_id: String,
#[config(secret)]
client_secret: String,
}
let openbao_url = std::env::var("OPENBAO_URL")
.or_else(|_| std::env::var("VAULT_ADDR"))
.ok();
match openbao_url {
Some(url) => {
let kv_mount =
std::env::var("OPENBAO_KV_MOUNT").unwrap_or_else(|_| "secret".to_string());
let auth_mount =
std::env::var("OPENBAO_AUTH_MOUNT").unwrap_or_else(|_| "userpass".to_string());
let skip_tls = std::env::var("OPENBAO_SKIP_TLS")
.map(|v| v == "true")
.unwrap_or(false);
match OpenbaoSecretStore::new(
url,
kv_mount,
auth_mount,
skip_tls,
std::env::var("OPENBAO_TOKEN").ok(),
std::env::var("OPENBAO_USERNAME").ok(),
std::env::var("OPENBAO_PASSWORD").ok(),
std::env::var("HARMONY_SSO_URL").ok(),
std::env::var("HARMONY_SSO_CLIENT_ID").ok(),
None,
None,
)
.await
{
Ok(store) => {
let store_source: Arc<dyn ConfigSource> =
Arc::new(StoreSource::new("harmony".to_string(), store));
println!("OpenBao connected. Full chain: env → sqlite → openbao");
ConfigManager::new(vec![env_source, Arc::clone(&sqlite) as _, store_source])
}
Err(e) => {
eprintln!(
"Warning: OpenBao unavailable ({e}), using local chain: env → sqlite"
);
ConfigManager::new(vec![env_source, sqlite])
}
}
}
None => {
println!("OPENBAO_URL not set. Using local chain: env → sqlite");
ConfigManager::new(vec![env_source, sqlite])
// Read `T`, or store `default` and read it back to prove the round-trip.
async fn round_trip<T>(client: &ConfigClient, default: T) -> anyhow::Result<()>
where
T: Config + std::fmt::Debug + Clone + PartialEq,
{
match client.get::<T>().await {
Ok(found) => info!("[{}] read existing: {:?}", T::KEY, found.masked()),
Err(harmony_config::ConfigError::NotFound { .. }) => {
client.set(&default).await?;
let back: T = client.get().await?;
anyhow::ensure!(back == default, "round-trip mismatch for {}", T::KEY);
info!("[{}] stored + verified: {:?}", T::KEY, back.masked());
}
Err(e) => return Err(e.into()),
}
Ok(())
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
env_logger::init();
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
let manager = build_manager().await;
let client = ConfigClient::for_namespace("harmony").await;
println!("\n=== harmony_config OpenBao Chain Demo ===\n");
round_trip(&client, AppConfig::default()).await?;
round_trip(&client, DatabaseCredentials::default()).await?;
println!("1. Attempting to get AppConfig (expect NotFound on first run)...");
match manager.get::<AppConfig>().await {
Ok(config) => {
println!(" Found: {:?}", config);
}
Err(harmony_config::ConfigError::NotFound { .. }) => {
println!(" NotFound - no config stored yet");
}
Err(e) => {
println!(" Error: {:?}", e);
}
}
println!("\n2. Setting AppConfig via set()...");
let config = AppConfig {
host: "production.example.com".to_string(),
port: 443,
};
manager.set(&config).await?;
println!(" Set: {:?}", config);
println!("\n3. Getting AppConfig back...");
let retrieved: AppConfig = manager.get().await?;
println!(" Retrieved: {:?}", retrieved);
assert_eq!(config, retrieved);
println!("\n4. Demonstrating env override...");
println!(" HARMONY_CONFIG_AppConfig env var overrides all backends");
let env_config = AppConfig {
host: "env-override.example.com".to_string(),
port: 9090,
};
unsafe {
std::env::set_var(
"HARMONY_CONFIG_AppConfig",
serde_json::to_string(&env_config)?,
);
}
let from_env: AppConfig = manager.get().await?;
println!(" Got from env: {:?}", from_env);
assert_eq!(env_config.host, "env-override.example.com");
unsafe {
std::env::remove_var("HARMONY_CONFIG_AppConfig");
}
println!("\n=== Done! ===");
println!("Config persisted at ~/.local/share/harmony/config/config.db");
// No default + Secret class: prompts, with `client_secret` read via Password.
let api: ApiCredentials = client.get_or_prompt().await?;
info!("[{}] resolved: {:?}", ApiCredentials::KEY, api.masked());
Ok(())
}

8
harmony_config/examples/prompting.rs
View File

@@ -12,7 +12,7 @@
use std::sync::Arc;
use harmony_config::{Config, ConfigManager, EnvSource, PromptSource, SqliteSource};
use harmony_config::{Config, ConfigClient, EnvSource, PromptSource, SqliteSource};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
@@ -27,8 +27,10 @@ struct UserConfig {
async fn main() -> anyhow::Result<()> {
env_logger::init();
let sqlite = SqliteSource::default().await?;
let manager = ConfigManager::new(vec![
// Namespace the SQLite file so this example's state doesn't
// collide with other harmony binaries that also use SqliteSource.
let sqlite = SqliteSource::for_namespace("harmony_config-prompting-example").await?;
let manager = ConfigClient::new(vec![
Arc::new(EnvSource),
Arc::new(sqlite),
Arc::new(PromptSource::new()),

645
harmony_config/src/lib.rs
View File

@@ -1,9 +1,13 @@
// Lets the derive macro emit `::harmony_config::…` paths that resolve both
// inside this crate and in downstream consumers.
extern crate self as harmony_config;
mod source;
use async_trait::async_trait;
use directories::ProjectDirs;
use interactive_parse::InteractiveParseObj;
use log::debug;
use log::warn;
use schemars::JsonSchema;
use serde::{Serialize, de::DeserializeOwned};
use std::path::PathBuf;
@@ -57,40 +61,107 @@ pub enum ConfigError {
SqliteError(String),
}
pub trait Config: Serialize + DeserializeOwned + JsonSchema + InteractiveParseObj + Sized {
const KEY: &'static str;
/// Hint passed through the `ConfigSource` chain; backends that care
/// (masking, prompting) inspect it, others ignore it. Elevated to `Secret`
/// automatically when any field — or the struct — is `#[config(secret)]`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum ConfigClass {
Standard,
Secret,
}
#[async_trait]
pub trait ConfigSource: Send + Sync {
async fn get(&self, key: &str) -> Result<Option<serde_json::Value>, ConfigError>;
pub trait Config: Serialize + DeserializeOwned + JsonSchema + InteractiveParseObj + Sized {
const KEY: &'static str;
const CLASS: ConfigClass = ConfigClass::Standard;
// Serialized names of `#[config(secret)]` fields, used for masking and
// password prompts. Matching is on the serialized name, so a secret
// field must not be `#[serde(rename)]`d — the derive records the Rust
// ident, and a rename would leak the value in cleartext.
const SECRET_FIELDS: &'static [&'static str] = &[];
}
async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError>;
/// Safe-display wrapper: `{:?}` renders `T::SECRET_FIELDS` as `"****"`.
/// Use it on any `Config` value headed for a log line or error message.
pub struct Masked<'a, T: Config>(&'a T);
fn should_persist(&self) -> bool {
true
impl<T: Config> std::fmt::Debug for Masked<'_, T> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let mut value = serde_json::to_value(self.0).map_err(|_| std::fmt::Error)?;
if let serde_json::Value::Object(map) = &mut value {
for name in T::SECRET_FIELDS {
if let Some(v) = map.get_mut(*name) {
*v = serde_json::Value::String("****".to_string());
}
}
}
write!(f, "{value:#}")
}
}
pub struct ConfigManager {
pub trait ConfigExt: Config {
fn masked(&self) -> Masked<'_, Self> {
Masked(self)
}
}
impl<T: Config> ConfigExt for T {}
#[async_trait]
pub trait ConfigSource: Send + Sync {
// `Ok(None)` means "absent here, try the next source"; `Err`
// short-circuits the chain. `class` is a hint most sources ignore.
async fn get(
&self,
class: ConfigClass,
key: &str,
) -> Result<Option<serde_json::Value>, ConfigError>;
async fn set(
&self,
class: ConfigClass,
key: &str,
value: &serde_json::Value,
) -> Result<(), ConfigError>;
}
pub struct ConfigClient {
sources: Vec<Arc<dyn ConfigSource>>,
}
impl ConfigManager {
impl ConfigClient {
pub fn new(sources: Vec<Arc<dyn ConfigSource>>) -> Self {
Self { sources }
}
/// Canonical chain `EnvSource → OpenBao → PromptSource`, scoped to
/// `namespace`. OpenBao is configured from env vars (see
/// [`openbao_from_env`]) and dropped from the chain if unset or
/// unreachable. SQLite is excluded on purpose: it stores cleartext on
/// disk and can't safely hold `Secret`-class config. For other chains,
/// build the sources yourself and pass them to [`ConfigClient::new`].
pub async fn for_namespace(namespace: &str) -> Self {
let mut sources: Vec<Arc<dyn ConfigSource>> = vec![Arc::new(EnvSource)];
if let Some(store) = openbao_from_env(namespace).await {
sources.push(store);
}
sources.push(Arc::new(PromptSource::new()));
Self::new(sources)
}
pub async fn get<T: Config>(&self) -> Result<T, ConfigError> {
for source in &self.sources {
if let Some(value) = source.get(T::KEY).await? {
let config: T =
serde_json::from_value(value).map_err(|e| ConfigError::Deserialization {
key: T::KEY.to_string(),
source: e,
})?;
debug!("Retrieved config for key {} from source", T::KEY);
return Ok(config);
if let Some(value) = source.get(T::CLASS, T::KEY).await? {
// A deser failure means the stored value is shaped for a
// different version of the struct (branch switch, rename);
// treat it as a miss for this source and fall through so a
// later source — or a re-prompt — overwrites the stale entry.
match serde_json::from_value::<T>(value) {
Ok(config) => return Ok(config),
Err(e) => warn!(
"Stale value for key {} in source; falling through ({e})",
T::KEY
),
}
}
}
Err(ConfigError::NotFound {
@@ -102,24 +173,8 @@ impl ConfigManager {
match self.get::<T>().await {
Ok(config) => Ok(config),
Err(ConfigError::NotFound { .. }) => {
let config =
T::parse_to_obj().map_err(|e| ConfigError::PromptError(e.to_string()))?;
let value =
serde_json::to_value(&config).map_err(|e| ConfigError::Serialization {
key: T::KEY.to_string(),
source: e,
})?;
for source in &self.sources {
if !source.should_persist() {
continue;
}
if source.set(T::KEY, &value).await.is_ok() {
break;
}
}
let config = PromptSource::new().prompt_for::<T>().await?;
self.set(&config).await?;
Ok(config)
}
Err(e) => Err(e),
@@ -133,22 +188,61 @@ impl ConfigManager {
})?;
for source in &self.sources {
source.set(T::KEY, &value).await?;
source.set(T::CLASS, T::KEY, &value).await?;
}
Ok(())
}
}
static CONFIG_MANAGER: Mutex<Option<Arc<ConfigManager>>> = Mutex::const_new(None);
/// Build an OpenBao-backed `StoreSource` from env vars, or `None` (with a
/// `warn!`) when `OPENBAO_URL`/`VAULT_ADDR` is unset or the connection
/// fails — so a missing OpenBao degrades the chain instead of breaking
/// startup. `jwt_role`/`jwt_auth_mount` are left to `OpenbaoSecretStore`'s
/// defaults, matching what `OpenbaoSetupScore` configures.
async fn openbao_from_env(namespace: &str) -> Option<Arc<dyn ConfigSource>> {
let Some(url) = std::env::var("OPENBAO_URL")
.or_else(|_| std::env::var("VAULT_ADDR"))
.ok()
else {
warn!("OpenBao URL not set; OpenBao source omitted from chain");
return None;
};
let env = |k: &str| std::env::var(k).ok();
let store = harmony_secret::OpenbaoSecretStore::new(
url,
env("OPENBAO_KV_MOUNT").unwrap_or_else(|| "secret".to_string()),
env("OPENBAO_AUTH_MOUNT").unwrap_or_else(|| "jwt".to_string()),
env("OPENBAO_SKIP_TLS").as_deref() == Some("true"),
env("OPENBAO_TOKEN"),
env("OPENBAO_USERNAME"),
env("OPENBAO_PASSWORD"),
env("HARMONY_SSO_URL"),
env("HARMONY_SSO_CLIENT_ID"),
None,
None,
)
.await;
match store {
Ok(store) => Some(Arc::new(StoreSource::new(namespace.to_string(), store))),
Err(e) => {
warn!("OpenBao unreachable ({e}); source omitted from chain");
None
}
}
}
static CONFIG_CLIENT: Mutex<Option<Arc<ConfigClient>>> = Mutex::const_new(None);
pub async fn init(sources: Vec<Arc<dyn ConfigSource>>) {
let mut manager = CONFIG_MANAGER.lock().await;
*manager = Some(Arc::new(ConfigManager::new(sources)));
let mut manager = CONFIG_CLIENT.lock().await;
*manager = Some(Arc::new(ConfigClient::new(sources)));
}
pub async fn get<T: Config>() -> Result<T, ConfigError> {
let manager = CONFIG_MANAGER.lock().await;
let manager = CONFIG_CLIENT.lock().await;
manager
.as_ref()
.ok_or(ConfigError::NoSources)?
@@ -157,7 +251,7 @@ pub async fn get<T: Config>() -> Result<T, ConfigError> {
}
pub async fn get_or_prompt<T: Config>() -> Result<T, ConfigError> {
let manager = CONFIG_MANAGER.lock().await;
let manager = CONFIG_CLIENT.lock().await;
manager
.as_ref()
.ok_or(ConfigError::NoSources)?
@@ -166,7 +260,7 @@ pub async fn get_or_prompt<T: Config>() -> Result<T, ConfigError> {
}
pub async fn set<T: Config>(config: &T) -> Result<(), ConfigError> {
let manager = CONFIG_MANAGER.lock().await;
let manager = CONFIG_CLIENT.lock().await;
manager
.as_ref()
.ok_or(ConfigError::NoSources)?
@@ -229,6 +323,10 @@ mod tests {
data: std::sync::Mutex<std::collections::HashMap<String, serde_json::Value>>,
get_count: AtomicUsize,
set_count: AtomicUsize,
/// Records each `(class, key)` pair observed on `get` / `set`
/// so tests can assert that `ConfigClient` forwards `T::CLASS`
/// to sources unchanged.
observed: std::sync::Mutex<Vec<(ConfigClass, String, &'static str)>>,
}
impl MockSource {
@@ -237,6 +335,7 @@ mod tests {
data: std::sync::Mutex::new(std::collections::HashMap::new()),
get_count: AtomicUsize::new(0),
set_count: AtomicUsize::new(0),
observed: std::sync::Mutex::new(Vec::new()),
}
}
@@ -245,6 +344,7 @@ mod tests {
data: std::sync::Mutex::new(data),
get_count: AtomicUsize::new(0),
set_count: AtomicUsize::new(0),
observed: std::sync::Mutex::new(Vec::new()),
}
}
@@ -255,18 +355,39 @@ mod tests {
fn set_call_count(&self) -> usize {
self.set_count.load(Ordering::SeqCst)
}
fn observed(&self) -> Vec<(ConfigClass, String, &'static str)> {
self.observed.lock().unwrap().clone()
}
}
#[async_trait]
impl ConfigSource for MockSource {
async fn get(&self, key: &str) -> Result<Option<serde_json::Value>, ConfigError> {
async fn get(
&self,
class: ConfigClass,
key: &str,
) -> Result<Option<serde_json::Value>, ConfigError> {
self.get_count.fetch_add(1, Ordering::SeqCst);
self.observed
.lock()
.unwrap()
.push((class, key.to_string(), "get"));
let data = self.data.lock().unwrap();
Ok(data.get(key).cloned())
}
async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> {
async fn set(
&self,
class: ConfigClass,
key: &str,
value: &serde_json::Value,
) -> Result<(), ConfigError> {
self.set_count.fetch_add(1, Ordering::SeqCst);
self.observed
.lock()
.unwrap()
.push((class, key.to_string(), "set"));
let mut data = self.data.lock().unwrap();
data.insert(key.to_string(), value.clone());
Ok(())
@@ -286,7 +407,7 @@ mod tests {
);
let source = Arc::new(MockSource::with_data(data));
let manager = ConfigManager::new(vec![source.clone()]);
let manager = ConfigClient::new(vec![source.clone()]);
let result: TestConfig = manager.get().await.unwrap();
assert_eq!(result, config);
@@ -307,7 +428,7 @@ mod tests {
let source1 = Arc::new(MockSource::new());
let source2 = Arc::new(MockSource::with_data(data2));
let manager = ConfigManager::new(vec![source1.clone(), source2.clone()]);
let manager = ConfigClient::new(vec![source1.clone(), source2.clone()]);
let result: TestConfig = manager.get().await.unwrap();
assert_eq!(result, config);
@@ -318,7 +439,7 @@ mod tests {
#[tokio::test]
async fn test_get_returns_not_found_when_no_source_has_key() {
let source = Arc::new(MockSource::new());
let manager = ConfigManager::new(vec![source.clone()]);
let manager = ConfigClient::new(vec![source.clone()]);
let result: Result<TestConfig, ConfigError> = manager.get().await;
assert!(matches!(result, Err(ConfigError::NotFound { .. })));
@@ -326,7 +447,7 @@ mod tests {
#[tokio::test]
async fn test_get_returns_error_with_no_sources() {
let manager = ConfigManager::new(vec![]);
let manager = ConfigClient::new(vec![]);
let result: Result<TestConfig, ConfigError> = manager.get().await;
assert!(matches!(result, Err(ConfigError::NotFound { .. })));
@@ -341,7 +462,7 @@ mod tests {
let source1 = Arc::new(MockSource::new());
let source2 = Arc::new(MockSource::new());
let manager = ConfigManager::new(vec![source1.clone(), source2.clone()]);
let manager = ConfigClient::new(vec![source1.clone(), source2.clone()]);
manager.set(&config).await.unwrap();
@@ -352,6 +473,228 @@ mod tests {
assert_eq!(result1, config);
}
#[tokio::test]
async fn test_derive_macro_emits_standard_class_with_no_secret_fields() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct PlainConfig {
host: String,
port: u16,
}
assert_eq!(PlainConfig::KEY, "PlainConfig");
assert_eq!(PlainConfig::CLASS, ConfigClass::Standard);
}
#[tokio::test]
async fn test_derive_macro_emits_secret_class_when_field_is_tagged() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct WithSecretField {
host: String,
#[config(secret)]
password: String,
}
assert_eq!(WithSecretField::CLASS, ConfigClass::Secret);
}
#[tokio::test]
async fn test_derive_macro_emits_secret_class_when_struct_is_tagged() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
#[config(secret)]
struct AllSecret {
token: String,
}
assert_eq!(AllSecret::CLASS, ConfigClass::Secret);
}
// -- SECRET_FIELDS derivation -----------------------------------------
#[tokio::test]
async fn test_derive_emits_empty_secret_fields_for_standard_struct() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct StandardPlain {
host: String,
port: u16,
}
assert_eq!(StandardPlain::CLASS, ConfigClass::Standard);
assert_eq!(StandardPlain::SECRET_FIELDS, &[] as &[&str]);
}
#[tokio::test]
async fn test_derive_emits_tagged_fields_for_per_field_secret() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct MixedSecret {
host: String,
#[config(secret)]
password: String,
#[config(secret)]
api_key: String,
non_secret: u16,
}
assert_eq!(MixedSecret::CLASS, ConfigClass::Secret);
// Names emitted in struct-field order; only the tagged ones.
assert_eq!(MixedSecret::SECRET_FIELDS, &["password", "api_key"]);
}
#[tokio::test]
async fn test_derive_emits_all_fields_for_struct_level_secret() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
#[config(secret)]
struct WholeStructSecret {
token: String,
refresh: String,
count: u32,
}
assert_eq!(WholeStructSecret::CLASS, ConfigClass::Secret);
assert_eq!(
WholeStructSecret::SECRET_FIELDS,
&["token", "refresh", "count"]
);
}
// -- Masked wrapper / ConfigExt::masked() -----------------------------
#[tokio::test]
async fn test_masked_replaces_secret_fields_with_stars() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct Mixed {
host: String,
#[config(secret)]
password: String,
}
let v = Mixed {
host: "db.example.com".to_string(),
password: "hunter2".to_string(),
};
let s = format!("{:?}", v.masked());
assert!(s.contains("db.example.com"), "host should be visible: {s}");
assert!(
s.contains("\"****\""),
"password should render as ****: {s}"
);
assert!(!s.contains("hunter2"), "raw password must not appear: {s}");
}
#[tokio::test]
async fn test_masked_passes_standard_struct_through_unmodified() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct PlainStandard {
name: String,
count: u32,
}
let v = PlainStandard {
name: "alice".to_string(),
count: 7,
};
let s = format!("{:?}", v.masked());
assert!(s.contains("alice"));
assert!(s.contains('7'));
assert!(!s.contains("****"));
}
#[tokio::test]
async fn test_masked_masks_every_field_for_struct_level_secret() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
#[config(secret)]
struct AllSecret {
token: String,
refresh: String,
}
let v = AllSecret {
token: "tok123".to_string(),
refresh: "ref456".to_string(),
};
let s = format!("{:?}", v.masked());
assert!(!s.contains("tok123"), "token must be masked: {s}");
assert!(!s.contains("ref456"), "refresh must be masked: {s}");
// Both fields should render as "****".
assert_eq!(s.matches("\"****\"").count(), 2, "got: {s}");
}
#[tokio::test]
async fn test_manual_impl_defaults_to_standard_class() {
// The trait gives CLASS a default of Standard so existing manual
// impls (like TestConfig above) compile without explicit updates.
assert_eq!(TestConfig::CLASS, ConfigClass::Standard);
}
#[tokio::test]
async fn test_config_manager_forwards_standard_class_to_sources_on_set() {
let source = Arc::new(MockSource::new());
let manager = ConfigClient::new(vec![source.clone()]);
let config = TestConfig {
name: "x".into(),
count: 1,
};
manager.set(&config).await.unwrap();
let observed = source.observed();
assert_eq!(observed.len(), 1);
let (class, key, op) = &observed[0];
assert_eq!(*class, ConfigClass::Standard);
assert_eq!(key, "TestConfig");
assert_eq!(*op, "set");
}
#[tokio::test]
async fn test_config_manager_forwards_secret_class_to_sources_on_set() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
struct SecretConfig {
#[config(secret)]
password: String,
}
assert_eq!(SecretConfig::CLASS, ConfigClass::Secret);
let source = Arc::new(MockSource::new());
let manager = ConfigClient::new(vec![source.clone()]);
manager
.set(&SecretConfig {
password: "hunter2".into(),
})
.await
.unwrap();
let (class, key, op) = source.observed().pop().unwrap();
assert_eq!(class, ConfigClass::Secret);
assert_eq!(key, "SecretConfig");
assert_eq!(op, "set");
}
#[tokio::test]
async fn test_config_manager_forwards_class_to_every_source_on_get() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq, Config)]
#[config(secret)]
struct StructLevelSecret {
token: String,
}
// Two empty sources so the chain probes both before returning NotFound.
let s1 = Arc::new(MockSource::new());
let s2 = Arc::new(MockSource::new());
let manager = ConfigClient::new(vec![s1.clone(), s2.clone()]);
let res: Result<StructLevelSecret, _> = manager.get().await;
assert!(matches!(res, Err(ConfigError::NotFound { .. })));
for src in [&s1, &s2] {
let observed = src.observed();
assert_eq!(observed.len(), 1);
let (class, key, op) = &observed[0];
assert_eq!(*class, ConfigClass::Secret);
assert_eq!(key, "StructLevelSecret");
assert_eq!(*op, "get");
}
}
#[tokio::test]
async fn test_derive_macro_generates_correct_key() {
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, PartialEq)]
@@ -372,7 +715,12 @@ mod tests {
let env_var = setup_env_vars("TestConfig", Some(r#"{"name":"from_env","count":7}"#));
let source = EnvSource;
let result = source.get(&env_var.replace("HARMONY_CONFIG_", "")).await;
let result = source
.get(
ConfigClass::Standard,
&env_var.replace("HARMONY_CONFIG_", ""),
)
.await;
unsafe {
std::env::remove_var(&env_var);
@@ -396,7 +744,12 @@ mod tests {
.unwrap();
rt.block_on(async {
let source = EnvSource;
let result = source.get(&env_var.replace("HARMONY_CONFIG_", "")).await;
let result = source
.get(
ConfigClass::Standard,
&env_var.replace("HARMONY_CONFIG_", ""),
)
.await;
assert!(result.unwrap().is_none());
});
});
@@ -408,7 +761,12 @@ mod tests {
let env_var = setup_env_vars("TestConfig", Some("not valid json"));
let source = EnvSource;
let result = source.get(&env_var.replace("HARMONY_CONFIG_", "")).await;
let result = source
.get(
ConfigClass::Standard,
&env_var.replace("HARMONY_CONFIG_", ""),
)
.await;
unsafe {
std::env::remove_var(&env_var);
@@ -430,7 +788,11 @@ mod tests {
std::fs::write(&config_path, serde_json::to_string(&config).unwrap()).unwrap();
let source = LocalFileSource::new(dir.path().to_path_buf());
let result = source.get("TestConfig").await.unwrap().unwrap();
let result = source
.get(ConfigClass::Standard, "TestConfig")
.await
.unwrap()
.unwrap();
let parsed: TestConfig = serde_json::from_value(result).unwrap();
assert_eq!(parsed, config);
@@ -442,7 +804,10 @@ mod tests {
let dir = tempdir().unwrap();
let source = LocalFileSource::new(dir.path().to_path_buf());
let result = source.get("NonExistentConfig").await.unwrap();
let result = source
.get(ConfigClass::Standard, "NonExistentConfig")
.await
.unwrap();
assert!(result.is_none());
}
@@ -459,7 +824,11 @@ mod tests {
};
source
.set("TestConfig", &serde_json::to_value(&config).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config).unwrap(),
)
.await
.unwrap();
@@ -484,11 +853,19 @@ mod tests {
};
source
.set("TestConfig", &serde_json::to_value(&config).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config).unwrap(),
)
.await
.unwrap();
let result = source.get("TestConfig").await.unwrap().unwrap();
let result = source
.get(ConfigClass::Standard, "TestConfig")
.await
.unwrap()
.unwrap();
let parsed: TestConfig = serde_json::from_value(result).unwrap();
assert_eq!(parsed, config);
@@ -502,7 +879,10 @@ mod tests {
let path = temp_file.path().to_path_buf();
let source = SqliteSource::open(path).await.unwrap();
let result = source.get("NonExistentConfig").await.unwrap();
let result = source
.get(ConfigClass::Standard, "NonExistentConfig")
.await
.unwrap();
assert!(result.is_none());
}
@@ -525,15 +905,27 @@ mod tests {
};
source
.set("TestConfig", &serde_json::to_value(&config1).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config1).unwrap(),
)
.await
.unwrap();
source
.set("TestConfig", &serde_json::to_value(&config2).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config2).unwrap(),
)
.await
.unwrap();
let result = source.get("TestConfig").await.unwrap().unwrap();
let result = source
.get(ConfigClass::Standard, "TestConfig")
.await
.unwrap()
.unwrap();
let parsed: TestConfig = serde_json::from_value(result).unwrap();
assert_eq!(parsed, config2);
@@ -561,17 +953,33 @@ mod tests {
let (r1, r2) = tokio::join!(
async {
source
.set("key1", &serde_json::to_value(&config1).unwrap())
.set(
ConfigClass::Standard,
"key1",
&serde_json::to_value(&config1).unwrap(),
)
.await
.unwrap();
source.get("key1").await.unwrap().unwrap()
source
.get(ConfigClass::Standard, "key1")
.await
.unwrap()
.unwrap()
},
async {
source
.set("key2", &serde_json::to_value(&config2).unwrap())
.set(
ConfigClass::Standard,
"key2",
&serde_json::to_value(&config2).unwrap(),
)
.await
.unwrap();
source.get("key2").await.unwrap().unwrap()
source
.get(ConfigClass::Standard, "key2")
.await
.unwrap()
.unwrap()
}
);
@@ -608,7 +1016,7 @@ mod tests {
let source1 = Arc::new(MockSource::with_data(data));
let source2 = Arc::new(MockSource::new());
let manager = ConfigManager::new(vec![source1.clone(), source2.clone()]);
let manager = ConfigClient::new(vec![source1.clone(), source2.clone()]);
let result: TestConfig = manager.get_or_prompt().await.unwrap();
assert_eq!(result, config);
@@ -626,7 +1034,7 @@ mod tests {
.unwrap();
let sqlite = Arc::new(sqlite);
let manager = ConfigManager::new(vec![sqlite.clone()]);
let manager = ConfigClient::new(vec![sqlite.clone()]);
let config = TestConfig {
name: "from_sqlite".to_string(),
@@ -634,7 +1042,11 @@ mod tests {
};
sqlite
.set("TestConfig", &serde_json::to_value(&config).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config).unwrap(),
)
.await
.unwrap();
@@ -653,7 +1065,7 @@ mod tests {
let sqlite = Arc::new(sqlite);
let env_source = Arc::new(EnvSource);
let manager = ConfigManager::new(vec![env_source.clone(), sqlite.clone()]);
let manager = ConfigClient::new(vec![env_source.clone(), sqlite.clone()]);
let sqlite_config = TestConfig {
name: "from_sqlite".to_string(),
@@ -665,7 +1077,11 @@ mod tests {
};
sqlite
.set("TestConfig", &serde_json::to_value(&sqlite_config).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&sqlite_config).unwrap(),
)
.await
.unwrap();
@@ -684,7 +1100,13 @@ mod tests {
}
#[tokio::test]
async fn test_branch_switching_scenario_deserialization_error() {
async fn test_branch_switching_scenario_deserialization_falls_through() {
// ADR 020 contract: a stale value (wrong shape for the current
// struct, e.g. after a branch switch that renamed or retyped a
// field) should be treated as a cache miss for that source. The
// chain falls through; if no source has a valid value, the result
// is `NotFound`, which `get_or_prompt` turns into a re-prompt
// that overwrites the stale entry.
use tempfile::NamedTempFile;
let temp_file = NamedTempFile::new().unwrap();
@@ -693,22 +1115,61 @@ mod tests {
.unwrap();
let sqlite = Arc::new(sqlite);
let manager = ConfigManager::new(vec![sqlite.clone()]);
let manager = ConfigClient::new(vec![sqlite.clone()]);
let old_config = serde_json::json!({
"name": "old_config",
"count": "not_a_number"
});
sqlite.set("TestConfig", &old_config).await.unwrap();
sqlite
.set(ConfigClass::Standard, "TestConfig", &old_config)
.await
.unwrap();
let result: Result<TestConfig, ConfigError> = manager.get().await;
assert!(matches!(result, Err(ConfigError::Deserialization { .. })));
assert!(matches!(result, Err(ConfigError::NotFound { .. })));
}
#[tokio::test]
async fn test_deserialization_failure_falls_through_to_next_source() {
// First source has a stale value; second source has a fresh one.
// The chain should skip past the stale entry and resolve from
// the second source.
use tempfile::NamedTempFile;
let temp_file = NamedTempFile::new().unwrap();
let stale_sqlite = SqliteSource::open(temp_file.path().to_path_buf())
.await
.unwrap();
let stale_sqlite = Arc::new(stale_sqlite);
let stale = serde_json::json!({"name": "stale", "count": "not_a_number"});
stale_sqlite
.set(ConfigClass::Standard, "TestConfig", &stale)
.await
.unwrap();
let fresh_config = TestConfig {
name: "fresh".to_string(),
count: 7,
};
let mut fresh_data = std::collections::HashMap::new();
fresh_data.insert(
"TestConfig".to_string(),
serde_json::to_value(&fresh_config).unwrap(),
);
let fresh = Arc::new(MockSource::with_data(fresh_data));
let manager = ConfigClient::new(vec![stale_sqlite.clone(), fresh.clone()]);
let result: TestConfig = manager.get().await.unwrap();
assert_eq!(result, fresh_config);
}
#[tokio::test]
async fn test_prompt_source_always_returns_none() {
let source = PromptSource::new();
let result = source.get("AnyKey").await.unwrap();
let result = source.get(ConfigClass::Standard, "AnyKey").await.unwrap();
assert!(result.is_none());
}
@@ -716,7 +1177,11 @@ mod tests {
async fn test_prompt_source_set_is_noop() {
let source = PromptSource::new();
let result = source
.set("AnyKey", &serde_json::json!({"test": "value"}))
.set(
ConfigClass::Standard,
"AnyKey",
&serde_json::json!({"test": "value"}),
)
.await;
assert!(result.is_ok());
}
@@ -726,13 +1191,17 @@ mod tests {
let source = PromptSource::new();
source
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::json!({"name": "test", "count": 42}),
)
.await
.unwrap();
let result = source.get("TestConfig").await.unwrap();
let result = source
.get(ConfigClass::Standard, "TestConfig")
.await
.unwrap();
assert!(result.is_none());
}
@@ -750,7 +1219,7 @@ mod tests {
let prompt_source = Arc::new(PromptSource::new());
let manager =
ConfigManager::new(vec![source1.clone(), sqlite.clone(), prompt_source.clone()]);
ConfigClient::new(vec![source1.clone(), sqlite.clone(), prompt_source.clone()]);
let result: Result<TestConfig, ConfigError> = manager.get().await;
assert!(matches!(result, Err(ConfigError::NotFound { .. })));
@@ -781,14 +1250,18 @@ mod tests {
let store_source = Arc::new(StoreSource::new("test".to_string(), AlwaysErrorStore));
let manager = ConfigManager::new(vec![store_source.clone(), sqlite.clone()]);
let manager = ConfigClient::new(vec![store_source.clone(), sqlite.clone()]);
let config = TestConfig {
name: "from_sqlite".to_string(),
count: 42,
};
sqlite
.set("TestConfig", &serde_json::to_value(&config).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config).unwrap(),
)
.await
.unwrap();
@@ -825,14 +1298,18 @@ mod tests {
let store_source = Arc::new(StoreSource::new("test".to_string(), NeverFindsStore));
let manager = ConfigManager::new(vec![store_source.clone(), sqlite.clone()]);
let manager = ConfigClient::new(vec![store_source.clone(), sqlite.clone()]);
let config = TestConfig {
name: "from_sqlite".to_string(),
count: 99,
};
sqlite
.set("TestConfig", &serde_json::to_value(&config).unwrap())
.set(
ConfigClass::Standard,
"TestConfig",
&serde_json::to_value(&config).unwrap(),
)
.await
.unwrap();

24
harmony_config/src/source/env.rs
View File

@@ -1,4 +1,4 @@
use crate::{ConfigError, ConfigSource};
use crate::{ConfigClass, ConfigError, ConfigSource};
use async_trait::async_trait;
pub struct EnvSource;
@@ -9,7 +9,11 @@ fn env_key_for(config_key: &str) -> String {
#[async_trait]
impl ConfigSource for EnvSource {
async fn get(&self, key: &str) -> Result<Option<serde_json::Value>, ConfigError> {
async fn get(
&self,
_class: ConfigClass,
key: &str,
) -> Result<Option<serde_json::Value>, ConfigError> {
let env_key = env_key_for(key);
match std::env::var(&env_key) {
@@ -27,23 +31,23 @@ impl ConfigSource for EnvSource {
}
}
async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> {
async fn set(
&self,
_class: ConfigClass,
key: &str,
value: &serde_json::Value,
) -> Result<(), ConfigError> {
let env_key = env_key_for(key);
let json_string = serde_json::to_string(value).map_err(|e| ConfigError::Serialization {
key: key.to_string(),
source: e,
})?;
// SAFETY: Setting environment variables is generally safe in single-threaded contexts.
// In multi-threaded contexts, this could cause races, but is acceptable for this use case
// as config is typically set once at startup.
// Rust 2024 makes `set_var` unsafe (data race if another thread reads
// env concurrently); config is set once at startup, so this is safe.
unsafe {
std::env::set_var(&env_key, &json_string);
}
Ok(())
}
fn should_persist(&self) -> bool {
false
}
}

20
harmony_config/src/source/local_file.rs
View File

@@ -2,8 +2,13 @@ use async_trait::async_trait;
use std::path::PathBuf;
use tokio::fs;
use crate::{ConfigError, ConfigSource};
use crate::{ConfigClass, ConfigError, ConfigSource};
/// Local file-backed config source (`<base_path>/<key>.json`).
///
/// ⚠️ Cleartext on disk and ignores `ConfigClass`, like
/// [`SqliteSource`](crate::SqliteSource) — non-secret config only, via an
/// explicit [`ConfigClient::new`](crate::ConfigClient::new).
pub struct LocalFileSource {
base_path: PathBuf,
}
@@ -24,7 +29,11 @@ impl LocalFileSource {
#[async_trait]
impl ConfigSource for LocalFileSource {
async fn get(&self, key: &str) -> Result<Option<serde_json::Value>, ConfigError> {
async fn get(
&self,
_class: ConfigClass,
key: &str,
) -> Result<Option<serde_json::Value>, ConfigError> {
let path = self.file_path_for(key);
match fs::read(&path).await {
@@ -45,7 +54,12 @@ impl ConfigSource for LocalFileSource {
}
}
async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> {
async fn set(
&self,
_class: ConfigClass,
key: &str,
value: &serde_json::Value,
) -> Result<(), ConfigError> {
fs::create_dir_all(&self.base_path).await?;
let path = self.file_path_for(key);

187
harmony_config/src/source/prompt.rs
View File

@@ -1,22 +1,41 @@
use async_trait::async_trait;
use std::sync::Arc;
use inquire::{CustomType, Password, PasswordDisplayMode, Text};
use schemars::schema::{InstanceType, RootSchema, Schema, SchemaObject, SingleOrVec};
use tokio::sync::Mutex;
use crate::{ConfigError, ConfigSource};
use crate::{Config, ConfigClass, ConfigError, ConfigSource};
pub struct PromptSource {
#[allow(dead_code)]
writer: Option<Arc<dyn std::io::Write + Send + Sync>>,
}
// Serialises interactive prompts process-wide: `inquire` assumes exclusive
// terminal ownership, so concurrent prompts would corrupt the terminal.
static PROMPT_MUTEX: Mutex<()> = Mutex::const_new(());
pub struct PromptSource;
impl PromptSource {
pub fn new() -> Self {
Self { writer: None }
Self
}
#[allow(dead_code)]
pub fn with_writer(writer: Arc<dyn std::io::Write + Send + Sync>) -> Self {
Self {
writer: Some(writer),
/// Prompt for a `T`. `Standard` structs go through `interactive_parse`
/// (full nested support); `Secret` structs go through a flat-field
/// walker that reads `#[config(secret)]` fields via `inquire::Password`.
pub async fn prompt_for<T: Config>(&self) -> Result<T, ConfigError> {
let _guard = PROMPT_MUTEX.lock().await;
let banner = format!(
"── Configuring `{}` ({:?}) — please fill the fields below ──",
T::KEY,
T::CLASS,
);
// inquire renders on stderr; match that channel and pad with blank
// lines so the banner stays separate from preceding log output.
eprintln!();
eprintln!("{banner}");
eprintln!();
match T::CLASS {
ConfigClass::Standard => {
T::parse_to_obj().map_err(|e| ConfigError::PromptError(e.to_string()))
}
ConfigClass::Secret => prompt_secret_struct::<T>(),
}
}
}
@@ -29,15 +48,153 @@ impl Default for PromptSource {
#[async_trait]
impl ConfigSource for PromptSource {
async fn get(&self, _key: &str) -> Result<Option<serde_json::Value>, ConfigError> {
async fn get(
&self,
_class: ConfigClass,
_key: &str,
) -> Result<Option<serde_json::Value>, ConfigError> {
Ok(None)
}
async fn set(&self, _key: &str, _value: &serde_json::Value) -> Result<(), ConfigError> {
async fn set(
&self,
_class: ConfigClass,
_key: &str,
_value: &serde_json::Value,
) -> Result<(), ConfigError> {
Ok(())
}
}
fn should_persist(&self) -> bool {
false
// Walks a Secret struct's schema, reading `T::SECRET_FIELDS` via Password
// and other fields via type-appropriate prompts. Only flat primitive
// fields are supported; anything else returns a clear PromptError.
fn prompt_secret_struct<T: Config>() -> Result<T, ConfigError> {
let root: RootSchema = schemars::schema_for!(T);
let object = root.schema.object.as_deref().ok_or_else(|| {
ConfigError::PromptError(format!(
"Secret struct `{}` has no JSON-object schema; the walker only \
supports plain structs",
T::KEY
))
})?;
let mut json = serde_json::Map::with_capacity(object.properties.len());
for (field_name, field_schema) in &object.properties {
let field_object = schema_object(field_schema, T::KEY, field_name)?;
let is_secret = T::SECRET_FIELDS.contains(&field_name.as_str());
let value = prompt_field::<T>(field_name, field_object, is_secret)?;
json.insert(field_name.clone(), value);
}
serde_json::from_value(serde_json::Value::Object(json)).map_err(|e| {
ConfigError::PromptError(format!(
"Failed to deserialize prompted values into `{}`: {e}",
T::KEY
))
})
}
fn schema_object<'a>(
schema: &'a Schema,
struct_name: &str,
field_name: &str,
) -> Result<&'a SchemaObject, ConfigError> {
match schema {
Schema::Object(obj) => Ok(obj),
Schema::Bool(_) => Err(ConfigError::PromptError(format!(
"Secret struct `{struct_name}` field `{field_name}` resolved to a \
boolean JSON-schema; use a concrete type."
))),
}
}
// Picks the inquire widget for one field from its JsonSchema. Secret
// strings use Password; other primitives use type-appropriate input.
// Returns Err for unsupported shapes — never a silent unmasked fallback.
fn prompt_field<T: Config>(
field_name: &str,
schema: &SchemaObject,
is_secret: bool,
) -> Result<serde_json::Value, ConfigError> {
let instance_type = schema.instance_type.as_ref().ok_or_else(|| {
ConfigError::PromptError(format!(
"Secret struct `{}` field `{field_name}` is not a flat primitive \
(string/integer/number/bool); flatten it or extend the walker.",
T::KEY
))
})?;
let single = match instance_type {
SingleOrVec::Single(boxed) => **boxed,
SingleOrVec::Vec(_) => {
return Err(ConfigError::PromptError(format!(
"Secret struct `{}` field `{field_name}` has a multi-type schema \
(e.g. nullable); the walker only handles single primitives.",
T::KEY
)));
}
};
let label = format!("{field_name}:");
match single {
InstanceType::String => {
if is_secret {
// Masked mode echoes `*` per keystroke (inquire's default
// Hidden mode shows nothing until enter).
let raw = Password::new(&label)
.with_display_mode(PasswordDisplayMode::Masked)
.without_confirmation()
.prompt()
.map_err(|e| {
ConfigError::PromptError(format!(
"Password prompt for `{}::{field_name}` failed: {e}",
T::KEY
))
})?;
Ok(serde_json::Value::String(raw))
} else {
let raw = Text::new(&label).prompt().map_err(|e| {
ConfigError::PromptError(format!(
"Text prompt for `{}::{field_name}` failed: {e}",
T::KEY
))
})?;
Ok(serde_json::Value::String(raw))
}
}
InstanceType::Integer => {
// i64 covers Config integer fields; serde narrows on deserialize.
let n = CustomType::<i64>::new(&label).prompt().map_err(|e| {
ConfigError::PromptError(format!(
"Integer prompt for `{}::{field_name}` failed: {e}",
T::KEY
))
})?;
Ok(serde_json::json!(n))
}
InstanceType::Number => {
let n = CustomType::<f64>::new(&label).prompt().map_err(|e| {
ConfigError::PromptError(format!(
"Number prompt for `{}::{field_name}` failed: {e}",
T::KEY
))
})?;
Ok(serde_json::json!(n))
}
InstanceType::Boolean => {
let b = CustomType::<bool>::new(&label).prompt().map_err(|e| {
ConfigError::PromptError(format!(
"Boolean prompt for `{}::{field_name}` failed: {e}",
T::KEY
))
})?;
Ok(serde_json::Value::Bool(b))
}
other => Err(ConfigError::PromptError(format!(
"Secret struct `{}` field `{field_name}` has unsupported type \
`{other:?}`; the walker only handles flat primitives.",
T::KEY
))),
}
}

27
harmony_config/src/source/sqlite.rs
View File

@@ -3,8 +3,15 @@ use sqlx::{SqlitePool, sqlite::SqlitePoolOptions};
use std::path::PathBuf;
use tokio::fs;
use crate::{ConfigError, ConfigSource};
use crate::{ConfigClass, ConfigError, ConfigSource};
/// Local SQLite-backed config cache at `<data_dir>/<namespace>/config.db`.
///
/// ⚠️ Stores values as cleartext JSON and ignores `ConfigClass`, so it must
/// never hold `Secret`-class config — it is excluded from the canonical
/// chain ([`ConfigClient::for_namespace`](crate::ConfigClient::for_namespace))
/// and only usable via an explicit [`ConfigClient::new`](crate::ConfigClient::new)
/// for non-secret offline caching.
pub struct SqliteSource {
pool: SqlitePool,
}
@@ -37,11 +44,14 @@ impl SqliteSource {
Ok(Self { pool })
}
pub async fn default() -> Result<Self, ConfigError> {
/// Scope the database to `namespace` so harmony binaries on one machine
/// don't share a `config.db`. Use the same namespace as `StoreSource`.
pub async fn for_namespace(namespace: &str) -> Result<Self, ConfigError> {
let path = crate::default_config_dir()
.ok_or_else(|| {
ConfigError::SqliteError("Could not determine default config directory".into())
})?
.join(namespace)
.join("config.db");
Self::open(path).await
}
@@ -49,7 +59,11 @@ impl SqliteSource {
#[async_trait]
impl ConfigSource for SqliteSource {
async fn get(&self, key: &str) -> Result<Option<serde_json::Value>, ConfigError> {
async fn get(
&self,
_class: ConfigClass,
key: &str,
) -> Result<Option<serde_json::Value>, ConfigError> {
let row: Option<(String,)> = sqlx::query_as("SELECT value FROM config WHERE key = ?")
.bind(key)
.fetch_optional(&self.pool)
@@ -69,7 +83,12 @@ impl ConfigSource for SqliteSource {
}
}
async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> {
async fn set(
&self,
_class: ConfigClass,
key: &str,
value: &serde_json::Value,
) -> Result<(), ConfigError> {
let json_string = serde_json::to_string(value).map_err(|e| ConfigError::Serialization {
key: key.to_string(),
source: e,

27
harmony_config/src/source/store.rs
View File

@@ -1,7 +1,8 @@
use async_trait::async_trait;
use harmony_secret::SecretStore;
use log::warn;
use crate::{ConfigError, ConfigSource};
use crate::{ConfigClass, ConfigError, ConfigSource};
pub struct StoreSource<S> {
namespace: String,
@@ -15,8 +16,14 @@ impl<S> StoreSource<S> {
}
#[async_trait]
// TODO(ADR 020-1): forward `_class` to `SecretStore` once that trait accepts
// a class hint; kept here so the ConfigSource boundary already matches ADR 020.
impl<S: SecretStore + 'static> ConfigSource for StoreSource<S> {
async fn get(&self, key: &str) -> Result<Option<serde_json::Value>, ConfigError> {
async fn get(
&self,
_class: ConfigClass,
key: &str,
) -> Result<Option<serde_json::Value>, ConfigError> {
match self.store.get_raw(&self.namespace, key).await {
Ok(bytes) => {
let value: serde_json::Value =
@@ -27,11 +34,23 @@ impl<S: SecretStore + 'static> ConfigSource for StoreSource<S> {
Ok(Some(value))
}
Err(harmony_secret::SecretStoreError::NotFound { .. }) => Ok(None),
Err(_) => Ok(None),
// Log before swallowing: a down/misconfigured OpenBao must not look identical to "key absent".
Err(e) => {
warn!(
"StoreSource: get for key '{key}' failed ({e}); treating as \
absent and falling through to the next source"
);
Ok(None)
}
}
}
async fn set(&self, key: &str, value: &serde_json::Value) -> Result<(), ConfigError> {
async fn set(
&self,
_class: ConfigClass,
key: &str,
value: &serde_json::Value,
) -> Result<(), ConfigError> {
let bytes = serde_json::to_vec(value).map_err(|e| ConfigError::Serialization {
key: key.to_string(),
source: e,

64
harmony_config_derive/src/lib.rs
View File

@@ -1,17 +1,23 @@
use proc_macro::TokenStream;
use proc_macro_crate::{FoundCrate, crate_name};
use quote::quote;
use syn::{DeriveInput, Ident, parse_macro_input};
use syn::{Attribute, Data, DeriveInput, Fields, Ident, parse_macro_input};
#[proc_macro_derive(Config)]
/// Derives `Config`, emitting `KEY`, `CLASS`, and `SECRET_FIELDS`. Mark a
/// field — or the whole struct — `#[config(secret)]` to elevate it to
/// `ConfigClass::Secret`. See `Config::SECRET_FIELDS` for the no-rename rule.
#[proc_macro_derive(Config, attributes(config))]
pub fn derive_config(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let struct_ident = &input.ident;
let key = struct_ident.to_string();
// Always emit a fully-qualified path: examples are separate binary
// crates whose `crate::` root lacks harmony_config's items, so the
// `extern crate self` alias in the lib makes `::harmony_config` resolve.
let config_crate_path = match crate_name("harmony_config") {
Ok(FoundCrate::Itself) => quote!(crate),
Ok(FoundCrate::Itself) => quote!(::harmony_config),
Ok(FoundCrate::Name(name)) => {
let ident = Ident::new(&name, proc_macro2::Span::call_site());
quote!(::#ident)
@@ -23,11 +29,63 @@ pub fn derive_config(input: TokenStream) -> TokenStream {
}
};
let struct_level_secret = has_secret_attr(&input.attrs);
// A struct-level `#[config(secret)]` makes every named field secret.
let mut secret_field_names: Vec<String> = Vec::new();
if let Data::Struct(ref data) = input.data
&& let Fields::Named(named) = &data.fields
{
for field in &named.named {
let Some(ident) = field.ident.as_ref() else {
continue;
};
if struct_level_secret || has_secret_attr(&field.attrs) {
secret_field_names.push(ident.to_string());
}
}
}
let is_secret = struct_level_secret || !secret_field_names.is_empty();
let class = if is_secret {
quote!(#config_crate_path::ConfigClass::Secret)
} else {
quote!(#config_crate_path::ConfigClass::Standard)
};
let secret_fields_const = if secret_field_names.is_empty() {
quote!() // fall back to the trait default `&[]`
} else {
let names = secret_field_names.iter().map(|n| quote!(#n));
quote! {
const SECRET_FIELDS: &'static [&'static str] = &[ #( #names ),* ];
}
};
let expanded = quote! {
impl #config_crate_path::Config for #struct_ident {
const KEY: &'static str = #key;
const CLASS: #config_crate_path::ConfigClass = #class;
#secret_fields_const
}
};
TokenStream::from(expanded)
}
fn has_secret_attr(attrs: &[Attribute]) -> bool {
attrs.iter().any(|attr| {
if !attr.path().is_ident("config") {
return false;
}
let mut found = false;
let _ = attr.parse_nested_meta(|meta| {
if meta.path.is_ident("secret") {
found = true;
}
Ok(())
});
found
})
}