The OpenBao chart's StatefulSet is OnDelete, so `rollout restart` silently no-ops on config changes — document deleting the pod + re-run to unseal instead. Replace the audit jq recipe's literal-path filter (can't match HMAC-hashed paths) with identity-field attribution, matching observed OpenBao 2.5.4 output. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
182 lines
6.3 KiB
Markdown
182 lines
6.3 KiB
Markdown
# Harmony SSO Example
|
|
|
|
Deploys Zitadel (identity provider) and OpenBao (secrets management) on a local k3d cluster, then demonstrates using them as `harmony_config` backends for shared config and secret management.
|
|
|
|
## Prerequisites
|
|
|
|
- Docker running
|
|
- Ports 8080 and 8200 free
|
|
- `/etc/hosts` entries (or use a local DNS resolver):
|
|
```
|
|
127.0.0.1 sso.harmony.local
|
|
127.0.0.1 bao.harmony.local
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Full deployment
|
|
|
|
```bash
|
|
# Deploy everything (OpenBao + Zitadel)
|
|
cargo run -p example-harmony-sso
|
|
|
|
# OpenBao only (faster, skip Zitadel)
|
|
cargo run -p example-harmony-sso -- --skip-zitadel
|
|
```
|
|
|
|
No `env.sh` to source: the example sets `HARMONY_SECRET_STORE=file` and
|
|
`HARMONY_SECRET_NAMESPACE=harmony-sso-example` at startup so the global
|
|
`harmony_secret` manager uses the local-file backend (writing to
|
|
`~/.local/share/harmony/secrets/`). Override either variable in your
|
|
shell if you want different behaviour.
|
|
|
|
### Config storage demo (token auth)
|
|
|
|
After deployment, run the config demo to verify `harmony_config` works with OpenBao:
|
|
|
|
```bash
|
|
cargo run -p example-harmony-sso -- --demo
|
|
```
|
|
|
|
This writes and reads a `SsoExampleConfig` through the `ConfigClient` chain (`EnvSource -> StoreSource<OpenbaoSecretStore>`), demonstrating environment variable overrides and persistent storage in OpenBao KV v2.
|
|
|
|
### SSO device flow demo
|
|
|
|
Requires a Zitadel application configured for device code grant:
|
|
|
|
```bash
|
|
HARMONY_SSO_CLIENT_ID=<zitadel-app-client-id> \
|
|
cargo run -p example-harmony-sso -- --sso-demo
|
|
```
|
|
|
|
### Cleanup
|
|
|
|
```bash
|
|
cargo run -p example-harmony-sso -- --cleanup
|
|
```
|
|
|
|
## What gets deployed
|
|
|
|
| Component | Namespace | Access |
|
|
|---|---|---|
|
|
| OpenBao (standalone, file storage) | `openbao` | `http://bao.harmony.local:8200` |
|
|
| Zitadel (with CNPG PostgreSQL) | `zitadel` | `http://sso.harmony.local:8080` |
|
|
|
|
### OpenBao configuration
|
|
|
|
- **Auth methods:** userpass, JWT
|
|
- **Secrets engine:** KV v2 at `secret/`
|
|
- **Policy:** `harmony-dev` grants CRUD on `secret/data/harmony/*`
|
|
- **Userpass credentials:** `harmony` / `harmony-dev-password`
|
|
- **JWT auth:** configured with Zitadel as OIDC provider, role `harmony-developer`
|
|
- **Unseal keys:** saved to `~/.local/share/harmony/openbao/unseal-keys.json`
|
|
- **Audit device:** file audit enabled at `/openbao/audit/audit.log` (see below)
|
|
|
|
## Audit log: who changed what
|
|
|
|
OpenBao's KV v2 version history shows *when* a value changed and *which*
|
|
version, but never *who* — the caller identity is not stored in KV
|
|
metadata, so the UI version list can't show the modifier. The modifier
|
|
is recorded only in the **audit log**.
|
|
|
|
A file audit device is declared at `/openbao/audit/audit.log` (on the
|
|
`auditStorage` PVC). Recent OpenBao refuses runtime audit enablement
|
|
via the API (`cannot enable audit device via API`), so `OpenbaoScore`
|
|
configures it **declaratively** in the OpenBao server config (an
|
|
`audit "file" "file"` stanza in the helm values). The device logs
|
|
every API request/response with the authenticated identity. Because
|
|
the SSO flow mints the OpenBao token from the Zitadel id_token with
|
|
`user_claim=email`, each write is attributed to the user's email.
|
|
|
|
Because the device is loaded from the server config at startup, it
|
|
applies only after the OpenBao pod (re)starts with the new config.
|
|
The OpenBao helm chart sets the server StatefulSet's `updateStrategy`
|
|
to `OnDelete` (by design — restarting OpenBao requires a manual
|
|
unseal), so a `helm upgrade` that changes the server config does
|
|
**not** recreate the pod. `kubectl rollout restart statefulset/openbao`
|
|
reports success but does nothing here — `OnDelete` only annotates the
|
|
template. You must delete the pod yourself, then re-run the example to
|
|
unseal the fresh (sealed) process:
|
|
|
|
```bash
|
|
kubectl delete pod openbao-0 -n openbao
|
|
cargo run -p example-harmony-sso # unseals the freshly-restarted pod
|
|
```
|
|
|
|
Confirm the device loaded with `bao audit list` below.
|
|
|
|
The `sys/audit` endpoint and reading the audit log both require the
|
|
**root token** — the `harmony-dev` policy on day-to-day tokens can't
|
|
touch them. Grab the root token from the unseal-keys cache:
|
|
|
|
```bash
|
|
ROOT=$(jq -r .root_token ~/.local/share/harmony/openbao/unseal-keys.json)
|
|
```
|
|
|
|
Confirm the device is on:
|
|
|
|
```bash
|
|
kubectl exec -n openbao openbao-0 -- \
|
|
sh -c "export VAULT_TOKEN=$ROOT && bao audit list"
|
|
# Path Type Description
|
|
# file/ file n/a
|
|
```
|
|
|
|
(`bao audit list` with no token returns `403 permission denied` — that
|
|
means "you didn't authenticate", not "audit is off".)
|
|
|
|
See who performed each operation (the audit log is JSON-lines on the
|
|
audit PVC; reading the file needs no OpenBao token, just pod access).
|
|
Because paths/values are HMAC-hashed by default you can't filter by a
|
|
literal key path — read the *identity* fields instead:
|
|
|
|
```bash
|
|
kubectl exec -n openbao openbao-0 -- cat /openbao/audit/audit.log | \
|
|
jq -c 'select(.type == "response" and .auth.display_name != null)
|
|
| {time, op: .request.operation,
|
|
who: .auth.display_name, role: .auth.metadata.role,
|
|
entity: .auth.entity_id}' | tail -n 15
|
|
```
|
|
|
|
SSO-authenticated operations are attributed to the user, e.g.:
|
|
|
|
```
|
|
{"time":"...","op":"update","who":"jwt-admin@zitadel.sso.harmony.local","role":"harmony-developer","entity":"81ddf1d7-..."}
|
|
```
|
|
|
|
The `jwt-` prefix is the JWT auth mount name; the rest is the email
|
|
from the Zitadel id_token (`user_claim=email`). Setup/unseal steps run
|
|
with the root token and show up as `"who":"root"`.
|
|
|
|
Notes:
|
|
- Audit entries HMAC-hash paths/values by default, so the *identity*
|
|
fields (`auth.display_name`, `auth.metadata`, `auth.entity_id`) are
|
|
what you rely on for attribution. If you need raw paths/values,
|
|
tune the audit device (`log_raw`, `hmac_accessor`) — not done here.
|
|
- This is the OpenBao-native answer; per-version "who" is not surfaced
|
|
in the UI.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Developer CLI
|
|
|
|
|
|-- harmony_config::ConfigClient
|
|
| |-- EnvSource (HARMONY_CONFIG_* env vars)
|
|
| |-- StoreSource<OpenbaoSecretStore>
|
|
| |-- Token auth (OPENBAO_TOKEN)
|
|
| |-- Cached token validation
|
|
| |-- Zitadel OIDC device flow (RFC 8628)
|
|
| |-- Userpass fallback
|
|
|
|
|
v
|
|
k3d cluster (harmony-example)
|
|
|-- OpenBao (KV v2 secrets engine)
|
|
| |-- JWT auth -> validates Zitadel id_tokens
|
|
| |-- userpass auth -> dev credentials
|
|
|
|
|
|-- Zitadel (OpenID Connect IdP)
|
|
|-- Device authorization grant
|
|
|-- Federated login (Google, GitHub, Entra ID)
|
|
```
|