harmony/docs/adr/027-multi-tenant-identity.md

Architecture Decision Record: Multi-Tenant Cloud Identity

Initial Author: Jean-Gabriel Gill-Couture

Initial Date: 2026-06-20

Last Updated Date: 2026-06-20

Status

Accepted. Phase 0 has been exercised by an initial tenant deployment.

Context

NationTech runs an open cloud: NationTech staff hold roles in one Zitadel org (cloud-admin, cloud-billing, …), and customers create tenants. A tenant owns a slice of every backing service — secrets under secret/data/<tenant>/* in OpenBao, a Harbor project <tenant>, Ceph RGW buckets <tenant>-*. The hard constraint:

A user (or CI identity) created in Zitadel must gain access to its tenant's secrets, registry, and storage without adding a new auth method to any service, and without modifying any app, per tenant.

The open question that triggered this ADR was Zitadel structure: one org per tenant, or one shared project with per-tenant roles? Phrased that way it looks like a service-integration choice. It is not.

Decision

Every backing service trusts the one Zitadel issuer + one shared audience exactly once, then authorizes on claims. Tenant scoping lives entirely in a frozen claim contract; the org/project structure is an internal isolation choice that can change behind it.

The claim contract (frozen — apps bind to this, never to tenant structure)

Claim	Meaning
issuer	https://sso.nationtech.io — the single root of trust.
aud	Resource Id of the shared cloud project — the single audience every service pins as bound_audiences. Never per-tenant.
groups	string array of <tenant>:<capability> entries, e.g. acme:deployer. The tenant slug is the universal resource prefix; the capability is a fixed tier.

Capability tiers are a fixed vocabulary: owner (full tenant control), deployer (CI — push images, read deploy secrets, apply workloads), viewer (read-only). The tenant slug is a human handle NationTech assigns (acme), decoupled from any Zitadel UUID, and is the literal prefix in OpenBao paths, the Harbor project name, and the Ceph bucket prefix.

Zitadel structure (Phase 0 — single org, shared project)

• One org: nationtech. NationTech creates all identities for now.
• One shared project, cloud. Its Resource Id is the audience for everything. One API application (Private Key JWT) under it makes that id a valid aud.
• Per-tenant = roles, not structure. Each tenant contributes roles <slug>:owner / <slug>:deployer / <slug>:viewer to the shared project. Granting a user or service user one of these roles is the entire "add user to tenant" operation.
• One project-level Action flattens granted role keys into the groups claim (Zitadel's roles claim is a map; a string-array groups is what the groups model below consumes — same Action as ADR-025).

Service integration (reuses ADR-025's groups model verbatim)

Each service is configured once, never per tenant:

• OpenBao — JWT auth configured against the one issuer; one shared jwt role pinning bound_audiences = <cloud project id> and groups_claim = groups. Per tenant+capability there is one OpenBao external group <slug>:<cap> and one policy granting its subtree (secret/data/<slug>/*), attached to that group. A token carrying acme:deployer resolves the policy at request time — no role edit, no re-auth config. (Identical mechanism and write-cost properties to ADR-025's fleet device access.)
• Harbor / Ceph RGW — same shape: bind the one OIDC issuer + audience once; map groups membership to the tenant's project / bucket-prefix policy.

Onboarding tenant N is therefore pure provisioning — add roles + Action entry, add one external group + policy per service — with zero app code, auth-method, or audience change. This is the constraint, satisfied.

Phasing (the structure evolves; the contract does not)

Phase	Trigger	Change	Claim contract
0 (now)	tenant #1	single org, shared cloud project; tenant encoded as <slug>:<cap> role keys flattened into groups. Zero new code.	unchanged
1	tenants #2–3	add a first-class tenant claim (Zitadel Action) + OpenBao claim_mappings + one templated policy secret/data/{{…metadata.tenant}}/*, collapsing per-tenant groups into one policy. One-time, not per-tenant.	tenant becomes a distinct claim; groups keeps capability
2	self-service onboarding	org per tenant + Zitadel project grants (B2B); slug becomes org metadata; audience stays the shared cloud project so services still bind one aud.	unchanged — apps never notice

Rationale

• Org-vs-project is not a service question. Because every service validates one issuer + one audience and reads claims, the Zitadel structure is free to change behind the claim contract. Picking the minimal structure now (single org) and deferring org-per-tenant to when self-service actually demands it is YAGNI applied to identity.
• One audience is the whole trick. If audience were per-tenant, every service would need a new bound_audiences per tenant — exactly the per-tenant service change the constraint forbids. A single shared-project audience is what makes onboarding pure provisioning.
• Reuse, don't reinvent. The group→policy authorization, its O(1) per-change write cost, and the roles→groups Action already exist and are argued in ADR-025. Multi-tenant cloud access and fleet device access are the same problem (signed group claim gates a prefixed secret subtree); they share one mechanism.
• Slug as prefix, not UUID. Binding resources to a human slug keeps OpenBao paths, Harbor projects, and buckets legible and stable across the Phase 2 org migration, where the UUID does change.

Consequences

Pros

• New tenant: provisioning only — no app, auth-config, or audience change.
• One issuer + one audience across all services; one place to reason about trust.
• Authorization is human-auditable: role grants in Zitadel, group→resource policies on each service.
• The Phase 2 org migration is invisible to apps (they read claims).

Cons / accepted trade-offs

• Phase 0 isolation is logical, not org-hard. All tenants share one org and project; a Zitadel-admin compromise spans tenants. Acceptable while NationTech creates every identity; Phase 2's org-per-tenant closes it.
• Zitadel Action required until first-class groups/claims GA — small, version-controlled server-side JS in the token path (shared with ADR-025).
• Capability tiers are coarse (owner/deployer/viewer). Finer grants mean more group→policy pairs; revisit only at a real third instance.

Alternatives considered

• Org per tenant from day one (Phase 2 now). Correct end state, but Zitadel org creation + project grants are unbuilt Scores and self-service isn't needed yet — premature. Deferred, not rejected.
• Per-tenant audience (a project per tenant). Forces every service to add a bound_audiences per tenant — the precise per-service change the constraint forbids. Rejected.
• Tenant in OpenBao only (out-of-band mapping). Drops Zitadel as the source of truth for who-belongs-to-what; reintroduces a second place to manage membership. Rejected.

Additional Notes

Verified Score gaps as of 2026-06-20 (build order tracks the phases): the Zitadel Score supports projects, roles, single-org user grants, machine users + keys, API apps, and the groups flatten Action — it lacks org creation, project grants, and arbitrary custom claims. The OpenBao Score supports JWT config + the external-groups model (ADR-025) — it lacks claim_mappings and templated-policy metadata population. Phase 0 needs none of the gaps; Phase 1 needs the OpenBao ones; Phase 2 needs the Zitadel ones.

Relationship to other ADRs — ADR-025: identical groups-as-boundary mechanism, reused not forked. ADR-020/020-1: harmony_config / OpenBao+Zitadel as the single config+secret entry point. ADR-023: all Zitadel/OpenBao configuration lands via Scores. ADR-011: multi-tenant cluster isolation, the workload-side counterpart of this identity-side split.