harmony/docs/adr/027-multi-tenant-identity.md

# Architecture Decision Record: Multi-Tenant Cloud Identity

Initial Author: Jean-Gabriel Gill-Couture

Initial Date: 2026-06-20

Last Updated Date: 2026-06-20

## Status

Accepted. Phase 0 has been exercised by an initial tenant deployment.

## Context

NationTech runs an open cloud: NationTech staff hold roles in one Zitadel
org (`cloud-admin`, `cloud-billing`, …), and customers create **tenants**.
A tenant owns a slice of every backing service — secrets under
`secret/data/<tenant>/*` in OpenBao, a Harbor project `<tenant>`, Ceph RGW
buckets `<tenant>-*`. The hard constraint:

> A user (or CI identity) created in Zitadel must gain access to its tenant's
> secrets, registry, and storage **without adding a new auth method to any
> service, and without modifying any app, per tenant.**

The open question that triggered this ADR was Zitadel structure: one
**org per tenant**, or one shared **project** with per-tenant roles? Phrased
that way it looks like a service-integration choice. It is not.

## Decision

**Every backing service trusts the *one* Zitadel issuer + *one* shared
audience exactly once, then authorizes on claims. Tenant scoping lives
entirely in a frozen claim contract; the org/project structure is an
internal isolation choice that can change behind it.**

### The claim contract (frozen — apps bind to this, never to tenant structure)

| Claim | Meaning |
|---|---|
| issuer | `https://sso.nationtech.io` — the single root of trust. |
| `aud` | Resource Id of the **shared `cloud` project** — the single audience every service pins as `bound_audiences`. Never per-tenant. |
| `groups` | string array of `<tenant>:<capability>` entries, e.g. `acme:deployer`. The tenant slug is the universal resource prefix; the capability is a fixed tier. |

Capability tiers are a fixed vocabulary: `owner` (full tenant control),
`deployer` (CI — push images, read deploy secrets, apply workloads),
`viewer` (read-only). The **tenant slug** is a human handle NationTech
assigns (`acme`), decoupled from any Zitadel UUID, and is the literal prefix
in OpenBao paths, the Harbor project name, and the Ceph bucket prefix.

### Zitadel structure (Phase 0 — single org, shared project)

- **One org**: `nationtech`. NationTech creates all identities for now.
- **One shared project**, `cloud`. Its Resource Id is the audience for
  *everything*. One API application (Private Key JWT) under it makes that id
  a valid `aud`.
- **Per-tenant = roles, not structure.** Each tenant contributes roles
  `<slug>:owner` / `<slug>:deployer` / `<slug>:viewer` to the shared project.
  Granting a user or service user one of these roles is the entire
  "add user to tenant" operation.
- One project-level **Action** flattens granted role keys into the `groups`
  claim (Zitadel's roles claim is a map; a string-array `groups` is what the
  groups model below consumes — same Action as ADR-025).

### Service integration (reuses ADR-025's groups model verbatim)

Each service is configured **once**, never per tenant:

- **OpenBao** — JWT auth configured against the one issuer; **one** shared
  jwt role pinning `bound_audiences = <cloud project id>` and
  `groups_claim = groups`. Per tenant+capability there is one OpenBao
  *external group* `<slug>:<cap>` and one policy granting its subtree
  (`secret/data/<slug>/*`), attached to that group. A token carrying
  `acme:deployer` resolves the policy at request time — no role edit, no
  re-auth config. (Identical mechanism and write-cost properties to
  ADR-025's fleet device access.)
- **Harbor / Ceph RGW** — same shape: bind the one OIDC issuer + audience
  once; map `groups` membership to the tenant's project / bucket-prefix
  policy.

Onboarding tenant N is therefore pure **provisioning** — add roles + Action
entry, add one external group + policy per service — with **zero** app code,
auth-method, or audience change. This is the constraint, satisfied.

### Phasing (the structure evolves; the contract does not)

| Phase | Trigger | Change | Claim contract |
|---|---|---|---|
| **0 (now)** | tenant #1 | single org, shared `cloud` project; tenant encoded as `<slug>:<cap>` role keys flattened into `groups`. Zero new code. | unchanged |
| **1** | tenants #2–3 | add a first-class `tenant` claim (Zitadel Action) + OpenBao `claim_mappings` + **one** templated policy `secret/data/{{…metadata.tenant}}/*`, collapsing per-tenant groups into one policy. One-time, not per-tenant. | `tenant` becomes a distinct claim; `groups` keeps capability |
| **2** | self-service onboarding | **org per tenant** + Zitadel project grants (B2B); slug becomes org metadata; audience stays the shared `cloud` project so services still bind one `aud`. | unchanged — apps never notice |

## Rationale

- **Org-vs-project is not a service question.** Because every service
  validates one issuer + one audience and reads claims, the Zitadel
  structure is free to change behind the claim contract. Picking the
  *minimal* structure now (single org) and deferring org-per-tenant to
  when self-service actually demands it is YAGNI applied to identity.
- **One audience is the whole trick.** If audience were per-tenant, every
  service would need a new `bound_audiences` per tenant — exactly the
  per-tenant service change the constraint forbids. A single shared-project
  audience is what makes onboarding pure provisioning.
- **Reuse, don't reinvent.** The group→policy authorization, its O(1)
  per-change write cost, and the roles→`groups` Action already exist and are
  argued in ADR-025. Multi-tenant cloud access and fleet device access are
  the same problem (signed group claim gates a prefixed secret subtree); they
  share one mechanism.
- **Slug as prefix, not UUID.** Binding resources to a human slug keeps
  OpenBao paths, Harbor projects, and buckets legible and stable across the
  Phase 2 org migration, where the UUID *does* change.

## Consequences

**Pros**
- New tenant: provisioning only — no app, auth-config, or audience change.
- One issuer + one audience across all services; one place to reason about
  trust.
- Authorization is human-auditable: role grants in Zitadel, group→resource
  policies on each service.
- The Phase 2 org migration is invisible to apps (they read claims).

**Cons / accepted trade-offs**
- **Phase 0 isolation is logical, not org-hard.** All tenants share one org
  and project; a Zitadel-admin compromise spans tenants. Acceptable while
  NationTech creates every identity; Phase 2's org-per-tenant closes it.
- **Zitadel Action required** until first-class groups/claims GA — small,
  version-controlled server-side JS in the token path (shared with ADR-025).
- **Capability tiers are coarse** (`owner`/`deployer`/`viewer`). Finer grants
  mean more group→policy pairs; revisit only at a real third instance.

## Alternatives considered

- **Org per tenant from day one (Phase 2 now).** Correct end state, but
  Zitadel org creation + project grants are unbuilt Scores and self-service
  isn't needed yet — premature. Deferred, not rejected.
- **Per-tenant audience (a project per tenant).** Forces every service to
  add a `bound_audiences` per tenant — the precise per-service change the
  constraint forbids. Rejected.
- **Tenant in OpenBao only (out-of-band mapping).** Drops Zitadel as the
  source of truth for who-belongs-to-what; reintroduces a second place to
  manage membership. Rejected.

## Additional Notes

Verified Score gaps as of 2026-06-20 (build order tracks the phases): the
Zitadel Score supports projects, roles, single-org user grants, machine
users + keys, API apps, and the `groups` flatten Action — it lacks org
creation, project grants, and arbitrary custom claims. The OpenBao Score
supports JWT config + the external-groups model (ADR-025) — it lacks
`claim_mappings` and templated-policy metadata population. Phase 0 needs none
of the gaps; Phase 1 needs the OpenBao ones; Phase 2 needs the Zitadel ones.

Relationship to other ADRs — **ADR-025**: identical groups-as-boundary
mechanism, reused not forked. **ADR-020/020-1**: `harmony_config` /
OpenBao+Zitadel as the single config+secret entry point. **ADR-023**: all
Zitadel/OpenBao configuration lands via Scores. **ADR-011**: multi-tenant
cluster isolation, the workload-side counterpart of this identity-side split.