Jean-Gabriel Gill-Couture
5e3b0522e0
Some checks failed
Run Check Script / check (pull_request) Failing after 13s
102 lines
4.6 KiB
Markdown
102 lines
4.6 KiB
Markdown
# Fleet and Zitadel FAQ
|
|
|
|
This page describes the authentication path used by the public Fleet production
|
|
composition.
|
|
|
|
## What does Zitadel provision for Fleet?
|
|
|
|
`ZitadelSetupScore` creates a Fleet project, an API application for NATS, a web
|
|
PKCE application for the production dashboard, the `fleet-admin` and `device`
|
|
roles, and the operator machine identity. Device enrollment creates a separate
|
|
machine user and JSON key for each device.
|
|
|
|
An application is an OIDC client configuration. Users belong to a Zitadel
|
|
organization; project role grants associate a user with roles in the Fleet
|
|
project. A project role is not a Zitadel administrator role.
|
|
|
|
## How does a device authenticate?
|
|
|
|
The agent keeps the JSON machine key at
|
|
`/etc/fleet-agent/zitadel-key.json`. On each NATS connection it:
|
|
|
|
1. signs a JWT assertion whose `iss` and `sub` are the machine user ID;
|
|
2. sends the assertion to `<issuer>/oauth/v2/token` with the RFC 7523
|
|
JWT-bearer grant;
|
|
3. requests `openid`, project roles, and the Fleet project audience;
|
|
4. takes `id_token` from the response and presents it as the NATS bearer token.
|
|
|
|
Production requires `id_token`, not `access_token`. Zitadel access tokens are
|
|
opaque by default. The ID token is a JWT that the callout can verify through
|
|
Zitadel's JWKS and that contains the audience, `client_id`, and role claims used
|
|
by Fleet.
|
|
|
|
The assertion lasts 60 seconds. The ID token is cached in memory and replaced
|
|
before its JWT `exp`; no refresh token is stored. The machine key remains valid
|
|
until it expires or is removed in Zitadel.
|
|
|
|
## How is a device ID derived?
|
|
|
|
Enrollment conventionally creates the username `device-<device-id>`. The
|
|
callout reads the ID token's `client_id` claim and strips `device-`. It validates
|
|
the result before inserting it into device-scoped NATS subjects.
|
|
|
|
## What happens while a device is offline?
|
|
|
|
The machine key remains on disk. Once the network returns, a NATS connection
|
|
attempt obtains a fresh ID token if the cached one is near expiry. The agent
|
|
cannot mint while Zitadel is unreachable, so reconnect waits until both Zitadel
|
|
and NATS are available.
|
|
|
|
## Are machine keys equivalent to PATs?
|
|
|
|
Both permit impersonation if copied. A machine key signs short-lived assertions
|
|
sent to Zitadel, while a PAT is itself a bearer credential. Zitadel allows more
|
|
than one key per machine user, which permits installing a replacement before
|
|
removing the old key. Fleet does not currently use TPM-backed key storage.
|
|
|
|
## Does enrollment make OpenBao secrets available?
|
|
|
|
No. The same ID token can authenticate to an OpenBao JWT role, but enrollment
|
|
must also configure the agent's OpenBao endpoint. Group membership, OpenBao
|
|
policies, policy attachments, and secret values must exist separately.
|
|
|
|
The operator has optional group-source and OpenBao grant-sync support. The
|
|
public production composition does not wire the required group source or
|
|
OpenBao administrator token, so those behaviors are disabled there.
|
|
|
|
## How does dashboard login differ?
|
|
|
|
The dashboard uses an OIDC authorization-code flow with PKCE. It validates the
|
|
returned ID token, stores it in an encrypted private cookie, and requires the
|
|
`fleet-admin` role. Its Zitadel application is configured to include roles in
|
|
the ID token. See [Operator dashboard SSO](./operator-dashboard-sso.md).
|
|
|
|
## Why is the IAM administrator PAT in a Kubernetes Secret?
|
|
|
|
`ZitadelSetupScore` uses the PAT for management API operations. Kubernetes
|
|
Secrets are not encrypted at rest unless the cluster enables encryption at
|
|
rest. Anyone who can read this Secret can act with its privileges. Moving this
|
|
bootstrap credential to a stronger storage path remains production-hardening
|
|
work.
|
|
|
|
## Token reference
|
|
|
|
| Material | Storage | Purpose |
|
|
|---|---|---|
|
|
| Machine JSON key | Device filesystem | Signs JWT-bearer assertions |
|
|
| JWT assertion | Memory and token request | Proves possession of the machine key; valid for 60 seconds |
|
|
| OIDC ID token | Agent memory and NATS connection | Verifiable bearer used by the NATS callout and OpenBao JWT login |
|
|
| Zitadel access token | Token response only | Opaque by default; not used for production Fleet bearer authentication |
|
|
| NATS user JWT | NATS authorization exchange | Carries the permissions issued by the auth callout |
|
|
|
|
## Code map
|
|
|
|
| Topic | File |
|
|
|---|---|
|
|
| Fleet identity composition | `fleet/harmony-fleet-deploy/src/app.rs` |
|
|
| Machine-key token exchange | `harmony_zitadel_jwt/src/lib.rs` |
|
|
| Shared agent/operator credentials | `fleet/harmony-fleet-auth/src/credentials.rs` |
|
|
| Callout validation | `nats/callout/src/zitadel.rs` |
|
|
| Device enrollment | `fleet/harmony-fleet-deploy/src/device_setup.rs` |
|
|
| Manual diagnosis | [Manual token mint](./fleet-manual-token-mint.md) |
|