Jean-Gabriel Gill-Couture
c20db5b361
Some checks failed
Run Check Script / check (pull_request) Failing after 19s
142 lines
8.1 KiB
Markdown
142 lines
8.1 KiB
Markdown
# Architecture Decision Record: Template Hydration for Kubernetes Manifest Generation
|
|
|
|
Initial Author: Jean-Gabriel Gill-Couture & Sylvain Tremblay
|
|
|
|
Initial Date: 2025-01-23
|
|
|
|
Last Updated Date: 2025-01-23
|
|
|
|
## Status
|
|
|
|
Implemented
|
|
|
|
## Context
|
|
|
|
Harmony's philosophy is built on three guiding principles: Infrastructure as Resilient Code, Prove It Works — Before You Deploy, and One Unified Model. Our goal is to shift validation and verification as left as possible—ideally to compile time—rather than discovering errors at deploy time.
|
|
|
|
After investigating a few approaches such as compile-checked Askama templates to generate Kubernetes manifests for Helm charts, we found again that this approach suffered from several fundamental limitations:
|
|
|
|
* **Late Validation:** Typos in template syntax or field names are only discovered at deployment time, not during compilation. A mistyped `metadata.name` won't surface until Helm attempts to render the template.
|
|
* **Brittle Maintenance:** Templates are string-based with limited IDE support. Refactoring requires grep-and-replace across YAML-like template files, risking subtle breakage.
|
|
* **Hard-to-Test Logic:** Testing template output requires mocking the template engine and comparing serialized strings rather than asserting against typed data structures.
|
|
* **No Type Safety:** There is no guarantee that the generated YAML will be valid Kubernetes resources without runtime validation.
|
|
|
|
We also faced a strategic choice around Helm: use it as both *templating engine* and *packaging mechanism*, or decouple these concerns. While Helm's ecosystem integration (Harbor, ArgoCD, OCI registry support) is valuable, the Jinja-like templating is at odds with Harmony's "code-first" ethos.
|
|
|
|
## Decision
|
|
|
|
We will adopt the **Template Hydration Pattern**—constructing Kubernetes manifests programmatically using strongly-typed `kube-rs` objects, then serializing them to YAML files for packaging into Helm charts.
|
|
|
|
Specifically:
|
|
|
|
* **Write strongly typed `k8s_openapi` Structs:** All Kubernetes resources (Deployment, Service, ConfigMap, etc.) will be constructed using the typed structs generated by `k8s_openapi`.
|
|
* **Direct Serialization to YAML:** Rather than rendering templates, we use `serde_yaml::to_string()` to serialize typed objects directly into YAML manifests. This way, YAML is only used as a data-transfer format and not a templating/programming language - which it is not.
|
|
* **Helm as Packaging-Only:** Helm's role is reduced to packaging pre-rendered templates into a tarball and pushing to OCI registries. No template rendering logic resides within Helm.
|
|
* **Ecosystem Preservation:** The generated Helm charts remain fully compatible with Harbor, ArgoCD, and any Helm-compatible tool—the only difference is that the `templates/` directory contains static YAML files.
|
|
|
|
The implementation in `backend_app.rs` demonstrates this pattern:
|
|
|
|
```rust
|
|
let deployment = Deployment {
|
|
metadata: ObjectMeta {
|
|
name: Some(self.name.clone()),
|
|
labels: Some([("app.kubernetes.io/name".to_string(), self.name.clone())].into()),
|
|
..Default::default()
|
|
},
|
|
spec: Some(DeploymentSpec { /* ... */ }),
|
|
..Default::default()
|
|
};
|
|
|
|
let deployment_yaml = serde_yaml::to_string(&deployment)?;
|
|
fs::write(templates_dir.join("deployment.yaml"), deployment_yaml)?;
|
|
```
|
|
|
|
## Rationale
|
|
|
|
**Aligns with "Infrastructure as Resilient Code"**
|
|
|
|
Harmony's first principle states that infrastructure should be treated like application code. By expressing Kubernetes manifests as Rust structs, we gain:
|
|
|
|
* **Refactorability:** Rename a label and the compiler catches all usages.
|
|
* **IDE Support:** Autocomplete for all Kubernetes API fields; documentation inline.
|
|
* **Code Navigation:** Jump to definition shows exactly where a value comes from.
|
|
|
|
**Achieves "Prove It Works — Before You Deploy"**
|
|
|
|
The compiler now validates that:
|
|
|
|
* All required fields are populated (Rust's `Option` type prevents missing fields).
|
|
* Field types match expectations (ports are integers, not strings).
|
|
* Enums contain valid values (e.g., `ServiceType::ClusterIP`).
|
|
|
|
This moves what was runtime validation into compile-time checks, fulfilling the "shift left" promise.
|
|
|
|
**Enables True Unit Testing**
|
|
|
|
Developers can now write unit tests that assert directly against typed objects:
|
|
|
|
```rust
|
|
let deployment = create_deployment(&app);
|
|
assert_eq!(deployment.spec.unwrap().replicas.unwrap(), 3);
|
|
assert_eq!(deployment.metadata.name.unwrap(), "my-app");
|
|
```
|
|
|
|
No string parsing, no YAML serialization, no fragile assertions against rendered output.
|
|
|
|
**Preserves Ecosystem Benefits**
|
|
|
|
By generating standard Helm chart structures, Harmony retains compatibility with:
|
|
|
|
* **OCI Registries (Harbor, GHCR):** `helm push` works exactly as before.
|
|
* **ArgoCD:** Syncs and manages releases using the generated charts.
|
|
* **Existing Workflows:** Teams already consuming Helm charts see no change.
|
|
|
|
The Helm tarball becomes a "dumb pipe" for transport, which is arguably its ideal role.
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
|
|
* **Compile-Time Safety:** A broad class of errors (typos, missing fields, type mismatches) is now caught at build time.
|
|
* **Better Developer Experience:** IDE autocomplete, inline documentation, and refactor support significantly reduce the learning curve for Kubernetes manifests.
|
|
* **Testability:** Unit tests can validate manifest structure without integration or runtime checks.
|
|
* **Auditability:** The source-of-truth for manifests is now pure Rust—easier to review in pull requests than template logic scattered across files.
|
|
* **Future-Extensibility:** CustomResources (CRDs) can be supported via `kopium`-generated Rust types, maintaining the same strong typing.
|
|
|
|
### Negative
|
|
|
|
* **API Schema Drift:** Kubernetes API changes require regenerating `k8s_openapi` types and updating code. A change in a struct field will cause the build to fail—intentionally, but still requiring the pipeline to be updated.
|
|
* **Verbosity:** Typed construction is more verbose than the equivalent template. Builder patterns or helper functions will be needed to keep code readable.
|
|
* **Learning Curve:** Contributors must understand both the Kubernetes resource spec *and* the Rust type system, rather than just YAML.
|
|
* **Debugging Shift:** When debugging generated YAML, you now trace through Rust code rather than template files—more precise but different mental model.
|
|
|
|
## Alternatives Considered
|
|
|
|
### 1. Enhance Askama with Compile-Time Validation
|
|
*Pros:* Stay within familiar templating paradigm; minimal code changes.
|
|
*Cons:* Rust's type system cannot fully express Kubernetes schema validation without significant macro boilerplate. Errors would still surface at template evaluation time, not compilation.
|
|
|
|
### 2. Use Helm SDK Programmatically (Go)
|
|
*Pros:* Direct access to Helm's template engine; no YAML serialization step.
|
|
*Cons:* Would introduce a second language (Go) into a Rust codebase, increasing cognitive load and compilation complexity. No improvement in compile-time safety.
|
|
|
|
### 3. Raw YAML String Templating (Manual)
|
|
*Pros:* Maximum control; no external dependencies.
|
|
*Cons:* Even more error-prone than Askama; no structure validation; string concatenation errors abound.
|
|
|
|
### 4. Use Kustomize for All Manifests
|
|
*Pros:* Declarative overlays; standard tool.
|
|
*Cons:* Kustomize is itself a layer over YAML templates with its own DSL. It does not provide compile-time type safety and would require externalizing manifest management outside Harmony's codebase.
|
|
|
|
__Note that this template hydration architecture still allows to override templates with tools like kustomize when required__
|
|
|
|
## Additional Notes
|
|
|
|
**Scalability to Future Topologies**
|
|
|
|
The Template Hydration pattern enables future Harmony architectures to generate manifests dynamically based on topology context. For example, a `CostTopology` might adjust resource requests based on cluster pricing, manipulating the typed `Deployment::spec` directly before serialization.
|
|
|
|
**Implementation Status**
|
|
|
|
As of this writing, the pattern is implemented for `BackendApp` deployments (`backend_app.rs`). The next phase is to extend this pattern across all application modules (`webapp.rs`, etc.) and to standardize on this approach for any new implementations.
|