adr/017-1-Nats-Clusters-Interconnection-Topology.md

@@ -0,0 +1,189 @@
+### 1. ADR 017-1: NATS Cluster Interconnection & Trust Topology
+
+# Architecture Decision Record: NATS Cluster Interconnection & Trust Topology
+
+**Status:** Proposed
+**Date:** 2026-01-12
+**Precedes:** [017-Staleness-Detection-for-Failover.md]
+
+## Context
+
+In ADR 017, we defined the failover mechanisms for the Harmony mesh. However, for a Primary (Site A) and a Replica (Site B) to communicate securely—or for the Global Mesh to function across disparate locations—we must establish a robust Transport Layer Security (TLS) strategy.
+
+Our primary deployment platform is OKD (Kubernetes). While OKD provides an internal `service-ca`, it is designed primarily for intra-cluster service-to-service communication. It lacks the flexibility required for:
+1.  **Public/External Gateway Identities:** NATS Gateways need to identify themselves via public DNS names or external IPs, not just internal `.svc` cluster domains.
+2.  **Cross-Cluster Trust:** We need a mechanism to allow Cluster A to trust Cluster B without sharing a single private root key.
+
+## Decision
+
+We will implement an **"Islands of Trust"** topology using **cert-manager** on OKD.
+
+### 1. Per-Cluster Certificate Authorities (CA)
+
+* We explicitly **reject** the use of a single "Supercluster CA" shared across all sites.
+    * Instead, every Harmony Cluster (Site A, Site B, etc.) will generate its own unique Self-Signed Root CA managed by `cert-manager` inside that cluster.
+*   **Lifecycle:** Root CAs will have a long duration (e.g., 10 years) to minimize rotation friction, while Leaf Certificates (NATS servers) will remain short-lived (e.g., 90 days) and rotate automatically.
+
+> Note : The decision to have a single CA for various workloads managed by Harmony on each deployment, or to have multiple CA for each service that requires interconnection is not made yet. This ADR leans towards one CA per service. This allows for maximum flexibility. But the direction might change and no clear decision has been made yet. The alternative of establishing that each cluster/harmony deployment has a single identity could make mTLS very simple between tenants.
+
+### 2. Trust Federation via Bundle Exchange
+
+To enable secure communication (mTLS) between clusters (e.g., for NATS Gateways or Leaf Nodes):
+
+*   **No Private Keys are shared.**
+*   We will aggregate the **Public CA Certificates** of all trusted clusters into a shared `ca-bundle.pem`.
+*   This bundle is distributed to the NATS configuration of every node.
+*   **Verification Logic:** When Site A connects to Site B, Site A verifies Site B's certificate against the bundle. Since Site B's CA public key is in the bundle, the connection is accepted.
+
+### 3. Tooling
+
+* We will use **cert-manager** (deployed via Operator on OKD) rather than OKD's built-in `service-ca`. This provides us with standard CRDs (`Issuer`, `Certificate`) to manage the lifecycle, rotation, and complex SANs (Subject Alternative Names) required for external connectivity.
+* Harmony will manage installation, configuration and bundle creation across all sites
+
+## Rationale
+
+**Security Blast Radius (The "Key Leak" Scenario)**
+If we used a single global CA and the private key for Site A was compromised (e.g., physical theft of a server from a basement), the attacker could impersonate *any* site in the global mesh.
+By using Per-Cluster CAs:
+*   If Site A is compromised, only Site A's identity is stolen.
+*   We can "evict" Site A from the mesh simply by removing Site A's Public CA from the `ca-bundle.pem` on the remaining healthy clusters and reloading. The attacker can no longer authenticate.
+
+**Decentralized Autonomy**
+This aligns with the "Humane Computing" vision. A local cluster owns its identity. It does not depend on a central authority to issue its certificates. It can function in isolation (offline) indefinitely without needing to "phone home" to renew credentials.
+
+## Consequences
+
+**Positive**
+*   **High Security:** Compromise of one node does not compromise the global mesh.
+*   **Flexibility:** Easier to integrate with third-party clusters or partners by simply adding their public CA to the bundle.
+*   **Standardization:** `cert-manager` is the industry standard, making the configuration portable to non-OKD K8s clusters if needed.
+
+**Negative**
+*   **Configuration Complexity:** We must manage a mechanism to distribute the `ca-bundle.pem` containing public keys to all sites. This should be automated (e.g., via a Harmony Agent) to ensure timely updates and revocation.
+*   **Revocation Latency:** Revoking a compromised cluster requires updating and reloading the bundle on all other clusters. This is slower than OCSP/CRL but acceptable for infrastructure-level trust if automation is in place.
+
+---
+
+# 2. Concrete overview of the process, how it can be implemented manually across multiple OKD clusters
+
+All of this will be automated via Harmony, but to understand correctly the process it is outlined in details here :
+
+## 1. Deploying and Configuring cert-manager on OKD
+
+While OKD has a built-in `service-ca` controller, it is "opinionated" and primarily signs certs for internal services (like `my-svc.my-namespace.svc`). It is **not suitable** for the Harmony Global Mesh because you cannot easily control the Subject Alternative Names (SANs) for external routes (e.g., `nats.site-a.nationtech.io`), nor can you easily export its CA to other clusters.
+
+**The Solution:** Use the **cert-manager Operator for Red Hat OpenShift**.
+
+### Step 1: Install the Operator
+1.  Log in to the OKD Web Console.
+2.  Navigate to **Operators** -> **OperatorHub**.
+3.  Search for **"cert-manager"**.
+4.  Choose the **"cert-manager Operator for Red Hat OpenShift"** (Red Hat provided) or the community version.
+5.  Click **Install**. Use the default settings (Namespace: `cert-manager-operator`).
+
+### Step 2: Create the "Island" CA (The Issuer)
+Once installed, you define your cluster's unique identity. Apply this YAML to your NATS namespace.
+
+```yaml
+# filepath: k8s/01-issuer.yaml
+apiVersion: cert-manager.io/v1
+kind: Issuer
+metadata:
+  name: harmony-selfsigned-issuer
+  namespace: harmony-nats
+spec:
+  selfSigned: {}
+---
+# This generates the unique Root CA for THIS specific cluster
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: harmony-root-ca
+  namespace: harmony-nats
+spec:
+  isCA: true
+  commonName: "harmony-site-a-ca" # CHANGE THIS per cluster (e.g., site-b-ca)
+  duration: 87600h # 10 years
+  renewBefore: 2160h # 3 months before expiry
+  secretName: harmony-root-ca-secret
+  privateKey:
+    algorithm: ECDSA
+    size: 256
+  issuerRef:
+    name: harmony-selfsigned-issuer
+    kind: Issuer
+    group: cert-manager.io
+---
+# This Issuer uses the Root CA generated above to sign NATS certs
+apiVersion: cert-manager.io/v1
+kind: Issuer
+metadata:
+  name: harmony-ca-issuer
+  namespace: harmony-nats
+spec:
+  ca:
+    secretName: harmony-root-ca-secret
+```
+
+### Step 3: Generate the NATS Server Certificate
+This certificate will be used by the NATS server. It includes both internal DNS names (for local clients) and external DNS names (for the global mesh).
+
+```yaml
+# filepath: k8s/02-nats-cert.yaml
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: nats-server-cert
+  namespace: harmony-nats
+spec:
+  secretName: nats-server-tls
+  duration: 2160h # 90 days
+  renewBefore: 360h # 15 days
+  issuerRef:
+    name: harmony-ca-issuer
+    kind: Issuer
+  # CRITICAL: Define all names this server can be reached by
+  dnsNames:
+  - "nats"
+  - "nats.harmony-nats.svc"
+  - "nats.harmony-nats.svc.cluster.local"
+  - "*.nats.harmony-nats.svc.cluster.local"
+  - "nats-gateway.site-a.nationtech.io" # External Route for Mesh
+```
+
+## 2. Implementing the "Islands of Trust" (Trust Bundle)
+
+To make Site A and Site B talk, you need to exchange **Public Keys**.
+
+1.  **Extract Public CA from Site A:**
+    ```bash
+    oc get secret harmony-root-ca-secret -n harmony-nats -o jsonpath='{.data.ca\.crt}' | base64 -d > site-a.crt
+    ```
+2.  **Extract Public CA from Site B:**
+    ```bash
+    oc get secret harmony-root-ca-secret -n harmony-nats -o jsonpath='{.data.ca\.crt}' | base64 -d > site-b.crt
+    ```
+3.  **Create the Bundle:**
+    Combine them into one file.
+    ```bash
+    cat site-a.crt site-b.crt > ca-bundle.crt
+    ```
+4.  **Upload Bundle to Both Clusters:**
+    Create a ConfigMap or Secret in *both* clusters containing this combined bundle.
+    ```bash
+    oc create configmap nats-trust-bundle --from-file=ca.crt=ca-bundle.crt -n harmony-nats
+    ```
+5.  **Configure NATS:**
+    Mount this ConfigMap and point NATS to it.
+
+    ```conf
+    # nats.conf snippet
+    tls {
+      cert_file: "/etc/nats-certs/tls.crt"
+      key_file:  "/etc/nats-certs/tls.key"
+      # Point to the bundle containing BOTH Site A and Site B public CAs
+      ca_file:   "/etc/nats-trust/ca.crt"
+    }
+    ```
+
+This setup ensures that Site A can verify Site B's certificate (signed by `harmony-site-b-ca`) because Site B's CA is in Site A's trust store, and vice versa, without ever sharing the private keys that generated them.

examples/cert_manager/Cargo.toml

@@ -0,0 +1,19 @@
+[package]
+name = "cert_manager"
+edition = "2024"
+version.workspace = true
+readme.workspace = true
+license.workspace = true
+publish = false
+
+[dependencies]
+harmony = { path = "../../harmony" }
+harmony_cli = { path = "../../harmony_cli" }
+harmony_types = { path = "../../harmony_types" }
+cidr = { workspace = true }
+tokio = { workspace = true }
+harmony_macros = { path = "../../harmony_macros" }
+log = { workspace = true }
+env_logger = { workspace = true }
+url = { workspace = true }
+assert_cmd = "2.0.16"

examples/cert_manager/src/main.rs

@@ -0,0 +1,42 @@
+use harmony::{
+    inventory::Inventory,
+    modules::cert_manager::{
+        capability::CertificateManagementConfig, score_cert_management::CertificateManagementScore,
+        score_certificate::CertificateScore, score_issuer::CertificateIssuerScore,
+    },
+    topology::K8sAnywhereTopology,
+};
+
+#[tokio::main]
+async fn main() {
+    let config = CertificateManagementConfig {
+        namespace: Some("test".to_string()),
+        acme_issuer: None,
+        ca_issuer: None,
+        self_signed: true,
+    };
+
+    let issuer_name = "test-self-signed-issuer".to_string();
+    let issuer = CertificateIssuerScore {
+        issuer_name: issuer_name.clone(),
+        config: config.clone(),
+    };
+
+    let cert = CertificateScore {
+        config: config.clone(),
+        issuer_name,
+        cert_name: "test-self-signed-cert".to_string(),
+        common_name: None,
+        dns_names: Some(vec!["test.dns.name".to_string()]),
+        is_ca: Some(false),
+    };
+
+    harmony_cli::run(
+        Inventory::autoload(),
+        K8sAnywhereTopology::from_env(),
+        vec![Box::new(issuer), Box::new(cert)],
+        None,
+    )
+    .await
+    .unwrap();
+}

examples/nats-supercluster/Cargo.toml

@@ -0,0 +1,19 @@
+[package]
+name = "example-nats-supercluster"
+edition = "2024"
+version.workspace = true
+readme.workspace = true
+license.workspace = true
+publish = false
+
+[dependencies]
+harmony = { path = "../../harmony" }
+harmony_cli = { path = "../../harmony_cli" }
+harmony_types = { path = "../../harmony_types" }
+cidr = { workspace = true }
+tokio = { workspace = true }
+harmony_macros = { path = "../../harmony_macros" }
+log = { workspace = true }
+env_logger = { workspace = true }
+url = { workspace = true }
+k8s-openapi.workspace = true

examples/nats-supercluster/env_example.sh

@@ -0,0 +1,6 @@
+# Cluster 1
+export HARMONY_NATS_SITE_1="kubeconfig=$HOME/.config/nt/kube/config,context=your_cluster_1_kube_context_name"
+export HARMONY_NATS_SITE_1_DOMAIN="your_cluster_1_public_domain"
+# Cluster 2
+export HARMONY_NATS_SITE_2="kubeconfig=$HOME/.config/nt/kube/config,context=your_cluster_2_kube_context_name"
+export HARMONY_NATS_SITE_2_DOMAIN="your_cluster_2_public_domain"

examples/nats-supercluster/src/main.rs

@@ -0,0 +1,481 @@
+use std::{collections::BTreeMap, str::FromStr};
+
+use harmony::{
+    interpret::{InterpretError, Outcome},
+    inventory::Inventory,
+    modules::{
+        cert_manager::{
+            capability::{CertificateManagement, CertificateManagementConfig},
+            crd::CaIssuer,
+        },
+        helm::chart::{HelmChartScore, HelmRepository, NonBlankString},
+        k8s::resource::K8sResourceScore,
+        okd::{
+            crd::route::{RoutePort, RouteSpec, RouteTargetReference, TLSConfig},
+            route::OKDRouteScore,
+        },
+    },
+    score::Score,
+    topology::{
+        HelmCommand, K8sAnywhereConfig, K8sAnywhereTopology, K8sclient, TlsRouter, Topology,
+    },
+};
+use harmony_macros::hurl;
+use k8s_openapi::{
+    ByteString, api::core::v1::Secret, apimachinery::pkg::apis::meta::v1::ObjectMeta,
+};
+use log::{debug, info};
+
+#[tokio::main]
+async fn main() -> Result<(), InterpretError> {
+    let namespace = "nats-supercluster-test";
+    let self_signed_issuer_name = "harmony-self-signed-issuer";
+    let ca_issuer_name = "harmony-ca-issuer";
+    let root_ca_cert_name = "harmony-root-ca";
+
+    log::info!("starting nats supercluster bootstrap");
+
+    // --------------------------------------------------
+    // 1. Build site contexts
+    // --------------------------------------------------
+
+    let site1 = site(
+        "HARMONY_NATS_SITE_1",
+        "HARMONY_NATS_SITE_1_DOMAIN",
+        "nats-sto1-cert-test1",
+    );
+
+    let site2 = site(
+        "HARMONY_NATS_SITE_2",
+        "HARMONY_NATS_SITE_2_DOMAIN",
+        "nats-cb1-cert-test2",
+    );
+
+    // --------------------------------------------------
+    // 2. Ensure clusters are reachable
+    // --------------------------------------------------
+
+    log::info!("ensuring both topologies are ready");
+
+    tokio::try_join!(site1.topology.ensure_ready(), site2.topology.ensure_ready(),)?;
+
+    // --------------------------------------------------
+    // 3. Create certificates
+    // --------------------------------------------------
+
+    log::info!("creating certificates");
+
+    let root_ca_config = CertificateManagementConfig {
+        namespace: Some(namespace.into()),
+        acme_issuer: None,
+        ca_issuer: Some(CaIssuer {
+            secret_name: format!("{}-tls", root_ca_cert_name),
+        }),
+        self_signed: false,
+    };
+
+    let self_signed_config = CertificateManagementConfig {
+        namespace: Some(namespace.to_string().clone()),
+        acme_issuer: None,
+        ca_issuer: None,
+        self_signed: true,
+    };
+
+    tokio::try_join!(
+        create_nats_certs(
+            site1.topology.clone(),
+            &site1.cluster,
+            ca_issuer_name,
+            &root_ca_config,
+            self_signed_issuer_name,
+            &self_signed_config,
+            root_ca_cert_name
+        ),
+        create_nats_certs(
+            site2.topology.clone(),
+            &site2.cluster,
+            ca_issuer_name,
+            &root_ca_config,
+            self_signed_issuer_name,
+            &self_signed_config,
+            root_ca_cert_name
+        ),
+    )?;
+
+    // --------------------------------------------------
+    // 4. Build CA bundle
+    // --------------------------------------------------
+
+    log::info!("building supercluster CA bundle");
+
+    let mut ca_bundle = Vec::new();
+
+    ca_bundle.push(
+        site1
+            .topology
+            .get_ca_certificate(root_ca_cert_name.to_string(), &root_ca_config)
+            .await?,
+    );
+    ca_bundle.push(
+        site2
+            .topology
+            .get_ca_certificate(root_ca_cert_name.to_string(), &root_ca_config)
+            .await?,
+    );
+
+    // --------------------------------------------------
+    // 5. Build Scores
+    // --------------------------------------------------
+
+    log::info!("building scores");
+
+    let site1_scores = vec![
+        build_ca_bundle_secret_score(
+            site1.topology.clone(),
+            &site1.cluster,
+            &ca_bundle,
+            namespace.into(),
+        )
+        .await,
+        build_route_score(site1.topology.clone(), &site1.cluster, namespace.into()).await,
+        build_deploy_nats_score(
+            site1.topology.clone(),
+            &site1.cluster,
+            vec![&site2.cluster],
+            namespace.into(),
+        )
+        .await,
+    ];
+
+    let site2_scores = vec![
+        build_ca_bundle_secret_score(
+            site2.topology.clone(),
+            &site2.cluster,
+            &ca_bundle,
+            namespace.into(),
+        )
+        .await,
+        build_route_score(site2.topology.clone(), &site2.cluster, namespace.into()).await,
+        build_deploy_nats_score(
+            site2.topology.clone(),
+            &site2.cluster,
+            vec![&site1.cluster],
+            namespace.into(),
+        )
+        .await,
+    ];
+
+    // --------------------------------------------------
+    // 6. Apply Scores
+    // --------------------------------------------------
+
+    log::info!("applying scores");
+
+    tokio::try_join!(
+        apply_scores(site1.topology.clone(), site1_scores),
+        apply_scores(site2.topology.clone(), site2_scores),
+    )?;
+
+    log::info!("supercluster bootstrap complete");
+    log::info!(
+        "Enjoy! You can test your nats cluster by running : `kubectl exec -n {namespace} -it deployment/nats-box -- nats pub test hi`"
+    );
+    Ok(())
+}
+
+async fn apply_scores<T: Topology + 'static>(
+    topology: T,
+    scores: Vec<Box<dyn Score<T>>>,
+) -> Result<(), InterpretError> {
+    info!("applying {} scores", scores.len());
+
+    harmony_cli::run(Inventory::autoload(), topology, scores, None)
+        .await
+        .map_err(|e| InterpretError::new(e.to_string()))?;
+
+    Ok(())
+}
+
+fn site(
+    topo_env: &str,
+    domain_env: &str,
+    cluster_name: &'static str,
+) -> SiteContext<K8sAnywhereTopology> {
+    let domain = std::env::var(domain_env).expect("missing domain env");
+
+    let topology =
+        K8sAnywhereTopology::with_config(K8sAnywhereConfig::remote_k8s_from_env_var(topo_env));
+
+    SiteContext {
+        topology,
+        cluster: NatsCluster {
+            replicas: 1,
+            name: cluster_name,
+            gateway_advertise: format!("{cluster_name}-gw.{domain}:443"),
+            dns_name: format!("{cluster_name}-gw.{domain}"),
+            supercluster_ca_secret_name: "nats-supercluster-ca-bundle",
+            tls_cert_name: "nats-gateway",
+            jetstream_enabled: "false",
+        },
+    }
+}
+
+struct SiteContext<T> {
+    topology: T,
+    cluster: NatsCluster,
+}
+
+struct NatsCluster {
+    replicas: usize,
+    name: &'static str,
+    gateway_advertise: String,
+    dns_name: String,
+    supercluster_ca_secret_name: &'static str,
+    tls_cert_name: &'static str,
+    jetstream_enabled: &'static str,
+}
+
+async fn create_nats_certs<T: Topology + CertificateManagement>(
+    topology: T,
+    cluster: &NatsCluster,
+    ca_issuer_name: &str,
+    ca_cert_mgmt_config: &CertificateManagementConfig,
+    self_signed_issuer_name: &str,
+    self_signed_cert_config: &CertificateManagementConfig,
+    root_ca_cert_name: &str,
+) -> Result<Outcome, InterpretError> {
+    //the order is pretty important
+
+    debug!(
+        "Applying certs to ns {:#?}",
+        ca_cert_mgmt_config.namespace.clone()
+    );
+
+    debug!("creating issuer '{}'", self_signed_issuer_name);
+    topology
+        .create_issuer(
+            self_signed_issuer_name.to_string(),
+            &self_signed_cert_config,
+        )
+        .await?;
+
+    debug!("creating certificate {root_ca_cert_name}");
+    topology
+        .create_certificate(
+            root_ca_cert_name.to_string(),
+            self_signed_issuer_name.to_string(),
+            Some(format!("harmony-{}-ca", cluster.name)),
+            None,
+            Some(true),
+            ca_cert_mgmt_config,
+        )
+        .await?;
+
+    debug!("creating issuer '{}'", ca_issuer_name);
+    topology
+        .create_issuer(ca_issuer_name.to_string(), ca_cert_mgmt_config)
+        .await?;
+
+    debug!("creating certificate {}", cluster.tls_cert_name);
+    topology
+        .create_certificate(
+            cluster.tls_cert_name.to_string(),
+            ca_issuer_name.to_string(),
+            None,
+            Some(vec![cluster.dns_name.clone()]),
+            Some(true),
+            ca_cert_mgmt_config,
+        )
+        .await?;
+
+    Ok(Outcome::success("success".to_string()))
+}
+
+async fn build_ca_bundle_secret(
+    namespace: &str,
+    nats_cluster: &NatsCluster,
+    bundle: &Vec<String>,
+) -> Secret {
+    Secret {
+        metadata: ObjectMeta {
+            name: Some(nats_cluster.supercluster_ca_secret_name.to_string()),
+            namespace: Some(namespace.to_string()),
+            ..Default::default()
+        },
+        data: Some(build_secret_data(bundle).await),
+        immutable: Some(false),
+        type_: Some("Opaque".to_string()),
+        string_data: None,
+    }
+}
+
+async fn build_secret_data(bundle: &Vec<String>) -> BTreeMap<String, ByteString> {
+    let mut data = BTreeMap::new();
+
+    data.insert(
+        "ca.crt".to_string(),
+        ByteString(bundle.join("\n").into_bytes()),
+    );
+
+    data
+}
+
+async fn build_ca_bundle_secret_score<T: Topology + K8sclient + 'static>(
+    _topology: T,
+    nats_cluster: &NatsCluster,
+    ca_bundle: &Vec<String>,
+    namespace: String,
+) -> Box<dyn Score<T>> {
+    let bundle_secret = build_ca_bundle_secret(&namespace, nats_cluster, ca_bundle).await;
+    debug!(
+        "deploying secret to ns: {} \nsecret: {:#?}",
+        namespace, bundle_secret
+    );
+    let k8ssecret = K8sResourceScore::single(bundle_secret, Some(namespace));
+    Box::new(k8ssecret)
+}
+
+async fn build_route_score<T: Topology + K8sclient + 'static>(
+    _topology: T,
+    cluster: &NatsCluster,
+    namespace: String,
+) -> Box<dyn Score<T>> {
+    let route = OKDRouteScore {
+        name: cluster.name.to_string(),
+        namespace,
+        spec: RouteSpec {
+            to: RouteTargetReference {
+                kind: "Service".to_string(),
+                name: cluster.name.to_string(),
+                weight: Some(100),
+            },
+            host: Some(cluster.dns_name.clone()),
+            port: Some(RoutePort { target_port: 7222 }),
+            tls: Some(TLSConfig {
+                insecure_edge_termination_policy: None,
+                termination: "passthrough".to_string(),
+                ..Default::default()
+            }),
+            wildcard_policy: None,
+            ..Default::default()
+        },
+    };
+    Box::new(route)
+}
+
+async fn build_deploy_nats_score<T: Topology + HelmCommand + TlsRouter + 'static>(
+    topology: T,
+    cluster: &NatsCluster,
+    peers: Vec<&NatsCluster>,
+    namespace: String,
+) -> Box<dyn Score<T>> {
+    let mut gateway_gateways = String::new();
+    for peer in peers {
+        // Construct wss:// URLs on port 443 for the remote gateways
+        gateway_gateways.push_str(&format!(
+            r#"
+        - name: {}
+          urls:
+            - nats://{}"#,
+            peer.name, peer.gateway_advertise
+        ));
+    }
+    let domain = topology.get_internal_domain().await.unwrap().unwrap();
+
+    // Inject gateway config into the 'merge' block to comply with chart structure
+    let values_yaml = Some(format!(
+        r#"config:
+  merge:
+    authorization:
+      default_permissions:
+        publish: ["TEST.*"]
+        subscribe:  ["PUBLIC.>"]
+      users:
+      # - user: "admin"
+      #   password: "admin_1"
+      #   permissions:
+      #       publish: ">"
+      #       subscribe: ">"
+      - password: "enGk0cgZUabM6bN6FXHT"
+        user: "testUser"
+    accounts:
+      system:
+          users:
+              - user: "admin"
+                password: "admin_2"
+    logtime: true
+    debug: true
+    trace: true
+    system_account: system
+  cluster:
+    name: {cluster_name}
+    enabled: true
+    replicas: {replicas}
+  jetstream:
+    enabled: {jetstream_enabled}
+    fileStorage:
+      enabled: true
+      size: 10Gi
+      storageDirectory: /data/jetstream
+  leafnodes:
+    enabled: false
+  websocket:
+    enabled: false
+    ingress:
+      enabled: true
+      className: openshift-default
+      pathType: Prefix
+      hosts: 
+        - nats-ws.{domain}
+  gateway:
+    enabled: true
+    port: 7222
+    name: {cluster_name}
+    merge:
+        advertise: {gateway_advertise}
+        gateways: {gateway_gateways}
+    tls:
+        enabled: true
+        secretName: {tls_secret_name}
+        # merge:
+        #    ca_file: "/etc/nats-certs/gateway/ca.crt"
+service:
+  ports:
+    gateway:
+      enabled: true
+tlsCA:
+    enabled: true
+    secretName: {supercluster_ca_secret_name}
+natsBox:
+  container:
+    image:
+      tag: nonroot"#,
+        cluster_name = cluster.name,
+        replicas = cluster.replicas,
+        domain = domain,
+        gateway_gateways = gateway_gateways,
+        gateway_advertise = cluster.gateway_advertise,
+        tls_secret_name = format!("{}-tls", cluster.tls_cert_name),
+        jetstream_enabled = cluster.jetstream_enabled,
+        supercluster_ca_secret_name = cluster.supercluster_ca_secret_name,
+    ));
+
+    debug!("Prepared Helm Chart values : \n{values_yaml:#?}");
+    let nats = HelmChartScore {
+        namespace: Some(NonBlankString::from_str(&namespace).unwrap()),
+        release_name: NonBlankString::from_str(&cluster.name).unwrap(),
+        chart_name: NonBlankString::from_str("nats/nats").unwrap(),
+        chart_version: None,
+        values_overrides: None,
+        values_yaml,
+        create_namespace: true,
+        install_only: false,
+        repository: Some(HelmRepository::new(
+            "nats".to_string(),
+            hurl!("https://nats-io.github.io/k8s/helm/charts/"),
+            true,
+        )),
+    };
+
+    Box::new(nats)
+}

harmony/src/domain/topology/k8s_anywhere/k8s_anywhere.rs

@@ -14,11 +14,24 @@ use tokio::sync::OnceCell;
 
 use crate::{
     executors::ExecutorError,
-    interpret::InterpretStatus,
+    interpret::{InterpretStatus, Outcome},
     inventory::Inventory,
     modules::{
+        cert_manager::{
+            capability::{CertificateManagement, CertificateManagementConfig},
+            crd::{
+                certificate::Certificate, issuer::Issuer,
+                score_k8s_certificate::K8sCertManagerCertificateScore,
+                score_k8s_issuer::K8sCertManagerIssuerScore,
+            },
+            operator::CertManagerOperatorScore,
+            score_cert_management::CertificateManagementScore,
+        },
         k3d::K3DInstallationScore,
-        k8s::ingress::{K8sIngressScore, PathType},
+        k8s::{
+            apps::crd::Subscription,
+            ingress::{K8sIngressScore, PathType},
+        },
         monitoring::{
             grafana::{grafana::Grafana, helm::helm_grafana::grafana_helm_chart_score},
             kube_prometheus::crd::{
@@ -384,6 +397,148 @@ impl Serialize for K8sAnywhereTopology {
     }
 }
 
+#[async_trait]
+impl CertificateManagement for K8sAnywhereTopology {
+    async fn install(&self) -> Result<Outcome, ExecutorError> {
+        let cert_management_operator = CertManagerOperatorScore::default();
+
+        cert_management_operator
+            .interpret(&Inventory::empty(), self)
+            .await
+            .map_err(|e| ExecutorError::UnexpectedError(e.to_string()))?;
+
+        Ok(Outcome::success(format!(
+            "Installed cert-manager into ns: {}",
+            cert_management_operator.namespace
+        )))
+    }
+
+    async fn ensure_certificate_management_ready(&self) -> Result<Outcome, ExecutorError> {
+        let k8s_client = self.k8s_client().await.unwrap();
+
+        match k8s_client
+            .get_resource::<Subscription>("cert-manager", Some("openshift-operators"))
+            .await
+            .map_err(|e| ExecutorError::UnexpectedError(format!("{}", e)))?
+        {
+            Some(subscription) => {
+                trace!("subscription {:#?}", subscription,);
+                Ok(Outcome::success(format!("Certificate Management Ready",)))
+            }
+            None => self.install().await,
+        }
+    }
+
+    async fn create_issuer(
+        &self,
+        issuer_name: String,
+        config: &CertificateManagementConfig,
+    ) -> Result<Outcome, ExecutorError> {
+        let issuer_score = K8sCertManagerIssuerScore {
+            issuer_name: issuer_name.clone(),
+            config: config.clone(),
+        };
+
+        issuer_score
+            .interpret(&Inventory::empty(), self)
+            .await
+            .map_err(|e| ExecutorError::UnexpectedError(e.to_string()))?;
+
+        Ok(Outcome::success(format!(
+            "issuer of kind {} is ready",
+            issuer_name
+        )))
+    }
+
+    async fn create_certificate(
+        &self,
+        cert_name: String,
+        issuer_name: String,
+        common_name: Option<String>,
+        dns_names: Option<Vec<String>>,
+        is_ca: Option<bool>,
+        config: &CertificateManagementConfig,
+    ) -> Result<Outcome, ExecutorError> {
+        self.certificate_issuer_ready(
+            issuer_name.clone(),
+            self.k8s_client().await.unwrap(),
+            config,
+        )
+        .await?;
+
+        let cert = K8sCertManagerCertificateScore {
+            cert_name: cert_name,
+            issuer_name,
+            config: config.clone(),
+            common_name,
+            is_ca,
+            dns_names,
+        };
+        cert.interpret(&Inventory::empty(), self)
+            .await
+            .map_err(|e| ExecutorError::UnexpectedError(e.to_string()))?;
+
+        Ok(Outcome::success(format!(
+            "Created cert into ns: {:#?}",
+            config.namespace.clone()
+        )))
+    }
+
+    async fn get_ca_certificate(
+        &self,
+        cert_name: String,
+        config: &CertificateManagementConfig,
+    ) -> Result<String, ExecutorError> {
+        let namespace = config.namespace.clone().unwrap();
+
+        let client = self.k8s_client().await.unwrap();
+
+        if let Some(certificate) = client
+            .get_resource::<Certificate>(&cert_name, Some(&namespace))
+            .await
+            .map_err(|e| ExecutorError::UnexpectedError(format!("{}", e)))?
+        {
+            let secret_name = certificate.spec.secret_name.clone();
+
+            trace!("Secret Name {:#?}", secret_name);
+            if let Some(secret) = client
+                .get_resource::<Secret>(&secret_name, Some(&namespace))
+                .await
+                .map_err(|e| {
+                    ExecutorError::UnexpectedError(format!(
+                        "secret {} not found in namespace {}: {}",
+                        secret_name, namespace, e
+                    ))
+                })?
+            {
+                let ca_cert = secret
+                    .data
+                    .as_ref()
+                    .and_then(|d| d.get("ca.crt"))
+                    .ok_or_else(|| {
+                        ExecutorError::UnexpectedError("Secret missing key 'ca.crt'".into())
+                    })?;
+
+                let ca_cert = String::from_utf8(ca_cert.0.clone()).map_err(|_| {
+                    ExecutorError::UnexpectedError("ca.crt is not valid UTF-8".into())
+                })?;
+
+                return Ok(ca_cert);
+            } else {
+                Err(ExecutorError::UnexpectedError(format!(
+                    "Error getting secret associated with cert_name: {}",
+                    cert_name
+                )))
+            }
+        } else {
+            return Err(ExecutorError::UnexpectedError(format!(
+                "Certificate {} not found in namespace {}",
+                cert_name, namespace
+            )));
+        }
+    }
+}
+
 impl K8sAnywhereTopology {
     pub fn from_env() -> Self {
         Self {
@@ -403,6 +558,38 @@ impl K8sAnywhereTopology {
         }
     }
 
+    pub async fn certificate_issuer_ready(
+        &self,
+        issuer_name: String,
+        k8s_client: Arc<K8sClient>,
+        config: &CertificateManagementConfig,
+    ) -> Result<Outcome, ExecutorError> {
+        let ns = config
+            .namespace
+            .clone()
+            .ok_or_else(|| ExecutorError::UnexpectedError("namespace is required".to_string()))?;
+
+        match k8s_client
+            .get_resource::<Issuer>(&issuer_name, Some(&ns))
+            .await
+        {
+            Ok(Some(_cert_issuer)) => Ok(Outcome::success(format!(
+                "issuer of kind {} is ready",
+                issuer_name
+            ))),
+
+            Ok(None) => Err(ExecutorError::UnexpectedError(format!(
+                "Issuer {} not present in namespace {}",
+                issuer_name, ns
+            ))),
+
+            Err(e) => Err(ExecutorError::UnexpectedError(format!(
+                "Failed to fetch Issuer {}: {}",
+                issuer_name, e
+            ))),
+        }
+    }
+
     pub async fn get_k8s_distribution(&self) -> Result<&KubernetesDistribution, PreparationError> {
         self.k8s_distribution
             .get_or_try_init(async || {
@@ -1089,6 +1276,12 @@ impl Topology for K8sAnywhereTopology {
             .await
             .map_err(PreparationError::new)?;
 
+        let cert_mgmt = CertificateManagementScore {};
+        cert_mgmt
+            .interpret(&Inventory::empty(), self)
+            .await
+            .map_err(|e| PreparationError::new(format!("{}", e)))?;
+
         match self.is_helm_available() {
             Ok(()) => Ok(PreparationOutcome::Success {
                 details: format!("{} + helm available", k8s_state.message.clone()),

harmony/src/modules/cert_manager/capability.rs

@@ -0,0 +1,46 @@
+use async_trait::async_trait;
+use serde::Serialize;
+
+use crate::{
+    executors::ExecutorError,
+    interpret::Outcome,
+    modules::cert_manager::crd::{AcmeIssuer, CaIssuer},
+};
+
+///TODO rust doc explaining issuer, certificate etc
+#[async_trait]
+pub trait CertificateManagement: Send + Sync {
+    async fn install(&self) -> Result<Outcome, ExecutorError>;
+
+    async fn ensure_certificate_management_ready(&self) -> Result<Outcome, ExecutorError>;
+
+    async fn create_issuer(
+        &self,
+        issuer_name: String,
+        config: &CertificateManagementConfig,
+    ) -> Result<Outcome, ExecutorError>;
+
+    async fn create_certificate(
+        &self,
+        cert_name: String,
+        issuer_name: String,
+        common_name: Option<String>,
+        dns_names: Option<Vec<String>>,
+        is_ca: Option<bool>,
+        config: &CertificateManagementConfig,
+    ) -> Result<Outcome, ExecutorError>;
+
+    async fn get_ca_certificate(
+        &self,
+        cert_name: String,
+        config: &CertificateManagementConfig,
+    ) -> Result<String, ExecutorError>;
+}
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CertificateManagementConfig {
+    pub namespace: Option<String>,
+    pub acme_issuer: Option<AcmeIssuer>,
+    pub ca_issuer: Option<CaIssuer>,
+    pub self_signed: bool,
+}

harmony/src/modules/cert_manager/crd/certificate.rs

@@ -0,0 +1,113 @@
+use kube::{CustomResource, api::ObjectMeta};
+use serde::{Deserialize, Serialize};
+
+#[derive(CustomResource, Deserialize, Serialize, Clone, Debug)]
+#[kube(
+    group = "cert-manager.io",
+    version = "v1",
+    kind = "Certificate",
+    plural = "certificates",
+    namespaced = true,
+    schema = "disabled"
+)]
+#[serde(rename_all = "camelCase")]
+pub struct CertificateSpec {
+    /// Name of the Secret where the certificate will be stored
+    pub secret_name: String,
+
+    /// Common Name (optional but often discouraged in favor of SANs)
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub common_name: Option<String>,
+
+    /// DNS Subject Alternative Names
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub dns_names: Option<Vec<String>>,
+
+    /// IP Subject Alternative Names
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub ip_addresses: Option<Vec<String>>,
+
+    /// Certificate duration (e.g. "2160h")
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub duration: Option<String>,
+
+    /// How long before expiry cert-manager should renew
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub renew_before: Option<String>,
+
+    /// Reference to the Issuer or ClusterIssuer
+    pub issuer_ref: IssuerRef,
+
+    /// Is this a CA certificate
+    #[serde(skip_serializing_if = "Option::is_none")]
+    #[serde(rename = "isCA")]
+    pub is_ca: Option<bool>,
+
+    /// Private key configuration
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub private_key: Option<PrivateKey>,
+}
+
+impl Default for Certificate {
+    fn default() -> Self {
+        Certificate {
+            metadata: ObjectMeta::default(),
+            spec: CertificateSpec::default(),
+        }
+    }
+}
+
+impl Default for CertificateSpec {
+    fn default() -> Self {
+        Self {
+            secret_name: String::new(),
+            common_name: None,
+            dns_names: None,
+            ip_addresses: None,
+            duration: None,
+            renew_before: None,
+            issuer_ref: IssuerRef::default(),
+            is_ca: None,
+            private_key: None,
+        }
+    }
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct IssuerRef {
+    pub name: String,
+
+    /// Either "Issuer" or "ClusterIssuer"
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub kind: Option<String>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub group: Option<String>,
+}
+
+impl Default for IssuerRef {
+    fn default() -> Self {
+        Self {
+            name: String::new(),
+            kind: None,
+            group: None,
+        }
+    }
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct PrivateKey {
+    /// RSA or ECDSA
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub algorithm: Option<String>,
+
+    /// Key size (e.g. 2048, 4096)
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub size: Option<u32>,
+
+    /// Rotation policy: "Never" or "Always"
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub rotation_policy: Option<String>,
+}

harmony/src/modules/cert_manager/crd/cluster_issuer.rs

@@ -0,0 +1,44 @@
+use kube::{CustomResource, api::ObjectMeta};
+use serde::{Deserialize, Serialize};
+
+use crate::modules::cert_manager::crd::{AcmeIssuer, CaIssuer, SelfSignedIssuer};
+
+#[derive(CustomResource, Deserialize, Serialize, Clone, Debug)]
+#[kube(
+    group = "cert-manager.io",
+    version = "v1",
+    kind = "ClusterIssuer",
+    plural = "clusterissuers",
+    namespaced = false,
+    schema = "disabled"
+)]
+#[serde(rename_all = "camelCase")]
+pub struct ClusterIssuerSpec {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub self_signed: Option<SelfSignedIssuer>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub ca: Option<CaIssuer>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub acme: Option<AcmeIssuer>,
+}
+
+impl Default for ClusterIssuer {
+    fn default() -> Self {
+        ClusterIssuer {
+            metadata: ObjectMeta::default(),
+            spec: ClusterIssuerSpec::default(),
+        }
+    }
+}
+
+impl Default for ClusterIssuerSpec {
+    fn default() -> Self {
+        Self {
+            self_signed: None,
+            ca: None,
+            acme: None,
+        }
+    }
+}

harmony/src/modules/cert_manager/crd/issuer.rs

@@ -0,0 +1,44 @@
+use kube::{CustomResource, api::ObjectMeta};
+use serde::{Deserialize, Serialize};
+
+use crate::modules::cert_manager::crd::{AcmeIssuer, CaIssuer, SelfSignedIssuer};
+
+#[derive(CustomResource, Deserialize, Serialize, Clone, Debug)]
+#[kube(
+    group = "cert-manager.io",
+    version = "v1",
+    kind = "Issuer",
+    plural = "issuers",
+    namespaced = true,
+    schema = "disabled"
+)]
+#[serde(rename_all = "camelCase")]
+pub struct IssuerSpec {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub self_signed: Option<SelfSignedIssuer>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub ca: Option<CaIssuer>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub acme: Option<AcmeIssuer>,
+}
+
+impl Default for Issuer {
+    fn default() -> Self {
+        Issuer {
+            metadata: ObjectMeta::default(),
+            spec: IssuerSpec::default(),
+        }
+    }
+}
+
+impl Default for IssuerSpec {
+    fn default() -> Self {
+        Self {
+            self_signed: None,
+            ca: None,
+            acme: None,
+        }
+    }
+}

harmony/src/modules/cert_manager/crd/mod.rs

@@ -0,0 +1,65 @@
+use serde::{Deserialize, Serialize};
+
+pub mod certificate;
+pub mod cluster_issuer;
+pub mod issuer;
+//pub mod score_cluster_issuer;
+pub mod score_k8s_certificate;
+pub mod score_k8s_issuer;
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct CaIssuer {
+    /// Secret containing `tls.crt` and `tls.key`
+    pub secret_name: String,
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug, Default)]
+#[serde(rename_all = "camelCase")]
+pub struct SelfSignedIssuer {}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct AcmeIssuer {
+    pub server: String,
+    pub email: String,
+
+    /// Secret used to store the ACME account private key
+    pub private_key_secret_ref: SecretKeySelector,
+
+    pub solvers: Vec<AcmeSolver>,
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct SecretKeySelector {
+    pub name: String,
+    pub key: String,
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct AcmeSolver {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub http01: Option<Http01Solver>,
+
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub dns01: Option<Dns01Solver>,
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct Dns01Solver {}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct Http01Solver {
+    pub ingress: IngressSolver,
+}
+
+#[derive(Deserialize, Serialize, Clone, Debug)]
+#[serde(rename_all = "camelCase")]
+pub struct IngressSolver {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub class: Option<String>,
+}

harmony/src/modules/cert_manager/crd/score_k8s_certificate.rs

@@ -0,0 +1,54 @@
+use kube::api::ObjectMeta;
+use serde::Serialize;
+
+use crate::{
+    interpret::Interpret,
+    modules::{
+        cert_manager::{
+            capability::CertificateManagementConfig,
+            crd::certificate::{Certificate, CertificateSpec, IssuerRef},
+        },
+        k8s::resource::K8sResourceScore,
+    },
+    score::Score,
+    topology::{K8sclient, Topology},
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct K8sCertManagerCertificateScore {
+    pub cert_name: String,
+    pub issuer_name: String,
+    pub common_name: Option<String>,
+    pub dns_names: Option<Vec<String>>,
+    pub is_ca: Option<bool>,
+    pub config: CertificateManagementConfig,
+}
+
+impl<T: Topology + K8sclient> Score<T> for K8sCertManagerCertificateScore {
+    fn name(&self) -> String {
+        "CertificateScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        let cert = Certificate {
+            metadata: ObjectMeta {
+                name: Some(self.cert_name.clone()),
+                namespace: self.config.namespace.clone(),
+                ..Default::default()
+            },
+            spec: CertificateSpec {
+                secret_name: format!("{}-tls", self.cert_name.clone()),
+                issuer_ref: IssuerRef {
+                    name: self.issuer_name.clone(),
+                    kind: Some("Issuer".into()),
+                    group: Some("cert-manager.io".into()),
+                },
+                common_name: self.common_name.clone(),
+                is_ca: self.is_ca.clone(),
+                dns_names: self.dns_names.clone(),
+                ..Default::default()
+            },
+        };
+        K8sResourceScore::single(cert, self.config.namespace.clone()).create_interpret()
+    }
+}

harmony/src/modules/cert_manager/crd/score_k8s_cluster_issuer.rs

@@ -0,0 +1,51 @@
+use kube::api::ObjectMeta;
+use serde::Serialize;
+
+use crate::{
+    interpret::Interpret,
+    modules::{
+        cert_manager::crd::{
+            AcmeIssuer, CaIssuer, SelfSignedIssuer,
+            cluster_issuer::{ClusterIssuer, ClusterIssuerSpec},
+        },
+        k8s::resource::K8sResourceScore,
+    },
+    score::Score,
+    topology::{K8sclient, Topology},
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct ClusterIssuerScore {
+    name: String,
+    acme_issuer: Option<AcmeIssuer>,
+    ca_issuer: Option<CaIssuer>,
+    self_signed: bool,
+}
+
+impl<T: Topology + K8sclient> Score<T> for ClusterIssuerScore {
+    fn name(&self) -> String {
+        "ClusterIssuerScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        let metadata = ObjectMeta {
+            name: Some(self.name.clone()),
+            namespace: None,
+            ..ObjectMeta::default()
+        };
+
+        let spec = ClusterIssuerSpec {
+            acme: self.acme_issuer.clone(),
+            ca: self.ca_issuer.clone(),
+            self_signed: if self.self_signed {
+                Some(SelfSignedIssuer::default())
+            } else {
+                None
+            },
+        };
+
+        let cluster_issuer = ClusterIssuer { metadata, spec };
+
+        K8sResourceScore::single(cluster_issuer, None).create_interpret()
+    }
+}

harmony/src/modules/cert_manager/crd/score_k8s_issuer.rs

@@ -0,0 +1,52 @@
+use kube::api::ObjectMeta;
+use serde::Serialize;
+
+use crate::{
+    interpret::Interpret,
+    modules::{
+        cert_manager::{
+            capability::CertificateManagementConfig,
+            crd::{
+                SelfSignedIssuer,
+                issuer::{Issuer, IssuerSpec},
+            },
+        },
+        k8s::resource::K8sResourceScore,
+    },
+    score::Score,
+    topology::{K8sclient, Topology},
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct K8sCertManagerIssuerScore {
+    pub issuer_name: String,
+    pub config: CertificateManagementConfig,
+}
+
+impl<T: Topology + K8sclient> Score<T> for K8sCertManagerIssuerScore {
+    fn name(&self) -> String {
+        "IssuerScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        let metadata = ObjectMeta {
+            name: Some(self.issuer_name.clone()),
+            namespace: self.config.namespace.clone(),
+            ..ObjectMeta::default()
+        };
+
+        let spec = IssuerSpec {
+            acme: self.config.acme_issuer.clone(),
+            ca: self.config.ca_issuer.clone(),
+            self_signed: if self.config.self_signed {
+                Some(SelfSignedIssuer::default())
+            } else {
+                None
+            },
+        };
+
+        let issuer = Issuer { metadata, spec };
+
+        K8sResourceScore::single(issuer, self.config.namespace.clone()).create_interpret()
+    }
+}

harmony/src/modules/cert_manager/mod.rs

@@ -1,3 +1,9 @@
+pub mod capability;
 pub mod cluster_issuer;
+pub mod crd;
 mod helm;
+pub mod operator;
+pub mod score_cert_management;
+pub mod score_certificate;
+pub mod score_issuer;
 pub use helm::*;

harmony/src/modules/cert_manager/operator.rs

@@ -0,0 +1,64 @@
+use kube::api::ObjectMeta;
+use serde::Serialize;
+
+use crate::{
+    interpret::Interpret,
+    modules::k8s::{
+        apps::crd::{Subscription, SubscriptionSpec},
+        resource::K8sResourceScore,
+    },
+    score::Score,
+    topology::{K8sclient, Topology, k8s::K8sClient},
+};
+
+/// Install the Cert-Manager Operator via RedHat Community Operators registry.redhat.io/redhat/community-operator-index:v4.19
+/// This Score creates a Subscription CR in the specified namespace
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CertManagerOperatorScore {
+    pub namespace: String,
+    pub channel: String,
+    pub install_plan_approval: String,
+    pub source: String,
+    pub source_namespace: String,
+}
+
+impl Default for CertManagerOperatorScore {
+    fn default() -> Self {
+        Self {
+            namespace: "openshift-operators".to_string(),
+            channel: "stable".to_string(),
+            install_plan_approval: "Automatic".to_string(),
+            source: "community-operators".to_string(),
+            source_namespace: "openshift-marketplace".to_string(),
+        }
+    }
+}
+
+impl<T: Topology + K8sclient> Score<T> for CertManagerOperatorScore {
+    fn name(&self) -> String {
+        "CertManagerOperatorScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        let metadata = ObjectMeta {
+            name: Some("cert-manager".to_string()),
+            namespace: Some(self.namespace.clone()),
+            ..ObjectMeta::default()
+        };
+
+        let spec = SubscriptionSpec {
+            channel: Some(self.channel.clone()),
+            config: None,
+            install_plan_approval: Some(self.install_plan_approval.clone()),
+            name: "cert-manager".to_string(),
+            source: self.source.clone(),
+            source_namespace: self.source_namespace.clone(),
+            starting_csv: None,
+        };
+
+        let subscription = Subscription { metadata, spec };
+
+        K8sResourceScore::single(subscription, Some(self.namespace.clone())).create_interpret()
+    }
+}

harmony/src/modules/cert_manager/score_cert_management.rs

@@ -0,0 +1,59 @@
+use async_trait::async_trait;
+use harmony_types::id::Id;
+use serde::Serialize;
+
+use crate::{
+    data::Version,
+    interpret::{Interpret, InterpretError, InterpretName, InterpretStatus, Outcome},
+    inventory::Inventory,
+    modules::cert_manager::capability::{CertificateManagement, CertificateManagementConfig},
+    score::Score,
+    topology::Topology,
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CertificateManagementScore {}
+
+impl<T: Topology + CertificateManagement> Score<T> for CertificateManagementScore {
+    fn name(&self) -> String {
+        "CertificateManagementScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        Box::new(CertificateManagementInterpret {})
+    }
+}
+
+#[derive(Debug)]
+struct CertificateManagementInterpret {}
+
+#[async_trait]
+impl<T: Topology + CertificateManagement> Interpret<T> for CertificateManagementInterpret {
+    async fn execute(
+        &self,
+        _inventory: &Inventory,
+        topology: &T,
+    ) -> Result<Outcome, InterpretError> {
+        CertificateManagement::ensure_certificate_management_ready(topology)
+            .await
+            .map_err(|e| InterpretError::new(e.to_string()))?;
+
+        Ok(Outcome::success(format!("CertificateManagement is ready")))
+    }
+
+    fn get_name(&self) -> InterpretName {
+        InterpretName::Custom("CertificateManagementInterpret")
+    }
+
+    fn get_version(&self) -> Version {
+        todo!()
+    }
+
+    fn get_status(&self) -> InterpretStatus {
+        todo!()
+    }
+
+    fn get_children(&self) -> Vec<Id> {
+        todo!()
+    }
+}

harmony/src/modules/cert_manager/score_certificate.rs

@@ -0,0 +1,89 @@
+use async_trait::async_trait;
+use harmony_types::id::Id;
+use log::trace;
+use serde::Serialize;
+
+use crate::{
+    data::Version,
+    interpret::{Interpret, InterpretError, InterpretName, InterpretStatus, Outcome},
+    inventory::Inventory,
+    modules::cert_manager::capability::{CertificateManagement, CertificateManagementConfig},
+    score::Score,
+    topology::Topology,
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CertificateScore {
+    pub cert_name: String,
+    pub issuer_name: String,
+    pub common_name: Option<String>,
+    pub dns_names: Option<Vec<String>>,
+    pub is_ca: Option<bool>,
+    pub config: CertificateManagementConfig,
+}
+
+impl<T: Topology + CertificateManagement> Score<T> for CertificateScore {
+    fn name(&self) -> String {
+        "CertificateCreationScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        Box::new(CertificateInterpret {
+            cert_name: self.cert_name.clone(),
+            issuer_name: self.issuer_name.clone(),
+            common_name: self.common_name.clone(),
+            dns_names: self.dns_names.clone(),
+            is_ca: self.is_ca.clone(),
+            config: self.config.clone(),
+        })
+    }
+}
+
+#[derive(Debug)]
+struct CertificateInterpret {
+    cert_name: String,
+    issuer_name: String,
+    common_name: Option<String>,
+    dns_names: Option<Vec<String>>,
+    is_ca: Option<bool>,
+    config: CertificateManagementConfig,
+}
+
+#[async_trait]
+impl<T: Topology + CertificateManagement> Interpret<T> for CertificateInterpret {
+    async fn execute(
+        &self,
+        inventory: &Inventory,
+        topology: &T,
+    ) -> Result<Outcome, InterpretError> {
+        let _certificate = topology
+            .create_certificate(
+                self.cert_name.clone(),
+                self.issuer_name.clone(),
+                self.common_name.clone(),
+                self.dns_names.clone(),
+                self.is_ca.clone(),
+                &self.config,
+            )
+            .await
+            .map_err(|e| InterpretError::new(e.to_string()))?;
+
+        Ok(Outcome::success(format!("Installed CertificateManagement")))
+    }
+
+    fn get_name(&self) -> InterpretName {
+        InterpretName::Custom("CertificateManagementInterpret")
+    }
+
+    fn get_version(&self) -> Version {
+        todo!()
+    }
+
+    fn get_status(&self) -> InterpretStatus {
+        todo!()
+    }
+
+    fn get_children(&self) -> Vec<Id> {
+        todo!()
+    }
+}

harmony/src/modules/cert_manager/score_issuer.rs

@@ -0,0 +1,71 @@
+use async_trait::async_trait;
+use harmony_types::id::Id;
+use log::debug;
+use serde::Serialize;
+
+use crate::{
+    data::Version,
+    interpret::{Interpret, InterpretError, InterpretName, InterpretStatus, Outcome},
+    inventory::Inventory,
+    modules::cert_manager::capability::{CertificateManagement, CertificateManagementConfig},
+    score::Score,
+    topology::Topology,
+};
+
+#[derive(Debug, Clone, Serialize)]
+pub struct CertificateIssuerScore {
+    pub issuer_name: String,
+    pub config: CertificateManagementConfig,
+}
+
+impl<T: Topology + CertificateManagement> Score<T> for CertificateIssuerScore {
+    fn name(&self) -> String {
+        "CertificateIssuerScore".to_string()
+    }
+
+    fn create_interpret(&self) -> Box<dyn Interpret<T>> {
+        Box::new(CertificateIssuerInterpret {
+            issuer_name: self.issuer_name.clone(),
+            config: self.config.clone(),
+        })
+    }
+}
+
+#[derive(Debug)]
+struct CertificateIssuerInterpret {
+    issuer_name: String,
+    config: CertificateManagementConfig,
+}
+
+#[async_trait]
+impl<T: Topology + CertificateManagement> Interpret<T> for CertificateIssuerInterpret {
+    async fn execute(
+        &self,
+        inventory: &Inventory,
+        topology: &T,
+    ) -> Result<Outcome, InterpretError> {
+        debug!("issuer name: {}", self.issuer_name.clone());
+        let _cert_issuer = topology
+            .create_issuer(self.issuer_name.clone(), &self.config)
+            .await
+            .map_err(|e| InterpretError::new(e.to_string()))?;
+
+        Ok(Outcome::success(format!("Installed CertificateManagement")))
+    }
+
+    fn get_name(&self) -> InterpretName {
+        InterpretName::Custom("CertificateManagementInterpret")
+    }
+
+    fn get_version(&self) -> Version {
+        todo!()
+    }
+
+    fn get_status(&self) -> InterpretStatus {
+        todo!()
+    }
+
+    fn get_children(&self) -> Vec<Id> {
+        todo!()
+    }
+}

harmony/src/modules/okd/crd/ingresses_config.rs

@@ -211,4 +211,3 @@ pub struct ObjectReference {
     pub namespace: String,
     pub resource: String,
 }
-

harmony/src/modules/okd/crd/mod.rs

@@ -1,3 +1,3 @@
+pub mod ingresses_config;
 pub mod nmstate;
 pub mod route;
-pub mod ingresses_config;