Compare commits

..

5 Commits

57 changed files with 1047 additions and 622 deletions

31
Cargo.lock generated
View File

@@ -690,6 +690,23 @@ dependencies = [
"tokio",
]
[[package]]
name = "brocade-switch"
version = "0.1.0"
dependencies = [
"async-trait",
"brocade",
"env_logger",
"harmony",
"harmony_cli",
"harmony_macros",
"harmony_types",
"log",
"serde",
"tokio",
"url",
]
[[package]]
name = "brotli"
version = "8.0.2"
@@ -2479,6 +2496,19 @@ dependencies = [
"tokio",
]
[[package]]
name = "harmony_inventory_builder"
version = "0.1.0"
dependencies = [
"cidr",
"harmony",
"harmony_cli",
"harmony_macros",
"harmony_types",
"tokio",
"url",
]
[[package]]
name = "harmony_macros"
version = "0.1.0"
@@ -2544,6 +2574,7 @@ dependencies = [
name = "harmony_types"
version = "0.1.0"
dependencies = [
"log",
"rand 0.9.2",
"serde",
"url",

View File

@@ -1,6 +1,4 @@
# Harmony
Open-source infrastructure orchestration that treats your platform like first-class code.
# Harmony : Open-source infrastructure orchestration that treats your platform like first-class code
_By [NationTech](https://nationtech.io)_
@@ -20,7 +18,9 @@ All in **one strongly-typed Rust codebase**.
From a **developer laptop** to a **global production cluster**, a single **source of truth** drives the **full software lifecycle.**
## The Harmony Philosophy
---
## 1 · The Harmony Philosophy
Infrastructure is essential, but it shouldnt be your core business. Harmony is built on three guiding principles that make modern platforms reliable, repeatable, and easy to reason about.
@@ -32,18 +32,9 @@ Infrastructure is essential, but it shouldnt be your core business. Harmony i
These principles surface as simple, ergonomic Rust APIs that let teams focus on their product while trusting the platform underneath.
## Where to Start
---
We have a comprehensive set of documentation right here in the repository.
| I want to... | Start Here |
| ----------------- | ------------------------------------------------------------------ |
| Get Started | [Getting Started Guide](./docs/guides/getting-started.md) |
| See an Example | [Use Case: Deploy a Rust Web App](./docs/use-cases/rust-webapp.md) |
| Explore | [Documentation Hub](./docs/README.md) |
| See Core Concepts | [Core Concepts Explained](./docs/concepts.md) |
## Quick Look: Deploy a Rust Webapp
## 2 · Quick Start
The snippet below spins up a complete **production-grade Rust + Leptos Webapp** with monitoring. Swap it for your own scores to deploy anything from microservices to machine-learning pipelines.
@@ -101,33 +92,63 @@ async fn main() {
}
```
To run this:
Run it:
- Clone the repository: `git clone https://git.nationtech.io/nationtech/harmony`
- Install dependencies: `cargo build --release`
- Run the example: `cargo run --example try_rust_webapp`
```bash
cargo run
```
## Documentation
Harmony analyses the code, shows an execution plan in a TUI, and applies it once you confirm. Same code, same binary—every environment.
All documentation is in the `/docs` directory.
---
- [Documentation Hub](./docs/README.md): The main entry point for all documentation.
- [Core Concepts](./docs/concepts.md): A detailed look at Score, Topology, Capability, Inventory, and Interpret.
- [Component Catalogs](./docs/catalogs/README.md): Discover all available Scores, Topologies, and Capabilities.
- [Developer Guide](./docs/guides/developer-guide.md): Learn how to write your own Scores and Topologies.
## 3 · Core Concepts
## Architectural Decision Records
| Term | One-liner |
| ---------------- | ---------------------------------------------------------------------------------------------------- |
| **Score<T>** | Declarative description of the desired state (e.g., `LAMPScore`). |
| **Interpret<T>** | Imperative logic that realises a `Score` on a specific environment. |
| **Topology** | An environment (local k3d, AWS, bare-metal) exposing verified _Capabilities_ (Kubernetes, DNS, …). |
| **Maestro** | Orchestrator that compiles Scores + Topology, ensuring all capabilities line up **at compile-time**. |
| **Inventory** | Optional catalogue of physical assets for bare-metal and edge deployments. |
- [ADR-001 · Why Rust](adr/001-rust.md)
- [ADR-003 · Infrastructure Abstractions](adr/003-infrastructure-abstractions.md)
- [ADR-006 · Secret Management](adr/006-secret-management.md)
- [ADR-011 · Multi-Tenant Cluster](adr/011-multi-tenant-cluster.md)
A visual overview is in the diagram below.
## Contribute
[Harmony Core Architecture](docs/diagrams/Harmony_Core_Architecture.drawio.svg)
Discussions and roadmap live in [Issues](https://git.nationtech.io/nationtech/harmony/-/issues). PRs, ideas, and feedback are welcome!
---
## License
## 4 · Install
Prerequisites:
- Rust
- Docker (if you deploy locally)
- `kubectl` / `helm` for Kubernetes-based topologies
```bash
git clone https://git.nationtech.io/nationtech/harmony
cd harmony
cargo build --release # builds the CLI, TUI and libraries
```
---
## 5 · Learning More
- **Architectural Decision Records** dive into the rationale
- [ADR-001 · Why Rust](adr/001-rust.md)
- [ADR-003 · Infrastructure Abstractions](adr/003-infrastructure-abstractions.md)
- [ADR-006 · Secret Management](adr/006-secret-management.md)
- [ADR-011 · Multi-Tenant Cluster](adr/011-multi-tenant-cluster.md)
- **Extending Harmony** write new Scores / Interprets, add hardware like OPNsense firewalls, or embed Harmony in your own tooling (`/docs`).
- **Community** discussions and roadmap live in [GitLab issues](https://git.nationtech.io/nationtech/harmony/-/issues). PRs, ideas, and feedback are welcome!
---
## 6 · License
Harmony is released under the **GNU AGPL v3**.

View File

@@ -1,6 +1,6 @@
use std::net::{IpAddr, Ipv4Addr};
use brocade::BrocadeOptions;
use brocade::{BrocadeOptions, ssh};
use harmony_secret::{Secret, SecretManager};
use harmony_types::switch::PortLocation;
use serde::{Deserialize, Serialize};
@@ -16,23 +16,28 @@ async fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
// let ip = IpAddr::V4(Ipv4Addr::new(10, 0, 0, 250)); // old brocade @ ianlet
let ip = IpAddr::V4(Ipv4Addr::new(192, 168, 55, 101)); // brocade @ sto1
let ip = IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)); // brocade @ sto1
// let ip = IpAddr::V4(Ipv4Addr::new(192, 168, 4, 11)); // brocade @ st
let switch_addresses = vec![ip];
let config = SecretManager::get_or_prompt::<BrocadeSwitchAuth>()
.await
.unwrap();
// let config = SecretManager::get_or_prompt::<BrocadeSwitchAuth>()
// .await
// .unwrap();
let brocade = brocade::init(
&switch_addresses,
22,
&config.username,
&config.password,
Some(BrocadeOptions {
// &config.username,
// &config.password,
"admin",
"password",
BrocadeOptions {
dry_run: true,
ssh: ssh::SshOptions {
port: 2222,
..Default::default()
},
..Default::default()
}),
},
)
.await
.expect("Brocade client failed to connect");
@@ -54,6 +59,7 @@ async fn main() {
}
println!("--------------");
todo!();
let channel_name = "1";
brocade.clear_port_channel(channel_name).await.unwrap();

View File

@@ -140,7 +140,7 @@ impl BrocadeClient for FastIronClient {
async fn configure_interfaces(
&self,
_interfaces: Vec<(String, PortOperatingMode)>,
_interfaces: &Vec<(String, PortOperatingMode)>,
) -> Result<(), Error> {
todo!()
}

View File

@@ -14,11 +14,12 @@ use async_trait::async_trait;
use harmony_types::net::MacAddress;
use harmony_types::switch::{PortDeclaration, PortLocation};
use regex::Regex;
use serde::Serialize;
mod fast_iron;
mod network_operating_system;
mod shell;
mod ssh;
pub mod ssh;
#[derive(Default, Clone, Debug)]
pub struct BrocadeOptions {
@@ -118,7 +119,7 @@ impl fmt::Display for InterfaceType {
}
/// Defines the primary configuration mode of a switch interface, representing mutually exclusive roles.
#[derive(Debug, PartialEq, Eq, Clone)]
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub enum PortOperatingMode {
/// The interface is explicitly configured for Brocade fabric roles (ISL or Trunk enabled).
Fabric,
@@ -141,12 +142,11 @@ pub enum InterfaceStatus {
pub async fn init(
ip_addresses: &[IpAddr],
port: u16,
username: &str,
password: &str,
options: Option<BrocadeOptions>,
options: BrocadeOptions,
) -> Result<Box<dyn BrocadeClient + Send + Sync>, Error> {
let shell = BrocadeShell::init(ip_addresses, port, username, password, options).await?;
let shell = BrocadeShell::init(ip_addresses, username, password, options).await?;
let version_info = shell
.with_session(ExecutionMode::Regular, |session| {
@@ -208,7 +208,7 @@ pub trait BrocadeClient: std::fmt::Debug {
/// Configures a set of interfaces to be operated with a specified mode (access ports, ISL, etc.).
async fn configure_interfaces(
&self,
interfaces: Vec<(String, PortOperatingMode)>,
interfaces: &Vec<(String, PortOperatingMode)>,
) -> Result<(), Error>;
/// Scans the existing configuration to find the next available (unused)

View File

@@ -187,7 +187,7 @@ impl BrocadeClient for NetworkOperatingSystemClient {
async fn configure_interfaces(
&self,
interfaces: Vec<(String, PortOperatingMode)>,
interfaces: &Vec<(String, PortOperatingMode)>,
) -> Result<(), Error> {
info!("[Brocade] Configuring {} interface(s)...", interfaces.len());
@@ -204,9 +204,12 @@ impl BrocadeClient for NetworkOperatingSystemClient {
PortOperatingMode::Trunk => {
commands.push("switchport".into());
commands.push("switchport mode trunk".into());
commands.push("no spanning-tree shutdown".into());
commands.push("switchport trunk allowed vlan all".into());
commands.push("no switchport trunk tag native-vlan".into());
commands.push("spanning-tree shutdown".into());
commands.push("no fabric isl enable".into());
commands.push("no fabric trunk enable".into());
commands.push("no shutdown".into());
}
PortOperatingMode::Access => {
commands.push("switchport".into());

View File

@@ -16,7 +16,6 @@ use tokio::time::timeout;
#[derive(Debug)]
pub struct BrocadeShell {
ip: IpAddr,
port: u16,
username: String,
password: String,
options: BrocadeOptions,
@@ -27,33 +26,31 @@ pub struct BrocadeShell {
impl BrocadeShell {
pub async fn init(
ip_addresses: &[IpAddr],
port: u16,
username: &str,
password: &str,
options: Option<BrocadeOptions>,
options: BrocadeOptions,
) -> Result<Self, Error> {
let ip = ip_addresses
.first()
.ok_or_else(|| Error::ConfigurationError("No IP addresses provided".to_string()))?;
let base_options = options.unwrap_or_default();
let options = ssh::try_init_client(username, password, ip, base_options).await?;
let brocade_ssh_client_options =
ssh::try_init_client(username, password, ip, options).await?;
Ok(Self {
ip: *ip,
port,
username: username.to_string(),
password: password.to_string(),
before_all_commands: vec![],
after_all_commands: vec![],
options,
options: brocade_ssh_client_options,
})
}
pub async fn open_session(&self, mode: ExecutionMode) -> Result<BrocadeSession, Error> {
BrocadeSession::open(
self.ip,
self.port,
self.options.ssh.port,
&self.username,
&self.password,
self.options.clone(),

View File

@@ -2,6 +2,7 @@ use std::borrow::Cow;
use std::sync::Arc;
use async_trait::async_trait;
use log::debug;
use russh::client::Handler;
use russh::kex::DH_G1_SHA1;
use russh::kex::ECDH_SHA2_NISTP256;
@@ -10,29 +11,43 @@ use russh_keys::key::SSH_RSA;
use super::BrocadeOptions;
use super::Error;
#[derive(Default, Clone, Debug)]
#[derive(Clone, Debug)]
pub struct SshOptions {
pub preferred_algorithms: russh::Preferred,
pub port: u16,
}
impl Default for SshOptions {
fn default() -> Self {
Self {
preferred_algorithms: Default::default(),
port: 22,
}
}
}
impl SshOptions {
fn ecdhsa_sha2_nistp256() -> Self {
fn ecdhsa_sha2_nistp256(port: u16) -> Self {
Self {
preferred_algorithms: russh::Preferred {
kex: Cow::Borrowed(&[ECDH_SHA2_NISTP256]),
key: Cow::Borrowed(&[SSH_RSA]),
..Default::default()
},
port,
..Default::default()
}
}
fn legacy() -> Self {
fn legacy(port: u16) -> Self {
Self {
preferred_algorithms: russh::Preferred {
kex: Cow::Borrowed(&[DH_G1_SHA1]),
key: Cow::Borrowed(&[SSH_RSA]),
..Default::default()
},
port,
..Default::default()
}
}
}
@@ -57,18 +72,21 @@ pub async fn try_init_client(
ip: &std::net::IpAddr,
base_options: BrocadeOptions,
) -> Result<BrocadeOptions, Error> {
let mut default = SshOptions::default();
default.port = base_options.ssh.port;
let ssh_options = vec![
SshOptions::default(),
SshOptions::ecdhsa_sha2_nistp256(),
SshOptions::legacy(),
default,
SshOptions::ecdhsa_sha2_nistp256(base_options.ssh.port),
SshOptions::legacy(base_options.ssh.port),
];
for ssh in ssh_options {
let opts = BrocadeOptions {
ssh,
ssh: ssh.clone(),
..base_options.clone()
};
let client = create_client(*ip, 22, username, password, &opts).await;
debug!("Creating client {ip}:{} {username}", ssh.port);
let client = create_client(*ip, ssh.port, username, password, &opts).await;
match client {
Ok(_) => {

Binary file not shown.

View File

@@ -1,33 +1 @@
# Harmony Documentation Hub
Welcome to the Harmony documentation. This is the main entry point for learning everything from core concepts to building your own Score, Topologies, and Capabilities.
## 1. Getting Started
If you're new to Harmony, start here:
- [**Getting Started Guide**](./guides/getting-started.md): A step-by-step tutorial that takes you from an empty project to deploying your first application.
- [**Core Concepts**](./concepts.md): A high-level overview of the key concepts in Harmony: `Score`, `Topology`, `Capability`, `Inventory`, `Interpret`, ...
## 2. Use Cases & Examples
See how to use Harmony to solve real-world problems.
- [**OKD on Bare Metal**](./use-cases/okd-on-bare-metal.md): A detailed walkthrough of bootstrapping a high-availability OKD cluster from physical hardware.
- [**Deploy a Rust Web App**](./use-cases/deploy-rust-webapp.md): A quick guide to deploying a monitored, containerized web application to a Kubernetes cluster.
## 3. Component Catalogs
Discover existing, reusable components you can use in your Harmony projects.
- [**Scores Catalog**](./catalogs/scores.md): A categorized list of all available `Scores` (the "what").
- [**Topologies Catalog**](./catalogs/topologies.md): A list of all available `Topologies` (the "where").
- [**Capabilities Catalog**](./catalogs/capabilities.md): A list of all available `Capabilities` (the "how").
## 4. Developer Guides
Ready to build your own components? These guides show you how.
- [**Writing a Score**](./guides/writing-a-score.md): Learn how to create your own `Score` and `Interpret` logic to define a new desired state.
- [**Writing a Topology**](./guides/writing-a-topology.md): Learn how to model a new environment (like AWS, GCP, or custom hardware) as a `Topology`.
- [**Adding Capabilities**](./guides/adding-capabilities.md): See how to add a `Capability` to your custom `Topology`.
Not much here yet, see the `adr` folder for now. More to come in time!

View File

@@ -1,7 +0,0 @@
# Component Catalogs
This section is the "dictionary" for Harmony. It lists all the reusable components available out-of-the-box.
- [**Scores Catalog**](./scores.md): Discover all available `Scores` (the "what").
- [**Topologies Catalog**](./topologies.md): A list of all available `Topologies` (the "where").
- [**Capabilities Catalog**](./capabilities.md): A list of all available `Capabilities` (the "how").

View File

@@ -1,40 +0,0 @@
# Capabilities Catalog
A `Capability` is a specific feature or API that a `Topology` offers. `Interpret` logic uses these capabilities to execute a `Score`.
This list is primarily for developers **writing new Topologies or Scores**. As a user, you just need to know that the `Topology` you pick (like `K8sAnywhereTopology`) provides the capabilities your `Scores` (like `ApplicationScore`) need.
<!--toc:start-->
- [Capabilities Catalog](#capabilities-catalog)
- [Kubernetes & Application](#kubernetes-application)
- [Monitoring & Observability](#monitoring-observability)
- [Networking (Core Services)](#networking-core-services)
- [Networking (Hardware & Host)](#networking-hardware-host)
<!--toc:end-->
## Kubernetes & Application
- **K8sClient**: Provides an authenticated client to interact with a Kubernetes API (create/read/update/delete resources).
- **HelmCommand**: Provides the ability to execute Helm commands (install, upgrade, template).
- **TenantManager**: Provides methods for managing tenants in a multi-tenant cluster.
- **Ingress**: Provides an interface for managing ingress controllers and resources.
## Monitoring & Observability
- **Grafana**: Provides an API for configuring Grafana (datasources, dashboards).
- **Monitoring**: A general capability for configuring monitoring (e.g., creating Prometheus rules).
## Networking (Core Services)
- **DnsServer**: Provides an interface for creating and managing DNS records.
- **LoadBalancer**: Provides an interface for configuring a load balancer (e.g., OPNsense, MetalLB).
- **DhcpServer**: Provides an interface for managing DHCP leases and host bindings.
- **TftpServer**: Provides an interface for managing files on a TFTP server (e.g., iPXE boot files).
## Networking (Hardware & Host)
- **Router**: Provides an interface for configuring routing rules, typically on a firewall like OPNsense.
- **Switch**: Provides an interface for configuring a physical network switch (e.g., managing VLANs and port channels).
- **NetworkManager**: Provides an interface for configuring host-level networking (e.g., creating bonds and bridges on a node).

View File

@@ -1,102 +0,0 @@
# Scores Catalog
A `Score` is a declarative description of a desired state. Find the Score you need and add it to your `harmony!` block's `scores` array.
<!--toc:start-->
- [Scores Catalog](#scores-catalog)
- [Application Deployment](#application-deployment)
- [OKD / Kubernetes Cluster Setup](#okd-kubernetes-cluster-setup)
- [Cluster Services & Management](#cluster-services-management)
- [Monitoring & Alerting](#monitoring-alerting)
- [Infrastructure & Networking (Bare Metal)](#infrastructure-networking-bare-metal)
- [Infrastructure & Networking (Cluster)](#infrastructure-networking-cluster)
- [Tenant Management](#tenant-management)
- [Utility](#utility)
<!--toc:end-->
## Application Deployment
Scores for deploying and managing end-user applications.
- **ApplicationScore**: The primary score for deploying a web application. Describes the application, its framework, and the features it requires (e.g., monitoring, CI/CD).
- **HelmChartScore**: Deploys a generic Helm chart to a Kubernetes cluster.
- **ArgoHelmScore**: Deploys an application using an ArgoCD Helm chart.
- **LAMPScore**: A specialized score for deploying a classic LAMP (Linux, Apache, MySQL, PHP) stack.
## OKD / Kubernetes Cluster Setup
This collection of Scores is used to provision an entire OKD cluster from bare metal. They are typically used in order.
- **OKDSetup01InventoryScore**: Discovers and catalogs the physical hardware.
- **OKDSetup02BootstrapScore**: Configures the bootstrap node, renders iPXE files, and kicks off the SCOS installation.
- **OKDSetup03ControlPlaneScore**: Renders iPXE configurations for the control plane nodes.
- **OKDSetupPersistNetworkBondScore**: Configures network bonds on the nodes and port channels on the switches.
- **OKDSetup04WorkersScore**: Renders iPXE configurations for the worker nodes.
- **OKDSetup06InstallationReportScore**: Runs post-installation checks and generates a report.
- **OKDUpgradeScore**: Manages the upgrade process for an existing OKD cluster.
## Cluster Services & Management
Scores for installing and managing services _inside_ a Kubernetes cluster.
- **K3DInstallationScore**: Installs and configes a local K3D (k3s-in-docker) cluster. Used by `K8sAnywhereTopology`.
- **CertManagerHelmScore**: Deploys the `cert-manager` Helm chart.
- **ClusterIssuerScore**: Configures a `ClusterIssuer` for `cert-manager`, (e.g., for Let's Encrypt).
- **K8sNamespaceScore**: Ensures a Kubernetes namespace exists.
- **K8sDeploymentScore**: Deploys a generic `Deployment` resource to Kubernetes.
- **K8sIngressScore**: Configures an `Ingress` resource for a service.
## Monitoring & Alerting
Scores for configuring observability, dashboards, and alerts.
- **ApplicationMonitoringScore**: A generic score to set up monitoring for an application.
- **ApplicationRHOBMonitoringScore**: A specialized score for setting up monitoring via the Red Hat Observability stack.
- **HelmPrometheusAlertingScore**: Configures Prometheus alerts via a Helm chart.
- **K8sPrometheusCRDAlertingScore**: Configures Prometheus alerts using the `PrometheusRule` CRD.
- **PrometheusAlertScore**: A generic score for creating a Prometheus alert.
- **RHOBAlertingScore**: Configures alerts specifically for the Red Hat Observability stack.
- **NtfyScore**: Configures alerts to be sent to a `ntfy.sh` server.
## Infrastructure & Networking (Bare Metal)
Low-level scores for managing physical hardware and network services.
- **DhcpScore**: Configures a DHCP server.
- **OKDDhcpScore**: A specialized DHCP configuration for the OKD bootstrap process.
- **OKDBootstrapDhcpScore**: Configures DHCP specifically for the bootstrap node.
- **DhcpHostBindingScore**: Creates a specific MAC-to-IP binding in the DHCP server.
- **DnsScore**: Configures a DNS server.
- **OKDDnsScore**: A specialized DNS configuration for the OKD cluster (e.g., `api.*`, `*.apps.*`).
- **StaticFilesHttpScore**: Serves a directory of static files (e.g., a documentation site) over HTTP.
- **TftpScore**: Configures a TFTP server, typically for serving iPXE boot files.
- **IPxeMacBootFileScore**: Assigns a specific iPXE boot file to a MAC address in the TFTP server.
- **OKDIpxeScore**: A specialized score for generating the iPXE boot scripts for OKD.
- **OPNsenseShellCommandScore**: Executes a shell command on an OPNsense firewall.
## Infrastructure & Networking (Cluster)
Network services that run inside the cluster or as part of the topology.
- **LoadBalancerScore**: Configures a general-purpose load balancer.
- **OKDLoadBalancerScore**: Configures the high-availability load balancers for the OKD API and ingress.
- **OKDBootstrapLoadBalancerScore**: Configures the load balancer specifically for the bootstrap-time API endpoint.
- **K8sIngressScore**: Configures an Ingress controller or resource.
- [HighAvailabilityHostNetworkScore](../../harmony/src/modules/okd/host_network.rs): Configures network bonds on a host and the corresponding port-channels on the switch stack for high-availability.
## Tenant Management
Scores for managing multi-tenancy within a cluster.
- **TenantScore**: Creates a new tenant (e.g., a namespace, quotas, network policies).
- **TenantCredentialScore**: Generates and provisions credentials for a new tenant.
## Utility
Helper scores for discovery and inspection.
- **LaunchDiscoverInventoryAgentScore**: Launches the agent responsible for the `OKDSetup01InventoryScore`.
- **DiscoverHostForRoleScore**: A utility score to find a host matching a specific role in the inventory.
- **InspectInventoryScore**: Dumps the discovered inventory for inspection.

View File

@@ -1,59 +0,0 @@
# Topologies Catalog
A `Topology` is the logical representation of your infrastructure and its `Capabilities`. You select a `Topology` in your Harmony project to define _where_ your `Scores` will be applied.
<!--toc:start-->
- [Topologies Catalog](#topologies-catalog)
- [HAClusterTopology](#haclustertopology)
- [K8sAnywhereTopology](#k8sanywheretopology)
<!--toc:end-->
### HAClusterTopology
- **`HAClusterTopology::autoload()`**
This `Topology` represents a high-availability, bare-metal cluster. It is designed for production-grade deployments like OKD.
It models an environment consisting of:
- At least 3 cluster nodes (for control plane/workers)
- 2 redundant firewalls (e.g., OPNsense)
- 2 redundant network switches
**Provided Capabilities:**
This topology provides a rich set of capabilities required for bare-metal provisioning and cluster management, including:
- `K8sClient` (once the cluster is bootstrapped)
- `DnsServer`
- `LoadBalancer`
- `DhcpServer`
- `TftpServer`
- `Router` (via the firewalls)
- `Switch`
- `NetworkManager` (for host-level network config)
---
### K8sAnywhereTopology
- **`K8sAnywhereTopology::from_env()`**
This `Topology` is designed for development and application deployment. It provides a simple, abstract way to deploy to _any_ Kubernetes cluster.
**How it works:**
1. By default (`from_env()` with no env vars), it automatically provisions a **local K3D (k3s-in-docker) cluster** on your machine. This is perfect for local development and testing.
2. If you provide a `KUBECONFIG` environment variable, it will instead connect to that **existing Kubernetes cluster** (e.g., your staging or production OKD cluster).
This allows you to use the _exact same code_ to deploy your application locally as you do to deploy it to production.
**Provided Capabilities:**
- `K8sClient`
- `HelmCommand`
- `TenantManager`
- `Ingress`
- `Monitoring`
- ...and more.

View File

@@ -1,40 +0,0 @@
# Core Concepts
Harmony's design is based on a few key concepts. Understanding them is the key to unlocking the framework's power.
### 1. Score
- **What it is:** A **Score** is a declarative description of a desired state. It's a "resource" that defines _what_ you want to achieve, not _how_ to do it.
- **Example:** `ApplicationScore` declares "I want this web application to be running and monitored."
### 2. Topology
- **What it is:** A **Topology** is the logical representation of your infrastructure and its abilities. It's the "where" your Scores will be applied.
- **Key Job:** A Topology's most important job is to expose which `Capabilities` it supports.
- **Example:** `HAClusterTopology` represents a bare-metal cluster and exposes `Capabilities` like `NetworkManager` and `Switch`. `K8sAnywhereTopology` represents a Kubernetes cluster and exposes the `K8sClient` `Capability`.
### 3. Capability
- **What it is:** A **Capability** is a specific feature or API that a `Topology` offers. It's the "how" a `Topology` can fulfill a `Score`'s request.
- **Example:** The `K8sClient` capability offers a way to interact with a Kubernetes API. The `Switch` capability offers a way to configure a physical network switch.
### 4. Interpret
- **What it is:** An **Interpret** is the execution logic that makes a `Score` a reality. It's the "glue" that connects the _desired state_ (`Score`) to the _environment's abilities_ (`Topology`'s `Capabilities`).
- **How it works:** When you apply a `Score`, Harmony finds the matching `Interpret` for your `Topology`. This `Interpret` then uses the `Capabilities` provided by the `Topology` to execute the necessary steps.
### 5. Inventory
- **What it is:** An **Inventory** is the physical material (the "what") used in a cluster. This is most relevant for bare-metal or on-premise topologies.
- **Example:** A list of nodes with their roles (control plane, worker), CPU, RAM, and network interfaces. For the `K8sAnywhereTopology`, the inventory might be empty or autoloaded, as the infrastructure is more abstract.
---
### How They Work Together (The Compile-Time Check)
1. You **write a `Score`** (e.g., `ApplicationScore`).
2. Your `Score`'s `Interpret` logic requires certain **`Capabilities`** (e.g., `K8sClient` and `Ingress`).
3. You choose a **`Topology`** to run it on (e.g., `HAClusterTopology`).
4. **At compile-time**, Harmony checks: "Does `HAClusterTopology` provide the `K8sClient` and `Ingress` capabilities that `ApplicationScore` needs?"
- **If Yes:** Your code compiles. You can be confident it will run.
- **If No:** The compiler gives you an error. You've just prevented a "config-is-valid-but-platform-is-wrong" runtime error before you even deployed.

View File

@@ -1,42 +0,0 @@
# Getting Started Guide
Welcome to Harmony! This guide will walk you through installing the Harmony framework, setting up a new project, and deploying your first application.
We will build and deploy the "Rust Web App" example, which automatically:
1. Provisions a local K3D (Kubernetes in Docker) cluster.
2. Deploys a sample Rust web application.
3. Sets up monitoring for the application.
## Prerequisites
Before you begin, you'll need a few tools installed on your system:
- **Rust & Cargo:** [Install Rust](https://www.rust-lang.org/tools/install)
- **Docker:** [Install Docker](https://docs.docker.com/get-docker/) (Required for the K3D local cluster)
- **kubectl:** [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) (For inspecting the cluster)
## 1. Install Harmony
First, clone the Harmony repository and build the project. This gives you the `harmony` CLI and all the core libraries.
```bash
# Clone the main repository
git clone https://git.nationtech.io/nationtech/harmony
cd harmony
# Build the project (this may take a few minutes)
cargo build --release
```
...
## Next Steps
Congratulations, you've just deployed an application using true infrastructure-as-code!
From here, you can:
- [Explore the Catalogs](../catalogs/README.md): See what other [Scores](../catalogs/scores.md) and [Topologies](../catalogs/topologies.md) are available.
- [Read the Use Cases](../use-cases/README.md): Check out the [OKD on Bare Metal](./use-cases/okd-on-bare-metal.md) guide for a more advanced scenario.
- [Write your own Score](../guides/writing-a-score.md): Dive into the [Developer Guide](./guides/developer-guide.md) to start building your own components.

BIN
empty_database.sqlite Normal file

Binary file not shown.

View File

@@ -0,0 +1,19 @@
[package]
name = "brocade-switch"
edition = "2024"
version.workspace = true
readme.workspace = true
license.workspace = true
[dependencies]
harmony = { path = "../../harmony" }
harmony_cli = { path = "../../harmony_cli" }
harmony_macros = { path = "../../harmony_macros" }
harmony_types = { path = "../../harmony_types" }
tokio.workspace = true
url.workspace = true
async-trait.workspace = true
serde.workspace = true
log.workspace = true
env_logger.workspace = true
brocade = { path = "../../brocade" }

View File

@@ -0,0 +1,157 @@
use std::str::FromStr;
use async_trait::async_trait;
use brocade::{BrocadeOptions, PortOperatingMode};
use harmony::{
data::Version,
infra::brocade::BrocadeSwitchClient,
interpret::{Interpret, InterpretError, InterpretName, InterpretStatus, Outcome},
inventory::Inventory,
score::Score,
topology::{
HostNetworkConfig, PortConfig, PreparationError, PreparationOutcome, Switch, SwitchClient,
SwitchError, Topology,
},
};
use harmony_macros::ip;
use harmony_types::{id::Id, net::MacAddress, switch::PortLocation};
use log::{debug, info};
use serde::Serialize;
#[tokio::main]
async fn main() {
let switch_score = BrocadeSwitchScore {
port_channels_to_clear: vec![
Id::from_str("17").unwrap(),
Id::from_str("19").unwrap(),
Id::from_str("18").unwrap(),
],
ports_to_configure: vec![
(PortLocation(2, 0, 17), PortOperatingMode::Trunk),
(PortLocation(2, 0, 19), PortOperatingMode::Trunk),
(PortLocation(1, 0, 18), PortOperatingMode::Trunk),
],
};
harmony_cli::run(
Inventory::autoload(),
SwitchTopology::new().await,
vec![Box::new(switch_score)],
None,
)
.await
.unwrap();
}
#[derive(Clone, Debug, Serialize)]
struct BrocadeSwitchScore {
port_channels_to_clear: Vec<Id>,
ports_to_configure: Vec<PortConfig>,
}
impl<T: Topology + Switch> Score<T> for BrocadeSwitchScore {
fn name(&self) -> String {
"BrocadeSwitchScore".to_string()
}
#[doc(hidden)]
fn create_interpret(&self) -> Box<dyn Interpret<T>> {
Box::new(BrocadeSwitchInterpret {
score: self.clone(),
})
}
}
#[derive(Debug)]
struct BrocadeSwitchInterpret {
score: BrocadeSwitchScore,
}
#[async_trait]
impl<T: Topology + Switch> Interpret<T> for BrocadeSwitchInterpret {
async fn execute(
&self,
_inventory: &Inventory,
topology: &T,
) -> Result<Outcome, InterpretError> {
info!("Applying switch configuration {:?}", self.score);
debug!(
"Clearing port channel {:?}",
self.score.port_channels_to_clear
);
topology
.clear_port_channel(&self.score.port_channels_to_clear)
.await
.map_err(|e| InterpretError::new(e.to_string()))?;
debug!("Configuring interfaces {:?}", self.score.ports_to_configure);
topology
.configure_interface(&self.score.ports_to_configure)
.await
.map_err(|e| InterpretError::new(e.to_string()))?;
Ok(Outcome::success("switch configured".to_string()))
}
fn get_name(&self) -> InterpretName {
InterpretName::Custom("BrocadeSwitchInterpret")
}
fn get_version(&self) -> Version {
todo!()
}
fn get_status(&self) -> InterpretStatus {
todo!()
}
fn get_children(&self) -> Vec<Id> {
todo!()
}
}
struct SwitchTopology {
client: Box<dyn SwitchClient>,
}
#[async_trait]
impl Topology for SwitchTopology {
fn name(&self) -> &str {
"SwitchTopology"
}
async fn ensure_ready(&self) -> Result<PreparationOutcome, PreparationError> {
Ok(PreparationOutcome::Noop)
}
}
impl SwitchTopology {
async fn new() -> Self {
let mut options = BrocadeOptions::default();
options.ssh.port = 2222;
let client =
BrocadeSwitchClient::init(&vec![ip!("127.0.0.1")], &"admin", &"password", options)
.await
.expect("Failed to connect to switch");
let client = Box::new(client);
Self { client }
}
}
#[async_trait]
impl Switch for SwitchTopology {
async fn setup_switch(&self) -> Result<(), SwitchError> {
todo!()
}
async fn get_port_for_mac_address(
&self,
_mac_address: &MacAddress,
) -> Result<Option<PortLocation>, SwitchError> {
todo!()
}
async fn configure_port_channel(&self, _config: &HostNetworkConfig) -> Result<(), SwitchError> {
todo!()
}
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError> {
self.client.clear_port_channel(ids).await
}
async fn configure_interface(&self, ports: &Vec<PortConfig>) -> Result<(), SwitchError> {
self.client.configure_interface(ports).await
}
}

View File

@@ -2,7 +2,7 @@ use harmony::{
inventory::Inventory,
modules::{
dummy::{ErrorScore, PanicScore, SuccessScore},
inventory::LaunchDiscoverInventoryAgentScore,
inventory::{HarmonyDiscoveryStrategy, LaunchDiscoverInventoryAgentScore},
},
topology::LocalhostTopology,
};
@@ -18,6 +18,7 @@ async fn main() {
Box::new(PanicScore {}),
Box::new(LaunchDiscoverInventoryAgentScore {
discovery_timeout: Some(10),
discovery_strategy: HarmonyDiscoveryStrategy::MDNS,
}),
],
None,

View File

@@ -0,0 +1,15 @@
[package]
name = "harmony_inventory_builder"
edition = "2024"
version.workspace = true
readme.workspace = true
license.workspace = true
[dependencies]
harmony = { path = "../../harmony" }
harmony_cli = { path = "../../harmony_cli" }
harmony_macros = { path = "../../harmony_macros" }
harmony_types = { path = "../../harmony_types" }
tokio.workspace = true
url.workspace = true
cidr.workspace = true

View File

@@ -0,0 +1,11 @@
cargo build -p harmony_inventory_builder --release --target x86_64-unknown-linux-musl
SCRIPT_DIR="$(dirname ${0})"
cd "${SCRIPT_DIR}/docker/"
cp ../../../target/x86_64-unknown-linux-musl/release/harmony_inventory_builder .
docker build . -t hub.nationtech.io/harmony/harmony_inventory_builder
docker push hub.nationtech.io/harmony/harmony_inventory_builder

View File

@@ -0,0 +1,10 @@
FROM debian:12-slim
RUN mkdir /app
WORKDIR /app/
COPY harmony_inventory_builder /app/
ENV RUST_LOG=info
CMD ["sleep", "infinity"]

View File

@@ -0,0 +1,36 @@
use harmony::{
inventory::{HostRole, Inventory},
modules::inventory::{DiscoverHostForRoleScore, HarmonyDiscoveryStrategy},
topology::LocalhostTopology,
};
use harmony_macros::cidrv4;
#[tokio::main]
async fn main() {
let discover_worker = DiscoverHostForRoleScore {
role: HostRole::Worker,
number_desired_hosts: 3,
discovery_strategy: HarmonyDiscoveryStrategy::SUBNET {
cidr: cidrv4!("192.168.0.1/25"),
port: 25000,
},
};
let discover_control_plane = DiscoverHostForRoleScore {
role: HostRole::ControlPlane,
number_desired_hosts: 3,
discovery_strategy: HarmonyDiscoveryStrategy::SUBNET {
cidr: cidrv4!("192.168.0.1/25"),
port: 25000,
},
};
harmony_cli::run(
Inventory::autoload(),
LocalhostTopology::new(),
vec![Box::new(discover_worker), Box::new(discover_control_plane)],
None,
)
.await
.unwrap();
}

View File

@@ -39,10 +39,10 @@ async fn main() {
.expect("Failed to get credentials");
let switches: Vec<IpAddr> = vec![ip!("192.168.33.101")];
let brocade_options = Some(BrocadeOptions {
let brocade_options = BrocadeOptions {
dry_run: *harmony::config::DRY_RUN,
..Default::default()
});
};
let switch_client = BrocadeSwitchClient::init(
&switches,
&switch_auth.username,

View File

@@ -31,10 +31,10 @@ pub async fn get_topology() -> HAClusterTopology {
.expect("Failed to get credentials");
let switches: Vec<IpAddr> = vec![ip!("192.168.1.101")]; // TODO: Adjust me
let brocade_options = Some(BrocadeOptions {
let brocade_options = BrocadeOptions {
dry_run: *harmony::config::DRY_RUN,
..Default::default()
});
};
let switch_client = BrocadeSwitchClient::init(
&switches,
&switch_auth.username,

View File

@@ -26,10 +26,10 @@ pub async fn get_topology() -> HAClusterTopology {
.expect("Failed to get credentials");
let switches: Vec<IpAddr> = vec![ip!("192.168.1.101")]; // TODO: Adjust me
let brocade_options = Some(BrocadeOptions {
let brocade_options = BrocadeOptions {
dry_run: *harmony::config::DRY_RUN,
..Default::default()
});
};
let switch_client = BrocadeSwitchClient::init(
&switches,
&switch_auth.username,

View File

@@ -35,10 +35,10 @@ async fn main() {
.expect("Failed to get credentials");
let switches: Vec<IpAddr> = vec![ip!("192.168.5.101")]; // TODO: Adjust me
let brocade_options = Some(BrocadeOptions {
let brocade_options = BrocadeOptions {
dry_run: *harmony::config::DRY_RUN,
..Default::default()
});
};
let switch_client = BrocadeSwitchClient::init(
&switches,
&switch_auth.username,

View File

@@ -5,10 +5,6 @@ version.workspace = true
readme.workspace = true
license.workspace = true
[[example]]
name = "try_rust_webapp"
path = "src/main.rs"
[dependencies]
harmony = { path = "../../harmony" }
harmony_cli = { path = "../../harmony_cli" }

View File

@@ -152,10 +152,10 @@ impl PhysicalHost {
pub fn parts_list(&self) -> String {
let PhysicalHost {
id,
category,
category: _,
network,
storage,
labels,
labels: _,
memory_modules,
cpus,
} = self;
@@ -226,8 +226,8 @@ impl PhysicalHost {
speed_mhz,
manufacturer,
part_number,
serial_number,
rank,
serial_number: _,
rank: _,
} = mem;
parts_list.push_str(&format!(
"\n{}Gb, {}Mhz, Manufacturer ({}), Part Number ({})",

View File

@@ -4,6 +4,8 @@ use std::error::Error;
use async_trait::async_trait;
use derive_new::new;
use crate::inventory::HostRole;
use super::{
data::Version, executors::ExecutorError, inventory::Inventory, topology::PreparationError,
};

View File

@@ -1,4 +1,5 @@
use async_trait::async_trait;
use brocade::PortOperatingMode;
use harmony_macros::ip;
use harmony_types::{
id::Id,
@@ -8,7 +9,7 @@ use harmony_types::{
use log::debug;
use log::info;
use crate::infra::network_manager::OpenShiftNmStateNetworkManager;
use crate::{infra::network_manager::OpenShiftNmStateNetworkManager, topology::PortConfig};
use crate::topology::PxeOptions;
use crate::{data::FileContent, executors::ExecutorError};
@@ -298,6 +299,16 @@ impl Switch for HAClusterTopology {
Ok(())
}
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError> {
todo!()
}
async fn configure_interface(
&self,
ports: &Vec<PortConfig>,
) -> Result<(), SwitchError> {
todo!()
}
}
#[async_trait]
@@ -521,4 +532,6 @@ impl SwitchClient for DummyInfra {
) -> Result<u8, SwitchError> {
unimplemented!("{}", UNIMPLEMENTED_DUMMY_INFRA)
}
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError> {todo!()}
async fn configure_interface(&self, ports: &Vec<PortConfig>) -> Result<(), SwitchError> {todo!()}
}

View File

@@ -7,6 +7,7 @@ use std::{
};
use async_trait::async_trait;
use brocade::PortOperatingMode;
use derive_new::new;
use harmony_types::{
id::Id,
@@ -214,6 +215,8 @@ impl From<String> for NetworkError {
}
}
pub type PortConfig = (PortLocation, PortOperatingMode);
#[async_trait]
pub trait Switch: Send + Sync {
async fn setup_switch(&self) -> Result<(), SwitchError>;
@@ -224,6 +227,8 @@ pub trait Switch: Send + Sync {
) -> Result<Option<PortLocation>, SwitchError>;
async fn configure_port_channel(&self, config: &HostNetworkConfig) -> Result<(), SwitchError>;
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError>;
async fn configure_interface(&self, ports: &Vec<PortConfig>) -> Result<(), SwitchError>;
}
#[derive(Clone, Debug, PartialEq)]
@@ -283,6 +288,9 @@ pub trait SwitchClient: Debug + Send + Sync {
channel_name: &str,
switch_ports: Vec<PortLocation>,
) -> Result<u8, SwitchError>;
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError>;
async fn configure_interface(&self, ports: &Vec<PortConfig>) -> Result<(), SwitchError>;
}
#[cfg(test)]

View File

@@ -14,7 +14,7 @@ use k8s_openapi::{
},
apimachinery::pkg::util::intstr::IntOrString,
};
use kube::Resource;
use kube::{Resource, api::DynamicObject};
use log::debug;
use serde::de::DeserializeOwned;
use serde_json::json;

View File

@@ -1,12 +1,13 @@
use async_trait::async_trait;
use brocade::{BrocadeClient, BrocadeOptions, InterSwitchLink, InterfaceStatus, PortOperatingMode};
use harmony_types::{
id::Id,
net::{IpAddress, MacAddress},
switch::{PortDeclaration, PortLocation},
};
use option_ext::OptionExt;
use crate::topology::{SwitchClient, SwitchError};
use crate::topology::{PortConfig, SwitchClient, SwitchError};
#[derive(Debug)]
pub struct BrocadeSwitchClient {
@@ -18,9 +19,9 @@ impl BrocadeSwitchClient {
ip_addresses: &[IpAddress],
username: &str,
password: &str,
options: Option<BrocadeOptions>,
options: BrocadeOptions,
) -> Result<Self, brocade::Error> {
let brocade = brocade::init(ip_addresses, 22, username, password, options).await?;
let brocade = brocade::init(ip_addresses, username, password, options).await?;
Ok(Self { brocade })
}
}
@@ -59,7 +60,7 @@ impl SwitchClient for BrocadeSwitchClient {
}
self.brocade
.configure_interfaces(interfaces)
.configure_interfaces(&interfaces)
.await
.map_err(|e| SwitchError::new(e.to_string()))?;
@@ -111,6 +112,24 @@ impl SwitchClient for BrocadeSwitchClient {
Ok(channel_id)
}
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError> {
for i in ids {
self.brocade
.clear_port_channel(&i.to_string())
.await
.map_err(|e| SwitchError::new(e.to_string()))?;
}
Ok(())
}
async fn configure_interface(&self, ports: &Vec<PortConfig>) -> Result<(), SwitchError> {
// FIXME hardcoded TenGigabitEthernet = bad
let ports = ports.iter().map(|p| (format!("TenGigabitEthernet {}", p.0), p.1.clone())).collect();
self.brocade
.configure_interfaces(&ports)
.await
.map_err(|e| SwitchError::new(e.to_string()))?;
Ok(())
}
}
#[cfg(test)]
@@ -147,8 +166,8 @@ mod tests {
let configured_interfaces = brocade.configured_interfaces.lock().unwrap();
assert_that!(*configured_interfaces).contains_exactly(vec![
(first_interface.name.clone(), PortOperatingMode::Access),
(second_interface.name.clone(), PortOperatingMode::Access),
(first_interface.port_location, PortOperatingMode::Access),
(second_interface.port_location, PortOperatingMode::Access),
]);
}
@@ -255,10 +274,10 @@ mod tests {
async fn configure_interfaces(
&self,
interfaces: Vec<(String, PortOperatingMode)>,
interfaces: &Vec<(String, PortOperatingMode)>,
) -> Result<(), Error> {
let mut configured_interfaces = self.configured_interfaces.lock().unwrap();
*configured_interfaces = interfaces;
*configured_interfaces = interfaces.clone();
Ok(())
}

View File

@@ -121,7 +121,7 @@ mod test {
#[test]
fn deployment_to_dynamic_roundtrip() {
// Create a sample Deployment with nested structures
let mut deployment = Deployment {
let deployment = Deployment {
metadata: ObjectMeta {
name: Some("my-deployment".to_string()),
labels: Some({

View File

@@ -135,6 +135,8 @@ impl OpenShiftNmStateNetworkManager {
description: Some(format!("Member of bond {bond_name}")),
r#type: nmstate::InterfaceType::Ethernet,
state: "up".to_string(),
mtu: Some(switch_port.interface.mtu),
mac_address: Some(switch_port.interface.mac_address.to_string()),
ipv4: Some(nmstate::IpStackSpec {
enabled: Some(false),
..Default::default()
@@ -160,7 +162,7 @@ impl OpenShiftNmStateNetworkManager {
interfaces.push(nmstate::Interface {
name: bond_name.to_string(),
description: Some(format!("HARMONY - Network bond for host {host}")),
description: Some(format!("Network bond for host {host}")),
r#type: nmstate::InterfaceType::Bond,
state: "up".to_string(),
copy_mac_from,

View File

@@ -8,7 +8,6 @@ mod tftp;
use std::sync::Arc;
pub use management::*;
use opnsense_config_xml::Host;
use tokio::sync::RwLock;
use crate::{executors::ExecutorError, topology::LogicalHost};

View File

@@ -19,8 +19,11 @@ pub struct DhcpScore {
pub host_binding: Vec<HostBinding>,
pub next_server: Option<IpAddress>,
pub boot_filename: Option<String>,
/// Boot filename to be provided to PXE clients identifying as BIOS
pub filename: Option<String>,
/// Boot filename to be provided to PXE clients identifying as uefi but NOT iPXE
pub filename64: Option<String>,
/// Boot filename to be provided to PXE clients identifying as iPXE
pub filenameipxe: Option<String>,
pub dhcp_range: (IpAddress, IpAddress),
pub domain: Option<String>,

View File

@@ -5,11 +5,10 @@ use serde::{Deserialize, Serialize};
use crate::{
data::Version,
hardware::PhysicalHost,
infra::inventory::InventoryRepositoryFactory,
interpret::{Interpret, InterpretError, InterpretName, InterpretStatus, Outcome},
inventory::{HostRole, Inventory},
modules::inventory::LaunchDiscoverInventoryAgentScore,
modules::inventory::{HarmonyDiscoveryStrategy, LaunchDiscoverInventoryAgentScore},
score::Score,
topology::Topology,
};
@@ -17,11 +16,13 @@ use crate::{
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DiscoverHostForRoleScore {
pub role: HostRole,
pub number_desired_hosts: i16,
pub discovery_strategy : HarmonyDiscoveryStrategy,
}
impl<T: Topology> Score<T> for DiscoverHostForRoleScore {
fn name(&self) -> String {
"DiscoverInventoryAgentScore".to_string()
format!("DiscoverHostForRoleScore({:?})", self.role)
}
fn create_interpret(&self) -> Box<dyn Interpret<T>> {
@@ -48,13 +49,15 @@ impl<T: Topology> Interpret<T> for DiscoverHostForRoleInterpret {
);
LaunchDiscoverInventoryAgentScore {
discovery_timeout: None,
discovery_strategy: self.score.discovery_strategy.clone(),
}
.interpret(inventory, topology)
.await?;
let host: PhysicalHost;
let mut chosen_hosts = vec![];
let host_repo = InventoryRepositoryFactory::build().await?;
let mut assigned_hosts = 0;
loop {
let all_hosts = host_repo.get_all_hosts().await?;
@@ -75,15 +78,24 @@ impl<T: Topology> Interpret<T> for DiscoverHostForRoleInterpret {
match ans {
Ok(choice) => {
info!(
"Selected {} as the {:?} node.",
choice.summary(),
self.score.role
"Assigned role {:?} for node {}",
self.score.role,
choice.summary()
);
host_repo
.save_role_mapping(&self.score.role, &choice)
.await?;
host = choice;
break;
chosen_hosts.push(choice);
assigned_hosts += 1;
info!(
"Found {assigned_hosts} hosts for role {:?}",
self.score.role
);
if assigned_hosts == self.score.number_desired_hosts {
break;
}
}
Err(inquire::InquireError::OperationCanceled) => {
info!("Refresh requested. Fetching list of discovered hosts again...");
@@ -100,8 +112,13 @@ impl<T: Topology> Interpret<T> for DiscoverHostForRoleInterpret {
}
Ok(Outcome::success(format!(
"Successfully discovered host {} for role {:?}",
host.summary(),
"Successfully discovered {} hosts {} for role {:?}",
self.score.number_desired_hosts,
chosen_hosts
.iter()
.map(|h| h.summary())
.collect::<Vec<String>>()
.join(", "),
self.score.role
)))
}

View File

@@ -1,6 +1,10 @@
mod discovery;
pub mod inspect;
use std::net::Ipv4Addr;
use cidr::{Ipv4Cidr, Ipv4Inet};
pub use discovery::*;
use tokio::time::{Duration, timeout};
use async_trait::async_trait;
use harmony_inventory_agent::local_presence::DiscoveryEvent;
@@ -24,6 +28,7 @@ use harmony_types::id::Id;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct LaunchDiscoverInventoryAgentScore {
pub discovery_timeout: Option<u64>,
pub discovery_strategy: HarmonyDiscoveryStrategy,
}
impl<T: Topology> Score<T> for LaunchDiscoverInventoryAgentScore {
@@ -43,6 +48,12 @@ struct DiscoverInventoryAgentInterpret {
score: LaunchDiscoverInventoryAgentScore,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum HarmonyDiscoveryStrategy {
MDNS,
SUBNET { cidr: cidr::Ipv4Cidr, port: u16 },
}
#[async_trait]
impl<T: Topology> Interpret<T> for DiscoverInventoryAgentInterpret {
async fn execute(
@@ -57,6 +68,37 @@ impl<T: Topology> Interpret<T> for DiscoverInventoryAgentInterpret {
),
};
match self.score.discovery_strategy {
HarmonyDiscoveryStrategy::MDNS => self.launch_mdns_discovery().await,
HarmonyDiscoveryStrategy::SUBNET { cidr, port } => {
self.launch_cidr_discovery(&cidr, port).await
}
};
Ok(Outcome::success(
"Discovery process completed successfully".to_string(),
))
}
fn get_name(&self) -> InterpretName {
InterpretName::DiscoverInventoryAgent
}
fn get_version(&self) -> Version {
todo!()
}
fn get_status(&self) -> InterpretStatus {
todo!()
}
fn get_children(&self) -> Vec<Id> {
todo!()
}
}
impl DiscoverInventoryAgentInterpret {
async fn launch_mdns_discovery(&self) {
harmony_inventory_agent::local_presence::discover_agents(
self.score.discovery_timeout,
|event: DiscoveryEvent| -> Result<(), String> {
@@ -112,6 +154,8 @@ impl<T: Topology> Interpret<T> for DiscoverInventoryAgentInterpret {
cpus,
};
// FIXME only save the host when it is new or something changed in it.
// we currently are saving the host every time it is discovered.
let repo = InventoryRepositoryFactory::build()
.await
.map_err(|e| format!("Could not build repository : {e}"))
@@ -132,25 +176,111 @@ impl<T: Topology> Interpret<T> for DiscoverInventoryAgentInterpret {
Ok(())
},
)
.await;
Ok(Outcome::success(
"Discovery process completed successfully".to_string(),
))
.await
}
fn get_name(&self) -> InterpretName {
InterpretName::DiscoverInventoryAgent
}
// async fn launch_cidr_discovery(&self, cidr : &Ipv4Cidr, port: u16) {
// todo!("launnch cidr discovery for {cidr} : {port}
// - Iterate over all possible addresses in cidr
// - make calls in batches of 20 attempting to reach harmony inventory agent on <addr, port> using same as above harmony_inventory_agent::client::get_host_inventory(&address, port)
// - Log warn when response is 404, it means the port was used by something else unexpected
// - Log error when response is 5xx
// - Log debug when no response (timeout 15 seconds)
// - Log info when found and response is 2xx
// ");
// }
async fn launch_cidr_discovery(&self, cidr: &Ipv4Cidr, port: u16) {
let addrs: Vec<Ipv4Inet> = cidr.iter().collect();
let total = addrs.len();
info!(
"Starting CIDR discovery for {} hosts on {}/{} (port {})",
total,
cidr.network_length(),
cidr,
port
);
fn get_version(&self) -> Version {
todo!()
}
let batch_size: usize = 20;
let timeout_secs = 5;
let request_timeout = Duration::from_secs(timeout_secs);
fn get_status(&self) -> InterpretStatus {
todo!()
}
let mut current_batch = 0;
let num_batches = addrs.len() / batch_size;
fn get_children(&self) -> Vec<Id> {
todo!()
for batch in addrs.chunks(batch_size) {
current_batch += 1;
info!("Starting query batch {current_batch} of {num_batches}, timeout {timeout_secs}");
let mut tasks = Vec::with_capacity(batch.len());
for addr in batch {
let addr = addr.address().to_string();
let port = port;
let task = tokio::spawn(async move {
match timeout(
request_timeout,
harmony_inventory_agent::client::get_host_inventory(&addr, port),
)
.await
{
Ok(Ok(host)) => {
info!("Found and response is 2xx for {addr}:{port}");
// Reuse the same conversion to PhysicalHost as MDNS flow
let harmony_inventory_agent::hwinfo::PhysicalHost {
storage_drives,
storage_controller,
memory_modules,
cpus,
chipset,
network_interfaces,
management_interface,
host_uuid,
} = host;
let host = PhysicalHost {
id: Id::from(host_uuid),
category: HostCategory::Server,
network: network_interfaces,
storage: storage_drives,
labels: vec![Label {
name: "discovered-by".to_string(),
value: "harmony-inventory-agent".to_string(),
}],
memory_modules,
cpus,
};
// Save host to inventory
let repo = InventoryRepositoryFactory::build()
.await
.map_err(|e| format!("Could not build repository : {e}"))
.unwrap();
if let Err(e) = repo.save(&host).await {
log::debug!("Failed to save host {}: {e}", host.id);
} else {
info!("Saved host id {}, summary : {}", host.id, host.summary());
}
}
Ok(Err(e)) => {
log::info!("Error querying inventory agent on {addr}:{port} : {e}");
}
Err(_) => {
// Timeout for this host
log::debug!("No response (timeout) for {addr}:{port}");
}
}
});
tasks.push(task);
}
// Wait for this batch to complete
for t in tasks {
let _ = t.await;
}
}
info!("CIDR discovery completed");
}
}

View File

@@ -4,7 +4,7 @@ use crate::{
infra::inventory::InventoryRepositoryFactory,
interpret::{Interpret, InterpretError, InterpretName, InterpretStatus, Outcome},
inventory::{HostRole, Inventory},
modules::inventory::DiscoverHostForRoleScore,
modules::inventory::{DiscoverHostForRoleScore, HarmonyDiscoveryStrategy},
score::Score,
topology::HAClusterTopology,
};
@@ -104,6 +104,8 @@ When you can dig them, confirm to continue.
bootstrap_host = hosts.into_iter().next().to_owned();
DiscoverHostForRoleScore {
role: HostRole::Bootstrap,
number_desired_hosts: 1,
discovery_strategy: HarmonyDiscoveryStrategy::MDNS,
}
.interpret(inventory, topology)
.await?;

View File

@@ -6,7 +6,7 @@ use crate::{
inventory::{HostRole, Inventory},
modules::{
dhcp::DhcpHostBindingScore, http::IPxeMacBootFileScore,
inventory::DiscoverHostForRoleScore, okd::templates::BootstrapIpxeTpl,
inventory::{DiscoverHostForRoleScore, HarmonyDiscoveryStrategy}, okd::templates::BootstrapIpxeTpl,
},
score::Score,
topology::{HAClusterTopology, HostBinding},
@@ -58,38 +58,39 @@ impl OKDSetup03ControlPlaneInterpret {
inventory: &Inventory,
topology: &HAClusterTopology,
) -> Result<Vec<PhysicalHost>, InterpretError> {
const REQUIRED_HOSTS: usize = 3;
const REQUIRED_HOSTS: i16 = 3;
let repo = InventoryRepositoryFactory::build().await?;
let mut control_plane_hosts = repo.get_host_for_role(&HostRole::ControlPlane).await?;
let control_plane_hosts = repo.get_host_for_role(&HostRole::ControlPlane).await?;
while control_plane_hosts.len() < REQUIRED_HOSTS {
info!(
"Discovery of {} control plane hosts in progress, current number {}",
REQUIRED_HOSTS,
control_plane_hosts.len()
);
// This score triggers the discovery agent for a specific role.
DiscoverHostForRoleScore {
role: HostRole::ControlPlane,
}
.interpret(inventory, topology)
.await?;
control_plane_hosts = repo.get_host_for_role(&HostRole::ControlPlane).await?;
info!(
"Discovery of {} control plane hosts in progress, current number {}",
REQUIRED_HOSTS,
control_plane_hosts.len()
);
// This score triggers the discovery agent for a specific role.
DiscoverHostForRoleScore {
role: HostRole::ControlPlane,
number_desired_hosts: REQUIRED_HOSTS,
discovery_strategy: HarmonyDiscoveryStrategy::MDNS,
}
.interpret(inventory, topology)
.await?;
if control_plane_hosts.len() < REQUIRED_HOSTS {
Err(InterpretError::new(format!(
let control_plane_hosts = repo.get_host_for_role(&HostRole::ControlPlane).await?;
if control_plane_hosts.len() < REQUIRED_HOSTS as usize {
return Err(InterpretError::new(format!(
"OKD Requires at least {} control plane hosts, but only found {}. Cannot proceed.",
REQUIRED_HOSTS,
control_plane_hosts.len()
)))
} else {
// Take exactly the number of required hosts to ensure consistency.
Ok(control_plane_hosts
.into_iter()
.take(REQUIRED_HOSTS)
.collect())
)));
}
// Take exactly the number of required hosts to ensure consistency.
Ok(control_plane_hosts
.into_iter()
.take(REQUIRED_HOSTS as usize)
.collect())
}
/// Configures DHCP host bindings for all control plane nodes.

View File

@@ -417,7 +417,6 @@ pub struct EthernetSpec {
#[serde(rename_all = "kebab-case")]
pub struct BondSpec {
pub mode: String,
#[serde(alias = "port")]
pub ports: Vec<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub options: Option<BTreeMap<String, Value>>,

View File

@@ -12,74 +12,6 @@ use crate::{
topology::{HostNetworkConfig, NetworkInterface, NetworkManager, Switch, SwitchPort, Topology},
};
/// Configures high-availability networking for a set of physical hosts.
///
/// This is an opinionated Score that creates a resilient network configuration.
/// It assumes hosts have at least two network interfaces connected
/// to redundant switches for high availability.
///
/// The Score's `Interpret` logic will:
/// 1. Setup the switch with sane defaults (e.g. mark interfaces as switchports for discoverability).
/// 2. Discover which switch ports each host's interfaces are connected to (via MAC address).
/// 3. Create a network bond (e.g. LACP) on the host itself using these interfaces.
/// 4. Configure a corresponding port-channel on the switch(es) for those ports.
///
/// This ensures that both the host and the switch are configured to treat the
/// multiple links as a single, aggregated, and redundant connection.
///
/// Hosts with 0 or 1 detected interfaces will be skipped, as bonding is not
/// applicable.
///
/// <div class="warning">
/// The implementation is currently _not_ idempotent, even though it should be.
/// Running it more than once on the same host might result in duplicated bond configurations.
/// </div>
///
/// <div class="warning">
/// This Score is not named well. A better name would be
/// `HighAvailabilityHostNetworkScore`, or something similar to better express the intent.
/// </div>
///
/// # Requirements
///
/// This Score can only be applied to a [Topology] that implements both the
/// [NetworkManager] (to configure the host-side bond) and [Switch]
/// (to configure the switch-side port-channel) capabilities.
///
/// # Current limitations
///
/// ## 1. No rollback logic & limited idempotency
///
/// If any of the steps described above fails, the Score will not attempt to revert any changes
/// already applied. Which could render the host or switch in an inconsistent state.
///
/// ## 2. Propagation delays on the switch
///
/// It might take some time for the sane defaults in step 1) to be applied. In some cases,
/// it was observed that the switch takes up to 5min to actually apply the config.
///
/// But this Score's Interpret doesn't wait and directly proceeds to step 2) to discover
/// the MAC addresses. Which could result interfaces being skipped because their corresponding port
/// on the switch couldn't be found.
///
/// TODO: Validate that the switch is in the expected state before continuing.
///
/// ## 3. Bond configuration
///
/// To find the next available bond id, the current
/// [NetworkManager](crate::infra::network_manager::OpenShiftNmStateNetworkManager) implementation
/// simply checks for existing bonds named `bond[n]` and take the next available `n` number.
///
/// It doesn't check that there are already a bond for the interfaces that should be bonded. Which
/// might result in a duplicate bond being created.
///
/// TODO: Make sure the interfaces to aggregate are not already bonded.
///
/// # Future improvements
///
/// Along with the `TODO` items above, splitting this Score into multiple smaller ones would be
/// beneficial. It has a lot of moving parts and some of them could be used on their own to make
/// operations on a cluster easier.
#[derive(Debug, Clone, Serialize)]
pub struct HostNetworkConfigurationScore {
pub hosts: Vec<PhysicalHost>,
@@ -319,14 +251,14 @@ impl<T: Topology + NetworkManager + Switch> Interpret<T> for HostNetworkConfigur
#[cfg(test)]
mod tests {
use assertor::*;
use brocade::PortOperatingMode;
use harmony_types::{net::MacAddress, switch::PortLocation};
use lazy_static::lazy_static;
use crate::{
hardware::HostCategory,
topology::{
HostNetworkConfig, NetworkError, PreparationError, PreparationOutcome, SwitchError,
SwitchPort,
HostNetworkConfig, NetworkError, PortConfig, PreparationError, PreparationOutcome, SwitchError, SwitchPort
},
};
use std::{
@@ -760,5 +692,14 @@ mod tests {
Ok(())
}
async fn clear_port_channel(&self, ids: &Vec<Id>) -> Result<(), SwitchError> {
todo!()
}
async fn configure_interface(
&self,
port_config: &Vec<PortConfig>,
) -> Result<(), SwitchError> {
todo!()
}
}
}

View File

@@ -0,0 +1,11 @@
cargo build -p harmony_inventory_agent --release --target x86_64-unknown-linux-musl
SCRIPT_DIR="$(dirname ${0})"
cd "${SCRIPT_DIR}/docker/"
cp ../../target/x86_64-unknown-linux-musl/release/harmony_inventory_agent .
docker build . -t hub.nationtech.io/harmony/harmony_inventory_agent
docker push hub.nationtech.io/harmony/harmony_inventory_agent

View File

@@ -0,0 +1 @@
harmony_inventory_agent

View File

@@ -0,0 +1,17 @@
FROM debian:12-slim
# install packages required to make these commands available : lspci, lsmod, dmidecode, smartctl, ip
RUN apt-get update && \
apt-get install -y --no-install-recommends pciutils kmod dmidecode smartmontools iproute2 && \
rm -rf /var/lib/apt/lists/*
RUN mkdir /app
WORKDIR /app/
COPY harmony_inventory_agent /app/
ENV RUST_LOG=info
CMD [ "/app/harmony_inventory_agent" ]

View File

@@ -0,0 +1,117 @@
apiVersion: v1
kind: Namespace
metadata:
name: harmony-inventory-agent
labels:
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/audit: privileged
pod-security.kubernetes.io/warn: privileged
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: harmony-inventory-agent
namespace: harmony-inventory-agent
---
# Grant the built-in "privileged" SCC to the SA
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: use-privileged-scc
namespace: harmony-inventory-agent
rules:
- apiGroups: ["security.openshift.io"]
resources: ["securitycontextconstraints"]
resourceNames: ["privileged"]
verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: use-privileged-scc
namespace: harmony-inventory-agent
subjects:
- kind: ServiceAccount
name: harmony-inventory-agent
namespace: harmony-inventory-agent
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: use-privileged-scc
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: harmony-inventory-agent
namespace: harmony-inventory-agent
spec:
selector:
matchLabels:
app: harmony-inventory-agent
template:
metadata:
labels:
app: harmony-inventory-agent
spec:
serviceAccountName: harmony-inventory-agent
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
tolerations:
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
containers:
- name: inventory-agent
image: hub.nationtech.io/harmony/harmony_inventory_agent
imagePullPolicy: Always
env:
- name: RUST_LOG
value: "harmony_inventory_agent=trace,info"
resources:
limits:
cpu: 200m
memory: 256Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
privileged: true
# optional: leave the rest unset since privileged SCC allows it
#
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: harmony-inventory-builder
namespace: harmony-inventory-agent
spec:
replicas: 1
strategy: {}
selector:
matchLabels:
app: harmony-inventory-builder
template:
metadata:
labels:
app: harmony-inventory-builder
spec:
serviceAccountName: harmony-inventory-agent
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: inventory-agent
image: hub.nationtech.io/harmony/harmony_inventory_builder
imagePullPolicy: Always
env:
- name: RUST_LOG
value: "harmony_inventory_builder=trace,info"
resources:
limits:
cpu: 200m
memory: 256Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
privileged: true
# optional: leave the rest unset since privileged SCC allows it

View File

@@ -1,5 +1,5 @@
use harmony_types::net::MacAddress;
use log::{debug, warn};
use log::{debug, trace, warn};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::fs;
@@ -121,20 +121,48 @@ pub struct ManagementInterface {
impl PhysicalHost {
pub fn gather() -> Result<Self, String> {
trace!("Start gathering physical host information");
let mut sys = System::new_all();
trace!("System new_all called");
sys.refresh_all();
trace!("System refresh_all called");
Self::all_tools_available()?;
trace!("All tools_available success");
let storage_drives = Self::gather_storage_drives()?;
trace!("got storage drives");
let storage_controller = Self::gather_storage_controller()?;
trace!("got storage controller");
let memory_modules = Self::gather_memory_modules()?;
trace!("got memory_modules");
let cpus = Self::gather_cpus(&sys)?;
trace!("got cpus");
let chipset = Self::gather_chipset()?;
trace!("got chipsets");
let network_interfaces = Self::gather_network_interfaces()?;
trace!("got network_interfaces");
let management_interface = Self::gather_management_interface()?;
trace!("got management_interface");
let host_uuid = Self::get_host_uuid()?;
Ok(Self {
storage_drives: Self::gather_storage_drives()?,
storage_controller: Self::gather_storage_controller()?,
memory_modules: Self::gather_memory_modules()?,
cpus: Self::gather_cpus(&sys)?,
chipset: Self::gather_chipset()?,
network_interfaces: Self::gather_network_interfaces()?,
management_interface: Self::gather_management_interface()?,
host_uuid: Self::get_host_uuid()?,
storage_drives,
storage_controller,
memory_modules,
cpus,
chipset,
network_interfaces,
management_interface,
host_uuid,
})
}
@@ -208,6 +236,8 @@ impl PhysicalHost {
));
}
debug!("All tools found!");
Ok(())
}
@@ -231,7 +261,10 @@ impl PhysicalHost {
fn gather_storage_drives() -> Result<Vec<StorageDrive>, String> {
let mut drives = Vec::new();
trace!("Starting storage drive discovery using lsblk");
// Use lsblk with JSON output for robust parsing
trace!("Executing 'lsblk -d -o NAME,MODEL,SERIAL,SIZE,ROTA,WWN -n -e 7 --json'");
let output = Command::new("lsblk")
.args([
"-d",
@@ -245,13 +278,18 @@ impl PhysicalHost {
.output()
.map_err(|e| format!("Failed to execute lsblk: {}", e))?;
trace!(
"lsblk command executed successfully (status: {:?})",
output.status
);
if !output.status.success() {
return Err(format!(
"lsblk command failed: {}",
String::from_utf8_lossy(&output.stderr)
));
let stderr_str = String::from_utf8_lossy(&output.stderr);
debug!("lsblk command failed: {stderr_str}");
return Err(format!("lsblk command failed: {stderr_str}"));
}
trace!("Parsing lsblk JSON output");
let json: Value = serde_json::from_slice(&output.stdout)
.map_err(|e| format!("Failed to parse lsblk JSON output: {}", e))?;
@@ -260,6 +298,8 @@ impl PhysicalHost {
.and_then(|v| v.as_array())
.ok_or("Invalid lsblk JSON: missing 'blockdevices' array")?;
trace!("Found {} blockdevices in lsblk output", blockdevices.len());
for device in blockdevices {
let name = device
.get("name")
@@ -268,52 +308,72 @@ impl PhysicalHost {
.to_string();
if name.is_empty() {
trace!("Skipping unnamed device entry: {:?}", device);
continue;
}
trace!("Inspecting block device: {name}");
// Extract metadata fields
let model = device
.get("model")
.and_then(|v| v.as_str())
.map(|s| s.trim().to_string())
.unwrap_or_default();
trace!("Model for {name}: '{}'", model);
let serial = device
.get("serial")
.and_then(|v| v.as_str())
.map(|s| s.trim().to_string())
.unwrap_or_default();
trace!("Serial for {name}: '{}'", serial);
let size_str = device
.get("size")
.and_then(|v| v.as_str())
.ok_or("Missing 'size' in lsblk device")?;
trace!("Reported size for {name}: {}", size_str);
let size_bytes = Self::parse_size(size_str)?;
trace!("Parsed size for {name}: {} bytes", size_bytes);
let rotational = device
.get("rota")
.and_then(|v| v.as_bool())
.ok_or("Missing 'rota' in lsblk device")?;
trace!("Rotational flag for {name}: {}", rotational);
let wwn = device
.get("wwn")
.and_then(|v| v.as_str())
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty() && s != "null");
trace!("WWN for {name}: {:?}", wwn);
let device_path = Path::new("/sys/block").join(&name);
trace!("Sysfs path for {name}: {:?}", device_path);
trace!("Reading logical block size for {name}");
let logical_block_size = Self::read_sysfs_u32(
&device_path.join("queue/logical_block_size"),
)
.map_err(|e| format!("Failed to read logical block size for {}: {}", name, e))?;
trace!("Logical block size for {name}: {}", logical_block_size);
trace!("Reading physical block size for {name}");
let physical_block_size = Self::read_sysfs_u32(
&device_path.join("queue/physical_block_size"),
)
.map_err(|e| format!("Failed to read physical block size for {}: {}", name, e))?;
trace!("Physical block size for {name}: {}", physical_block_size);
trace!("Determining interface type for {name}");
let interface_type = Self::get_interface_type(&name, &device_path)?;
trace!("Interface type for {name}: {}", interface_type);
trace!("Getting SMART status for {name}");
let smart_status = Self::get_smart_status(&name)?;
trace!("SMART status for {name}: {:?}", smart_status);
let mut drive = StorageDrive {
name: name.clone(),
@@ -330,19 +390,31 @@ impl PhysicalHost {
// Enhance with additional sysfs info if available
if device_path.exists() {
trace!("Enhancing drive {name} with extra sysfs metadata");
if drive.model.is_empty() {
trace!("Reading model from sysfs for {name}");
drive.model = Self::read_sysfs_string(&device_path.join("device/model"))
.unwrap_or(format!("Failed to read model for {}", name));
.unwrap_or_else(|_| format!("Failed to read model for {}", name));
}
if drive.serial.is_empty() {
trace!("Reading serial from sysfs for {name}");
drive.serial = Self::read_sysfs_string(&device_path.join("device/serial"))
.unwrap_or(format!("Failed to read serial for {}", name));
.unwrap_or_else(|_| format!("Failed to read serial for {}", name));
}
} else {
trace!(
"Sysfs path {:?} not found for drive {name}, skipping extra metadata",
device_path
);
}
debug!("Discovered storage drive: {drive:?}");
drives.push(drive);
}
debug!("Discovered total {} storage drives", drives.len());
trace!("All discovered dives: {drives:?}");
Ok(drives)
}
@@ -418,6 +490,8 @@ impl PhysicalHost {
}
}
debug!("Found storage controller {controller:?}");
Ok(controller)
}
@@ -486,6 +560,7 @@ impl PhysicalHost {
}
}
debug!("Found memory modules {modules:?}");
Ok(modules)
}
@@ -501,22 +576,30 @@ impl PhysicalHost {
frequency_mhz: global_cpu.frequency(),
});
debug!("Found cpus {cpus:?}");
Ok(cpus)
}
fn gather_chipset() -> Result<Chipset, String> {
Ok(Chipset {
let chipset = Chipset {
name: Self::read_dmi("baseboard-product-name")?,
vendor: Self::read_dmi("baseboard-manufacturer")?,
})
};
debug!("Found chipset {chipset:?}");
Ok(chipset)
}
fn gather_network_interfaces() -> Result<Vec<NetworkInterface>, String> {
let mut interfaces = Vec::new();
let sys_net_path = Path::new("/sys/class/net");
trace!("Reading /sys/class/net");
let entries = fs::read_dir(sys_net_path)
.map_err(|e| format!("Failed to read /sys/class/net: {}", e))?;
trace!("Got entries {entries:?}");
for entry in entries {
let entry = entry.map_err(|e| format!("Failed to read directory entry: {}", e))?;
@@ -525,6 +608,7 @@ impl PhysicalHost {
.into_string()
.map_err(|_| "Invalid UTF-8 in interface name")?;
let iface_path = entry.path();
trace!("Inspecting interface {iface_name} path {iface_path:?}");
// Skip virtual interfaces
if iface_name.starts_with("lo")
@@ -535,70 +619,101 @@ impl PhysicalHost {
|| iface_name.starts_with("tun")
|| iface_name.starts_with("wg")
{
trace!(
"Skipping interface {iface_name} because it appears to be virtual/unsupported"
);
continue;
}
// Check if it's a physical interface by looking for device directory
if !iface_path.join("device").exists() {
trace!(
"Skipping interface {iface_name} since {iface_path:?}/device does not exist"
);
continue;
}
trace!("Reading MAC address for {iface_name}");
let mac_address = Self::read_sysfs_string(&iface_path.join("address"))
.map_err(|e| format!("Failed to read MAC address for {}: {}", iface_name, e))?;
let mac_address = MacAddress::try_from(mac_address).map_err(|e| e.to_string())?;
trace!("MAC address for {iface_name}: {mac_address}");
let speed_mbps = if iface_path.join("speed").exists() {
match Self::read_sysfs_u32(&iface_path.join("speed")) {
Ok(speed) => Some(speed),
let speed_path = iface_path.join("speed");
let speed_mbps = if speed_path.exists() {
trace!("Reading speed for {iface_name} from {:?}", speed_path);
match Self::read_sysfs_u32(&speed_path) {
Ok(speed) => {
trace!("Speed for {iface_name}: {speed} Mbps");
Some(speed)
}
Err(e) => {
debug!(
"Failed to read speed for {}: {} . This is expected to fail on wifi interfaces.",
"Failed to read speed for {}: {} (this may be expected on WiFi interfaces)",
iface_name, e
);
None
}
}
} else {
trace!("Speed file not found for {iface_name}, skipping");
None
};
trace!("Reading operstate for {iface_name}");
let operstate = Self::read_sysfs_string(&iface_path.join("operstate"))
.map_err(|e| format!("Failed to read operstate for {}: {}", iface_name, e))?;
trace!("Operstate for {iface_name}: {operstate}");
trace!("Reading MTU for {iface_name}");
let mtu = Self::read_sysfs_u32(&iface_path.join("mtu"))
.map_err(|e| format!("Failed to read MTU for {}: {}", iface_name, e))?;
trace!("MTU for {iface_name}: {mtu}");
trace!("Reading driver for {iface_name}");
let driver =
Self::read_sysfs_symlink_basename(&iface_path.join("device/driver/module"))
.map_err(|e| format!("Failed to read driver for {}: {}", iface_name, e))?;
trace!("Driver for {iface_name}: {driver}");
trace!("Reading firmware version for {iface_name}");
let firmware_version = Self::read_sysfs_opt_string(
&iface_path.join("device/firmware_version"),
)
.map_err(|e| format!("Failed to read firmware version for {}: {}", iface_name, e))?;
trace!("Firmware version for {iface_name}: {firmware_version:?}");
// Get IP addresses using ip command with JSON output
trace!("Fetching IP addresses for {iface_name}");
let (ipv4_addresses, ipv6_addresses) = Self::get_interface_ips_json(&iface_name)
.map_err(|e| format!("Failed to get IP addresses for {}: {}", iface_name, e))?;
trace!("Interface {iface_name} has IPv4: {ipv4_addresses:?}, IPv6: {ipv6_addresses:?}");
interfaces.push(NetworkInterface {
name: iface_name,
let is_up = operstate == "up";
trace!("Constructing NetworkInterface for {iface_name} (is_up={is_up})");
let iface = NetworkInterface {
name: iface_name.clone(),
mac_address,
speed_mbps,
is_up: operstate == "up",
is_up,
mtu,
ipv4_addresses,
ipv6_addresses,
driver,
firmware_version,
});
};
debug!("Discovered interface: {iface:?}");
interfaces.push(iface);
}
debug!("Discovered total {} network interfaces", interfaces.len());
trace!("Interfaces collected: {interfaces:?}");
Ok(interfaces)
}
fn gather_management_interface() -> Result<Option<ManagementInterface>, String> {
if Path::new("/dev/ipmi0").exists() {
let mgmt = if Path::new("/dev/ipmi0").exists() {
Ok(Some(ManagementInterface {
kind: "IPMI".to_string(),
address: None,
@@ -612,11 +727,16 @@ impl PhysicalHost {
}))
} else {
Ok(None)
}
};
debug!("Found management interface {mgmt:?}");
mgmt
}
fn get_host_uuid() -> Result<String, String> {
Self::read_dmi("system-uuid")
let uuid = Self::read_dmi("system-uuid");
debug!("Found uuid {uuid:?}");
uuid
}
// Helper methods
@@ -709,7 +829,8 @@ impl PhysicalHost {
Ok("Ramdisk".to_string())
} else {
// Try to determine from device path
let subsystem = Self::read_sysfs_string(&device_path.join("device/subsystem"))?;
let subsystem = Self::read_sysfs_string(&device_path.join("device/subsystem"))
.unwrap_or(String::new());
Ok(subsystem
.split('/')
.next_back()
@@ -779,6 +900,8 @@ impl PhysicalHost {
size.map(|s| s as u64)
}
// FIXME when scanning an interface that is part of a bond/bridge we won't get an address on the
// interface, we should be looking at the bond/bridge device. For example, br-ex on k8s nodes.
fn get_interface_ips_json(iface_name: &str) -> Result<(Vec<String>, Vec<String>), String> {
let mut ipv4 = Vec::new();
let mut ipv6 = Vec::new();

View File

@@ -1,4 +1,4 @@
use log::{debug, error, info, warn};
use log::{debug, error, info, trace, warn};
use mdns_sd::{ServiceDaemon, ServiceInfo};
use std::collections::HashMap;
@@ -12,6 +12,7 @@ use crate::{
/// This function is synchronous and non-blocking. It spawns a background Tokio task
/// to handle the mDNS advertisement for the lifetime of the application.
pub fn advertise(service_port: u16) -> Result<(), PresenceError> {
trace!("starting advertisement process for port {service_port}");
let host_id = match PhysicalHost::gather() {
Ok(host) => Some(host.host_uuid),
Err(e) => {
@@ -20,11 +21,15 @@ pub fn advertise(service_port: u16) -> Result<(), PresenceError> {
}
};
trace!("Found host id {host_id:?}");
let instance_name = format!(
"inventory-agent-{}",
host_id.clone().unwrap_or("unknown".to_string())
);
trace!("Found host id {host_id:?}, name : {instance_name}");
let spawned_msg = format!("Spawned local presence advertisement task for '{instance_name}'.");
tokio::spawn(async move {

View File

@@ -28,7 +28,7 @@ async fn inventory() -> impl Responder {
async fn main() -> std::io::Result<()> {
env_logger::init();
let port = env::var("HARMONY_INVENTORY_AGENT_PORT").unwrap_or_else(|_| "8080".to_string());
let port = env::var("HARMONY_INVENTORY_AGENT_PORT").unwrap_or_else(|_| "25000".to_string());
let port = port
.parse::<u16>()
.expect(&format!("Invalid port number, cannot parse to u16 {port}"));

View File

@@ -9,3 +9,4 @@ license.workspace = true
serde.workspace = true
url.workspace = true
rand.workspace = true
log.workspace = true

View File

@@ -1,6 +1,6 @@
use serde::{Deserialize, Serialize};
#[derive(Copy, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, PartialOrd, Ord)]
#[derive(Copy, Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize, PartialOrd, Ord)]
pub struct MacAddress(pub [u8; 6]);
impl MacAddress {
@@ -19,14 +19,6 @@ impl From<&MacAddress> for String {
}
}
impl std::fmt::Debug for MacAddress {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_tuple("MacAddress")
.field(&String::from(self))
.finish()
}
}
impl std::fmt::Display for MacAddress {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&String::from(self))

View File

@@ -1,4 +1,6 @@
use std::{fmt, str::FromStr};
use log::trace;
use serde::Serialize;
/// Simple error type for port parsing failures.
#[derive(Debug)]
@@ -21,7 +23,7 @@ impl fmt::Display for PortParseError {
/// Represents the atomic, physical location of a switch port: `<Stack>/<Module>/<Port>`.
///
/// Example: `1/1/1`
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone)]
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone, Serialize)]
pub struct PortLocation(pub u8, pub u8, pub u8);
impl fmt::Display for PortLocation {
@@ -70,6 +72,11 @@ impl FromStr for PortLocation {
pub enum PortDeclaration {
/// A single switch port defined by its location. Example: `PortDeclaration::Single(1/1/1)`
Single(PortLocation),
/// A Named port, often used for virtual ports such as PortChannels. Example
/// ```rust
/// PortDeclaration::Named("1".to_string())
/// ```
Named(String),
/// A strictly sequential range defined by two endpoints using the hyphen separator (`-`).
/// All ports between the endpoints (inclusive) are implicitly included.
/// Example: `PortDeclaration::Range(1/1/1, 1/1/4)`
@@ -130,8 +137,14 @@ impl PortDeclaration {
return Ok(PortDeclaration::Set(start_port, end_port));
}
let location = PortLocation::from_str(port_str)?;
Ok(PortDeclaration::Single(location))
match PortLocation::from_str(port_str) {
Ok(loc) => Ok(PortDeclaration::Single(loc)),
Err(e) => {
trace!("Failed to parse PortLocation {port_str} : {e}");
trace!("Falling back on named port");
Ok(PortDeclaration::Named(port_str.to_string()))
}
}
}
}
@@ -141,6 +154,7 @@ impl fmt::Display for PortDeclaration {
PortDeclaration::Single(port) => write!(f, "{port}"),
PortDeclaration::Range(start, end) => write!(f, "{start}-{end}"),
PortDeclaration::Set(start, end) => write!(f, "{start}*{end}"),
PortDeclaration::Named(name) => write!(f, "{name}"),
}
}
}

View File

@@ -195,7 +195,7 @@ pub struct System {
pub disablechecksumoffloading: u8,
pub disablesegmentationoffloading: u8,
pub disablelargereceiveoffloading: u8,
pub ipv6allow: u8,
pub ipv6allow: Option<u8>,
pub powerd_ac_mode: String,
pub powerd_battery_mode: String,
pub powerd_normal_mode: String,
@@ -226,6 +226,7 @@ pub struct System {
pub dns6gw: Option<String>,
pub dns7gw: Option<String>,
pub dns8gw: Option<String>,
pub prefer_ipv4: Option<String>,
pub dnsallowoverride: u8,
pub dnsallowoverride_exclude: Option<MaybeString>,
}
@@ -329,6 +330,7 @@ pub struct Range {
pub struct StaticMap {
pub mac: String,
pub ipaddr: String,
pub cid: Option<MaybeString>,
pub hostname: String,
pub descr: Option<MaybeString>,
pub winsserver: MaybeString,
@@ -764,9 +766,19 @@ pub struct Jobs {
pub struct Job {
#[yaserde(attribute = true)]
pub uuid: MaybeString,
#[yaserde(rename = "name")]
pub name: MaybeString,
pub name: Option<MaybeString>,
// Add other fields as needed
pub origin: Option<MaybeString>,
pub enabled: Option<MaybeString>,
pub minutes: Option<MaybeString>,
pub hours: Option<MaybeString>,
pub days: Option<MaybeString>,
pub months: Option<MaybeString>,
pub weekdays: Option<MaybeString>,
pub who: Option<MaybeString>,
pub command: Option<MaybeString>,
pub parameters: Option<MaybeString>,
pub description: Option<MaybeString>,
}
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]
@@ -895,28 +907,28 @@ pub struct Proxy {
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]
pub struct ProxyGeneral {
pub enabled: i8,
pub error_pages: String,
pub error_pages: Option<MaybeString>,
#[yaserde(rename = "icpPort")]
pub icp_port: MaybeString,
pub logging: Logging,
#[yaserde(rename = "alternateDNSservers")]
pub alternate_dns_servers: MaybeString,
#[yaserde(rename = "dnsV4First")]
pub dns_v4_first: i8,
pub dns_v4_first: Option<MaybeString>,
#[yaserde(rename = "forwardedForHandling")]
pub forwarded_for_handling: String,
pub forwarded_for_handling: Option<MaybeString>,
#[yaserde(rename = "uriWhitespaceHandling")]
pub uri_whitespace_handling: String,
pub uri_whitespace_handling: Option<MaybeString>,
#[yaserde(rename = "enablePinger")]
pub enable_pinger: i8,
#[yaserde(rename = "useViaHeader")]
pub use_via_header: i8,
pub use_via_header: Option<MaybeString>,
#[yaserde(rename = "suppressVersion")]
pub suppress_version: i32,
pub suppress_version: Option<MaybeString>,
#[yaserde(rename = "connecttimeout")]
pub connect_timeout: MaybeString,
pub connect_timeout: Option<MaybeString>,
#[yaserde(rename = "VisibleEmail")]
pub visible_email: String,
pub visible_email: Option<MaybeString>,
#[yaserde(rename = "VisibleHostname")]
pub visible_hostname: MaybeString,
pub cache: Cache,
@@ -953,7 +965,7 @@ pub struct LocalCache {
pub cache_mem: i32,
pub maximum_object_size: MaybeString,
pub maximum_object_size_in_memory: MaybeString,
pub memory_cache_mode: String,
pub memory_cache_mode: MaybeString,
pub size: i32,
pub l1: i32,
pub l2: i32,
@@ -965,13 +977,13 @@ pub struct LocalCache {
pub struct Traffic {
pub enabled: i32,
#[yaserde(rename = "maxDownloadSize")]
pub max_download_size: i32,
pub max_download_size: MaybeString,
#[yaserde(rename = "maxUploadSize")]
pub max_upload_size: i32,
pub max_upload_size: MaybeString,
#[yaserde(rename = "OverallBandwidthTrotteling")]
pub overall_bandwidth_trotteling: i32,
pub overall_bandwidth_trotteling: MaybeString,
#[yaserde(rename = "perHostTrotteling")]
pub per_host_trotteling: i32,
pub per_host_trotteling: MaybeString,
}
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]
@@ -988,7 +1000,7 @@ pub struct ParentProxy {
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]
pub struct Forward {
pub interfaces: String,
pub interfaces: MaybeString,
pub port: i32,
pub sslbumpport: i32,
pub sslbump: i32,
@@ -1033,9 +1045,9 @@ pub struct Acl {
pub google_apps: MaybeString,
pub youtube: MaybeString,
#[yaserde(rename = "safePorts")]
pub safe_ports: String,
pub safe_ports: MaybeString,
#[yaserde(rename = "sslPorts")]
pub ssl_ports: String,
pub ssl_ports: MaybeString,
#[yaserde(rename = "remoteACLs")]
pub remote_acls: RemoteAcls,
}
@@ -1051,9 +1063,9 @@ pub struct RemoteAcls {
pub struct Icap {
pub enable: i32,
#[yaserde(rename = "RequestURL")]
pub request_url: String,
pub request_url: MaybeString,
#[yaserde(rename = "ResponseURL")]
pub response_url: String,
pub response_url: MaybeString,
#[yaserde(rename = "SendClientIP")]
pub send_client_ip: i32,
#[yaserde(rename = "SendUsername")]
@@ -1061,7 +1073,7 @@ pub struct Icap {
#[yaserde(rename = "EncodeUsername")]
pub encode_username: i32,
#[yaserde(rename = "UsernameHeader")]
pub username_header: String,
pub username_header: MaybeString,
#[yaserde(rename = "EnablePreview")]
pub enable_preview: i32,
#[yaserde(rename = "PreviewSize")]
@@ -1076,9 +1088,9 @@ pub struct Authentication {
pub method: MaybeString,
#[yaserde(rename = "authEnforceGroup")]
pub auth_enforce_group: MaybeString,
pub realm: String,
pub credentialsttl: i32, // This field is already in snake_case
pub children: i32,
pub realm: MaybeString,
pub credentialsttl: MaybeString, // This field is already in snake_case
pub children: MaybeString,
}
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]
@@ -1293,6 +1305,7 @@ pub struct WireguardServerItem {
pub peers: String,
pub endpoint: MaybeString,
pub peer_dns: MaybeString,
pub debug: Option<MaybeString>,
}
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]
@@ -1477,6 +1490,7 @@ pub struct Ppp {
pub ports: Option<MaybeString>,
pub username: Option<MaybeString>,
pub password: Option<MaybeString>,
pub provider: Option<MaybeString>,
}
#[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]

View File

@@ -86,10 +86,7 @@ impl<'a> DhcpConfigLegacyISC<'a> {
mac,
ipaddr: ipaddr.to_string(),
hostname,
descr: Default::default(),
winsserver: Default::default(),
dnsserver: Default::default(),
ntpserver: Default::default(),
..Default::default()
};
existing_mappings.push(static_map);
@@ -126,9 +123,7 @@ impl<'a> DhcpConfigLegacyISC<'a> {
ipaddr: entry["ipaddr"].as_str().unwrap_or_default().to_string(),
hostname: entry["hostname"].as_str().unwrap_or_default().to_string(),
descr: entry["descr"].as_str().map(MaybeString::from),
winsserver: MaybeString::default(),
dnsserver: MaybeString::default(),
ntpserver: MaybeString::default(),
..Default::default()
})
.collect();