Commit Graph

69 Commits

Author SHA1 Message Date
701d8cfab9 feat: automatically discover inventory (#127)
## Fully automated inventory gathering now works!

Boot up harmony_inventory_agent with `cargo run -p harmony_inventory_agent`
Launch the DiscoverInventoryAgentScore , currently available this way :

`RUST_LOG=info cargo run -p example-cli -- -f Discover -y`

And you will have automatically all hosts saved to the database. Run `cargo sqlx setup` if you have not done it yet.

Co-authored-by: Ian Letourneau <ian@noma.to>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/127
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Co-committed-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
2025-08-31 22:45:07 +00:00
f9906cb419 refact: Move basic types to harmony_types crate to avoid external dependencies.
This includes Id, IpAddress, Url and some other heavily used types
2025-08-30 18:01:14 -04:00
cb4382fbb5 feat: Inventory PhysicalHost persistence with sqlx and local sqlite db 2025-08-30 16:48:56 -04:00
1eca2cc1a9 refact: Discovery callback takes a closure and fix git dependency hash 2025-08-30 16:48:13 -04:00
ec277bc13d feat: Allow building harmony statically with x86_64-unknown-linux-musl target 2025-08-29 14:49:44 -04:00
05e7b8075c feat(inventory agent): Local presence advertisement and discovery now works! Must be within the same LAN to share the multicast address though 2025-08-29 11:22:44 -04:00
b857412151 extract related logic into an OkdIpxeScore 2025-08-29 09:52:11 -04:00
0876f4e4f0 Merge remote-tracking branch 'origin/doc/pxe_test_setup' into doc/pxe_test_setup 2025-08-29 01:15:00 -04:00
6ac0e095a3 wip(inventory-agent): local presence advertisement and discovery using mdns almost working 2025-08-29 01:10:43 -04:00
Ian Letourneau
f180cc4c80 wip: rename harmony-secret* by harmony_secret* 2025-08-28 14:29:24 -04:00
57c3b01e66 chore: refactor pxe templates to jinja templates rendered by askama 2025-08-22 09:05:18 -04:00
e2a09efdee Merge remote-tracking branch 'origin/master' into doc/pxe_test_setup 2025-08-21 21:56:09 -04:00
da6610c625 wip: PXE setup for ipxe and okd files in progress 2025-08-21 17:28:17 -04:00
27c51e0ec5 feat(wip): Support opnsense 25.7 which defaults to dnsmasq instead of isc dhcp 2025-08-20 21:54:46 -04:00
72fb05b5cc fix(inventory_agent) : Agent now retreives correct dmidecode fields, fixed uuid generation which is unacceptable, fixed storage drive parsing, much better error handling, much more strict behavior which also leads to more complete output as missing fields will raise errors unless explicitely optional 2025-08-19 17:56:06 -04:00
6685b05cc5 wip(inventory_agent): Refactoring for better error handling in progress 2025-08-19 17:05:23 -04:00
3f34f868eb Merge remote-tracking branch 'origin/master' into feat/inventory_agent 2025-08-19 12:56:10 -04:00
70a65ed5d0 Merge remote-tracking branch 'origin/master' into feat/secrets 2025-08-19 12:00:19 -04:00
26e8e386b9 feat: Secret module works with infisical and local file storage backends 2025-08-19 11:59:21 -04:00
19cb7f73bc feat: Harmony inventory agent crate that exposes an endpoint listing the host hardware. Has to be reviewed, generated 99% by GLM-4.5 2025-08-19 11:24:20 -04:00
2a6a233fb2 feat: WIP add secrets module and macro crate 2025-08-15 14:40:39 -04:00
Ian Letourneau
8bb4a9d3f6 fix: remove indicatif in harmony_cli to simplify logging and fixing interactions 2025-08-15 11:26:54 -04:00
024084859e Monitor an application within a tenant (#86)
WIP: added implementation to deploy crd-alertmanagerconfigs
Co-authored-by: Ian Letourneau <letourneau.ian@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/86
Co-authored-by: Willem <wrolleman@nationtech.io>
Co-committed-by: Willem <wrolleman@nationtech.io>
2025-08-04 21:42:01 +00:00
06aab1f57f fix(cli): reduce noise & better track progress within Harmony (#91)
Introduce a way to instrument what happens within Harmony and around Harmony (e.g. in the CLI or in Composer).

The goal is to provide visual feedback to the end users and inform them of the progress of their tasks (e.g. deployment) as clearly as possible. It is important to also let them know of the outcome of their tasks (what was created, where to access stuff, etc.).

<img src="https://media.discordapp.net/attachments/1295353830300713062/1400289618636574741/demo.gif?ex=688c18d5&is=688ac755&hm=2c70884aacb08f7bd15cbb65a7562a174846906718aa15294bbb238e64febbce&=" />

## Changes

### Instrumentation architecture
Extensibility and ease of use is key here, while preserving type safety as much as possible.

The proposed API is quite simple:
```rs
// Emit an event
instrumentation::instrument(
    HarmonyEvent::TopologyPrepared {
        topology: "k8s-anywhere",
        outcome: Outcome::success("yay")
    }
);

// Consume events
instrumentation::subscribe("Harmony CLI Logger", async |event| {
    match event {
        HarmonyEvent::TopologyPrepared { name, outcome } => todo!(),
    }
});
```

#### Current limitations
* this API is not very extensible, but it could be easily changed to allow end users to define custom events in addition to Harmony core events
* we use a tokio broadcast channel behind the scene so only in process communication can happen, but it could be easily changed to a more flexible communication mechanism as implementation details are hidden

### `harmony_composer` VS `harmony_cli`
As Harmony Composer launches commands from Harmony (CLI), they both live in different processes. And because of this, we cannot easily make all the logging happens in one place (Harmony Composer) and get rid of Harmony CLI. At least not without introducing additional complexity such as communication through a server, unix socket, etc.

So for the time being, it was decided to preserve both `harmony_composer` and `harmony_cli` and let them independently log their stuff and handle their own responsibilities:
* `harmony_composer`: takes care only of setting up & packaging a project, delegates everything else to `harmony_cli`
* `harmony_cli`: takes care of configuring & running Harmony

### Logging & prompts
* [indicatif](https://github.com/console-rs/indicatif) is used to create progress bars and track progress within Harmony, Harmony CLI, and Harmony Composer
* [inquire](https://github.com/mikaelmello/inquire) is preserved, but was removed from `harmony` (core) as UI concerns shouldn't go that deep
  * note: for now the only prompt we had was simply deleted, we'll have to find a better way to prompt stuff in the future

## Todos
* [ ] Update/Create ADRs
* [ ] Continue instrumentation for missing branches
* [ ] Allow instrumentation to emit and subscribe to custom events

Co-authored-by: Ian Letourneau <letourneau.ian@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/91
Reviewed-by: johnride <jg@nationtech.io>
2025-07-31 19:35:33 +00:00
3be2fa246c fix: unjank the demo (#85)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/85
Reviewed-by: wjro <wrolleman@nationtech.io>
2025-07-11 14:32:16 +00:00
c11a31c7a9 wip: Fix ArgoApplication 2025-07-04 16:43:10 -04:00
539b8299ae feat(continuousdelivery): Local deployment implementation for demo purposes. Needs a lot of refactoring but it works (or almost works) 2025-07-03 11:55:10 -04:00
82119076cf fix: merge conflict 2025-07-02 13:46:26 -04:00
197770a603 feat: Add ntfy score (#69)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/69
2025-07-02 16:19:35 +00:00
ab69a2c264 feat: add service monitors support to prom (#66)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/66
Co-authored-by: taha <taha@noreply.git.nationtech.io>
Co-committed-by: taha <taha@noreply.git.nationtech.io>
2025-07-02 15:29:16 +00:00
e857efa92f fix merge conflict 2025-07-02 11:26:27 -04:00
7de9860dcf refactor: monitoring takes namespace from tenant 2025-07-02 11:14:24 -04:00
8ae0d6b548 feat: Application Interpret still WIP but now call ensure_installed on features, also introduced a rust app example, completed work on clone_box behavior 2025-07-01 22:44:44 -04:00
55143dcad4 Merge pull request 'feat: add dry-run functionality and similar dependency' (#62) from feat/dryRun into master
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/62
Reviewed-by: wjro <wrolleman@nationtech.io>
2025-06-26 15:14:25 +00:00
acfb93f1a2 feat: add dry-run functionality and similar dependency
- Implemented a dry-run mode for K8s resource patching, displaying diffs before applying changes.
- Added the `similar` dependency for calculating and displaying text diffs.
- Enhanced K8s resource application to handle various port specifications in NetworkPolicy ingress rules.
- Added support for port ranges and lists of ports in NetworkPolicy rules.
- Updated K8s client to utilize the dry-run configuration setting.
- Added configuration option `HARMONY_DRY_RUN` to enable or disable dry-run mode.
2025-06-24 14:54:22 -04:00
f437c40428 impl_monitoring_alerting_kube_prometheus (#64)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/64
Co-authored-by: Willem <wrolleman@nationtech.io>
Co-committed-by: Willem <wrolleman@nationtech.io>
2025-06-24 18:54:15 +00:00
e06548ac44 feat: Alerting module architecture to make it easy to use and extensible by external crates
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/61
Reviewed-by: johnride <jg@nationtech.io>
Co-authored-by: Willem <wrolleman@nationtech.io>
Co-committed-by: Willem <wrolleman@nationtech.io>
2025-06-19 14:37:16 +00:00
155e9bac28 feat: create harmony_composer initial version + rework CI (#58)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/58
Reviewed-by: johnride <jg@nationtech.io>
Co-authored-by: Taha Hawa <taha@taha.dev>
Co-committed-by: Taha Hawa <taha@taha.dev>
2025-06-18 19:52:37 +00:00
7bebc58615 feat: add tenant credential management (#63)
Adds the foundation for managing tenant credentials, including:

- `TenantCredentialScore` for scoring credential-related operations.
- `TenantCredentialManager` trait for creating users.
- `CredentialMetadata` struct to store credential information.
- `CredentialData` enum to hold credential content.
- `TenantCredentialBundle` struct to encapsulate metadata and content.

This provides a starting point for implementing credential creation, storage, and retrieval within the harmony system.

Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/63
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Co-committed-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
2025-06-17 18:28:04 +00:00
b94dd1e595 feat: add support for custom CIDR ingress/egress rules (#60)
- Added `additional_allowed_cidr_ingress` and `additional_allowed_cidr_egress` fields to `TenantNetworkPolicy` to allow specifying custom CIDR blocks for network access.
- Updated K8sTenantManager to parse and apply these CIDR rules to NetworkPolicy ingress and egress rules.
- Added `cidr` dependency to `harmony_macros` and a custom proc macro `cidrv4` to easily parse CIDR strings.
- Updated TenantConfig to default inter tenant and internet egress to deny all and added default empty vectors for CIDR ingress and egress.
- Updated ResourceLimits to implement default.

Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/60
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Co-committed-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
2025-06-12 15:24:03 +00:00
8e472e4c65 feat: Add Default implementation for Harmony Id along with documentation.
This Id implementation is optimized for ease of use. Ids are prefixed with the unix epoch and suffixed with 7 alphanumeric characters. But Ids can also contain any String the user wants to pass it
2025-06-08 21:23:29 -04:00
2ff70db0b1 wip: Tenant example project 2025-06-06 13:52:40 -04:00
e17ac1af83 Merge remote-tracking branch 'origin/master' into TenantManager_impl_k8s_anywhere 2025-06-04 16:14:21 -04:00
2b6d2e8606 fix:merge confict 2025-05-20 16:05:38 -04:00
bae7222d64 Our own Helm Command/Resource/Executor (WIP) (#13)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/13
Co-authored-by: Taha Hawa <taha@taha.dev>
Co-committed-by: Taha Hawa <taha@taha.dev>
2025-05-20 14:01:10 +00:00
f7d3da3ac9 fix merge conflict 2025-05-15 15:31:26 -04:00
b4c6848433 feat: added default monitoringStackScore implementation 2025-05-15 14:52:04 -04:00
0d94c537a0 feat: add ingress score (#32)
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/32
Reviewed-by: wjro <wrolleman@nationtech.io>
2025-05-15 16:11:40 +00:00
bc2bd2f2f4 feat: push docker image to registry and deploy with full tag
- Added functionality to tag and push the built Docker image to a specified registry.
- Modified deployment score to use the full image tag (including registry and project).
- Included error handling and logging for the `docker tag` and `docker push` commands.
- Updated the `K8sDeploymentScore` struct to include a namespace field and environment variables for database credentials.
- Added kebab-case conversion for deployment name and namespace.
- Implemented a check_output function for better error reporting.
2025-04-30 22:33:31 -04:00
87f6afc249 feat: add mariadb helm deployment to lamp interpreter
- Adds a `deploy_database` function to the `LAMPInterpret` struct to deploy a MariaDB database using Helm.
- Integrates `HelmCommand` trait requirement to the `LAMPInterpret` struct.
- Introduces `HelmChartScore` to manage MariaDB deployment.
- Adds namespace configuration for helm deployments.
- Updates trait bounds for `LAMPInterpret` to include `HelmCommand`.
- Implements `get_namespace` function to retrieve the namespace.
2025-04-30 15:40:26 -04:00