Introduce a way to instrument what happens within Harmony and around Harmony (e.g. in the CLI or in Composer).
The goal is to provide visual feedback to the end users and inform them of the progress of their tasks (e.g. deployment) as clearly as possible. It is important to also let them know of the outcome of their tasks (what was created, where to access stuff, etc.).
<img src="https://media.discordapp.net/attachments/1295353830300713062/1400289618636574741/demo.gif?ex=688c18d5&is=688ac755&hm=2c70884aacb08f7bd15cbb65a7562a174846906718aa15294bbb238e64febbce&=" />
## Changes
### Instrumentation architecture
Extensibility and ease of use is key here, while preserving type safety as much as possible.
The proposed API is quite simple:
```rs
// Emit an event
instrumentation::instrument(
HarmonyEvent::TopologyPrepared {
topology: "k8s-anywhere",
outcome: Outcome::success("yay")
}
);
// Consume events
instrumentation::subscribe("Harmony CLI Logger", async |event| {
match event {
HarmonyEvent::TopologyPrepared { name, outcome } => todo!(),
}
});
```
#### Current limitations
* this API is not very extensible, but it could be easily changed to allow end users to define custom events in addition to Harmony core events
* we use a tokio broadcast channel behind the scene so only in process communication can happen, but it could be easily changed to a more flexible communication mechanism as implementation details are hidden
### `harmony_composer` VS `harmony_cli`
As Harmony Composer launches commands from Harmony (CLI), they both live in different processes. And because of this, we cannot easily make all the logging happens in one place (Harmony Composer) and get rid of Harmony CLI. At least not without introducing additional complexity such as communication through a server, unix socket, etc.
So for the time being, it was decided to preserve both `harmony_composer` and `harmony_cli` and let them independently log their stuff and handle their own responsibilities:
* `harmony_composer`: takes care only of setting up & packaging a project, delegates everything else to `harmony_cli`
* `harmony_cli`: takes care of configuring & running Harmony
### Logging & prompts
* [indicatif](https://github.com/console-rs/indicatif) is used to create progress bars and track progress within Harmony, Harmony CLI, and Harmony Composer
* [inquire](https://github.com/mikaelmello/inquire) is preserved, but was removed from `harmony` (core) as UI concerns shouldn't go that deep
* note: for now the only prompt we had was simply deleted, we'll have to find a better way to prompt stuff in the future
## Todos
* [ ] Update/Create ADRs
* [ ] Continue instrumentation for missing branches
* [ ] Allow instrumentation to emit and subscribe to custom events
Co-authored-by: Ian Letourneau <letourneau.ian@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/91
Reviewed-by: johnride <jg@nationtech.io>
if kube-prometheus-operator is present installation of prometheus-operator will skip
outside of local k3d installation installation of operator is skipped
- Implemented a dry-run mode for K8s resource patching, displaying diffs before applying changes.
- Added the `similar` dependency for calculating and displaying text diffs.
- Enhanced K8s resource application to handle various port specifications in NetworkPolicy ingress rules.
- Added support for port ranges and lists of ports in NetworkPolicy rules.
- Updated K8s client to utilize the dry-run configuration setting.
- Added configuration option `HARMONY_DRY_RUN` to enable or disable dry-run mode.
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/58
Reviewed-by: johnride <jg@nationtech.io>
Co-authored-by: Taha Hawa <taha@taha.dev>
Co-committed-by: Taha Hawa <taha@taha.dev>
Adds the foundation for managing tenant credentials, including:
- `TenantCredentialScore` for scoring credential-related operations.
- `TenantCredentialManager` trait for creating users.
- `CredentialMetadata` struct to store credential information.
- `CredentialData` enum to hold credential content.
- `TenantCredentialBundle` struct to encapsulate metadata and content.
This provides a starting point for implementing credential creation, storage, and retrieval within the harmony system.
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/63
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Co-committed-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
- Added `additional_allowed_cidr_ingress` and `additional_allowed_cidr_egress` fields to `TenantNetworkPolicy` to allow specifying custom CIDR blocks for network access.
- Updated K8sTenantManager to parse and apply these CIDR rules to NetworkPolicy ingress and egress rules.
- Added `cidr` dependency to `harmony_macros` and a custom proc macro `cidrv4` to easily parse CIDR strings.
- Updated TenantConfig to default inter tenant and internet egress to deny all and added default empty vectors for CIDR ingress and egress.
- Updated ResourceLimits to implement default.
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/60
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Co-committed-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
This Id implementation is optimized for ease of use. Ids are prefixed with the unix epoch and suffixed with 7 alphanumeric characters. But Ids can also contain any String the user wants to pass it
- Added functionality to tag and push the built Docker image to a specified registry.
- Modified deployment score to use the full image tag (including registry and project).
- Included error handling and logging for the `docker tag` and `docker push` commands.
- Updated the `K8sDeploymentScore` struct to include a namespace field and environment variables for database credentials.
- Added kebab-case conversion for deployment name and namespace.
- Implemented a check_output function for better error reporting.
- Adds a `deploy_database` function to the `LAMPInterpret` struct to deploy a MariaDB database using Helm.
- Integrates `HelmCommand` trait requirement to the `LAMPInterpret` struct.
- Introduces `HelmChartScore` to manage MariaDB deployment.
- Adds namespace configuration for helm deployments.
- Updates trait bounds for `LAMPInterpret` to include `HelmCommand`.
- Implements `get_namespace` function to retrieve the namespace.
- Added `build_dockerfile` function to generate a Dockerfile based on the LAMP stack for the given project.
- Implemented `build_docker_image` to execute the docker build command and create the image.
- Configured user and permissions for apache.
- Included necessary apache configuration for security.
- Added error handling for docker build failures.
- Exposed port 80 for external access.
- Added basic serialization to Config struct.
- Adds functionality to download, install, and manage k3d clusters.
- Includes methods for downloading the latest release, creating clusters, and verifying cluster existence.
- Implements `ensure_k3d_installed`, `get_latest_release_tag`, `download_latest_release`, `is_k3d_installed`, `verify_cluster_exists`, `create_cluster` and `create_kubernetes_client`.
- Provides a `get_client` method to access the Kubernetes client.
- Includes unit tests for download and installation.
- Adds handling for different operating systems.
- Improves error handling and logging.
- Introduces a `K3d` struct to encapsulate k3d cluster management logic.
- Adds the ability to specify the cluster name during K3d initialization.
- Implemented functionality to fetch the latest k3d release tag from GitHub.
- Added logic to determine the appropriate binary URL based on the current platform.
- Implemented downloading and saving the binary to a specified directory.
- Included unit tests to verify the download and installation process.
- Added a `K3D_BIN_FILE_NAME` constant for clarity.
- Added logging for better debugging.
WIP: formatted score debug print into a table with a name header and the score information below
Co-authored-by: Jean-Gabriel Gill-Couture <jg@nationtech.io>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/11
Reviewed-by: johnride <jg@nationtech.io>
Co-authored-by: Willem <wrolleman@nationtech.io>
Co-committed-by: Willem <wrolleman@nationtech.io>
Co-authored-by: tahahawa <tahahawa@gmail.com>
Reviewed-on: https://git.nationtech.io/NationTech/harmony/pulls/9
Reviewed-by: johnride <jg@nationtech.io>
Co-authored-by: Taha Hawa <taha@taha.dev>
Co-committed-by: Taha Hawa <taha@taha.dev>
This commit adds `serde` dependency and derives `Serialize` trait for `Score` types. This is necessary for serialization and deserialization of these types, which is required to display Scores to various user interfaces
- Added `serde` dependency to `harmony_types/Cargo.toml`.
- Added `serde::Serialize` derive macro to `MacAddress` in `harmony_types/src/lib.rs`.
- Added `serde::Serialize` derive macro to `Config` in `opnsense-config/src/config/config.rs`.
- Added `serde::Serialize` derive macro to `Score` in `harmony_types/src/lib.rs`.
- Added `serde::Serialize` derive macro to `Config` and `Score` in relevant modules.
- Added placeholder `todo!()` implementations for `serialize` methods. These will be implemented in future commits.
- Implemented `OKDLoadBalancerScore` and integrated it as a `FrontendScore`.
- Added `FrontendScore` trait for TUI displayable scores.
- Implemented `Display` for `OKDLoadBalancerScore`.
- Updated `ScoreListWidget` to handle `FrontendScore` types.
- Included load balancer score in the TUI.
Introduce the `Topology` trait to ensure that `Maestro` can compile-time safely bind compatible `Scores` and `Topologies`. This refactoring includes updating `HarmonyTuiEvent`, `ScoreListWidget`, and related structures to work with generic `Topology` types, enhancing type safety and modularity.
- Introduce `LAMPScore` struct with additional fields: `domain`, `config`, and `php_version`.
- Define default implementation for `LAMPConfig`.
- Update `Url` enum to use `Url(url::Url)` instead of `Remote(url::Url)`.
- Adjust references in `HttpServer` and `TftpServer` implementations.
- Modify `Interpret` trait implementation to use `name()` method from `LAMPScore`.
