harmony/adr/020-monitoring-alerting-architecture/monitoring_v2/README.md

Monitoring v2 - Improved Architecture

This example demonstrates the improved monitoring architecture that addresses the "WTF/minute" issues in the original design.

Key Improvements

1. Single AlertChannel Trait with Generic Sender

The original design required 9-12 implementations for each alert channel (Discord, Webhook, etc.) - one for each sender type. The new design uses a single trait with generic sender parameterization:

pub trait AlertChannel<Sender: AlertSender> { async fn install_config(&self, sender: &Sender) -> Result<Outcome, InterpretError>; fn name(&self) -> String; fn as_any(&self) -> &dyn std::any::Any; }

Benefits:

• One Discord implementation works with all sender types
• Type safety at compile time
• No runtime dispatch overhead

2. MonitoringStack Abstraction

Instead of manually selecting CRDPrometheus vs KubePrometheus vs RHOBObservability, you now have a unified MonitoringStack that handles versioning:

let monitoring_stack = MonitoringStack::new(MonitoringApiVersion::V2CRD) .set_namespace("monitoring") .add_alert_channel(discord_receiver) .set_scrape_targets(vec![...]);

Benefits:

• Single source of truth for monitoring configuration
• Easy to switch between monitoring versions
• Automatic version-specific configuration

3. TenantMonitoringScore - True Composition

The original monitoring_with_tenant example just put tenant and monitoring as separate items in a vec. The new design truly composes them:

let tenant_score = TenantMonitoringScore::new("test-tenant", monitoring_stack);

This creates a single score that:

• Has tenant context
• Has monitoring configuration
• Automatically installs monitoring scoped to tenant namespace

Benefits:

• No more "two separate things" confusion
• Automatic tenant namespace scoping
• Clear ownership: tenant owns its monitoring

4. Versioned Monitoring APIs

Clear versioning makes it obvious which monitoring stack you're using:

pub enum MonitoringApiVersion { V1Helm, // Old Helm charts V2CRD, // Current CRDs V3RHOB, // RHOB (future) }

Benefits:

• No guessing which API version you're using
• Easy to migrate between versions
• Backward compatibility path

Comparison

Original Design (monitoring_with_tenant)

• Manual selection of each component
• Manual installation of both components
• Need to remember to pass both to harmony_cli::run
• Monitoring not scoped to tenant automatically

New Design (monitoring_v2)

• Single composed score
• One score does it all

Usage

cd examples/monitoring_v2 cargo run

Migration Path

To migrate from the old design to the new:

1. Replace individual alert channel implementations with AlertChannel
2. Use MonitoringStack instead of manual *Prometheus selection
3. Use TenantMonitoringScore instead of separate TenantScore + monitoring scores
4. Select monitoring version via MonitoringApiVersion