101 lines
4.2 KiB
Markdown
101 lines
4.2 KiB
Markdown
# OKD HA Cluster on KVM
|
||
|
||
Deploys a complete OKD high-availability cluster on a KVM hypervisor using
|
||
Harmony's KVM module. All infrastructure is defined in Rust — no YAML, no
|
||
shell scripts, no hand-crafted XML.
|
||
|
||
## What it creates
|
||
|
||
| Resource | Details |
|
||
|-------------------|------------------------------------------|
|
||
| Virtual network | `harmonylan` — 192.168.100.0/24, NAT |
|
||
| OPNsense VM | 2 vCPU / 4 GiB RAM — gateway + PXE |
|
||
| Control plane ×3 | 4 vCPU / 16 GiB RAM — `cp0` … `cp2` |
|
||
| Worker ×3 | 8 vCPU / 32 GiB RAM — `worker0` … `worker2` |
|
||
|
||
## Architecture
|
||
|
||
All VMs share the same `harmonylan` virtual network. OPNsense sits on both
|
||
that network and the host bridge, acting as the gateway and PXE server.
|
||
|
||
```
|
||
Host network (bridge)
|
||
│
|
||
┌───────┴──────────┐
|
||
│ OPNsense │ 192.168.100.1
|
||
│ gateway + PXE │
|
||
└───────┬──────────┘
|
||
│
|
||
│ harmonylan (192.168.100.0/24)
|
||
├─────────────┬──────────────────┬──────────────────┐
|
||
│ │ │ │
|
||
┌───────┴──┐ ┌──────┴───┐ ┌──────────┴─┐ ┌──────────┴─┐
|
||
│ cp0 │ │ cp1 │ │ cp2 │ │ worker0 │
|
||
│ .10 │ │ .11 │ │ .12 │ │ .20 │
|
||
└──────────┘ └──────────┘ └────────────┘ └──────┬─────┘
|
||
│
|
||
┌───────┴────┐
|
||
│ worker1 │
|
||
│ .21 │
|
||
└───────┬────┘
|
||
│
|
||
┌───────┴────┐
|
||
│ worker2 │
|
||
│ .22 │
|
||
└────────────┘
|
||
```
|
||
|
||
All nodes PXE boot from the network interface. OPNsense serves the OKD
|
||
bootstrap images via TFTP/iPXE and handles DHCP for the whole subnet.
|
||
|
||
## Prerequisites
|
||
|
||
- Linux host with KVM/QEMU and libvirt installed
|
||
- `libvirt-dev` headers (for building the `virt` crate)
|
||
- A `default` storage pool configured in libvirt
|
||
- Sufficient disk space (~550 GiB for all VM images)
|
||
|
||
## Running
|
||
|
||
```bash
|
||
cargo run --bin kvm_okd_ha_cluster
|
||
```
|
||
|
||
Set `RUST_LOG=info` (or `debug`) to control verbosity.
|
||
|
||
## Configuration
|
||
|
||
| Environment variable | Default | Description |
|
||
|-------------------------|--------------------|-------------------------------------|
|
||
| `HARMONY_KVM_URI` | `qemu:///system` | Libvirt connection URI |
|
||
| `HARMONY_KVM_IMAGE_DIR` | harmony data dir | Directory for qcow2 disk images |
|
||
|
||
For a remote KVM host over SSH:
|
||
|
||
```bash
|
||
export HARMONY_KVM_URI="qemu+ssh://user@myhost/system"
|
||
```
|
||
|
||
## What happens after `cargo run`
|
||
|
||
The program defines all resources in libvirt but does not start any VMs.
|
||
Next steps:
|
||
|
||
1. Start OPNsense: `virsh start opnsense-harmony`
|
||
2. Connect to the OPNsense web UI at `https://192.168.100.1`
|
||
3. Configure DHCP, TFTP, and the iPXE menu for OKD
|
||
4. Start the control plane and worker nodes — they will PXE boot and begin
|
||
the OKD installation automatically
|
||
|
||
## Cleanup
|
||
|
||
```bash
|
||
for vm in opnsense-harmony cp0-harmony cp1-harmony cp2-harmony \
|
||
worker0-harmony worker1-harmony worker2-harmony; do
|
||
virsh destroy "$vm" 2>/dev/null || true
|
||
virsh undefine "$vm" --remove-all-storage 2>/dev/null || true
|
||
done
|
||
virsh net-destroy harmonylan 2>/dev/null || true
|
||
virsh net-undefine harmonylan 2>/dev/null || true
|
||
```
|