harmony/examples/opnsense_vm_integration

OPNsense VM Integration Example

Fully automated end-to-end integration test: boots an OPNsense VM via KVM, bootstraps SSH and API access without any manual browser interaction, installs packages, runs 11 Harmony Scores, and verifies idempotency (runs all Scores twice, asserts zero duplicates). CI-friendly.

Quick start

# 1. One-time setup (libvirt, Docker compatibility)
./examples/opnsense_vm_integration/setup-libvirt.sh

# 2. Verify prerequisites
cargo run -p opnsense-vm-integration -- --check

# 3. Boot + bootstrap + integration test (fully unattended)
cargo run -p opnsense-vm-integration -- --full

# 4. Clean up
cargo run -p opnsense-vm-integration -- --clean

Or use the build script:

./build/opnsense-e2e.sh              # check + boot + test
./build/opnsense-e2e.sh --download   # download image first
./build/opnsense-e2e.sh --clean      # tear down

That's it. No browser clicks, no manual SSH setup, no wizard interaction.

What happens during --full

1. Downloads OPNsense 26.1 nano image (~350MB, cached after first download)
2. Injects config.xml with virtio interface assignments (vtnet0=LAN, vtnet1=WAN)
3. Creates a 4 GiB qcow2 disk and boots via KVM (1 vCPU, 1GB RAM, 4 NICs)
4. Waits for web UI to respond (~20s)
5. Automated bootstrap via OPNsenseBootstrap:
   • Logs in (root/opnsense) with CSRF token handling
   • Aborts the initial setup wizard
   • Enables SSH with root login and password auth
   • Changes web GUI port to 9443 (avoids HAProxy conflicts)
   • Restarts lighttpd via SSH to apply the port change
6. Creates OPNsense API key via SSH (PHP script)
7. Installs os-haproxy via firmware API
8. Runs 11 Scores configuring the entire firewall
9. Verifies all configurations via REST API assertions
10. Idempotency test: runs all 11 Scores again, asserts entity counts are unchanged

Step-by-step mode

If you prefer to separate boot and test:

# Boot + bootstrap (creates VM, enables SSH, sets port)
cargo run -p opnsense-vm-integration -- --boot

# Run integration test (assumes VM is bootstrapped)
cargo run -p opnsense-vm-integration

# Check VM status at any time
cargo run -p opnsense-vm-integration -- --status

Prerequisites

System requirements

• Linux with KVM support (Intel VT-x/AMD-V)
• ~10 GB free disk space
• ~15 minutes for first run (image download + firmware update)
• Subsequent runs: ~2 minutes

Required packages

Arch/Manjaro:

sudo pacman -S libvirt qemu-full dnsmasq

Fedora:

sudo dnf install libvirt qemu-kvm dnsmasq

Ubuntu/Debian:

sudo apt install libvirt-daemon-system qemu-kvm dnsmasq

Automated setup

./examples/opnsense_vm_integration/setup-libvirt.sh

This handles: user group membership, libvirtd startup, default storage pool, Docker FORWARD policy conflict.

After running setup, apply group membership:

newgrp libvirt

Docker + libvirt compatibility

Docker sets the iptables FORWARD policy to DROP, which blocks libvirt's NAT networking. The setup script detects this and switches libvirt to the iptables firewall backend so both coexist.

Scores applied

#	Score	What it configures
1	LoadBalancerScore	HAProxy with 2 frontends, backends with TCP health checks
2	DhcpScore	DHCP range, 2 static host bindings, PXE boot options
3	TftpScore	TFTP server serving boot files
4	NodeExporterScore	Prometheus node exporter
5	VlanScore	2 VLANs (tags 100, 200) on vtnet0
6	FirewallRuleScore	Firewall filter rules with logging
7	OutboundNatScore	Source NAT for outbound traffic
8	BinatScore	Bidirectional 1:1 NAT
9	VipScore	Virtual IPs (IP aliases)
10	DnatScore	Port forwarding rules
11	LaggScore	Link aggregation (vtnet2+vtnet3)

All Scores are idempotent: the test runs them twice and asserts entity counts are unchanged. This catches duplicate creation bugs that mock tests cannot detect.

Network architecture

Host (192.168.1.10) --- virbr-opn bridge --- OPNsense LAN (192.168.1.1)
                         192.168.1.0/24        vtnet0
                         NAT to internet

                    --- virbr0 (default) --- OPNsense WAN (DHCP)
                         192.168.122.0/24      vtnet1
                         NAT to internet

Environment variables

Variable	Default	Description
RUST_LOG	(unset)	Log level: info, debug, trace
HARMONY_KVM_URI	qemu:///system	Libvirt connection URI
HARMONY_KVM_IMAGE_DIR	~/.local/share/harmony/kvm/images	Cached disk images

Troubleshooting

VM won't start / permission denied Ensure your user is in the libvirt group and that the image directory is traversable by the qemu user. The setup script handles this.

192.168.1.0/24 conflict If your host network already uses this subnet, the VM will be unreachable. Edit the constants in src/main.rs to use a different subnet.

HAProxy install fails OPNsense may need a firmware update first. The integration test attempts this automatically. If it fails, connect to the web UI at https://192.168.1.1:9443 and update manually.

Serial console access

virsh -c qemu:///system console opn-integration
# Press Ctrl+] to exit