Liberator = single-box on-prem stack. 1x Mikrotik hAP ax2 + 1x Proxmox host running OPNsense + NixDns + Win Server DC + Nextcloud + Mail + PMG LXC + optional Win clients, all VLAN-segmented.

Roles

Role discriminator = host.Description. Pattern already in use at src/NETSetup/Config/Translators/DeviceTranslator.cs:297.

Role host.Description Mikrotik upstream OPNsense VM VLAN flow

Liberator master

Liberator

yes

yes

full

Power Station

Power Station

future (cluster)

no

none, flat vmbr0

Vanilla Proxmox

Proxmox or other

no

no

none, flat vmbr0

Example customer config (TestCompany style):

.AddDevice("Liberator", DeviceTypes.Server)
    .WithSerialNumber("lib")
    .WithDescription("Liberator")

.AddDevice("Power-1", DeviceTypes.Server)
    .WithSerialNumber("power-1")
    .WithDescription("Power Station")

Detection

Two-layer mechanism:

  1. Translator-time (handled by LiberatorWithWinDCTranslator and friends): customer .cs builder marks single host with WithDescription("Liberator"). Translator, after the device-foreach loop, synthesises 4 child VMs (Nextcloud 100@<libSerial>, Mail 102@<libSerial>, Dns 103@<libSerial>, OPNsense 104@<libSerial>) into config.DirectoryService.Machines. Power Station hosts (WithDescription("Power Station")) get NO child synthesis. Description is consumed here and not propagated to Machine - by design.

  2. Runtime gate in UpdateMethods.RunProxmoxUpdate: the resulting config shape determines the flow. Single check:

var runFullNetworkFlow =
    GetOrderedVmsForHost(config, host.SerialNumber)
        .Any(v => v.OperatingSystem == OS.OPNSense);

OPNsense-presence IS the canonical runtime signal. Liberator hosts always have OPNsense synthesised; Power Station / vanilla Proxmox hosts never do.

Translator-time Description Resulting config runFullNetworkFlow

Liberator

OPNsense + Nextcloud + Mail + Dns Machines synthesised

true

Power Station

bare host, no synthesised VMs

false

Proxmox / empty / other

bare host

false

If !runFullNetworkFlow: skip OPNsense provision, skip Mikrotik API push, skip OPNsense API push, skip Proxmox VLAN bridge config, skip VM NIC retag. VMs (Nix / Win / PMG) explicitly listed in config still get provisioned on flat vmbr0 untagged via the same GetOrderedVmsForHost serial-matching code path.

Topology

@startuml Liberator-Network

title Liberator network topology - hAP ax2 + Proxmox + OPNsense + VLANs

' Role discriminator = host.Description (DeviceTranslator.cs:297 pattern):
'   "Liberator"     -> full flow: Mikrotik upstream + OPNsense on host + VLANs
'   "Power Station" -> bare Proxmox, future cluster compute, no router/VLAN
'   "Proxmox" / other -> bare Proxmox, no router/VLAN
' Defensive double-gate: isLiberator && hasOpnsenseVm in config for host.SerialNumber.
'
' VLAN map: 10 = server tier, 20 = LAN clients, 250 = OOB/MGMT, 254 = Guest.
' WAN untagged on Mikrotik ether1.
' DHCP source of truth: OPNsense per VLAN. Static reservations from config.Computers[].MAC.
' DNS resolver: NixDns VM running Unbound. OPNsense forwards all clients to NixDns IP.

skinparam rectangle {
  BackgroundColor<<infra>> #E8F0FE
  BackgroundColor<<svc>>   #E6F4EA
  BackgroundColor<<fw>>    #FCE8E6
  BackgroundColor<<dns>>   #FEF7E0
  BackgroundColor<<net>>   #F3E8FD
}

cloud "ISP / GPON CPE\nuntagged DHCP" as NET

node "Mikrotik hAP ax2 (L2 only post phase 2)\nmgmt IP VLAN250 .1\nNO NAT, NO DHCP server" as MT <<net>> {
  rectangle "ether1 (2.5G) WAN\nuntagged DHCP" as MT_E1
  rectangle "bonding1 = ether2 + ether3\nLACP 802.3ad\ntrunk: 10,20,250,254" as MT_BOND
  rectangle "ether4 access\nVLAN20 untagged" as MT_E4
  rectangle "ether5 access (spare LAN port)\nVLAN20 untagged" as MT_E5
  rectangle "wlan 2.4 + 5 GHz\nSSID Liberator\nWPA2/3 -> VLAN20" as MT_W_LAN
  rectangle "wlan 2.4 GHz\nSSID Liberator-Guest\nopen, no portal -> VLAN254" as MT_W_GUEST
  rectangle "wlan 5 GHz hidden\nSSID admin-oob-lib\nWPA3 -> VLAN250" as MT_W_ADMIN
}

NET <--> MT_E1 : public IP

node "Proxmox host (Liberator)\nbond0 = enp1s0 + enp2s0 LACP\nvmbr0 VLAN-aware trunk\nhost mgmt = vmbr0v250 = .2" as PVE <<infra>> {

  rectangle "OPNsense VM (Option Y, 5 vNIC)\nserial 104@lib\n  net0 untagged   = WAN\n  net1 tag=10     = .1 server tier gw\n  net2 tag=20     = .1 LAN gw + DHCP\n  net3 tag=250    = .3 MGMT\n  net4 tag=254    = .1 Guest gw\nFW deny-default inter-VLAN\nNAT outbound\nDNS forwarder -> NixDns .53\nREST API key first-boot installed" as OPN <<fw>>

  rectangle "NixDns VM (Unbound)\nserial 103@lib\nvNIC tag=10 = .53\nVLAN10\nresolver + blocking +\nreverse + DDNS" as DNS <<dns>>
  rectangle "Win Server DC\nserial dc@lib\nvNIC tag=10 = .10\nVLAN10\nDNS = 127.0.0.1, .53" as DC <<svc>>
  rectangle "Nextcloud VM\nserial 100@lib\nvNIC tag=10 = .50\nVLAN10" as NC <<svc>>
  rectangle "Mail VM\nserial 102@lib\nvNIC tag=10 = .51\nVLAN10" as MAIL <<svc>>
  rectangle "PMG LXC\nctid 200@lib\nvNIC tag=10 = .52\nVLAN10" as PMG <<svc>>
  rectangle "Win11 client VM\nserial 201@lib\nvNIC tag=20\nDHCP from OPNsense VLAN20" as W11 <<svc>>
}

MT_BOND <==> PVE : LACP 802.3ad trunk\n10 / 20 / 250 / 254

OPN -[hidden]- DNS
OPN -[hidden]- DC

DNS --> OPN  : VLAN10
DC --> OPN   : VLAN10
NC --> OPN   : VLAN10
MAIL --> OPN : VLAN10
PMG --> OPN  : VLAN10
W11 --> OPN  : VLAN20

MT_E4 ..> OPN : VLAN20 access (LAN clients reach OPNsense via Mikrotik bridge)
MT_W_LAN ..> W11 : VLAN20 (WiFi clients)
MT_W_ADMIN ..> PVE : VLAN250 OOB (admin laptop -> Proxmox host independent of OPNsense)

note bottom of OPN
  **Option Y: 5 tagged vNICs**
  Hypervisor enforces VLAN, defence in depth.
  qm create:
    -net0 virtio,bridge=vmbr0
    -net1 virtio,bridge=vmbr0,tag=10
    -net2 virtio,bridge=vmbr0,tag=20
    -net3 virtio,bridge=vmbr0,tag=250
    -net4 virtio,bridge=vmbr0,tag=254
  Other VMs: single tagged vNIC each.
end note

note right of PVE
  Out-of-band mgmt path:
    laptop -> admin-oob-lib WiFi (hidden, VLAN250)
    -> Mikrotik bridges VLAN250 over LACP trunk
    -> Proxmox host .2 reachable
    even if OPNsense down.
end note

note left of MT
  **Phase 1 bootstrap** (today's flash-mikrotik):
    flat NAT, Mikrotik DHCP, single vmbr0 on host.
  **Phase 2 cutover** (additive then strip):
    add VLAN bridge alongside flat
    -> swing Proxmox + VMs
    -> enable OPNsense WAN VLAN
    -> strip Mikrotik NAT + DHCP.
  Re-pushed every netsetup update cycle (idempotent).
end note

note top of NET
  **Role gate**: host.Description == "Liberator"
  AND OPNsense VM in config for host.SerialNumber.
  Otherwise (Power Station / Proxmox / other):
  skip entire Mikrotik + OPNsense + VLAN flow.
  VMs still provisioned on flat vmbr0 untagged.
end note

@enduml

VLAN Map

VLAN Purpose Hosts OPNsense if

(untagged)

WAN

ISP/GPON CPE

vtnet0 DHCP client

10

Server tier

NixDns .53, DC .10, NC .50, Mail .51, PMG .52, OPNsense .1

vtnet1 = .1

20

LAN clients

Win clients, printers, scanners, WiFi Liberator

vtnet2 = .1, DHCP server

250

OOB / MGMT

Mikrotik .1, Proxmox host .2, OPNsense mgmt .3, WiFi admin-oob-lib

vtnet3 = .3

254

Guest

WiFi Liberator-Guest, internet-only

vtnet4 = .1, isolated

Inter-VLAN firewall = deny default. Explicit allows VLAN20 → VLAN10 for AD, DNS, SMB, print, HTTPS. Guest VLAN254 internet-only, no inter-VLAN.

WiFi (hAP ax2)

SSID Band Auth VLAN

Liberator

2.4 + 5 GHz

WPA2/3 PSK

20

Liberator-Guest

2.4 GHz

open, no captive portal

254

admin-oob-lib

5 GHz hidden

WPA3 PSK

250

admin-oob-lib = the one OOB path. If OPNsense dies, admin joins hidden SSID, lands on VLAN250, reaches Proxmox host .2 and Mikrotik .1 directly via L2. No OPNsense in path.

Mikrotik Port Layout

Port Role Config

ether1 (2.5G)

WAN

untagged, DHCP client from GPON

ether2, ether3

LACP bond to Proxmox

trunk 10/20/250/254

ether4

LAN client access

VLAN20 untagged

ether5

LAN client access spare

VLAN20 untagged

No hardware LACP offload on hAP ax2 (Aldrin2 switch chip), bond runs in software. Fine for Liberator scale (2x 1 GbE to host).

Proxmox Host Network (post phase 2)

auto bond0
iface bond0 inet manual
    bond-slaves enp1s0 enp2s0
    bond-mode 802.3ad
    bond-miimon 100

auto vmbr0
iface vmbr0 inet manual
    bridge-ports bond0
    bridge-vlan-aware yes
    bridge-vids 10 20 250 254

auto vmbr0v250
iface vmbr0v250 inet static
    address 10.0.250.2/24
    gateway 10.0.250.3
    bridge_ports bond0.250
    bridge_stp off
    bridge_fd 0

ifupdown2 reload always run in commit/rollback safe-mode so mgmt loss auto-rolls back.

OPNsense VM NIC Layout (Option Y)

Five tagged vNICs, hypervisor enforces tags. Defence in depth: even if OPNsense misconfigures a VLAN child, hypervisor blocks cross-VLAN leak.

qm create <vmid> \
  --bios seabios \
  --memory 2048 --cores 2 \
  --agent enabled=1 \
  --scsihw virtio-scsi-single \
  --net0 virtio,bridge=vmbr0 \
  --net1 virtio,bridge=vmbr0,tag=10 \
  --net2 virtio,bridge=vmbr0,tag=20 \
  --net3 virtio,bridge=vmbr0,tag=250 \
  --net4 virtio,bridge=vmbr0,tag=254
qm importdisk <vmid> /var/lib/vz/template/iso/opnsense.img <storage>
qm set <vmid> --scsi0 <storage>:vm-<vmid>-disk-0 --boot order=scsi0
qm start <vmid>

Per-VLAN tcpdump at host = tcpdump -i tap<vmid>i2 (just VLAN20 traffic) etc.

Other VM NIC Layout

Each non-OPNsense VM gets one tagged vNIC:

VM kind Tag VLAN purpose

NixDns, DC, Nextcloud, Mail, PMG LXC

10

server tier

Win11 clients

20

LAN clients

DHCP and DNS Source of Truth

  • DHCP server: OPNsense per VLAN. Mikrotik DHCP stripped at end of cutover.

  • DHCP static reservations: derived from config.Computers[].MAC (machine map). Not from Employees - employees have no MACs.

  • DNS resolver: NixDns VM (Unbound). OPNsense DNS forwarder points all clients here. NixDns handles resolution, blocking, reverse, DDNS, reverse-proxy hints if configured.

Update Flow (every netsetup update cycle, idempotent)

@startuml Liberator-Network-Push

title netsetup update push - Liberator network converge (every cycle, idempotent)

actor Admin
participant "netsetup update\n(on Proxmox host)" as NS
participant "UpdateMethods\nRunProxmoxUpdate" as UM
participant "EnsureCustomerVms" as ECV
participant "ProvisionOpnsenseVm" as POPN
participant "OpnsenseInstall\n(SSH first-boot)" as INST
participant "ConfigureOpnsenseVlans\n(REST API push)" as COV
participant "ConfigureMikrotikVlans\n(REST API push)" as CMV
participant "ConfigureProxmoxBridges\n(/etc/network/interfaces +\nsafe ifreload)" as CPB
participant "RewireVmNicsToVlans\n(qm set tag=...)" as RWN
participant "OPNsense VM" as OPN
participant "Mikrotik hAP ax2" as MT
participant "Other VMs (Nix / Win / PMG)" as VMS

Admin -> NS: netsetup update
NS -> UM: RunProxmoxUpdate(host, config)
UM -> UM: post-install gate (log file)\napt update + full-upgrade\nEnsureProxmoxResolvConfSecondaryDns

== Role gate ==
UM -> UM: isLiberator = host.Description == "Liberator"
UM -> UM: hasOpnsenseVm = GetOrderedVmsForHost(config, host.SerialNumber)\n  .Any(v => v.OS == OS.OPNSense)
UM -> UM: runFullNetworkFlow = isLiberator && hasOpnsenseVm

alt runFullNetworkFlow

  UM -> ECV: Run(host, config, services)

  == Phase 3d.3: OPNsense provision (Option Y, 5 vNIC) ==
  ECV -> POPN: provision (default raw image, first only)
  POPN -> OPN: qm create <vmid>\n  --net0 virtio,bridge=vmbr0\n  --net1 virtio,bridge=vmbr0,tag=10\n  --net2 virtio,bridge=vmbr0,tag=20\n  --net3 virtio,bridge=vmbr0,tag=250\n  --net4 virtio,bridge=vmbr0,tag=254\nqm importdisk + qm set --scsi0 + qm start

  alt /conf/.netsetup-installed missing
    ECV -> INST: OpnsenseInstall.Install(host, machine, config)
    INST -> INST: build config.xml:\n  vtnet0..4 assigned\n  DHCP per VLAN (10/20/254)\n  static maps from config.Computers[].MAC\n  DNS forwarder -> NixDns .53\n  FW deny-default inter-VLAN\n  allow VLAN20 -> VLAN10 (AD/DNS/SMB/print/HTTPS)\n  admin user + api_key/api_secret
    INST -> OPN: ssh root@<wanIp> (master key)\nscp config.xml -> /conf/\nconfigctl service reload all
    INST -> INST: persist api_key/secret to\n/etc/netsetup/opnsense-api.env
  end

  == Phase 3d.x: other VMs (single tagged vNIC each) ==
  ECV -> VMS: NixDns/DC/NC/Mail/PMG -> tag=10\nWin11 client -> tag=20\n(serial host matching unchanged)

  == Phase 3e: VLAN converge (every cycle, drift-correcting) ==

  UM -> COV: push OPNsense REST API
  COV -> OPN: probe /api/core/system/status (basic auth)
  COV -> OPN: PUT /api/dhcpv4/static_map\nPUT /api/dnsforwarder/settings\nPUT /api/firewall/alias/...\nPUT /api/firewall/filter/...\nPOST /api/firewall/apply

  UM -> CMV: push Mikrotik REST API
  CMV -> MT: bridge VLAN-aware on\nadd VLANs 10,20,250,254\nbonding1 LACP 802.3ad\n  slaves = ether2, ether3\ntrunk ports tagged\nether4, ether5 access VLAN20\nwifi SSIDs: Liberator(20)\n             Liberator-Guest(254)\n             admin-oob-lib(250 hidden)\nMikrotik IP VLAN250 .1\nremove flat NAT\ndisable Mikrotik DHCP server

  UM -> CPB: rewrite /etc/network/interfaces:\n  bond0 LACP enp1s0+enp2s0\n  vmbr0 bridge-vlan-aware\n  vmbr0v250 host mgmt = .2\nifupdown2 reload safe-mode\n(rollback if mgmt lost)

  UM -> RWN: for each existing VM, drift-check tag:\n  qm set <vmid> -netN ...,tag=<vlan>\nonly when tag missing or wrong\n(idempotent)

  == Phase 3f: btrfs scrub ==
  UM -> UM: btrfs scrub each mounted fs

else !runFullNetworkFlow (Power Station / vanilla Proxmox / other)
  UM -> ECV: Run(host, config, services)
  ECV -> VMS: VMs provisioned on flat vmbr0 untagged\n(serial host matching unchanged)
  note right
    Skipped:
      ProvisionOpnsenseVm
      OpnsenseInstall
      ConfigureOpnsenseVlans
      ConfigureMikrotikVlans
      ConfigureProxmoxBridges
      RewireVmNicsToVlans
    Bare Proxmox path. Power Station
    cluster wiring = future TODO.
  end note
  UM -> UM: btrfs scrub each mounted fs
end

UM --> NS: update complete
NS --> Admin: exit 0

note over UM
  Cutover sequence (additive then strip) handled by the
  ordering above: OPNsense API stages all interfaces first
  (VLAN children created), Mikrotik adds VLAN bridge alongside
  flat, Proxmox swings VM NICs to tags, OPNsense WAN VLAN if
  enabled, finally Mikrotik strips flat NAT + DHCP.
  Every step idempotent so re-running netsetup update
  is always safe and converges drift.
end note

@enduml

Order:

  1. EnsureLatestBinaryOrReexec (Phase 3a)

  2. EnsureProxmoxPostInstall log-gated (Phase 3b)

  3. apt update + full-upgrade (Phase 3c)

  4. EnsureCustomerVms (Phase 3d)

    1. OPNsense first (Option Y 5-NIC, gated)

    2. NixDns + DC + NC + Mail + PMG (single tagged vNIC each)

    3. Win11 clients

  5. ConfigureOpnsenseVlans REST API push (Phase 3e, gated)

  6. ConfigureMikrotikVlans REST API push (Phase 3e, gated)

  7. ConfigureProxmoxBridges interfaces + safe ifreload (Phase 3e, gated)

  8. RewireVmNicsToVlans qm set drift correction (Phase 3e, gated)

  9. btrfs scrub (Phase 3f)

Steps 5-8 gated on runFullNetworkFlow. Every step idempotent: re-pushing on each cycle is safe and corrects drift.

Cutover Sequence (Phase 1 baseline → Phase 2 target)

Phase 1 baseline = today’s flash-mikrotik config: flat NAT, Mikrotik DHCP, single vmbr0 on host, OPNsense optional standalone.

Phase 2 target = the VLAN topology above.

Cutover = additive then strip, runs inside a single netsetup update cycle on a freshly flashed Liberator:

  1. OPNsense API: stage all interfaces + DHCP + FW. WAN VLAN if disabled.

  2. Mikrotik API: add VLAN-aware bridge + VLANs + bond + WiFi SSIDs alongside flat bridge. Both paths work in parallel.

  3. Proxmox: rewrite interfaces → bond + VLAN-aware vmbr0. ifreload safe-mode. Attach VM NICs with tags. Verify ping to OPNsense LAN .1.

  4. Enable OPNsense WAN VLAN if. Internet now flows via OPNsense.

  5. Mikrotik API: remove flat-NAT + disable Mikrotik DHCP. Mikrotik = pure L2.

If any step breaks: ifupdown2 rollback restores Proxmox mgmt; admin OOB WiFi reaches Mikrotik VLAN250; Mikrotik 5s reset button returns to factory.

Future: Cluster + Power Station

Power Station = Proxmox node added to a Liberator cluster for compute (e.g. AI workloads). Identified by host.Description == "Power Station". Today: bare Proxmox flow only, no router, no VLAN. Implementation deferred.

Planned wiring when implemented:

  • Power Station uplink trunked from Liberator master’s bond (VLAN10 server tier extends, plus a dedicated corosync VLAN TBD).

  • Proxmox cluster join via pvecm from Power Station to Liberator master.

  • VMs hosted on Power Station get same tag=10 vNICs as if hosted on master.

  • No OPNsense duplicate. Power station relies on master’s OPNsense for routing/firewall.

Not implemented. When wired, this section gets rewritten with the real flow.

Skipped Behaviour on Non-Liberator Hosts

For host.Description != "Liberator" OR no OPNsense VM in config for host.SerialNumber:

  • No ProvisionOpnsenseVm

  • No OpnsenseInstall

  • No ConfigureOpnsenseVlans / ConfigureMikrotikVlans

  • No ConfigureProxmoxBridges (host stays single-NIC vmbr0 untagged)

  • No RewireVmNicsToVlans

VMs (Nix / Windows / PMG LXC) still provisioned via the same GetOrderedVmsForHost serial-matching code path, but land on flat vmbr0 with no VLAN tag. Same code, gated flow.