Background

  • Old code derived default gateway from config.DirectoryService.Get<IRouter>().SingleOrDefault().IP.

  • Wrong for greenfield: firewall (OPNsense, etc.) doesn’t exist as VM yet at install time.

  • Proxmox host installer (answer.toml), DC static-IP setup (Stage3Windows), and Linux/PE config emit (MainWinPE) all need a real reachable gateway during install - not a future VM IP.

  • Hence: split gateway-during-install (real upstream) from router-as-firewall (future VM, may not exist yet).

Phase 1 - Upstream gateway (current)

  • NETSetupConfig.GatewayIP carries the actual upstream port IP.

  • ISP/DC gateway, customer’s existing network gateway, or test bench’s real router.

  • All gateway-emitting code paths read GatewayIP first.

  • Fallback to Get<IRouter>().IP only when GatewayIP == IP.None (legacy customer JSONs).

  • Test fixture auto-populates GatewayIP from the Hyper-V external switch’s bound NIC default route:

  • VirtualHyperVTestBase.EnsureFreeIP

  • VirtualHyperVTestBase.PatchDhcpToHostGateway

  • Test bench reuses the real upstream - no fake gateway baked into fixtures.

Phase 2 - Firewall takeover (shipped)

  • Trigger: OPNsense VM provisioned by EnsureCustomerVms AND reachable on its LAN IP via HTTPS:443.

  • Helper: src/NETSetup/Helpers/EnsureGatewayTakeover.cs. Runs every netsetup update cycle at Phase 3e (after ConfigureOpnsenseVlans + RewireVmNicsToVlans).

  • Idempotent skip when config.GatewayIP already == OPNsense LAN IP. Best-effort: failures log warning + continue.

  • After flip:

  • Proxmox host: awk-in-place rewrite of gateway line under iface vmbr0 in /etc/network/interfaces + ifreload -a.

  • In-memory: config.GatewayIP = opnsenseLan so NixOS VMs re-template networking.defaultGateway next UpdateNixosVm cycle.

  • Customer JSON on disk keeps Phase 1 upstream IP unchanged (by design — reinstall must start from real upstream).

  • Outcomes (unit-test enum): NoOpnsense, OpnsenseUnreachable, AlreadyTakenOver, FlippedToOpnsense.

Code map (where gateway is read today)

  • src/NETSetup/Extensions/ProxmoxAutoInstallExtensions.cs - answer.toml network-manual section.

  • src/NETSetup/Stages/2-WinPE/MainWinPE.cs - Linux config bundle written to USB SYSTEM partition.

  • src/NETSetup/Stages/3-NETSetupWindows/Stage3Windows.cs + WindowsServerMethods.SetIPv4Configuration - DC static IPv4.

  • src/NETSetup/Helpers/EnsureGatewayTakeover.cs - Phase 2 host gateway flip + in-memory config.GatewayIP rewrite.

  • src/NETSetup/Config/Nix/Templates/NixTemplateFactory.cs - reads config.GatewayIP each emit, so NixOS VMs follow Phase 2 flip automatically.

Open follow-ups

  • Windows DC NIC rewrite: extend WindowsServerMethods.SetIPv4Configuration (PSDirect) so existing Windows VMs follow the Phase 2 flip. Today: Windows VMs keep their install-time gateway until reinstall.

  • Multi-firewall HA / VRRP: gateway should track the VRRP virtual IP, not a single firewall’s LAN IP.