solution: src/NETSetup.sln project: NETSetup testProject: NETSetup.Tests language: csharp testCommand: dotnet test src/NETSetup.sln --nologo --verbosity minimal ---
1. In Development
1.1. nixos-vm-provisioning
GOAL: Proxmox host running NETSetup provisions customer VMs per NETSetupConfig during netsetup update. NixOS-backed VMs (Nextcloud, IPFS, generic NixOS) boot generic nixos.iso, NETSetup SSHes in using master key, scps C#-generated flake + host config, runs install.sh remotely. Windows VMs use existing per-VM offline create-iso pipe. Result: fully hands-off VM deploy from a single netsetup update on the Proxmox host.
Decisions:
-
ISO artifact:
nixos.isoon Proxmox ISO storage. Dropbox holdsnixos.iso.7z+nixos.iso.sha256(ArchiveSync + SevenZipCommand pattern — see Serenacompressed_image_driver_sync). -
Bootstrap SSH: single master NETSetup keypair. Pubkey committed at
src/NETSetup/nixos/netsetup-master.pub, baked intoiso/iso.nixrootauthorized_keys. Privkey shipped with NETSetup binary undersrc/NETSetup/Resources/netsetup-master.key(embedded resource, extracted to/NETSetup/.ssh/netsetup-master.keychmod 0600 on use). -
OS trigger:
Machine.OperatingSystem.Name.StartsWith("Nix")→ nix-backed.OS.NixOS(contracts) routes toNixHostTemplate. New NETSetup-local OS constantsNixNextcloud,NixIPFSdefined inNETSetup.Config.Nix.NetsetupOS(NOT in contracts lib). -
Templates: abstract
NixHostTemplatebase + concreteNixNextcloudTemplate,NixIpfsTemplate. Templates exposeGenerateDefaultNix()(host config body) andGenerateHardwareNix(). Content derived fromsrc/NETSetup/nixos/hosts/default-nextcloud/default.nixanddefault-ipfs/default.nix.GenerateFlakeNix()exists but is no longer called byProvisionNixosVm— see post-implementation pivot below. -
Per-VM emit shape (post-pivot, 2026-04-30): flat
config.nix+hardware.nix+ sopssecrets.yaml+.default.keyat the per-VM generated dir root. No emittedflake.nix— the base nix tree at/iso/nixos-config/flake.nixalready exposesnixosConfigurations.default = mkHost "default" ./config.nix(nixos/hosts.nix:113). Overlaying our flake strippeddisko + lanzaboote + sopsmodules fromsharedHostModules, causingdisko-installto crash withartifacts[1]: unbound variable(nodiskoScriptartifact when disko module isn’t loaded). Reusing the base flake keeps every shared module in scope. -
End-to-end test ISO bootstrap:
ProxmoxNixosNextcloudTestsself-bootstraps every prerequisite in[TestInitialize]plusnixos.isoper run:-
EnsureChocoPackageCommand(nested NETCommand-style wrapper aroundchoco install <pkg> -y, idempotent) ensuressopsis on PATH forSopsCommand. -
EnsureWslDistroCommand(nested NETCommand-style wrapper aroundwsl --import) ensures theNixOSWSL distro is registered.IsDistroRegisteredshort-circuits onwsl -l -qso already-imported distros skip the Dropbox fetch. When absent, the rootfs tarball is pulled fromImages/NixOSWsl.tar.gzon Dropbox (TestContext.GetImageRemoteSource()prefers the local Dropbox folder for instant copy, falls back to the API). The Dropbox sub-path is encoded as a const on the test class so it’s the single point of update if the image moves. -
BuildNixosIsoCommand(wsl.exe -d NixOS — nix build .#isoagainst the localsrc/NETSetup/nixosflake →C:\NETSetupDEV\nixos.iso) builds the ISO, thenpveHost.UploadISOships it sha-driven so unchanged builds skip the wire transfer.
-
-
Why
wsl --importinstead ofwsl --install -d NixOS? The Microsoft Store catalogue version is not always the pinned NixOS-WSL build the flake expects (nixpkgs lock + cached flake inputs). Importing a tarball we publish ourselves keeps WSL state reproducible across dev boxes. -
install.shinvoked with-h default(not the per-VM hostname). Per-VM OS hostname identity preserved vianetworking.hostNameset inside the emittedconfig.nixbyNixHostTemplate. -
Installer SOPS age key: single committed pair at
src/NETSetup/nixos/.sops/installer.{key,pub}. Public key pinned asProvisionNixosVm.InstallerAgePublicKeyconstant. Private key shipped via csproj<Content CopyToOutputDirectory="Always" CopyToPublishDirectory="Always">, deployed to/NETSetup/.sops/installer.key(700/600) bynetsetup-first-boot.shsection 4c.ProvisionNixosVm.EmitConfigBundlegenerates a random 32-byte hex LUKS password, encrypts it viaSopsCommandto a per-VMsecrets.yaml, and copies the installer private key into.default.keysoinstall.shL156 can decrypt during disko-install. Local generated dir is wiped on success (TryCleanupConfigDir). -
Disk: plain btrfs (no LUKS). Default
/dev/sda; VirtIO VMs use/dev/vda.NixHostTemplatetakes device param. -
Entry point: extend
UpdateMethods.RunProxmoxUpdate(appendEnsureCustomerVms(host, config)at end).InstallCommand.InstallStage.ProxmoxVMsstays a stub. -
New commands live in
NETSetup/NETCommand/— no changes to sharedosisa.NETCommand.NixRemoteInstallCommandcomposesLinuxCommandBaseforssh/scp.QmGuestCommandwraps Proxmox-APInodes/<node>/qemu/<vmid>/agentvia corsinvest client (reuse existingProxmox/Hostwrappers fromosisa.SystemManagement.Proxmox). -
No
System.IO: useAbsolutePath+PhysicalFileOperator. No rawProcess: use NETCommand builders. -
VM identification: reuse
CreateVirtualMachines.IsForHost+GetOrderedVmsForHost(DC-first ordering preserved). -
Local emission path for generated nix configs:
AbsolutePath("/NETSetup/generated-nix") / vmName(Linux Proxmox host), wiped + rewritten each update cycle. -
UPDATE (2026-07-13, IPFS cluster secrets - commit
a997e484, live test green end-to-end): NixIPFS provision now GENERATE + EMIT its own sops secrets on top ofluks_password. Per-VMsecrets.yamlalso carryipfs/swarm_key(multiline psk block/key/swarm/psk/1.0.0/+/base16/+ 64-lowercase-hex) andipfs/cluster_secret(single lineCLUSTER_SECRET=<64-lowercase-hex>, a systemd EnvironmentFile line), both nested under oneipfs:map. Those two are cluster-shared:IpfsClusterSecretsGenerator.EnsureOn(config)mint them ONCE per customer/cluster (RandomNumberGenerator.Fill(byte[32])+Convert.ToHexStringLower), stash on new persistedNETSetupConfig.IpfsCluster(IpfsClusterOptionsholdingIpfsSwarmKey+IpfsClusterSecretVogen VOs), STABLE across re-provision (regen would partition the cluster). Key stringsipfs/swarm_key/ipfs/cluster_secretare consts, MUST matchipfs-private-cluster.nix:297,304.NixIpfsTemplate.GetRequiredSecrets(config)throwMissingIpfsClusterSecretsException : NETSetupExceptionwhen a NixIPFS host emit withconfig.IpfsCluster == null. luks stays UPPERCASE hex; ipfs entropy is LOWERCASE hex. -
UPDATE (2026-07-13, secrets.yaml renderer): emission gain nested-key + block-scalar support via new pure
SecretsYamlBuilder. Single-segment keys stay FLAT (luks back-compat);a/bkeys nest (so bothipfs/*group under oneipfs:map). Multiline value → YAML block scalar|(content indent = key indent + 2); single-line → inline-quoted (\+"escaped).SopsCommandnow holdIReadOnlyList<NixSecretEntry> Entries(was flat dictPayload);RenderedYamlInputdelegate to the builder. Templates declare their needs via new virtualNixHostTemplate.GetRequiredSecrets(config)(base =Array.Empty);EmitConfigBundlecallEnsureOn(config)then loopGetRequiredSecretsinto the same builder.InstallNixosVm.Runalso callEnsureOn(config)at top (inline-built configs). -
UPDATE (2026-07-13, non-interactive LUKS): encrypted
disko-installis NOW non-interactive viapasswordFile = "/tmp/luks.key"indisk-layouts.nix(both encrypted + dual-boot luks blocks; alsodisk/encrypted.nix+disk/dual-boot.nix).install.sh(L195-196) write the decrypted sopsluks_passwordto/tmp/luks.key(chmod 600) BEFOREdisko-install, delete it after TPM enroll (L306). WHY: withoutpasswordFiledisko prompt for the passphrase on stdin, get NOTHING over the ssh pipe,luksFormatabort, and the nix closure copy into the still-unmounted/mnt/disko-install-rootRAM tmpfs →No space left on device. This is the root-cause of the earlier IPFS install failures. -
UPDATE (2026-07-13, NixIPFS VM sizing): NixIPFS VM get 8 GiB RAM (installer build the closure in a RAM-backed tmpfs store - undersize = OOM) + 64 GiB disk (IPFS repo live on the system disk). Contrast:
NixHostTemplatedefault 2 GiB / 32 GiB;NixNextcloudTemplate8 GiB / 256 GiB. -
UPDATE (2026-07-13): the
ProxmoxNixosNextcloudTestsISO uploader connect to the Proxmox host by LAN IP, NOT theosisaLib03hostname (DNS not resolvable from every dev box).-
✓ Add NETSetup-local
OSconstants and nix-detection extension: createsrc/NETSetup/Config/Nix/NetsetupOS.cswith public static readonlyOperatingSystem NixNextcloud = OperatingSystem.From("NixNextcloud", OperatingSystemFamily.Linux_GNU)andNixIPFS = OperatingSystem.From("NixIPFS", OperatingSystemFamily.Linux_GNU). Createsrc/NETSetup/Extensions/NixOsExtensions.cswithpublic static bool IsNixBacked(this OperatingSystem os)returning true whenos == OS.NixOSORos.Name.StartsWith("Nix", StringComparison.OrdinalIgnoreCase). Assertions: -
NetsetupOS.NixNextcloud.Name.Should().Be(SomeNixNextcloudName)
-
NetsetupOS.NixNextcloud.Family.Should().Be(OperatingSystemFamily.Linux_GNU)
-
NetsetupOS.NixIPFS.Name.Should().Be(SomeNixIpfsName)
-
OS.NixOS.IsNixBacked().Should().BeTrue()
-
NetsetupOS.NixNextcloud.IsNixBacked().Should().BeTrue()
-
NetsetupOS.NixIPFS.IsNixBacked().Should().BeTrue()
-
OS.Windows11.IsNixBacked().Should().BeFalse()
-
OS.Proxmox.IsNixBacked().Should().BeFalse()
-
OS.Unknown.IsNixBacked().Should().BeFalse()
-
✓ Extend
src/NETSetup/nixos/iso/iso.nixwith qemu-guest-agent + master pubkey: addservices.qemuGuest.enable = true;andusers.users.root.openssh.authorizedKeys.keys = [ (lib.fileContents ../netsetup-master.pub) ];(createsrc/NETSetup/nixos/netsetup-master.pubplaceholder committed with a real ed25519 pubkey). Keepservices.openssh.enable = true, ensurePermitRootLogin = "prohibit-password"(or"yes"during bootstrap). Preserve all existing options (volumeID, contents, install-nixos.sh script, supportedFilesystems). Assertions: -
isoNixContent.Should().Contain("services.qemuGuest.enable = true")
-
isoNixContent.Should().Contain("users.users.root.openssh.authorizedKeys.keys")
-
isoNixContent.Should().Contain("netsetup-master.pub")
-
isoNixContent.Should().Contain("services.openssh.enable = true")
-
pubFileExists.Should().BeTrue()
-
pubFileContent.Should().StartWith("ssh-ed25519 ")
-
✓ Create
build-nixos-imageskill + standalone script: add.claude/skills/build-nixos-image/SKILL.md(description: "Build NixOS installer ISO, 7z, sha256, upload to Dropbox") plussrc/NETSetup/nixos/scripts/build-nixos-image.shthat (1)cdto nixos dir,git add -A(flakes see only tracked), (2)nix build .#iso --out-link /tmp/nixos-iso-result, (3) resolve ISO file via symlink, (4)7z a -mx=9 /tmp/nixos.iso.7z <iso>, (5)sha256sum /tmp/nixos.iso.7z > /tmp/nixos.iso.sha256, (6)cpboth to"$DROPBOX/NETSetup/ISOs/". Script must be idempotent, detect missing nix/7z and error clearly, and exit non-zero on any step failure. Skill instructs running under WSL distronixos(modeled afterupdate-nixos-imageskill). Assertions: -
skillFileExists.Should().BeTrue()
-
skillContent.Should().Contain("build-nixos-image")
-
skillContent.Should().Contain("WSL")
-
scriptFileExists.Should().BeTrue()
-
scriptContent.Should().Contain("nix build .#iso")
-
scriptContent.Should().Contain("7z a")
-
scriptContent.Should().Contain("sha256sum")
-
scriptContent.Should().Contain("set -euo pipefail")
-
scriptIsExecutable.Should().BeTrue() (file mode check)
-
✓ Implement C# Nix templates under
src/NETSetup/Config/Nix/Templates/: abstractNixHostTemplatewith propertiesHostname(MachineName),DiskDevice(string, default/dev/sda),UserName(string, defaultadmin),TimeZone(string, defaultUTC),AuthorizedKeys(IReadOnlyList<string>) and virtual methodsGenerateFlakeNix(),GenerateHardwareNix(), abstractGenerateDefaultNix(). ConcreteNixNextcloudTemplateandNixIpfsTemplateoverrideGenerateDefaultNix()with content ported fromsrc/NETSetup/nixos/hosts/default-nextcloud/default.nixanddefault-ipfs/default.nix(literal string). FactoryNixTemplateFactory.ForMachine(Machine machine)returns concrete template based onmachine.OperatingSystem:NixNextcloud→ NixNextcloudTemplate,NixIPFS→ NixIpfsTemplate,NixOS→NixHostTemplategeneric subclassNixGenericTemplate, other → throwsNETSetupException. Assertions: -
nextcloudTemplate.GenerateDefaultNix().Should().Contain("services.nextcloud.enable = true")
-
nextcloudTemplate.GenerateDefaultNix().Should().Contain(SomeHostname)
-
nextcloudTemplate.GenerateFlakeNix().Should().Contain("description")
-
nextcloudTemplate.GenerateHardwareNix().Should().Contain(SomeDiskDevice)
-
ipfsTemplate.GenerateDefaultNix().Should().Contain("services.ipfs.enable = true")
-
NixTemplateFactory.ForMachine(nextcloudMachine).Should().BeOfType<NixNextcloudTemplate>()
-
NixTemplateFactory.ForMachine(ipfsMachine).Should().BeOfType<NixIpfsTemplate>()
-
NixTemplateFactory.ForMachine(nixosMachine).Should().BeOfType<NixGenericTemplate>()
-
act.Should().Throw<NETSetupException>() for non-nix machine
-
template.GenerateDefaultNix().Should().Contain(SomeAuthorizedKey)
-
✓ Implement Proxmox guest-agent IP discovery:
src/NETSetup/Extensions/QmGuestAgentExtensions.cswithpublic static IPAddress DiscoverVmIpViaQmAgent(this Host host, int vmId, TimeSpan deadline, ILogger logger). Uses corsinvest_client.Nodes[<node>].Qemu[vmId].Agent.NetworkGetInterfaces.NetworkGetInterfaces()(or equivalent), parses JSONdata.result[].ip-addresses[]filteringip-address-type == "ipv4", excluding127.0.0.0/8and169.254.0.0/16. Polls every 2s until first match or deadline. ThrowsNETSetupExceptionwithvmIdin message on timeout. Assertions: -
result.Should().Be(IPAddress.Parse(SomeValidIp))
-
result.AddressFamily.Should().Be(AddressFamily.InterNetwork)
-
(loopback-only scenario) act.Should().Throw<NETSetupException>().Where(e ⇒ e.Message.Contains(SomeVmId.ToString()))
-
(link-local-only scenario) act.Should().Throw<NETSetupException>()
-
(timeout with no response) act.Should().Throw<NETSetupException>().Where(e ⇒ e.Message.Contains("timeout"))
-
(multiple ipv4) result.Should().Be(IPAddress.Parse(SomeFirstNonLoopbackIp))
-
✓ Implement
NixRemoteInstallCommandundersrc/NETSetup/NETCommand/NixRemoteInstallCommand.csusing NETCommand builder pattern. Builder exposesWithIp(IPAddress),WithHostname(string),WithMasterKey(AbsolutePath keyPath),WithConfigDir(AbsolutePath localGeneratedDir),WithDiskDevice(string)(default/dev/sda),CopyHostSshKeys(bool).Execute()runs: (1)ssh-keyscantarget → known_hosts, (2)scp -r <localDir>/* root@<ip>:/nixos-config/, (3)ssh root@<ip> "/nixos-config/scripts/install.sh -h <name> -d <device>"with 30min timeout. Each phase is aLinuxCommandBaseinternally. ReturnsNixRemoteInstallResultwithSuccess,StandardOutput,StandardError,Phase(KeyScan/Scp/Install) on failure. Assertions: -
builder.WithIp(SomeIp).WithHostname(SomeHostname).WithMasterKey(SomeKeyPath).WithConfigDir(SomeConfigDir).Build().Should().NotBeNull()
-
act.Should().Throw<ArgumentException>() when WithIp not called
-
act.Should().Throw<ArgumentException>() when WithHostname not called
-
command.RenderedScpArgs.Should().Contain("root@" + SomeIp)
-
command.RenderedScpArgs.Should().Contain("/nixos-config/")
-
command.RenderedSshArgs.Should().Contain("install.sh -h " + SomeHostname)
-
command.RenderedSshArgs.Should().Contain("-d " + SomeDiskDevice)
-
command.RenderedSshArgs.Should().Contain("-i " + SomeKeyPath)
-
result.Phase.Should().Be(InstallPhase.Install) on successful full run
-
✓ Implement
EnsureNixosIsoOnProxmoxHost(ISystemHost host, IFileProvider dropbox): checks/var/lib/vz/template/iso/nixos.isoexistence AND sha256 match against DropboxNETSetup/ISOs/nixos.iso.sha256. If missing or mismatch: downloadsnixos.iso.7zto temp, verifies sha256, extracts viaSevenZipCommandto/var/lib/vz/template/iso/nixos.iso, logs action. Idempotent — returns silently if match. Lives atsrc/NETSetup/Helpers/EnsureNixosIso.cs. Assertions: -
act.Should().NotThrow() when ISO present + hash matches
-
result.Action.Should().Be(EnsureAction.Downloaded) when missing
-
result.Action.Should().Be(EnsureAction.Replaced) when hash mismatch
-
result.Action.Should().Be(EnsureAction.UpToDate) when already current
-
act.Should().Throw<NETSetupException>() when Dropbox sha256 file missing
-
act.Should().Throw<NETSetupException>() when downloaded archive sha256 fails verification
-
isoPath.FileExists().Should().BeTrue() after successful run
-
✓ Implement
ProvisionNixosVm(Host proxmoxHost, Machine machine, NETSetupConfig config, ISystemHost sysHost)insrc/NETSetup/Stages/5-NETSetupLinux/ProvisionNixosVm.cs: (1) buildVirtualMachineentity fromMachine(name, id from serial prefix, CPU/RAM/disk from machine hardware,IsoName = "nixos.iso"), (2) skip if VM already exists with matching serial description, (3) callproxmoxHost.CreateVirtualMachine(vm, autoStart: true), (4)DiscoverVmIpViaQmAgent(vm.Id, TimeSpan.FromMinutes(5)), (5) pick template viaNixTemplateFactory.ForMachine(machine), emit files toAbsolutePath("/NETSetup/generated-nix") / machine.Name, (6) runNixRemoteInstallCommandbuilder with generated dir + master key path, (7) log result, throwNETSetupExceptionon failure. Assertions: -
(new VM scenario) proxmoxHost.Received().CreateVirtualMachine(Arg.Is<VirtualMachine>(vm ⇒ vm.IsoName == SomeNixosIsoName))
-
(existing VM scenario) proxmoxHost.DidNotReceive().CreateVirtualMachine(Arg.Any<VirtualMachine>())
-
result.VmIp.Should().Be(IPAddress.Parse(SomeDiscoveredIp))
-
result.GeneratedConfigDir.Should().Be(SomeExpectedGeneratedDir)
-
act.Should().Throw<NETSetupException>() when DiscoverVmIpViaQmAgent times out
-
act.Should().Throw<NETSetupException>() when NixRemoteInstall fails
-
generatedDir.DirectoryExists().Should().BeTrue() after successful emission (during install — wiped on success post-pivot)
-
(generatedDir / "config.nix").FileExists().Should().BeTrue()
-
(generatedDir / "hardware.nix").FileExists().Should().BeTrue()
-
(generatedDir / "secrets.yaml").FileExists().Should().BeTrue()
-
(generatedDir / ".default.key").FileExists().Should().BeTrue()
-
✓ Implement
ProvisionWindowsVmOnProxmox(Host proxmoxHost, Machine machine, NETSetupConfig config, ISystemHost sysHost, IServiceProvider services)insrc/NETSetup/Stages/5-NETSetupLinux/ProvisionWindowsVmOnProxmox.cs: (1) skip if VM exists, (2) derive ticket + hostname frommachine.Name+ config, (3) invoke existingCreateIsoCommandoffline path (MainISO.CreateOfflineProductionISO+CopyToProxmoxStorageIfApplicable) to produceNETSetup-{ticket}-{host}.isoin/var/lib/vz/template/iso/, (4) buildVirtualMachinewith that ISO as main, stealth hardware iffShouldUseStealthHardware(machine), (5)proxmoxHost.CreateVirtualMachine(vm, autoStart: true). Reuses existingCreateVirtualMachines.ShouldUseStealthHardware. Assertions: -
(new Windows 11 VM) proxmoxHost.Received().CreateVirtualMachine(Arg.Is<VirtualMachine>(vm ⇒ vm.IsoName.StartsWith("NETSetup-")))
-
(WindowsServer VM) proxmoxHost.Received().CreateVirtualMachine(Arg.Any<VirtualMachine>())
-
(existing VM) proxmoxHost.DidNotReceive().CreateVirtualMachine(Arg.Any<VirtualMachine>())
-
producedIsoPath.Should().Be(ProxmoxDefaultIsoStoragePath + "/NETSetup-" + SomeTicket + "-" + SomeHostname + ".iso")
-
result.UsedStealthHardware.Should().BeTrue() when machine is Windows 11
-
result.UsedStealthHardware.Should().BeFalse() when machine is WindowsServer
-
act.Should().Throw<NETSetupException>() when offline ISO creation fails
-
✓ Wire VM provisioning into
UpdateMethods.RunProxmoxUpdate: add new methodEnsureCustomerVms(ISystemHost host, NETSetupConfig config, IServiceProvider services)called at end ofRunProxmoxUpdate(afterRunBtrfsScrub). Method: (1) connect to localProxmoxvia DI, (2)GetHostAsync(host.MachineName), (3) callEnsureNixosIsoOnProxmoxHostiff any nix-backed machine targets this host, (4)GetOrderedVmsForHost(config, host.SerialNumber)— iterate with DC first, (5) dispatch:IsNixBacked→ProvisionNixosVm, else Windows →ProvisionWindowsVmOnProxmox, other → log skip. Errors per-VM are logged + continue (one bad VM must not block others). Final aggregate throws if any VM failed. Assertions: -
(no VMs for host) proxmoxHost.DidNotReceive().CreateVirtualMachine(Arg.Any<VirtualMachine>())
-
(mixed nix+windows VMs) proxmoxHost.Received(SomeExpectedCount).CreateVirtualMachine(Arg.Any<VirtualMachine>())
-
(DC-first ordering) firstCreatedVmSerial.Should().StartWith(SomeDcVmIdPrefix)
-
(all nix, no windows) ensureNixosIsoCalled.Should().BeTrue()
-
(only windows, no nix) ensureNixosIsoCalled.Should().BeFalse()
-
(one VM fails, others succeed) proxmoxHost.Received(SomeTotalCount).CreateVirtualMachine(Arg.Any<VirtualMachine>())
-
(any failure) act.Should().Throw<NETSetupException>().Where(e ⇒ e.Message.Contains(SomeFailedVmName))
-
(all success) act.Should().NotThrow()
-
✓ Documentation: add section "Proxmox VM provisioning during update" to
netsetup.adoc(after existing Proxmox sections) describing the two flows (NixOS SSH-deploy + Windows ISO-deploy), required keys/paths, update cadence. Add PUML sequence diagramsrc/NETSetup/docs/NETSetupProvisionVms.pumlshowing: netsetup-update → EnsureNixosIso → CreateVirtualMachine → qm guest-agent → scp config → ssh install.sh → reboot. Include innetsetup.adocviainclude::docs/NETSetupProvisionVms.puml[]. All narrative in caveman full per CLAUDE.md. Assertions: -
adocContent.Should().Contain("Proxmox VM provisioning")
-
adocContent.Should().Contain("nixos.iso")
-
adocContent.Should().Contain("NETSetupProvisionVms.puml")
-
pumlFileExists.Should().BeTrue()
-
pumlContent.Should().Contain("@startuml")
-
pumlContent.Should().Contain("EnsureNixosIso")
-
pumlContent.Should().Contain("qm guest")
-
pumlContent.Should().Contain("scp")
-
pumlContent.Should().Contain("install.sh")
-
2. In Development
2.1. osisa-mikrotik
GOAL: Provide a self-contained MikroTik RouterOS REST API client library (src/osisa.Mikrotik/) mirroring the osisa.OPNSense architecture (CRUD repos + UoW + DataStoreReference + Contracts-only public surface + IMikrotikConverge entry point). Covers broad resource families (bridges, VLANs, interfaces, bonding, IP/DHCP/DNS, firewall NAT+filter, WiFi v7+, user, system). Includes NETSetup-side bridge classes (MikrotikApiCredentials, MikrotikDesiredStateBuilder, MikrotikConvergeBridge) wired into RunProxmoxUpdate for the Liberator VLAN cutover flow. MikroTik REST API uses standard HTTP verbs (GET=print, PUT=add, PATCH=set, DELETE=remove), Basic Auth, paths mirror console menu (/rest/<menu>), all values encoded as strings, resource IDs use *N format.
Decisions:
-
Full CRUD/UoW shape mirroring osisa.OPNSense - same DataStoreReference/UnitOfWork/Repository pattern for future osisa.Rest.Repository migration path
-
Project lives in netsetup repo:
src/osisa.Mikrotik/+src/osisa.Mikrotik.Tests/, added tosrc/NETSetup.sln -
OPNsense-aligned repo method names: GetAllAsync, GetAsync, AddAsync, SetAsync, DeleteAsync
-
WiFi v7+ only (
/interface/wifipackage, not legacy/interface/wireless) - hAP ax2 runs RouterOS 7 only -
Batch-then-push UoW: changes buffered in-memory, CommitAsync pushes all pending PUT/PATCH/DELETE in sequence. No rollback on failure - throw MikrotikException with Applied[] + Failed[] delta
-
Entity base tracks both
.id(*Nformat) andName(nullable) for human-readable converge diffs -
Typed C# properties with custom MikrotikStringConverter (System.Text.Json) handling MikroTik’s "all values are strings" encoding (string to bool/int/long deserialization)
-
Auth via env file
/etc/netsetup/mikrotik-api.env(MIKROTIK_URL, MIKROTIK_USER, MIKROTIK_PASSWORD) - same KEY=VALUE pattern as OPNsense -
MikrotikDesiredState is flat typed DTO with arrays per resource family (BridgeSpec[], BridgePortSpec[], etc.)
-
v1 MikrotikDesiredStateBuilder populates full Liberator VLAN cutover: bridges, VLANs, ports, WiFi SSIDs, DHCP disable, NAT disable, bonding, system identity
-
Converge safety guard: only delete entries matching desired state or prefixed with "netsetup-" - never delete user-created resources
-
Contracts-only public surface: NETSetup sees only IMikrotikConverge + MikrotikConnection + MikrotikDesiredState + *Spec records. All repo/entity/UoW types are internal (InternalsVisibleTo osisa.Mikrotik.Tests)
-
MikroTik REST API reference: https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API
Status (2026-05-12, commits 77a43f1e + 4f94c1b9 + 03316fc6 + be41eafe + d9af2b05 + 311e7421 + 94c3e555 + 5525a0a7 + 7c0b9064 + 848a97d5 on feature/proxmoxvms, mikrotik feature COMPLETE: all 9 RALPH tasks landed, 249 mikrotik tests green = 213 in osisa.Mikrotik.Tests + 36 NETSetup-side helper tests (10 MikrotikApiCredentialsTests + 18 MikrotikDesiredStateBuilderTests + 8 MikrotikConvergeBridgeTests). Plus 17 PhysicalMikrotikIntegrationTests gated on real hardware (not in CI). See mikrotik_feature_summary.md for the architecture-at-a-glance.):
-
Scaffold landed:
src/osisa.Mikrotik/+src/osisa.Mikrotik.Tests/wired intosrc/NETSetup.sln.osisa.Mikrotik.csprojtargets$(DefaultNetTargetFramework), pullsMicrosoft.Extensions.DependencyInjection.Abstractions+Microsoft.Extensions.Logging.Abstractions+osisa.Validation, exposes internals via<InternalsVisibleTo Include="osisa.Mikrotik.Tests" />. -
Public surface (
namespace osisa.Mikrotik.Contracts):MikrotikConnection,MikrotikDesiredState,IMikrotikConverge,MikrotikConvergeResult,MikrotikResourceDelta, plus the 16*Specrecords (BridgeSpec,BridgePortSpec,BridgeVlanSpec,InterfaceSpec,BondingSpec,IpAddressSpec,IpRouteSpec,DhcpServerSpec,DhcpLeaseSpec,DnsSpec,FirewallNatRuleSpec,FirewallFilterRuleSpec,WifiConfigurationSpec,WifiSecuritySpec,UserSpec,SystemIdentitySpec). DI:osisa.Mikrotik.MikrotikServiceCollectionExtensions.AddMikrotik(IServiceCollection). Exception:osisa.Mikrotik.Exceptions.MikrotikException(carriesStatusCode + ErrorMessage + Detail?, message format"MikroTik {status}: {message}"or"MikroTik {status}: {message} ({detail})"). -
Transport (
Client/MikrotikHttpClient, internal sealedIDisposable): real impl. Basic auth"Basic " + base64(user + ":" + pw)exposed on internalAuthorizationHeadertest seam.HttpClient.BaseAddress = BaseUrl + "/", paths built via"rest/" + path.TrimStart('/'). Verbs:GetAsync<T>(list,[]on empty),GetSingleAsync<T>,PutAsync<T>(add),PatchAsync(path,id,body),DeleteAsync(path,id),PostAsync<T>(action endpoints). 4xx-5xx responses parse{error,message,detail}→MikrotikException; non-JSON body keepsReasonPhraseas message. Self-signed cert skip viaHttpClientHandler.ServerCertificateCustomValidationCallback(default on). SharedJsonSerializerOptionswithPropertyNamingPolicy=null+DefaultIgnoreCondition.WhenWritingNull+MikrotikStringConverter. Test-only constructor accepts a pre-wiredHttpClientforFakeHttpMessageHandler. -
JSON (
Json/MikrotikStringConverter, internal sealedJsonConverterFactory): real impl.CanConvertclaims exactlybool | int | long | string.CreateConverteryields per-type nested converters. Read tolerates BOTH the MikroTik all-strings form ("true","42") AND raw JSON values (true,42) - resilient to per-version drift. Write ALWAYS emits a quoted JSON string (writer.WriteStringValue); numeric values formatted withInvariantCultureto avoid German-locale thousands-separator regressions. String converter coerces rawTrue/False/Numbertokens into"true"/"false"/decimal-string so MikroTik fields that occasionally come back as a raw number (e.g.pvid) still bind to astring?property. -
Entity base (
Entities/MikrotikEntity.cs, internal abstract): real impl. Two writable string properties -[JsonPropertyName(".id")] string? Id(literal MikroTik wire key".id", dot-prefixed - non-negotiable) and[JsonPropertyName("name")] string? Name. Both nullable - new entities pre-PUT haveId == null, families without a name field leaveName == null. Concrete entities (task 4) inherit this base + add family-specificstring?/bool?/int?/long?properties with[JsonPropertyName("kebab-case")]where MikroTik’s wire key diverges from C# PascalCase. -
Repository base (
Repositories/MikrotikRepositoryBase<T> : IMikrotikRepository<T> where T : MikrotikEntity, internal abstract): real impl. Concrete subclasses only overrideabstract string ResourcePath(e.g."interface/bridge").GetAllAsyncdelegatesHttpClient.GetAsync<T>(ResourcePath, ct);GetAsync(id, ct)→HttpClient.GetSingleAsync<T>(ResourcePath + "/" + id, ct).Add/Set/DeletebufferPendingOperationrecords into aList<PendingOperation>;IsDirty == Pending.Count > 0.FlushAsync(ct)iterates pending ops in insertion order:Add → PutAsync(returns saved entity,Applied[]capturessaved.Id ?? ""),Set → PatchAsync(ResourcePath, op.Id, body),Delete → DeleteAsync(ResourcePath, op.Id). Per-optry/catchoverExceptioncaptures intoMikrotikFlushFailure(Id, Error);Pending.Clear()runs unconditionally after the loop. Result typeMikrotikFlushResult(IReadOnlyList<string> Applied, IReadOnlyList<MikrotikFlushFailure> Failed). Pending op carriesPendingOperationKindenum (Add | Set | Delete) + id (empty string for Add until device assigns) + nullableobject?body. No retry, no rollback - caller decides via Applied/Failed split.GetAllAsync+Add+Set+Deletearevirtualso singleton + read-only subclasses can override. -
Entities + repositories (17 resource families): all
internal sealed. Entities inheritMikrotikEntityand decorate properties with[JsonPropertyName("kebab-case")]where the MikroTik wire key diverges. The catalog (with REST paths + kebab keys + repo kind):Bridge(interface/bridge,vlan-filtering,protocol-mode);BridgePort(interface/bridge/port,bridge,interface,pvid,tagged);BridgeVlan(interface/bridge/vlan,bridge,tagged,untagged,vlan-ids);RouterInterface(interface- renamed to dodge the C# keyword clash;type,default-name,disabled,running,mac-address);Bonding(interface/bonding,slaves,mode,transmit-hash-policy,lacp-rate);IpAddress(ip/address,address,network,interface,disabled,comment);IpRoute(ip/route,dst-address,gateway,distance,comment);DhcpServer(ip/dhcp-server,interface,disabled,address-pool);DhcpLease(ip/dhcp-server/lease,address,mac-address,comment,disabled);DnsSettingsSINGLETON (ip/dns,servers,allow-remote-requests);FirewallNatRule(ip/firewall/nat,chain,action,out-interface,in-interface,in-interface-list,disabled,comment);FirewallFilterRule(ip/firewall/filter,chain,action,protocol,dst-port,in-interface,in-interface-list,comment,disabled);WifiConfiguration(interface/wifi,ssid,country,channel,security);WifiSecurity(interface/wifi/security,authentication-types,passphrase);User(user,password,group);SystemIdentitySINGLETON (system/identity, hostname via inheritedName);SystemResourceSINGLETON + READ-ONLY (system/resource,platform,board-name,version,uptime,free-memory : long?,total-memory : long?-Add/Set/DeletethrowNotSupportedException). Singleton repos overrideGetAllAsyncto wrap the single object in a one-entry array. Repository <→ Contracts spec name drift to bridge in the mapper (task 6):Bridge.Protocol → protocol-modevs specVlanFiltering(spec keeps only Name + VlanFiltering);BridgePort.Tagged(entity, literal wire key) vsBridgePortSpec.TaggedVlans(caller-friendly semantic). -
Unit of work (
UnitOfWork/IMikrotikUnitOfWork+MikrotikUnitOfWork, internal): real impl. Exposes oneIMikrotikRepository<T>getter per resource family (17 props), all backed by a singleMikrotikHttpClientinjected in the ctor.CommitAsync(ct)iterates the 17 repos in declaration order via a privateFlushIfDirtyAsync<T>helper: skips repos with!IsDirty, otherwise awaitsFlushAsyncand appends to the result list. Sequential (never parallel - shared HttpClient + MikroTik’s REST engine misorders rapid PATCH bursts). Caller is responsible for dependency ordering inside the desired state (bridges before bridge ports, etc.). Result isMikrotikCommitResult(IReadOnlyList<MikrotikFlushResult> Results)withHasFailures ⇒ Results.Any(r ⇒ r.Failed.Count > 0). EmptyResults(no repos were dirty) != failure. -
Data store + reference (
DataStore/, all internal): real impl.MikrotikDataStore : IMikrotikDataStore, IDisposableis the canonical HttpClient owner - production ctor(MikrotikConnection)builds a freshMikrotikHttpClient(connection, NullLogger.Instance)+ aMikrotikUnitOfWork(httpClient)and disposes the HttpClient onDispose()(idempotent via_disposedflag). Internal test-seam ctor accepts a pre-built HttpClient forFakeHttpMessageHandler.MikrotikDataStoreReference : IMikrotikDataStoreReferencecarries aMikrotikConnectionand aFunc<MikrotikConnection, MikrotikHttpClient>factory (defaults tonew MikrotikHttpClient(connection, NullLogger.Instance); internal ctor lets tests inject).ExistsAsync(ct)probesGET /rest/system/resourcevia a throw-away HttpClient (using var http = _httpFactory(_connection)), returnstrueon any successful deserialization,falseon anyException(auth failure, timeout, JSON parse, cancellation - all reduce to "not reachable").OpenDataStore()builds a fresh HttpClient +MikrotikDataStore(caller owns disposal).MikrotikDataStoreReferenceProvideris a stateless factory returning a new reference per call. Provider’sGetReferencereturn type tightened fromobjecttoIMikrotikDataStoreReferencein this commit. -
Entity mapper (
Mapping/MikrotikEntityMapper, internal static): real impl. 32 static methods - oneToEntity(XxxSpec)+ oneToSpec(Xxx entity)per family for 16 spec families (SystemResource excluded - read-only, noSpeccounterpart). Mechanical copy-property-to-property, no business logic.ToEntityprojects spec record positionals to entity nullable properties as-is.ToSpeccollapses entity nullables to spec record positionals viaentity.X ?? string.Empty/entity.X ?? false/entity.Pvid ?? 0(spec records are non-nullable positional records). The notable bridge:BridgePortSpec.TaggedVlans<→BridgePort.Tagged(semantic spec name vs literal wire-key entity property).Bridge.Protocolis NOT inBridgeSpecand does not round-trip - the mapper writes nothing for it; future extension is additive. -
Converge orchestrator (
MikrotikConverge, internal sealed: IMikrotikConverge): real impl, replacing the scaffold-timeNotImplementedExceptionstub. Ctor takesIMikrotikDataStoreReferenceProvider + ILogger<MikrotikConverge>.RunAsyncflow: (1)Precondition.NotNullon connection + desired; (2) reachability gate viareference.ExistsAsync(ct)→ if false, log warning + returnMikrotikConvergeResult(Reachable=false, Deltas=Array.Empty); (3)using var store = reference.OpenDataStore()→ 16-repo UoW; (4) 16RunResourceAsync(label, () ⇒ ConvergeXxxAsync(…))calls in fixed declaration order (Bridge → BridgePort → BridgeVlan → Interface → Bonding → IpAddress → IpRoute → DhcpServer → DhcpLease → FirewallNatRule → FirewallFilterRule → WifiConfiguration → WifiSecurity → User → Dns → SystemIdentity), each captured into thedeltaslist; (5)uow.CommitAsync(ct)wrapped in try/catch (warn + swallow on failure); (6) returnMikrotikConvergeResult(true, deltas.ToArray())with all 16 entries.RunResourceAsync<T>is the per-resource isolation seam: catches everyException, logs warning with the family label, returnsMikrotikResourceDelta(label, 0, 0, 0)so failures appear as zero-delta entries (3-layer failure pattern: per-op viaFlushAsync.Failed, per-family viaRunResourceAsync, global commit via the outer try/catch). Match keys per family:Namefor Bridge/Interface/Bonding/Wifi*/User/DhcpServer; compositeBridge+Interfacefor BridgePort; compositeBridge+VlanIdsfor BridgeVlan;Addressfor IpAddress; compositeDstAddress+Gatewayfor IpRoute;MacAddress(case-insensitive!) for DhcpLease; compositeChain+Commentfor Firewall*; FirstOrDefault for Dns/SystemIdentity singletons. Delete safety guard viaprivate const string NetsetupPrefix = "netsetup-"+IsNetsetupManaged(label): only 5 families have a delete loop (Bridge, Bonding, IpAddress, FirewallNatRule, FirewallFilterRule) and they only delete entries whoseNameorCommentstarts withnetsetup-- manually-created MikroTik resources NEVER get touched. Singleton converges (ConvergeDnsAsync+ConvergeSystemIdentityAsync) treatnulldesired spec as a no-op zero-delta (so non-Liberator hosts can leaveMikrotikDesiredState.Dns = nullandSystemIdentity = nulland the family is visited but does nothing). Interface and DhcpServer converges are Set-only (no Add path - device boots with a fixed set, converge togglesDisabled). IpRoute, FirewallFilterRule, WifiSecurity, User are Add-only (no Set, no Delete - reflects Liberator cutover semantics). -
osisa.Mikrotik library is FEATURE-COMPLETE for adoc tasks 1-6 (Contracts + transport + JSON + entity catalog + UoW/DataStore + mapper + converge). NETSetup-side tasks 7 (credentials parser) + 8 (desired-state builder) + 9 (converge bridge + EnsureCustomerVms wire-in) are also done. All 9 RALPH tasks landed - feature COMPLETE. Single known follow-up: the task 9 wire-in chose
EnsureCustomerVms.Run(paired withOpnsenseConvergeBridge.Runat lines 177-182) instead of the adoc-spec "explicit Phase 3e step 4 inRunProxmoxUpdate`" - operationally equivalent today since `EnsureCustomerVmsruns insideRunProxmoxUpdate; revisit if/when explicitConfigureProxmoxBridges+RewireVmNicsToVlanshelpers are added. -
NETSetup credentials parser (
src/NETSetup/Helpers/MikrotikApiCredentials.cs,public static): real impl.public const string ApiEnvPath = "/etc/netsetup/mikrotik-api.env".TryLoad()→MikrotikConnection?reads the env file via(AbsolutePath)ApiEnvPath+path.FileExists()/path.ReadAllText()(osisa IO conventions); returnsnullif the file is missing.Parse(string)→MikrotikConnection?is a pure helper: empty/whitespace content returnsnull; per-line iteration overSplit('\n', RemoveEmptyEntries)withTrim()strips CRLF, skip-on-blank-or-#, split on first=, value getsTrim('"')to supportKEY="value"shell-style env files. Recognized keys:MIKROTIK_URL,MIKROTIK_USER,MIKROTIK_PASSWORD. Unknown keys silently ignored (forward-compat). Missing required key THROWSNETSetupException("<KEY> missing in mikrotik-api.env")- intentional divergence fromOpnsenseApiCredentials(returns null on incomplete) because a present-but-malformed file means flash-mikrotik wrote it wrong, which is a real bug to surface. The resultingMikrotikConnection.AllowSelfSignedCertificatekeeps its defaulttrue. NETSetup.csproj has aProjectReferencetoosisa.Mikrotik.csprojenabling Contracts visibility. 10/10 tests green inMikrotikApiCredentialsTestscovering valid parse + 3 missing-key throws + comments + quoted values + empty content + nonexistent file + unknown extra keys. -
NETSetup converge bridge (
src/NETSetup/Helpers/MikrotikConvergeBridge.cs,public static): real impl, FINAL task.public static void Run(ISystemHost host, NETSetupConfig config, IServiceProvider? services = null). Synchronous (.GetAwaiter().GetResult()onIMikrotikConverge.RunAsync) to match the synchronousEnsureCustomerVmsflow. Best-effort: every failure path log + swallow, NEVER throws past the catch-allExceptionhandler. Phases: (1)ArgumentNullExceptionon null host/config; (2)CredentialsProvider()(defaults toMikrotikApiCredentials.TryLoad,internal static Func<MikrotikConnection?>test seam) - null → info log "creds not found" + return; (3)LiberatorGate.IsFullLiberatorFlow(host, config) ? "Liberator" : ""+MikrotikDesiredStateBuilder.Build(config, hostDescription); (4) privateIsEmptyState4-property heuristic (Bridges.Length == 0 && SystemIdentity is null && WifiConfigurations.Length == 0 && Users.Length == 0) → info log "skipping … empty desired state" + return; (5)ResolveConverge(services)(DI lookup → fallback to self-builtServiceCollectionwithAddLogging() + AddMikrotik()ifIMikrotikConvergenot registered); (6) inside try:converge.RunAsync(connection, desired).GetAwaiter().GetResult()→!result.Reachable→ warn "unreachable" + return, otherwise iterateresult.Deltasinfo-logging"MikroTik converge {Resource}: +{Added}/~{Updated}/-{Deleted}"; (7) catch-allException→ warn "converge threw - continuing" with exception attached. TheLiberatorGatecheck lives INSIDE the bridge body (caller minimalism -EnsureCustomerVmscalls unconditionally). Wired intosrc/NETSetup/Helpers/EnsureCustomerVms.csat lines 177-182, immediately afterOpnsenseConvergeBridge.Run, NOT insideRunProxmoxUpdateas the adoc spec suggested - the two REST-converge bridges paired side-by-side is more discoverable, and operational cadence is identical becauseEnsureCustomerVmsitself runs insideRunProxmoxUpdate. 8/8 tests green covering null-creds skip, non-Liberator empty-state skip, unreachable warn, exception swallow, DI-vs-fallback ResolveConverge. -
NETSetup desired-state builder (
src/NETSetup/Helpers/MikrotikDesiredStateBuilder.cs,public static): real impl. Pure mapper, zero IO.public const string ManagedPrefix = "netsetup-"(matches delete-safety prefix) +public const string DefaultIdentityName = "Lib_AP".Build(NETSetupConfig config, string hostDescription)→MikrotikDesiredState. Non-Liberator (anything other than literal"Liberator"withStringComparison.Ordinal) returnsnew MikrotikDesiredState()= all-empty arrays + null singletons (no-op converge). Liberator branch emits the recipe:Bridges = [BridgeSpec("bridge1", "yes")](VLAN filtering on, non-prefixed well-known name);BridgePorts4 entries (ether2 + ether3 trunk withPvid=1+ tagged"10,20,250,254"; ether4 + ether5 access withPvid=20+ empty tagged);BridgeVlans4 entries (VLAN 10/20/250/254 on bridge1, tagged on ether2+ether3, VLAN 20 also untagged on ether4+ether5);IpAddresses = [IpAddressSpec("10.0.250.1/24", "bridge1", "netsetup-mgmt")];DhcpServers + FirewallNatRules + FirewallFilterRulesempty (OPNsense owns DHCP/NAT/filter);WifiConfigurations3 entries on wifi1/wifi2/wifi3 for SSIDsLiberator+Liberator-Guest+admin-oob-lib, countryCH, channel2ghz-onlyn,5ghz-ax/2ghz-onlyn/5ghz-ax, eachSecurityfield referencing aWifiSecuritySpec.Name;WifiSecurities3 entries with PSK profiles (wpa2-psk,wpa3-pskfor Liberator + open for Guest +wpa3-pskfor admin-oob), passwords fromconfig.LiberatorWifiPassword+config.MikrotikAdminPassword(null → empty placeholder);Users = [UserSpec("admin", adminPassword, "full")];SystemIdentity = new SystemIdentitySpec(config.SelectedHostname ?? "Lib_AP"). New NETSetupConfig fields (added in this commit):LiberatorWifiPassword+MikrotikAdminPassword(bothpublic string?; latter doubles as admin user pw AND admin-oob-lib WPA3 PSK - single source of truth). Adoc-spec drift: spec mentionedMachineName.Value, impl uses the realNETSetupConfig.SelectedHostnamefield. 18/18 tests green. -
UPDATE (2026-05-29, dual-WAN inbound): the desired-state builder no longer leaves
FirewallNatRules + FirewallFilterRulesempty for inbound published services - it now emits per-ServedPortnetsetup-dnat-{port}dst-nat +netsetup-allow-{port}forward-accept rules. New constMikrotikDesiredStateBuilder.WanInterfaceList = "WAN". Those rules setInInterfaceList = "WAN"with EMPTYInInterface(NOT literalether1). Reason: behind an XGS-PON modem the WAN arrives tagged on VLAN 10 so ingress isether1-vlan10, notether1; an ether1-pinned port-forward would never match and inbound mail/nextcloud would be dead. Matching the WAN interface-list (which holds BOTHether1+ether1-vlan10) fires the rule on whichever iface carries the WAN. osisa.Mikrotik gainedInInterfaceListonFirewallNatRuleSpec+FirewallFilterRuleSpec(+ entities + mapper + NAT drift-check). DEPENDENCY: the converge REFERENCES theWANlist but does NOT create it - it relies on the flash-mikrotik.rsc(MikrotikRscBuilder, new constWanVlanId = "10") having created theWANinterface-list + both members + a dhcp-client on each iface. Flash-then-update is the normal order. (OPNsense still owns the per-VLAN DHCP + outbound NAT + inter-VLAN filter; only the inbound published-service port-forwards live on the Mikrotik.) -
Tests: 213 green (42 in `94c3e555`). Contracts + records (Connection, DesiredState, ConvergeResult, ResourceDelta, all 16 spec records, exception) + consolidated `MikrotikContractsTests` for DI + serialization shape. `MikrotikHttpClientTests` covers Basic auth header, 4xx/5xx -> exception mapping, JSON read-write round-trip, self-signed-cert toggle. `MikrotikStringConverterTests` covers raw-and-quoted Read and always-quoted Write per primitive. `MikrotikEntityTests` covers `.id` + `name` JSON round-trip and nullable defaults. `MikrotikRepositoryBaseTests` covers IsDirty transitions, GetAll/Get routing, FlushAsync verb dispatch (Add->PUT/Set->PATCH/Delete->DELETE), partial-fail capture (continue on per-op exception, Applied vs Failed split), Pending unconditional clear, empty-batch no-op. `MikrotikFlushResultTests` covers the result + failure record shapes. `MikrotikRepositoriesTests` (+32 in `d9af2b05`) asserts 17 `ResourcePath` strings + per-family entity JSON deserialization through `MikrotikStringConverter` (kebab-case keys, bool/long round-trip) + singleton-wrap behaviour for Dns/SystemIdentity/SystemResource + read-only `NotSupportedException` guard for SystemResource Add/Set/Delete. `MikrotikUnitOfWorkTests` ( in
311e7421) covers 17-repo getter non-null + CommitAsync with 0/1/3 dirty repos + sequential flush order + HasFailures propagation.MikrotikCommitResultTestscoversHasFailurescomputation across mixed inner outcomes.MikrotikDataStoreReferenceTestscoversExistsAsynctrue on 200, false on refused/401/timeout/non-JSON,OpenDataStorereturns a usable store, factory seam exercised.MikrotikServiceCollectionExtensionsTestsassertsAddMikrotikregistersIMikrotikConverge+IMikrotikDataStoreReferenceProvideras singletons (idempotent on multiple calls).MikrotikEntityMapperTests(+ in94c3e555) coversToEntity/ToSpecround-trip per family + null projection (string → "", bool → false, int → 0) + theBridgePortSpec.TaggedVlans<→BridgePort.Taggeddrift bridge.MikrotikConvergeTests(+ in94c3e555) covers reachability-false short-circuit, golden-path mixed Add/Set/Delete, per-resource failure isolation (one family throws, others still run with correct deltas),netsetup--prefix delete safety, singleton Dns/SystemIdentity Set behaviour,nulldesired Dns = no-op zero-delta, all 16 families appear in the result.PhysicalMikrotikIntegrationTests(17 tests,Skip = "Requires a physical MikroTik device.") runs manually against a real hAP ax2 during commissioning - NOT executed in CI.TestInfrastructure/FakeHttpMessageHandler.csroute-keyed (mirrorsosisa.OPNSense.Tests);TestInfrastructure/TestValues.cshosts shared constants. (Test agent renamed the auth-header property assertionAuthHeader→AuthorizationHeaderto match the shipped name.)-
✓ Create project scaffold +
MikrotikHttpClient+MikrotikStringConverter+MikrotikException: Createsrc/osisa.Mikrotik/osisa.Mikrotik.csproj(net10,<InternalsVisibleTo Include="osisa.Mikrotik.Tests" />) andsrc/osisa.Mikrotik.Tests/osisa.Mikrotik.Tests.csproj(MSTest + FluentAssertions). Add both tosrc/NETSetup.sln. CreateClient/MikrotikHttpClient.cs(internal) wrappingHttpClientwith: Basic auth header (Authorization: Basic base64(username:password)), self-signed cert skip viaHttpClientHandler.ServerCertificateCustomValidationCallback, typed methodsGetAsync<T>(string path)returningT(GET/rest/{path}),GetSingleAsync<T>(string path)for singleton resources,PutAsync<T>(string path, T entity)returningTwith.id(PUT/rest/{path}),PatchAsync(string path, string id, object body)(PATCH/rest/{path}/{id}),DeleteAsync(string path, string id)(DELETE/rest/{path}/{id}),PostAsync<T>(string path, object body)returningT(POST/rest/{path}). Error responses (HTTP 4xx/5xx) parsed from JSON{"error": N, "message": "…", "detail": "…"}intoMikrotikException. CreateJson/MikrotikStringConverter.cs- customSystem.Text.Json.Serialization.JsonConverterfactory handling MikroTik’s all-strings encoding: deserializes"true"/"false"tobool, numeric strings toint/long, preservesstringpassthrough. Registered on the sharedJsonSerializerOptionsused byMikrotikHttpClient. CreateExceptions/MikrotikException.cswithint StatusCode,string ErrorMessage,string? Detail. Createsrc/osisa.Mikrotik.Tests/TestInfrastructure/FakeHttpMessageHandler.cs(same pattern as osisa.OPNSense.Tests) with route-keyed response dictionary. Assertions: -
httpClient.AuthHeader.Should().Be("Basic " + Convert.ToBase64String(Encoding.UTF8.GetBytes(SomeUser + ":" + SomePassword)))
-
getResult.Should().HaveCount(ExpectedCount)
-
getResult[0].Id.Should().Be(SomeEntityId)
-
putResult.Id.Should().NotBeNullOrEmpty()
-
act.Should().ThrowAsync<MikrotikException>().Where(e ⇒ e.StatusCode == 404 && e.ErrorMessage == SomeErrorMessage)
-
MikrotikStringConverter deserializing "true" .Should().Be(true)
-
MikrotikStringConverter deserializing "42" .Should().Be(42)
-
MikrotikStringConverter deserializing "ether1" .Should().Be("ether1")
-
MikrotikStringConverter serializing true .Should().Be("true")
-
MikrotikStringConverter serializing 42 .Should().Be("42")
-
✓ Create Contracts public surface types: Create
Contracts/namespace insrc/osisa.Mikrotik/with all public types.MikrotikConnectionrecord:string BaseUrl,string Username,string Password,bool AllowSelfSignedCertificate = true.IMikrotikConvergeinterface:Task<MikrotikConvergeResult> RunAsync(MikrotikConnection connection, MikrotikDesiredState desired, CancellationToken ct).MikrotikConvergeResultrecord:bool Reachable,MikrotikResourceDelta[] Deltas.MikrotikResourceDeltarecord:string ResourceType,int Added,int Updated,int Deleted. Desired state spec records (all inContracts/):BridgeSpec(string Name, string VlanFiltering),BridgePortSpec(string Bridge, string Interface, int Pvid, string TaggedVlans),BridgeVlanSpec(string Bridge, string Tagged, string Untagged, string VlanIds),InterfaceSpec(string Name, bool Disabled),BondingSpec(string Name, string Slaves, string Mode, string TransmitHashPolicy, string LacpRate),IpAddressSpec(string Address, string Interface, string Comment),IpRouteSpec(string DstAddress, string Gateway, string Comment),DhcpServerSpec(string Name, bool Disabled),DhcpLeaseSpec(string Address, string MacAddress, string Comment),DnsSpec(string Servers, bool AllowRemoteRequests),FirewallNatRuleSpec(string Chain, string Action, string OutInterface, bool Disabled, string Comment),FirewallFilterRuleSpec(string Chain, string Action, string Protocol, string DstPort, string InInterface, string Comment, bool Disabled),WifiConfigurationSpec(string Name, string Ssid, string Country, string Channel, string Security),WifiSecuritySpec(string Name, string AuthenticationTypes, string Passphrase),UserSpec(string Name, string Password, string Group),SystemIdentitySpec(string Name).MikrotikDesiredStaterecord: typed array properties for each spec (BridgeSpec[] Bridges,BridgePortSpec[] BridgePorts,BridgeVlanSpec[] BridgeVlans,InterfaceSpec[] Interfaces,BondingSpec[] Bondings,IpAddressSpec[] IpAddresses,IpRouteSpec[] IpRoutes,DhcpServerSpec[] DhcpServers,DhcpLeaseSpec[] DhcpLeases,DnsSpec? Dns,FirewallNatRuleSpec[] FirewallNatRules,FirewallFilterRuleSpec[] FirewallFilterRules,WifiConfigurationSpec[] WifiConfigurations,WifiSecuritySpec[] WifiSecurities,UserSpec[] Users,SystemIdentitySpec? SystemIdentity).MikrotikServiceCollectionExtensions.AddMikrotik(this IServiceCollection services)registeringIMikrotikConverge+ provider as singletons. Assertions: -
new MikrotikConnection(SomeUrl, SomeUser, SomePassword).BaseUrl.Should().Be(SomeUrl)
-
new MikrotikConnection(SomeUrl, SomeUser, SomePassword).AllowSelfSignedCertificate.Should().BeTrue()
-
new MikrotikDesiredState with empty arrays .Bridges.Should().BeEmpty()
-
new BridgeSpec(SomeBridgeName, SomeVlanFiltering).Name.Should().Be(SomeBridgeName)
-
new MikrotikConvergeResult(true, Array.Empty<MikrotikResourceDelta>()).Reachable.Should().BeTrue()
-
new MikrotikResourceDelta(SomeResourceType, 1, 2, 0).Added.Should().Be(1)
-
✓ Create
MikrotikEntitybase +MikrotikRepositoryBase<T>with batch buffering: CreateEntities/MikrotikEntity.cs(internal) - base class with[JsonPropertyName(".id")] string? Id { get; set; }andstring? Name { get; set; }. All MikroTik entity types will inherit from this. CreateRepositories/IMikrotikRepository.cs(internal) with:Task<IReadOnlyList<T>> GetAllAsync(CancellationToken ct),Task<T> GetAsync(string id, CancellationToken ct),void Add(T entity)(buffers),void Set(string id, T entity)(buffers),void Delete(string id)(buffers),Task<MikrotikFlushResult> FlushAsync(CancellationToken ct),bool IsDirty. CreateRepositories/MikrotikRepositoryBase.cs(internal abstract): stores pending operations in aList<PendingOperation>(discriminated union of Add/Set/Delete),GetAllAsyncdelegates toMikrotikHttpClient.GetAsync<List<T>>(ResourcePath),GetAsyncdelegates toMikrotikHttpClient.GetSingleAsync<T>(ResourcePath + "/" + id), Add/Set/Delete append to pending list,FlushAsynciterates pending ops pushing via HttpClient (PUT for adds, PATCH for sets, DELETE for deletes) collectingApplied[]andFailed[]intoMikrotikFlushResult, clears pending on success. Abstractstring ResourcePath { get; }. CreateRepositories/MikrotikFlushResult.cs(internal):IReadOnlyList<string> Applied,IReadOnlyList<(string Id, Exception Error)> Failed. Assertions: -
entity.Id.Should().Be(SomeEntityId)
-
entity.Name.Should().Be(SomeEntityName)
-
repo.IsDirty.Should().BeFalse() initially
-
repo.Add(someEntity); repo.IsDirty.Should().BeTrue()
-
getAllResult.Should().HaveCount(ExpectedCount) (via FakeHttpMessageHandler)
-
getResult.Id.Should().Be(SomeEntityId)
-
flushResult.Applied.Should().HaveCount(ExpectedAppliedCount) after Add+Set+Delete
-
flushResult.Failed.Should().BeEmpty() on success
-
repo.IsDirty.Should().BeFalse() after successful FlushAsync
-
flushResult on mid-batch failure: flushResult.Applied.Should().HaveCount(SomePartialCount) and flushResult.Failed.Should().HaveCount(1)
-
✓ Create all concrete entity types + repository implementations for 17 MikroTik resource families: In
Entities/create internal entity classes inheritingMikrotikEntity, each with typed properties (viaMikrotikStringConverter) matching the MikroTik REST API JSON fields.Bridge.cs:string? VlanFiltering,string? Protocol.BridgePort.cs:string? Bridge,string? Interface,int? Pvid,string? TaggedVlans(mapped fromtagged).BridgeVlan.cs:string? Bridge,string? Tagged,string? Untagged,string? VlanIds(mapped fromvlan-ids).RouterInterface.cs:string? Type,string? DefaultName,bool? Disabled,string? Running,string? MacAddress.Bonding.cs:string? Slaves,string? Mode,string? TransmitHashPolicy(mapped fromtransmit-hash-policy),string? LacpRate(mapped fromlacp-rate).IpAddress.cs:string? Address,string? Network,string? Interface,bool? Disabled,string? Comment.IpRoute.cs:string? DstAddress(mapped fromdst-address),string? Gateway,string? Distance,string? Comment.DhcpServer.cs:string? Interface, bool? Disabled`,string? AddressPool.DhcpLease.cs:string? Address,string? MacAddress(mapped frommac-address),string? Comment,bool? Disabled.DnsSettings.cs:string? Servers,bool? AllowRemoteRequests(mapped fromallow-remote-requests).FirewallNatRule.cs:string? Chain,string? Action,string? OutInterface(mapped fromout-interface),bool? Disabled,string? Comment.FirewallFilterRule.cs:string? Chain,string? Action,string? Protocol,string? DstPort(mapped fromdst-port),string? InInterface(mapped fromin-interface),string? Comment,bool? Disabled.WifiConfiguration.cs:string? Ssid,string? Country,string? Channel,string? Security.WifiSecurity.cs:string? AuthenticationTypes(mapped fromauthentication-types),string? Passphrase.User.cs:string? Password,string? Group.SystemIdentity.cs(singleton).SystemResource.cs(read-only):string? Platform,string? BoardName,string? Version,string? Uptime,long? FreeMemory,long? TotalMemory. InRepositories/create concrete repos:BridgeRepository(pathinterface/bridge),BridgePortRepository(interface/bridge/port),BridgeVlanRepository(interface/bridge/vlan),InterfaceRepository(interface),BondingRepository(interface/bonding),IpAddressRepository(ip/address),IpRouteRepository(ip/route),DhcpServerRepository(ip/dhcp-server),DhcpLeaseRepository(ip/dhcp-server/lease),DnsSettingsRepository(ip/dns) - override GetAllAsync for singleton (GET returns single object, wrap in array),FirewallNatRuleRepository(ip/firewall/nat),FirewallFilterRuleRepository(ip/firewall/filter),WifiConfigurationRepository(interface/wifi),WifiSecurityRepository(interface/wifi/security),UserRepository(user),SystemIdentityRepository(system/identity) - singleton override,SystemResourceRepository(system/resource) - read-only (Add/Set/Delete throwNotSupportedException). Use[JsonPropertyName("kebab-case")]attributes on entity properties where MikroTik JSON key differs from C# PascalCase. Assertions: -
bridgeRepo.ResourcePath.Should().Be("interface/bridge")
-
ipAddressRepo.ResourcePath.Should().Be("ip/address")
-
firewallNatRepo.ResourcePath.Should().Be("ip/firewall/nat")
-
wifiConfigRepo.ResourcePath.Should().Be("interface/wifi")
-
systemResourceRepo.ResourcePath.Should().Be("system/resource")
-
Bridge deserialized from
{“.id”:"*1","name":"bridge1","vlan-filtering":"true"}: result.Id.Should().Be("*1"), result.Name.Should().Be("bridge1"), result.VlanFiltering.Should().Be("true") -
IpAddress deserialized from sample JSON: result.Address.Should().Be(SomeAddress), result.Interface.Should().Be(SomeInterface)
-
DnsSettingsRepository.GetAllAsync wraps singleton response in array: result.Should().HaveCount(1)
-
SystemResourceRepository.Add should throw NotSupportedException
-
FirewallFilterRule deserialized with
dst-portmapped to DstPort property -
✓ Create UoW + DataStore + DataStoreReference + Provider + DI registration: In
UnitOfWork/createIMikrotikUnitOfWork.cs(internal): exposes all 17 repos as read-only properties (IMikrotikRepository<Bridge> Bridges,IMikrotikRepository<BridgePort> BridgePorts, etc.) +Task<MikrotikCommitResult> CommitAsync(CancellationToken ct). CreateMikrotikCommitResult.cs(internal):IReadOnlyList<MikrotikFlushResult> Results,bool HasFailures. CreateMikrotikUnitOfWork.cs(internal): constructor takesMikrotikHttpClient, instantiates all 17 concrete repos,CommitAsynciterates repos whereIsDirty == true, callsFlushAsyncon each, aggregates intoMikrotikCommitResult. InDataStore/createIMikrotikDataStore.cs(internal):IMikrotikUnitOfWork UnitOfWork { get; },IDisposable. CreateMikrotikDataStore.cs: createsMikrotikHttpClientfromMikrotikConnection, passes toMikrotikUnitOfWork. CreateIMikrotikDataStoreReference.cs(internal):Task<bool> ExistsAsync(CancellationToken ct),IMikrotikDataStore OpenDataStore(). CreateMikrotikDataStoreReference.cs:ExistsAsyncprobesGET /rest/system/resource- returns true on 200, false on any exception (connection refused, timeout, auth failure). CreateIMikrotikDataStoreReferenceProvider.cs(internal):IMikrotikDataStoreReference GetReference(MikrotikConnection connection). CreateMikrotikDataStoreReferenceProvider.cs. In root, updateMikrotikServiceCollectionExtensions.AddMikrotik: registerIMikrotikDataStoreReferenceProvider→MikrotikDataStoreReferenceProvider(singleton),IMikrotikConverge→MikrotikConverge(singleton). Assertions: -
uow.CommitAsync with no dirty repos: result.Results.Should().BeEmpty()
-
uow.Bridges.Add(someBridge); uow.CommitAsync: result.Results.Should().HaveCount(1)
-
uow.CommitAsync with 3 dirty repos: result.Results.Should().HaveCount(3)
-
uow.CommitAsync with one repo failing: result.HasFailures.Should().BeTrue()
-
dataStoreRef.ExistsAsync returns true on 200 response
-
dataStoreRef.ExistsAsync returns false on connection refused
-
provider.GetReference(someConn).Should().NotBeNull()
-
AddMikrotik DI: services.BuildServiceProvider().GetRequiredService<IMikrotikConverge>().Should().NotBeNull()
-
AddMikrotik DI: services.BuildServiceProvider().GetRequiredService<IMikrotikDataStoreReferenceProvider>().Should().NotBeNull()
-
✓ Create
MikrotikEntityMapper+MikrotikConvergeorchestrator: CreateMapping/MikrotikEntityMapper.cs(internal static) with bidirectional mapping methods between Contracts *Spec records and internal entity types. OneToEntity/ToSpecmethod pair per resource family (e.g.,Bridge ToEntity(BridgeSpec spec),BridgeSpec ToSpec(Bridge entity)). Handle nullable fields. CreateMikrotikConverge.cs(internal, implementsIMikrotikConverge):RunAsyncflow: (1)provider.GetReference(connection), (2)ExistsAsync- if false, returnMikrotikConvergeResult(Reachable: false, Deltas: []), (3)OpenDataStore()→ UoW, (4) per resource family:GetAllAsynccurrent state, diff against desired by name/key matching (name for bridges/interfaces/bondings/wifi/users/identity, address for IPs, chain+comment for firewall rules, mac-address for DHCP leases), queueAddfor missing,Setfor changed (compare all properties),Deletefor entries present on device but not in desired state AND (name starts with "netsetup-" OR matches an entry in desired state that was removed - safety guard: never delete user-created resources), (5)uow.CommitAsync(), (6) buildMikrotikConvergeResultwith per-resourceMikrotikResourceDelta, (7) per-resource failures captured as warnings in result (same isolation pattern as osisa.OPNSense) - one resource family failing must not block others. DnsSettings and SystemIdentity are singletons: diff by comparing current vs desired, Set if different, no Add/Delete. Assertions: -
mapper: MikrotikEntityMapper.ToEntity(someBridgeSpec).Name.Should().Be(SomeBridgeName)
-
mapper: MikrotikEntityMapper.ToSpec(someBridgeEntity).VlanFiltering.Should().Be(SomeVlanFiltering)
-
converge unreachable: result.Reachable.Should().BeFalse()
-
converge golden path (add 2 bridges + set 1 IP + delete 1 NAT rule): result.Deltas.Should().Contain(d ⇒ d.ResourceType == "Bridge" && d.Added == 2)
-
converge empty desired state: result.Deltas.Should().AllSatisfy(d ⇒ d.Added == 0 && d.Updated == 0 && d.Deleted == 0)
-
converge per-resource failure: bridges fail, IPs succeed → result still contains IP delta with correct counts
-
converge safety guard: existing "manual-rule" not in desired state → NOT deleted. existing "netsetup-old-rule" not in desired state → deleted
-
converge singleton (SystemIdentity): current "MikroTik" + desired "Lib_AP" → delta Updated == 1
-
✓ Create NETSetup
MikrotikApiCredentialsenv file parser: Createsrc/NETSetup/Helpers/MikrotikApiCredentials.cswith staticParse(string content)returningMikrotikConnection. Reads KEY=VALUE format (one per line,#comments, empty lines skipped):MIKROTIK_URL→BaseUrl,MIKROTIK_USER→Username,MIKROTIK_PASSWORD→Password. Missing required field (URL, USER, PASSWORD) throwsNETSetupExceptionwith field name in message. AddTryReadFromDisk(AbsolutePath path, ILogger logger)→MikrotikConnection?: returns null if file doesn’t exist (info log), otherwise reads + Parse. Env file path constant:/etc/netsetup/mikrotik-api.env. Test class insrc/NETSetup.Tests/HelperTests/MikrotikApiCredentialsTests.cs. Assertions: -
Parse valid content: result.BaseUrl.Should().Be(SomeMikrotikUrl)
-
Parse valid content: result.Username.Should().Be(SomeMikrotikUser)
-
Parse valid content: result.Password.Should().Be(SomeMikrotikPassword)
-
Parse missing MIKROTIK_URL: act.Should().Throw<NETSetupException>().Where(e ⇒ e.Message.Contains("MIKROTIK_URL"))
-
Parse missing MIKROTIK_USER: act.Should().Throw<NETSetupException>().Where(e ⇒ e.Message.Contains("MIKROTIK_USER"))
-
Parse with comments and empty lines: result.BaseUrl.Should().Be(SomeMikrotikUrl)
-
TryReadFromDisk with nonexistent file: result.Should().BeNull()
-
✓ Create NETSetup
MikrotikDesiredStateBuilderfor Liberator VLAN cutover: Createsrc/NETSetup/Helpers/MikrotikDesiredStateBuilder.cswithBuild(NETSetupConfig config, string hostDescription)returningMikrotikDesiredState. ForhostDescription == "Liberator"(role gate from ): Bridges = one "bridge1" with VlanFiltering="yes". BridgePorts = ether2 pvid=1 tagged 10,20,250,254 (trunk to Proxmox), ether3 pvid=1 tagged 10,20,250,254 (trunk to Proxmox), ether4 pvid=20 (LAN access), ether5 pvid=20 (spare LAN). BridgeVlans = VLAN 10,20,250,254 on bridge1 with appropriate tagged/untagged ports. IpAddresses = 10.0.250.1/24 on bridge1 VLAN 250 (OOB/MGMT). DhcpServers = all found servers set Disabled=true (OPNsense handles DHCP). FirewallNatRules = masquerade rule set Disabled=true (OPNsense handles NAT). FirewallFilterRules = empty (OPNsense handles filtering). WifiConfigurations = "Liberator" on wifi1 (2.4+5 GHz) VLAN 20, "Liberator-Guest" on wifi2 open VLAN 254, "admin-oob-lib" hidden WPA3 VLAN 250. WifiSecurities = WPA2/3 PSK for Liberator, WPA3 for admin-oob-lib, none for Guest. SystemIdentity = hostname from config or "Lib_AP". Users = admin withconfig.MikrotikAdminPassword(from flash-mikrotik flashed password). For non-LiberatorhostDescription: returnMikrotikDesiredStatewith all empty arrays and null singletons (no-op converge). WiFi passwords sourced fromNETSetupConfigfields (existing WiFi password config or flash-mikrotik defaults). Test class insrc/NETSetup.Tests/HelperTests/MikrotikDesiredStateBuilderTests.cs. Assertions: -
Liberator: result.Bridges.Should().HaveCount(1)
-
Liberator: result.Bridges[0].Name.Should().Be("bridge1")
-
Liberator: result.Bridges[0].VlanFiltering.Should().Be("yes")
-
Liberator: result.BridgePorts.Should().HaveCount(4) (ether2, ether3, ether4, ether5)
-
Liberator: result.BridgePorts.First(p ⇒ p.Interface == "ether2").Pvid.Should().Be(1)
-
Liberator: result.BridgeVlans.Should().Contain(v ⇒ v.VlanIds == "10")
-
Liberator: result.DhcpServers.Should().AllSatisfy(s ⇒ s.Disabled.Should().BeTrue())
-
Liberator: result.FirewallNatRules.Should().AllSatisfy(r ⇒ r.Disabled.Should().BeTrue())
-
Liberator: result.WifiConfigurations.Should().HaveCount(3)
-
Liberator: result.WifiConfigurations.Should().Contain(w ⇒ w.Ssid == "Liberator")
-
Liberator: result.WifiConfigurations.Should().Contain(w ⇒ w.Ssid == "Liberator-Guest")
-
Liberator: result.WifiConfigurations.Should().Contain(w ⇒ w.Ssid == "admin-oob-lib")
-
Liberator: result.SystemIdentity.Should().NotBeNull()
-
non-Liberator: result.Bridges.Should().BeEmpty()
-
non-Liberator: result.SystemIdentity.Should().BeNull()
-
✓ Create NETSetup
MikrotikConvergeBridge+ wire intoRunProxmoxUpdate: Createsrc/NETSetup/Helpers/MikrotikConvergeBridge.cswithRun(ISystemHost host, NETSetupConfig config, IServiceProvider services, ILogger logger). Best-effort pattern (same asOpnsenseConvergeBridge): (1)MikrotikApiCredentials.TryReadFromDisk- null → info log "no mikrotik credentials" + return, (2)MikrotikDesiredStateBuilder.Build(config, host.Description)- empty desired state (non-Liberator) → info log + return, (3) resolveIMikrotikConvergefromservices(build one-shotServiceCollectionwithAddMikrotik()if not pre-registered), (4)converge.RunAsync(connection, desired, ct), (5) if!result.Reachable→ warn log "MikroTik unreachable at {url}" + return, (6) log each delta, (7) catch all exceptions → warn log + continue (never throw). Wire intosrc/NETSetup/Helpers/UpdateMethods.csRunProxmoxUpdateas step 4 from Liberator design (afterEnsureCustomerVms/OpnsenseConvergeBridge, beforeConfigureProxmoxBridges), gated onrunFullNetworkFlow(host.Description == "Liberator" && hasOpnsenseVm). Test class insrc/NETSetup.Tests/HelperTests/MikrotikConvergeBridgeTests.cs. Assertions: -
missing creds file: should not throw, logger should contain "no mikrotik credentials"
-
non-Liberator host: should not throw, should not call IMikrotikConverge.RunAsync
-
unreachable MikroTik: should not throw, logger should contain "unreachable"
-
successful converge: IMikrotikConverge.RunAsync.Received(1)
-
IMikrotikConverge.RunAsync throws: should not throw (caught), logger should contain warning
-
RunProxmoxUpdate calls MikrotikConvergeBridge.Run when isLiberator && hasOpnsenseVm
-
RunProxmoxUpdate skips MikrotikConvergeBridge.Run when !isLiberator
-
3. In Development
3.1. vm-datastore-v5
GOAL: Virtual machines are handled like records in a database. The virtualisation environment (Hyper-V, Proxmox) is the database provider; a host/scope is the data store; a VM is a record. Adding a VM is an Insert, removing one is a Delete, changing one is an Update - and an Update on a hypervisor means stop → change → restart. The whole surface follows the netbase V5 DataStore schema (DataStoreReferenceBase5 + DependencyInjectionDataStoreReferenceBuilder5 + IDataStore2<TUnitOfWork> + IUnitOfWork + ICanRead/ICanInsert/ICanUpdate/ICanDelete), and the tests follow the netbase test patterns (DataStoreReference5TestBase, UnitOfWorkRepositoryTestsBase3, UnitTestsBase, TestValues Some* constants, TypedTestDataBuilderDataSource).
ACCEPTANCE CRITERION (this is what "done" means): two test classes exist that differ only in the wiring of the provider - HyperVVirtualMachineReadAllTests (local Hyper-V) and ProxmoxVirtualMachineReadAllTests (osisaLib03) - both derive from one shared VirtualMachineReadAllTestBase<TReference, TBuilder> that holds the entire test body, and both deliver the correct VM count for their connection. Every line of scenario logic lives in the base; the derived classes contribute a ConfigureBuilder and a native-API count oracle and nothing else. If a reader can diff the two files and see only the provider wiring, the abstraction is proven.
NON-GOAL / HARD CONSTRAINT: This iteration is purely additive. EnsureCustomerVms, CreateVirtualMachines, ProvisionNixosVm, InstallNixosVm, UpdateNixosVm, ProvisionWindowsVmOnProxmox, ProvisionOpnsenseVm and UpdateMethods.RunProxmoxUpdate keep their current runtime behaviour byte for byte. Nothing in the new layer is called from the production update path in this iteration. The DB semantics become visible in the tests - the new suite is the proof that the model holds - not in a changed netsetup update. Wiring the planner into RunProxmoxUpdate is a deliberate follow-up spec, gated behind a green suite here.
Decisions:
-
Is there already a UnitOfWork pattern here? YES - and it is reused, not reinvented.
ISystemManagementUnitOfWork(ships inosisa.SystemManagement.Windows, built onDataStoreReferenceBase5+DataStoreNameDependencyInjectionDataStoreReferenceBuilder5+CanGetSingleUnitOfWorkDataStoreReferenceBase) is already consumed in this repo - seeNETSetup.Tests/ExeTests/GetTests.csandNETSetup.Tests/UEFIDriverTests.cs:var provider = new MachineScopeProvider(); var dataStoreReference = provider.GetReferences().First(); var dataStore = dataStoreReference.OpenDataStore<ISystemManagementUnitOfWork>(); var unitOfWork = dataStore.GetUnitOfWork(); var repository = unitOfWork.GetRepository<IDisk>(); var disks = repository.ReadAll();That is exactly the shape this spec needs, one level up: the machine scope is the data store,
IDiskis the record,ReadAll()is the query. The VM layer is the same sentence with a Proxmox/Hyper-V data store andIVirtualMachineas the record.MachineScopeProvider3 : IDataStoreReferenceProvider3is the provider precedent to copy. -
Suggested UnitOfWork interface - deliberately mirrors
ISystemManagementUnitOfWorkso the two read identically at the call site:namespace osisa.VirtualMachines.Contracts; /// <summary> /// Unit of work over one virtualisation environment. The environment (Hyper-V scope, /// Proxmox node) is the data store; a virtual machine is a record. /// </summary> public interface IVirtualizationUnitOfWork : IUnitOfWork { /// <summary>Virtual machines in this environment. Insert / Update / Delete are buffered /// until <see cref="IUnitOfWork.Commit"/>.</summary> IVirtualMachineRepository VirtualMachines { get; } } /// <summary>Full-CRUD repository of virtual machines.</summary> public interface IVirtualMachineRepository : IGenericRepository<VirtualMachineSettings> { // inherits ICanRead / ICanInsert / ICanUpdate / ICanDelete<VirtualMachineSettings> }The named
VirtualMachinesproperty is the ergonomic accessor;unitOfWork.GetRepository<VirtualMachineSettings>()MUST resolve the same instance, so both idioms work and the genericISystemManagementUnitOfWorkhabit carries over unchanged. -
AddReference2→ the real API isAddReference3<TBuilder>(Action<TBuilder>)onDataStoreReferenceProvider3.Builder.osisa.Repositoryexposes exactly two:AddReference(Func<IServiceProvider, ICanGetAnyUnitOfWork>)(Shape A) andAddReference3<TBuilder>(Action<TBuilder>)(Shape B). There is noAddReference2. Both providers are registered on one builder, one call each:_inner = new DataStoreReferenceProvider3.Builder(services) .AddReference3<HyperVVmDataStoreReference5.Builder>(b => b .WithDataStoreName(LocalHyperVStoreName) .WithManagementScope(LiveMethods.ConnectToLiveLocalSystem(@"root\virtualization\v2"))) .AddReference3<ProxmoxVmDataStoreReference5.Builder>(b => b .WithDataStoreName(OsisaLib03StoreName) .WithHost(proxmox.GetHost(OsisaLib03StoreName))) .Build();GetReferences()then yields two references, and the sameOpenDataStore3<IVirtualizationUnitOfWork>()→GetUnitOfWork()→VirtualMachines.ReadAll()walk works against both. That walk is the validation the acceptance criterion asks for. -
New projects:
src/osisa.VirtualMachines/+src/osisa.VirtualMachines.Tests/, both added tosrc/NETSetup.sln. Target$(DefaultNetTargetFramework). Test project name is{Source}.Tests- capitalT, plurals- per netbase convention. Precedent to mirror for the library shape:osisa.Mikrotik(Contracts-only public surface, internal repositories/UoW/DataStore,InternalsVisibleTofor its.Tests). -
Provider = environment. Two real providers plus one in-memory provider:
-
HyperVVmDataStoreReference5- a Hyper-VManagementScope(root\virtualization\v2) is one data store. -
ProxmoxVmDataStoreReference5- a Proxmox node (osisa.SystemManagement.Proxmox.Host) is one data store. -
MemoryVmDataStoreReference5- deterministic in-process store. This is what makes the CRUD semantics testable with zero hypervisor, and it is the store the netbaseUnitOfWorkRepositoryTestsBase3CRUD suite runs against.
-
-
Provider constants live in NETSetup, NOT in netbase:
WellKnownDataStoreProvidersshipsSqlite/SqlServer/Memory/Mock/Excel/Postgres/Restonly. AddNetsetupDataStoreProviders(public static class) withHyperV,Proxmox,MemoryVmstring constants, used by[DataStoreProvider(…)]on each reference. -
VirtualMachineSettingsis a projection, not a competitor, ofosisa.SystemManagement.Contracts.Interfaces.IVirtualMachine- the read-model both hypervisor libraries already share. A flat provider-neutral record is what lets one repository, one UoW and one test base span Hyper-V and Proxmox; the native types stay behind the operations seam. -
Record identity: keyed by
SerialNumber({VMID}@{HostSerial}), NOT byMachineName. The serial is the already-existing primary key of the NETSetup VM model (CreateVirtualMachines.IsForHostparses it) and it survives a rename; a name change must read as anUpdate, notDelete+Insert.VirtualMachineKeyis a Vogen VO wrapping the serial string. For a VM discovered on a host that carries no NETSetup serial (hand-created), the key falls back to{VmId}@{HostName}soReadAllnever drops a record it cannot name. -
Change buffering (batch-then-commit, same as
MikrotikUnitOfWork):Insert/Update/Deletemutate an in-memory pending list and flipIsDirty; nothing touches the hypervisor untilCommit().NoCommittherefore genuinely means "the VM was never touched" - that is the assertion that proves the transaction boundary is real.ReadAllis always live and unbuffered. -
Commit ordering is fixed and asserted: Delete first, then Update, then Insert. Deleting before inserting frees vmids and MACs for the inserts in the same commit; inserting last means a failed insert cannot orphan a half-updated VM.
-
Update== stop → change → restart is the core semantic and is modelled explicitly, not implied:-
VmOperationenum:Read,Insert,Stop,Change,Start,Delete. -
IVirtualMachineOperations(internal, one impl per provider) exposesRead,Insert,Stop,Change,Start,Deleteas six separate calls. -
Commit()expands one pendingUpdateinto exactly three recorded operations, in order:Stop,Change,Start. -
A VM that was already stopped before the update is NOT restarted -
Startis emitted only when the pre-updatePowerStatewasRunning. Restarting a deliberately-stopped VM would be a behaviour change smuggled in through the abstraction. -
An
Updatewhose diff is empty is a no-op: zero operations recorded, no stop, no restart. Idempotentnetsetup updatecycles must not bounce healthy VMs.
-
-
VmCommitResult(public record) carriesIReadOnlyList<VmOperationRecord> Operations(Key,Operation,Ordinal) plusInserted/Updated/Deletedcounts andFailedkeys. Tests assert on this list - it is the observable proof of stop/change/restart. -
Provider mapping of the six operations. No new hypervisor code - every verb already exists in the shipped libraries:
-
Hyper-V (
osisa.SystemManagement.HyperV):Read→ManagementScope.ReadAllVirtualMachines()(ReadVirtualMachine(name)for single);Insert→ManagementScope.InsertVirtualMachine(already DB-named - seeNETSetup.Helpers.VMHelper.CreateVM);Stop→VMHelper.StopVM(vm.System.Stop());Start→VMHelper.StartVM(vm.System.Start());Delete→RemoveVirtualMachineAsync/DestroyVirtualMachine. -
Proxmox (
osisa.SystemManagement.Proxmox):Read→Host.GetVirtualMachines()/TryGetVirtualMachine;Insert→Host.CreateVirtualMachine(vm, autoStart, storage, isoStorage);Stop→StopVirtualMachineAsync(ShutdownVirtualMachineAsyncwhen the guest agent is up);Start→StartVirtualMachineAsync;Delete→RemoveVirtualMachineAsync. -
VMHelper.RebootVMandRebootVirtualMachineAsyncare deliberately NOT used to implementUpdate: reboot is one atomic guest operation and would hide theChangestep between stop and start.
-
-
Test targets, from the existing in-repo infrastructure - do not invent new ones:
-
Proxmox: node
osisaLib03, API192.168.3.233:8006, userroot, passworddefault4244$- same constantsProxmoxNixosNextcloudTestsandProxmoxStandaloneWin11NodeTestsalready use. Connect by LAN IP, not hostname:osisaLib03is not DNS-resolvable from every dev box. -
Hyper-V:
LiveMethods.ConnectToLiveLocalSystem(@"root\virtualization\v2"), exactly asVirtualHyperVTestBase.HyperVScopedoes.
-
-
How "the correct VM count" is asserted. The count is environment-dependent, so a hard-coded number would be a false green on any other box. The oracle is the native API of the same provider: the DataStore count must equal
scope.ReadAllVirtualMachines().Count()(Hyper-V) resp.pveHost.GetVirtualMachines().Count()(Proxmox). One abstractExpectedVmCounton the base, one line in each derived class. The base additionally asserts the count is> 0(an empty result would satisfy an equality-only oracle trivially) andTestContext.WriteLine`s every VM name so the run is diagnosable. Both live classes carry `[TestCategory("local")]andAssert.Inconclusivewhen their environment is absent - the suite must stay green on a build agent with no Hyper-V and no route to192.168.3.233. -
Commit()is fail-fast per record but aggregating across records: a failing record records its partial operations, is added toVmCommitResult.Failed, and the loop continues;Commit()throwsVmDataStoreExceptionat the end listing every failed key - same contract asEnsureCustomerVms.RunCore, so a future wire-in is a drop-in. -
VmDesiredStateBuilder(NETSetup side,public static, pure, zero IO) reusesCreateVirtualMachines.GetOrderedVmsForHostverbatim so desired-state ordering (OPNsense → DC → rest → NixDns last) stays the same single source of truth as today’s provisioning order. No duplicated ordering logic. -
VmDataStorePlanner(NETSetup side,public static, pure, zero IO) diffs desired vs live. Delete safety guard, mirroringMikrotikDesiredStateBuilder.ManagedPrefix: a live VM is only ever proposed for deletion when its serial is hosted on this host (CreateVirtualMachines.IsForHost). A hand-created VM, or one belonging to another host, is never inDeletes.VmChangePlanhas noApplyin this iteration - the planner is read-only by design, and that is what keeps the spec behaviour-neutral. -
No
System.IO, no rawProcess:AbsolutePath+PhysicalFileOperatorand the NETCommand builders, per repo convention. -
Test packages for
src/osisa.VirtualMachines.Tests/:osisa.Repository,osisa.Repository.Contracts,osisa.Repository.TestInfrastructure,osisa.Domain.Mock,osisa.Specifications,osisa.TestInfrastructure,osisa.TestInfrastructure.Contracts,osisa.Hosting.TestInfrastructure,osisa.SystemManagement.HyperV,osisa.SystemManagement.Proxmox, MSTest + FluentAssertions. RISK:osisaRepositoryVersionis pinned at0.19.0-ModelFeature.191while the restoredosisa.Repository.TestInfrastructurecarryingDataStoreReference5TestBaseis0.19.0-ModelFeature.292+. If.191does not shipDataStoreReference5TestBase, bumposisaRepositoryVersionto the netbase-current0.19.0-ModelFeature.292in one isolated commit and re-run the full suite BEFORE starting task 9. -
Test values:
src/osisa.VirtualMachines.Tests/TestInfrastructure/TestValues.cswithSome*constants only (SomeVmName,SomeOtherVmName,SomeHostSerial,SomeVmSerial,SomeMemoryBytes,SomeUpdatedMemoryBytes,SomeCoreCount,SingleItemCount,TwoItemCount,SixOperationCount, …). NoValue, noInteger1.
Status (2026-08-30, SOFT-DELETE CONTRACT MOVED TO NETBASE + HYPER-V FIRMWARE BY OS, UNCOMMITTED in BOTH repo): four change, two of them BEHAVIOUR.
-
IPermanentDeletions<TEntity>left this repo. It now ship in netbaseosisa.Repository.Contracts/Repositories/, deriveIRepositoryBase<TEntity>, and keep the same two member (ReadSoftDeleted()+ExecutePermanentDeletionsAsync(ct)). The oldosisa.Virtualization/Repositories/IPermanentDeletions[TEntity].csis DELETED; consumer only repoint the using. Reason: a two-step deletion is a repository capability, so a netbase TEST BASE can gate on it - which is impossible while the interface live in a consumer library. Semantics unchanged, see the netsetup rootNETSetup.adocsection "Deletion is two steps".-
Doc crefs in
osisa.Virtualizationmust now be FULLY QUALIFIED:osisa.Repository.Contractsexport aMachineNameof its own and a bare cref go ambiguous. -
The interface is INVARIANT on purpose. Covariance was tried and dropped -
out TEntitywould foreclose a laterPermanentlyDeleteAsync(TEntity)forever.
-
-
CRUD suites moved v3 → v4. Netbase gained
UnitOfWorkRepositoryTestsBase4[…]plus the[…,TKey]keyed variant: v3 plus SIX soft-delete case, gated onRepositorySupportsPermanentDeletions.VirtualMachineRepositoryCrudTestBase2(and with it the HyperV + Proxmox live suites) sit on v4 now - 8 → 14 case per provider suite.ReadSoftDeletedCount()andExecutePermanentDeletions()arevirtualbecause a live hypervisor is a SHARED store; the base override both and filter throughIsOwn, and the destroy route through the existingExecuteOwnDeletionsguard, which goAssert.Inconclusiverather than destroy a guest the suite did not create. New tests + a fake cover the contract inosisa.Repository.Contracts.Tests.NETConfigDataStoreTests2deliberately STAY on v3: its store is JSON-backed, so the six cases would mark and destroy a config entry, not a guest. -
BEHAVIOUR: Hyper-V firmware is now decided by the row OS and applied AT CREATION. New
Operations/HyperVFirmware.cs+ a changedHyperVVmOperations: a Windows-family row is created with Secure Boot ON and a vTPM, anything else with both OFF, passed as theVirtualBIOSconstructor argument. Both halves are needed - Windows 11 demand the pair, and the Microsoft UEFI CA reject the unsigned nixos ISO the moment Secure Boot is on. NewContracts/OperatingSystemExtensions.Is(this OperatingSystem, OperatingSystemFamily)is the one place that answer "is this a Windows row".Warningenabling a vTPM AFTER the guest exist is UNRECOVERABLE. Measured on the bench this session: the vTPM seal guest state to itself, so a guest that was installed without one refuse to start with 0xC000A002(STATUS_AUTH_TAG_MISMATCH) once one is added. There is no later switch - which is exactly why this belong in the create call and NOT in anUpdatea synchronisation could apply.HyperVVirtualMachineRepositoryCrudTests2therefore provision a Windows 11 shaped guest, 64 GB, that actually BOOTS (a row with no OS would only ever prove the Linux half, and a guest that never boot never exercise the firmware it was given), gate on its boot image fixture (Assert.Inconclusivewhen the image is absent - the netbaseVirtualMachinector refuse a guest with no ISO), and honourNETSETUP_KEEP_CRUD_GUESTSto keep the created guests for inspection instead of destroying them.COST, no full-suite figure available: only ONE case was timed this session,
UnitOfWorkRepositoryTestBase_Insert_Commiton Hyper-V - 11m29s on the old Minimal shape, then 18m19s and 20m30s once the guest became the booting 64 GB Windows 11 shape. Do NOT extrapolate a 14-case suite time from that. Measure the suite before quoting one. -
BEHAVIOUR:
osisa.OPNSense.Rest/UnitOfWork/FirewallGroupsUnitOfWork3.csported. NetbaseUnitOfWorkBase3take anILoggerinstead of anITransaction3and declare an abstractOnCommit(); the transaction is HELD by the unit of work and flushed inOnCommit. This is the port the version hold recorded below (see the 2026-08-24 audit entry) said would be needed - it is done, and the hold insrc/Directory.Packages.propsis released. -
PACKAGE VERSIONS in both repo currently point at LOCALLY PACKED builds. They are local-only and must not be committed, so no version number is written here - a doc that name an unpublished package send the next reader to a feed that does not have it.
-
BEHAVIOUR:
Stages/0-Iso/S5NETSetupDev.UpdateISOfixed. Two defects, both of which produced a SILENT bad outcome:-
Deadlock. It blocked on
builder.Build().ExecuteAsync().Result, which deadlock under a test host’s SynchronizationContext. Symptom: publish complete, then zero CPU and zero bytes until the case time out three hours later - it read as a slow build, not as a hang. NowTask.Run) ⇒ command.ExecuteAsync(.GetAwaiter().GetResult(). -
1-byte ISO. The result was logged at
Debugand the exit code NEVER checked, so a failed build look identical to a good one: the caller moved a file that was never written and handed a guest an unbootable medium.UpdateISOnow THROW on non-zero exit, and the new privateEnsureIsoBuiltverify the image exist and is>= MinimumPlausibleIsoBytes(1 MB).
-
Bench findings from the SAME session, measured on osisaLib03 (192.168.3.233) and the local Hyper-V
bench. These are environment + creation-path rules, not repository work, so they live in full in
docs/winpe.adoc section "Deploy Blockers" (durable rules) and in the
netsetup-windows-node-proxmox skill reference references/deploy-blockers.md (command recipes).
Recorded here only because they decide whether a run on this bench can be trusted:
-
A live suite that provision an INSTALL guest must disable ballooning / dynamic memory. WinPE have no balloon driver, so the guest get squeezed and the DISM apply HANG at zero bytes rather than fail.
VirtualHyperVTestBasetoday callDisableDynamicMemoryonly for non-Windows rows - and a Windows row install through WinPE just the same, so the OS gate is the wrong gate. -
The vTPM measurement behind the firmware change above. Enabling a vTPM after the guest have state give
0xC000A002STATUS_AUTH_TAG_MISMATCHand is unrecoverable - already stated in the firmware bullet, repeated here because it is what makes "AT CREATION" non-negotiable. -
DEFECT, creation path: default
cpu=x86-64-v3cannot start on a host without AVX2. osisaLib03 lack avx2/bmi1/bmi2/fma/movbe/abm and QEMU exit code 1.ProxmoxWindowsVmFactory,ProvisionNixosVmandProxmoxVmOperationsalready override tohost; any NEW creation path must, or the guest is created and never start. -
CreateVirtualMachinecan return true whileqmcreatefailed. The failure surface later asFailed to find VirtualMachine … Sequence contains no matching elementfrom the round-trip read-
which read like a READ defect and is not. Read
/nodes/<node>/tasks.
-
-
devboxisois the reason a Proxmox Win11 test take seconds. It is a CIFS mount of the dev boxNETSetupISOshare whosetemplate\isois a junction toC:\NETSetupDEV, so the host REFERENCE the 11.9 GB ISO. Offline → no guest can be created, and uploading is not a fallback (0.15 MB/s measured, ~20 h). -
A
[TestCategory("Prod")]test report GREEN having run nothing.src/Test.Build.propsforcelocal.runsettings, whoseTestCategory!=Prodfilter apply at DISCOVERY, so the method is never enumerated and an inherited base-class case pass instead. Use--settings src/prod.runsettings, andFullyQualifiedName~(notName=) because these methods are[DataRow]-driven. Any Prod result quoted in this file must have been produced that way.
Status (2026-08-23, LANDED + GREEN, 18/18 tests): the acceptance criterion is met. dotnet test --filter "FullyQualifiedName~DataStoreTests" → Fehler: 0, erfolgreich: 18 (9 shared test method x 2 provider class). Live count delivered: LocalHyperV = 10 VMs, osisaLib03 = 9 VMs, each equal to its own native API count.
-
Code landed in
src/NETSetup/DataStore/+src/NETSetup/Exceptions/VmDataStoreException.cs; test insrc/NETSetup.Tests/DataStoreTests/. NOT a separateosisa.VirtualMachinesproject - NETSetup already referenceosisa.SystemManagement.HyperV+.Proxmox, so only three PackageReference (osisa.Repository,osisa.Repository.Contracts,osisa.Specifications.Contracts) were added and no sln entry was needed. Splitting the library out stay a follow-up. -
Reference base is
CanGetSingleUnitOfWorkDataStoreReferenceBase<VmConnectionStringBuilder, TSelf, IVirtualizationUnitOfWork>(sharedVmDataStoreReferenceBase<TSelf>), NOTDataStoreReferenceBase5direct - a VM store serve exactly ONE unit of work type, which is the same reasonMachineScopeReference5use that base. -
Builders are plain nested
Builderclass, notDataStoreNameDependencyInjectionDataStoreReferenceBuilder5: that base demandSetDataStore+ DI plumbing this store does not need, and the live handle (management scope, Proxmox host) is injected, never serialised. -
VirtualizationDataStoreReferenceProvider3wrap each single-UoW reference in the netbaseSingleAsAnyUnitOfWorkDataStoreReference<IVirtualizationUnitOfWork>soGetReferences()satisfyIDataStoreReferenceProvider3.AddReference2does not exist inosisa.Repository- see the decision above. -
VirtualMachineKeyis a plainstringon the record for now, not a Vogen VO. Vogen wrapping is a follow-up and does not change any signature above. -
KEY DERIVATION BUG FOUND AND FIXED live:
SerialNumber.From("9000@osisaLib03")yieldValue == "osisaLib03"andHostedId == "9000"- the vmid is SPLIT OFF. Keying on.Valuecollide every NETSetup VM on a node onto one key. A hosted serial must be REASSEMBLED as{HostedId}@{Value}. Non-hosted guest report a hardware serial,"0"or"empty"(all non-unique), so they key on the node-qualified VM name instead. Hyper-V has no vmid and no unique BIOS serial, so it always key on the scope-qualified name.ReadAll_ShouldReturnUniquelyKeyedRecordsis what caught it. -
Still open from the task list below: the in-memory provider (task 4), the cross-provider
GetReferences()walk (task 9), the netbaseDataStoreReference5TestBase/UnitOfWorkRepositoryTestsBase3suite (task 10), the stop-change-restart semantics test (task 11),VmDesiredStateBuilder(task 12),VmDataStorePlanner(task 13), the neutrality regression test (task 14) and the doc/puml task (task 15). Nothing in the landed code is called fromnetsetup update, so behaviour is unchanged by construction.
Status (2026-08-24, UNIT-OF-WORK INVARIANT AUDIT): osisa.Virtualization.Tests 54/54; live DataStoreTests 26/26; regression 988 / 0.
THE INVARIANT, stated by the repo owner and now audited: Insert / Update / Delete on the repository do NOTHING. The environment change only on Commit of the unit of work. No commit - no insert, update or delete.
-
HOLDS, verified line by line.
Insert/Update/DeleteonlyEnqueueinto_pending. All five mutating seam call (InsertAsync,DeleteAsync,StopAsync,ChangeAsync,StartAsync) live insideFlushAsync/ApplyUpdateAsync, andFlushAsynchave exactly ONE caller repo-wide:CommitAsync.FlushAsyncandRollbackare deliberately NOT onIVirtualMachineRepository, so no consumer can flush without committing. Read is live and unbuffered by design - a pending insert is invisible toReadAlluntil commit, declared on the contract. -
NOT PROVED BY DERIVING netbase
UnitOfWorkRepositoryTestsBase3, which is what this repo normally use. Environment blocker, not a choice:osisaRepositoryVersionis pinned at0.19.0-ModelFeature.191, NO csproj in netsetup consumeosisa.Repository.TestInfrastructureso that version was never pulled into the local NuGet cache, and the GitHub Packages feed answer 403 without credential.VersionOverrideto.300fail too - its transitiveosisa.Json/osisa.Stringsat.300are equally absent. DO NOT fix this by bumpingosisaRepositoryVersion: this file record that.292change theUnitOfWorkBase3ctor and require portingosisa.OPNSense.Rest, and that the last attempt measured 188 build error and 0 runnable test. The real fix is aread:packagestoken plus a singlePackageReferenceat.191- every.191transitive dep is already cached.SUPERSEDED (2026-08-30): the hold is released.
osisa.OPNSense.RestIS ported to the newUnitOfWorkBase3(ILoggerctor + abstractOnCommit(), transaction held and flushed there), so the repository version could move and the live CRUD suites now DO derive the netbase base - v4, not v3. Keep the paragraph above: it is the record of WHY the hold existed, and the 188-build-error attempt it describe is what a naive bump still cost. -
IN ITS PLACE
VirtualMachineRepositoryCrudTestsmirror the six base case test-for-test (Insert/Update/DeletexCommit/NoCommit), assert on READ-BACK STATE the way the base do, and addCommitAsyncvariant, an all-three-verbs-uncommitted case and the dispose case. Shaped so swapping to the real base is mechanical. -
MUTATION TESTED, twice, because a guard nobody has seen fail is not a guard. (1) Making
Insertapply immediately turn 8 test red. (2) EmptyingRollback()originally left ALL 48 GREEN - the two dispose test could not fail, one verifying through a fresh unit of work owning a different buffer, the other only asserting the fake had recorded nothing.Dispose_ShouldClearTheBufferItselfassert the buffer directly and now catch it. -
CommitAsyncnow refuse a DISPOSED unit of work (ObjectDisposedException). It used to flush, inert only as a side effect ofRollbackclearing. -
ChangeAsyncDEFECT, fixed by refusing rather than pretending. Both provider apply NO field. Driven through the update path that mean: stop the guest, change nothing, restart, reportUpdated++for work never done - and since nothing was applied the same difference persist, so EVERY reconciliation bounce the guest again.IVirtualMachineOperations.CanApplyChangesnow declare the capability, both provider return FALSE, and the repository refuse the update BEFOREStopAsync. The ORDER is the point: throwing from insideChangeAsyncwould leave the guest POWERED OFF, because a per-record failure is caught and the restart never run. Same rule the provider already follow forNetworkName/Storage/VmId- a provider that cannot honour a field say so instead of silently dropping it. -
KNOWN LIMIT of the unit suite: the fake DOES apply a change, so
Update_Commit_ShouldApplyTheChangeprove the REPOSITORY buffering, never a provider’s CRUD. No provider apply fields yet, so there is nothing to prove on that side untilChangeAsyncis implemented.
Status (2026-08-24, ARCHITECT APPROVED after five review rounds): dotnet build src/NETSetup.sln clean; osisa.Virtualization.Tests 40/40; live DataStoreTests 26/26 against real Hyper-V + osisaLib03, both guest SSH-connectable; regression 988 passed / 0 failed.
The review chain kept finding ONE defect class wearing different clothes: an update that silently did nothing and reported a clean commit. Each round closed one instance; the last round closed the CONDITION instead. Worth reading before touching the matching or reporting code.
-
KEY MATCHING, round 3+4: a row built by
VirtualMachineBuildercarry no key, and that is the NORMAL update path. Matching it needed three corrections. (1) The builder used to inventFrom(name), a third key shape no provider project - removed. (2) A name-based fallback compared case-SENSITIVELY (MachineNamedo not normalise) while the key path did not - removed. (3) Matching only the projected key missed every Proxmox guest carrying a NETSetup serial, because those projectFromSerialwhile a keyless row key asenv/name. Final shape:ResolveLiveRE-KEY every live row asFromEnvironment(EnvironmentName, r.Name)and compare that, so there is ONE comparison and it is always a normalisedVirtualMachineKey. Comparing the projected key as well would be dead code - a serial never contain the separator a re-keyed name always has. -
MATCHING DEPEND ON UNIQUE NAMES. Proxmox do not enforce unique guest name, so the live suite assert
ReadAll_ShouldReturnUniquelyNamedRecordsalongside the key assertion. Two guest sharing a name on one node would make an update ambiguous. -
REPORTING, round 4: "no live counterpart" and "nothing differs" used to share one silent
return false, and THAT shared silence is what let all three matching bugs survive.VmCommitResultnow carryNotFound, andApplyUpdateAsyncreturn anUpdateOutcometri-state (Applied / Unchanged / NotFound) so a MISS is caller-visible and testable rather than only a log line. -
NAMING, round 4:
VmOperationRecordandVmCommitFailurenow carryMachineName Nameas well as the key, because the key is EMPTY for a keyless row - the documented path - so a key-only record rendered every operation of that path anonymous and a failure rendered asfailed for: .naming nothing.VmCommitFailure.Describe()fall back to the name. Stop/Change/Start are keyed from the RESOLVED LIVE row, never the desired one. -
LOGGER:
CommitLogger ?? Logger ?? NullLogger.Instance- an explicitly configured logger WIN. The other order madeWithLoggera no-op whenever the service provider happened to supply a logger.WithLoggeris wired at all four live test call site. -
ResolveIpAddressreturnIPAddress?, not a string. The Proxmox side already hold a typed address and used to throw the type away, which forced a hand-rolled dotted-quad check in the base. -
STILL DEFERRED, deliberately:
ChangeAsyncon both provider only RESOLVE the machine, it apply no field - so an update currently stop and restart a guest without changing anything. That is the update spec’s work and it is documented at both call site. Also open:ICanReadAsync<T>/IGenericRepositoryAsync<T>are shipped by netbase but not implemented here, so the twoReadoverload stay sync-over-async; andVmConnectionStringBuilder/VirtualizationDataStoreReferenceProvider3/VirtualizationDataStorehave no dedicated unit test, being thin and covered end-to-end by the live suite.
Status (2026-08-23, EXTRACTED TO osisa.Virtualization + VO CONVERSION, GREEN): the layer left NETSetup and became a library, and the raw-primitive surface is gone. dotnet build src/NETSetup.sln clean; osisa.Virtualization.Tests 34/34; live DataStoreTests 24/24 against real Hyper-V + osisaLib03; regression 988 passed / 0 failed.
-
LIBRARY:
src/NETSetup/DataStore/is GONE. Code now live insrc/osisa.Virtualization/(Contracts/ ValueObjects/ DataStore/ Operations/ Repositories/ UnitOfWork/ Exceptions/) plussrc/osisa.Virtualization.Tests/, both registered insrc/NETSetup.sln, consumed by NETSetup and NETSetup.Tests viaProjectReference(the in-solution convention - the PackageReference rule is for CROSS-solution deps). ONE project with aContracts/folder, following osisa.Mikrotik / osisa.OPNSense / osisa.Inventory rather than the two-assembly split.VmDataStoreExceptionbecameVirtualizationException : Exception- a library must not depend on the app assembly. -
VALUE OBJECTS: new Vogen
VirtualMachineKey(IKeyOf<VirtualMachineSettings>, only two factories -FromSerial(SerialNumber)andFromEnvironment(MachineName, MachineName)),VmId,VirtualSwitchName,StorageName,WmiNamespace. REUSED rather than reinvented:MachineName,OperatingSystem,DataSize(twice).CoresstayintandAutoStartstaybool- a genuine count and a genuine two-state. -
THE
IsoSPLIT is the substantive design change. Onestringused to mean an absolute local path on Hyper-V and a storage-relative name on Proxmox, which is why the test base could only assertEndWith("nixos.iso"). The row now carryISubPath Iso(the NAME) plusStorageName IsoStorage, and each provider resolve it against its own root - Proxmox against the storage, Hyper-V through the newIBootImageResolver(implemented byNETSetup.Helpers.BootImageResolverover the keyed operator). The identical row now go to both hypervisor unchanged. -
ASYNC:
IVirtualMachineOperationsis Task-returning, the repository flush is async, and the unit of work implementICanCommitAsync.CommitAsyncis the real path; the synchronousCommit()delegate to it at the OUTER boundary only, because netbaseIUnitOfWorkdeclare a synchronousCommit. OnlyCommit()is genuinely FORCED (netbase declare it synchronous with no async counterpart); the twoReadoverload are a second and third sync-over-async point that could be lifted, because netbase also shipICanReadAsync<T>/IGenericRepositoryAsync<T>- not implemented yet. The code comment name all three rather than claiming one. -
SEVEN DEFECTS FIXED. The sharpest: the "empty diff is a no-op" guard never worked. It compared full record equality, but a read-back record never carry
Iso/NetworkName/Storage, so a desired record built with an ISO ALWAYS compared unequal and every idempotent cycle stopped and restarted a healthy machine - the exact behaviour the code claimed to prevent.VirtualMachineSettings.DescribesSameMachineAsnow compare only the fieldsChangecan apply. Also: Proxmox insert now honourrecord.VmId(WithVmId(PickFreeVmId(…))used to be silently dropped); a failed record no longer escapeFlushleaving the unit of work dirty; the real exception survive intoVmCommitFailureand the aggregate throw; Hyper-VStop/Startthrow instead of silently no-op while the commit log claim they ran. -
A provider that CANNOT honour a field now throw instead of discarding it silently - Hyper-V on
Storage/IsoStorage/VmId, Proxmox onNetworkName. A field a provider drop without a word is the same class of lie as the dual-meaningIsowas. -
VOGEN BUG FOUND: a
[Instance]carrying backslashes is emitted into a NON-verbatim literal without re-escaping, soroot\virtualization\v2silently becamerootirtualization2(the\vpair read as a vertical tab) and every Hyper-V connection failed. Anything with a backslash must be built throughFromin a plain static property instead.WmiNamespace_HyperV_ShouldKeepItsBackslashesguard it. -
KEY MATCHING, fixed after review:
VirtualMachineBuildermust NOT invent a key. It used to default toFrom(name), a THIRD key shape neither provider ever project, so the repository could never match the row and every update of a built row was a silent no-op reported as a clean commit - the same defect as the empty-diff bug, re-entering through the builder. A built row now leave the key empty and the repository key it exactly the way a provider key a keyless guest, so there is ONE match path. -
KEY CASE, fixed after review:
VirtualMachineKeydocumented case-insensitive comparison but only an explicitComparisonValueuppercased - the generated==/GetHashCodestayed ordinal, and the repository dictionary use the DEFAULT comparer. A VogenNormalizeInputnow lower-case on creation so every consumer obey one rule. Null is passed THROUGH that hook on purpose: Vogen run it BEFOREValidate, so coalescing there would turnFrom(null)into a silent empty key. -
NETSetupFileOperators.Iso2renamed toBootImages(+DefaultBootImagesRoot/BootImagesRoot) - numbered names say nothing, and the key needed a five-line comment to justify itself. -
The raw
ProcessStartInfoin the Hyper-V test is gone. Two nested NETCommand facade command (GetVmNetworkAdapterCommand,NeighborLookupCommand) delegate toPowershellCommand.Builder().UseEncoded(), carry aRenderedShellCommandtest seam, and set$ErrorActionPreference = 'Stop'so a cmdlet failure is distinguishable from an empty result - the old helper swallowed exit code and stderr entirely. -
ADDRESS DISCOVERY, learned the hard way: do NOT use
ArpScanner.FindIPByMACto find a guest on the Default Switch. It actively ARP-SWEEP every address of the subnet, and that switch is a /20 - roughly 4000 synchronousSendARPcalls, which crashed the test host unscoped and outlasted the whole boot timeout when scoped. READ the neighbour cache withGet-NetNeighborinstead; the guest is already in it by the time sshd is up. Hyper-V connect went from 6m50s (fail) to 48s (pass).
Status (2026-08-23, VM BUILDER + INSERT + CONNECT, LANDED + GREEN, 22/22 tests): a machine is now DESCRIBED once with a fluent builder and INSERTED into either environment, and the resulting guest is provably reachable.
-
VirtualMachineBuilder(src/NETSetup/DataStore/VirtualMachineBuilder.cs) set every property through aWith…call:WithName,WithKey,WithOperatingSystem,WithVmId,WithMemory/WithMemoryBytes,WithCores,WithDisk/WithDiskBytes,WithPowerState,WithIso,WithNetwork,WithStorage,WithIsoStorage,WithAutoStart.Create()start empty,From(record)start from an existing row,Build()demand a name. -
VirtualMachineSettingsgained OPTIONAL init property (Iso,NetworkName,Storage,IsoStorage,AutoStart). The positional signature is untouched, so every existing construction keep compiling - purely additive. -
Inserton both provider now realise those value with the SAME hardware shapeProvisionNixosVmalready use:Processor+RandomAccessModule+VMDiskon Proxmox, and theVirtualMachine(name, iso, switch, disk, cpu, ram)ctor on Hyper-V. The 4-arg Hyper-V ctor default to 4 cores / 4 GiB, so aWithCores/WithMemorywould silently vanish - the wider overload is used whenever the row specify them. -
Hyper-V gen 2 guest need TWO firmware knob before an unsigned Linux ISO boot: Secure Boot OFF and the optical drive FIRST. (UPDATED 2026-08-30: Secure Boot is no longer OFF for everybody - the row OS decide it, Windows family get Secure Boot + vTPM at CREATION. See the top status entry. The optical-drive-first rule is unchanged.) Both are set natively through
Msvm_VirtualSystemSettingData+ModifySystemSettings(no PowerShell, no raw process).ComputerSystemMethods.ModifySettingDatawas tried first and fail with "failed to modify resources" - it target RESOURCE setting, not the SYSTEM setting where firmware knob live. -
Boot source selection: the DVD and the hard disk BOTH describe themselves as
EFI SCSI Deviceon a gen 2 guest, so a description match pick the wrong device. A boot source InstanceID is the drive own device path plus a trailing marker, so the DVD is matched on THAT. -
IPFS node row (identical on both): 8 GiB RAM, 2 core, 64 GiB disk,
nixos.iso, autostart. Verified on the Hyper-V guest:ProcessorCount 2,MemoryStartup 8 GiB. -
CONNECT PROOF: a real ssh session as root, authenticated with
netsetup-master.key(its public half is baked intonixos.isorootauthorized_keys), thenuname -aexecuted and asserted - a half-open socket cannot pass. Both guest answerLinux nixos-installer 6.12.78 … x86_64 GNU/Linux.TestIpfsNixProxmoxon osisaLib03 andTestIpfsNixHyperVon the local Hyper-V are LEFT RUNNING on purpose so they stay connectable. -
Address discovery is provider wiring: Proxmox use the existing
DiscoverVmIpViaQmAgent(the ISO enable qemu-guest-agent); Hyper-V ask integration service first and fall back to adapter MAC → host arp cache, because the NixOS live image run no Hyper-V KVP daemon. -
The local boot image live at
C:\NETSetupImages\nixos.iso, NOT underC:\NETSetupDEV: the Stage0 dev-env test wipe that folder, and a running guest hold its mounted ISO locked - 15 Stage0 test failed with a sharing violation until the image moved out.-
✓ Scaffold
osisa.VirtualMachines+ Contracts public surface: Createsrc/osisa.VirtualMachines/osisa.VirtualMachines.csproj(targets$(DefaultNetTargetFramework),<InternalsVisibleTo Include="osisa.VirtualMachines.Tests" />, referencesosisa.Repository,osisa.Repository.Contracts,osisa.Specifications,osisa.Validation,osisa.SystemManagement.Contracts,Microsoft.Extensions.DependencyInjection.Abstractions,Microsoft.Extensions.Logging.Abstractions) andsrc/osisa.VirtualMachines.Tests/osisa.VirtualMachines.Tests.csproj. Add both tosrc/NETSetup.sln. Createnamespace osisa.VirtualMachines.Contractswith:VirtualMachineKey(Vogen VO overstring,From(SerialNumber)+From(int vmId, string hostName)fallback),VmPowerStateenum (Unknown,Stopped,Running),VmOperationenum (Read,Insert,Stop,Change,Start,Delete),VirtualMachineSettingsrecord (VirtualMachineKey Key,string Name,string OperatingSystem,int VmId,long MemoryBytes,int Cores,long DiskBytes,VmPowerState PowerState),VmOperationRecordrecord (VirtualMachineKey Key,VmOperation Operation,int Ordinal),VmCommitResultrecord (IReadOnlyList<VmOperationRecord> Operations,int Inserted,int Updated,int Deleted,IReadOnlyList<VirtualMachineKey> Failed),VmDataStoreException : Exception,NetsetupDataStoreProvidersstatic class (HyperV,Proxmox,MemoryVmconsts). Assertions: -
VirtualMachineKey.From(SerialNumber.From(SomeVmSerial)).Value.Should().Be(SomeVmSerial)
-
VirtualMachineKey.From(SomeVmId, SomeHostName).Value.Should().Be(SomeVmId + "@" + SomeHostName)
-
new VirtualMachineSettings(SomeKey, SomeVmName, …).Key.Should().Be(SomeKey)
-
new VirtualMachineSettings(…).PowerState.Should().Be(VmPowerState.Stopped)
-
two VirtualMachineSettings with identical values .Should().Be(each other)
-
Enum.GetValues<VmOperation>().Should().HaveCount(SixOperationCount)
-
VmOperation.Stop.Should().NotBe(VmOperation.Change)
-
NetsetupDataStoreProviders.HyperV.Should().Be(nameof(NetsetupDataStoreProviders.HyperV))
-
NetsetupDataStoreProviders.Proxmox.Should().NotBe(NetsetupDataStoreProviders.MemoryVm)
-
new VmCommitResult(Array.Empty<VmOperationRecord>(), 0, 0, 0, Array.Empty<VirtualMachineKey>()).Operations.Should().BeEmpty()
-
typeof(VmDataStoreException).Should().BeDerivedFrom<Exception>()
-
✓ Create
IVirtualMachineRepository+VirtualMachineRepositoryBasewith buffered Insert/Update/Delete: CreateContracts/IVirtualMachineRepository.csasIGenericRepository<VirtualMachineSettings>so it inherits netbaseICanRead,ICanInsert,ICanUpdate,ICanDeleteofVirtualMachineSettings. CreateRepositories/VirtualMachineRepositoryBase.cs(internal abstract): holdsIVirtualMachineOperations Operations(injected), a pending-change list, andbool IsDirty.Insert/Update/Deleteonly enqueue;ReadAll/ReadSinglego straight through toOperations.Read(always live).FlushAsyncapplies pending changes in the fixed order Delete → Update → Insert and returns theVmOperationRecordlist, expanding eachUpdateintoStop→Change→Start(Startomitted when the pre-update record’sPowerState != Running; the whole triple omitted when the diff is empty). Clears pending +IsDirtyon success. Assertions: -
repository.IsDirty.Should().BeFalse() initially
-
repository.Insert(SomeRecord); repository.IsDirty.Should().BeTrue()
-
repository.Insert(SomeRecord); operations.Recorded.Should().BeEmpty()
-
repository.Delete(SomeRecord); repository.IsDirty.Should().BeTrue()
-
repository.ReadAll() with pending inserts: operations.Recorded.Should().ContainSingle(o ⇒ o.Operation == VmOperation.Read)
-
after FlushAsync: repository.IsDirty.Should().BeFalse()
-
flush with one Insert + one Delete: recorded.Select(o ⇒ o.Operation).Should().ContainInOrder(VmOperation.Delete, VmOperation.Insert)
-
flush with one Update on a Running record: recorded.Select(o ⇒ o.Operation).Should().ContainInOrder(VmOperation.Stop, VmOperation.Change, VmOperation.Start)
-
flush with one Update on a Stopped record: recorded.Select(o ⇒ o.Operation).Should().ContainInOrder(VmOperation.Stop, VmOperation.Change) and .Should().NotContain(VmOperation.Start)
-
flush with an Update whose desired equals live: recorded.Should().BeEmpty()
-
typeof(IVirtualMachineRepository).Should().Implement<ICanInsert<VirtualMachineSettings>>()
-
typeof(IVirtualMachineRepository).Should().Implement<ICanUpdate<VirtualMachineSettings>>()
-
typeof(IVirtualMachineRepository).Should().Implement<ICanDelete<VirtualMachineSettings>>()
-
typeof(IVirtualMachineRepository).Should().Implement<ICanRead<VirtualMachineSettings>>()
-
✓ Create
IVirtualizationUnitOfWork+VirtualizationUnitOfWorkwithCommit: CreateContracts/IVirtualizationUnitOfWork.cs(: IUnitOfWork) exposingIVirtualMachineRepository VirtualMachines { get; }- shaped after the existingISystemManagementUnitOfWorksoGetRepository<VirtualMachineSettings>()resolves the SAME instance as the named property. CreateUnitOfWork/VirtualizationUnitOfWork.cs(internal,: UnitOfWorkBase3where the base fits, else the interface directly): ctor takesIVirtualMachineOperations+ILogger.Commit()flushes the repository whenIsDirty, aggregates per-record failures intoVmCommitResult.Failedand throwsVmDataStoreExceptionlisting every failed key; a cleanCommit()on a non-dirty UoW is a no-op returning an empty result.Dispose()drops pending changes without applying them. Assertions: -
unitOfWork.VirtualMachines.Should().NotBeNull()
-
unitOfWork.GetRepository<VirtualMachineSettings>().Should().BeSameAs(unitOfWork.VirtualMachines)
-
typeof(IVirtualizationUnitOfWork).Should().Implement<IUnitOfWork>()
-
Commit() on a non-dirty unit of work: result.Operations.Should().BeEmpty()
-
Commit() on a non-dirty unit of work: operations.Recorded.Should().BeEmpty()
-
Insert then Commit(): result.Inserted.Should().Be(SingleItemCount)
-
Insert + Update + Delete then Commit(): result.Deleted.Should().Be(SingleItemCount) && result.Updated.Should().Be(SingleItemCount) && result.Inserted.Should().Be(SingleItemCount)
-
Insert then Dispose() without Commit(): operations.Recorded.Should().BeEmpty()
-
one failing record among three: act.Should().Throw<VmDataStoreException>().WithMessage("" + SomeVmSerial + "")
-
one failing record among three: the other two records .Should() still be applied
-
Commit() twice with no changes in between: second result.Operations.Should().BeEmpty()
-
❏ Create
MemoryVmDataStoreReference5(V5 in-memory provider): CreateDataStore/VmConnectionStringBuilder.cs(: DbConnectionStringBuilder,DataStoreName+HostName+ nestedValueNamesclass) andDataStore/MemoryVmDataStoreReference5.csdecorated[DataStoreProvider(NetsetupDataStoreProviders.MemoryVm)], derivingDataStoreReferenceBase5<VmConnectionStringBuilder, MemoryVmDataStoreReference5>. OverrideExists(),EnsureDeleted(),GetName()(→DataStoreName.From(ConnectionStringBuilder.DataStoreName)) andOpenDataStoreImplementation<TUnitOfWork>(out bool isNew). NestedBuilder : DataStoreNameDependencyInjectionDataStoreReferenceBuilder5<MemoryVmDataStoreReference5, VmConnectionStringBuilder, Builder>withWithDataStoreName+WithSeed(params VirtualMachineSettings[]). BackingInMemoryVmOperationskeeps records in a dictionary keyed byVirtualMachineKeyand records everyVmOperationit is asked to perform. Assertions: -
new MemoryVmDataStoreReference5.Builder(Services).WithDataStoreName(TestContext.TestName).Build().Should().NotBeNull()
-
reference.Exists().Should().BeFalse() before open
-
reference.OpenDataStore3<IVirtualizationUnitOfWork>(); reference.Exists().Should().BeTrue()
-
reference.Name.ToString().Should().Be(TestContext.TestName)
-
reference.ConnectionString.Should().Contain(nameof(VmConnectionStringBuilder.ValueNames.DataStoreName))
-
reference.ConnectionString.Should().Contain(NetsetupDataStoreProviders.MemoryVm)
-
MemoryVmDataStoreReference5.FromConnectionString(FromConnectionString, Services).Name.ToString().Should().Be(SomeDataStoreName)
-
seeded with two records: unitOfWork.VirtualMachines.ReadAll().Should().HaveCount(TwoItemCount)
-
reference.EnsureDeleted(); reference.Exists().Should().BeFalse()
-
✓ Create
HyperVVmDataStoreReference5+HyperVVmOperations: CreateDataStore/HyperVVmDataStoreReference5.csdecorated[DataStoreProvider(NetsetupDataStoreProviders.HyperV)], sameDataStoreReferenceBase5shape, nestedBuilderwithWithDataStoreName+WithManagementScope(ManagementScope).Exists()probes the scope. CreateOperations/HyperVVmOperations.cs(internal: IVirtualMachineOperations):Read→ManagementScope.ReadAllVirtualMachines()projected toVirtualMachineSettings;Insert→ManagementScope.InsertVirtualMachine(already DB-shaped - seeNETSetup.Helpers.VMHelper.CreateVM);Stop→VMHelper.StopVM;Start→VMHelper.StartVM;Delete→RemoveVirtualMachineAsync/DestroyVirtualMachine;Changeapplies diffed fields only.VMHelper.RebootVMMUST NOT appear - reboot collapses stop and start into one guest operation and would hide theChangestep. Assertions: -
typeof(HyperVVmDataStoreReference5).Should().BeDecoratedWith<DataStoreProviderAttribute>()
-
reference.ConnectionString.Should().Contain(NetsetupDataStoreProviders.HyperV)
-
operations.Read() on a stubbed scope with two VMs .Should().HaveCount(TwoItemCount)
-
operations.Read() maps name + state: record.Name.Should().Be(SomeVmName) && record.PowerState.Should().Be(VmPowerState.Running)
-
operations.Insert(SomeRecord): stubbed scope InsertVirtualMachine called exactly once
-
operations.Stop(SomeKey) then operations.Start(SomeKey): stub records Stop before Start
-
HyperVVmOperations source .Should().NotContain(nameof(VMHelper.RebootVM))
-
✓ Create
ProxmoxVmDataStoreReference5+ProxmoxVmOperations: CreateDataStore/ProxmoxVmDataStoreReference5.csdecorated[DataStoreProvider(NetsetupDataStoreProviders.Proxmox)], nestedBuilderwithWithDataStoreName+WithHost(PveHost).Exists()pings the node through the existingosisa.SystemManagement.Proxmoxwrapper. CreateOperations/ProxmoxVmOperations.cs(internal: IVirtualMachineOperations):Read→Host.GetVirtualMachines()projected toVirtualMachineSettings;Insert→Host.CreateVirtualMachine;Stop→StopVirtualMachineAsync(ShutdownVirtualMachineAsyncwhen the guest agent responds);Start→StartVirtualMachineAsync;Delete→RemoveVirtualMachineAsync;Changeapplies diffed fields only.RebootVirtualMachineAsyncMUST NOT be used forUpdate. No new hypervisor code - every call goes through the existing wrapper behind theIVirtualMachineOperationsseam so unit tests never touch a real node. Assertions: -
typeof(ProxmoxVmDataStoreReference5).Should().BeDecoratedWith<DataStoreProviderAttribute>()
-
reference.ConnectionString.Should().Contain(NetsetupDataStoreProviders.Proxmox)
-
operations.Read() on a stubbed host returning two qemu entries .Should().HaveCount(TwoItemCount)
-
operations.Read() maps vmid + name: record.VmId.Should().Be(SomeVmId) && record.Name.Should().Be(SomeVmName)
-
operations.Read() maps a running guest: record.PowerState.Should().Be(VmPowerState.Running)
-
operations.Insert(SomeRecord): stubbed host CreateVirtualMachine called exactly once
-
operations.Stop(SomeKey) then operations.Start(SomeKey): stub records Stop before Start
-
ProxmoxVmOperations source .Should().NotContain("RebootVirtualMachine")
-
unreachable host: reference.Exists().Should().BeFalse()
-
[~] Create
VirtualizationDataStoreReferenceProvider3wiring BOTH providers on one builder: CreateDataStore/VirtualizationDataStoreReferenceProvider3.cs(: IDataStoreReferenceProvider3, Shape B - hold an innerDataStoreReferenceProvider3.Builderand delegateGetReference/GetReferences), registeringAddReference3<HyperVVmDataStoreReference5.Builder>(…)ANDAddReference3<ProxmoxVmDataStoreReference5.Builder>(…). NOTE: the API isAddReference3<TBuilder>(Action<TBuilder>);osisa.Repositoryhas noAddReference2. Store names are consts on the class:LocalHyperVStoreName = "LocalHyperV",OsisaLib03StoreName = "osisaLib03". MirrorsMachineScopeProvider3fromosisa.SystemManagement.Windows. PARTIAL (2026-08-24): the provider + builder landed and the store key moved to the test side asDataStoreTests/OsisaLib03.Store/LocalHyperV.Store(they name a BENCH environment, which is test knowledge, not library knowledge). The DI half -VirtualMachinesServiceCollectionExtensions.AddVirtualMachines()+AddUnitOfWorkMapping<IVirtualizationUnitOfWork, VirtualizationUnitOfWork>()- is NOT built: nothing resolve the store through DI yet, every reference is built through its own builder. Do it when a consumer actually need DI resolution. Assertions: -
provider.GetReferences().Should().HaveCount(TwoItemCount)
-
provider.GetReference(DataStoreName.From(LocalHyperVStoreName)).Should().NotBeNull()
-
provider.GetReference(DataStoreName.From(OsisaLib03StoreName)).Should().NotBeNull()
-
provider.GetReference(DataStoreName.From(SomeUnknownStoreName)).Should().BeNull()
-
provider.GetReferences().Select(r ⇒ r.GetType()).Should().Contain(typeof(HyperVVmDataStoreReference5))
-
provider.GetReferences().Select(r ⇒ r.GetType()).Should().Contain(typeof(ProxmoxVmDataStoreReference5))
-
every reference .Should().BeAssignableTo<ICanGetAnyUnitOfWork>()
-
(OPEN, DI half not built) services.AddVirtualMachines(); services.BuildServiceProvider().GetRequiredService<IDataStoreReferenceProvider3>().Should().NotBeNull()
-
VirtualizationDataStoreReferenceProvider3 source .Should().NotContain("AddReference2")
-
✓ ACCEPTANCE: one shared read-all test base + two provider-wiring-only test classes: Create
src/osisa.VirtualMachines.Tests/ReadAll/VirtualMachineReadAllTestBase.cs-public abstract class VirtualMachineReadAllTestBase<TReference, TBuilder> : HostingUnitTestsBaseholding the ENTIRE scenario: build the reference fromConfigureBuilder,OpenDataStore3<IVirtualizationUnitOfWork>(),GetUnitOfWork(),VirtualMachines.ReadAll(), count + name dump. Abstract members, and ONLY these:ConfigureBuilder(TBuilder builder),int ExpectedVmCount(the native-API oracle),bool EnvironmentAvailable(→Assert.Inconclusivewhen false). Then createReadAll/HyperVVirtualMachineReadAllTests.cs(ConfigureBuilder→WithManagementScope(LiveMethods.ConnectToLiveLocalSystem(@"root\virtualization\v2")),ExpectedVmCount→scope.ReadAllVirtualMachines().Count()) andReadAll/ProxmoxVirtualMachineReadAllTests.cs(ConfigureBuilder→WithHost(proxmox.GetHost("osisaLib03"))against192.168.3.233:8006root/default4244$by LAN IP - NOT hostname, it is not DNS-resolvable everywhere;ExpectedVmCount→pveHost.GetVirtualMachines().Count()). Both[TestClass],[TestCategory("local")]. THE TWO FILES MUST DIFFER ONLY IN THAT WIRING - no test method, no assertion, no arrange step may live in a derived class. Assertions: -
ReadAll_ShouldReturnTheSameCountAsTheNativeApi (inherited): records.Count.Should().Be(ExpectedVmCount)
-
ReadAll_ShouldReturnAtLeastOneVm (inherited): records.Should().NotBeEmpty()
-
ReadAll_ShouldReturnUniquelyKeyedRecords (inherited): records.Select(r ⇒ r.Key).Should().OnlyHaveUniqueItems()
-
ReadAll_ShouldPopulateEveryName (inherited): records.Should().OnlyContain(r ⇒ !string.IsNullOrWhiteSpace(r.Name))
-
ReadAll_ShouldClassifyEveryPowerState (inherited): records.Should().OnlyContain(r ⇒ r.PowerState != VmPowerState.Unknown)
-
OpenDataStore_ShouldReturnAVirtualizationUnitOfWork (inherited): unitOfWork.Should().BeAssignableTo<IVirtualizationUnitOfWork>()
-
ReadAll_ShouldNotRecordAnyWriteOperation (inherited): unitOfWork.VirtualMachines.IsDirty.Should().BeFalse()
-
HyperV class: result count .Should().Be(local ManagementScope.ReadAllVirtualMachines().Count())
-
Proxmox class: result count .Should().Be(osisaLib03 PveHost.GetVirtualMachines().Count())
-
typeof(HyperVVirtualMachineReadAllTests).GetMethods(DeclaredOnly | Public).Where(has TestMethodAttribute).Should().BeEmpty()
-
typeof(ProxmoxVirtualMachineReadAllTests).GetMethods(DeclaredOnly | Public).Where(has TestMethodAttribute).Should().BeEmpty()
-
both classes declare exactly the same member names (ConfigureBuilder, ExpectedVmCount, EnvironmentAvailable)
-
❏ Add the cross-provider walk over
GetReferences(): Createsrc/osisa.VirtualMachines.Tests/ReadAll/VirtualizationDataStoreReferenceProvider3ReadAllTests : HostingUnitTestsBase,[TestCategory("local")]. ResolvesVirtualizationDataStoreReferenceProvider3, iteratesGetReferences(), and for EACH reference runs the identicalOpenDataStore3<IVirtualizationUnitOfWork>()→GetUnitOfWork()→VirtualMachines.ReadAll()walk, writing"{storeName}: {count} VMs"per reference. This is the validation that oneAddReference3registration list spans both environments with one code path. Assertions: -
GetReferences().Should().HaveCount(TwoItemCount)
-
the walk over both references .Should().NotThrow()
-
counts.Keys.Should().BeEquivalentTo(LocalHyperVStoreName, OsisaLib03StoreName)
-
counts[LocalHyperVStoreName].Should().Be(local ManagementScope.ReadAllVirtualMachines().Count())
-
counts[OsisaLib03StoreName].Should().Be(osisaLib03 PveHost.GetVirtualMachines().Count())
-
counts.Values.Should().OnlyContain(c ⇒ c > 0)
-
the walk uses exactly one shared code path (no per-provider branch in the test body)
-
❏ Add the netbase DataStore test suites (
DataStoreReference5TestBase+UnitOfWorkRepositoryTestsBase3): Createsrc/osisa.VirtualMachines.Tests/TestInfrastructure/TestValues.cs(Some*constants only). CreateMemoryVmDataStoreReference5Tests : DataStoreReference5TestBase<MemoryVmDataStoreReference5, MemoryVmDataStoreReference5.Builder>-[TestClass],[TestCategory(DataStoreReferenceTest)],DefaultUnitOfWorkTestMapping ⇒ new(typeof(IVirtualizationUnitOfWork), typeof(VirtualizationUnitOfWork)),FromConnectionString,ConfigureregisteringAddUnitOfWorkMapping<IVirtualizationUnitOfWork, VirtualizationUnitOfWork>(), plusAssertConnectionString/AssertDataStoreType/AssertFromConnectionString/ConfigureBuilder. CreateHyperVVmDataStoreReference5TestsandProxmoxVmDataStoreReference5Testsfrom the same base against stubbed operations. CreateVirtualMachineRepositoryCrudTests : UnitOfWorkRepositoryTestsBase3<VirtualMachineSettings, IVirtualMachineRepository, IVirtualizationUnitOfWork>implementingCreateDataStore,CreateEntity(index, unitOfWork)($"{SomeVmName}_{index}"),CreateUnitOfWork,ExpectedInitialCount,GetRepository,OnEnsureDeleteDataStore,UpdateEntity(bumpMemoryBytes). The inherited CRUD suite IS the "VMs are database records" proof. Assertions: -
Insert_Commit (inherited): count after commit .Should().Be(ExpectedInitialCount + SingleItemCount)
-
Insert_NoCommit (inherited): count .Should().Be(ExpectedInitialCount)
-
Update_Commit (inherited): reloaded entity .MemoryBytes.Should().Be(SomeUpdatedMemoryBytes)
-
Update_NoCommit (inherited): reloaded entity .MemoryBytes.Should().Be(SomeMemoryBytes)
-
Delete_Commit (inherited): count after commit .Should().Be(ExpectedInitialCount - SingleItemCount)
-
Delete_NoCommit (inherited): count .Should().Be(ExpectedInitialCount)
-
RepositorySupportsInsert.Should().BeTrue()
-
RepositorySupportsUpdate.Should().BeTrue()
-
RepositorySupportsDelete.Should().BeTrue()
-
RepositorySupportsRead.Should().BeTrue()
-
Ex(UnitOfWorkTestMapping) resolves IVirtualizationUnitOfWork for all three providers
-
❏ Add the explicit stop-change-restart semantics tests: Create
src/osisa.VirtualMachines.Tests/VirtualizationUnitOfWorkSemanticsTests : UnitTestsBase- one test per sentence of the goal, asserting againstVmCommitResult.Operations(orderedVmOperationRecordlist), never against logger text. Assertions: -
AddingAVm_ShouldBeAnInsert: result.Operations.Select(o ⇒ o.Operation).Should().Equal(VmOperation.Insert)
-
RemovingAVm_ShouldBeADelete: result.Operations.Select(o ⇒ o.Operation).Should().Equal(VmOperation.Delete)
-
ChangingARunningVm_ShouldStopChangeAndRestart: result.Operations.Select(o ⇒ o.Operation).Should().Equal(VmOperation.Stop, VmOperation.Change, VmOperation.Start)
-
ChangingAStoppedVm_ShouldStopAndChangeButNotRestart: result.Operations.Select(o ⇒ o.Operation).Should().Equal(VmOperation.Stop, VmOperation.Change)
-
ChangingNothing_ShouldRecordNoOperation: result.Operations.Should().BeEmpty()
-
CommitOrder_ShouldBeDeleteThenUpdateThenInsert: result.Operations.Select(o ⇒ o.Operation).Should().ContainInOrder(VmOperation.Delete, VmOperation.Stop, VmOperation.Change, VmOperation.Start, VmOperation.Insert)
-
PendingChanges_BeforeCommit_ShouldTouchNothing: operations.Recorded.Should().BeEmpty()
-
Dispose_WithoutCommit_ShouldTouchNothing: operations.Recorded.Should().BeEmpty()
-
Operations ordinals .Should().BeInAscendingOrder()
-
PartialFailure_ShouldThrowListingOnlyFailedKeys: exception.Message.Should().Contain(SomeVmSerial) and .Should().NotContain(SomeOtherVmSerial)
-
❏ Create NETSetup-side
VmDesiredStateBuilder(pure, no IO): Createsrc/NETSetup/Helpers/VmDesiredStateBuilder.cs(public static).Build(NETSetupConfig config, SerialNumber hostSerial)→IReadOnlyList<VirtualMachineSettings>. MUST delegate ordering toCreateVirtualMachines.GetOrderedVmsForHost- do not reimplement the OPNsense-first / DC-second / NixDns-last rule. Maps eachMachineto aVirtualMachineSettings(KeyfromSerialNumber,NamefromMachineName,OperatingSystemfromMachine.OperatingSystem.Name, sizing from the same template defaultsProvisionNixosVmalready applies). Zero IO, zero hypervisor calls, no logger. Assertions: -
Build with an empty config .Should().BeEmpty()
-
Build .Should().HaveCount(config machines for this host)
-
Build .Should().NotContain(a machine whose serial targets another host)
-
Build.Select(r ⇒ r.Name).Should().Equal(CreateVirtualMachines.GetOrderedVmsForHost(config, hostSerial).Select(m ⇒ m.MachineName.Value))
-
Build.First().OperatingSystem.Should().Be(OS.OPNSense.Name)
-
Build.Last().OperatingSystem.Should().Be(NetsetupOS.NixDns.Name)
-
Build keys .Should().OnlyHaveUniqueItems()
-
Build called twice with the same config .Should().BeEquivalentTo(each other)
-
❏ Create NETSetup-side
VmDataStorePlanner(pure diff, delete-safety guarded): Createsrc/NETSetup/Helpers/VmDataStorePlanner.cs(public static) +VmChangePlanrecord (IReadOnlyList<VirtualMachineSettings> Inserts,Updates,Deletes).Plan(desired, live, hostSerial)diffs onVirtualMachineKey: in desired but not live →Inserts; in both but with a differing field →Updates; in live but not desired →Deletes, but only when the live record’s serial is hosted onhostSerial(CreateVirtualMachines.IsForHost). A hand-created VM, or one belonging to another host, is never proposed for deletion.VmChangePlanhas noApplyin this iteration - read-only by design sonetsetup updatebehaviour is untouched. Assertions: -
Plan(desired: one new, live: empty).Inserts.Should().HaveCount(SingleItemCount)
-
Plan(desired: empty, live: one hosted).Deletes.Should().HaveCount(SingleItemCount)
-
Plan(desired: empty, live: one foreign-host serial).Deletes.Should().BeEmpty()
-
Plan(desired: empty, live: one unhosted hand-created serial).Deletes.Should().BeEmpty()
-
Plan(identical desired and live).Inserts.Should().BeEmpty() && .Updates.Should().BeEmpty() && .Deletes.Should().BeEmpty()
-
Plan(same key, different MemoryBytes).Updates.Should().HaveCount(SingleItemCount)
-
Plan(same key, different Name).Updates.Should().HaveCount(SingleItemCount)
-
Plan(same key, different Name).Inserts.Should().BeEmpty()
-
typeof(VmChangePlan).GetMethod("Apply").Should().BeNull()
-
Plan called twice with the same inputs .Should().BeEquivalentTo(each other)
-
❏ Add the behaviour-neutrality regression tests: Create
src/NETSetup.Tests/HelperTests/VmDataStoreBehaviourNeutralityTests : UnitTestsBaseproving the new layer changed nothing.EnsureCustomerVms.RunCoremust still produce the exact outcome sequence it produces today for the same input, and must not reference the new types.UpdateMethods.RunProxmoxUpdatemust not callVmDataStorePlanneror open a VM data store in this iteration. Assertions: -
EnsureCustomerVms.RunCore with the existing fixture: outcomes .Should().BeEquivalentTo(the pre-change expected sequence)
-
EnsureCustomerVms.RunCore dispatch order .Should().Equal(DC first, then the rest)
-
EnsureCustomerVms.cs source .Should().NotContain(nameof(VmDataStorePlanner))
-
EnsureCustomerVms.cs source .Should().NotContain("VirtualizationUnitOfWork")
-
UpdateMethods.RunProxmoxUpdate source .Should().NotContain(nameof(VmDataStorePlanner))
-
CreateVirtualMachines.GetOrderedVmsForHost .Should().Equal(the golden fixture captured before this feature)
-
dotnet test src/NETSetup.sln: every previously green test .Should().still be green
-
❏ Document the DataStore VM model in
netsetup.adoc+ a sequence diagram: Add a "Virtual machines as database records" section to the repository-rootnetsetup.adocexplaining the record/provider/Insert/Delete/stop-change-restart model, the Hyper-V and Proxmox provider mapping, theAddReference3two-provider registration, and that the planner is read-only for now. Addsrc/NETSetup/docs/NETSetupVmDataStore.pumlshowingNETSetupConfig→VmDesiredStateBuilder→VmDataStorePlanner→IVirtualizationUnitOfWork.Commit→ Delete / Stop-Change-Start / Insert against the provider. Add a siblingNetsetupAdocVmDataStoreTestsnext toNetsetupAdocProvisionVmsTests, anchoring the repo root on.gitexactly as the existing class does. Assertions: -
adocContent.Should().Contain("Virtual machines as database records")
-
adocContent.Should().Contain("Insert")
-
adocContent.Should().Contain("stop")
-
adocContent.Should().Contain("restart")
-
adocContent.Should().Contain("IVirtualizationUnitOfWork")
-
adocContent.Should().Contain("AddReference3")
-
adocContent.Should().Contain("NETSetupVmDataStore.puml")
-
pumlFileExists.Should().BeTrue()
-
pumlContent.Should().Contain("@startuml")
-
pumlContent.Should().Contain("VmDesiredStateBuilder")
-
pumlContent.Should().Contain("VmDataStorePlanner")
-
pumlContent.Should().Contain("Stop")
-
pumlContent.Should().Contain("Change")
-
pumlContent.Should().Contain("Start")
-
4. Done
4.1. osisa-inventory
GOAL: Active network discovery library. Sweep address, find device NOBODY told it about - any OS, no agent, no credential - and classify them. Fill the gap the existing tooling leave: scan-network verb only CIM-probe KNOWN Windows host over PS-remoting, NETSetup.Helpers.ArpScanner is a MikroTik-flow primitive, osisa.LanSweeper only READ the commercial LanSweeper product DB. Full detail: osisa.Inventory.adoc.
Decisions:
-
New project pair
src/osisa.Inventory/+src/osisa.Inventory.Tests/, wired intosrc/NETSetup.sln. Same shape asosisa.Mikrotik/osisa.OPNSense: packable lib (Package.Build.props),$(DefaultNetTargetFramework),InternalsVisibleTothe test project. -
Three probe behind one
IHostProbeinterface, scanner run ALL per address and merge:ArpHostProbe(Win32SendARP→ MAC, Windows + same-segment only),IcmpHostProbe(Ping→ round-trip, cross-segment but silent on ICMP-blocking host),TcpPortHostProbe(TcpClientover port list → open port, the one that feed classification). Rationale: no single technique see every device. -
Probe NEVER throw for unreachable host - return
HostProbeOutcome.NoResponse. Only bad INPUT throw. Address answering nothing is dropped from the result: a scan report device, never silence. -
ScanTargetExpanderCOUNT before it materialize → mistyped/8throwScanTargetLimitExceededExceptioninstead of 16 million probe. Concurrency bounded viaSemaphoreSlimfromInventoryScanRequest.MaxConcurrency. -
Reuse osisa VO for address / port / name:
IP,Port,MachineNamefromosisa.SystemManagement.Contracts. New Vogen VO only where none exist:MacAddress(canonicalAA:BB:CC:DD:EE:FF+FromAnyFormatfor dash/dot/bare/lowercase wire input),OuiPrefix,VendorName(withUnknowninstance). -
MacAddress.TryFromAnyFormatuseout MacAddress?NOTout MacAddress- Vogen VOG009 prohibit handing outdefault. -
DeviceClassifierrule are ORDERED most-specific-first, port + vendor only. SMB+SSH → Nas BEFORE the Windows rule, else every NAS read as Windows. -
OuiVendorTable.Defaultis a CURATED STARTER SET (~35 entry), NOT the IEEE registry, and NOT verified against it. Extensible by ctor injection. Full-registry file load deliberately not wired. -
ArpHostProbemust qualifySystem.OperatingSystem.IsWindows()-osisa.SystemManagement.Contracts.Elementsalso defineOperatingSystem. -
Library stand ALONE: no CLI verb, no
netsetup updatehook, noNETSetupConfigfeed. Deliberate - keep the scope to the project. -
"Scan the whole network" need NO configured range:
ILocalNetworkProvider/LocalNetworkProviderread the machine own NIC and emit CIDR specifier straight intoInventoryScanRequest.Targets. Skip down link, loopback, tunnel, link-local169.254.*, non-contiguous netmask, and anything wider thanInventoryScanDefaults.MinimumLocalPrefixLength(/22). Floor exist because a NIC on a/16is near always VPN or container bridge - nobody mean to walk 65k address. -
Live-wire test gated the repo way: class-level
[Ignore("manual: …")]+[TestCategory("Integration")], same asPhysicalOpnsenseIntegrationTests. Never run in a normaldotnet test. -
LocalNetworkProviderTestsDO touch the real NIC but assert only SHAPE (prefix floor, no loopback, CIDR the expander agree with, own address inside own network) - suite stay green on any box, including one with no usable NIC. -
PROXY-ARP GUARD (found by actually running the live sweep, NOT by design): Windows
SendARPagainst an address routed at a VPN / TAP adapter get a SYNTHESIZED reply, so EVERY address answer with the same MAC. First live run on a dev box with an OpenVPN TAP adapter (MAC block00:FF:*) report all 254 address of192.168.4.0/24as device.NetworkScanner.RemoveProxyArpArtifactsnow drop a group when the address answered ARP ONLY and more thanInventoryScanDefaults.MaxAddressesPerMacAddress(4) address share that MAC. Anything that ALSO answer ICMP/TCP is kept - real evidence outrank the heuristic, so the TAP gateway itself still appear. Discard logged at warning naming MAC + count, never silent.
Status (2026-08-11, uncommitted): 33 production .cs + 16 test .cs. dotnet build clean. dotnet test 89 pass / 3 skipped (the live scan, [Ignore] by design).
Verified on dev box, discovery half via .agents/scratch/show-local-networks.cs (zero traffic, NIC read only): LocalNetworkProvider resolve 2 attached network - 192.168.4.0/24 (Local Area Connection, a TAP adapter) + 192.168.3.0/24 (vEthernet External_WIFI) = 508 address.
Verified LIVE (authorized run against the osisa corp LAN, 1 m 13 s): 508 address probed → 252 ARP-only ghost discarded → 42 real device. Result cross-check against known inventory: osisaLib01 .200 + osisaLib02 .202 + .215 + .217 + .233 all classified Hypervisor off port 8006, osisaDC06 .201 Windows, CHOCOSERVER .220, SAGESERVER .221, osisaNextcloud .225 Linux, FWROE04 .254 (ICMP only, no port), a Printer at .241 off 9100/515/631. Classification and reverse-DNS both behaving.
OPEN DEFECT from that run: ArpHostProbe pass sourceIp = 0 to SendARP, so Windows choose the outgoing interface and on a TAP-equipped box choose wrong - MAC resolved for only 3 of 42 device, so vendor read Unknown for the rest. Fix: inject ILocalNetworkProvider into the probe, match the target to the NIC whose subnet hold it, pass THAT address as sourceIp. Not applied yet.
Timing note for anyone running the live test: three probe run in SEQUENCE per address and an unanswered Win32 SendARP block ~2 s, so budget ~3 s per 64 silent address. One mostly-empty /24 at concurrency 128 is ~8-15 s. dotnet build clean (0 error). dotnet test 76/76 green, all offline via StubHostProbe + StubHostNameResolver - nothing touch the wire.
-
✓ Project scaffold + VO + entity + exception + contract + service + DI extension,
src/osisa.Inventory/. -
✓ Test project
src/osisa.Inventory.Tests/- 85 offline test overMacAddress,OuiPrefix,ScanTargetExpander(incl. limit ceiling),LocalNetworkProvider,OuiVendorResolver,DeviceClassifier,NetworkScanner(responding + silent host),AddInventoryregistration. -
✓ Whole-network scan:
ILocalNetworkProvider+LocalNetworkProvider+InventoryScanDefaults.MinimumLocalPrefixLength, registered inAddInventory. -
✓ Live example
src/osisa.Inventory.Tests/LiveNetworkScanIntegrationTests.cs- 3 test (sweep every attached network / assert answering method recorded / sweep env-configured range viaNETSETUP_INVENTORY_RANGE), prints an address+kind+mac+vendor+hostname+port table to the test output. -
✓ Project doc
src/osisa.Inventory/osisa.Inventory.adoc- surface, classify rule table, safety, OUI-table caveat, limit. -
✓ Proxy-ARP guard
NetworkScanner.RemoveProxyArpArtifacts+InventoryScanDefaults.MaxAddressesPerMacAddress, covered byNetworkScannerProxyArpTests(ARP-only shared MAC → dropped) andNetworkScannerSharedMacWithRealEvidenceTests(shared MAC + ICMP → kept). -
✓ Free-tool evaluation vs LanSweeper, measured 2026-08-11/12. LanSweeper is commercial + closed, so it is a BENCHMARK not an option (
src/osisa.LanSweeperonly read one customer’s existing install). Full requirement matrix, every measured number, licensing analysis, blocker list and phased plan: docs/inventory-stack-evaluation.adoc. Verdict: no single free LanSweeper exist, but a free STACK cover it - GLPI + GLPI-Agent (GPLv2) match per-asset depth, nmap win discovery, Lextm.SharpSnmpLib (MIT) own the gear layer. Licensing catch: nmap is NPSL, so invoke-as-prerequisite only, NEVER bundle in NETSetup without a paid OEM license; SharpSnmpLib is MIT and IS shippable. The real gap is not tooling - it is no scanning credential (remote WMI/DCOMZugriff verweigert, WinRM filtered) and SNMP enabled on exactly 1 of 254 device. -
❏ OPEN (supersedes the
sourceIpfix): wrap nmap behind a NETCommand builder + parse-oXXML, keeposisa.Inventorydomain model on top. Retire the curated OUI table AND theArpHostProbesourceIp = 0defect in one move, and gain service version + OS fingerprint. Cheaper than reimplementing probes. -
❏ OPEN: pull ARP + DHCP lease table from MikroTik / OPNsense over API instead of sweeping - zero traffic, immune to the routing trap, and
osisa.Mikrotik+osisa.OPNSense.Restclient already exist in this repo. Highest value per effort. -
❏ OPEN (infra, not code): credentialed inventory is blocked on this network - remote WMI/DCOM returns
Zugriff verweigertand WinRM 5985 isfiltered. Needs a dedicated scanning account with local-admin + firewall rule before ANY tool can reach LanSweeper depth. -
❏ OPEN:
scan-inventoryCLI verb insrc/NETSetup/CLI/Commands/Network/writing JSON via the existingScanOutputPath/ScanProbeRunneroutput convention. -
❏ OPEN: load full IEEE MA-L registry from file instead of the curated table.
-
❏ OPEN: feed discovered device into
NETSetupConfig/ customer.csdraft, or push to an infrastructure DB (NetBox / Snipe-IT). Parked reference source for the push shape:src/osisa.Inventory/Reference/ExternalNetworkScannerSample.cs(external ICMP-only sample, fully commented out,prefix = parked). Read its header before borrowing - it hardcode the API token and fire 254 unbounded ping. The osisa way is a NETCommand builder orosisa.Rest.RepositoryUnitOfWork, not rawHttpClient.