I've been running with this for a while, no regressions noticed.

11 active VMs, lots of crosstalk network and disk IO, and lots of
outbound network IO.

-peter


On 2026 Aug 06 (Thu) at 17:09:35 -0400 (-0400), Dave Voutila wrote:
:Dave Voutila <dv@sisu.io> writes:
:
:Another update, fixing building GENERIC. Missed some ifdefs for
:MULTIPROCESSOR.
:
:> Updated diff that applies to the tree again. Recent commit to vmd(8)
:> broke the original diff.
:>
:> There's a different vmd commit I'm making later today but by my testing
:> that shouldn't conflict with this diff.
:>
:> Dave Voutila <dv@sisu.io> writes:
:>
:>> Calling on vmm(4)/vmd(8) users to give this diff a test please!
:>>
:>> This is a major breaking change I'm proposing to land soon. It changes
:>> the vmm(4) design to create a new file type for each VM. When a VM is
:>> created, the caller gets a new file descriptor for using ioctl(2) calls
:>> against that VM.
:>>
:>> This fundamentally changes the design from knowing a "magic number"
:>> (today, the vm id) to possessing a capability (the open file
:>> descriptor).
:>>
:>> Why do this?
:>>
:>>  - Better isolate which process can manipulate a VM. Today it's a hacky
:>>    combination of knowing the magic number and vmm(4) tracking an owning
:>>    PID. The current design scares me.
:>>
:>>  - Better scope the vmm(4) API into basically a control plane vs. data
:>>    plane design. A privileged process with an open /dev/vmm special file
:>>    can launch/terminate VMs, but the manipulation of each VM and VCPU is
:>>    done via its own dedicated file.
:>>
:>>  - Using files helps simplify the lifecycle management of VMs in the
:>>    kernel by tying their lifetime to a file. vmd(8) processes emulating
:>>    the VM will have their file descriptor closed by the kernel even if
:>>    the process is killed (e.g. via segfault). This simplifies cleanup of
:>>    dead VMs.
:>>
:>> Bonus: there's a slight chance this improves perf slightly, but since
:>> that's not the goal I'm not benchmarking. Given a new ioctl(2) path for
:>> the new VM file type, it may reduce contention on the kernel lock.
:>>
:>> Note: this is unlike Linux/KVM which uses not only a file per VM, but
:>> one per VCPU. I see no reason for that level of complication at the moment.
:>>
:>> I don't expect detailed review yet given the size of this. I plan on
:>> sending separate mails breaking this down into kernel vs. userland (the
:>> userland stuff is a lot of deck chair shuffling) as well as updates to
:>> vmm.4 man page and fstat(1).
:>>
:>> To test:
:>>
:>> 1. build and install new kernel and reboot
:>> 2. symlink (recommended) or install headers in /usr/include:
:>>      amd64/vmmvar.h --> sys/arch/amd64/include/vmmvar.h
:>>      dev/vmm/vmm.h --> sys/dev/vmm/vmm.h
:>>      dev/ic/pspvar.h --> sys/dev/ic/pspvar.h
:>
:> Also, you need sys/file.h --> sys/sys/file.h if building fstat(1).
:>
:>> 3. build and install vmd(8) and vmctl(8) (if this fails, check your
:>>    headers in /usr/include/ point to the patched ones in the tree)
:>> 4. run your existing VMs...you should see no discernable difference in
:>>    behavior.
:>>
:>> If things break or behavior changes, please report to me including:
:>>
:>> * cpu0 details from dmesg(1)
:>> * details on how the vm is started (/etc/vm.conf contents or vmctl(8)
:>>   args)
:>> * vmd(8) debug output if any (try running vmd in the foreground via
:>>   something like: # $(which vmd) -dvv
:>>
:>> ** I specifically need help testing SEV-related usage as I have no
:>> commercial grade AMD hardware with all the cool SEV stuff. **
:>>
:>> Once I land this I'll no longer lose sleep worrying about the current
:>> design and can turn my attentions to MMIO/instruction emulation needed
:>> for getting to SMP ;)
:>>
:>> Thanks!
:>>