From: Crystal Kolipe Subject: Re: [EXT] Re: vmd(8): Use 32-bit direct kernel launch for both amd64 and i386 To: tech@openbsd.org Date: Wed, 20 Aug 2025 17:38:21 +0100 On Wed, Aug 20, 2025 at 05:51:46PM +0200, Hans-Jrg Hxer wrote: > On Wed, Aug 20, 2025 at 04:27:44PM +0100, Crystal Kolipe wrote: > > On Wed, Aug 20, 2025 at 04:23:12PM +0200, Hans-Jrg Hxer wrote: > > > On Wed, Aug 20, 2025 at 02:23:45PM +0100, Crystal Kolipe wrote: > > > > On Wed, Aug 20, 2025 at 02:24:34PM +0200, Hans-Jrg Hxer wrote: > > > > > The diff can be tested with amd64 and i386 ramdisk kernels like this: > > > > > > > > > > # vmctl start -c -b i386/bsd.rd myvm > > > > > # vmctl start -c -b amd64/bsd.rd myvm > > > > > > > > > > Using a BIOS boot image (eg. /etc/firmware/vmm-bios) is not affected by > > > > > this change. > > > > > > > > On a few of our systems this diff is throwing vm_resetcpu: failed, and the > > > > corresponding vms don't start. > > > > > > > > Does this change rely on any other recent commits? > > > > > > > > If not, I suspect that some of our local changes are breaking it. > > > > > > > > We have various production systems deployed with vms that boot directly in to > > > > the kernel without using a bios image, including several custom kernel > > > > configs, so I would like to test this on them, but unfortunately most of the > > > > interesting systems are running a heavily modified 7.7-release rather than > > > > -current. > > > > > > > > > > First of all: Thanks for testing! > > > > > > This diff does rely on commits more recent than 7.7-release. It won't > > > apply cleanly on 7.7-release sources. To understand what you are doing: > > > You are running a currnet vmd with this patch on a 7.7-release based > > > system? > > > > Not exactly, the host systems are running 7.7-release with various local > > changes, (unrelated to vmd). I just updated loadfile_elf.c to HEAD, (one > > commit by bluhm@), and then applied your patch. > > > > When I first read the diff, I thought it was a no-op for booting i386 kernels, > > so I was surprised that that broke. Looking more closely, I see that you're > > setting bits in CR0 that were not set before. Reverting that makes i386 vms > > boot again on this hacked up 7.7-release system, which is what I would expect. > > yes, it's not a no-op for i386. Up to now i386 is launched with paging > already enabled. And running on a page table provided by vmd. When doing > a firmware boot, our boot loaders setup a flat 32-bit environment with > paging disabled. So that's different to what we do on direct launch. > > We could keep paging enabled. However, amd64 locore0 enables PAE to > use 64-bit page table entries. This needs to be done before paging is > enabled (order is set PAE, then set PG). That's why I now disable paging. > > Do your modified systems actually boot when using firmware boot? Yes, firmware booting works as expected. But this configuration is rarely tested, in fact most of our host systems don't even have a bios image installed.