Index | Thread | Search

From:
Dave Voutila <dv@sisu.io>
Subject:
Re: Reworking VMM's nested paging & guest memory (de-vmspace-ification)
To:
Alexander Bluhm <alexander.bluhm@gmx.net>
Cc:
tech@openbsd.org, mlarkin@openbsd.org, Hans-Joerg_Hoexer@genua.de
Date:
Tue, 15 Apr 2025 15:56:20 -0400

Download raw body.

Thread
Alexander Bluhm <alexander.bluhm@gmx.net> writes:

> On Mon, Apr 14, 2025 at 08:18:46PM -0400, Dave Voutila wrote:
>> OKs welcome, but I'd prefer some test reports before looking to commit,
>> but since we're recently unlocked from release prep, I'd prefer sooner
>> rather than later.
>
> It seems to work on my AMD CPU.  But with AMD SEV enabled, guest
> does not start.
>

Hmm. My AMD machine is too old and doesn't have a psp(4). :(

> Apr 15 20:14:37 ot52 vmd[99497]: psp_activate: ioctl: Input/output error
> Apr 15 20:14:37 ot52 vmd[99497]: sev_activate: failed to activate guest: 0x1:0x3

This is a pretty early failure. From my reading of sev.c it doesn't seem
to even have anything to do with the guest memory unless I'm missing
something.

This seems odd. Hans-Joerg, any idea? I'm pretty sure I didn't touch
anything to do with asids or SEV. The guest memory is mapped into the
process differently, but at this point it should be mapped and
available to vmd...otherwise other things would fail.

> Apr 15 20:14:37 ot52 vmd[99497]: vmd: SEV activatation failed for VCPU 0 failed - exiting.
> Apr 15 20:14:37 ot52 vmd[99497]: psp_guest_shutdown: ioctl: Input/output error
> Apr 15 20:14:37 ot52 vmd[99497]: failed to deactivate guest
> Apr 15 20:14:37 ot52 vmd[99497]: start_vm: could not shutdown SEV
> Apr 15 20:14:37 ot52 vmd[57809]: SIOCSIFDESCR: Device not configured
> Apr 15 20:14:37 ot52 vmd[57809]: SIOCBRDGADD: Invalid argument
> Apr 15 20:14:37 ot52 vmd[57809]: SIOCSIFRDOMAIN: Device not configured
> Apr 15 20:14:37 ot52 vmd[57809]: SIOCGIFFLAGS: Device not configured
>
> OpenBSD 7.7-current (GENERIC.MP) #0: Tue Apr 15 16:36:45 CEST 2025
>     root@ot52.obsd-lab.genua.de:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 274654408704 (261930MB)
> avail mem = 266303348736 (253966MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 3.3 @ 0xa9d1c000 (75 entries)
> bios0: vendor American Megatrends Inc. version "2.5" date 09/26/2022
> bios0: genua gsXXL
> efi0 at bios0: UEFI 2.8
> efi0: American Megatrends rev 0x50016
> acpi0 at bios0: ACPI 6.0
> acpi0: sleep states S0 S5
> acpi0: tables DSDT FACP SSDT SPMI SSDT FIDT MCFG SSDT SSDT BERT HPET IVRS PCCT SSDT CRAT CDIT SSDT WSMT APIC ERST HEST
> acpi0: wakeup devices B000(S3) C000(S3) B010(S3) C010(S3) B030(S3) C030(S3) B020(S3) C020(S3) B100(S3) C100(S3) B110(S3) C110(S3) B130(S3) C130(S3) B120(S3) C120(S3)
> acpitimer0 at acpi0: 3579545 Hz, 32 bits
> acpimcfg0 at acpi0
> acpimcfg0: addr 0xe0000000, bus 0-255
> acpihpet0 at acpi0: 14318180 Hz
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu0: cpuid 1
> edx=178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT>
> ecx=76da320b<SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND>
> cpu0: cpuid 6 eax=4<ARAT> ecx=1<EFFFREQ>
> cpu0: cpuid 7.0 ebx=219c97a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA> ecx=40068c<UMIP,PKU> edx=10
> cpu0: cpuid d.1 eax=f<XSAVEOPT,XSAVEC,XGETBV1,XSAVES>
> cpu0: cpuid 80000001 edx=2fd3fbff<NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG>
> ecx=75c237ff<LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX>
> cpu0: cpuid 80000007 edx=6799<HWPSTATE,ITSC>
> cpu0: cpuid 80000008 ebx=91bef75f<INVLPGB,IBPB,IBRS,STIBP,STIBP_ALL,IBRS_PREF,IBRS_SM,SSBD>
> cpu0: cpuid 8000001F eax=101fd3f<SME,SEV,PFLUSH_MSR,SEVES,SEVSNP,VMPL,SECTSC,HWECACHECOH,REQ64BITHOST,RESTINJ,ALTINJ,DBGSTSW,IBSDISALLOW,VTE,VMSARPROT> ecx=1fd edx=5
> cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB 64b/line 8-way L2 cache, 32MB 64b/line 16-way L3 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
> cpu0: apic clock running at 100MHz
> cpu0: mwait min=64, max=64, C-substates=1.1, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu3: smt 0, core 3, package 0
> cpu4 at mainbus0: apid 8 (application processor)
> cpu4: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu4: smt 0, core 4, package 0
> cpu5 at mainbus0: apid 10 (application processor)
> cpu5: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu5: smt 0, core 5, package 0
> cpu6 at mainbus0: apid 12 (application processor)
> cpu6: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu6: smt 0, core 6, package 0
> cpu7 at mainbus0: apid 14 (application processor)
> cpu7: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu7: smt 0, core 7, package 0
> cpu8 at mainbus0: apid 16 (application processor)
> cpu8: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu8: smt 0, core 8, package 0
> cpu9 at mainbus0: apid 18 (application processor)
> cpu9: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu9: smt 0, core 9, package 0
> cpu10 at mainbus0: apid 20 (application processor)
> cpu10: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu10: smt 0, core 10, package 0
> cpu11 at mainbus0: apid 22 (application processor)
> cpu11: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu11: smt 0, core 11, package 0
> cpu12 at mainbus0: apid 24 (application processor)
> cpu12: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu12: smt 0, core 12, package 0
> cpu13 at mainbus0: apid 26 (application processor)
> cpu13: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu13: smt 0, core 13, package 0
> cpu14 at mainbus0: apid 28 (application processor)
> cpu14: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu14: smt 0, core 14, package 0
> cpu15 at mainbus0: apid 30 (application processor)
> cpu15: AMD EPYC 73F3 16-Core Processor, 3500.00 MHz, 19-01-01, patch 0a0011d5
> cpu15: smt 0, core 15, package 0
> cpu16 at mainbus0: apid 1 (application processor)
> cpu16: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu16: smt 1, core 0, package 0
> cpu17 at mainbus0: apid 3 (application processor)
> cpu17: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu17: smt 1, core 1, package 0
> cpu18 at mainbus0: apid 5 (application processor)
> cpu18: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu18: smt 1, core 2, package 0
> cpu19 at mainbus0: apid 7 (application processor)
> cpu19: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu19: smt 1, core 3, package 0
> cpu20 at mainbus0: apid 9 (application processor)
> cpu20: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu20: smt 1, core 4, package 0
> cpu21 at mainbus0: apid 11 (application processor)
> cpu21: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu21: smt 1, core 5, package 0
> cpu22 at mainbus0: apid 13 (application processor)
> cpu22: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu22: smt 1, core 6, package 0
> cpu23 at mainbus0: apid 15 (application processor)
> cpu23: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu23: smt 1, core 7, package 0
> cpu24 at mainbus0: apid 17 (application processor)
> cpu24: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu24: smt 1, core 8, package 0
> cpu25 at mainbus0: apid 19 (application processor)
> cpu25: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu25: smt 1, core 9, package 0
> cpu26 at mainbus0: apid 21 (application processor)
> cpu26: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu26: smt 1, core 10, package 0
> cpu27 at mainbus0: apid 23 (application processor)
> cpu27: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu27: smt 1, core 11, package 0
> cpu28 at mainbus0: apid 25 (application processor)
> cpu28: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu28: smt 1, core 12, package 0
> cpu29 at mainbus0: apid 27 (application processor)
> cpu29: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu29: smt 1, core 13, package 0
> cpu30 at mainbus0: apid 29 (application processor)
> cpu30: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu30: smt 1, core 14, package 0
> cpu31 at mainbus0: apid 31 (application processor)
> cpu31: AMD EPYC 73F3 16-Core Processor, 3500.01 MHz, 19-01-01, patch 0a0011d5
> cpu31: smt 1, core 15, package 0
> ioapic0 at mainbus0: apid 240 pa 0xfec00000, version 21, 24 pins, can't remap
> ioapic1 at mainbus0: apid 241 pa 0xb5400000, version 21, 32 pins, can't remap
> ioapic2 at mainbus0: apid 242 pa 0xf4180000, version 21, 32 pins, can't remap
> ioapic3 at mainbus0: apid 243 pa 0xb4180000, version 21, 32 pins, can't remap
> ioapic4 at mainbus0: apid 244 pa 0xf5180000, version 21, 32 pins, can't remap
> acpiprt0 at acpi0: bus 192 (S0D0)
> acpiprt1 at acpi0: bus -1 (D0A0)
> acpiprt2 at acpi0: bus -1 (D0A1)
> acpiprt3 at acpi0: bus 193 (D0A2)
> acpiprt4 at acpi0: bus 194 (D0A3)
> acpiprt5 at acpi0: bus 195 (D0A4)
> acpiprt6 at acpi0: bus -1 (BR14)
> acpiprt7 at acpi0: bus 197 (D0A5)
> acpiprt8 at acpi0: bus 198 (D0A6)
> acpiprt9 at acpi0: bus 199 (B000)
> acpiprt10 at acpi0: bus 200 (C000)
> acpiprt11 at acpi0: bus -1 (C001)
> acpiprt12 at acpi0: bus -1 (C002)
> acpiprt13 at acpi0: bus 128 (S0D1)
> acpiprt14 at acpi0: bus -1 (D1A0)
> acpiprt15 at acpi0: bus -1 (D1A1)
> acpiprt16 at acpi0: bus -1 (D1A2)
> acpiprt17 at acpi0: bus -1 (D1A3)
> acpiprt18 at acpi0: bus 129 (D1B0)
> acpiprt19 at acpi0: bus 130 (D1B1)
> acpiprt20 at acpi0: bus 131 (B010)
> acpiprt21 at acpi0: bus 132 (C010)
> acpiprt22 at acpi0: bus -1 (C011)
> acpiprt23 at acpi0: bus -1 (C012)
> acpiprt24 at acpi0: bus 0 (PCI0)
> acpiprt25 at acpi0: bus -1 (D3A0)
> acpiprt26 at acpi0: bus -1 (D3A1)
> acpiprt27 at acpi0: bus -1 (D3B0)
> acpiprt28 at acpi0: bus -1 (D3B1)
> acpiprt29 at acpi0: bus -1 (D3B2)
> acpiprt30 at acpi0: bus -1 (D3B3)
> acpiprt31 at acpi0: bus 1 (B030)
> acpiprt32 at acpi0: bus 2 (C030)
> acpiprt33 at acpi0: bus -1 (C031)
> acpiprt34 at acpi0: bus -1 (C032)
> acpiprt35 at acpi0: bus 64 (S0D2)
> acpiprt36 at acpi0: bus -1 (D2A0)
> acpiprt37 at acpi0: bus -1 (D2A1)
> acpiprt38 at acpi0: bus -1 (D2B0)
> acpiprt39 at acpi0: bus -1 (D2B1)
> acpiprt40 at acpi0: bus -1 (D2B2)
> acpiprt41 at acpi0: bus -1 (D2B3)
> acpiprt42 at acpi0: bus 65 (B020)
> acpiprt43 at acpi0: bus 66 (C020)
> acpiprt44 at acpi0: bus 67 (C021)
> acpiprt45 at acpi0: bus 68 (C022)
> acpipci0 at acpi0 S0D0: 0x00000000 0x00000011 0x00000001
> acpipci1 at acpi0 S0D1: 0x00000000 0x00000011 0x00000001
> acpipci2 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
> acpicmos0 at acpi0
> com0 at acpi0 UAR1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
> com1 at acpi0 UAR2 addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
> com1: console
> "IPI0001" at acpi0 not configured
> amdgpio0 at acpi0 GPIO uid 0 addr 0xfed81500/0x400 irq 7, 184 pins
> acpipci3 at acpi0 S0D2: 0x00000000 0x00000011 0x00000001
> acpibtn0 at acpi0: PWRB
> acpicpu0 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu1 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu2 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu3 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu4 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu5 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu6 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu7 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu8 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu9 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu10 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu11 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu12 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu13 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu14 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu15 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu16 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu17 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu18 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu19 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu20 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu21 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu22 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu23 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu24 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu25 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu26 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu27 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu28 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu29 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu30 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> acpicpu31 at acpi0: C2(0@30 io@0x814), C1(0@1 mwait), PSS
> "PNP0C33" at acpi0 not configured
> "PNP0C14" at acpi0 not configured
> acpipwrres0 at acpi0: P0SA
> acpipwrres1 at acpi0: P3SA
> acpipwrres2 at acpi0: P0SA
> acpipwrres3 at acpi0: P3SA
> acpipwrres4 at acpi0: P0SA
> acpipwrres5 at acpi0: P3SA
> acpipwrres6 at acpi0: P0SA
> acpipwrres7 at acpi0: P3SA
> acpipwrres8 at acpi0: P0SA
> acpipwrres9 at acpi0: P3SA
> acpipwrres10 at acpi0: P0SA
> acpipwrres11 at acpi0: P3SA
> acpipwrres12 at acpi0: P0SA
> acpipwrres13 at acpi0: P3SA
> acpipwrres14 at acpi0: P0SA
> acpipwrres15 at acpi0: P3SA
> acpipwrres16 at acpi0: P0SA
> acpipwrres17 at acpi0: P3SA
> acpipwrres18 at acpi0: P0SA
> acpipwrres19 at acpi0: P3SA
> acpipwrres20 at acpi0: P0SA
> acpipwrres21 at acpi0: P3SA
> acpipwrres22 at acpi0: P0SA
> acpipwrres23 at acpi0: P3SA
> acpipwrres24 at acpi0: P0SA
> acpipwrres25 at acpi0: P3SA
> acpipwrres26 at acpi0: P0SA
> acpipwrres27 at acpi0: P3SA
> acpipwrres28 at acpi0: P0SA
> acpipwrres29 at acpi0: P3SA
> acpipwrres30 at acpi0: P0SA
> acpipwrres31 at acpi0: P3SA
> ipmi at mainbus0 not configured
> cpu0: 3500 MHz: speeds: 3500 2500 1500 MHz
> pci0 at mainbus0 bus 192
> ksmn0 at pci0 dev 0 function 0 "AMD 17h Root Complex" rev 0x00
> vendor "AMD", unknown product 0x164f (class system subclass IOMMU, rev 0x01) at pci0 dev 0 function 2 not configured
> pchb0 at pci0 dev 1 function 0 "AMD 17h Host" rev 0x00
> pchb1 at pci0 dev 2 function 0 "AMD 17h Host" rev 0x00
> pchb2 at pci0 dev 3 function 0 "AMD 17h Host" rev 0x00
> ppb0 at pci0 dev 3 function 3 "AMD 17h PCIE" rev 0x00: msi
> pci1 at ppb0 bus 193
> xhci0 at pci1 dev 0 function 0 "ASMedia ASM1042A xHCI" rev 0x00: msix, xHCI 1.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "ASMedia xHCI root hub" rev 3.00/1.00 addr 1
> ppb1 at pci0 dev 3 function 4 "AMD 17h PCIE" rev 0x00: msi
> pci2 at ppb1 bus 194
> xhci1 at pci2 dev 0 function 0 "ASMedia ASM1042A xHCI" rev 0x00: msix, xHCI 1.0
> usb1 at xhci1: USB revision 3.0
> uhub1 at usb1 configuration 1 interface 0 "ASMedia xHCI root hub" rev 3.00/1.00 addr 1
> ppb2 at pci0 dev 3 function 5 "AMD 17h PCIE" rev 0x00: msi
> pci3 at ppb2 bus 195
> ppb3 at pci3 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x04
> pci4 at ppb3 bus 196
> "ASPEED Technology AST2000" rev 0x41 at pci4 dev 0 function 0 not configured
> ppb4 at pci0 dev 3 function 6 "AMD 17h PCIE" rev 0x00: msi
> pci5 at ppb4 bus 197
> ahci0 at pci5 dev 0 function 0 "ASMedia ASM1061 AHCI" rev 0x02: msi, AHCI 1.2
> ahci0: port 0: 6.0Gb/s
> scsibus1 at ahci0: 32 targets
> sd0 at scsibus1 targ 0 lun 0: <ATA, Samsung SSD 850, EMT0> naa.5002538d40ff4db3
> sd0: 238475MB, 512 bytes/sector, 488397168 sectors, thin
> ppb5 at pci0 dev 3 function 7 "AMD 17h PCIE" rev 0x00: msi
> pci6 at ppb5 bus 198
> bge0 at pci6 dev 0 function 0 "Broadcom BCM5720" rev 0x00, BCM5720 A0 (0x5720000), APE firmware NCSI 1.5.1.0: msi, address 00:25:90:bc:ff:2a
> brgphy0 at bge0 phy 1: BCM5720C, rev. 0
> bge1 at pci6 dev 0 function 1 "Broadcom BCM5720" rev 0x00, BCM5720 A0 (0x5720000), APE firmware NCSI 1.5.1.0: msi, address 00:25:90:bc:ff:2b
> brgphy1 at bge1 phy 2: BCM5720C, rev. 0
> pchb3 at pci0 dev 4 function 0 "AMD 17h Host" rev 0x00
> pchb4 at pci0 dev 5 function 0 "AMD 17h Host" rev 0x00
> pchb5 at pci0 dev 7 function 0 "AMD 17h Host" rev 0x00
> ppb6 at pci0 dev 7 function 1 "AMD 17h PCIE" rev 0x00
> pci7 at ppb6 bus 199
> vendor "AMD", unknown product 0x148a (class instrumentation unknown subclass 0x00, rev 0x00) at pci7 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci7 dev 0 function 2 not configured
> pchb6 at pci0 dev 8 function 0 "AMD 17h Host" rev 0x00
> ppb7 at pci0 dev 8 function 1 "AMD 17h PCIE" rev 0x00
> pci8 at ppb7 bus 200
> vendor "AMD", unknown product 0x1485 (class instrumentation unknown subclass 0x00, rev 0x00) at pci8 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci8 dev 0 function 2 not configured
> pci9 at mainbus0 bus 128
> ksmn1 at pci9 dev 0 function 0 "AMD 17h Root Complex" rev 0x00
> vendor "AMD", unknown product 0x164f (class system subclass IOMMU, rev 0x01) at pci9 dev 0 function 2 not configured
> pchb7 at pci9 dev 1 function 0 "AMD 17h Host" rev 0x00
> pchb8 at pci9 dev 2 function 0 "AMD 17h Host" rev 0x00
> pchb9 at pci9 dev 3 function 0 "AMD 17h Host" rev 0x00
> ppb8 at pci9 dev 3 function 1 "AMD 17h PCIE" rev 0x00: msi
> pci10 at ppb8 bus 129
> ppb9 at pci9 dev 3 function 2 "AMD 17h PCIE" rev 0x00: msi
> pci11 at ppb9 bus 130
> pchb10 at pci9 dev 4 function 0 "AMD 17h Host" rev 0x00
> pchb11 at pci9 dev 5 function 0 "AMD 17h Host" rev 0x00
> pchb12 at pci9 dev 7 function 0 "AMD 17h Host" rev 0x00
> ppb10 at pci9 dev 7 function 1 "AMD 17h PCIE" rev 0x00
> pci12 at ppb10 bus 131
> vendor "AMD", unknown product 0x148a (class instrumentation unknown subclass 0x00, rev 0x00) at pci12 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci12 dev 0 function 2 not configured
> pchb13 at pci9 dev 8 function 0 "AMD 17h Host" rev 0x00
> ppb11 at pci9 dev 8 function 1 "AMD 17h PCIE" rev 0x00
> pci13 at ppb11 bus 132
> vendor "AMD", unknown product 0x1485 (class instrumentation unknown subclass 0x00, rev 0x00) at pci13 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci13 dev 0 function 2 not configured
> pci14 at mainbus0 bus 0
> ksmn2 at pci14 dev 0 function 0 "AMD 17h Root Complex" rev 0x00
> vendor "AMD", unknown product 0x164f (class system subclass IOMMU, rev 0x01) at pci14 dev 0 function 2 not configured
> pchb14 at pci14 dev 1 function 0 "AMD 17h Host" rev 0x00
> pchb15 at pci14 dev 2 function 0 "AMD 17h Host" rev 0x00
> pchb16 at pci14 dev 3 function 0 "AMD 17h Host" rev 0x00
> pchb17 at pci14 dev 4 function 0 "AMD 17h Host" rev 0x00
> pchb18 at pci14 dev 5 function 0 "AMD 17h Host" rev 0x00
> pchb19 at pci14 dev 7 function 0 "AMD 17h Host" rev 0x00
> ppb12 at pci14 dev 7 function 1 "AMD 17h PCIE" rev 0x00
> pci15 at ppb12 bus 1
> vendor "AMD", unknown product 0x148a (class instrumentation unknown subclass 0x00, rev 0x00) at pci15 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci15 dev 0 function 2 not configured
> pchb20 at pci14 dev 8 function 0 "AMD 17h Host" rev 0x00
> ppb13 at pci14 dev 8 function 1 "AMD 17h PCIE" rev 0x00
> pci16 at ppb13 bus 2
> vendor "AMD", unknown product 0x1485 (class instrumentation unknown subclass 0x00, rev 0x00) at pci16 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci16 dev 0 function 2 not configured
> xhci2 at pci16 dev 0 function 3 vendor "AMD", unknown product 0x148c rev 0x00: msix, xHCI 1.10
> usb2 at xhci2: USB revision 3.0
> uhub2 at usb2 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00 addr 1
> piixpm0 at pci14 dev 20 function 0 "AMD FCH SMBus" rev 0x61: polling
> iic0 at piixpm0
> iic1 at piixpm0
> pcib0 at pci14 dev 20 function 3 "AMD FCH LPC" rev 0x51
> pchb21 at pci14 dev 24 function 0 vendor "AMD", unknown product 0x1650 rev 0x00
> pchb22 at pci14 dev 24 function 1 vendor "AMD", unknown product 0x1651 rev 0x00
> pchb23 at pci14 dev 24 function 2 vendor "AMD", unknown product 0x1652 rev 0x00
> pchb24 at pci14 dev 24 function 3 vendor "AMD", unknown product 0x1653 rev 0x00
> pchb25 at pci14 dev 24 function 4 vendor "AMD", unknown product 0x1654 rev 0x00
> pchb26 at pci14 dev 24 function 5 vendor "AMD", unknown product 0x1655 rev 0x00
> pchb27 at pci14 dev 24 function 6 vendor "AMD", unknown product 0x1656 rev 0x00
> pchb28 at pci14 dev 24 function 7 vendor "AMD", unknown product 0x1657 rev 0x00
> isa0 at pcib0
> isadma0 at isa0
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> pci17 at mainbus0 bus 64
> ksmn3 at pci17 dev 0 function 0 "AMD 17h Root Complex" rev 0x00
> vendor "AMD", unknown product 0x164f (class system subclass IOMMU, rev 0x01) at pci17 dev 0 function 2 not configured
> pchb29 at pci17 dev 1 function 0 "AMD 17h Host" rev 0x00
> pchb30 at pci17 dev 2 function 0 "AMD 17h Host" rev 0x00
> pchb31 at pci17 dev 3 function 0 "AMD 17h Host" rev 0x00
> pchb32 at pci17 dev 4 function 0 "AMD 17h Host" rev 0x00
> pchb33 at pci17 dev 5 function 0 "AMD 17h Host" rev 0x00
> pchb34 at pci17 dev 7 function 0 "AMD 17h Host" rev 0x00
> ppb14 at pci17 dev 7 function 1 "AMD 17h PCIE" rev 0x00
> pci18 at ppb14 bus 65
> vendor "AMD", unknown product 0x148a (class instrumentation unknown subclass 0x00, rev 0x00) at pci18 dev 0 function 0 not configured
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci18 dev 0 function 2 not configured
> pchb35 at pci17 dev 8 function 0 "AMD 17h Host" rev 0x00
> ppb15 at pci17 dev 8 function 1 "AMD 17h PCIE" rev 0x00
> pci19 at ppb15 bus 66
> vendor "AMD", unknown product 0x1485 (class instrumentation unknown subclass 0x00, rev 0x00) at pci19 dev 0 function 0 not configured
> ccp0 at pci19 dev 0 function 1 "AMD 17h Crypto" rev 0x00: msix
> psp0 at ccp0: vers 2, api 1.52, build 4, SEV, SEV-ES
> vendor "AMD", unknown product 0x1498 (class crypto subclass miscellaneous, rev 0x00) at pci19 dev 0 function 2 not configured
> xhci3 at pci19 dev 0 function 3 vendor "AMD", unknown product 0x148c rev 0x00: msix, xHCI 1.10
> usb3 at xhci3: USB revision 3.0
> uhub3 at usb3 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00 addr 1
> ppb16 at pci17 dev 8 function 2 "AMD 17h PCIE" rev 0x00
> pci20 at ppb16 bus 67
> ahci1 at pci20 dev 0 function 0 "AMD FCH AHCI" rev 0x51: msi, AHCI 1.3.1
> scsibus2 at ahci1: 32 targets
> ppb17 at pci17 dev 8 function 3 "AMD 17h PCIE" rev 0x00
> pci21 at ppb17 bus 68
> ahci2 at pci21 dev 0 function 0 "AMD FCH AHCI" rev 0x51: msi, AHCI 1.3.1
> scsibus3 at ahci2: 32 targets
> vmm0 at mainbus0: SVM/RVI
> efifb0 at mainbus0: 1024x768, 32bpp
> wsdisplay0 at efifb0 mux 1
> wsdisplay0: screen 0-5 added (std, vt100 emulation)
> uhub4 at uhub2 port 2 configuration 1 interface 0 "ATEN International product 0x7000" rev 2.00/0.00 addr 2
> uhidev0 at uhub4 port 1 configuration 1 interface 0 "ATEN International product 0x2419" rev 1.10/1.00 addr 3
> uhidev0: iclass 3/1
> ukbd0 at uhidev0: 8 variable keys, 6 key codes
> wskbd0 at ukbd0 mux 1
> wskbd0: connecting to wsdisplay0
> uhidev1 at uhub4 port 1 configuration 1 interface 1 "ATEN International product 0x2419" rev 1.10/1.00 addr 3
> uhidev1: iclass 3/1
> ums0 at uhidev1: 3 buttons, Z dir
> wsmouse0 at ums0 mux 0
> vscsi0 at root
> scsibus4 at vscsi0: 256 targets
> softraid0 at root
> scsibus5 at softraid0: 256 targets
> root on sd0a (6dcfb315de8e2773.a) swap on sd0b dump on sd0b