On 21.03.2025 14:24, Stefan Fritsch wrote:
>Hi,
>
>On Sun, 16 Mar 2025, Stefan Fritsch wrote:
>> there were some reports that vio on KVM/qemu sometimes panics with
>>
>>  vq_size not power of two: 65535
>>
>> but I could never reproduce it. bluhm@ now got me a test setup where the
>> bsd kernel is PXE booted on qemu in 440fx mode, and there it is
>> reproducible.
>>
>> After some debugging it seems that seabios or ipxe maps the PCI BARs at
>> 0x380000000000-0x380080000000 which is outside the allowed range in
>> pci_init_extents(). On the other hand, in 440fx mode, qemu seems to
>> produce ACPI 1.x tables and there is a check in acpipci_attach() that for
>> ACPI < 5.x, the PCI infos from _CRS are not used. OpenBSD will then
>> disable the BARs and when mapping them again in vio_attach(), it will
>> sometimes choose adresses that do not work, reads return 0xff and writes
>> are ignored. I guess this is becuase the address (in my case 0xbff14000)
>> lies outside the PCI window of the emulated chipset.
>>
>> I have put dmesg, acpi tables and other info at
>> https://www.sfritsch.de/~stf/vq-panic/
>>
>> Qemu in q35 mode produces ACPI 3.x tables, so it may also be affected.
>>
>> There may be three ways to fix this:
>>
>> 1) increase the allowed range for pcimem in pci_init_extents(). This is
>> what the diff below does.
>>
>> 2) somehow make acpipci_attach() use the ACPI infos on qemu. I have
>> verified that removing the version check fixes the issue. Since removing
>> the version check seems to break many other systems, this would have to be
>> a qemu specific quirk.
>>
>> 3) try to make OpenBSD reliably map the BARs somewhere where it works. Is
>> there a way for OpenBSD to get the info where the PCI window is without
>> trusting ACPI?
>>
>> I remember at least one report of this issue on i386. Any idea how to fix
>> it there?
>
>Mark Patruck noticed that these issues seem to be caused by some
>relatively recent changes in seabios.
>
>https://mail.coreboot.org/hyperkitty/list/seabios@seabios.org/message/R7FOQMMYWVX577QNIA2AKUAGOZKNJIAP/
>https://gitlab.com/qemu-project/seabios/-/commit/df9dd418b3b0e586cb208125094620fc7f90f23d
>
>A workaround seems to be to configure the VM with <= 3GB memory.
>
>The problem may become more wide-spread with 7.7, since we now default to
>virtio 1.x, which uses MMIO on qemu, compared to virtio 0.9 which uses PIO
>BARs. Therefore it would be nice to get a fix in before the release, if it
>is not too late already.
>
>The diff below uses acpipci / _CRS also with old ACPI versions if running
>on a hypervisor. I think the chance that it will break unrelated systems
>is low. It does not change behavior on vmd, where no acpi attaches at all.

With the first solution (1), as well as the diff below (2) i don't get
any conflicts like

0:28:0: bridge mem address conflict 0x383800000000/0x800000000
0:28:1: bridge mem address conflict 0x383000000000/0x800000000
0:28:2: bridge mem address conflict 0x382800000000/0x800000000
0:28:3: bridge mem address conflict 0x382000000000/0x800000000
0:30:0: bridge mem address conflict 0x380000000000/0x2000000000
5:1:0: bridge mem address conflict 0x381800000000/0x800000000
5:4:0: bridge mem address conflict 0x380000000000/0x800000000
6:3:0: mem address conflict 0x381800000000/0x4000
6:18:0: mem address conflict 0x381800004000/0x4000
9:1:0: mem address conflict 0x380000000000/0x4000

anymore and also passed through devices work flawlessly

ixl0 at pci1 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 0, FW 9.152.77998 API 1.15, msix, 4 queues, address 40:a6:b7:c0:b5:b2
ixl1 at pci1 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 1, FW 9.152.77998 API 1.15, msix, 4 queues, address 40:a6:b7:c0:b5:b3

when running QEMU >=9.0. Both solutions work.

>ok?
>
>
>diff --git a/sys/arch/amd64/pci/acpipci.c b/sys/arch/amd64/pci/acpipci.c
>index 51cd1360383..2e3236772bb 100644
>--- a/sys/arch/amd64/pci/acpipci.c
>+++ b/sys/arch/amd64/pci/acpipci.c
>@@ -152,7 +152,7 @@ acpipci_attach(struct device *parent, struct device *self, void *aux)
>
> 	aml_parse_resource(&res, acpipci_parse_resources, sc);
>
>-	if (sc->sc_acpi->sc_major < 5) {
>+	if (sc->sc_acpi->sc_major < 5 && (cpu_ecxfeature & CPUIDECX_HV) == 0) {
> 		extent_destroy(sc->sc_ioex);
> 		extent_destroy(sc->sc_memex);
>
>

--
Mark Patruck ( mark at wrapped.cx )
GPG key 0xF2865E51 / 187F F6D3 EE04 1DCE 1C74  F644 0D3C F66F F286 5E51
  
https://www.wrapped.cx