From: YASUOKA Masahiko <yasuoka@openbsd.org>
Subject: Re: iavf: panic on QEMU with i440fx chipset emulation
To: naito.yuichiro@gmail.com
Cc: kettenis@openbsd.org, tech@openbsd.org
Date: Mon, 07 Oct 2024 10:53:56 +0900

On Mon, 07 Oct 2024 10:22:00 +0900 (JST)
Yuichiro NAITO <naito.yuichiro@gmail.com> wrote:
> From: YASUOKA Masahiko <yasuoka@openbsd.org>
> Subject: Re: iavf: panic on QEMU with i440fx chipset emulation
> Date: Fri, 04 Oct 2024 17:19:28 +0900 (JST)
> 
>> On Wed, 25 Sep 2024 11:42:02 +0900 (JST)
>> Yuichiro NAITO <naito.yuichiro@gmail.com> wrote:
>>> Hi, While I'm testing iavf(4) on Linux QEMU with i440fx chipset emulation,
>>> I had the following panic message on my OpenBSD current build.
>>> I440fx is the default chipset emulation of QEMU. I know q35 chipset
>>> emulation is required for msix interrupts. But if users forget to select it,
>>> I tested what happens on the default chipset emulation. 
>> 
>> I think the root problem is this i440fx situation.
> 
> No, it's a separate issue.
> 
> I mean the iavf(4) panic occurs if an error happens between
> 'iavf_get_vf_resources' and 'if_atatch' in the 'iavf_attach' function.
> Looking at the 'iavf_attach' function, there are at least 3 functions
> that may cause an error, 'iavf_config_irq_map', 'pci_intr_map_msix',
> and 'pci_intr_establish' functions.
> The 'pci_intr_map_msix' case is 100% reproducible on the QEMU i440fx
> chipset emulation. The other cases are very rare but should be fixed for
> stability.

Yes, it might be true if there is a broken hardware which has the same
problem of qemu.  We are discussing to have a quirk for qemu.  For the
other hardwares than qemu, I think

diff --git a/sys/dev/pci/if_iavf.c b/sys/dev/pci/if_iavf.c
index d573d6725f4..e3eadf23904 100644
--- a/sys/dev/pci/if_iavf.c
+++ b/sys/dev/pci/if_iavf.c
@@ -766,6 +766,11 @@ iavf_attach(struct device *parent, struct device *self, void *aux)
 	pcireg_t memtype;
 	int tries;
 
+	if (pci_intr_msix_count(pa) <= 0) {
+		printf(": no msix interrupt\n");
+		return;
+	}
+
 	rw_init(&sc->sc_cfg_lock, "iavfcfg");
 
 	sc->sc_pc = pa->pa_pc;

this kind of workaround is enough.


>> The driver assumes the device works with msix.  The physical device is
>> connected on PCI-E bus and msix is usable of cource.  Since QEMU is
>> doing PCI-passthrough the device to the guest, I think QEMU should
>> configure a PCI bus which support msxi for the guest.  But actually
>> msix is not usable on OpenBSD guest.
>> 
>> On FreeBSD, it seems to work around this problem by PCI_QUIRK_ENABLE_MSI_VM
>> 
>>   https://github.com/freebsd/freebsd-src/blob/release/14.1.0/sys/dev/pci/pci.c#L279-L283
>> 
>> I think this kind of quirk is needed for us.
>> What do you think?
> 
> NetBSD also has the same quirk.
> 
> https://github.com/NetBSD/src/blob/affb7619d18b17a184eede63a4823f629a2893fc/sys/arch/x86/pci/pci_machdep.c#L273
> 
> I agree with you we add the same quirk for OpenBSD.
> I will send a patch for the PCI bus driver soon.

Thanks,