From: Mark Kettenis <mark.kettenis@xs4all.nl>
Subject: Re: ice max vectors
To: Alexander Bluhm <alexander.bluhm@gmx.net>
Cc: tech@openbsd.org
Date: Tue, 24 Feb 2026 23:18:07 +0100

> Date: Tue, 24 Feb 2026 22:35:31 +0100
> From: Alexander Bluhm <alexander.bluhm@gmx.net>
> 
> On Tue, Feb 24, 2026 at 09:45:36PM +0100, Mark Kettenis wrote:
> > > #define ICE_MAX_VECTORS		8 /* XXX this is pretty arbitrary */
> > > Does anyone know a more realistic number for ice than 8 ?
> > 
> > You could look at pcidump -v output on a few machines.  MSI should
> > report the number of vectors; for MSI-X the table size should be
> > equivalent.
> 
>  138:0:1: Intel E810-XXV SFP
>         0x0070: Capability 0x11: Extended Message Signalled Interrupts (MSI-X)
>                 Enabled: yes; table size 1024 (BAR 3:0)
> 
>  225:0:0: Intel E810-C QSFP
>         0x0070: Capability 0x11: Extended Message Signalled Interrupts (MSI-X)
>                 Enabled: yes; table size 1024 (BAR 3:0)
> 
> Should I change it to 1024?  Only sc_vectors could become larger,
> but it is limited by ncpus.
> 
>         sc->sc_nvectors = MIN(sc->sc_nmsix_max, ncpus);
>         sc->sc_nvectors = MIN(sc->sc_nvectors, ICE_MAX_VECTORS);
>         sc->sc_vectors = mallocarray(sizeof(*sc->sc_vectors), sc->sc_nvectors,
>             M_DEVBUF, M_WAITOK|M_CANFAIL|M_ZERO);

Hmm, this driver is confused about the number of interrupt vectors and
the number of queues.  Looks like it wants 1 + sc_nqueues interrupt
vectors, using the first one for management.  The management interrupt
is established on the primary CPU (possibly overflowing to another
CPU).  And the interrupts for the queues are spread around over the
CPUs.  Looks like the intent here is to limit the number of queues to
IF_MAX_VECTORS.

The driver already looks at the number of of MSI-X that the hardware
provides.  So I think I would simple remove ICE_MAX_VECTORS altogether
and just replace it with IF_MAX_VECTORS or IF_MAX_VECTORS + 1 where
appropriate.  I don't think it makes sense for sc_nvectors to be
larger than sc_nqueues.


> Index: dev/pci/if_ice.c
> ===================================================================
> RCS file: /data/mirror/openbsd/cvs/src/sys/dev/pci/if_ice.c,v
> diff -u -p -r1.68 if_ice.c
> --- dev/pci/if_ice.c	24 Feb 2026 20:14:29 -0000	1.68
> +++ dev/pci/if_ice.c	24 Feb 2026 21:29:55 -0000
> @@ -263,7 +263,7 @@ struct ice_intr_vector {
>  	pci_intr_handle_t	 ih;
>  };
>  
> -#define ICE_MAX_VECTORS			8 /* XXX this is pretty arbitrary */
> +#define ICE_MAX_VECTORS			1024
>  
>  static struct rwlock ice_sff_lock = RWLOCK_INITIALIZER("icesff");
>  
> @@ -308,8 +308,8 @@ struct ice_softc {
>  
>  	/* isc_* fields inherited from FreeBSD iflib struct if_softc_ctx */
>  	int isc_tx_nsegments;
> -	int isc_ntxd[8];
> -	int isc_nrxd[8];
> +	int isc_ntxd[MIN(ICE_MAX_VECTORS, IF_MAX_VECTORS)];
> +	int isc_nrxd[MIN(ICE_MAX_VECTORS, IF_MAX_VECTORS)];
>  	int isc_tx_tso_segments_max;
>  	int isc_tx_tso_size_max;
>  	int isc_tx_tso_segsize_max;
> @@ -339,7 +339,7 @@ struct ice_softc {
>  
>  	int rebuild_ticks;
>  
> -	int sw_intr[ICE_MAX_VECTORS];
> +	int sw_intr[MIN(ICE_MAX_VECTORS, IF_MAX_VECTORS) + 1];
>  };
>  
>  static int ice_rxrinfo(struct ice_softc *, struct if_rxrinfo *);
>