From: Alexander Bluhm <bluhm@openbsd.org>
Subject: Re: spread 8 network interrupt over cpu and softnet
To: Hrvoje Popovski <hrvoje@srce.hr>
Cc: tech@openbsd.org
Date: Wed, 12 Nov 2025 01:13:23 +0100

On Tue, Nov 11, 2025 at 08:51:22PM +0100, Hrvoje Popovski wrote:
> On 11.11.2025. 12:20, Alexander Bluhm wrote:
> > I would like to commit the global IF_MAX_VECTORS limit.  Then it
> > is easier for me to find optimal values for number of softnet threads
> > and interface queues.
> > 
> > ok?
> > 
> > bluhm
> 
> Hi,
> 
> does that mean that if I want to test 16 queues I need to change
> NET_TASKQ to 16 in /sys/net/if.c and
> IF_MAX_VECTORS to 16 in /sys/net/if.h ?

After the driver part commited, I want to use the diff below to
spread packets over more than 8 softnet tasks.  With 16 CPUs each
network card generates 8 interrupts, but two cards should distribute
packets over 16 tasks.

You may also try to increase IF_MAX_VECTORS to generate more
interrupts per card.  But that only works if the hardware supports
it.  And I don't know if more queues make things faster.  Of course
it also depends of the hash distribution of you traffic.

I have not tested these variants yet.

bluhm

Index: net/if.c
===================================================================
RCS file: /data/mirror/openbsd/cvs/src/sys/net/if.c,v
diff -u -p -r1.745 if.c
--- net/if.c	11 Nov 2025 07:56:50 -0000	1.745
+++ net/if.c	11 Nov 2025 17:56:37 -0000
@@ -238,7 +238,7 @@ struct softnet {
 	struct netstack	 sn_netstack;
 } __aligned(64);
 #ifdef MULTIPROCESSOR
-#define NET_TASKQ	8
+#define NET_TASKQ	16
 #else
 #define NET_TASKQ	1
 #endif
Index: net/ifq.c
===================================================================
RCS file: /data/mirror/openbsd/cvs/src/sys/net/ifq.c,v
diff -u -p -r1.62 ifq.c
--- net/ifq.c	28 Jul 2025 05:25:44 -0000	1.62
+++ net/ifq.c	11 Nov 2025 17:55:33 -0000
@@ -255,7 +255,7 @@ void
 ifq_init(struct ifqueue *ifq, struct ifnet *ifp, unsigned int idx)
 {
 	ifq->ifq_if = ifp;
-	ifq->ifq_softnet = net_tq(idx);
+	ifq->ifq_softnet = net_tq(ifp->if_index * IF_MAX_VECTORS + idx);
 	ifq->ifq_softc = NULL;
 
 	mtx_init(&ifq->ifq_mtx, IPL_NET);