From: Andrew Lemin <andrew.lemin@gmail.com>
Subject: Re: UDP parallel input
To: Alexander Bluhm <bluhm@openbsd.org>
Cc: tech@openbsd.org
Date: Fri, 26 Jul 2024 17:25:13 +1000

Hi,
This is amazing news! Well done :)

Just to clarify, you mention "you need multiple CPUs and network interfaces
that support multiqueue".
Mellanox cards have multiple queues;
https://github.com/openbsd/src/blob/master/sys/dev/pci/if_mcx.c

However 'mcx' is missing from your list "These are bnxt, igc, ix, ixl,
ngbe, vmx".

Am I correct in assuming that Mellanox cards will be able to realise these
changes, and your list of example drivers here was just a example and not
an exclusive list?

Thanks again for the hard work!
Andy Lemin


On Tue, 23 Jul 2024 at 23:43, Alexander Bluhm <bluhm@openbsd.org> wrote:

> Hi,
>
> mvs@ has completed the final bits to make socket buffer MP safe for
> UDP packets.  This means that we can run UDP input on multiple
> threads.  Diff below activates this.
>
> Usually IP processing runs on multiple softnet threads with shared
> net lock.  From there local packets are queued and processed by one
> thread with exclusive net lock.  If the PR_MPINPUT flag is set,
> protocol input is called directly from IP input on multiple threads,
> shared net lock and no additional queueing.
>
> Note that you need kernel sources from at least Sun Jul 21.  Then
> apply this diff and test UDP local delivery to sockets.  To see an
> performance effect, you need multiple CPUs and network interfaces
> that support multiqueue.  These are bnxt, igc, ix, ixl, ngbe, vmx.
>
> Please test!
>
> bluhm
>
> Index: netinet/in_proto.c
> ===================================================================
> RCS file: /data/mirror/openbsd/cvs/src/sys/netinet/in_proto.c,v
> diff -u -p -r1.106 in_proto.c
> --- netinet/in_proto.c  13 Jul 2024 12:00:11 -0000      1.106
> +++ netinet/in_proto.c  22 Jul 2024 11:40:55 -0000
> @@ -185,7 +185,7 @@ const struct protosw inetsw[] = {
>    .pr_type     = SOCK_DGRAM,
>    .pr_domain   = &inetdomain,
>    .pr_protocol = IPPROTO_UDP,
> -  .pr_flags    = PR_ATOMIC|PR_ADDR|PR_SPLICE|PR_MPSOCKET,
> +  .pr_flags    = PR_ATOMIC|PR_ADDR|PR_SPLICE|PR_MPINPUT|PR_MPSOCKET,
>    .pr_input    = udp_input,
>    .pr_ctlinput = udp_ctlinput,
>    .pr_ctloutput        = ip_ctloutput,
> Index: netinet6/in6_proto.c
> ===================================================================
> RCS file: /data/mirror/openbsd/cvs/src/sys/netinet6/in6_proto.c,v
> diff -u -p -r1.116 in6_proto.c
> --- netinet6/in6_proto.c        19 Jul 2024 16:58:32 -0000      1.116
> +++ netinet6/in6_proto.c        22 Jul 2024 11:40:55 -0000
> @@ -136,7 +136,7 @@ const struct protosw inet6sw[] = {
>    .pr_type     = SOCK_DGRAM,
>    .pr_domain   = &inet6domain,
>    .pr_protocol = IPPROTO_UDP,
> -  .pr_flags    = PR_ATOMIC|PR_ADDR|PR_SPLICE|PR_MPSOCKET,
> +  .pr_flags    = PR_ATOMIC|PR_ADDR|PR_SPLICE|PR_MPINPUT|PR_MPSOCKET,
>    .pr_input    = udp_input,
>    .pr_ctlinput = udp6_ctlinput,
>    .pr_ctloutput        = ip6_ctloutput,
>
>