Index | Thread | Search

From:
Jan Klemkow <jan@openbsd.org>
Subject:
Re: SoftLRO diff for cnmac/octeon
To:
Janne Johansson <icepic.dz@gmail.com>
Cc:
"tech@openbsd.org" <tech@openbsd.org>, Visa Hankala <visa@openbsd.org>
Date:
Thu, 29 Jan 2026 21:45:33 +0100

Download raw body.

Thread
On Thu, Jan 29, 2026 at 04:00:48PM +0100, Janne Johansson wrote:
> This is an update for the diff I sent when Jan Klemkow first sent out
> the proposal for SoftLRO last year for certain interfaces.
> It still gives me large speedups (*)  for incoming TCP and I have been
> running it for a long while.
> 
> In case gmail as usual mangles the whitespaces, the diff is also available here:
> 
> https://s3.sto3.safedc.net/jj-public-files/cnmac.softlro.diff

Thanks for the diff, but...

Patching file dev/if_cnmac.c using Plan A...
Hunk #1 succeeded at 55.
Hunk #2 failed at 109.
Hunk #3 failed at 310.
Hunk #4 failed at 318.
Hunk #5 failed at 1251.
4 out of 5 hunks failed--saving rejects to dev/if_cnmac.c.rej
Hmm...  Ignoring the trailing garbage.
done

Even the downloaded version does not apply to cvs HEAD because of
missing tabs.

> Index: dev/if_cnmac.c
> ===================================================================
> RCS file: /cvs/src/sys/arch/octeon/dev/if_cnmac.c,v
> diff -u -p -u -r1.86 if_cnmac.c
> --- dev/if_cnmac.c      20 May 2024 23:13:33 -0000      1.86
> +++ dev/if_cnmac.c      29 Jan 2026 14:35:19 -0000
> @@ -55,6 +55,9 @@
>  #include <net/if_media.h>
>  #include <netinet/in.h>
>  #include <netinet/if_ether.h>
> +#include <netinet/tcp.h>
> +#include <netinet/tcp_timer.h>
> +#include <netinet/tcp_var.h>
> 
>  #if NBPFILTER > 0
>  #include <net/bpf.h>
> @@ -106,6 +109,7 @@
> 
>  CTASSERT(MCLBYTES >= OCTEON_POOL_SIZE_PKT + CACHELINESIZE);
> 
> +
>  void   cnmac_buf_init(struct cnmac_softc *);
> 
>  int    cnmac_match(struct device *, void *, void *);
> @@ -306,7 +310,7 @@ cnmac_attach(struct device *parent, stru
>         strncpy(ifp->if_xname, sc->sc_dev.dv_xname, sizeof(ifp->if_xname));
>         ifp->if_softc = sc;
>         ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
> -       ifp->if_xflags = IFXF_MPSAFE;
> +       ifp->if_xflags = IFXF_MPSAFE|IFXF_LRO;
>         ifp->if_ioctl = cnmac_ioctl;
>         ifp->if_qstart = cnmac_start;
>         ifp->if_watchdog = cnmac_watchdog;
> @@ -314,7 +318,8 @@ cnmac_attach(struct device *parent, stru
>         ifq_init_maxlen(&ifp->if_snd, max(GATHER_QUEUE_SIZE, IFQ_MAXLEN));
> 
>         ifp->if_capabilities = IFCAP_VLAN_MTU | IFCAP_CSUM_TCPv4 |
> -           IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6;
> +           IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6 |
> +           IFCAP_LRO;
> 
>         cn30xxgmx_set_filter(sc->sc_gmx_port);
> 
> @@ -1246,7 +1251,10 @@ cnmac_recv(struct cnmac_softc *sc, uint6
>                             M_TCP_CSUM_IN_OK | M_UDP_CSUM_IN_OK;
>         }
> 
> -       ml_enqueue(ml, m);
> +       if (ISSET(ifp->if_xflags, IFXF_LRO))
> +         tcp_softlro_glue(ml, m, ifp);
> +       else
> +         ml_enqueue(ml, m);
> 
>         return nmbuf;
> 
> 
> 
> 
> Benching with iperf3 -s running on an Edgerouter 8-Pro with "ifconfig
> cnmac0 -tcplro"
> 
> jj@jjs-MacBook-Air:~$ iperf3 -c 172.31.31.102
> Connecting to host 172.31.31.102, port 5201
> [  5] local 172.31.31.1 port 53101 connected to 172.31.31.102 port 5201
> [ ID] Interval           Transfer     Bitrate
> [  5]   0.00-1.00   sec  50.9 MBytes   426 Mbits/sec
> [  5]   1.00-2.01   sec  49.0 MBytes   410 Mbits/sec
> [  5]   2.01-3.00   sec  50.5 MBytes   424 Mbits/sec
> [  5]   3.00-4.00   sec  51.1 MBytes   429 Mbits/sec
> [  5]   4.00-5.00   sec  48.4 MBytes   405 Mbits/sec
> [  5]   5.00-6.00   sec  49.5 MBytes   415 Mbits/sec
> [  5]   6.00-7.00   sec  49.5 MBytes   416 Mbits/sec
> [  5]   7.00-8.01   sec  51.1 MBytes   428 Mbits/sec
> [  5]   8.01-9.00   sec  49.2 MBytes   414 Mbits/sec
> [  5]   9.00-10.00  sec  48.8 MBytes   409 Mbits/sec
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate
> [  5]   0.00-10.00  sec   498 MBytes   418 Mbits/sec                  sender
> [  5]   0.00-10.02  sec   498 MBytes   417 Mbits/sec                  receiver
> 
> iperf Done.
> 
> After enabling LRO again:
> 
> jj@jjs-MacBook-Air:~$ iperf3 -c 172.31.31.102
> Connecting to host 172.31.31.102, port 5201
> [  5] local 172.31.31.1 port 53103 connected to 172.31.31.102 port 5201
> [ ID] Interval           Transfer     Bitrate
> [  5]   0.00-1.00   sec   109 MBytes   910 Mbits/sec
> [  5]   1.00-2.01   sec   113 MBytes   942 Mbits/sec
> [  5]   2.01-3.00   sec   112 MBytes   943 Mbits/sec
> [  5]   3.00-4.00   sec   111 MBytes   928 Mbits/sec
> [  5]   4.00-5.00   sec   111 MBytes   933 Mbits/sec
> [  5]   5.00-6.00   sec   112 MBytes   939 Mbits/sec
> [  5]   6.00-7.00   sec   112 MBytes   937 Mbits/sec
> [  5]   7.00-8.00   sec   112 MBytes   938 Mbits/sec
> [  5]   8.00-9.00   sec   111 MBytes   933 Mbits/sec
> [  5]   9.00-10.00  sec   112 MBytes   935 Mbits/sec
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval           Transfer     Bitrate
> [  5]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  sender
> [  5]   0.00-10.01  sec  1.09 GBytes   932 Mbits/sec                  receiver
> 
> iperf Done.
> 
> Almost same diff tested on a friends ER-4 with 7.8 gives 200 MBit/s
> extra on those tests.
> 
> -- 
> May the most significant bit of your life be positive.
>