From: Janne Johansson Subject: SoftLRO diff for cnmac/octeon To: "tech@openbsd.org" , Jan Klemkow , Visa Hankala Date: Thu, 29 Jan 2026 16:00:48 +0100 This is an update for the diff I sent when Jan Klemkow first sent out the proposal for SoftLRO last year for certain interfaces. It still gives me large speedups (*) for incoming TCP and I have been running it for a long while. In case gmail as usual mangles the whitespaces, the diff is also available here: https://s3.sto3.safedc.net/jj-public-files/cnmac.softlro.diff Index: dev/if_cnmac.c =================================================================== RCS file: /cvs/src/sys/arch/octeon/dev/if_cnmac.c,v diff -u -p -u -r1.86 if_cnmac.c --- dev/if_cnmac.c 20 May 2024 23:13:33 -0000 1.86 +++ dev/if_cnmac.c 29 Jan 2026 14:35:19 -0000 @@ -55,6 +55,9 @@ #include #include #include +#include +#include +#include #if NBPFILTER > 0 #include @@ -106,6 +109,7 @@ CTASSERT(MCLBYTES >= OCTEON_POOL_SIZE_PKT + CACHELINESIZE); + void cnmac_buf_init(struct cnmac_softc *); int cnmac_match(struct device *, void *, void *); @@ -306,7 +310,7 @@ cnmac_attach(struct device *parent, stru strncpy(ifp->if_xname, sc->sc_dev.dv_xname, sizeof(ifp->if_xname)); ifp->if_softc = sc; ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; - ifp->if_xflags = IFXF_MPSAFE; + ifp->if_xflags = IFXF_MPSAFE|IFXF_LRO; ifp->if_ioctl = cnmac_ioctl; ifp->if_qstart = cnmac_start; ifp->if_watchdog = cnmac_watchdog; @@ -314,7 +318,8 @@ cnmac_attach(struct device *parent, stru ifq_init_maxlen(&ifp->if_snd, max(GATHER_QUEUE_SIZE, IFQ_MAXLEN)); ifp->if_capabilities = IFCAP_VLAN_MTU | IFCAP_CSUM_TCPv4 | - IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6; + IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6 | + IFCAP_LRO; cn30xxgmx_set_filter(sc->sc_gmx_port); @@ -1246,7 +1251,10 @@ cnmac_recv(struct cnmac_softc *sc, uint6 M_TCP_CSUM_IN_OK | M_UDP_CSUM_IN_OK; } - ml_enqueue(ml, m); + if (ISSET(ifp->if_xflags, IFXF_LRO)) + tcp_softlro_glue(ml, m, ifp); + else + ml_enqueue(ml, m); return nmbuf; Benching with iperf3 -s running on an Edgerouter 8-Pro with "ifconfig cnmac0 -tcplro" jj@jjs-MacBook-Air:~$ iperf3 -c 172.31.31.102 Connecting to host 172.31.31.102, port 5201 [ 5] local 172.31.31.1 port 53101 connected to 172.31.31.102 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 50.9 MBytes 426 Mbits/sec [ 5] 1.00-2.01 sec 49.0 MBytes 410 Mbits/sec [ 5] 2.01-3.00 sec 50.5 MBytes 424 Mbits/sec [ 5] 3.00-4.00 sec 51.1 MBytes 429 Mbits/sec [ 5] 4.00-5.00 sec 48.4 MBytes 405 Mbits/sec [ 5] 5.00-6.00 sec 49.5 MBytes 415 Mbits/sec [ 5] 6.00-7.00 sec 49.5 MBytes 416 Mbits/sec [ 5] 7.00-8.01 sec 51.1 MBytes 428 Mbits/sec [ 5] 8.01-9.00 sec 49.2 MBytes 414 Mbits/sec [ 5] 9.00-10.00 sec 48.8 MBytes 409 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 498 MBytes 418 Mbits/sec sender [ 5] 0.00-10.02 sec 498 MBytes 417 Mbits/sec receiver iperf Done. After enabling LRO again: jj@jjs-MacBook-Air:~$ iperf3 -c 172.31.31.102 Connecting to host 172.31.31.102, port 5201 [ 5] local 172.31.31.1 port 53103 connected to 172.31.31.102 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 109 MBytes 910 Mbits/sec [ 5] 1.00-2.01 sec 113 MBytes 942 Mbits/sec [ 5] 2.01-3.00 sec 112 MBytes 943 Mbits/sec [ 5] 3.00-4.00 sec 111 MBytes 928 Mbits/sec [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec [ 5] 5.00-6.00 sec 112 MBytes 939 Mbits/sec [ 5] 6.00-7.00 sec 112 MBytes 937 Mbits/sec [ 5] 7.00-8.00 sec 112 MBytes 938 Mbits/sec [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec [ 5] 9.00-10.00 sec 112 MBytes 935 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec sender [ 5] 0.00-10.01 sec 1.09 GBytes 932 Mbits/sec receiver iperf Done. Almost same diff tested on a friends ER-4 with 7.8 gives 200 MBit/s extra on those tests. -- May the most significant bit of your life be positive.