Download raw body.
SoftLRO diff for cnmac/octeon
On Tue, Mar 31, 2026 at 05:26:21PM +0200, Kirill A. Korinsky wrote:
> On Wed, 04 Feb 2026 20:30:13 +0100,
> Janne Johansson <icepic.dz@gmail.com> wrote:
> >
> > > > The SMALL_KERNEL diff is at
> > > >
> > > > https://s3.sto3.safedc.net/jj-public-files/small-kernel-cnmac-softlro.diff
> > >
> > > I can't boot kernel with this patch, it fails as:
> > >
> > > starting network
> > >
> > > Trap cause = 2 Frame 0x980000000fd97878
> > > Trap PC 0xffffffff8119f2a4 RA 0xffffffff8119f5ec fault 0x0
> > > cnmac_recv_mbuf+0x134 (77e0000d9f08a,1,980000000fd97a00,980000000fd979fc) ra 0xffffffff8119f5ec sp 0x980000000fd979d0, sz 32
> > >
> >
> > Thanks for testing, I'll rig something up with wg and see if I can
> > provoke the error with it.
> >
>
> Here a bit reworked version of your last diff.
>
> On my ER-4 with this diff I see:
>
> % iperf3-darwin -t 30 -c 172.31.3.1
> Connecting to host 172.31.3.1, port 5201
> [ 5] local 172.31.3.11 port 49306 connected to 172.31.3.1 port 5201
> [ ID] Interval Transfer Bitrate Retr Cwnd RTT
> [ 5] 0.00-1.00 sec 84.8 MBytes 710 Mbits/sec 5792 2.61 MBytes 7ms
> [ 5] 1.00-2.00 sec 102 MBytes 862 Mbits/sec 7240 1004 KBytes 9ms
> [ 5] 2.00-3.00 sec 106 MBytes 890 Mbits/sec 0 1.06 MBytes 9ms
> [ 5] 3.00-4.00 sec 101 MBytes 844 Mbits/sec 0 1.16 MBytes 12ms
> [ 5] 4.00-5.00 sec 105 MBytes 878 Mbits/sec 0 1.27 MBytes 9ms
> [ 5] 5.00-6.00 sec 99.8 MBytes 837 Mbits/sec 8688 1020 KBytes 8ms
> [ 5] 6.00-7.00 sec 105 MBytes 879 Mbits/sec 0 1.10 MBytes 10ms
> [ 5] 7.00-8.00 sec 104 MBytes 869 Mbits/sec 0 1.18 MBytes 13ms
> [ 5] 8.00-9.00 sec 106 MBytes 886 Mbits/sec 0 1.24 MBytes 5ms
> [ 5] 9.00-10.00 sec 105 MBytes 883 Mbits/sec 0 1.28 MBytes 9ms
> [ 5] 10.00-11.00 sec 108 MBytes 903 Mbits/sec 0 1.31 MBytes 13ms
> [ 5] 11.00-12.00 sec 105 MBytes 878 Mbits/sec 10136 1011 KBytes 8ms
> [ 5] 12.00-13.00 sec 103 MBytes 864 Mbits/sec 0 1.10 MBytes 10ms
> [ 5] 13.00-14.00 sec 103 MBytes 862 Mbits/sec 0 1.18 MBytes 8ms
> [ 5] 14.00-15.00 sec 102 MBytes 851 Mbits/sec 0 1.24 MBytes 11ms
> [ 5] 15.00-16.00 sec 99.3 MBytes 835 Mbits/sec 0 1.28 MBytes 7ms
> [ 5] 16.00-17.00 sec 104 MBytes 868 Mbits/sec 0 1.31 MBytes 5ms
> [ 5] 17.00-18.00 sec 95.1 MBytes 800 Mbits/sec 1448 549 KBytes 4ms
> [ 5] 18.00-19.00 sec 90.2 MBytes 756 Mbits/sec 0 694 KBytes 7ms
> [ 5] 19.00-20.00 sec 93.1 MBytes 781 Mbits/sec 0 867 KBytes 10ms
> [ 5] 20.00-21.00 sec 97.2 MBytes 815 Mbits/sec 0 1015 KBytes 10ms
> [ 5] 21.00-22.00 sec 107 MBytes 902 Mbits/sec 0 1.13 MBytes 11ms
> [ 5] 22.00-23.00 sec 101 MBytes 847 Mbits/sec 0 1.25 MBytes 10ms
> [ 5] 23.00-24.00 sec 91.3 MBytes 767 Mbits/sec 2896 557 KBytes 2ms
> [ 5] 24.00-25.00 sec 82.6 MBytes 691 Mbits/sec 0 660 KBytes 7ms
> [ 5] 25.00-26.00 sec 90.8 MBytes 763 Mbits/sec 0 783 KBytes 1ms
> [ 5] 26.00-27.00 sec 99.4 MBytes 832 Mbits/sec 0 949 KBytes 9ms
> [ 5] 27.00-28.00 sec 97.8 MBytes 822 Mbits/sec 0 1.06 MBytes 1ms
> [ 5] 28.00-29.01 sec 105 MBytes 878 Mbits/sec 0 1.19 MBytes 11ms
> [ 5] 29.01-30.00 sec 109 MBytes 916 Mbits/sec 0 1.31 MBytes 13ms
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval Transfer Bitrate Retr
> [ 5] 0.00-30.00 sec 2.93 GBytes 839 Mbits/sec 36200 sender
> [ 5] 0.00-30.16 sec 2.93 GBytes 834 Mbits/sec receiver
>
> iperf Done.
>
> So far it seems stable enough to actually use it.
>
> The diff:
>
> Index: sys/arch/octeon/dev/if_cnmac.c
> ===================================================================
> RCS file: /home/cvs/src/sys/arch/octeon/dev/if_cnmac.c,v
> diff -u -p -r1.86 if_cnmac.c
> --- sys/arch/octeon/dev/if_cnmac.c 20 May 2024 23:13:33 -0000 1.86
> +++ sys/arch/octeon/dev/if_cnmac.c 31 Mar 2026 14:32:09 -0000
> @@ -55,6 +55,11 @@
> #include <net/if_media.h>
> #include <netinet/in.h>
> #include <netinet/if_ether.h>
> +#ifndef SMALL_KERNEL
> +#include <netinet/tcp.h>
> +#include <netinet/tcp_timer.h>
> +#include <netinet/tcp_var.h>
> +#endif
>
> #if NBPFILTER > 0
> #include <net/bpf.h>
> @@ -307,6 +312,9 @@ cnmac_attach(struct device *parent, stru
> ifp->if_softc = sc;
> ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
> ifp->if_xflags = IFXF_MPSAFE;
> +#ifndef SMALL_KERNEL
> + ifp->if_xflags |= IFXF_LRO;
> +#endif
> ifp->if_ioctl = cnmac_ioctl;
> ifp->if_qstart = cnmac_start;
> ifp->if_watchdog = cnmac_watchdog;
> @@ -315,6 +323,9 @@ cnmac_attach(struct device *parent, stru
>
> ifp->if_capabilities = IFCAP_VLAN_MTU | IFCAP_CSUM_TCPv4 |
> IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6;
> +#ifndef SMALL_KERNEL
> + ifp->if_capabilities |= IFCAP_LRO;
> +#endif
>
> cn30xxgmx_set_filter(sc->sc_gmx_port);
>
> @@ -1214,6 +1225,9 @@ cnmac_recv(struct cnmac_softc *sc, uint6
> struct mbuf *m;
> uint64_t word2;
> int nmbuf = 0;
> +#ifndef SMALL_KERNEL
> + struct ether_extracted ext;
> +#endif
Please move this definition next to ether_extract_headers() inside
the if block, to reduce #ifndef'ing.
>
> word2 = work[2];
>
> @@ -1246,7 +1260,19 @@ cnmac_recv(struct cnmac_softc *sc, uint6
> M_TCP_CSUM_IN_OK | M_UDP_CSUM_IN_OK;
> }
>
> - ml_enqueue(ml, m);
> +#ifndef SMALL_KERNEL
> + if (__predict_true(ISSET(ifp->if_xflags, IFXF_LRO)) &&
> + __predict_true(!ISSET(word2, PIP_WQE_WORD2_IP_NI)) &&
I would leave these branch hints out as they mainly add visual noise
here. Also, the predict true hint does not seem correct with IFXF_LRO
because it is a valid admin choice to not use LRO.
With the above tweaks, OK visa@ after release.
Tested with EdgeRouter Lite (CN5020) and EdgeRouter Pro (CN6120).
> + ISSET(word2, PIP_WQE_WORD2_IP_TU) &&
> + !ISSET(word2, PIP_WQE_WORD2_IP_FR | PIP_WQE_WORD2_IP_LE)) {
> + ether_extract_headers(m, &ext);
> + if (ext.tcp != NULL)
> + tcp_softlro_glue(ml, m, ifp);
> + else
> + ml_enqueue(ml, m);
> + } else
> +#endif
> + ml_enqueue(ml, m);
>
> return nmbuf;
>
>
>
> --
> wbr, Kirill
>
SoftLRO diff for cnmac/octeon