From: Kirill A. Korinsky Subject: Re: SoftLRO diff for cnmac/octeon To: Janne Johansson Cc: Jan Klemkow , tech@openbsd.org, visa@openbsd.org Date: Wed, 04 Feb 2026 20:17:00 +0100 On Wed, 04 Feb 2026 11:04:22 +0100, Janne Johansson wrote: > > > The following macro is needed, or your diff breaks SMALL_KERNELs like > > the bsd.rd. > > > > #ifndef SMALL_KERNEL > > > > > - ml_enqueue(ml, m); > > > + if (ISSET(ifp->if_xflags, IFXF_LRO)) > > > > It there a way do differentiate between TCP and over packets? > > So, you could use the same pattern, as in the other drivers? > > Is there a programmers guide for this chip, which might help? > > Tried looking at the NDK fbsd 11 has in their sys/contrib and some > linux cavium sources, but since its all named differently, it is very > had to discern if there is a simple way to notice those. > Also tested on ERL3 now, bumps tcp receive from 100-ish Mbit to 400Mbit/s. > > The SMALL_KERNEL diff is at > > https://s3.sto3.safedc.net/jj-public-files/small-kernel-cnmac-softlro.diff > > Since it was softlro that pulls in the netinet/tcp* headers, I put > that behind ifndef SMALL_KERNEL too, no need to pull them in if not > gluing packets. > I can't boot kernel with this patch, it fails as: starting network Trap cause = 2 Frame 0x980000000fd97878 Trap PC 0xffffffff8119f2a4 RA 0xffffffff8119f5ec fault 0x0 cnmac_recv_mbuf+0x134 (77e0000d9f08a,1,980000000fd97a00,980000000fd979fc) ra 0xffffffff8119f5ec sp 0x980000000fd979d0, sz 32 cnmac_recv+0x74 (77e0000d9f08a,1,980000000fd97a00,f7dfddd3709a3dd6) ra 0xffffffff8119d624 sp 0x980000000fd979f0, sz 80 cnmac_intr+0xfc (77e0000d9f08a,308f6a10c7e82567,980000000fd97a00,f7dfddd3709a3dd6) ra 0xffffffff8134e680 sp 0x980000000fd97a40, sz 96 octciu_intr_bank+0x270 (77e0000d9f08a,308f6a10c7e82567,980000000fd97a00,7bece6ec01216f6e) ra 0xffffffff8134dd84 sp 0x980000000fd97aa0, sz 160 octciu_intr0+0x94 (77e0000d9f08a,308f6a10c7e82567,c7c93030b1561ef2,7bece6ec01216f6e) ra 0xffffffff814e8028 sp 0x980000000fd97b40, sz 64 interrupt+0x170 (77e0000d9f08a,790736f741a134ac,c7c93030b1561ef2,7bece6ec01216f6e) ra 0xffffffff8133d214 sp 0x980000000fd97b80, sz 64 User-level: pid 74299 stopped on non ddb fault Stopped at cnmac_recv_mbuf+0x134: ld v1,32(t8) ddb{0}> also, probably previous one crashed inside wg on reboot, unfortently I had lost stacktrace :( > Index: if_cnmac.c > =================================================================== > RCS file: /cvs/src/sys/arch/octeon/dev/if_cnmac.c,v > diff -u -p -u -r1.86 if_cnmac.c > --- if_cnmac.c 20 May 2024 23:13:33 -0000 1.86 > +++ if_cnmac.c 4 Feb 2026 08:55:25 -0000 > @@ -55,6 +55,11 @@ > #include > #include > #include > +#ifndef SMALL_KERNEL > +#include > +#include > +#include > +#endif /* SMALL_KERNEL */ > > #if NBPFILTER > 0 > #include > @@ -306,7 +311,7 @@ cnmac_attach(struct device *parent, stru > strncpy(ifp->if_xname, sc->sc_dev.dv_xname, sizeof(ifp->if_xname)); > ifp->if_softc = sc; > ifp->if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST; > - ifp->if_xflags = IFXF_MPSAFE; > + ifp->if_xflags = IFXF_MPSAFE|IFXF_LRO; > ifp->if_ioctl = cnmac_ioctl; > ifp->if_qstart = cnmac_start; > ifp->if_watchdog = cnmac_watchdog; > @@ -314,7 +319,8 @@ cnmac_attach(struct device *parent, stru > ifq_init_maxlen(&ifp->if_snd, max(GATHER_QUEUE_SIZE, IFQ_MAXLEN)); > > ifp->if_capabilities = IFCAP_VLAN_MTU | IFCAP_CSUM_TCPv4 | > - IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6; > + IFCAP_CSUM_UDPv4 | IFCAP_CSUM_TCPv6 | IFCAP_CSUM_UDPv6 | > + IFCAP_LRO; > > cn30xxgmx_set_filter(sc->sc_gmx_port); > > @@ -1246,7 +1252,13 @@ cnmac_recv(struct cnmac_softc *sc, uint6 > M_TCP_CSUM_IN_OK | M_UDP_CSUM_IN_OK; > } > > - ml_enqueue(ml, m); > +#ifndef SMALL_KERNEL > + if (__predict_true(ISSET(ifp->if_xflags, IFXF_LRO))) > + tcp_softlro_glue(ml, m, ifp); > + else > +#else > + ml_enqueue(ml, m); > +#endif > > return nmbuf; > > > -- > May the most significant bit of your life be positive. > -- wbr, Kirill