From: "Theo de Raadt" <deraadt@openbsd.org>
Subject: Re: igc(4): fix recv. jumbo frames
To: Alexander Bluhm <bluhm@openbsd.org>
Cc: Jan Klemkow <jan@openbsd.org>, tech@openbsd.org
Date: Wed, 31 Jul 2024 10:39:25 -0600

Alexander Bluhm <bluhm@openbsd.org> wrote:

> On Tue, Jul 30, 2024 at 08:12:08AM +0200, Jan Klemkow wrote:
> > The DMA mapping and allocation of mbufs for jumbo frames uses different
> > sizes.  Thus, we are ending up with corrupt mbufs, which leads to panics
> > in later part of the TCP/IP stack:
> > 
> > panic: kernel diagnostic assertion "M_DATABUF(m) + M_SIZE(m) >= (m->m_data + m->m_len)"
> > failed: file "/usr/src/sys/kern/uipc_mbuf.c", line 1364
> > 
> > With the following diff, we use the same size for mapping and
> > allocation.
> > 
> > ok?
> 
> I have tested this diff successfully.  Before jumbo frames with
> igc(4) paniced, now TCP is faster.
> 
> But are we allocating too much in the non-jumbo case?
>     sc->rx_mbuf_sz = MAX_JUMBO_FRAME_SIZE + ETHER_ALIGN;
>     m = MCLGETL(NULL, M_DONTWAIT, sc->rx_mbuf_sz)
> 
> This means for each receive packet we allocate a 16k mbuf cluster.
> Usually only 1500 byte are needed.
> 
> ix(4) has this code and manages to chain the mbufs correctly.
>     /* Use 2k clusters, even for jumbo frames */
>     sc->rx_mbuf_sz = MCLBYTES + ETHER_ALIGN;

This hardware chains fragments in the ring?

It's a tradeoff.  jumbos are rare, so why reserve so much memory to them
on hardware which can do better.