From: Claudio Jeker Subject: Re: TSO with small packets To: Marcus Glocker Cc: Alexander Bluhm , tech@openbsd.org Date: Sun, 11 Feb 2024 08:49:13 +0100 On Sat, Feb 10, 2024 at 11:22:44PM +0100, Marcus Glocker wrote: > On Sat, Feb 10, 2024 at 10:22:03PM +0100, Claudio Jeker wrote: > > > > The watchdog timeouts seen with em(4) tso enabled [1] do only happen when > > > em_tso_setup() receives an empty tcp frame, so the payload is zero. > > > I can understand that the hardware fails when we ask to tso zero bytes > > > of payload -- This shouldn't happen. > > > > > > Those empty tcp frames are only arriving in em_tso_setup() when ix(4) > > > forwards packets. I suspect the issue somewhere in > > > > > > http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/dev/pci/if_ix.c.diff?r1=1.196&r2=1.197 > > > > > > But I don't understand this LSO code good enough to spot what might go > > > wrong there. I can see that when this happens, ix(4) forwards a packet > > > with 60 bytes (vlan header + 2 bytes payload?) m_pkthdr.len, and 1 byte > > > m_pkthdr.ph_mss. Why this results in em_tso_setup() receiving an > > > packet with 54 bytes (no payload), I don't understand. > > > > This is because the minimal Ethernet packet is 64 bytes but since vlan > > header is used it is reduced to 60. There is a trailer in this packet > > since the payload (IP len) is less than 60 / 64 bytes. > > > > 54 bytes is probably 12 (ETH HDR) + 20 (IP) + 20 (TCP) + 2 Payload > > Isn't ETH HDR still 14 bytes (without vlan)? My calculation was > > 14 (ETH HDR) + 20 (IP) + 20 (TCP) = 0 Payload Right. Vlan header is 4 bytes. Did you manage to look at the TCP flags and sequence numbers? It indeed makes no sense to use TSO for such a packet :) > > > The updated diff skips empty tcp frames em_tso_setup(). With that the > > > watchdog timeouts reported by Hrvoje are gone. But obviously it's not > > > the solution for the issue in ix(4). > > > > Yes, this smells like a ix(4) LRO bug that may need a software workaround > > in ix(4) itself. -- :wq Claudio