From: David Gwynne Subject: Re: ix(4): fix tx dma segment size To: tech@openbsd.org Date: Wed, 22 Apr 2026 11:43:24 +1000 On Tue, Apr 21, 2026 at 02:10:26PM +0200, Jan Klemkow wrote: > On Tue, Apr 21, 2026 at 03:06:47PM +1000, David Gwynne wrote: > > i thought it was weird that an MI driver like ix(4) is using an MD > > number like PAGE_SIZE for the maximum segment size in its tx dma > > maps. the manual says tx segments can be 16k (and goes into a 16bit > > field), so let's try using the documented value here. > > > > the rx descs also use 16 * 1024 as a magic number here, so it's in > > keeping with other code in the same driver. > > > > ive only tested this on an Intel 82599 rev 0x01 on arm64, so tests > > from other chips and sparc64 (which has an 8k page size) would be > > appreciated. > > works in my setups > > ok jan@ cool, thanks. we dusted a t4-1 with ix(4) inside off and tried it out. the ix1:0:ix-txq:1 and ix1:0:txq:1 stats seem to show the stack pushing large packets on to the hardware compared to what the hardware puts on the wire, so im assuming large clusters made it into the transmit path. dlg@t4-1 ~$ netstat -m 748 mbufs in use: 740 mbufs allocated to data 1 mbuf allocated to packet headers 7 mbufs allocated to socket names and addresses 0/80 mbuf 2048 byte clusters in use (current/peak) 553/1185 mbuf 2112 byte clusters in use (current/peak) 0/16 mbuf 4096 byte clusters in use (current/peak) 0/16 mbuf 8192 byte clusters in use (current/peak) 185/350 mbuf 9344 byte clusters in use (current/peak) 0/10 mbuf 12288 byte clusters in use (current/peak) 0/48 mbuf 16384 byte clusters in use (current/peak) 0/48 mbuf 65536 byte clusters in use (current/peak) 10344/10360/131072 Kbytes allocated to network (current/peak/max) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines 1 defrag mbuf allocation 846 prepend mbuf allocation 0 pullup mbuf allocation 0 pullup memory copy 0 pulldown mbuf allocation 0 pulldown memory copy dlg@t4-1 ~$ netstat -sp tcp tcp: 947225 packets sent 174820 data packets (68483396 bytes) 16 data packets (8288 bytes) retransmitted 0 fast retransmitted packets 651588 ack-only packets (730194 delayed) 0 URG only packets 0 window probe packets 120833 window update packets 16 control packets 0 packets software-checksummed 0 output TSO large packets chopped in software 3427 output TSO large packets to device 38046 output TSO generated packets sent to network 0 bad TSO packets dropped 1412337 packets received 129242 acks (for 68482682 bytes) 140 duplicate acks 0 acks for unsent data 0 acks for old data 1325149 packets (7122666881 bytes) received in-sequence 270 completely duplicate packets (487736 bytes) 137 old duplicate packets 5 packets with some duplicate data (18824 bytes duplicated) 56336 out-of-order packets (266824364 bytes) 0 packets (0 bytes) of data after window 0 window probes 1 window update packet 2 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded for missing IPsec protection 0 discarded due to memory shortage 0 packets software-checksummed 0 bad/missing md5 checksums 0 good md5 checksums 0 input LRO generated packets glued in software 697848 input LRO generated packets from device 4506091 input LRO small packets received from network 0 bad LRO packets dropped 6 connection requests 9 connection accepts 14 connections established (including accepts) 14 connections closed (including 2 drops) 0 connections drained 0 embryonic connections dropped 129244 segments updated rtt (of 102306 attempts) 16 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 26647 correct ACK header predictions 1228331 correct data packet header predictions 19 PCB cache misses 1 dropped due to no socket 0 dropped as socket is closing 0 ECN connections accepted 0 ECE packets received 0 CWR packets received 0 CE packets received 0 ECT packets sent 0 ECE packets sent 0 CWR packets sent cwr by fastrecovery: 6 cwr by timeout: 16 cwr by ecn: 0 0 bad connection attempts 0 SYN packets dropped due to queue or memory full 9 SYN cache entries added 0 hash collisions 9 completed 0 aborted (no space to build PCB) 0 timed out 0 dropped due to overflow 0 dropped due to bucket overflow 0 dropped due to RST 0 dropped due to ICMP unreachable 0 SYN,ACKs retransmitted 0 duplicate SYNs received for entries already in the cache 0 SYNs dropped (no route or no space) 1 SYN cache seed with new random 293 hash bucket array size in current SYN cache 0 entries in current SYN cache, limit is 10255 0 longest bucket length in current SYN cache, limit is 105 99991 uses of current SYN cache left 6 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 153 SACK options received 52803 SACK options sent 0 SACK options dropped dlg@t4-1 ~$ kstat ix1::: ix1:0:ix-rxq:0 packets: 3030955 packets bytes: 4531394140 bytes qdrops: 0 packets ix1:0:ix-rxq:1 packets: 2172116 packets bytes: 3238183677 bytes qdrops: 0 packets ix1:0:ix-rxq:2 packets: 2132 packets bytes: 411481 bytes qdrops: 0 packets ix1:0:ix-rxq:3 packets: 2079 packets bytes: 644032 bytes qdrops: 0 packets ix1:0:ix-rxq:4 packets: 2808 packets bytes: 671761 bytes qdrops: 0 packets ix1:0:ix-rxq:5 packets: 7774 packets bytes: 847054 bytes qdrops: 0 packets ix1:0:ix-rxq:6 packets: 5288 packets bytes: 974665 bytes qdrops: 0 packets ix1:0:ix-rxq:7 packets: 1701 packets bytes: 342626 bytes qdrops: 0 packets ix1:0:ix-stats:0 crc errs: 0 packets tx link xon: 0 packets rx link xon: 0 packets tx link xoff: 0 packets rx link xoff: 0 packets rx 64B: 56 packets rx 65-127B: 77395 packets rx 128-255B: 4875 packets rx 256-511B: 11043 packets rx 512-1023B: 47548 packets rx 1024-maxB: 5083936 packets tx good: 947230 packets rx good: 7794368848 bytes tx good: 137786158 bytes rx undersize: 0 packets rx fragment: 0 packets rx oversize: 0 packets rx jabber: 0 packets rx total: 7795233207 bytes rx total: 5230423 packets tx total: 947230 packets rx good: 5224853 packets rx bcast: 26422 packets rx mcast: 11551 packets tx 64B: 162 packets tx 65-127B: 806325 packets tx 128-255B: 100555 packets tx 256-511B: 324 packets tx 512-1023B: 1576 packets tx 1024-maxB: 38288 packets tx mcast: 2972 packets tx bcast: 32 packets ix1:0:ix-txq:0 packets: 3065 packets bytes: 380582 bytes ix1:0:ix-txq:1 packets: 39814 packets bytes: 59619838 bytes ix1:0:ix-txq:2 packets: 46 packets bytes: 5813 bytes ix1:0:ix-txq:3 packets: 83 packets bytes: 10932 bytes ix1:0:ix-txq:4 packets: 20275 packets bytes: 2380826 bytes ix1:0:ix-txq:5 packets: 883932 packets bytes: 71594376 bytes ix1:0:ix-txq:6 packets: 2 packets bytes: 160 bytes ix1:0:ix-txq:7 packets: 13 packets bytes: 4207 bytes ix1:0:rxq:0 packets: 886976 packets bytes: 4377768912 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 439270 dequeues: 438954 ix1:0:rxq:1 packets: 507983 packets bytes: 3119662441 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 274200 dequeues: 272644 ix1:0:rxq:2 packets: 2129 packets bytes: 402779 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 2089 dequeues: 2089 ix1:0:rxq:3 packets: 2025 packets bytes: 632800 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 1917 dequeues: 1917 ix1:0:rxq:4 packets: 2808 packets bytes: 660529 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 2679 dequeues: 2679 ix1:0:rxq:5 packets: 7718 packets bytes: 812268 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 7657 dequeues: 7657 ix1:0:rxq:6 packets: 5294 packets bytes: 954173 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 5119 dequeues: 5117 ix1:0:rxq:7 packets: 1701 packets bytes: 335822 bytes fdrops: 0 packets qdrops: 0 packets errors: 0 packets qlen: 0 packets enqueues: 1625 dequeues: 1625 ix1:0:txq:0 packets: 3066 packets bytes: 380334 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:1 packets: 5200 packets bytes: 57176058 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:2 packets: 46 packets bytes: 5629 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:3 packets: 83 packets bytes: 10600 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:4 packets: 20276 packets bytes: 2300024 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:5 packets: 883932 packets bytes: 68058648 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:6 packets: 2 packets bytes: 152 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0 ix1:0:txq:7 packets: 13 packets bytes: 4155 bytes qdrops: 0 packets errors: 0 packets qlen: 0 packets maxqlen: 255 packets oactive: false oactives: 0