From: Mark Kettenis Subject: Re: bounce buffer mbuf defrag To: Alexander Bluhm Cc: tech@openbsd.org Date: Thu, 29 Aug 2024 19:16:54 +0200 > Date: Thu, 29 Aug 2024 17:28:58 +0200 > From: Alexander Bluhm > Cc: tech@openbsd.org > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > > On Thu, Aug 29, 2024 at 04:57:41PM +0200, Mark Kettenis wrote: > > > Date: Wed, 28 Aug 2024 14:37:45 +0200 > > > From: Alexander Bluhm > > > > > > Hi, > > > > > > In my tests I have seen packet drops vio_encap() when using bounce > > > buffers. Some mbufs seem to be too fragmented for the pre allocated > > > bounce buffer pages. Calling m_defrag() fixes the problem. > > > > Hmm. How does this happen? Is this a case for which the function > > would have return EFBIG at the end in the non-bounce-buffer case? > > The number of bounce buffers we allocate up front is a rough estimate. > In case of a mbuf chain the virtual addesses are fragmented, so we > need more segments. With TSO the TCP layer sends large 64K packets. > So all bounce pages are used up with real data, but we need two > pages when we split segements. > > I am trying the diff below. Allocate enough pages for size and > nsegments extra pages when we split the mbuf chain into segemnts. > If TCP socket send buffer is more fragmented than nsegments, > m_defrag() will rescue us. Well, I think that allocate nsegments of extra pages would be somewhat excessive. The current approach of MAX((size / PAGE_SIZE) + 1, nsegmenst) seems reasonable to me and falling back on m_defrag() seems a reasonable approach. > Without bouncing I see no problems in my tests. > > bluhm > > Index: arch/amd64/amd64/bus_dma.c > =================================================================== > RCS file: /data/mirror/openbsd/cvs/src/sys/arch/amd64/amd64/bus_dma.c,v > diff -u -p -r1.58 bus_dma.c > --- arch/amd64/amd64/bus_dma.c 28 Aug 2024 18:21:15 -0000 1.58 > +++ arch/amd64/amd64/bus_dma.c 29 Aug 2024 14:36:07 -0000 > @@ -140,7 +140,7 @@ _bus_dmamap_create(bus_dma_tag_t t, bus_ > > if (use_bounce_buffer) { > /* this many pages plus one in case we get split */ > - npages = round_page(size) / PAGE_SIZE + 1; > + npages = round_page(size) / PAGE_SIZE + nsegments; > if (npages < nsegments) > npages = nsegments; > mapsize += sizeof(struct vm_page *) * npages; > >