From: Claudio Jeker <cjeker@diehard.n-r-g.com>
Subject: Re: Reduce KERNEL_LOCK() contention in uvn_detach()
To: Mark Kettenis <mark.kettenis@xs4all.nl>
Cc: Martin Pieuchot <mpi@grenadille.net>, tech@openbsd.org
Date: Tue, 22 Oct 2024 21:31:56 +0200

On Tue, Oct 22, 2024 at 08:56:16PM +0200, Mark Kettenis wrote:
> > Date: Sat, 19 Oct 2024 15:16:03 +0200
> > From: Martin Pieuchot <mpi@grenadille.net>
> > 
> > The reaper is currently running mostly without grabbing the KERNEL_LOCK().
> > The contending exception is the teardown path of mmaped files: uvn_detach().
> > 
> > We can easily improve this case by grabbing the KERNEL_LOCK() only if UVM
> > has released its last reference on the vnode.
> > 
> > Here it's ok to interleave the KERNEL_LOCK() and a rwlock because the
> > KERNEL_LOCK() is always released when failing to grab a rwlock.  The
> > same trick has been (is?) used in the network stack with the NET_LOCK().
> > 
> > This shifts the contention when using multiple reaper paths as tested by
> > claudio@.
> > 
> > ok?
> 
> ok kettenis@

Also OK claudio@
 
> > Index: uvm/uvm_vnode.c
> > ===================================================================
> > RCS file: /cvs/src/sys/uvm/uvm_vnode.c,v
> > diff -u -p -r1.133 uvm_vnode.c
> > --- uvm/uvm_vnode.c	24 Jul 2024 12:16:21 -0000	1.133
> > +++ uvm/uvm_vnode.c	8 Oct 2024 09:14:47 -0000
> > @@ -306,15 +306,14 @@ uvn_detach(struct uvm_object *uobj)
> >  	struct vnode *vp;
> >  	int oldflags;
> >  
> > -	KERNEL_LOCK();
> >  	rw_enter(uobj->vmobjlock, RW_WRITE);
> >  	uobj->uo_refs--;			/* drop ref! */
> >  	if (uobj->uo_refs) {			/* still more refs */
> >  		rw_exit(uobj->vmobjlock);
> > -		KERNEL_UNLOCK();
> >  		return;
> >  	}
> >  
> > +	KERNEL_LOCK();
> >  	/* get other pointers ... */
> >  	uvn = (struct uvm_vnode *) uobj;
> >  	vp = uvn->u_vnode;
> > 
> > 
> 

-- 
:wq Claudio