From: Alexander Bluhm <bluhm@openbsd.org>
Subject: Re: unlock nd6 hiint
To: tech@openbsd.org
Date: Thu, 30 Jan 2025 22:19:02 +0100

On Sun, Jan 26, 2025 at 06:32:23PM +0100, Alexander Bluhm wrote:
> Hi,
> 
> TCP input is calling nd6_nud_hint() so I want to get rid of the
> exclusiv net lock.  Better use nd6 mutex in combination with shared
> net lock.

I did run this diff though my performance tests.  With IPv6 TCP I
see 1.5% decrease in throughput.  This is more or less what happens
due to a per packet mutex in the hot path.  As I see no other way
to run tcp_input() in parallel.  ND6 complexity requires to read
and update ln_state by multiple CPU.  We can always optmize later
and make ND6 locking smarter.

> ok?

anyone?

> bluhm
> 
> Index: netinet6/nd6.c
> ===================================================================
> RCS file: /data/mirror/openbsd/cvs/src/sys/netinet6/nd6.c,v
> diff -u -p -r1.283 nd6.c
> --- netinet6/nd6.c	4 Sep 2024 07:54:52 -0000	1.283
> +++ netinet6/nd6.c	26 Jan 2025 00:11:47 -0000
> @@ -709,21 +709,25 @@ nd6_nud_hint(struct rtentry *rt)
>  	struct llinfo_nd6 *ln;
>  	struct ifnet *ifp;
>  
> -	NET_ASSERT_LOCKED_EXCLUSIVE();
> -
> -	ifp = if_get(rt->rt_ifidx);
> -	if (ifp == NULL)
> -		return;
> +	NET_ASSERT_LOCKED();
>  
>  	if ((rt->rt_flags & RTF_GATEWAY) != 0 ||
>  	    (rt->rt_flags & RTF_LLINFO) == 0 ||
> -	    rt->rt_llinfo == NULL || rt->rt_gateway == NULL ||
> +	    rt->rt_gateway == NULL ||
>  	    rt->rt_gateway->sa_family != AF_LINK) {
>  		/* This is not a host route. */
> -		goto out;
> +		return;
>  	}
>  
> +	ifp = if_get(rt->rt_ifidx);
> +	if (ifp == NULL)
> +		return;
> +
> +	mtx_enter(&nd6_mtx);
> +
>  	ln = (struct llinfo_nd6 *)rt->rt_llinfo;
> +	if (ln == NULL)
> +		goto out;
>  	if (ln->ln_state < ND6_LLINFO_REACHABLE)
>  		goto out;
>  
> @@ -739,6 +743,7 @@ nd6_nud_hint(struct rtentry *rt)
>  	if (!ND6_LLINFO_PERMANENT(ln))
>  		nd6_llinfo_settimer(ln, ifp->if_nd->reachable);
>  out:
> +	mtx_leave(&nd6_mtx);
>  	if_put(ifp);
>  }
>