On Fri, Jun 13, 2025 at 01:53:13PM +1000, David Gwynne wrote:
> On Thu, Jun 12, 2025 at 10:34:54PM +0200, Alexander Bluhm wrote:
> > On Thu, Jun 12, 2025 at 10:58:34PM +0300, Vitaliy Makkoveev wrote:
> > > On Thu, Jun 12, 2025 at 07:55:20PM +0200, Alexander Bluhm wrote:
> > > > Hi,
> > > > 
> > > > syzcaller has found this crash in lo(4) calling bpf(4).
> > > > 
> > > > https://syzkaller.appspot.com/bug?extid=74f8141966171d61a0ab
> > > > 
> > > > panic: kernel diagnostic assertion "nlevel >= IPL_NONE" failed: file "/syzkaller/managers/multicore/kernel/sys/arch/amd64/amd64/intr.c", line 781
> > > > Stopped at      db_enter+0x25:  addq    $0x8,%rsp
> > > >     TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
> > > > *245945  56013      0     0x14000      0x200    1  softnet1
> > > >  316891  13736      0     0x14000 0x40000200    0  softclock
> > > > db_enter() at db_enter+0x25 sys/arch/amd64/amd64/db_interface.c:438
> > > > panic(ffffffff8346d347) at panic+0x1e5 sys/kern/subr_prf.c:198
> > > > __assert(ffffffff83415b99,ffffffff8335e4f5,30d,ffffffff833c132c) at __assert+0x29 sys/kern/subr_prf.c:-1
> > > > splraise(deadbeef) at splraise+0xbd sys/arch/amd64/amd64/intr.c:781
> > > > mtx_enter_try(ffff8000014e8608) at mtx_enter_try+0x97 sys/kern/kern_lock.c:276
> > > > mtx_enter(ffff8000014e8608) at mtx_enter+0x62 sys/kern/kern_lock.c:253
> > > > _bpf_mtap(ffff800000064c40,fffffd8068509300,ffff80002a216880,2) at _bpf_mtap+0x2fb sys/net/bpf.c:1410
> > > > bpf_mtap_af(ffff800000064c40,18,fffffd8068509300,2) at bpf_mtap_af+0x71 sys/net/bpf.c:1502
> > > > if_input_local(ffff800000b2c000,fffffd8068509300,18,ffffffff83970b58) at if_input_local+0xb2 sys/net/if.c:802
> > > > loinput(ffff800000b2c000,fffffd8068509300,ffffffff83970b58) at loinput+0x73 sys/net/if_loop.c:254
> > > > if_input_process(ffff800000b2c000,ffff80002a216a58,1) at if_input_process+0x1d5 sys/net/if.c:1012
> > > > ifiq_process(ffff800000b2d200) at ifiq_process+0xcd sys/net/ifq.c:874
> > > > taskq_thread(ffff80000002c100) at taskq_thread+0x157 sys/kern/kern_task.c:446
> > > > end trace frame: 0x0, count: 2
> > > > 
> > > > The splraise(deadbeef) looks very much like a use after free.
> > > > 
> > > > if_input_local() checks ifp->if_bpf and bpfdetach() sets ifp->if_bpf
> > > > to NULL.  As if_input_local() runs with shared net lock, bpfdetach()
> > > > should be locked exclusively.  As bpf_mtap() may be called by
> > > > driver's RX interrupt, we also need splnet().
> 
> raising the spl only applies to the current cpu, it does not block
> interrupts on any other cpu, so splnet is not enough to block rx
> interrupts.
> 
> > > > 
> > > > ok?
> > > > 
> > > 
> > > The interface should be detached from if_idxmap and it should have no
> > > queue routines running at this point. Could this be side effect of
> > > syncache regressions?
> > 
> > This panic is 3 days old.  There we only had a socket leak in the
> > syn cache.  A leak does not trigger deadbeef.
> > https://syzkaller.appspot.com/bug?extid=74f8141966171d61a0ab
> > 
> > And we had another one in bpf 4 days ago.
> > https://syzkaller.appspot.com/bug?extid=0177016a8bf3d5252ca3
> > 
> > Maybe it is a more general bpf bug.
> 
> this might be my fault after src/sys/net/if_loop.c r1.101 where i made
> lo(4) multiq.
> 
> interfaces should stop in flight packet processing (including
> if_input_process) before tearing state down. this is usually done
> when the interface is brought down, which drivers usually call during
> detach, but lo(4) is special. it relied on if_detach to do this work,
> but it appears that if_detach was lazy and only ran ifq_barrier against
> the 0th ifq. the diff below makes it run ifq_barrier against all the
> ifqs. this is consistent with the other ifq handling further down in
> if_detach.
> 

ok mvs

> Index: if.c
> ===================================================================
> RCS file: /cvs/src/sys/net/if.c,v
> diff -u -p -r1.734 if.c
> --- if.c	9 May 2025 03:12:36 -0000	1.734
> +++ if.c	13 Jun 2025 03:50:28 -0000
> @@ -1202,8 +1202,10 @@ if_detach(struct ifnet *ifp)
>  	ifp->if_qstart = if_detached_qstart;
>  
>  	/* Wait until the start routines finished. */
> -	ifq_barrier(&ifp->if_snd);
> -	ifq_clr_oactive(&ifp->if_snd);
> +	for (i = 0; i < ifp->if_nifqs; i++) {
> +		ifq_barrier(ifp->if_ifqs[i]);
> +		ifq_clr_oactive(ifp->if_ifqs[i]);
> +	}
>  
>  #if NBPFILTER > 0
>  	bpfdetach(ifp);
>