On Thu, Jun 12, 2025 at 10:34:54PM +0200, Alexander Bluhm wrote:
> On Thu, Jun 12, 2025 at 10:58:34PM +0300, Vitaliy Makkoveev wrote:
> > On Thu, Jun 12, 2025 at 07:55:20PM +0200, Alexander Bluhm wrote:
> > > Hi,
> > > 
> > > syzcaller has found this crash in lo(4) calling bpf(4).
> > > 
> > > https://syzkaller.appspot.com/bug?extid=74f8141966171d61a0ab
> > > 
> > > panic: kernel diagnostic assertion "nlevel >= IPL_NONE" failed: file "/syzkaller/managers/multicore/kernel/sys/arch/amd64/amd64/intr.c", line 781
> > > Stopped at      db_enter+0x25:  addq    $0x8,%rsp
> > >     TID    PID    UID     PRFLAGS     PFLAGS  CPU  COMMAND
> > > *245945  56013      0     0x14000      0x200    1  softnet1
> > >  316891  13736      0     0x14000 0x40000200    0  softclock
> > > db_enter() at db_enter+0x25 sys/arch/amd64/amd64/db_interface.c:438
> > > panic(ffffffff8346d347) at panic+0x1e5 sys/kern/subr_prf.c:198
> > > __assert(ffffffff83415b99,ffffffff8335e4f5,30d,ffffffff833c132c) at __assert+0x29 sys/kern/subr_prf.c:-1
> > > splraise(deadbeef) at splraise+0xbd sys/arch/amd64/amd64/intr.c:781
> > > mtx_enter_try(ffff8000014e8608) at mtx_enter_try+0x97 sys/kern/kern_lock.c:276
> > > mtx_enter(ffff8000014e8608) at mtx_enter+0x62 sys/kern/kern_lock.c:253
> > > _bpf_mtap(ffff800000064c40,fffffd8068509300,ffff80002a216880,2) at _bpf_mtap+0x2fb sys/net/bpf.c:1410
> > > bpf_mtap_af(ffff800000064c40,18,fffffd8068509300,2) at bpf_mtap_af+0x71 sys/net/bpf.c:1502
> > > if_input_local(ffff800000b2c000,fffffd8068509300,18,ffffffff83970b58) at if_input_local+0xb2 sys/net/if.c:802
> > > loinput(ffff800000b2c000,fffffd8068509300,ffffffff83970b58) at loinput+0x73 sys/net/if_loop.c:254
> > > if_input_process(ffff800000b2c000,ffff80002a216a58,1) at if_input_process+0x1d5 sys/net/if.c:1012
> > > ifiq_process(ffff800000b2d200) at ifiq_process+0xcd sys/net/ifq.c:874
> > > taskq_thread(ffff80000002c100) at taskq_thread+0x157 sys/kern/kern_task.c:446
> > > end trace frame: 0x0, count: 2
> > > 
> > > The splraise(deadbeef) looks very much like a use after free.
> > > 
> > > if_input_local() checks ifp->if_bpf and bpfdetach() sets ifp->if_bpf
> > > to NULL.  As if_input_local() runs with shared net lock, bpfdetach()
> > > should be locked exclusively.  As bpf_mtap() may be called by
> > > driver's RX interrupt, we also need splnet().

raising the spl only applies to the current cpu, it does not block
interrupts on any other cpu, so splnet is not enough to block rx
interrupts.

> > > 
> > > ok?
> > > 
> > 
> > The interface should be detached from if_idxmap and it should have no
> > queue routines running at this point. Could this be side effect of
> > syncache regressions?
> 
> This panic is 3 days old.  There we only had a socket leak in the
> syn cache.  A leak does not trigger deadbeef.
> https://syzkaller.appspot.com/bug?extid=74f8141966171d61a0ab
> 
> And we had another one in bpf 4 days ago.
> https://syzkaller.appspot.com/bug?extid=0177016a8bf3d5252ca3
> 
> Maybe it is a more general bpf bug.

this might be my fault after src/sys/net/if_loop.c r1.101 where i made
lo(4) multiq.

interfaces should stop in flight packet processing (including
if_input_process) before tearing state down. this is usually done
when the interface is brought down, which drivers usually call during
detach, but lo(4) is special. it relied on if_detach to do this work,
but it appears that if_detach was lazy and only ran ifq_barrier against
the 0th ifq. the diff below makes it run ifq_barrier against all the
ifqs. this is consistent with the other ifq handling further down in
if_detach.

Index: if.c
===================================================================
RCS file: /cvs/src/sys/net/if.c,v
diff -u -p -r1.734 if.c
--- if.c	9 May 2025 03:12:36 -0000	1.734
+++ if.c	13 Jun 2025 03:50:28 -0000
@@ -1202,8 +1202,10 @@ if_detach(struct ifnet *ifp)
 	ifp->if_qstart = if_detached_qstart;
 
 	/* Wait until the start routines finished. */
-	ifq_barrier(&ifp->if_snd);
-	ifq_clr_oactive(&ifp->if_snd);
+	for (i = 0; i < ifp->if_nifqs; i++) {
+		ifq_barrier(ifp->if_ifqs[i]);
+		ifq_clr_oactive(ifp->if_ifqs[i]);
+	}
 
 #if NBPFILTER > 0
 	bpfdetach(ifp);