From: Tom Smyth Subject: Re: Bug Reported by me today Create, Bridge, change bridge timeout and then destroy bridge kernel panic To: Theo de Raadt Cc: tech@openbsd.org Date: Tue, 3 Nov 2026 22:47:07 +0000 Apologies Theo all, for full context the email was intended to follow on from the bug report sent earlier today, with subject subject: bridge(4) (7.9 and 8.0): timeout set on a down bridge, then destroy: panic in timeout wheel Tom Smyth 16:26 (6 hours ago) to bugs@ Folks, I encountered this bug on AMD64 OpenBSD 7.9 and OpenBSD8.0 testing and analysis assisted with AI, but the bug is real and confirmed in person, manual test and recreation ogma-tl1-crash80# ifconfig bridge57 create ogma-tl1-crash80# ifconfig bridge57 timeout 5 ogma-tl1-crash80# ifconfig bridge57 destroy ogma-tl1-crash80# ogma-tl1-crash80# ogma-tl1-crash80# ogma-tl1-crash80# uvm_fault(0xffffffff82b3f2b8, 0xffff800000d21558, 0, 2) -> e fatal page fault in supervisor mode trap type 6 code 2 rip ffffffff817c441c cs 8 rflags 10202 cr2 ffff800000d21558 cpl d rsp ffff800045150dc0 gsbase 0xffffffff82ae5ff0 kgsbase 0x0 panic: trap type 6, code=2, pc=ffffffff817c441c Starting stack trace... panic(ffffffff826e19fc) at panic+0x170 kerntrap(ffff800045150d10) at kerntrap+0x15d alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b timeout_hardclock_update() at timeout_hardclock_update+0xfc clockintr_hardclock(ffffffff82ae6de8,ffff800045150f20,0) at clockintr_hardclock+0xa8 clockintr_dispatch(ffff800045150f20) at clockintr_dispatch+0x254 lapic_clockintr(0,0) at lapic_clockintr+0x43 Xresume_lapic_ltimer() at Xresume_lapic_ltimer+0x2a acpicpu_idle() at acpicpu_idle+0x2b3 sched_idle(ffffffff82ae5ff0) at sched_idle+0x244 end trace frame: 0x0, count: 247 End of stack trace. dump to dev 4,1 not possible >Synopsis: bridge timeout set while down, then destroy: panic (use after free of sc_brtimeout) >Category: kernel >Environment: System : OpenBSD 7.9 (also reproduced on OpenBSD 8.0, see below) Details : OpenBSD 7.9 (GENERIC.MP) #19: Mon Oct 19 02:26:40 MDT 2026 (syspatch kernel) and OpenBSD 8.0 (GENERIC.MP) #2: Mon Oct 19 12:55:12 MDT 2026 (syspatch kernel), same host, 2 vCPUs, 4 GB RAM Architecture: OpenBSD.amd64 Machine : amd64, QEMU/KVM guest on Proxmox VE (Standard PC (Q35 + ICH9, 2009)), virtio devices, Intel Xeon X5650 host CPU, 2 vCPUs, 3 GB RAM >Description: Setting the bridge address cache timeout on a bridge(4) that has never been brought up and then destroying the bridge leaves the bridge's sc_brtimeout queued in the timeout wheel after the softc has been freed. The next time the wheel touches that bucket the kernel faults (write to an unmapped page): uvm_fault(0xffffffff82b1dea0, 0xffff800001349558, 0, 2) -> e fatal page fault in supervisor mode trap type 6 code 2 rip ffffffff81aff933 cs 8 rflags 10202 cr2 ffff800001349558 cpl d rsp ffff80002ad0a230 gsbase 0xffffffff82abcff0 kgsbase 0x0 panic: trap type 6, code=2, pc=ffffffff81aff933 Starting stack trace... panic(ffffffff826a2667) at panic+0x170 kerntrap(ffff80002ad0a180) at kerntrap+0x15d alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b timeout_hardclock_update() at timeout_hardclock_update+0x103 clockintr_hardclock(ffffffff82abdde0,ffff80002ad0a390,0) at clockintr_hardclock+0xa8 clockintr_dispatch(ffff80002ad0a390) at clockintr_dispatch+0x259 lapic_clockintr(0,0) at lapic_clockintr+0x43 Xresume_lapic_ltimer() at Xresume_lapic_ltimer+0x2a acpicpu_idle() at acpicpu_idle+0x2b3 sched_idle(ffffffff82abcff0) at sched_idle+0x24b end trace frame: 0x0, count: 247 End of stack trace. An earlier panic on the same machine, from a test loop doing the same thing with a few more ioctls (maxaddr), was identical apart from addresses: uvm_fault(0xffffffff82aecb78, 0xffff800001348558, 0, 2) -> e fatal page fault in supervisor mode trap type 6 code 2 rip ffffffff81bcef33 cs 8 rflags 10206 cr2 ffff800001348558 cpl d rsp ffff80002ad0a090 gsbase 0xffffffff82ac6ff0 kgsbase 0x0 panic: trap type 6, code=2, pc=ffffffff81bcef33 Starting stack trace... panic(ffffffff826a4947) at panic+0x170 kerntrap(ffff80002ad09fe0) at kerntrap+0x15d alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b timeout_hardclock_update() at timeout_hardclock_update+0x73 clockintr_hardclock(ffffffff82ac7de0,ffff80002ad0a1f0,0) at clockintr_hardclock+0xa8 clockintr_dispatch(ffff80002ad0a1f0) at clockintr_dispatch+0x259 lapic_clockintr(0,0) at lapic_clockintr+0x43 Xresume_lapic_ltimer() at Xresume_lapic_ltimer+0x2a acpicpu_idle() at acpicpu_idle+0x2b3 sched_idle(ffffffff82ac6ff0) at sched_idle+0x24b end trace frame: 0x0, count: 247 End of stack trace. (Both traces were recovered from the message buffer after the reboot. The crash dump started both times but did not complete, so there is no vmcore.) A different machine with the same kernel panicked hours after a similar sequence (maxaddr 100, timeout 86400 on a bridge that was never up, destroy), with the same kind of fault (write, cr2 at offset 0x550 in an unmapped page) in softclock_process_tick_timeout+0x1e3, called from softclock(), at the CIRCQ_INSERT_TAIL into a wheel bucket (rcx pointed into timeout_wheel). I believe this is the same bug, but I did not reproduce that one directly. From ddb on that machine: ddb{0}> show panic *cpu0: uvm_fault(0xffffffff82b9a548, 0xffff800000f2f550, 0, 2) -> e ddb{0}> bt softclock_process_tick_timeout(fffffd810875fc20,1) at softclock_process_tick_timeout+0x1e3 softclock(0) at softclock+0x10e softintr_dispatch(0) at softintr_dispatch+0x101 dosoftint(0) at dosoftint+0x3f Xsoftclock() at Xsoftclock+0x27 end of kernel ddb{0}> show registers (excerpt) rdi 0xfffffd810875fc20 rdx 0xffff800000f2f550 rcx 0xffffffff82af38e0 timeout_wheel+0x13f0 rax 0xffff800000f2f550 r15 0xfffffd810875fc20 rip 0xffffffff81168de3 softclock_process_tick_timeout+0x1e3 softclock_process_tick_timeout+0x1e3: movq %rdi,0(%rax) Source analysis (7.9 release sys.tar.gz: if_bridge.c 1.382, bridgectl.c 1.25, kern_timeout.c 1.112; no 7.9 errata patch touches these files): 1. sys/net/bridgectl.c, bridgectl_ioctl(), SIOCBRDGSTO (lines 109-120) arms the timeout without looking at IFF_RUNNING: 115 sc->sc_brttimeout = bparam->ifbrp_ctime; 116 if (bparam->ifbrp_ctime != 0) 117 timeout_add_sec(&sc->sc_brtimeout, sc->sc_brttimeout); Otherwise the timeout is only armed in bridge_init() (if_bridge.c:776-777), when the bridge goes up, and by bridge_rtage() re-arming itself (bridgectl.c:352-353). bridge_rtage() itself returns at once if the bridge is not running (bridgectl.c:326-327). 2. sys/net/if_bridge.c, bridge_stop() (lines 786-798) returns early on a bridge that is not running, so it never reaches the timeout_del_barrier(): 790 if (!ISSET(ifp->if_flags, IFF_RUNNING)) 791 return; ... 797 timeout_del_barrier(&sc->sc_brtimeout); 3. sys/net/if_bridge.c, bridge_clone_destroy() (lines 221-246) relies on bridge_stop() and does not delete sc_brtimeout itself before freeing the softc: 233 bridge_stop(sc); ... 244 free(sc, M_DEVBUF, sizeof *sc); So after "timeout N" on a bridge that is down, the timeout stays in the wheel and the softc is freed under it. struct bridge_softc is bigger than MAXALLOCSAVE (sc_rts[] alone is 1024 LIST_HEADs), so free() gives the pages back to the kernel map and the next wheel operation on that bucket (CIRCQ_CONCAT in MOVEBUCKET from timeout_hardclock_update(), kern_timeout.c:120- 122/149-157/609, or CIRCQ_INSERT_TAIL in softclock_process_tick_timeout(), kern_timeout.c:742) writes through the stale to_list and faults. If the pages were reused instead, the result would be silent corruption of the wheel. sys/net/if_bridge.c is unchanged in -current (still 1.382), and the SIOCBRDGSTO and bridge_rtage() code in bridgectl.c 1.26 is unchanged as well, so I believe -current is affected too. Also reproduced on OpenBSD 8.0: OpenBSD 8.0 (GENERIC.MP) #2: Mon Oct 19 12:55:12 MDT 2026 (syspatch kernel), amd64, same kind of QEMU/KVM guest, 2 vCPUs, 4 GB RAM. The same three commands panicked 2 out of 2 times within about 30 seconds. Traces from the message buffer after the reboot: Run 1: uvm_fault(0xffffffff82b9ec40, 0xffff800000bdf550, 0, 2) -> e fatal page fault in supervisor mode trap type 6 code 2 rip ffffffff82521daa cs 8 rflags 10202 cr2 ffff800000bdf550 cpl d rsp ffff800045151790 panic: trap type 6, code=2, pc=ffffffff82521daa panic(ffffffff826e1fef) at panic+0x170 kerntrap(ffff8000451516e0) at kerntrap+0x15d alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b softclock_process_tick_timeout(ffffffff82b092a0,1) at softclock_process_tick_timeout+0x1da softclock(0) at softclock+0x10e softintr_dispatch(0) at softintr_dispatch+0x105 dosoftint(0) at dosoftint+0x3f Xsoftclock() at Xsoftclock+0x27 acpicpu_idle() at acpicpu_idle+0x2b3 sched_idle(ffffffff82b26ff0) at sched_idle+0x244 Run 2: uvm_fault(0xffffffff82b3f2b8, 0xffff800000d20558, 0, 2) -> e fatal page fault in supervisor mode trap type 6 code 2 rip ffffffff817c3e96 cs 8 rflags 10202 cr2 ffff800000d20558 cpl d rsp ffff8000451f8d80 panic: trap type 6, code=2, pc=ffffffff817c3e96 panic(ffffffff826e19fc) at panic+0x170 kerntrap(ffff8000451f8cd0) at kerntrap+0x15d alltraps_kern_meltdown() at alltraps_kern_meltdown+0x7b timeout_del(ffff800045229aa0) at timeout_del+0x46 sleep_finish(16e362ee0,1) at sleep_finish+0x1a1 msleep_nsec(...) at msleep_nsec+0xcc kqueue_sleep(ffffef8160726228,ffff8000451f91b8) at kqueue_sleep+0xc7 kqueue_scan(...) at kqueue_scan+0xc1 sys_kevent(...) at sys_kevent+0x38f syscall(ffff8000451f9310) at syscall+0x60d Xsyscall() at Xsyscall+0x128 Both are again write faults at page offset 0x550 / 0x558 in an unmapped page, as on 7.9. Run 2 hits the stale wheel entry from an unrelated timeout_del() (a kevent sleep timer in the same bucket), which shows any timeout user can trip over it. >How-To-Repeat: As root, on a machine where bridge57 does not exist: ifconfig bridge57 create ifconfig bridge57 timeout 5 ifconfig bridge57 destroy sleep 15 The machine panics when the 5 second timeout is due (2 out of 2 runs). No members, no load and no other interfaces are needed. The bridge is never brought up. Expected: the bridge is destroyed and nothing else happens. Actual: kernel panic (page fault) in the timeout wheel about 5 seconds later. These variants did not panic (each followed by an 8 second wait): create; destroy create; up; destroy create; maxaddr 100; destroy create; timeout 5; up; destroy create; up; timeout 5; destroy create; up; timeout 5; down; destroy All of these either never arm the timeout, or have the bridge running at destroy time (or pass through bridge_stop() while it is running), so the timeout is deleted. With "timeout 86400" instead of 5 there was no panic within 5 minutes of load; the stale entry presumably waits in a higher wheel level until its bucket is touched. >Fix: The diff below is UNTESTED (not even compiled). It deletes the timeout in bridge_clone_destroy() whatever the IFF_RUNNING state, and only arms it in SIOCBRDGSTO when the bridge is running (bridge_init() arms it when the bridge goes up, so the new value still takes effect then). Either hunk alone should stop the panic; the first one is the actual fix. Untested workaround: "ifconfig bridgeN timeout 0" before destroy (SIOCBRDGSTO with 0 calls timeout_del()), or bring the bridge up before setting the timeout. Index: sys/net/if_bridge.c --- sys/net/if_bridge.c.orig +++ sys/net/if_bridge.c @@ -231,6 +231,7 @@ NET_ASSERT_UNLOCKED(); bridge_stop(sc); + timeout_del_barrier(&sc->sc_brtimeout); bridge_rtflush(sc, IFBF_FLUSHALL); while ((bif = SMR_SLIST_FIRST_LOCKED(&sc->sc_iflist)) != NULL) bridge_ifremove(bif); Index: sys/net/bridgectl.c --- sys/net/bridgectl.c.orig +++ sys/net/bridgectl.c @@ -113,9 +113,11 @@ break; } sc->sc_brttimeout = bparam->ifbrp_ctime; - if (bparam->ifbrp_ctime != 0) - timeout_add_sec(&sc->sc_brtimeout, sc->sc_brttimeout); - else + if (bparam->ifbrp_ctime != 0) { + if (ISSET(ifp->if_flags, IFF_RUNNING)) + timeout_add_sec(&sc->sc_brtimeout, + sc->sc_brttimeout); + } else timeout_del(&sc->sc_brtimeout); break; case SIOCBRDGGTO: On Tue, 3 Nov 2026 at 21:48, Theo de Raadt wrote: > > Sorry this is a poor effort. > > 1. does not explain the problem. > 2. does not try to interpret the problem > 3. AI tells me this is the solution > 4. Patch indents are broken. > > >Folks I tested the following patch ( with assistance from ai) > >and it stopped the panic on AMD64 on 8.0 on 5 separate runs > > > >Index: sys/net/if_bridge.c > >--- sys/net/if_bridge.c.orig > >+++ sys/net/if_bridge.c > >@@ -231,6 +231,7 @@ > > NET_ASSERT_UNLOCKED(); > > > > bridge_stop(sc); > >+ timeout_del_barrier(&sc->sc_brtimeout); > > bridge_rtflush(sc, IFBF_FLUSHALL); > > while ((bif = SMR_SLIST_FIRST_LOCKED(&sc->sc_iflist)) != NULL) > > bridge_ifremove(bif); > >Index: sys/net/bridgectl.c > >--- sys/net/bridgectl.c.orig > >+++ sys/net/bridgectl.c > >@@ -113,9 +113,11 @@ > > break; > > } > > sc->sc_brttimeout = bparam->ifbrp_ctime; > >- if (bparam->ifbrp_ctime != 0) > >- timeout_add_sec(&sc->sc_brtimeout, sc->sc_brttimeout); > >- else > >+ if (bparam->ifbrp_ctime != 0) { > >+ if (ISSET(ifp->if_flags, IFF_RUNNING)) > >+ timeout_add_sec(&sc->sc_brtimeout, > >+ sc->sc_brttimeout); > >+ } else > > timeout_del(&sc->sc_brtimeout); > > break; > > case SIOCBRDGGTO: > > > > -- Kindest regards, Tom Smyth.