From: Greg Schaefer <gsgs7878@proton.me>
Subject: sched_choosecpu returning curcpu panics sched_chooseproc/SPCF_SHOULDHALT
To: "tech@openbsd.org" <tech@openbsd.org>
Date: Wed, 11 Feb 2026 21:28:17 +0000

While experimenting with avoiding scheduling processes on idle C6+
high-exit-latency sleeping cores, I triggered an unexpected panic
when sched_choosecpu() returned its fallback value of curcpu().

The panic occurs because the sched_chooseproc() SPCF_SHOULDHALT
code uses setrunqueue(NULL, ...) (choose a core) followed by if
(p->p_cpu == curcpu()) KASSERT(p->p_flag & P_CPUPEG).

In practice, this does not happen as secondary cores are removed
from sched_all_cpus immediately prior to SPCF_SHOULDHALT (with only
the primary remaining) and thus sched_chooseproc() chooses it.
Extra filtering logic (like my experiment) or empty sched_all_cpus
can allow sched_chooseproc() to return curcpu().

My workaround is below-- SPCF_SHOULDHALT moves all non-pegged procs
explicitly to the primary core and lets the main sched_chooseproc()
logic handle pegged and idle. Another approach is sched_choosecpu()
returning cpuset_first(&sched_all_cpus) as the default, but that
returns NULL if sched_all_cpus is ever empty. Not sure its worth
changing, but figured it was worth an FYI.

diff -u /usr/src/sys78/kern/kern_sched.c /usr/src/run78/kern/kern_sched.c
--- /usr/src/sys78/kern/kern_sched.c	Thu Jun 12 15:37:58 2025
+++ /usr/src/run78/kern/kern_sched.c	Sun Feb  8 15:47:40 2026
@@ -331,24 +331,17 @@
 		if (spc->spc_whichqs) {
 			for (queue = 0; queue < SCHED_NQS; queue++) {
 				while ((p = TAILQ_FIRST(&spc->spc_qs[queue]))) {
+					/* move non-pegged procs to primary cpu */
+					while ((p != NULL) && (p->p_flag & P_CPUPEG))
+						p = TAILQ_NEXT(p, p_runq);
+					if (p == NULL)
+						break;
 					remrunqueue(p);
-					setrunqueue(NULL, p, p->p_runpri);
-					if (p->p_cpu == curcpu()) {
-						KASSERT(p->p_flag & P_CPUPEG);
-						goto again;
-					}
+					setrunqueue(cpu_info_list, p, p->p_runpri);
 				}
 			}
 		}
-		p = spc->spc_idleproc;
-		if (p == NULL)
-			panic("no idleproc set on CPU%d",
-			    CPU_INFO_UNIT(curcpu()));
-		p->p_stat = SRUN;
-		KASSERT(p->p_wchan == NULL);
-		return (p);
 	}
-again:
 #endif

 	if (spc->spc_whichqs) {
@@ -500,11 +493,11 @@
 	struct cpu_info *ci;
 	struct cpuset set;

-	KASSERT((self->ci_schedstate.spc_schedflags & SPCF_SHOULDHALT) == 0);
-
 	/* Don't steal if we don't want to schedule processes in this CPU. */
 	if (!cpuset_isset(&sched_all_cpus, self))
 		return (NULL);
+
+	KASSERT((self->ci_schedstate.spc_schedflags & SPCF_SHOULDHALT) == 0);

 	cpuset_copy(&set, &sched_queued_cpus);