The switch to per CPU mode relies on the new user not running yet. The comment above the code says so: the new user is not visible in the system, and the task that forks it runs the fixup. A failed execve() is missed. It rejoins its mm in sched_mm_cid_after_execve(): the new user is the running task, and it runs the fixup itself. If it is preempted before the fixup is done, mm_get_cid() can loop forever at its schedule in, with the runqueue lock held. In the diagram below, the machine has two possible CPUs. A and B are threads of one process, which is in per task mode with two CIDs. C is the vfork() child of B. A holds mm_cid.mutex from sched_mm_cid_fork(A) on. CPU 0 CPU 1 A: execve() sched_mm_cid_before_execve() A leaves, users = 1 B: vfork() sched_mm_cid_fork(C) users = 2 C takes the last CID A: execve() fails sched_mm_cid_after_execve() sched_mm_cid_fork(A) users = 3, switch to per CPU mode A is active and has no CID A is preempted A is scheduled in mm_cid_schedin(A) mm_get_cid() loops: B and C hold both CIDs C: exit() sched_mm_cid_exit() waits for mm_cid.mutex, which A holds Nothing ends the loop. Only the fixup could free a CID, and A is the task that runs it. A guest with two CPUs hangs on this sequence when a SCHED_FIFO task wakes every 50us on CPU 0 to preempt A. CPU 0 stays in __schedule() and CPU 1 in queued_spin_lock_slowpath(). With this change the same guest completes 20000 attempts. Fix this by keeping a running joiner inactive until the fixups are done. A schedule in gives it no CID before that. sched_mm_cid_fork() then marks it active, and it takes its CID under the runqueue lock. Fixes: 653fda7ae73d ("sched/mmcid: Switch over to the new mechanism") Signed-off-by: Mathieu Desnoyers Cc: stable@vger.kernel.org --- kernel/sched/core.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ab68e8605eed..510cd775723b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -11054,7 +11054,16 @@ static void sched_mm_cid_fork(struct task_struct *t) return; } - if (!sched_mm_cid_add_user(t, mm)) { + flip = sched_mm_cid_add_user(t, mm); + /* + * A running joiner takes no CID at a schedule in before the + * fixups are done. The users it joins can hold every CID + * until then, and the fixups are its own work. + */ + if (t == current) + t->mm_cid.active = 0; + + if (!flip) { if (!cid_on_cpu(mm->mm_cid.mode)) t->mm_cid.cid = mm_get_cid(mm); break; @@ -11064,7 +11073,6 @@ static void sched_mm_cid_fork(struct task_struct *t) * Handle the mode change and transfer current's CID. A * running joiner has no CID yet. It takes one below. */ - flip = true; percpu = cid_on_cpu(mm->mm_cid.mode); if (!percpu) mm_cid_transit_to_task(current, pcp); @@ -11077,18 +11085,16 @@ static void sched_mm_cid_fork(struct task_struct *t) mm_cid_fixup_tasks_to_cpus(); } else { mm_cid_fixup_cpus_to_tasks(mm); - /* - * A running joiner may have taken a CID at a - * schedule in. If not, it takes one below. - */ + /* A running joiner takes its CID below. */ if (t != current) t->mm_cid.cid = mm_get_cid(mm); } } - /* A running joiner with no CID takes one now, see above. */ + /* The running joiner takes part from here on, see above. */ if (t == current) { guard(task_rq_lock)(t); + t->mm_cid.active = 1; if (t->mm_cid.cid == MM_CID_UNSET) mm_cid_schedin(t); } -- 2.47.3