sched_mm_cid_fork() makes a task a user of the CIDs of its mm. In per CPU mode the new user gets no CID there. It takes the CID of its CPU when it is scheduled in. This relies on the new user not running yet. That holds for a fork() child. A successful execve() joins a new mm as its first user, and the first user gets a CID directly. A failed execve() is missed. bprm_execve() leaves the mm in sched_mm_cid_before_execve(). Its error path rejoins the same mm in sched_mm_cid_after_execve(). If the execve() fails before begin_new_exec() kills the other threads, they are still users of the mm. The mm can then be in per CPU mode. The task is running, so it returns to user space with no schedule in. Its CID is still MM_CID_UNSET, and rseq_force_update() makes the kernel write it to the rseq area. User space reads an mm_cid of 0x80000000 until its next context switch. A program that indexes per-CID data with it would read out of bounds. An execve() of a file that no binfmt handler recognises fails in that window. In testing, with 20 threads on a machine with 8 CPUs: mm_cid 2147483648 (0x80000000) after a failed execve, round 0 Fix this by giving a running joiner its CID in sched_mm_cid_fork(). It takes the CID a schedule in would give it, under the runqueue lock. The switch back to per task mode has the same problem. Since commit 7dbc307b6c72 ("sched/mmcid: Fix affinity-triggered switch back to per-task mode"), a join can do the switch that an affinity change queued. For a running joiner: 1. sched_mm_cid_fork() runs mm_cid_fixup_cpus_to_tasks(). 2. The joiner is preempted. It takes a CID at its schedule in. 3. sched_mm_cid_fork() allocates a CID for the new user. That overwrites the CID of step 2. The bit of the first CID stays set in the bitmap, with no owner. In testing, three threads on two CPUs then read an mm_cid of 3 or more. Leave that allocation to a joiner that is not running. A running joiner takes its CID as above if it has none. Fixes: 653fda7ae73d ("sched/mmcid: Switch over to the new mechanism") Signed-off-by: Mathieu Desnoyers Cc: stable@vger.kernel.org --- kernel/sched/core.c | 43 +++++++++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1a088d4576f5..f2a981d418eb 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -10986,10 +10986,19 @@ static bool sched_mm_cid_add_user(struct task_struct *t, struct mm_struct *mm) return mm_update_max_cids(mm); } +/* + * Make @t a user of the CIDs of its mm. Called on fork(2) and on + * execve(2). A failed execve(2) rejoins the mm it left. + * + * In per CPU mode a new user gets no CID here. It takes the CID of its + * CPU when it is scheduled in. A task that joins while it is running + * returns to user space before that, so it takes its CID here. This is + * the case of a failed execve(2) when the mm has other users. + */ static void sched_mm_cid_fork(struct task_struct *t) { struct mm_struct *mm = t->mm; - bool percpu; + bool percpu = false, flip = false; if (!mm) return; @@ -11012,10 +11021,14 @@ static void sched_mm_cid_fork(struct task_struct *t) if (!sched_mm_cid_add_user(t, mm)) { if (!cid_on_cpu(mm->mm_cid.mode)) t->mm_cid.cid = mm_get_cid(mm); - return; + break; } - /* Handle the mode change and transfer current's CID */ + /* + * Handle the mode change and transfer current's CID. A + * running joiner has no CID yet. It takes one below. + */ + flip = true; percpu = cid_on_cpu(mm->mm_cid.mode); if (!percpu) mm_cid_transit_to_task(current, pcp); @@ -11023,11 +11036,25 @@ static void sched_mm_cid_fork(struct task_struct *t) mm_cid_transit_to_cpu(current, pcp); } - if (percpu) { - mm_cid_fixup_tasks_to_cpus(); - } else { - mm_cid_fixup_cpus_to_tasks(mm); - t->mm_cid.cid = mm_get_cid(mm); + if (flip) { + if (percpu) { + mm_cid_fixup_tasks_to_cpus(); + } else { + mm_cid_fixup_cpus_to_tasks(mm); + /* + * A running joiner may have taken a CID at a + * schedule in. If not, it takes one below. + */ + if (t != current) + t->mm_cid.cid = mm_get_cid(mm); + } + } + + /* A running joiner with no CID takes one now, see above. */ + if (t == current) { + guard(task_rq_lock)(t); + if (t->mm_cid.cid == MM_CID_UNSET) + mm_cid_schedin(t); } } -- 2.47.3