nldsq_cursor_next_task() unlinks the cursor and returns NULL when the DSQ walk is finished. A later bpf_iter_scx_dsq_next() call sees an empty cursor and nldsq_next_task(dsq, NULL, rev) starts again from the DSQ head; the old tasks are still queued, so the iterator returns them again. The verifier relies on the iterator contract that once next() returns NULL it keeps returning NULL: process_iter_next_call() marks the iterator DRAINED and treats later next() results as the constant zero. A restarted DSQ iteration therefore hands the BPF program a kernel pointer while the verifier believes R0 == 0. Remember the drained state in the cursor flags and return NULL on every later next() call. The cursor has spare flag bits and the BPF-visible iterator layout is unchanged. Fixes: 650ba21b131e ("sched_ext: Implement DSQ iterator") Signed-off-by: Kang Chen --- v2: - Shorten the code comment. kernel/sched/ext/ext.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c index f74ffb3b1aca..8a7a33e203c2 100644 --- a/kernel/sched/ext/ext.c +++ b/kernel/sched/ext/ext.c @@ -580,12 +580,14 @@ enum scx_dsq_iter_flags { /* iterate in the reverse dispatch order */ SCX_DSQ_ITER_REV = 1U << 16, + __SCX_DSQ_ITER_DRAINED = 1U << 28, __SCX_DSQ_ITER_HAS_REENQ = 1U << 29, __SCX_DSQ_ITER_HAS_SLICE = 1U << 30, __SCX_DSQ_ITER_HAS_VTIME = 1U << 31, __SCX_DSQ_ITER_USER_FLAGS = SCX_DSQ_ITER_REV, __SCX_DSQ_ITER_ALL_FLAGS = __SCX_DSQ_ITER_USER_FLAGS | + __SCX_DSQ_ITER_DRAINED | __SCX_DSQ_ITER_HAS_REENQ | __SCX_DSQ_ITER_HAS_SLICE | __SCX_DSQ_ITER_HAS_VTIME, @@ -11028,13 +11030,18 @@ __bpf_kfunc int bpf_iter_scx_dsq_new(struct bpf_iter_scx_dsq *it, u64 dsq_id, __bpf_kfunc struct task_struct *bpf_iter_scx_dsq_next(struct bpf_iter_scx_dsq *it) { struct bpf_iter_scx_dsq_kern *kit = (void *)it; + struct task_struct *p; - if (!kit->dsq) + if (!kit->dsq || (kit->cursor.flags & __SCX_DSQ_ITER_DRAINED)) return NULL; guard(raw_spinlock_irqsave)(&kit->dsq->lock); - return nldsq_cursor_next_task(&kit->cursor, kit->dsq); + p = nldsq_cursor_next_task(&kit->cursor, kit->dsq); + if (!p) + kit->cursor.flags |= __SCX_DSQ_ITER_DRAINED; + + return p; } /** -- 2.34.1