From: Yun Lu Cancellation clears the task-work slot, but a BPF program can still hold the deleted element's value pointer in its read-side section. Scheduling through that pointer can publish a new context. Its reference keeps the context alive, not the element referenced by ctx->map_val: 1. A BPF program looks up E and retains its value pointer. 2. Another task deletes E, cancels its task work and clears the slot. 3. The original reader publishes a new context C, schedules task work and leaves its trace read-side section. 4. C enters RUNNING outside the grace period's waiting set for E, before the final destructor marks C FREED. 5. The destructor frees E without waiting for C. The callback then accesses E through C->map_val. A controlled reproducer on the unmodified kernel at commit 7c3188f84b54 ("selftests/bpf: Time tc_edt from its first packet with data") reported: BUG: KASAN: slab-use-after-free in array_map_update_elem+0x4b8/0x7f0 Read of size 8 at addr ffff88800e61be98 by task uaf_repro/91 Call Trace: __asan_memcpy+0x23/0x60 array_map_update_elem+0x4b8/0x7f0 bpf_prog_976f3fa6d36fbef6_cb_probe_step+0xaf/0xe5 bpf_prog_ac7215eae4609044_process_work_cb+0x12c/0x234 bpf_task_work_callback+0x351/0x970 task_work_run+0x116/0x200 Allocated by task 90: alloc_bulk+0x238/0x360 bpf_mem_alloc_init+0x2fd/0x930 rhtab_map_alloc+0x352/0x430 Freed by task 9: kfree+0x1b8/0x550 free_all.isra.0+0x5b/0x110 __free_rcu+0x87/0x1b0 srcu_invoke_callbacks+0x360/0x5c0 After publishing C and claiming PENDING, look up the key and require the returned pointer to match the retained value. This is before irq_work is queued: either lookup rejects the removed value or deletion finds the published context and marks it FREED. Take regular RCU around the lookup because sleepable callers hold only a trace read-side section. On rejection, conditionally roll PENDING back to STANDBY so a concurrent FREED is preserved, and drop this attempt's callback reference. The idle context remains available for normal cancellation and reclamation. Check membership before the usercnt policy, so a removed value returns -ENOENT when this check is reached even if usercnt is zero. Compare value addresses as well as keys: replacement must reject the old value, while reinsertion at the same address is the current member. Live in-place updates retain their existing behavior. Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs") Suggested-by: Alexei Starovoitov Signed-off-by: Yun Lu --- kernel/bpf/helpers.c | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 281fedabf7b0..a2c199aed893 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1148,6 +1148,18 @@ static void *map_key_from_value(struct bpf_map *map, void *value, u32 *arr_idx) return (void *)value - round_up(map->key_size, 8); } +/* Does lookup still return this value under its key? */ +static bool bpf_map_value_is_linked(struct bpf_map *map, void *value) +{ + u32 idx; + void *key; + + /* Sleepable callers need regular RCU for the map lookup itself. */ + guard(rcu)(); + key = map_key_from_value(map, value, &idx); + return map->ops->map_lookup_elem(map, key) == value; +} + enum bpf_async_type { BPF_ASYNC_TYPE_TIMER = 0, BPF_ASYNC_TYPE_WQ, @@ -4774,6 +4786,16 @@ static struct bpf_task_work_ctx *bpf_task_work_acquire_ctx(struct bpf_task_work return ERR_PTR(-EBUSY); } + /* + * ctx is published and PENDING, with no irq_work queued. Either + * lookup rejects the value or deletion finds ctx. + */ + if (!bpf_map_value_is_linked(map, (void *)tw - map->record->task_work_off)) { + (void)cmpxchg(&ctx->state, BPF_TW_PENDING, BPF_TW_STANDBY); + bpf_task_work_ctx_put(ctx); + return ERR_PTR(-ENOENT); + } + /* * If no process or bpffs is holding a reference to the map, no new callbacks should be * scheduled. This does not address any race or correctness issue, but rather is a policy @@ -4842,7 +4864,12 @@ release_prog: * @callback: pointer to BPF subprogram to call * @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier * - * Return: 0 if task work has been scheduled successfully, negative error code otherwise + * A value reinserted at the same address is the current member. Success + * queues irq_work but does not guarantee callback delivery. + * + * Return: 0 if the scheduling irq_work was queued; -ENOENT if the membership + * lookup does not return this value; -EBUSY for a busy context or no user + * references; another negative error code on failure. */ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct bpf_task_work *tw, void *map__const_map, bpf_task_work_callback_t callback, @@ -4860,7 +4887,7 @@ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct b * @callback: pointer to BPF subprogram to call * @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier * - * Return: 0 if task work has been scheduled successfully, negative error code otherwise + * Return: As for bpf_task_work_schedule_signal(). */ __bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw, void *map__const_map, bpf_task_work_callback_t callback, -- 2.43.0