From: Yuan Chen rhtab_delete_elem() and rhtab_map_update_existing() eagerly call bpf_obj_free_fields() when an element is deleted or its value is replaced. This runs kptr destructors (and unpins uptrs, frees list/rbtree roots) in the caller's execution context, which is unsafe for BPF programs running in NMI context (e.g. perf_event programs attached to hardware PMU overflows): referenced kptr destructors may take locks or otherwise cannot run in NMI. Commit a3a81d247651 ("bpf: Cancel special fields on map value recycle") switched the hash map and array recycle paths to bpf_obj_cancel_fields(), which only cancels NMI-safe fields (timer, workqueue, task_work), but it missed the resizable hashtab. rhtab_map_update_existing() even documents the intended "cancel" semantics while still calling bpf_obj_free_fields(). Fix by cancelling only NMI-safe fields on rhtab update/delete and leaving referenced kptrs (and other fields requiring full destruction) attached to the recycled element, to be destroyed by rhtab_mem_dtor() when the element is eventually freed. This matches the hash map semantics and keeps the element's kptr reference accounting balanced: the reference stays owned by the element and is released exactly once by the final destruction path. Verified with a selftest (perf_event NMI program overwriting a rhtab element that holds a referenced task kptr): before this patch the NMI update releases the kptr and the subsequent probe observes NULL in the slot; after the patch the kptr is inherited and the probe observes it non-NULL. Fixes: a3a81d247651 ("bpf: Cancel special fields on map value recycle") Signed-off-by: Yuan Chen --- kernel/bpf/hashtab.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 9f394e1aa2e8..70fa1e1b4b00 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -2865,14 +2865,23 @@ static int rhtab_map_alloc_check(union bpf_attr *attr) return htab_map_alloc_check(attr); } -static void rhtab_check_and_free_fields(struct bpf_rhtab *rhtab, - struct rhtab_elem *elem) +static void rhtab_cancel_fields(struct bpf_rhtab *rhtab, + struct rhtab_elem *elem) { if (IS_ERR_OR_NULL(rhtab->map.record)) return; - bpf_obj_free_fields(rhtab->map.record, - rhtab_elem_value(elem, rhtab->map.key_size)); + /* + * Only cancel NMI-safe fields (timer, workqueue, task_work) here. + * kptr/uptr/list/rbtree destruction must not run from arbitrary BPF + * execution contexts (e.g. NMI), so leave those fields attached to + * the recycled element and let rhtab_mem_dtor() destroy them once the + * element is eventually freed. This matches the hash map semantics + * introduced by a3a81d247651 ("bpf: Cancel special fields on map value + * recycle"). + */ + bpf_map_free_internal_structs(&rhtab->map, + rhtab_elem_value(elem, rhtab->map.key_size)); } static void rhtab_mem_dtor(void *obj, void *ctx) @@ -2964,8 +2973,8 @@ static int rhtab_delete_elem(struct bpf_rhtab *rhtab, struct rhtab_elem *elem, v rhtab_read_elem_value(&rhtab->map, copy, elem, flags); check_and_init_map_value(&rhtab->map, copy); } - /* Release internal structs: kptr, bpf_timer, task_work, wq */ - rhtab_check_and_free_fields(rhtab, elem); + /* Cancel NMI-safe fields; full destruction happens in rhtab_mem_dtor */ + rhtab_cancel_fields(rhtab, elem); bpf_mem_cache_free_rcu(&rhtab->ma, elem); return 0; } @@ -3024,10 +3033,11 @@ static long rhtab_map_update_existing(struct bpf_map *map, struct rhtab_elem *el * BPF_F_LOCK, matching arraymap semantics. * * copy_map_value() skips special-field offsets, so old timers/ - * kptrs/etc. still sit in the slot. Cancel them after the copy - * to match arraymap's update semantics. + * kptrs/etc. still sit in the slot. Cancel the NMI-safe ones after + * the copy to match arraymap's update semantics; referenced kptrs + * stay attached and are destroyed by rhtab_mem_dtor(). */ - rhtab_check_and_free_fields(rhtab, elem); + rhtab_cancel_fields(rhtab, elem); return 0; } -- 2.54.0