Confine bpf_task_park() to the main body of the program and the functions it calls directly, at any depth, and refuse it in every callback frame: the synchronous callbacks of bpf_loop(), bpf_for_each_map_elem(), bpf_find_vma(), bpf_user_ringbuf_drain() and the graph helpers, the asynchronous callbacks of bpf_timer, bpf_wq, bpf_task_work and bpf_call_rcu(), and the exception callback. The park is sound where the only thing around the program is its runner, which holds the program, its maps and its context for the whole invocation. A callback has more around it. Kernel code called by the program, or a worker acting on its behalf, holds state across the callback that was not written with a sleep that drops the program's protection in mind, and each case would need its own audit: - bpf_for_each_map_elem() keeps an iteration position into the map across the callback. The verifier already refuses sleeping in the hash map variant; nothing would refuse it in the array variant. - A bpf_wq callback runs from a kworker, and bpf_wq_work() holds the program through nothing but a Tasks Trace RCU read section, the very section the park drops. Today the free path survives only because bpf_async_cb_rcu_tasks_trace_free() notices work_busy() and waits for another grace period, which a park of unbounded length turns into an endless chain of grace periods after the element is deleted, and the comment there assumes the callback stays inside the section. - A bpf_task_work callback also runs from exit_task_work() for an exiting task, after exit_signals() has set PF_EXITING; from then on wants_signal() refuses every signal for it, so a park without a timeout in that callback could not be interrupted. - The exception callback runs while the program is being unwound after bpf_throw(); nothing in it should block. Rather than audit and special-case each, the rule is that the park may only appear where the program is the only thing running: the main body, or a frame reached from it through plain calls. The check walks every frame of the current state and refuses the call if any is a callback frame. Asynchronous callbacks are verified as their own state with the callback as frame 0, so the walk sees them too. A static function that parks and is also used as a callback is rejected on the callback path, which is the right outcome: it cannot be made safe on that path. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/verifier.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e90cc127aedb..b42782c7f209 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -11222,6 +11222,31 @@ static void invalidate_non_owning_refs(struct bpf_verifier_env *env) })); } +/* + * bpf_task_park() is confined to the main body of the program and the + * functions it calls directly. A callback runs from kernel code that holds + * state of its own around the call, an iteration position, a work item or + * timer that owns the program only through Tasks Trace RCU, an exiting task, + * none of it audited against a sleep that drops the program's protection. + * Refuse the park in any callback frame, synchronous, asynchronous or + * exception. Asynchronous callbacks are verified as a state of their own + * with the callback as frame 0, so the walk over the frames sees them too. + */ +static bool in_callback_frames(struct bpf_verifier_env *env) +{ + struct bpf_verifier_state *vstate = env->cur_state; + int i; + + for (i = 0; i <= vstate->curframe; i++) { + struct bpf_func_state *frame = vstate->frame[i]; + + if (frame->in_callback_fn || frame->in_async_callback_fn || + frame->in_exception_callback_fn) + return true; + } + return false; +} + /* * bpf_task_park() sleeps without the Tasks Trace RCU read section and the * disabled migration that a sleepable program otherwise runs under. A dynptr @@ -16366,6 +16391,12 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, } if (is_bpf_task_park_kfunc(meta.func_id)) { + if (in_callback_frames(env)) { + verbose(env, "bpf_task_park() is not allowed in a callback\n"); + bpf_diag_policy(env, insn_idx, "kfunc bpf_task_park", "the call is in a callback", + "Call bpf_task_park() from the main program or from a function it calls directly, not from a callback."); + return -EINVAL; + } err = check_park_stack_objects(env, insn_idx); if (err) return err; -- 2.53.0