The kernel event that makes a parked task worth waking is usually not one a syscall program sees. A thread parked until memory runs short wants the wakeup of kswapd or the start of direct reclaim to end its sleep; one parked for a queue wants the enqueue, which may happen in a tracing program, a callback of a BPF scheduler or a networking program. The program on that event has the task, as an argument or as a kptr in a map it shares with the waiter. Register the wake for BPF_PROG_TYPE_UNSPEC, which makes it available to every program type, in a kfunc set of its own. It takes a referenced, trusted, or RCU-protected task, so a task kptr read under bpf_rcu_read_lock() works without a reference per wake. The park stays with syscall programs: the audit that confined it there concerned the runner, which has to hold the program across a sleep that drops its Tasks Trace RCU protection, and the wake sleeps nowhere and touches only the task and its pending word. What the wake does depend on is a context in which try_to_wake_up() may run. It takes the task's pi_lock and a runqueue lock, and a program can run in kernel code that holds either: a tracing program on sched_waking runs under the pi_lock and one on sched_switch under the rq lock, as do the callbacks of a BPF scheduler, while a perf event program can run in NMI context, where neither lock may be taken. All of these run with IRQs disabled, and a wake asked for with IRQs disabled is queued for irq_work instead of being issued. A program that runs with IRQs enabled holds neither lock, whatever its type, so no program type needs a rule of its own. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/task_park.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/task_park.c b/kernel/bpf/task_park.c index 0e3d7f3ec597..9de1b496ead1 100644 --- a/kernel/bpf/task_park.c +++ b/kernel/bpf/task_park.c @@ -281,13 +281,31 @@ BTF_KFUNCS_START(bpf_task_park_kfunc_ids) BTF_ID_FLAGS(func, bpf_task_park, KF_SLEEPABLE) BTF_KFUNCS_END(bpf_task_park_kfunc_ids) +BTF_KFUNCS_START(bpf_task_wake_kfunc_ids) +BTF_ID_FLAGS(func, bpf_task_wake, KF_RCU) +BTF_KFUNCS_END(bpf_task_wake_kfunc_ids) + static const struct btf_kfunc_id_set bpf_task_park_kfunc_set = { .owner = THIS_MODULE, .set = &bpf_task_park_kfunc_ids, }; +static const struct btf_kfunc_id_set bpf_task_wake_kfunc_set = { + .owner = THIS_MODULE, + .set = &bpf_task_wake_kfunc_ids, +}; + +/* + * The park stays with syscall programs, whose runner holds the program across + * the sleep. The wake touches only the task and defers itself where it cannot + * be issued, so every program type has it, to wake a parked task from the + * kernel event that makes it worth waking. + */ static int __init bpf_task_park_kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_task_park_kfunc_set); + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_task_park_kfunc_set); + return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC, &bpf_task_wake_kfunc_set); } late_initcall(bpf_task_park_kfunc_init); -- 2.53.0