decay_va_pool_node() walks every per-pool free-list entry under
vmap_purge_lock and merges each vmap_area into a global RB-tree via
reclaim_list_global() without yielding. The outer loop has no
rescheduling point, so when many vmap areas are queued the function
can monopolize the CPU long enough to trigger an RCU self-detected
stall:
rcu: INFO: rcu_preempt self-detected stall on CPU
rcu: 2-...0: (6344 ticks this GP) idle=853c/1/0x4000000000000000 softirq=41536/41536 fqs=3211
rcu: (t=6528 jiffies g=37549 q=4652 ncpus=4)
CPU: 2 UID: 0 PID: 1516 Comm: syz.5.318 Not tainted 7.0.0-rc7 #4 PREEMPT(full)
Call Trace:
finish_task_switch.isra.0+0x23e/0x990 kernel/sched/core.c:5155
context_switch kernel/sched/core.c:5301 [inline]
__schedule+0xb3d/0x3680 kernel/sched/core.c:6911
preempt_schedule_common+0x44/0xc0 kernel/sched/core.c:7095
preempt_schedule_thunk+0x16/0x30 arch/x86/entry/thunk.S:12
__raw_spin_unlock include/linux/spinlock_api_smp.h:169 [inline]
_raw_spin_unlock+0x43/0x50 kernel/locking/spinlock.c:186
reclaim_list_global mm/vmalloc.c:2213 [inline]
decay_va_pool_node+0xccf/0x1070 mm/vmalloc.c:2273
__purge_vmap_area_lazy+0x136/0xc80 mm/vmalloc.c:2361
_vm_unmap_aliases+0x469/0x6e0 mm/vmalloc.c:2996
change_page_attr_set_clr+0x24d/0x4a0 arch/x86/mm/pat/set_memory.c:2082
set_memory_rox+0xc2/0x110 arch/x86/mm/pat/set_memory.c:2314
create_trampoline arch/x86/kernel/ftrace.c:421 [inline]
arch_ftrace_update_trampoline+0x79d/0xb50 arch/x86/kernel/ftrace.c:479
ftrace_update_trampoline+0x45/0x360 kernel/trace/ftrace.c:8391
__register_ftrace_function+0x238/0x340 kernel/trace/ftrace.c:365
ftrace_startup+0x3b/0x370 kernel/trace/ftrace.c:3098
register_ftrace_function_nolock+0x5e/0x160 kernel/trace/ftrace.c:9162
register_ftrace_function+0x32b/0x4c0 kernel/trace/ftrace.c:9189
perf_ftrace_function_register kernel/trace/trace_event_perf.c:494 [inline]
perf_ftrace_event_register+0x159/0x240 kernel/trace/trace_event_perf.c:518
perf_trace_event_open kernel/trace/trace_event_perf.c:184 [inline]
perf_trace_event_init kernel/trace/trace_event_perf.c:206 [inline]
perf_trace_event_init+0x17b/0xad0 kernel/trace/trace_event_perf.c:193
perf_trace_init+0x176/0x290 kernel/trace/trace_event_perf.c:226
perf_tp_event_init+0xa6/0x120 kernel/events/core.c:11270
perf_try_init_event+0x103/0x930 kernel/events/core.c:13029
perf_init_event kernel/events/core.c:13127 [inline]
perf_event_alloc.part.0+0x11dd/0x4970 kernel/events/core.c:13402
perf_event_alloc kernel/events/core.c:13283 [inline]
__do_sys_perf_event_open+0x764/0x2eb0 kernel/events/core.c:13924
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0xa9/0x580 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x76/0x7e
Add cond_resched() at the bottom of the outer loop in
decay_va_pool_node(). At that point the per-pool spinlock has already
been released and the outer vmap_purge_lock is a mutex, so sleeping
is safe.
Found by Syzkaller.
Fixes: 72210662c5a2 ("mm: vmalloc: offload free_vmap_area_lock lock")
Signed-off-by: Sechang Lim
---
mm/vmalloc.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 61caa55a4402..78e064a9c4c7 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -2268,6 +2268,8 @@ decay_va_pool_node(struct vmap_node *vn, bool full_decay)
WRITE_ONCE(vn->pool[i].len, pool_len);
spin_unlock(&vn->pool_lock);
}
+
+ cond_resched();
}
reclaim_list_global(&decay_list);
--
2.43.0