From: Hui Zhu lru_gen_change_state() migrates folios in batches of up to MAX_LRU_BATCH (4096) folios under lruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the same lru_lock spin with interrupts disabled for that long. The caller already reschedules between batches, but a batch itself only ends after 4096 folios. Stop it early when a resched is pending, or when the lock has waiters and the current hold has moved at least MIN_LRU_BATCH folios; the caller restarts where it left off. TIF_NEED_RESCHED can still be set with interrupts off: a remote resched_curr() sets it directly in the target's thread_info. The minimum batch avoids lock bouncing under sustained contention, while a waiter now waits ~15us instead of ~1ms. This follows lru_gen_look_around() in this file, security/keys/gc.c and jbd2_log_do_checkpoint(), and commit 6e9be217a3ce ("mm/mglru: use a smaller batch for reclaim"). Signed-off-by: Hui Zhu --- Changlog: v2: Also yield on lock contention after a minimum batch (Kairui Song). mm/vmscan.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 6a1af9b92349..c1231925159f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -5533,7 +5533,10 @@ static bool fill_evictable(struct lruvec *lruvec) success = lru_gen_add_folio(lruvec, folio, !active); VM_WARN_ON_ONCE(!success); - if (!--remaining) + /* stop early to shorten the irq-off section */ + if (!--remaining || need_resched() || + (MAX_LRU_BATCH - remaining >= MIN_LRU_BATCH && + spin_is_contended(&lruvec->lru_lock))) return false; } } @@ -5562,7 +5565,10 @@ static bool drain_evictable(struct lruvec *lruvec) VM_WARN_ON_ONCE(!success); lruvec_add_folio(lruvec, folio); - if (!--remaining) + /* stop early to shorten the irq-off section */ + if (!--remaining || need_resched() || + (MAX_LRU_BATCH - remaining >= MIN_LRU_BATCH && + spin_is_contended(&lruvec->lru_lock))) return false; } } -- 2.53.0