For some embedded systems, .text segments are often fixed. In situations of high memory pressure, these fixed segments may be reclaimed by the system, leading to iowait when these segments will be used again. The iowait problem becomes even more severe due to the following reasons: 1. The reclaimed code segments are often those that handle exceptional scenarios, which are not frequently executed. When memory pressure increases, the entire system can become sluggish, leading to execution of these seldom-used exception-handling code segments. Since these segments are more likely to be reclaimed from memory, this exacerbates system sluggishness. 2. The reclaimed code segments used for exception handling are often shared by multiple tasks, causing these tasks to wait on the folio's PG_locked bit, further increasing I/O wait. 3. Under memory pressure, the reclamation of code segments is often scattered and randomly distributed, slowing down the efficiency of block device reads and further exacerbating I/O wait. While this issue could be addressed by preloading a library mlock all executable segments, it would lead to many code segments that are never used being locked, resulting in memory waste. In systems where code execution is relatively fixed, preventing currently in-use code segments from being reclaimed makes sense. This acts as a self-adaptive way for the system to lock the necessary portions, which saves memory compared to locking all code segments with mlock. Introduce /proc/sys/vm/skipexec_enabled that can be set to 1 to enable this feature. When this feature is enabled, during memory reclamation logic, a flag TTU_SKIP_EXEC will be passed to try_to_unmap, allowing try_to_unmap_one to check if the vma has the VM_EXEC attribute when flag TTU_SKIP_EXEC is present. If the VM_EXEC attribute is set, it will skip the unmap operation. In the same scenario of locking a large file with vmtouch -l, our tests showed that without enabling the skipexec_enabled feature, the number of occurrences where iowait exceeded 20ms was 47,457, the longest iowait is 3 seconds. After enabling the skipexec_enabled feature, the number of occurrences dropped to only 34, the longest iowait is only 44ms, and none of these 34 instances were due to page cache file pages causing I/O wait. Signed-off-by: Xin Zhao --- include/linux/rmap.h | 1 + include/linux/writeback.h | 1 + mm/page-writeback.c | 14 ++++++++++++-- mm/rmap.c | 3 +++ mm/vmscan.c | 2 ++ 5 files changed, 19 insertions(+), 2 deletions(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index daa92a585..6a919f27e 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -101,6 +101,7 @@ enum ttu_flags { * do a final flush if necessary */ TTU_RMAP_LOCKED = 0x80, /* do not grab rmap lock: * caller holds it */ + TTU_SKIP_EXEC = 0x100,/* skip VM_MAYEXEC when unmap */ }; #ifdef CONFIG_MMU diff --git a/include/linux/writeback.h b/include/linux/writeback.h index f48e8ccff..16cf08028 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -343,6 +343,7 @@ extern struct wb_domain global_wb_domain; extern unsigned int dirty_writeback_interval; extern unsigned int dirty_expire_interval; extern int laptop_mode; +extern int skipexec_enabled; void global_dirty_limits(unsigned long *pbackground, unsigned long *pdirty); unsigned long wb_calc_thresh(struct bdi_writeback *wb, unsigned long thresh); diff --git a/mm/page-writeback.c b/mm/page-writeback.c index ccdeb0e84..e7c4a35ad 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -101,7 +101,6 @@ static unsigned long vm_dirty_bytes; * The interval between `kupdate'-style writebacks */ unsigned int dirty_writeback_interval = 5 * 100; /* centiseconds */ - EXPORT_SYMBOL_GPL(dirty_writeback_interval); /* @@ -114,9 +113,11 @@ unsigned int dirty_expire_interval = 30 * 100; /* centiseconds */ * a full sync is triggered after this time elapses without any disk activity. */ int laptop_mode; - EXPORT_SYMBOL(laptop_mode); +int skipexec_enabled; +EXPORT_SYMBOL(skipexec_enabled); + /* End of sysctl-exported parameters */ struct wb_domain global_wb_domain; @@ -2334,6 +2335,15 @@ static const struct ctl_table vm_page_writeback_sysctls[] = { .mode = 0644, .proc_handler = proc_dointvec_jiffies, }, + { + .procname = "skipexec_enabled", + .data = &skipexec_enabled, + .maxlen = sizeof(skipexec_enabled), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, + }, }; #endif diff --git a/mm/rmap.c b/mm/rmap.c index f955f02d5..5f528a03a 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1864,6 +1864,9 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, unsigned long hsz = 0; int ptes = 0; + if ((flags & TTU_SKIP_EXEC) && (vma->vm_flags & VM_EXEC)) + return false; + /* * When racing against e.g. zap_pte_range() on another cpu, * in between its ptep_get_and_clear_full() and folio_remove_rmap_*(), diff --git a/mm/vmscan.c b/mm/vmscan.c index 670fe9fae..c9ca65aa9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1350,6 +1350,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, if (folio_test_pmd_mappable(folio)) flags |= TTU_SPLIT_HUGE_PMD; + if (skipexec_enabled) + flags |= TTU_SKIP_EXEC; /* * Without TTU_SYNC, try_to_unmap will only begin to * hold PTL from the first present PTE within a large -- 2.34.1