The zonelist_update_seq retry was limited to CONFIG_MEMORY_HOTREMOVE, presumably because only hot-remove takes zones out of a zonelist. However, build_zonerefs_node() rewrites the zonelists in place, so onlining the first block of a zone shifts the entries that follow it in every zonelist containing that zone. This can cause a spurious allocation failure for a __GFP_THISNODE caller. For a node with only ZONE_NORMAL, onlining its first ZONE_MOVABLE block rewrites the node's ZONELIST_NOFALLBACK list as: [NORMAL, NULL] -> [MOVABLE, NULL] -> [MOVABLE, NORMAL, NULL] A concurrent GFP_KERNEL | __GFP_THISNODE allocation can observe the intermediate list, skip ZONE_MOVABLE, and find no eligible zone. With CONFIG_MEMORY_HOTREMOVE=n, check_retry_zonelist() returns 0 and the allocation fails even though the node has free ZONE_NORMAL memory. Gate the sequence checks on CONFIG_MEMORY_HOTPLUG, which covers every configuration that rebuilds zonelists after boot. Fixes: 3d36424b3b58 ("mm/page_alloc: fix race condition between build_all_zonelists and page allocation") Signed-off-by: Gregory Price (Meta) --- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7682aecc2c075..81db19f2ab551 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4425,7 +4425,7 @@ static DEFINE_SEQLOCK(zonelist_update_seq); static unsigned int zonelist_iter_begin(void) { - if (IS_ENABLED(CONFIG_MEMORY_HOTREMOVE)) + if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG)) return read_seqbegin(&zonelist_update_seq); return 0; @@ -4433,7 +4433,7 @@ static unsigned int zonelist_iter_begin(void) static unsigned int check_retry_zonelist(unsigned int seq) { - if (IS_ENABLED(CONFIG_MEMORY_HOTREMOVE)) + if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG)) return read_seqretry(&zonelist_update_seq, seq); return seq; -- 2.53.0-Meta A cpuset update changes the task's mems_allowed and rebinds its mempolicy in separate steps under mems_allowed_seq, but the check reads both without it. This can cause a spurious numa placement miss under some cpuset/mempolicy combinations. A reader that loads one mask before the update and the other after it can see a spurious empty intersection and drop the nodemask. For example, a task with cpuset.mems={0} and MPOL_BIND={0}. If the cpuset moves to mems={1,2}, the policy is rebound to {1}. The allocator can race and compare the old mems_allowed {0} against the new policy {1} - which will result in a dropped nodemask and a misplaced allocation on node 2. Simply retry the intersection until mems_allowed_seq is stable. This covers the task policy, which is rebound under mems_allowed_seq. VMA policies are rebound afterwards by mpol_rebind_mm(), outside mems_allowed_seq, and are not covered by this change. Fixes: cc9a6c877661 ("cpuset: mm: reduce large amounts of memory barrier related damage v3") Signed-off-by: Gregory Price (Meta) --- kernel/cgroup/cpuset.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 3f52717c19654..8ee239d5c6b48 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -4354,10 +4354,21 @@ nodemask_t cpuset_mems_allowed(struct task_struct *tsk) * @nodemask: the nodemask to be checked * * Are any of the nodes in the nodemask allowed in current->mems_allowed? + * + * A cpuset update rebinds the task's mempolicy and changes mems_allowed in + * separate steps, so retry if an update raced with the check. */ int cpuset_nodemask_valid_mems_allowed(const nodemask_t *nodemask) { - return nodes_intersects(*nodemask, current->mems_allowed); + unsigned int seq; + int ret; + + do { + seq = read_mems_allowed_begin(); + ret = nodes_intersects(*nodemask, current->mems_allowed); + } while (read_mems_allowed_retry(seq)); + + return ret; } /* -- 2.53.0-Meta