The BPF verifier validates the alignment of PTR_TO_MAP_VALUE accesses in check_ptr_alignment() only against the offset relative to the start of the map value, assuming that the map value base address itself is at least 8-byte aligned. While hash maps enforce this invariant by rounding the key size up to 8 bytes (l->key + round_up(key_size, 8)), lpm_trie places the map value immediately after the key bytes at node->data + trie->data_size without padding. Because trie->data_size (key_size - 4) can be any value from 1 to 256, the value pointer returned by trie_lookup_elem() is only aligned to (trie->data_size % 8). In particular, for an IPv4 LPM trie (key_size = 8, data_size = 4), the map value base address is always 4-byte aligned (4 mod 8, and either 4 or 12 mod 16). The verifier accepts 8-byte atomic operations (BPF_ATOMIC | BPF_DW) at 8-byte aligned offsets within the value (e.g. offset 0 and offset 8), but at runtime on arm64 the JITed 64-bit atomic (stadd) faults with an unhandled hardware alignment fault (ESR.FSC = 0x21, crossing a 16-byte granule boundary on FEAT_LSE2 CPUs, or on any 4-byte aligned address without FEAT_LSE2) and panics the kernel: Unable to handle kernel paging request at virtual address 0xffff00000159db6c Mem abort info: ESR = 0x0000000096000021 EC = 0x25: DABT (current EL), IL = 32 bits FSC = 0x21: alignment fault Internal error: Oops: 0x96000021 [#1] SMP pc : bpf_prog_cc9155b98965fc5e+0x64/0x90 Call trace: bpf_prog_cc9155b98965fc5e+0x64/0x90 (P) bpf_dispatcher_xdp_func+0x14/0x28 bpf_test_run+0x11c/0x360 bpf_prog_test_run_xdp+0x358/0x548 Kernel panic - not syncing: Oops: Fatal exception in interrupt This issue was found while formally specifying eBPF map-value alignment invariants on Linux 7.3.5. Fix this by marking struct lpm_trie_node's data[] flexible array as __aligned(8) and rounding the value offset up to 8 bytes with round_up(trie->data_size, 8) in trie_alloc(), lpm_trie_node_alloc(), trie_lookup_elem(), and trie_mem_usage(), matching hashtab. Verified on Linux 7.3.5 (arm64 under QEMU) with an XDP program executing 64-bit atomic adds at offsets 0 and 8 of an IPv4 lpm_trie map value. Fixes: b95a5c4db09b ("bpf: add a longest prefix match trie map implementation") Assisted-by: LLM Signed-off-by: Tobias Philipp --- A standalone reproducer is available on request. kernel/bpf/lpm_trie.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 4ef9c8c9f3a2..9b4e38d16f21 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -27,7 +27,7 @@ struct lpm_trie_node { struct lpm_trie_node __rcu *child[2]; u32 prefixlen; u32 flags; - u8 data[]; + u8 data[] __aligned(8); }; struct lpm_trie { @@ -286,7 +286,7 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) if (!found) return NULL; - return found->data + trie->data_size; + return found->data + round_up(trie->data_size, 8); } static struct lpm_trie_node *lpm_trie_node_alloc(struct lpm_trie *trie, @@ -302,7 +302,7 @@ static struct lpm_trie_node *lpm_trie_node_alloc(struct lpm_trie *trie, node->flags = 0; if (value) - memcpy(node->data + trie->data_size, value, + memcpy(node->data + round_up(trie->data_size, 8), value, trie->map.value_size); return node; @@ -598,7 +598,8 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) raw_res_spin_lock_init(&trie->lock); /* Allocate intermediate and leaf nodes from the same allocator */ - leaf_size = sizeof(struct lpm_trie_node) + trie->data_size + + leaf_size = sizeof(struct lpm_trie_node) + + round_up(trie->data_size, 8) + trie->map.value_size; err = bpf_mem_alloc_init(&trie->ma, leaf_size, false); if (err) @@ -766,8 +767,9 @@ static u64 trie_mem_usage(const struct bpf_map *map) struct lpm_trie *trie = container_of(map, struct lpm_trie, map); u64 elem_size; - elem_size = sizeof(struct lpm_trie_node) + trie->data_size + - trie->map.value_size; + elem_size = sizeof(struct lpm_trie_node) + + round_up(trie->data_size, 8) + + trie->map.value_size; return elem_size * READ_ONCE(trie->n_entries); } -- 2.46.0