From: Yuan Chen When a module kfunc declares an implicit struct bpf_prog_aux * argument, the verifier must identify it so the kernel injects env->prog->aux into the correct register at runtime. The original check used is_kfunc_arg_prog_aux() which calls btf_types_are_same() to compare the module BTF type against vmlinux. Root Cause ---------- This issue was triggered by pahole 1.30 generating module BTF with incorrect type information, which caused the kernel's distilled base BTF deduplication for modules to fail. As a result, the module retained its own copy of struct bpf_prog_aux with a different BTF ID than vmlinux's definition. While pahole 1.31 fixed the BTF generation issue, the kernel must be robust against such inconsistencies: a BTF mismatch should result in a clean rejection, not a kernel crash or information disclosure. When the distilled base dedup fails and btf_types_are_same() cannot match the module's bpf_prog_aux type against vmlinux's, is_kfunc_arg_prog_aux() returned false and the code fell through silently without setting arg_prog. The kfunc then received whatever value was in the argument register and dereferenced it as a bpf_prog_aux pointer, leading to: BUG: kernel invalid pointer dereference, address: 00000000000009e2 RIP: bpf_prog_get_assoc_struct_ops+0xa/0xc0 RDI: 0x000000000000046d (stale register value) In the observed crash the stale value was the process PID, causing a dereference within the unmapped NULL page. However, an attacker able to control the register value -- for example by writing a BPF program that explicitly sets R2 before calling a KF_IMPLICIT_ARGS kfunc -- could redirect the dereference to arbitrary kernel memory, turning this into an information disclosure. The fix ensures the verifier either validates and injects the correct bpf_prog_aux pointer, or rejects the program outright -- no silent fallthrough that could be exploited. Crash Stack Trace ----------------- PID: 1133 TASK: ffff8881057d3900 CPU: 3 COMMAND: "test_progs" #0 machine_kexec at ffffffff812f6e26 #1 __crash_kexec at ffffffff8145a788 #2 crash_kexec at ffffffff8145ac24 #3 oops_end at ffffffff812bb67c #4 page_fault_oops at ffffffff813053a1 #5 exc_page_fault at ffffffff828e60a1 #6 asm_exc_page_fault at ffffffff810012a6 [exception RIP: bpf_prog_get_assoc_struct_ops+10] RIP: ffffffff815c024a RSP: ffffc90001b57e48 RFLAGS: 00010283 RAX: ffff8881057d3900 RBX: ffffc90001b57e68 RCX: ffff8881057d3900 RDX: 0000607d4d1768b8 RSI: 000000000000046d RDI: 000000000000046d #7 bpf_kfunc_multi_st_ops_test_1_assoc at ffffffffc0013a85 [bpf_testmod] #8 bpf_trace_run2 at ffffffff814f8332 #9 __traceiter_sys_enter at ffffffff81415f45 #10 trace_syscall_enter at ffffffff81416735 #11 do_syscall_64 at ffffffff828e06a1 Fix --- Introduce a two-layer argument-injection detection: 1. get_kfunc_arg_inject_type() -- lightweight name-based classification of injectable types (currently only KF_INJECT_ARG_PROG_AUX). This ensures we recognize injection candidates regardless of BTF type IDs. 2. is_kfunc_arg_prog_aux() -- strict type validation within the inject case; if validation fails the program is rejected with -EINVAL instead of silently bypassing injection setup. This design ensures that BTF inconsistencies result in a clean verification failure instead of a crash or a potential information disclosure, and the approach is extensible for future injection types. Fixes: 64e1360524b9 ("bpf: Verifier support for KF_IMPLICIT_ARGS") Signed-off-by: Yuan Chen --- kernel/bpf/verifier.c | 48 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 8dd79b735a69..928b6c42a4bf 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10857,6 +10857,39 @@ static bool is_kfunc_arg_prog_aux(const struct btf *btf, const struct btf_param return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_PROG_AUX_ID); } +/* + * Injectable argument types are implicit kfunc arguments whose value is + * injected by the kernel at call time rather than received from the BPF + * program. Use name-based matching for initial detection to avoid false + * negatives when a module's BTF references the type via a different BTF ID + * than vmlinux's. Actual type compatibility is still validated by the + * caller with btf_types_are_same(). + */ +enum kfunc_inject_arg_type { + KF_INJECT_ARG_NONE = 0, + KF_INJECT_ARG_PROG_AUX, +}; + +static enum kfunc_inject_arg_type get_kfunc_arg_inject_type( + const struct btf *btf, const struct btf_param *arg) +{ + const struct btf_type *t; + u32 res_id; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!t || !btf_type_is_ptr(t)) + return KF_INJECT_ARG_NONE; + + t = btf_type_skip_modifiers(btf, t->type, &res_id); + if (!t) + return KF_INJECT_ARG_NONE; + + if (strcmp(btf_type_name(btf, res_id), "bpf_prog_aux") == 0) + return KF_INJECT_ARG_PROG_AUX; + + return KF_INJECT_ARG_NONE; +} + /* * A kfunc with KF_IMPLICIT_ARGS has two prototypes in BTF: * - the _impl prototype with full arg list (meta->func_proto) @@ -11899,8 +11932,17 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ u32 ref_id, type_size; bool is_ret_buf_sz = false; int kf_arg_type; - - if (is_kfunc_arg_prog_aux(btf, &args[i])) { + enum kfunc_inject_arg_type inject_type; + + inject_type = get_kfunc_arg_inject_type(btf, &args[i]); + switch (inject_type) { + case KF_INJECT_ARG_PROG_AUX: + /* Validate the arg type against vmlinux's definition */ + if (!is_kfunc_arg_prog_aux(btf, &args[i])) { + verbose(env, "arg#%d implicit argument type mismatch, " + "expected struct bpf_prog_aux *\n", i); + return -EINVAL; + } /* Reject repeated use bpf_prog_aux */ if (meta->arg_prog) { verifier_bug(env, "Only 1 prog->aux argument supported per-kfunc"); @@ -11914,6 +11956,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ meta->arg_prog = true; cur_aux(env)->arg_prog = regno; continue; + default: + break; } if (is_kfunc_arg_ignore(btf, &args[i]) || is_kfunc_arg_implicit(meta, i)) -- 2.54.0