| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/08/11 00:58 | flow | patch-triage |
0mArgs:null Results: {
"EnableConfigs": [
"BPF_SYSCALL"
],
"FocusSymbols": [
"bpf_get_kfunc_ret_size",
"__btf_type_is_scalar_struct",
"bpf_jit_supports_kfunc_ret_reg_pair",
"btf_distill_func_proto",
"btf_prepare_func_args"
],
"KMSANReasoning": "The patch series introduces support for BPF kfuncs and subprograms to return values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. The changes are primarily within the BPF verifier (to track and validate the R2 register) and the JIT compilers (to properly move the return values into the expected BPF registers). \n\nFrom an uninitialized memory perspective:\n1. The BPF verifier statically enforces that BPF subprograms fully initialize both R0 and R2 before returning (via `check_reg_arg(..., SRC_OP)`).\n2. If a static subprogram leaks a stack pointer in R2, the verifier safely marks it as uninitialized (`bpf_mark_reg_not_init`), preventing the caller from reading it. This is a static verifier check, not a dynamic kernel memory issue.\n3. The patch does not introduce any new kernel-to-user data structures, complex allocations, or bounds checking logic that could expose uninitialized kernel memory.\n4. The only new kfuncs introduced are for testing purposes (`bpf_testmod`), and their return structures are fully initialized (e.g., using designated initializers which zero-fill padding).\n\nSince the risk of uninitialized memory usage or info-leaks is statically mitigated by the verifier and no new uninitialized kernel memory paths are introduced, a dedicated KMSAN fuzzing session is not required. Standard KASAN and BPF verifier tests are sufficient.",
"NeedsKMSAN": false,
"Reasoning": "The patch introduces support for BPF kfuncs returning values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. This involves significant changes to the BPF verifier (register tracking, liveness, backtracking, BTF validation) and JIT compilers (x86_64, arm64, riscv64). These are core functional changes to the BPF subsystem that are reachable from userspace and should be fuzzed.",
"WorthFuzzing": true
} |
| 1/1 | 2026/08/11 00:58 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit e58dc654dc59f196a5aa28b5e90faf612f8b4f83\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Tue Aug 11 00:58:42 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst\nindex 1004eb0bec617..71d3ff8695994 100644\n--- a/Documentation/bpf/kfuncs.rst\n+++ b/Documentation/bpf/kfuncs.rst\n@@ -575,6 +575,68 @@ is also covered by this recovery. A kfunc handed an arena pointer may\n therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking\n against the arena. Larger accesses must verify the range explicitly.\n \n+2.9 kfunc Return Values\n+-----------------------\n+\n+A kfunc may return a scalar, a pointer, or a small struct or union by\n+value. A scalar or pointer of up to 8 bytes is returned in R0, as usual.\n+\n+A struct or union returned by value must be composed only of scalars\n+(recursively), where a scalar is an integer or an enum; arrays of scalars are\n+allowed as members. Its bytes are handed back to the program as the raw\n+contents of R0 (and R2), so a pointer field would be laundered into a scalar\n+and escape the verifier's pointer provenance and reference tracking. A struct\n+or union with a pointer member is therefore rejected at load time, and so is\n+one with a floating-point member, which the ABI may not return in R0:R2 at\n+all.\n+\n+A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a\n+scalar-only struct or union, or an ``__int128``. Such a value is returned\n+in the register pair R0:R2, matching the convention LLVM uses for the BPF\n+target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or\n+union of 8 bytes or less is returned in R0 alone.\n+\n+::\n+\n+ struct bpf_pair { __u64 a, b; }; /* 16 bytes */\n+\n+ __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void)\n+ {\n+ struct bpf_pair p = { .a = 1, .b = 2 };\n+\n+ return p; /* p.a in R0, p.b in R2 */\n+ }\n+\n+Returning a value in the R0:R2 pair requires the JIT to place the second\n+half of the return value into R2, which not every architecture supports\n+right now. A kfunc with a return value larger than 8 bytes is therefore\n+rejected at load time on a JIT that does not advertise this capability (see\n+``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run\n+by the interpreter. A return value larger than 16 bytes is not supported.\n+\n+The same R0:R2 convention applies to a BPF subprogram, global or static,\n+that returns an ``__int128`` or a struct or union larger than 8 bytes. Such a\n+program also requires the JIT, since the interpreter propagates only R0 out\n+of a subprogram. A global subprogram is verified in isolation, so its\n+by-value struct or union return is restricted to scalars just like a kfunc's;\n+a static subprogram is verified inline and has no such restriction. The main\n+program cannot return more than 8 bytes, as its return value is the program's\n+exit code.\n+\n+A global subprogram must leave a scalar in *every* register of the pair, so\n+both halves of the returned value have to be assigned. Leaving the upper half\n+uninitialized is not merely untidy: the compiler is then free to leave R2\n+holding whatever it happened to hold, which for a subprogram taking a pointer\n+argument is typically that pointer. Handing the caller an unknown scalar built\n+from a pointer is a leak, so the verifier rejects it with::\n+\n+ At subprogram exit the register R2 is not a scalar value (...)\n+\n+Initialize the whole return value, for example ``struct pair p = {};``, to\n+avoid this. A static subprogram is exempt: it is verified inline, so an\n+unassigned R2 is simply passed back to the caller as uninitialized and only a\n+caller that reads it fails.\n+\n .. _BPF_kfunc_lifecycle_expectations:\n \n 3. kfunc lifecycle expectations\ndiff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c\nindex d14d297ebb967..a1febb4c5718c 100644\n--- a/arch/arm64/net/bpf_jit_comp.c\n+++ b/arch/arm64/net/bpf_jit_comp.c\n@@ -2330,6 +2330,11 @@ bool bpf_jit_supports_kfunc_call(void)\n \treturn true;\n }\n \n+bool bpf_jit_supports_kfunc_ret_reg_pair(void)\n+{\n+\treturn true;\n+}\n+\n bool bpf_jit_supports_stack_args(void)\n {\n \treturn true;\ndiff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c\nindex 6b9972b07c1b0..c8e94d4657e9b 100644\n--- a/arch/riscv/net/bpf_jit_comp64.c\n+++ b/arch/riscv/net/bpf_jit_comp64.c\n@@ -2111,6 +2111,11 @@ bool bpf_jit_supports_kfunc_call(void)\n \treturn true;\n }\n \n+bool bpf_jit_supports_kfunc_ret_reg_pair(void)\n+{\n+\treturn true;\n+}\n+\n bool bpf_jit_supports_ptr_xchg(void)\n {\n \treturn true;\ndiff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c\nindex 8dddb5d7af21b..01c34114c8502 100644\n--- a/arch/x86/net/bpf_jit_comp.c\n+++ b/arch/x86/net/bpf_jit_comp.c\n@@ -2647,6 +2647,22 @@ st:\t\t\tinsn_off = insn-\u003eoff;\n \t\t\t\treturn -EINVAL;\n \t\t\tif (priv_frame_ptr)\n \t\t\t\tpop_r9(\u0026prog);\n+\t\t\tif (src_reg == BPF_PSEUDO_KFUNC_CALL) {\n+\t\t\t\tconst struct btf_func_model *fm;\n+\n+\t\t\t\t/*\n+\t\t\t\t * A kfunc returning a \u003e8 byte aggregate hands the\n+\t\t\t\t * second half back in RDX (the native ABI's second\n+\t\t\t\t * return reg), but BPF expects it in R0:R2. BPF R0\n+\t\t\t\t * is RAX (no move needed), while BPF R2 is RSI, so\n+\t\t\t\t * copy RDX into RSI.\n+\t\t\t\t */\n+\t\t\t\tfm = bpf_jit_find_kfunc_model(bpf_prog, insn);\n+\t\t\t\tif (!fm)\n+\t\t\t\t\treturn -EFAULT;\n+\t\t\t\tif (fm-\u003eret_size \u003e 8)\n+\t\t\t\t\temit_mov_reg(\u0026prog, true, BPF_REG_2, BPF_REG_3);\n+\t\t\t}\n \t\t\tbreak;\n \t\t}\n \n@@ -4137,6 +4153,11 @@ bool bpf_jit_supports_kfunc_call(void)\n \treturn true;\n }\n \n+bool bpf_jit_supports_kfunc_ret_reg_pair(void)\n+{\n+\treturn true;\n+}\n+\n bool bpf_jit_supports_stack_args(void)\n {\n \treturn true;\ndiff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h\nindex 93f7c2075eeaa..1f516daa41dbf 100644\n--- a/include/linux/bpf_verifier.h\n+++ b/include/linux/bpf_verifier.h\n@@ -811,6 +811,8 @@ struct bpf_subprog_info {\n \tbool is_async_cb: 1;\n \tbool is_exception_cb: 1;\n \tbool args_cached: 1;\n+\t/* true if the return value is passed in the R0:R2 register pair */\n+\tbool ret_reg_pair: 1;\n \t/* true if bpf_fastcall stack region is used by functions that can't be inlined */\n \tbool keep_fastcall_stack: 1;\n \tbool changes_pkt_data: 1;\n@@ -1044,6 +1046,16 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env\n \treturn \u0026env-\u003esubprog_info[subprog];\n }\n \n+/*\n+ * True if @subprog returns its value in the R0:R2 register pair. Cached by\n+ * bpf_compute_subprog_ret_regs(), since this is queried on hot paths: at\n+ * every subprogram call and at every subprogram exit.\n+ */\n+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)\n+{\n+\treturn subprog_info(env, subprog)-\u003eret_reg_pair;\n+}\n+\n struct bpf_call_summary {\n \tu8 num_params;\n \tbool is_void;\n@@ -1435,6 +1447,10 @@ int bpf_jmp_offset(struct bpf_insn *insn);\n struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);\n void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);\n bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);\n+int bpf_get_kfunc_ret_size(const struct bpf_prog *prog, u32 func_id,\n+\t\t\t u16 btf_fd_idx, u8 *ret_size);\n+bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,\n+\t\t\t\t const struct btf_type *t, int rec);\n \n int bpf_find_subprog(struct bpf_verifier_env *env, int off);\n bool bpf_is_throw_kfunc(struct bpf_insn *insn);\ndiff --git a/include/linux/filter.h b/include/linux/filter.h\nindex 4edba8182db1b..f3c34fd70f2d2 100644\n--- a/include/linux/filter.h\n+++ b/include/linux/filter.h\n@@ -1213,6 +1213,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);\n bool bpf_jit_supports_subprog_tailcalls(void);\n bool bpf_jit_supports_percpu_insn(void);\n bool bpf_jit_supports_kfunc_call(void);\n+bool bpf_jit_supports_kfunc_ret_reg_pair(void);\n bool bpf_jit_supports_stack_args(void);\n bool bpf_jit_supports_arena_args(void);\n bool bpf_jit_supports_far_kfunc_call(void);\ndiff --git a/kernel/bpf/backtrack.c b/kernel/bpf/backtrack.c\nindex 40bd04421a991..fc8ecad6f01b0 100644\n--- a/kernel/bpf/backtrack.c\n+++ b/kernel/bpf/backtrack.c\n@@ -425,6 +425,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,\n \t\t\t\t */\n \t\t\t\tverifier_bug_if(idx + 1 != subseq_idx, env,\n \t\t\t\t\t\t\"extra insn from subprog\");\n+\t\t\t\t/*\n+\t\t\t\t * a global subprog returning more than 8 bytes\n+\t\t\t\t * sets R2 as well. R2 is part of the args mask\n+\t\t\t\t * checked just below, so it has to be cleared\n+\t\t\t\t * here rather than next to R0.\n+\t\t\t\t */\n+\t\t\t\tif (bt_is_reg_set(bt, BPF_REG_2) \u0026\u0026\n+\t\t\t\t bpf_ret_reg_pair(env, subprog))\n+\t\t\t\t\tbt_clear_reg(bt, BPF_REG_2);\n \t\t\t\t/* r1-r5 are invalidated after subprog call,\n \t\t\t\t * so for global func call it shouldn't be set\n \t\t\t\t * anymore\n@@ -508,6 +517,19 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,\n \t\t\t\treturn -ENOTSUPP;\n \t\t\t/* regular helper call sets R0 */\n \t\t\tbt_clear_reg(bt, BPF_REG_0);\n+\t\t\t/* a kfunc returning more than 8 bytes also sets R2 */\n+\t\t\tif (insn-\u003esrc_reg == BPF_PSEUDO_KFUNC_CALL \u0026\u0026\n+\t\t\t bt_is_reg_set(bt, BPF_REG_2)) {\n+\t\t\t\tu8 ret_size;\n+\t\t\t\tint err;\n+\n+\t\t\t\terr = bpf_get_kfunc_ret_size(env-\u003eprog, insn-\u003eimm, insn-\u003eoff,\n+\t\t\t\t\t\t\t \u0026ret_size);\n+\t\t\t\tif (verifier_bug_if(err, env, \"no kfunc desc for insn %d\", idx))\n+\t\t\t\t\treturn -EFAULT;\n+\t\t\t\tif (ret_size \u003e 8)\n+\t\t\t\t\tbt_clear_reg(bt, BPF_REG_2);\n+\t\t\t}\n \t\t\tif (bt_reg_mask(bt) \u0026 BPF_REGMASK_ARGS) {\n \t\t\t\t/* if backtracking was looking for registers R1-R5\n \t\t\t\t * they should have been found already.\n@@ -522,7 +544,30 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,\n \t\t\t\t\treturn -EFAULT;\n \t\t\t}\n \t\t} else if (opcode == BPF_EXIT) {\n-\t\t\tbool r0_precise;\n+\t\t\tbool from_subprog_call, r0_precise, r2_precise = false;\n+\n+\t\t\t/*\n+\t\t\t * BPF_EXIT in subprog or callback always returns\n+\t\t\t * right after the call instruction, so by checking\n+\t\t\t * whether the instruction at subseq_idx-1 is subprog\n+\t\t\t * call or not we can distinguish actual exit from\n+\t\t\t * *subprog* from exit from *callback*. In the former\n+\t\t\t * case, we need to propagate the precision of the\n+\t\t\t * return registers, if necessary. In the latter we\n+\t\t\t * never do that.\n+\t\t\t */\n+\t\t\tfrom_subprog_call = subseq_idx - 1 \u003e= 0 \u0026\u0026\n+\t\t\t\t\t bpf_pseudo_call(\u0026env-\u003eprog-\u003einsnsi[subseq_idx - 1]);\n+\t\t\tif (from_subprog_call \u0026\u0026 bt_is_reg_set(bt, BPF_REG_2)) {\n+\t\t\t\tstruct bpf_subprog_info *callee;\n+\n+\t\t\t\t/* 'idx' is the exit insn, so it is in the callee */\n+\t\t\t\tcallee = bpf_find_containing_subprog(env, idx);\n+\t\t\t\tif (verifier_bug_if(!callee, env,\n+\t\t\t\t\t\t \"no subprog contains exit insn %d\", idx))\n+\t\t\t\t\treturn -EFAULT;\n+\t\t\t\tr2_precise = bpf_ret_reg_pair(env, callee - env-\u003esubprog_info);\n+\t\t\t}\n \n \t\t\t/* Backtracking to a nested function call, 'idx' is a part of\n \t\t\t * the inner frame 'subseq_idx' is a part of the outer frame.\n@@ -535,23 +580,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,\n \t\t\tif (subseq_idx \u003e= 0 \u0026\u0026 bpf_calls_callback(env, subseq_idx))\n \t\t\t\tfor (i = BPF_REG_1; i \u003c= BPF_REG_5; i++)\n \t\t\t\t\tbt_clear_reg(bt, i);\n+\t\t\tif (r2_precise)\n+\t\t\t\tbt_clear_reg(bt, BPF_REG_2);\n \t\t\tif (bt_reg_mask(bt) \u0026 BPF_REGMASK_ARGS) {\n \t\t\t\tverifier_bug(env, \"backtracking exit unexpected regs %x\",\n \t\t\t\t\t bt_reg_mask(bt));\n \t\t\t\treturn -EFAULT;\n \t\t\t}\n \n-\t\t\t/* BPF_EXIT in subprog or callback always returns\n-\t\t\t * right after the call instruction, so by checking\n-\t\t\t * whether the instruction at subseq_idx-1 is subprog\n-\t\t\t * call or not we can distinguish actual exit from\n-\t\t\t * *subprog* from exit from *callback*. In the former\n-\t\t\t * case, we need to propagate r0 precision, if\n-\t\t\t * necessary. In the former we never do that.\n-\t\t\t */\n-\t\t\tr0_precise = subseq_idx - 1 \u003e= 0 \u0026\u0026\n-\t\t\t\t bpf_pseudo_call(\u0026env-\u003eprog-\u003einsnsi[subseq_idx - 1]) \u0026\u0026\n-\t\t\t\t bt_is_reg_set(bt, BPF_REG_0);\n+\t\t\tr0_precise = from_subprog_call \u0026\u0026 bt_is_reg_set(bt, BPF_REG_0);\n \n \t\t\tbt_clear_reg(bt, BPF_REG_0);\n \t\t\tif (bt_subprog_enter(bt))\n@@ -559,6 +596,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,\n \n \t\t\tif (r0_precise)\n \t\t\t\tbt_set_reg(bt, BPF_REG_0);\n+\t\t\tif (r2_precise)\n+\t\t\t\tbt_set_reg(bt, BPF_REG_2);\n \t\t\t/* r6-r9 and stack slots will stay set in caller frame\n \t\t\t * bitmasks until we return back from callee(s)\n \t\t\t */\ndiff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c\nindex 6606187ed4f43..5551abcea1d39 100644\n--- a/kernel/bpf/btf.c\n+++ b/kernel/bpf/btf.c\n@@ -7592,7 +7592,12 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,\n \t\treturn -EINVAL;\n \t}\n \tret = __get_type_size(btf, func-\u003etype, \u0026t);\n-\tif (ret \u003c 0 || btf_type_is_struct(t)) {\n+\t/*\n+\t * __get_type_size() already restricts a non-negative ret to void, a\n+\t * pointer, an int, an enum or a struct/union, so only the size is checked\n+\t * here.\n+\t */\n+\tif (ret \u003c 0 || ret \u003e 16) {\n \t\tbpf_log(log,\n \t\t\t\"The function %s return type %s is unsupported.\\n\",\n \t\t\ttname, btf_type_str(t));\n@@ -7965,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,\n \n /* Check whether the type is a valid return type. */\n static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,\n-\t\tconst struct btf_type *t, int subprog)\n+\t\tconst struct btf_type *t, int subprog, bool is_global)\n {\n \tu32 tags = 0;\n \tint err;\n@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt\n \tif (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))\n \t\treturn 0;\n \n+\tif (btf_type_is_struct(t) \u0026\u0026 t-\u003esize \u003c= 16) {\n+\t\t/*\n+\t\t * A \u003e8 byte struct/union is returned in the R0:R2 register pair.\n+\t\t * A global function is verified in isolation, so its caller models\n+\t\t * the return as an opaque R0:R2 scalar pair; it must therefore\n+\t\t * contain only scalars, otherwise a pointer field would be\n+\t\t * laundered into a scalar and escape provenance and reference\n+\t\t * tracking. That requirement is enforced here: do_check_common()\n+\t\t * propagates the error for global functions and for the main\n+\t\t * program.\n+\t\t *\n+\t\t * A local (static) function is verified inline and its R0:R2 are\n+\t\t * copied as precise register state (with the JIT forced on when\n+\t\t * the pair is consumed), so a pointer field stays tracked and needs\n+\t\t * no such restriction. Accepting it here is not by itself what\n+\t\t * makes it legal: btf_check_subprog_call() drops any error other\n+\t\t * than -EFAULT. What it avoids is needlessly marking the\n+\t\t * subprogram's BTF unreliable.\n+\t\t *\n+\t\t * The main program (subprog 0) takes the scalar-only path as well,\n+\t\t * but its return value is the program's exit code, so a \u003e8 byte\n+\t\t * return is rejected separately at BPF_EXIT.\n+\t\t */\n+\t\tbool local_func = subprog \u0026\u0026 !is_global;\n+\n+\t\tif (local_func || __btf_type_is_scalar_struct(env, btf, t, 0))\n+\t\t\treturn 0;\n+\t}\n+\n \treturn -EOPNOTSUPP;\n }\n \n@@ -8075,12 +8109,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)\n \t\treturn -EINVAL;\n \t}\n \n-\terr = btf_validate_return_type(env, btf, t, subprog);\n+\terr = btf_validate_return_type(env, btf, t, subprog, is_global);\n \tif (err) {\n \t\tif (is_global) {\n \t\t\tbpf_log(log,\n-\t\t\t\t\"Global function %s() return value not void or scalar. \"\n-\t\t\t\t\"Only those are supported.\\n\",\n+\t\t\t\t\"Global function %s() has unsupported return type. \"\n+\t\t\t\t\"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\\n\",\n \t\t\t\ttname);\n \t\t}\n \t\treturn err;\ndiff --git a/kernel/bpf/core.c b/kernel/bpf/core.c\nindex a3e1fae32eace..d98f4220e875e 100644\n--- a/kernel/bpf/core.c\n+++ b/kernel/bpf/core.c\n@@ -3303,6 +3303,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)\n \treturn false;\n }\n \n+bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)\n+{\n+\treturn false;\n+}\n+\n bool __weak bpf_jit_supports_stack_args(void)\n {\n \treturn false;\ndiff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c\nindex ef9a5a9228872..451edd74fa6f5 100644\n--- a/kernel/bpf/liveness.c\n+++ b/kernel/bpf/liveness.c\n@@ -2062,10 +2062,15 @@ static inline u32 mask_widen(u32 m) { return m | (m \u003c\u003c 16); }\n static inline u16 mask_lo(u32 m) { return (u16)m; }\n static inline u16 mask_hi(u32 m) { return (u16)(m \u003e\u003e 16); }\n \n-/* Compute info-\u003e{use,def} fields for the instruction */\n+/*\n+ * Compute info-\u003e{use,def} fields for the instruction. @ret_reg_pair tells\n+ * whether the subprogram containing @insn returns its value in the R0:R2\n+ * register pair, which matters for BPF_EXIT.\n+ */\n static void compute_insn_live_regs(struct bpf_verifier_env *env,\n \t\t\t\t struct bpf_insn *insn,\n-\t\t\t\t struct insn_live_regs *info)\n+\t\t\t\t struct insn_live_regs *info,\n+\t\t\t\t bool ret_reg_pair)\n {\n \tstruct bpf_call_summary cs;\n \tconst u8 class = BPF_CLASS(insn-\u003ecode);\n@@ -2196,7 +2201,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,\n \t\t\tbreak;\n \t\tcase BPF_EXIT:\n \t\t\tdef = 0;\n-\t\t\tuse = r0;\n+\t\t\tuse = ret_reg_pair ? (r0 | reg64_mask(BPF_REG_2)) : r0;\n \t\t\tbreak;\n \t\tcase BPF_CALL:\n \t\t\tdef = ALL_CALLER_SAVED_REGS;\n@@ -2233,8 +2238,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)\n \tstruct insn_live_regs *state;\n \tint insn_cnt = env-\u003eprog-\u003elen;\n \tu64 pos, insn_pos;\n-\tint err = 0, i, j;\n-\tbool changed;\n+\tint err = 0, i, j, subprog, start, end;\n+\tbool changed, ret_reg_pair;\n \n \t/* Use the following algorithm:\n \t * - define the following:\n@@ -2261,8 +2266,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)\n \t\tgoto out;\n \t}\n \n-\tfor (i = 0; i \u003c insn_cnt; ++i)\n-\t\tcompute_insn_live_regs(env, \u0026insns[i], \u0026state[i]);\n+\tfor (subprog = 0; subprog \u003c env-\u003esubprog_cnt; subprog++) {\n+\t\tstart = env-\u003esubprog_info[subprog].start;\n+\t\tend = env-\u003esubprog_info[subprog + 1].start;\n+\t\tret_reg_pair = bpf_ret_reg_pair(env, subprog);\n+\n+\t\tfor (i = start; i \u003c end; ++i)\n+\t\t\tcompute_insn_live_regs(env, \u0026insns[i], \u0026state[i], ret_reg_pair);\n+\t}\n \n \t/* Forward pass: resolve stack access through FP-derived pointers */\n \terr = bpf_compute_subprog_arg_access(env);\ndiff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c\nindex add3affc57035..f8294359c85df 100644\n--- a/kernel/bpf/verifier.c\n+++ b/kernel/bpf/verifier.c\n@@ -382,27 +382,75 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)\n \treturn aux \u0026\u0026 aux[subprog].linkage == BTF_FUNC_GLOBAL;\n }\n \n-static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)\n+/* Return type of a subprogram, NULL if it cannot be resolved */\n+static const struct btf_type *subprog_ret_type(struct bpf_verifier_env *env, int subprog)\n {\n-\tconst struct btf_type *type, *func, *func_proto;\n+\tconst struct btf_type *func, *func_proto;\n \tconst struct btf *btf = env-\u003eprog-\u003eaux-\u003ebtf;\n \tu32 btf_id;\n \n+\tif (!btf || !env-\u003eprog-\u003eaux-\u003efunc_info)\n+\t\treturn NULL;\n+\n \tbtf_id = env-\u003eprog-\u003eaux-\u003efunc_info[subprog].type_id;\n \n+\t/* Both already validated by prepare_btf_func() at prog load. */\n \tfunc = btf_type_by_id(btf, btf_id);\n-\tif (verifier_bug_if(!func, env, \"btf_id %u not found\", btf_id))\n-\t\treturn false;\n-\n \tfunc_proto = btf_type_by_id(btf, func-\u003etype);\n-\tif (!func_proto)\n-\t\treturn false;\n \n-\ttype = btf_type_skip_modifiers(btf, func_proto-\u003etype, NULL);\n-\tif (!type)\n-\t\treturn false;\n+\treturn btf_type_skip_modifiers(btf, func_proto-\u003etype, NULL);\n+}\n+\n+static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)\n+{\n+\tconst struct btf_type *type = subprog_ret_type(env, subprog);\n \n-\treturn btf_type_is_void(type);\n+\treturn type \u0026\u0026 btf_type_is_void(type);\n+}\n+\n+/*\n+ * Number of registers holding a function return value: a value of up to 8\n+ * bytes is returned in R0, a value of more than 8 bytes and no more than 16\n+ * bytes (an __int128 or a struct/union of such size) is returned in the R0:R2\n+ * register pair, with R2 holding the upper half.\n+ */\n+static u32 ret_regs_cnt(u32 size)\n+{\n+\treturn size \u003e 8 \u0026\u0026 size \u003c= 16 ? 2 : 1;\n+}\n+\n+/* Registers holding a function return value, in order. See ret_regs_cnt(). */\n+static const int ret_regs[] = { BPF_REG_0, BPF_REG_2 };\n+\n+/*\n+ * Resolve the return convention of every subprogram once, so that\n+ * bpf_ret_reg_pair() is a plain flag test on the hot paths that use it.\n+ */\n+static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)\n+{\n+\tconst struct btf_type *type;\n+\tint subprog;\n+\n+\tfor (subprog = 0; subprog \u003c env-\u003esubprog_cnt; subprog++) {\n+\t\ttype = subprog_ret_type(env, subprog);\n+\t\tif (type \u0026\u0026 (btf_type_is_struct(type) || btf_type_is_scalar(type)))\n+\t\t\tsubprog_info(env, subprog)-\u003eret_reg_pair = ret_regs_cnt(type-\u003esize) \u003e 1;\n+\t}\n+}\n+\n+/*\n+ * A \u003e8 byte BPF return changes the calling convention to R0:R2, so the\n+ * verifier can only allow it while the subprogram's prototype remains\n+ * reliable. Once BTF is marked unreliable, reject the feature instead of\n+ * silently falling back to R0-only semantics.\n+ */\n+static bool subprog_ret_pair_unreliable(struct bpf_verifier_env *env, int subprog)\n+{\n+\tstruct bpf_prog_aux *aux = env-\u003eprog-\u003eaux;\n+\n+\treturn bpf_ret_reg_pair(env, subprog) \u0026\u0026\n+\t aux-\u003efunc_info_aux \u0026\u0026\n+\t aux-\u003efunc_info_aux[subprog].unreliable;\n }\n \n static const char *subprog_name(const struct bpf_verifier_env *env, int subprog)\n@@ -2470,6 +2518,19 @@ int bpf_get_kfunc_addr(const struct bpf_prog *prog, u32 func_id,\n \treturn 0;\n }\n \n+int bpf_get_kfunc_ret_size(const struct bpf_prog *prog, u32 func_id,\n+\t\t\t u16 btf_fd_idx, u8 *ret_size)\n+{\n+\tconst struct bpf_kfunc_desc *desc;\n+\n+\tdesc = find_kfunc_desc(prog, func_id, btf_fd_idx);\n+\tif (!desc)\n+\t\treturn -EFAULT;\n+\n+\t*ret_size = desc-\u003efunc_model.ret_size;\n+\treturn 0;\n+}\n+\n #define BPF_FD_SLOT_BTF\t1UL\n \n static void fd_slot_set_map(struct bpf_fd_array *slot, struct bpf_map *map)\n@@ -2807,6 +2868,19 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)\n \terr = btf_distill_func_proto(\u0026env-\u003elog, kfunc.btf, kfunc.proto, kfunc.name, \u0026func_model);\n \tif (err)\n \t\treturn err;\n+\tif (func_model.ret_size \u003e 8) {\n+\t\tif (kfunc.flags \u0026\u0026 (*kfunc.flags \u0026 KF_FASTCALL)) {\n+\t\t\tverbose(env,\n+\t\t\t\t\"kfunc %s with \u003e8-byte return is not supported with KF_FASTCALL\\n\",\n+\t\t\t\tkfunc.name);\n+\t\t\treturn -EOPNOTSUPP;\n+\t\t}\n+\t\tif (!bpf_jit_supports_kfunc_ret_reg_pair()) {\n+\t\t\tverbose(env, \"kfunc %s with \u003e8-byte return is not supported by JIT\\n\",\n+\t\t\t\tkfunc.name);\n+\t\t\treturn -EOPNOTSUPP;\n+\t\t}\n+\t}\n \n \tmemset(\u0026meta, 0, sizeof(meta));\n \tmeta.btf = kfunc.btf;\n@@ -9388,6 +9462,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n \tu16 callee_incoming, stack_arg_cnt;\n \tstruct bpf_func_state *caller;\n \tint err, subprog, target_insn;\n+\tu32 i, nregs;\n \n \ttarget_insn = *insn_idx + insn-\u003eimm + 1;\n \tsubprog = bpf_find_subprog(env, target_insn);\n@@ -9399,6 +9474,11 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n \terr = btf_check_subprog_call(env, subprog, caller-\u003eregs);\n \tif (err == -EFAULT)\n \t\treturn err;\n+\tif (subprog_ret_pair_unreliable(env, subprog)) {\n+\t\tverbose(env, \"Func#%d ('%s') returns \u003e8 bytes, which requires reliable BTF\\n\",\n+\t\t\tsubprog, subprog_name(env, subprog));\n+\t\treturn -EINVAL;\n+\t}\n \tif (bpf_subprog_is_global(env, subprog)) {\n \t\tconst char *sub_name = subprog_name(env, subprog);\n \n@@ -9430,9 +9510,22 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n \t\tclear_caller_saved_regs(env, caller-\u003eregs);\n \t\tinvalidate_outgoing_stack_args(env, cur_func(env));\n \n-\t\t/* All non-void global functions return a 64-bit SCALAR_VALUE. */\n+\t\t/*\n+\t\t * A non-void global function returns a 64-bit SCALAR_VALUE in\n+\t\t * R0, or a \u003e8 byte SCALAR_VALUE in the R0:R2 register pair.\n+\t\t */\n \t\tif (!subprog_returns_void(env, subprog)) {\n-\t\t\tmark_reg_unknown(env, caller-\u003eregs, BPF_REG_0);\n+\t\t\tnregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;\n+\t\t\t/*\n+\t\t\t * The R0:R2 return convention is only implemented in the\n+\t\t\t * JIT: the interpreter propagates BPF_R0 alone out of a\n+\t\t\t * subprogram, so a caller reading R2 would see a stale\n+\t\t\t * value. Force the JIT once a caller can observe the pair.\n+\t\t\t */\n+\t\t\tif (nregs \u003e 1)\n+\t\t\t\tenv-\u003eprog-\u003ejit_required = 1;\n+\t\t\tfor (i = 0; i \u003c nregs; i++)\n+\t\t\t\tmark_reg_unknown(env, caller-\u003eregs, ret_regs[i]);\n \t\t}\n \n \t\tif (env-\u003esubprog_info[subprog].might_throw) {\n@@ -9754,10 +9847,19 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)\n \tstruct bpf_func_state *caller, *callee;\n \tstruct bpf_reg_state *r0;\n \tbool in_callback_fn;\n+\tu32 i, nregs;\n \tint err;\n \n \tcallee = state-\u003eframe[state-\u003ecurframe];\n \tr0 = \u0026callee-\u003eregs[BPF_REG_0];\n+\tif (subprog_ret_pair_unreliable(env, callee-\u003esubprogno)) {\n+\t\tverbose(env, \"Func#%d ('%s') returns \u003e8 bytes, which requires reliable BTF\\n\",\n+\t\t\tcallee-\u003esubprogno, subprog_name(env, callee-\u003esubprogno));\n+\t\treturn -EINVAL;\n+\t}\n+\tnregs = bpf_ret_reg_pair(env, callee-\u003esubprogno) ? 2 : 1;\n+\tif (nregs \u003e 1)\n+\t\tenv-\u003eprog-\u003ejit_required = 1;\n \tif (r0-\u003etype == PTR_TO_STACK) {\n \t\t/* technically it's ok to return caller's stack pointer\n \t\t * (or caller's caller's pointer) back to the caller,\n@@ -9793,8 +9895,23 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)\n \t\t\treturn -EFAULT;\n \t\t}\n \t} else {\n-\t\t/* return to the caller whatever r0 had in the callee */\n-\t\tcaller-\u003eregs[BPF_REG_0] = *r0;\n+\t\t/*\n+\t\t * return to the caller whatever the callee had in the\n+\t\t * return register(s)\n+\t\t */\n+\t\tfor (i = 0; i \u003c nregs; i++)\n+\t\t\tcaller-\u003eregs[ret_regs[i]] = callee-\u003eregs[ret_regs[i]];\n+\n+\t\t/*\n+\t\t * R2 carries only the upper half of a register pair return\n+\t\t * value. A stack pointer must not escape the callee (see the\n+\t\t * R0 case above), but there is no need to reject the whole\n+\t\t * program for it: hand the caller an uninitialized R2 instead,\n+\t\t * so that only a caller actually using the returned pointer\n+\t\t * fails.\n+\t\t */\n+\t\tif (nregs \u003e 1 \u0026\u0026 caller-\u003eregs[BPF_REG_2].type == PTR_TO_STACK)\n+\t\t\tbpf_mark_reg_not_init(env, \u0026caller-\u003eregs[BPF_REG_2]);\n \t}\n \n \t/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,\n@@ -10691,6 +10808,19 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn\n \treturn 0;\n }\n \n+/*\n+ * Mark the register(s) holding a @size byte kfunc return value as unknown\n+ * scalars. Both halves of a register pair are treated the same way.\n+ */\n+static void mark_kfunc_ret_regs(struct bpf_verifier_env *env,\n+\t\t\t\tstruct bpf_reg_state *regs, u32 size)\n+{\n+\tu32 i, nregs = ret_regs_cnt(size);\n+\n+\tfor (i = 0; i \u003c nregs; i++)\n+\t\tmark_reg_unknown(env, regs, ret_regs[i]);\n+}\n+\n static bool is_kfunc_acquire(struct bpf_call_arg_meta *meta)\n {\n \treturn meta-\u003ekfunc_flags \u0026 KF_ACQUIRE;\n@@ -10958,9 +11088,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_\n }\n \n /* Returns true if struct is composed of scalars, 4 levels of nesting allowed */\n-static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,\n-\t\t\t\t\tconst struct btf *btf,\n-\t\t\t\t\tconst struct btf_type *t, int rec)\n+bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,\n+\t\t\t\t const struct btf *btf,\n+\t\t\t\t const struct btf_type *t, int rec)\n {\n \tconst struct btf_type *member_type;\n \tconst struct btf_member *member;\n@@ -13167,10 +13297,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n \t}\n \n \tif (btf_type_is_scalar(t)) {\n-\t\tmark_reg_unknown(env, regs, BPF_REG_0);\n+\t\tmark_kfunc_ret_regs(env, regs, t-\u003esize);\n \t\tif (meta.btf == btf_vmlinux \u0026\u0026 (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||\n \t\t meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))\n \t\t\t__mark_reg_const_zero(env, \u0026regs[BPF_REG_0]);\n+\t} else if (btf_type_is_struct(t)) {\n+\t\t/*\n+\t\t * The returned struct comes back as raw register bits modeled\n+\t\t * as an unknown scalar, so it must contain only scalars:\n+\t\t * otherwise a pointer field would be laundered into a scalar\n+\t\t * and escape provenance and reference tracking.\n+\t\t */\n+\t\tif (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {\n+\t\t\tverbose(env,\n+\t\t\t\t\"kernel function %s returns %s %s that is not composed of scalars\\n\",\n+\t\t\t\tfunc_name, btf_type_str(t),\n+\t\t\t\tbtf_name_by_offset(desc_btf, t-\u003ename_off));\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tmark_kfunc_ret_regs(env, regs, t-\u003esize);\n \t} else if (btf_type_is_ptr(t)) {\n \t\tptr_type = btf_type_skip_modifiers(desc_btf, t-\u003etype, \u0026ptr_type_id);\n \t\terr = check_special_kfunc(env, \u0026meta, regs, insn_aux, ptr_type, desc_btf);\n@@ -16250,6 +16395,11 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)\n \t\t\tverbose(env, \"callback function not static\\n\");\n \t\t\treturn -EINVAL;\n \t\t}\n+\t\tif (bpf_ret_reg_pair(env, subprogno)) {\n+\t\t\tverbose(env,\n+\t\t\t\t\"callback function with \u003e8-byte return value is not supported\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n \n \t\tdst_reg-\u003etype = PTR_TO_FUNC;\n \t\tdst_reg-\u003esubprogno = subprogno;\n@@ -16617,37 +16767,61 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char\n \treturn 0;\n }\n \n-static int check_global_subprog_return_code(struct bpf_verifier_env *env)\n+static int check_global_ret_scalar_reg(struct bpf_verifier_env *env, u32 regno,\n+\t\t\t\t bool allow_arena_ptr_return)\n {\n-\tstruct bpf_reg_state *reg = reg_state(env, BPF_REG_0);\n-\tstruct bpf_func_state *cur_frame = cur_func(env);\n+\tstruct bpf_reg_state *reg;\n \tint err;\n \n-\tif (subprog_returns_void(env, cur_frame-\u003esubprogno))\n-\t\treturn 0;\n-\n-\terr = check_reg_arg(env, BPF_REG_0, SRC_OP);\n+\terr = check_reg_arg(env, regno, SRC_OP);\n \tif (err)\n \t\treturn err;\n \n \t/* Pointers to arena are safe to pass between subprograms. */\n-\tif (is_arena_reg(env, BPF_REG_0))\n+\tif (allow_arena_ptr_return \u0026\u0026 is_arena_reg(env, regno))\n \t\treturn 0;\n \n-\tif (is_pointer_value(env, BPF_REG_0)) {\n-\t\tverbose(env, \"R%d leaks addr as return value\\n\", BPF_REG_0);\n+\tif (is_pointer_value(env, regno)) {\n+\t\tverbose(env, \"R%d leaks addr as return value\\n\", regno);\n \t\treturn -EACCES;\n \t}\n \n+\treg = reg_state(env, regno);\n \tif (reg-\u003etype != SCALAR_VALUE) {\n-\t\tverbose(env, \"At subprogram exit the register R0 is not a scalar value (%s)\\n\",\n-\t\t\treg_type_str(env, reg-\u003etype));\n+\t\tverbose(env, \"At subprogram exit the register R%d is not a scalar value (%s)\\n\",\n+\t\t\tregno, reg_type_str(env, reg-\u003etype));\n \t\treturn -EINVAL;\n \t}\n \n \treturn 0;\n }\n \n+static int check_global_subprog_return_code(struct bpf_verifier_env *env)\n+{\n+\tstruct bpf_func_state *cur_frame = cur_func(env);\n+\tu32 subprog = cur_frame-\u003esubprogno;\n+\tu32 i, nregs;\n+\tint err;\n+\n+\tif (subprog_returns_void(env, subprog))\n+\t\treturn 0;\n+\n+\t/*\n+\t * An arena pointer is only a legitimate return value when it is the\n+\t * whole of it, that is when it is returned in R0 alone. Both halves of\n+\t * a register pair carry a piece of a \u003e8 byte scalar, so an arena\n+\t * pointer in either of them is a leak.\n+\t */\n+\tnregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;\n+\tfor (i = 0; i \u003c nregs; i++) {\n+\t\terr = check_global_ret_scalar_reg(env, ret_regs[i], nregs == 1);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+\n /* Bitmask with 1s for all caller saved registers */\n #define ALL_CALLER_SAVED_REGS ((1u \u003c\u003c CALLER_SAVED_REGS) - 1)\n \n@@ -17134,10 +17308,16 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,\n \t */\n \tif (cur_frame-\u003esubprogno \u0026\u0026\n \t !cur_frame-\u003ein_async_callback_fn \u0026\u0026\n-\t !cur_frame-\u003ein_exception_callback_fn)\n+\t !cur_frame-\u003ein_exception_callback_fn) {\n \t\terr = check_global_subprog_return_code(env);\n-\telse\n+\t} else {\n+\t\tif (!cur_frame-\u003esubprogno \u0026\u0026 bpf_ret_reg_pair(env, 0)) {\n+\t\t\tverbose(env,\n+\t\t\t\t\"return value larger than 8 bytes is not supported at program exit\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n \t\terr = check_return_code(env, BPF_REG_0, \"R0\");\n+\t}\n \tif (err)\n \t\treturn err;\n \treturn PROCESS_BPF_EXIT;\n@@ -18464,6 +18644,12 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)\n \t\t\t\tret = -EINVAL;\n \t\t\t\tgoto out;\n \t\t\t}\n+\t\t\tif (bpf_ret_reg_pair(env, subprog)) {\n+\t\t\t\tverbose(env,\n+\t\t\t\t\t\"exception cb cannot return value larger than 8 bytes\\n\");\n+\t\t\t\tret = -EINVAL;\n+\t\t\t\tgoto out;\n+\t\t\t}\n \n \t\t\t/* Also ensure the callback only has a single scalar argument. */\n \t\t\tif (sub-\u003earg_cnt != 1 || sub-\u003eargs[0].arg_type != ARG_ANYTHING) {\n@@ -19291,6 +19477,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,\n \t\t\treturn -EOPNOTSUPP;\n \t\t}\n \n+\t\t/*\n+\t\t * An extension replaces the target outright, so it has to match\n+\t\t * the target's return convention. Its own return value is capped\n+\t\t * at 8 bytes (a \u003e8 byte program return is rejected at BPF_EXIT),\n+\t\t * so it can never fill the R0:R2 pair the target's callers read.\n+\t\t * This cannot be left to btf_check_type_match() above, which\n+\t\t * compares return types by btf_type-\u003einfo only: an int carries no\n+\t\t * vlen, so a 16-byte __int128 and an 8-byte long compare equal.\n+\t\t */\n+\t\tif (prog_extension \u0026\u0026 tgt_info-\u003efmodel.ret_size \u003e 8) {\n+\t\t\tbpf_log(log,\n+\t\t\t\t\"Cannot replace function %s with a \u003e8 byte return value\\n\",\n+\t\t\t\ttname);\n+\t\t\treturn -EOPNOTSUPP;\n+\t\t}\n+\n \t\t/*\n \t\t * *.multi programs don't need an address during program\n \t\t * verification, we just take the module ref if needed.\n@@ -20280,6 +20482,9 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n \tif (ret \u003c 0)\n \t\tgoto skip_full_check;\n \n+\t/* must precede the first bpf_ret_reg_pair() user below */\n+\tbpf_compute_subprog_ret_regs(env);\n+\n \tret = bpf_compute_live_registers(env);\n \tif (ret \u003c 0)\n \t\tgoto skip_full_check;\ndiff --git a/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c b/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c\nnew file mode 100644\nindex 0000000000000..8be64d01ff4db\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c\n@@ -0,0 +1,206 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003ctest_progs.h\u003e\n+#include \"aggregate_ret_int128_c.skel.h\"\n+#include \"aggregate_ret_struct_c.skel.h\"\n+#include \"aggregate_ret_union_c.skel.h\"\n+#include \"aggregate_ret_kfunc_c.skel.h\"\n+#include \"aggregate_ret_run.skel.h\"\n+#include \"aggregate_ret_func.skel.h\"\n+#include \"aggregate_ret_kfunc.skel.h\"\n+\n+/*\n+ * The bpf_testmod kfuncs returning more than 8 bytes are only built on x86_64\n+ * and arm64 (see bpf_testmod.c); everywhere else the tests calling them are\n+ * skipped.\n+ */\n+static bool has_ret_pair_kfuncs(void)\n+{\n+#if defined(__x86_64__) || defined(__aarch64__)\n+\treturn true;\n+#else\n+\treturn false;\n+#endif\n+}\n+\n+static void run_prog(struct bpf_program *prog)\n+{\n+\tchar buf[64] = {};\n+\tint err, prog_fd;\n+\tLIBBPF_OPTS(bpf_test_run_opts, topts,\n+\t\t .data_in = buf,\n+\t\t .data_size_in = sizeof(buf),\n+\t\t .repeat = 1,\n+\t);\n+\n+\tprog_fd = bpf_program__fd(prog);\n+\terr = bpf_prog_test_run_opts(prog_fd, \u0026topts);\n+\tif (!ASSERT_OK(err, \"test_run\"))\n+\t\treturn;\n+\n+\tASSERT_EQ(topts.retval, 0, \"aggregate_ret_result\");\n+}\n+\n+/*\n+ * Run @prog as subtest @name. Where the register-pair return is unsupported\n+ * the subtest reports a skip instead, so that the list of subtests does not\n+ * depend on the compiler or on the architecture; @prog is unused then and may\n+ * be NULL, for a caller that could not even open its object.\n+ */\n+static void run_subtest(const char *name, struct bpf_program *prog, bool supported)\n+{\n+\tif (!test__start_subtest(name))\n+\t\treturn;\n+\n+\tif (!supported) {\n+\t\ttest__skip();\n+\t\treturn;\n+\t}\n+\n+\trun_prog(prog);\n+}\n+\n+static void test_int128_c(void)\n+{\n+\tstruct aggregate_ret_int128_c *skel;\n+\n+\tskel = aggregate_ret_int128_c__open_and_load();\n+\tif (!ASSERT_OK_PTR(skel, \"skel_int128_c_open_load\"))\n+\t\treturn;\n+\n+\trun_subtest(\"int128_c\", skel-\u003eprogs.aggregate_ret_int128_c_test,\n+\t\t skel-\u003erodata-\u003ehas_reg_pair_ret);\n+\n+\taggregate_ret_int128_c__destroy(skel);\n+}\n+\n+static void test_struct_c(void)\n+{\n+\tstruct aggregate_ret_struct_c *skel;\n+\n+\tskel = aggregate_ret_struct_c__open_and_load();\n+\tif (!ASSERT_OK_PTR(skel, \"skel_struct_c_open_load\"))\n+\t\treturn;\n+\n+\trun_subtest(\"struct_c\", skel-\u003eprogs.aggregate_ret_struct_c_test,\n+\t\t skel-\u003erodata-\u003ehas_reg_pair_ret);\n+\n+\trun_subtest(\"global_struct_c\", skel-\u003eprogs.aggregate_ret_global_struct_c_test,\n+\t\t skel-\u003erodata-\u003ehas_reg_pair_ret);\n+\n+\taggregate_ret_struct_c__destroy(skel);\n+}\n+\n+static void test_union_c(void)\n+{\n+\tstruct aggregate_ret_union_c *skel;\n+\n+\tskel = aggregate_ret_union_c__open_and_load();\n+\tif (!ASSERT_OK_PTR(skel, \"skel_union_c_open_load\"))\n+\t\treturn;\n+\n+\trun_subtest(\"union_c\", skel-\u003eprogs.aggregate_ret_union_c_test,\n+\t\t skel-\u003erodata-\u003ehas_reg_pair_ret);\n+\n+\taggregate_ret_union_c__destroy(skel);\n+}\n+\n+static void test_kfunc_c(void)\n+{\n+\tstruct aggregate_ret_kfunc_c *skel;\n+\tbool supported;\n+\tint err;\n+\n+\tskel = aggregate_ret_kfunc_c__open();\n+\tif (!ASSERT_OK_PTR(skel, \"skel_kfunc_c_open\"))\n+\t\treturn;\n+\n+\tsupported = skel-\u003erodata-\u003ehas_reg_pair_ret \u0026\u0026 has_ret_pair_kfuncs();\n+\n+\tif (supported) {\n+\t\t/*\n+\t\t * Where the JIT cannot hand the second half of a \u003e8-byte kfunc\n+\t\t * return back in R0:R2, bpf_add_kfunc_call() rejects the call\n+\t\t * with -EOPNOTSUPP. Asking the kernel keeps this test free of a\n+\t\t * list of the JITs that can, which would have to be updated as\n+\t\t * the rest of them learn.\n+\t\t */\n+\t\terr = aggregate_ret_kfunc_c__load(skel);\n+\t\tif (err == -EOPNOTSUPP)\n+\t\t\tsupported = false;\n+\t\telse if (!ASSERT_OK(err, \"skel_kfunc_c_load\"))\n+\t\t\tgoto out;\n+\t}\n+\n+\trun_subtest(\"kfunc_int128_c\", skel-\u003eprogs.aggregate_ret_kfunc_int128_c_test,\n+\t\t supported);\n+\n+\trun_subtest(\"kfunc_struct_c\", skel-\u003eprogs.aggregate_ret_kfunc_struct_c_test,\n+\t\t supported);\n+\n+out:\n+\taggregate_ret_kfunc_c__destroy(skel);\n+}\n+\n+static void test_run(void)\n+{\n+\tstruct aggregate_ret_run *skel;\n+\tbool kfunc_ok = true;\n+\tint err;\n+\n+\t/*\n+\t * Every program in this object shares __kfunc_btf_root(), so where the\n+\t * testmod kfuncs are absent the object cannot load at all -- including\n+\t * for the kfunc-free \"asm\" subtest.\n+\t */\n+\tif (!has_ret_pair_kfuncs()) {\n+\t\trun_subtest(\"asm\", NULL, false);\n+\t\trun_subtest(\"asm_kfunc\", NULL, false);\n+\t\trun_subtest(\"struct\", NULL, false);\n+\t\trun_subtest(\"union\", NULL, false);\n+\t\treturn;\n+\t}\n+\n+\tskel = aggregate_ret_run__open();\n+\tif (!ASSERT_OK_PTR(skel, \"skel_run_open\"))\n+\t\treturn;\n+\n+\terr = aggregate_ret_run__load(skel);\n+\tif (err == -EOPNOTSUPP) {\n+\t\tkfunc_ok = false;\n+\t\taggregate_ret_run__destroy(skel);\n+\n+\t\tskel = aggregate_ret_run__open();\n+\t\tif (!ASSERT_OK_PTR(skel, \"skel_run_reopen\"))\n+\t\t\treturn;\n+\n+\t\tbpf_program__set_autoload(skel-\u003eprogs.aggregate_ret_asm_kfunc_test, false);\n+\t\tbpf_program__set_autoload(skel-\u003eprogs.aggregate_ret_struct_test, false);\n+\t\tbpf_program__set_autoload(skel-\u003eprogs.aggregate_ret_union_test, false);\n+\n+\t\terr = aggregate_ret_run__load(skel);\n+\t}\n+\tif (!ASSERT_OK(err, \"skel_run_load\"))\n+\t\tgoto out;\n+\n+\trun_subtest(\"asm\", skel-\u003eprogs.aggregate_ret_asm_test, true);\n+\trun_subtest(\"asm_kfunc\", skel-\u003eprogs.aggregate_ret_asm_kfunc_test, kfunc_ok);\n+\trun_subtest(\"struct\", skel-\u003eprogs.aggregate_ret_struct_test, kfunc_ok);\n+\trun_subtest(\"union\", skel-\u003eprogs.aggregate_ret_union_test, kfunc_ok);\n+\n+out:\n+\taggregate_ret_run__destroy(skel);\n+}\n+\n+void test_aggregate_ret(void)\n+{\n+\ttest_int128_c();\n+\ttest_struct_c();\n+\ttest_union_c();\n+\ttest_kfunc_c();\n+\ttest_run();\n+\n+\tRUN_TESTS(aggregate_ret_func);\n+\tif (has_ret_pair_kfuncs())\n+\t\tRUN_TESTS(aggregate_ret_kfunc);\n+}\ndiff --git a/tools/testing/selftests/bpf/prog_tests/exceptions.c b/tools/testing/selftests/bpf/prog_tests/exceptions.c\nindex 3588d6f97fd4e..71d00c568d802 100644\n--- a/tools/testing/selftests/bpf/prog_tests/exceptions.c\n+++ b/tools/testing/selftests/bpf/prog_tests/exceptions.c\n@@ -5,6 +5,7 @@\n #include \"exceptions.skel.h\"\n #include \"exceptions_ext.skel.h\"\n #include \"exceptions_fail.skel.h\"\n+#include \"exceptions_ret_pair_fail.skel.h\"\n #include \"exceptions_assert.skel.h\"\n \n static char log_buf[1024 * 1024];\n@@ -12,6 +13,7 @@ static char log_buf[1024 * 1024];\n static void test_exceptions_failure(void)\n {\n \tRUN_TESTS(exceptions_fail);\n+\tRUN_TESTS(exceptions_ret_pair_fail);\n }\n \n static void test_exceptions_success(void)\ndiff --git a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c\nindex 2523c07a16c65..0b54f911015c5 100644\n--- a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c\n+++ b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c\n@@ -441,6 +441,19 @@ static void test_func_replace_int_with_void(void)\n \t\t\t\t \" doesn't match type INT of global_func2()\");\n }\n \n+static void test_func_replace_ret_pair(void)\n+{\n+\tconst char *msg = \"Cannot replace function agg_ret_target_func with a \u003e8 byte return\";\n+\n+\t/*\n+\t * An extension cannot replace a function whose return value comes back\n+\t * in the R0:R2 pair: the extension's own return is capped at 8 bytes,\n+\t * so it would leave R2 stale for the target's callers.\n+\t */\n+\ttest_obj_load_failure_common(\"freplace_ret_pair.bpf.o\",\n+\t\t\t\t \"./aggregate_ret_target.bpf.o\", msg);\n+}\n+\n static int find_prog_btf_id(const char *name, __u32 attach_prog_fd)\n {\n \tstruct bpf_prog_info info = {};\n@@ -660,6 +673,8 @@ void serial_test_fexit_bpf2bpf(void)\n \t\ttest_func_replace_progmap();\n \tif (test__start_subtest(\"freplace_int_with_void\"))\n \t\ttest_func_replace_int_with_void();\n+\tif (test__start_subtest(\"freplace_ret_pair\"))\n+\t\ttest_func_replace_ret_pair();\n \tif (test__start_subtest(\"freplace_void\"))\n \t\ttest_func_replace_void();\n \tif (test__start_subtest(\"sleepable_fentry_to_xdp\"))\ndiff --git a/tools/testing/selftests/bpf/prog_tests/timer.c b/tools/testing/selftests/bpf/prog_tests/timer.c\nindex 09ff21e1ad2f0..593e56d8964ea 100644\n--- a/tools/testing/selftests/bpf/prog_tests/timer.c\n+++ b/tools/testing/selftests/bpf/prog_tests/timer.c\n@@ -6,6 +6,7 @@\n #include \u003csys/syscall.h\u003e\n #include \"timer.skel.h\"\n #include \"timer_failure.skel.h\"\n+#include \"timer_ret_pair_fail.skel.h\"\n #include \"timer_interrupt.skel.h\"\n \n #define NUM_THR 8\n@@ -285,6 +286,7 @@ void serial_test_timer(void)\n \ttest_timer(timer);\n \n \tRUN_TESTS(timer_failure);\n+\tRUN_TESTS(timer_ret_pair_fail);\n }\n \n void serial_test_timer_stress(void)\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_func.c b/tools/testing/selftests/bpf/progs/aggregate_ret_func.c\nnew file mode 100644\nindex 0000000000000..0bc18450a46ef\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_func.c\n@@ -0,0 +1,420 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003clinux/bpf.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \"bpf_misc.h\"\n+\n+typedef unsigned __int128 u128;\n+\n+__naked u128 global_agg_good(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0x1234;\"\t/* low 64 bits */\n+\t\"r2 = 0x5678;\"\t/* high 64 bits */\n+\t\"exit;\"\n+\t);\n+}\n+\n+__naked u128 global_agg_bad(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+__naked u128 global_agg_bad_ptr(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = r10;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__success __retval(0)\n+int aggregate_ret_global(void *ctx)\n+{\n+\t__u64 lo, hi;\n+\n+\tasm volatile (\n+\t\"call %[global_agg_good];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: __imm(global_agg_good)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\");\n+\tif (lo != 0x1234)\n+\t\treturn 1;\n+\tif (hi != 0x5678)\n+\t\treturn 2;\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"R2 !read_ok\")\n+__naked int aggregate_ret_global_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[global_agg_bad];\"\n+\t\"r0 = r2;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(global_agg_bad)\n+\t: __clobber_all);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"At subprogram exit the register R2 is not a scalar value\")\n+__naked int aggregate_ret_global_ptr_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[global_agg_bad_ptr];\"\n+\t\"r0 = r2;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(global_agg_bad_ptr)\n+\t: __clobber_all);\n+}\n+\n+static __naked __noinline u128 static_agg_bad_ptr(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = r10;\"\t/* stack pointer placed in the second return register */\n+\t\"exit;\"\n+\t);\n+}\n+\n+/*\n+ * R2 is caller-saved and only copied from the callee at exit; a PTR_TO_STACK\n+ * left in it is turned into an uninitialized R2 in the caller. A caller that\n+ * never reads R2 is therefore unaffected and loads fine.\n+ */\n+SEC(\"tc\")\n+__success __retval(0)\n+__naked int aggregate_ret_static_ptr_unused(void)\n+{\n+\tasm volatile (\n+\t\"call %[static_agg_bad_ptr];\"\n+\t\"r0 = 0;\"\t\t/* R2 holds a stack pointer but is never read */\n+\t\"exit;\"\n+\t:\n+\t: __imm(static_agg_bad_ptr)\n+\t: __clobber_all);\n+}\n+\n+/* But a caller that does read the returned stack pointer is rejected. */\n+SEC(\"tc\")\n+__failure __msg(\"R2 !read_ok\")\n+__naked int aggregate_ret_static_ptr_read_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[static_agg_bad_ptr];\"\n+\t\"r0 = r2;\"\t\t/* using the returned stack pointer is rejected */\n+\t\"exit;\"\n+\t:\n+\t: __imm(static_agg_bad_ptr)\n+\t: __clobber_all);\n+}\n+\n+static __naked __noinline u128 static_agg_no_r2(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"R2 !read_ok\")\n+__naked int aggregate_ret_static_uninit_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[static_agg_no_r2];\"\n+\t\"r0 = r2;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(static_agg_no_r2)\n+\t: __clobber_all);\n+}\n+\n+static __naked __noinline u128 static_agg_precise(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = 4;\"\t/* second half; its value is made precise below */\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__success __retval(0)\n+__log_level(2)\n+__msg(\"mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 2: (57) r6 \u0026= 7\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2\")\n+__msg(\"mark_precise: frame0: regs=r2 stack= before 12: (95) exit\")\n+__msg(\"mark_precise: frame1: regs=r2 stack= before 11: (b7) r2 = 4\")\n+__naked int aggregate_ret_static_precise(void)\n+{\n+\tasm volatile (\n+\t\"call %[static_agg_precise];\"\n+\t\"r6 = r2;\"\t\t/* derived from the aggregate's second half */\n+\t\"r6 \u0026= 7;\"\t\t/* keep it in [0, 7] to index the stack */\n+\t\"r1 = r10;\"\n+\t\"r1 += -8;\"\n+\t\"r1 += r6;\"\t\t/* ptr += scalar marks r6 (hence R2) precise */\n+\t\"r0 = 0;\"\n+\t\"*(u8 *)(r1 + 0) = r0;\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(static_agg_precise)\n+\t: __clobber_all);\n+}\n+\n+SEC(\"tc\")\n+__success __retval(0)\n+__log_level(2)\n+__msg(\"mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 2: (57) r6 \u0026= 7\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2\")\n+__msg(\"mark_precise: frame0: regs=r2 stack= before 0: (85) call pc+9\")\n+__naked int aggregate_ret_global_precise(void)\n+{\n+\tasm volatile (\n+\t\"call %[global_agg_good];\"\n+\t\"r6 = r2;\"\t\t/* derived from the aggregate's second half */\n+\t\"r6 \u0026= 7;\"\t\t/* keep it in [0, 7] to index the stack */\n+\t\"r1 = r10;\"\n+\t\"r1 += -8;\"\n+\t\"r1 += r6;\"\t\t/* ptr += scalar marks r6 (hence R2) precise */\n+\t\"r0 = 0;\"\n+\t\"*(u8 *)(r1 + 0) = r0;\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(global_agg_good)\n+\t: __clobber_all);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"return value larger than 8 bytes is not supported at program exit\")\n+__naked u128 aggregate_ret_entry_fail(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = 0;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+#if __clang_major__ \u003e= 23\n+\n+struct pair {\n+\t__u64 hi;\n+\t__u64 lo;\n+};\n+\n+union upair {\n+\t__u64 halves[2];\n+\tstruct {\n+\t\t__u64 lo;\n+\t\t__u64 hi;\n+\t} parts;\n+};\n+\n+/* A by-value struct that smuggles a pointer, which must be rejected. */\n+struct with_ptr {\n+\tvoid *p;\n+\t__u64 x;\n+};\n+\n+/* A by-value union that smuggles a pointer, which must be rejected too. */\n+union upair_with_ptr {\n+\tvoid *p;\n+\t__u64 halves[2];\n+};\n+\n+/* Global subprogram returning a scalar-only 16-byte struct in R0:R2. */\n+__naked struct pair global_ret_struct(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0x1234;\"\t/* struct's first half */\n+\t\"r2 = 0x5678;\"\t/* struct's second half */\n+\t\"exit;\"\n+\t);\n+}\n+\n+/* Global subprogram returning a scalar-only 16-byte union in R0:R2. */\n+__naked union upair global_ret_union(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0x1234;\"\n+\t\"r2 = 0x5678;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__success __retval(0)\n+int aggregate_ret_global_struct(void *ctx)\n+{\n+\t__u64 lo, hi;\n+\n+\tasm volatile (\n+\t\"call %[global_ret_struct];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: __imm(global_ret_struct)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\");\n+\tif (lo != 0x1234)\n+\t\treturn 1;\n+\tif (hi != 0x5678)\n+\t\treturn 2;\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+__success __retval(0)\n+int aggregate_ret_global_union(void *ctx)\n+{\n+\t__u64 lo, hi;\n+\n+\tasm volatile (\n+\t\"call %[global_ret_union];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: __imm(global_ret_union)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\");\n+\tif (lo != 0x1234)\n+\t\treturn 1;\n+\tif (hi != 0x5678)\n+\t\treturn 2;\n+\treturn 0;\n+}\n+\n+__naked struct with_ptr global_ret_struct_ptr(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = 0;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"Global function global_ret_struct_ptr() has unsupported return type\")\n+__naked int aggregate_ret_global_struct_ptr_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[global_ret_struct_ptr];\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(global_ret_struct_ptr)\n+\t: __clobber_all);\n+}\n+\n+__naked union upair_with_ptr global_ret_union_ptr(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = 0;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"Global function global_ret_union_ptr() has unsupported return type\")\n+__naked int aggregate_ret_global_union_ptr_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[global_ret_union_ptr];\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(global_ret_union_ptr)\n+\t: __clobber_all);\n+}\n+\n+#endif /* __clang_major__ \u003e= 23 */\n+\n+static __naked u128 agg_callee(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 1;\"\n+\t\"r2 = 2;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__log_level(2)\n+__msg(\"Live regs before insn:\")\n+/*\n+ * R2 is read at the exit of agg_callee() (insn 5), which returns a pair, but\n+ * not at the exit of this program (insn 2), which returns an int.\n+ */\n+__msg(\"0: .12345.... (85) call pc+2\")\n+__msg(\"1: ..2....... (bf) r0 = r2\")\n+__msg(\"2: 0......... (95) exit\")\n+__msg(\"3: .......... (b7) r0 = 1\")\n+__msg(\"4: 0......... (b7) r2 = 2\")\n+__msg(\"5: 0.2....... (95) exit\")\n+__naked int aggregate_ret_live(void)\n+{\n+\tasm volatile (\n+\t\"call %[agg_callee];\"\n+\t\"r0 = r2;\"\n+\t\"exit;\"\n+\t:\n+\t: [agg_callee]\"i\"(agg_callee)\n+\t: __clobber_all);\n+}\n+\n+/*\n+ * A static subprogram is verified inline, so prepare_func_exit() hands the\n+ * caller the callee's actual R0:R2 register state rather than an opaque scalar\n+ * pair. A pointer in the returned struct therefore stays tracked and is usable\n+ * by the caller, which is why btf_validate_return_type() does not apply the\n+ * scalar-only restriction to a local function. Return the context pointer as\n+ * the upper half and dereference it in the caller.\n+ */\n+struct ptr_pair {\n+\tvoid *p;\n+\t__u64 x;\n+};\n+\n+static __naked __noinline struct ptr_pair static_ret_ptr_pair(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0;\"\n+\t\"r2 = r1;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__success __retval(0)\n+__naked int aggregate_ret_static_ptr_pair(void)\n+{\n+\tasm volatile (\n+\t\"call %[static_ret_ptr_pair];\"\n+\t\"r1 = *(u32 *)(r2 + 0);\"\t/* deref the returned ctx pointer */\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(static_ret_ptr_pair)\n+\t: __clobber_all);\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c\nnew file mode 100644\nindex 0000000000000..f2e09c8be0bed\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c\n@@ -0,0 +1,48 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+#if defined(__clang_major__) \u0026\u0026 __clang_major__ \u003e= 23\n+\n+const volatile bool has_reg_pair_ret = true;\n+\n+#define MIX_A\t0xdeadbeefcafef00dULL\n+#define MIX_B\t0x0123456789abcdefULL\n+\n+typedef unsigned __int128 u128;\n+\n+static __noinline u128 make_i128(__u64 a, __u64 b)\n+{\n+\treturn ((u128)(a + b) \u003c\u003c 64) | (a - b);\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_int128_c_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen ^ MIX_A;\n+\t__u64 b = skb-\u003elen ^ MIX_B;\n+\tu128 v;\n+\n+\tv = make_i128(a, b);\n+\tif ((__u64)(v \u003e\u003e 64) != a + b)\n+\t\treturn 1;\n+\tif ((__u64)v != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+#else\n+\n+const volatile bool has_reg_pair_ret = false;\n+\n+SEC(\"tc\")\n+int aggregate_ret_int128_c_test(struct __sk_buff *skb)\n+{\n+\treturn 0;\n+}\n+\n+#endif\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c\nnew file mode 100644\nindex 0000000000000..617724aa70156\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c\n@@ -0,0 +1,126 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \"bpf_misc.h\"\n+#include \"../test_kmods/bpf_testmod_kfunc.h\"\n+\n+/*\n+ * Reference kfunc addresses to force those BTF to be emitted. Taking the address\n+ * (rather than calling) avoids any dependence on the compiler lowering an\n+ * __int128 or struct return value, which the BPF backend only supports from\n+ * LLVM 23 on.\n+ */\n+void __kfunc_btf_root(void)\n+{\n+\tasm volatile (\"\"\n+\t:\n+\t: \"r\"(\u0026bpf_kfunc_call_test_i128),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_fastcall),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_ptr),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_ii),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_big));\n+}\n+\n+/*\n+ * bpf_add_kfunc_call() rejects a kfunc returning more than 8 bytes unless the\n+ * JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), so a test that has to\n+ * get past it is tagged with the architectures implementing it. The two tests\n+ * below that are rejected earlier, on KF_FASTCALL or on reading R2 after an\n+ * 8-byte struct return, behave the same everywhere and are not tagged.\n+ */\n+\n+SEC(\"tc\")\n+__arch_x86_64 __arch_arm64 __arch_riscv64\n+__success __retval(0)\n+__log_level(2)\n+__msg(\"mark_precise: frame0: last_idx 7 first_idx 0 subseq_idx -1\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 6: (07) r1 += -8\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 5: (bf) r1 = r10\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 4: (57) r6 \u0026= 7\")\n+__msg(\"mark_precise: frame0: regs=r6 stack= before 3: (bf) r6 = r2\")\n+__msg(\"mark_precise: frame0: regs=r2 stack= before 2: (85) call bpf_kfunc_call_test_i128\")\n+__naked int aggregate_ret_kfunc_precise(void)\n+{\n+\tasm volatile (\n+\t\"r1 = 1;\"\n+\t\"r2 = 2;\"\n+\t\"call %[bpf_kfunc_call_test_i128];\"\n+\t\"r6 = r2;\"\t\t/* second return half */\n+\t\"r6 \u0026= 7;\"\t\t/* keep it in [0, 7] to index the stack */\n+\t\"r1 = r10;\"\n+\t\"r1 += -8;\"\n+\t\"r1 += r6;\"\t\t/* ptr += scalar marks r6 (hence R2) precise */\n+\t\"r0 = 0;\"\n+\t\"*(u8 *)(r1 + 0) = r0;\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(bpf_kfunc_call_test_i128)\n+\t: __clobber_all);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"kfunc bpf_kfunc_call_test_ret_fastcall with \u003e8-byte return is not supported with KF_FASTCALL\")\n+__naked int aggregate_ret_kfunc_fastcall_fail(void)\n+{\n+\tasm volatile (\n+\t\"r1 = 1;\"\n+\t\"r2 = 2;\"\n+\t\"call %[bpf_kfunc_call_test_ret_fastcall];\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(bpf_kfunc_call_test_ret_fastcall)\n+\t: __clobber_all);\n+}\n+\n+SEC(\"tc\")\n+__arch_x86_64 __arch_arm64 __arch_riscv64\n+__failure __msg(\"is not composed of scalars\")\n+__naked int aggregate_ret_kfunc_ptr_fail(void)\n+{\n+\tasm volatile (\n+\t\"r1 = 0;\"\n+\t\"call %[bpf_kfunc_call_test_ret_ptr];\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(bpf_kfunc_call_test_ret_ptr)\n+\t: __clobber_all);\n+}\n+\n+SEC(\"tc\")\n+__failure __msg(\"R2 !read_ok\")\n+__naked int aggregate_ret_kfunc_small_no_r2(void)\n+{\n+\tasm volatile (\n+\t\"r1 = 0;\"\n+\t\"r2 = 0;\"\n+\t\"call %[bpf_kfunc_call_test_ret_ii];\"\n+\t\"r0 = r2;\"\t/* R2 is not a return register for a \u003c=8 byte struct */\n+\t\"exit;\"\n+\t:\n+\t: __imm(bpf_kfunc_call_test_ret_ii)\n+\t: __clobber_all);\n+}\n+\n+/*\n+ * A return value larger than 16 bytes does not fit in R0:R2 and is rejected by\n+ * btf_distill_func_proto(), before the KF_FASTCALL and JIT-capability checks,\n+ * so this behaves the same on every architecture.\n+ */\n+SEC(\"tc\")\n+__failure __msg(\"The function bpf_kfunc_call_test_ret_big return type STRUCT is unsupported\")\n+__naked int aggregate_ret_kfunc_too_big_fail(void)\n+{\n+\tasm volatile (\n+\t\"call %[bpf_kfunc_call_test_ret_big];\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(bpf_kfunc_call_test_ret_big)\n+\t: __clobber_all);\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c\nnew file mode 100644\nindex 0000000000000..fd000620f3141\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c\n@@ -0,0 +1,66 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \"../test_kmods/bpf_testmod_kfunc.h\"\n+\n+#if defined(__clang_major__) \u0026\u0026 __clang_major__ \u003e= 23\n+\n+const volatile bool has_reg_pair_ret = true;\n+\n+#define MIX_A\t0xdeadbeefcafef00dULL\n+#define MIX_B\t0x0123456789abcdefULL\n+\n+typedef unsigned __int128 u128;\n+\n+SEC(\"tc\")\n+int aggregate_ret_kfunc_int128_c_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen ^ MIX_A;\n+\t__u64 b = skb-\u003elen ^ MIX_B;\n+\tu128 v;\n+\n+\tv = bpf_kfunc_call_test_i128(a, b);\n+\tif ((__u64)(v \u003e\u003e 64) != a + b)\n+\t\treturn 1;\n+\tif ((__u64)v != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen ^ MIX_A;\n+\t__u64 b = skb-\u003elen ^ MIX_B;\n+\tstruct prog_test_ret_pair p;\n+\n+\tp = bpf_kfunc_call_test_ret_pair(a, b);\n+\tif (p.hi != a + b)\n+\t\treturn 1;\n+\tif (p.lo != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+#else\n+\n+const volatile bool has_reg_pair_ret = false;\n+\n+SEC(\"tc\")\n+int aggregate_ret_kfunc_int128_c_test(struct __sk_buff *skb)\n+{\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb)\n+{\n+\treturn 0;\n+}\n+\n+#endif\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_run.c b/tools/testing/selftests/bpf/progs/aggregate_ret_run.c\nnew file mode 100644\nindex 0000000000000..382ef3b90037c\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_run.c\n@@ -0,0 +1,178 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \"bpf_misc.h\"\n+#include \"../test_kmods/bpf_testmod_kfunc.h\"\n+\n+typedef unsigned __int128 u128;\n+\n+/*\n+ * Reference kfunc addresses to force those BTF to be emitted. Taking the address\n+ * (rather than calling) avoids any dependence on the compiler lowering an __int128\n+ * or struct return value, which the BPF backend only supports from LLVM 23 on.\n+ */\n+void __kfunc_btf_root(void)\n+{\n+\tasm volatile (\"\"\n+\t:\n+\t: \"r\"(\u0026bpf_kfunc_call_test_i128),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_pair),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_li),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_ii),\n+\t \"r\"(\u0026bpf_kfunc_call_test_ret_uu));\n+}\n+\n+#define I128_ASM_LO 0xABCDabcd12345678ULL\n+#define I128_ASM_HI 0x1234567890abcdefULL\n+\n+static __naked __noinline u128 make_i128_asm(void)\n+{\n+\tasm volatile (\n+\t\"r0 = %[lo] ll;\"\t/* low 64 bits */\n+\t\"r2 = %[hi] ll;\"\t/* high 64 bits */\n+\t\"exit;\"\n+\t:\n+\t: __imm_const(lo, I128_ASM_LO), __imm_const(hi, I128_ASM_HI)\n+\t);\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_asm_test(struct __sk_buff *skb)\n+{\n+\t__u64 lo, hi;\n+\n+\tasm volatile (\n+\t\"call %[callee];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: [callee]\"i\"(make_i128_asm)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\"\n+\t);\n+\tif (lo != I128_ASM_LO)\n+\t\treturn 1;\n+\tif (hi != I128_ASM_HI)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * R0 holds bytes 0..7 of a kfunc return value and R2 bytes 8..15, so where a\n+ * member sits inside a register depends on the endianness of the target.\n+ * Although arm64 supports both little and big endian, for simplicity, only\n+ * do little endian for now..\n+ */\n+SEC(\"tc\")\n+int aggregate_ret_asm_kfunc_test(struct __sk_buff *skb)\n+{\n+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n+\t__u64 a = skb-\u003elen;\n+\t__u64 b = skb-\u003elen ^ 0xdeadbeefULL;\n+\t__u64 lo, hi;\n+\n+\tasm volatile (\n+\t\"r1 = %[a];\"\n+\t\"r2 = %[b];\"\n+\t\"call %[kfunc];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: [a]\"r\"(a), [b]\"r\"(b), [kfunc]\"i\"(bpf_kfunc_call_test_i128)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\"\n+\t);\n+\tif (hi != a + b)\n+\t\treturn 1;\n+\tif (lo != a - b)\n+\t\treturn 2;\n+#endif\n+\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_struct_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen;\n+\t__u64 b = skb-\u003elen ^ 0xdeadbeefULL;\n+\t__u64 lo, hi;\n+\n+\t/* struct { u64 hi; u64 lo; }: R0 = hi, R2 = lo. */\n+\tasm volatile (\n+\t\"r1 = %[a];\"\n+\t\"r2 = %[b];\"\n+\t\"call %[kfunc];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: [a]\"r\"(a), [b]\"r\"(b), [kfunc]\"i\"(bpf_kfunc_call_test_ret_pair)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\"\n+\t);\n+\tif (lo != a + b)\n+\t\treturn 1;\n+\tif (hi != a - b)\n+\t\treturn 2;\n+\n+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__\n+\t/* struct { u64 a; int b; }: R0 = a, low 32 bits of R2 = b. */\n+\tasm volatile (\n+\t\"r1 = %[a];\"\n+\t\"r2 = %[b];\"\n+\t\"call %[kfunc];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: [a]\"r\"(a), [b]\"r\"(b), [kfunc]\"i\"(bpf_kfunc_call_test_ret_li)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\"\n+\t);\n+\tif (lo != a)\n+\t\treturn 3;\n+\tif ((int)hi != ~(int)b)\n+\t\treturn 4;\n+\n+\t/* struct { int a; int b; }: 8 bytes, packed into R0; R2 is not used. */\n+\tasm volatile (\n+\t\"r1 = %[a];\"\n+\t\"r2 = %[b];\"\n+\t\"call %[kfunc];\"\n+\t\"%[lo] = r0;\"\n+\t: [lo]\"=r\"(lo)\n+\t: [a]\"r\"(a), [b]\"r\"(b), [kfunc]\"i\"(bpf_kfunc_call_test_ret_ii)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\"\n+\t);\n+\tif ((int)lo != (int)a)\n+\t\treturn 5;\n+\tif ((int)(lo \u003e\u003e 32) != (int)b)\n+\t\treturn 6;\n+#endif\n+\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_union_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen;\n+\t__u64 b = skb-\u003elen ^ 0xdeadbeefULL;\n+\t__u64 lo, hi;\n+\n+\tasm volatile (\n+\t\"r1 = %[a];\"\n+\t\"r2 = %[b];\"\n+\t\"call %[kfunc];\"\n+\t\"%[lo] = r0;\"\n+\t\"%[hi] = r2;\"\n+\t: [lo]\"=r\"(lo), [hi]\"=r\"(hi)\n+\t: [a]\"r\"(a), [b]\"r\"(b), [kfunc]\"i\"(bpf_kfunc_call_test_ret_uu)\n+\t: \"r0\", \"r1\", \"r2\", \"r3\", \"r4\", \"r5\"\n+\t);\n+\tif (lo != a + b)\n+\t\treturn 1;\n+\tif (hi != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c\nnew file mode 100644\nindex 0000000000000..5296e41da3f00\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c\n@@ -0,0 +1,82 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+#if defined(__clang_major__) \u0026\u0026 __clang_major__ \u003e= 23\n+\n+const volatile bool has_reg_pair_ret = true;\n+\n+#define MIX_A\t0xdeadbeefcafef00dULL\n+#define MIX_B\t0x0123456789abcdefULL\n+\n+struct pair {\n+\t__u64 hi;\t/* R0 */\n+\t__u64 lo;\t/* R2 */\n+};\n+\n+static __noinline struct pair make_pair(__u64 a, __u64 b)\n+{\n+\tstruct pair p = { .hi = a + b, .lo = a - b };\n+\n+\treturn p;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_struct_c_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen ^ MIX_A;\n+\t__u64 b = skb-\u003elen ^ MIX_B;\n+\tstruct pair p;\n+\n+\tp = make_pair(a, b);\n+\tif (p.hi != a + b)\n+\t\treturn 1;\n+\tif (p.lo != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+__noinline struct pair make_pair_global(__u64 a, __u64 b)\n+{\n+\tstruct pair p = { .hi = a + b, .lo = a - b };\n+\n+\treturn p;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_global_struct_c_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen ^ MIX_A;\n+\t__u64 b = skb-\u003elen ^ MIX_B;\n+\tstruct pair p;\n+\n+\tp = make_pair_global(a, b);\n+\tif (p.hi != a + b)\n+\t\treturn 1;\n+\tif (p.lo != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+#else\n+\n+const volatile bool has_reg_pair_ret = false;\n+\n+SEC(\"tc\")\n+int aggregate_ret_struct_c_test(struct __sk_buff *skb)\n+{\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_global_struct_c_test(struct __sk_buff *skb)\n+{\n+\treturn 0;\n+}\n+\n+#endif\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_target.c b/tools/testing/selftests/bpf/progs/aggregate_ret_target.c\nnew file mode 100644\nindex 0000000000000..cffd8d7d3241a\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_target.c\n@@ -0,0 +1,29 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003clinux/bpf.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \"bpf_misc.h\"\n+\n+/* freplace target: a global subprogram returning 16 bytes in R0:R2. */\n+__naked unsigned __int128 agg_ret_target_func(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 0x1234;\"\n+\t\"r2 = 0x5678;\"\n+\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__naked int agg_ret_target(void)\n+{\n+\tasm volatile (\n+\t\"call %[agg_ret_target_func];\"\n+\t\"r0 = 0;\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(agg_ret_target_func)\n+\t: __clobber_all);\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c\nnew file mode 100644\nindex 0000000000000..5547fa6cbd495\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c\n@@ -0,0 +1,58 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+#if defined(__clang_major__) \u0026\u0026 __clang_major__ \u003e= 23\n+\n+const volatile bool has_reg_pair_ret = true;\n+\n+#define MIX_A\t0xdeadbeefcafef00dULL\n+#define MIX_B\t0x0123456789abcdefULL\n+\n+union pair {\n+\t__u64 halves[2];\n+\tstruct {\n+\t\t__u64 lo;\t/* R0 */\n+\t\t__u64 hi;\t/* R2 */\n+\t} parts;\n+};\n+\n+static __noinline union pair make_pair(__u64 a, __u64 b)\n+{\n+\tunion pair p;\n+\n+\tp.halves[0] = a + b;\n+\tp.halves[1] = a - b;\n+\treturn p;\n+}\n+\n+SEC(\"tc\")\n+int aggregate_ret_union_c_test(struct __sk_buff *skb)\n+{\n+\t__u64 a = skb-\u003elen ^ MIX_A;\n+\t__u64 b = skb-\u003elen ^ MIX_B;\n+\tunion pair p;\n+\n+\tp = make_pair(a, b);\n+\tif (p.parts.lo != a + b)\n+\t\treturn 1;\n+\tif (p.parts.hi != a - b)\n+\t\treturn 2;\n+\n+\treturn 0;\n+}\n+\n+#else\n+\n+const volatile bool has_reg_pair_ret = false;\n+\n+SEC(\"tc\")\n+int aggregate_ret_union_c_test(struct __sk_buff *skb)\n+{\n+\treturn 0;\n+}\n+\n+#endif\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c\nnew file mode 100644\nindex 0000000000000..a45db5d9c1d44\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c\n@@ -0,0 +1,10 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)\n+{\n+\tfor (;;)\n+\t\t;\n+}\ndiff --git a/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c\nnew file mode 100644\nindex 0000000000000..35506c7c5a916\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c\n@@ -0,0 +1,10 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer)\n+{\n+\tfor (;;)\n+\t\t;\n+}\ndiff --git a/tools/testing/selftests/bpf/progs/exceptions_fail.c b/tools/testing/selftests/bpf/progs/exceptions_fail.c\nindex ac44d60e50666..9708efb93683b 100644\n--- a/tools/testing/selftests/bpf/progs/exceptions_fail.c\n+++ b/tools/testing/selftests/bpf/progs/exceptions_fail.c\n@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)\n \n SEC(\"?tc\")\n __exception_cb(exception_cb_bad_ret_type1)\n-__failure __msg(\"Global function exception_cb_bad_ret_type1() return value not void or scalar.\")\n+__failure __msg(\"Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\")\n int reject_exception_cb_type_1(struct __sk_buff *ctx)\n {\n \tbpf_throw(0);\ndiff --git a/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c\nnew file mode 100644\nindex 0000000000000..842f86ad8659e\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c\n@@ -0,0 +1,30 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_tracing.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+#include \"bpf_misc.h\"\n+#include \"bpf_experimental.h\"\n+\n+__naked __noinline __used\n+unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)\n+{\n+\tasm volatile (\n+\t\"r0 = r1;\"\n+\t\"r2 = 0;\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+SEC(\"?tc\")\n+__exception_cb(exception_cb_bad_ret_type3)\n+__failure __msg(\"exception cb cannot return value larger than 8 bytes\")\n+__btf_func_path(\"btf__exceptions_ret_pair_fail.bpf.o\")\n+int reject_exception_cb_ret_pair(void *ctx)\n+{\n+\tbpf_throw(0);\n+\treturn 0;\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/freplace_ret_pair.c b/tools/testing/selftests/bpf/progs/freplace_ret_pair.c\nnew file mode 100644\nindex 0000000000000..84b701402ca67\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/freplace_ret_pair.c\n@@ -0,0 +1,20 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+#include \u003clinux/bpf.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+\n+/*\n+ * An extension replaces its target outright, so it has to match the target's\n+ * return convention. Its own return value is capped at 8 bytes, so it can\n+ * never fill the R0:R2 pair that the target's callers read, and the attach is\n+ * rejected. btf_check_type_match() cannot catch this: it compares return types\n+ * by btf_type-\u003einfo only, and an int carries no vlen, so the __u64 here and\n+ * the target's __int128 compare equal.\n+ */\n+SEC(\"freplace/agg_ret_target_func\")\n+__u64 new_agg_ret_target_func(void)\n+{\n+\treturn 0;\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c\nnew file mode 100644\nindex 0000000000000..29fd294dfd49b\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c\n@@ -0,0 +1,49 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */\n+\n+#include \u003clinux/bpf.h\u003e\n+#include \u003ctime.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \u003cbpf/bpf_tracing.h\u003e\n+#include \"bpf_misc.h\"\n+\n+char _license[] SEC(\"license\") = \"GPL\";\n+\n+struct elem {\n+\tstruct bpf_timer t;\n+};\n+\n+struct {\n+\t__uint(type, BPF_MAP_TYPE_ARRAY);\n+\t__uint(max_entries, 1);\n+\t__type(key, int);\n+\t__type(value, struct elem);\n+} timer_map SEC(\".maps\");\n+\n+__naked __noinline __used\n+static unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 0;\"\n+\t\t\"r2 = 0;\"\n+\t\t\"exit;\"\n+\t\t::: __clobber_all\n+\t);\n+}\n+\n+SEC(\"fentry/bpf_fentry_test1\")\n+__failure __msg(\"callback function with \u003e8-byte return value is not supported\")\n+__btf_func_path(\"btf__timer_ret_pair_fail.bpf.o\")\n+long BPF_PROG2(test_bad_ret_pair, int, a)\n+{\n+\tint key = 0;\n+\tstruct bpf_timer *timer;\n+\n+\ttimer = bpf_map_lookup_elem(\u0026timer_map, \u0026key);\n+\tif (timer) {\n+\t\tbpf_timer_init(timer, \u0026timer_map, CLOCK_BOOTTIME);\n+\t\tbpf_timer_set_callback(timer, timer_cb_ret_pair);\n+\t}\n+\n+\treturn 0;\n+}\ndiff --git a/tools/testing/selftests/bpf/progs/verifier_arena.c b/tools/testing/selftests/bpf/progs/verifier_arena.c\nindex b241bbcf54a8a..455b55296f355 100644\n--- a/tools/testing/selftests/bpf/progs/verifier_arena.c\n+++ b/tools/testing/selftests/bpf/progs/verifier_arena.c\n@@ -704,4 +704,42 @@ int check_arena_arg_ret(void *ctx)\n \treturn 0;\n }\n \n+struct arena_ret_pair {\n+\t__u64 lo;\n+\t__u64 hi;\n+};\n+\n+/*\n+ * A 16-byte value is returned in the R0:R2 register pair. A global subprogram\n+ * may return an arena pointer in R0, but R2 holds the upper half of a scalar\n+ * pair, so an arena pointer there is not a valid return value. The ld_imm64 of\n+ * the arena map is what links the arena to the program, without which the\n+ * addr_space_cast insn is not allowed.\n+ */\n+__naked struct arena_ret_pair global_ret_arena_ptr_in_r2(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = %[arena] ll;\"\n+\t\t\"r2 = 8192;\"\n+\t\t\"r2 = addr_space_cast(r2, 0x0, 0x1);\"\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_addr(arena)\n+\t\t: __clobber_all);\n+}\n+\n+SEC(\"syscall\")\n+__failure __msg(\"At subprogram exit the register R2 is not a scalar value (arena)\")\n+__naked int check_global_ret_arena_ptr_in_r2(void)\n+{\n+\tasm volatile (\n+\t\t\"call %[global_ret_arena_ptr_in_r2];\"\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm(global_ret_arena_ptr_in_r2)\n+\t\t: __clobber_all);\n+}\n+\n char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c\nindex a6133f7521f34..9ef95983da3ca 100644\n--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c\n+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c\n@@ -939,6 +939,83 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)\n \treturn 0;\n }\n \n+/*\n+ * A kfunc is only usable where the ABI hands its return value back in\n+ * registers. s390x, for example, returns a by-value struct or union through a\n+ * hidden pointer argument (sret) whatever its size. That pointer shifts every\n+ * declared argument by one register, and pahole, which maps parameters to\n+ * registers positionally, then skips the function with \"unexpected register\n+ * usage for parameter\". resolve_btfids reports \"no BTF func for kfunc\" and\n+ * leaves the ID at 0, which makes register_btf_kfunc_id_set() fail at module\n+ * init, so the module does not load at all.\n+ *\n+ * Restrict these kfuncs to the architectures where the return value comes back\n+ * in registers. A kfunc taking no argument has nothing for the sret pointer to\n+ * displace and needs no guard, whatever it returns.\n+ */\n+#if defined(__x86_64__) || defined(__aarch64__)\n+__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b)\n+{\n+\treturn (__int128)(((unsigned __int128)(a + b) \u003c\u003c 64) | (a - b));\n+}\n+\n+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)\n+{\n+\tstruct prog_test_ret_pair r = { .hi = a + b, .lo = a - b };\n+\n+\treturn r;\n+}\n+\n+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)\n+{\n+\tstruct prog_test_ret_pair r = { .hi = a + b, .lo = a - b };\n+\n+\treturn r;\n+}\n+\n+__bpf_kfunc struct prog_test_ret_li bpf_kfunc_call_test_ret_li(u64 a, int b)\n+{\n+\tstruct prog_test_ret_li r = { .a = a, .b = ~b };\n+\n+\treturn r;\n+}\n+\n+__bpf_kfunc union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(u64 a, u64 b)\n+{\n+\tunion prog_test_ret_uu r;\n+\n+\tr.halves[0] = a + b;\n+\tr.halves[1] = a - b;\n+\treturn r;\n+}\n+\n+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)\n+{\n+\tstruct prog_test_ret_ptr r = { .p = NULL, .tag = tag };\n+\n+\treturn r;\n+}\n+\n+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)\n+{\n+\tstruct prog_test_ret_ii r = { .a = a, .b = b };\n+\n+\treturn r;\n+}\n+#endif /* __x86_64__ || __aarch64__ */\n+\n+/*\n+ * Takes no argument on purpose: with no arguments there is nothing for the sret\n+ * pointer to displace, so this needs no architecture guard even though it\n+ * returns 24 bytes. See the comment on bpf_kfunc_call_test_i128() above.\n+ */\n+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)\n+{\n+\tstruct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };\n+\n+\treturn r;\n+}\n+\n __bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,\n \t\t\t\t\t u64 e, u64 f, u64 g, u64 h,\n \t\t\t\t\t u64 i, u64 j)\n@@ -1472,6 +1549,16 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2)\n BTF_ID_FLAGS(func, bpf_kfunc_call_test3)\n BTF_ID_FLAGS(func, bpf_kfunc_call_test4)\n BTF_ID_FLAGS(func, bpf_kfunc_call_test5)\n+#if defined(__x86_64__) || defined(__aarch64__)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_fastcall, KF_FASTCALL)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_li)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_uu)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ptr)\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)\n+#endif\n+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_big)\n BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)\n BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)\n BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)\ndiff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h\nindex c4383acb53c11..755973793e9b5 100644\n--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h\n+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h\n@@ -55,6 +55,44 @@ struct prog_test_big_arg {\n \t__u64 b;\n };\n \n+/*\n+ * A 16-byte struct returned by value from a kfunc: .hi comes back in R0 and\n+ * .lo in R2.\n+ */\n+struct prog_test_ret_pair {\n+\t__u64 hi;\n+\t__u64 lo;\n+};\n+\n+struct prog_test_ret_li {\t/* 16 bytes: R0:R2 */\n+\t__u64 a;\n+\tint b;\n+};\n+\n+struct prog_test_ret_ii {\t/* 8 bytes: R0 only */\n+\tint a;\n+\tint b;\n+};\n+\n+union prog_test_ret_uu {\t/* 16 bytes: R0:R2 */\n+\t__u64 halves[2];\n+\tstruct {\n+\t\t__u64 lo;\n+\t\t__u64 hi;\n+\t} parts;\n+};\n+\n+struct prog_test_ret_ptr {\t/* 16 bytes: contains a pointer */\n+\tvoid *p;\n+\t__u64 tag;\n+};\n+\n+struct prog_test_ret_big {\t/* 24 bytes: too large for R0:R2 */\n+\t__u64 a;\n+\t__u64 b;\n+\t__u64 c;\n+};\n+\n struct prog_test_fail1 {\n \tvoid *p;\n \tint x;\n@@ -130,6 +168,16 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;\n struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;\n long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;\n int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;\n+#ifdef __SIZEOF_INT128__\n+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;\n+#endif\n+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;\n+struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;\n+struct prog_test_ret_li bpf_kfunc_call_test_ret_li(__u64 a, int b) __ksym;\n+struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;\n+union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(__u64 a, __u64 b) __ksym;\n+struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;\n+struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym;\n __u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,\n \t\t\t __u64 e, __u64 f, __u64 g, __u64 h,\n \t\t\t __u64 i, __u64 j) __ksym;\n"
} |
| 2/1 | 2026/08/11 00:58 | agent | patch-evaluator |
0mModel:best-expensive Args: null Results: {
"EnableConfigs": [
"BPF_SYSCALL"
],
"FocusSymbols": [
"bpf_get_kfunc_ret_size",
"__btf_type_is_scalar_struct",
"bpf_jit_supports_kfunc_ret_reg_pair",
"btf_distill_func_proto",
"btf_prepare_func_args"
],
"Reasoning": "The patch introduces support for BPF kfuncs returning values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. This involves significant changes to the BPF verifier (register tracking, liveness, backtracking, BTF validation) and JIT compilers (x86_64, arm64, riscv64). These are core functional changes to the BPF subsystem that are reachable from userspace and should be fuzzed.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer. Your job is to review a provided patch series and determine if it makes functional changes to the kernel that should be fuzzed. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on your internal knowledge of the kernel. You must actively use your code access tools to examine the actual source code and confirm any assumptions. Return WorthFuzzing=false if the patch only contains: - Modifications to Documentation/, Kconfig files, or code comments. - Purely decorative changes, such as logging (e.g., pr_err, printk) or tracepoints. - Changes to numeric constants or macros that do not functionally alter execution flow. - Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware (e.g., usb gadget, mac80211_hwsim). - Code in vendor-specific PCIe switch, SmartNIC, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) that require physical PCIe hardware cards not emulated in standard QEMU. - Driver .remove, .shutdown, or pci_unregister_driver teardown callbacks (e.g., igb_remove) that are executed only during PCI hot-unplug or sysfs driver unbind operations. If it modifies reachable core kernel logic, drivers, or architectures, use your code search tools to verify the code can be executed, then return WorthFuzzing=true. When returning WorthFuzzing=true, you MUST ALSO: 1. Extract any specific kernel functions that should be heavily fuzzed into FocusSymbols. Avoid listing generic hot-path functions to prevent skewed test distributions. Prefer non-static, non-inlined API entrypoint functions over internal static helper functions (which are inlined by the compiler and do not have distinct symbol addresses). 2. Identify any specific CONFIG_ options required to properly test this new/modified feature. Go and look into the Kconfig files and check for ifdefs around the code, do not make assumptions. Also check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed for Kbuild to compile the code statically into vmlinux. List them in the EnableConfigs output array, and DO NOT add a 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: For your convenience, here is the diff of the changes:
commit e58dc654dc59f196a5aa28b5e90faf612f8b4f83
Author: syz-cluster <triage@syzkaller.com>
Date: Tue Aug 11 00:58:42 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
index 1004eb0bec617..71d3ff8695994 100644
--- a/Documentation/bpf/kfuncs.rst
+++ b/Documentation/bpf/kfuncs.rst
@@ -575,6 +575,68 @@ is also covered by this recovery. A kfunc handed an arena pointer may
therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking
against the arena. Larger accesses must verify the range explicitly.
+2.9 kfunc Return Values
+-----------------------
+
+A kfunc may return a scalar, a pointer, or a small struct or union by
+value. A scalar or pointer of up to 8 bytes is returned in R0, as usual.
+
+A struct or union returned by value must be composed only of scalars
+(recursively), where a scalar is an integer or an enum; arrays of scalars are
+allowed as members. Its bytes are handed back to the program as the raw
+contents of R0 (and R2), so a pointer field would be laundered into a scalar
+and escape the verifier's pointer provenance and reference tracking. A struct
+or union with a pointer member is therefore rejected at load time, and so is
+one with a floating-point member, which the ABI may not return in R0:R2 at
+all.
+
+A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a
+scalar-only struct or union, or an ``__int128``. Such a value is returned
+in the register pair R0:R2, matching the convention LLVM uses for the BPF
+target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or
+union of 8 bytes or less is returned in R0 alone.
+
+::
+
+ struct bpf_pair { __u64 a, b; }; /* 16 bytes */
+
+ __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void)
+ {
+ struct bpf_pair p = { .a = 1, .b = 2 };
+
+ return p; /* p.a in R0, p.b in R2 */
+ }
+
+Returning a value in the R0:R2 pair requires the JIT to place the second
+half of the return value into R2, which not every architecture supports
+right now. A kfunc with a return value larger than 8 bytes is therefore
+rejected at load time on a JIT that does not advertise this capability (see
+``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run
+by the interpreter. A return value larger than 16 bytes is not supported.
+
+The same R0:R2 convention applies to a BPF subprogram, global or static,
+that returns an ``__int128`` or a struct or union larger than 8 bytes. Such a
+program also requires the JIT, since the interpreter propagates only R0 out
+of a subprogram. A global subprogram is verified in isolation, so its
+by-value struct or union return is restricted to scalars just like a kfunc's;
+a static subprogram is verified inline and has no such restriction. The main
+program cannot return more than 8 bytes, as its return value is the program's
+exit code.
+
+A global subprogram must leave a scalar in *every* register of the pair, so
+both halves of the returned value have to be assigned. Leaving the upper half
+uninitialized is not merely untidy: the compiler is then free to leave R2
+holding whatever it happened to hold, which for a subprogram taking a pointer
+argument is typically that pointer. Handing the caller an unknown scalar built
+from a pointer is a leak, so the verifier rejects it with::
+
+ At subprogram exit the register R2 is not a scalar value (...)
+
+Initialize the whole return value, for example ``struct pair p = {};``, to
+avoid this. A static subprogram is exempt: it is verified inline, so an
+unassigned R2 is simply passed back to the caller as uninitialized and only a
+caller that reads it fails.
+
.. _BPF_kfunc_lifecycle_expectations:
3. kfunc lifecycle expectations
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index d14d297ebb967..a1febb4c5718c 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2330,6 +2330,11 @@ bool bpf_jit_supports_kfunc_call(void)
return true;
}
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return true;
+}
+
bool bpf_jit_supports_stack_args(void)
{
return true;
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index 6b9972b07c1b0..c8e94d4657e9b 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -2111,6 +2111,11 @@ bool bpf_jit_supports_kfunc_call(void)
return true;
}
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return true;
+}
+
bool bpf_jit_supports_ptr_xchg(void)
{
return true;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 8dddb5d7af21b..01c34114c8502 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2647,6 +2647,22 @@ st: insn_off = insn->off;
return -EINVAL;
if (priv_frame_ptr)
pop_r9(&prog);
+ if (src_reg == BPF_PSEUDO_KFUNC_CALL) {
+ const struct btf_func_model *fm;
+
+ /*
+ * A kfunc returning a >8 byte aggregate hands the
+ * second half back in RDX (the native ABI's second
+ * return reg), but BPF expects it in R0:R2. BPF R0
+ * is RAX (no move needed), while BPF R2 is RSI, so
+ * copy RDX into RSI.
+ */
+ fm = bpf_jit_find_kfunc_model(bpf_prog, insn);
+ if (!fm)
+ return -EFAULT;
+ if (fm->ret_size > 8)
+ emit_mov_reg(&prog, true, BPF_REG_2, BPF_REG_3);
+ }
break;
}
@@ -4137,6 +4153,11 @@ bool bpf_jit_supports_kfunc_call(void)
return true;
}
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return true;
+}
+
bool bpf_jit_supports_stack_args(void)
{
return true;
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 93f7c2075eeaa..1f516daa41dbf 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -811,6 +811,8 @@ struct bpf_subprog_info {
bool is_async_cb: 1;
bool is_exception_cb: 1;
bool args_cached: 1;
+ /* true if the return value is passed in the R0:R2 register pair */
+ bool ret_reg_pair: 1;
/* true if bpf_fastcall stack region is used by functions that can't be inlined */
bool keep_fastcall_stack: 1;
bool changes_pkt_data: 1;
@@ -1044,6 +1046,16 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
return &env->subprog_info[subprog];
}
+/*
+ * True if @subprog returns its value in the R0:R2 register pair. Cached by
+ * bpf_compute_subprog_ret_regs(), since this is queried on hot paths: at
+ * every subprogram call and at every subprogram exit.
+ */
+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
+{
+ return subprog_info(env, subprog)->ret_reg_pair;
+}
+
struct bpf_call_summary {
u8 num_params;
bool is_void;
@@ -1435,6 +1447,10 @@ int bpf_jmp_offset(struct bpf_insn *insn);
struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);
void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);
bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);
+int bpf_get_kfunc_ret_size(const struct bpf_prog *prog, u32 func_id,
+ u16 btf_fd_idx, u8 *ret_size);
+bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
+ const struct btf_type *t, int rec);
int bpf_find_subprog(struct bpf_verifier_env *env, int off);
bool bpf_is_throw_kfunc(struct bpf_insn *insn);
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 4edba8182db1b..f3c34fd70f2d2 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1213,6 +1213,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);
bool bpf_jit_supports_subprog_tailcalls(void);
bool bpf_jit_supports_percpu_insn(void);
bool bpf_jit_supports_kfunc_call(void);
+bool bpf_jit_supports_kfunc_ret_reg_pair(void);
bool bpf_jit_supports_stack_args(void);
bool bpf_jit_supports_arena_args(void);
bool bpf_jit_supports_far_kfunc_call(void);
diff --git a/kernel/bpf/backtrack.c b/kernel/bpf/backtrack.c
index 40bd04421a991..fc8ecad6f01b0 100644
--- a/kernel/bpf/backtrack.c
+++ b/kernel/bpf/backtrack.c
@@ -425,6 +425,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
*/
verifier_bug_if(idx + 1 != subseq_idx, env,
"extra insn from subprog");
+ /*
+ * a global subprog returning more than 8 bytes
+ * sets R2 as well. R2 is part of the args mask
+ * checked just below, so it has to be cleared
+ * here rather than next to R0.
+ */
+ if (bt_is_reg_set(bt, BPF_REG_2) &&
+ bpf_ret_reg_pair(env, subprog))
+ bt_clear_reg(bt, BPF_REG_2);
/* r1-r5 are invalidated after subprog call,
* so for global func call it shouldn't be set
* anymore
@@ -508,6 +517,19 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
return -ENOTSUPP;
/* regular helper call sets R0 */
bt_clear_reg(bt, BPF_REG_0);
+ /* a kfunc returning more than 8 bytes also sets R2 */
+ if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL &&
+ bt_is_reg_set(bt, BPF_REG_2)) {
+ u8 ret_size;
+ int err;
+
+ err = bpf_get_kfunc_ret_size(env->prog, insn->imm, insn->off,
+ &ret_size);
+ if (verifier_bug_if(err, env, "no kfunc desc for insn %d", idx))
+ return -EFAULT;
+ if (ret_size > 8)
+ bt_clear_reg(bt, BPF_REG_2);
+ }
if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
/* if backtracking was looking for registers R1-R5
* they should have been found already.
@@ -522,7 +544,30 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
return -EFAULT;
}
} else if (opcode == BPF_EXIT) {
- bool r0_precise;
+ bool from_subprog_call, r0_precise, r2_precise = false;
+
+ /*
+ * BPF_EXIT in subprog or callback always returns
+ * right after the call instruction, so by checking
+ * whether the instruction at subseq_idx-1 is subprog
+ * call or not we can distinguish actual exit from
+ * *subprog* from exit from *callback*. In the former
+ * case, we need to propagate the precision of the
+ * return registers, if necessary. In the latter we
+ * never do that.
+ */
+ from_subprog_call = subseq_idx - 1 >= 0 &&
+ bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+ if (from_subprog_call && bt_is_reg_set(bt, BPF_REG_2)) {
+ struct bpf_subprog_info *callee;
+
+ /* 'idx' is the exit insn, so it is in the callee */
+ callee = bpf_find_containing_subprog(env, idx);
+ if (verifier_bug_if(!callee, env,
+ "no subprog contains exit insn %d", idx))
+ return -EFAULT;
+ r2_precise = bpf_ret_reg_pair(env, callee - env->subprog_info);
+ }
/* Backtracking to a nested function call, 'idx' is a part of
* the inner frame 'subseq_idx' is a part of the outer frame.
@@ -535,23 +580,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
if (subseq_idx >= 0 && bpf_calls_callback(env, subseq_idx))
for (i = BPF_REG_1; i <= BPF_REG_5; i++)
bt_clear_reg(bt, i);
+ if (r2_precise)
+ bt_clear_reg(bt, BPF_REG_2);
if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
verifier_bug(env, "backtracking exit unexpected regs %x",
bt_reg_mask(bt));
return -EFAULT;
}
- /* BPF_EXIT in subprog or callback always returns
- * right after the call instruction, so by checking
- * whether the instruction at subseq_idx-1 is subprog
- * call or not we can distinguish actual exit from
- * *subprog* from exit from *callback*. In the former
- * case, we need to propagate r0 precision, if
- * necessary. In the former we never do that.
- */
- r0_precise = subseq_idx - 1 >= 0 &&
- bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]) &&
- bt_is_reg_set(bt, BPF_REG_0);
+ r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
bt_clear_reg(bt, BPF_REG_0);
if (bt_subprog_enter(bt))
@@ -559,6 +596,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
if (r0_precise)
bt_set_reg(bt, BPF_REG_0);
+ if (r2_precise)
+ bt_set_reg(bt, BPF_REG_2);
/* r6-r9 and stack slots will stay set in caller frame
* bitmasks until we return back from callee(s)
*/
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 6606187ed4f43..5551abcea1d39 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -7592,7 +7592,12 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
return -EINVAL;
}
ret = __get_type_size(btf, func->type, &t);
- if (ret < 0 || btf_type_is_struct(t)) {
+ /*
+ * __get_type_size() already restricts a non-negative ret to void, a
+ * pointer, an int, an enum or a struct/union, so only the size is checked
+ * here.
+ */
+ if (ret < 0 || ret > 16) {
bpf_log(log,
"The function %s return type %s is unsupported.\n",
tname, btf_type_str(t));
@@ -7965,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,
/* Check whether the type is a valid return type. */
static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,
- const struct btf_type *t, int subprog)
+ const struct btf_type *t, int subprog, bool is_global)
{
u32 tags = 0;
int err;
@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
return 0;
+ if (btf_type_is_struct(t) && t->size <= 16) {
+ /*
+ * A >8 byte struct/union is returned in the R0:R2 register pair.
+ * A global function is verified in isolation, so its caller models
+ * the return as an opaque R0:R2 scalar pair; it must therefore
+ * contain only scalars, otherwise a pointer field would be
+ * laundered into a scalar and escape provenance and reference
+ * tracking. That requirement is enforced here: do_check_common()
+ * propagates the error for global functions and for the main
+ * program.
+ *
+ * A local (static) function is verified inline and its R0:R2 are
+ * copied as precise register state (with the JIT forced on when
+ * the pair is consumed), so a pointer field stays tracked and needs
+ * no such restriction. Accepting it here is not by itself what
+ * makes it legal: btf_check_subprog_call() drops any error other
+ * than -EFAULT. What it avoids is needlessly marking the
+ * subprogram's BTF unreliable.
+ *
+ * The main program (subprog 0) takes the scalar-only path as well,
+ * but its return value is the program's exit code, so a >8 byte
+ * return is rejected separately at BPF_EXIT.
+ */
+ bool local_func = subprog && !is_global;
+
+ if (local_func || __btf_type_is_scalar_struct(env, btf, t, 0))
+ return 0;
+ }
+
return -EOPNOTSUPP;
}
@@ -8075,12 +8109,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return -EINVAL;
}
- err = btf_validate_return_type(env, btf, t, subprog);
+ err = btf_validate_return_type(env, btf, t, subprog, is_global);
if (err) {
if (is_global) {
bpf_log(log,
- "Global function %s() return value not void or scalar. "
- "Only those are supported.\n",
+ "Global function %s() has unsupported return type. "
+ "Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n",
tname);
}
return err;
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index a3e1fae32eace..d98f4220e875e 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3303,6 +3303,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)
return false;
}
+bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return false;
+}
+
bool __weak bpf_jit_supports_stack_args(void)
{
return false;
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index ef9a5a9228872..451edd74fa6f5 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -2062,10 +2062,15 @@ static inline u32 mask_widen(u32 m) { return m | (m << 16); }
static inline u16 mask_lo(u32 m) { return (u16)m; }
static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }
-/* Compute info->{use,def} fields for the instruction */
+/*
+ * Compute info->{use,def} fields for the instruction. @ret_reg_pair tells
+ * whether the subprogram containing @insn returns its value in the R0:R2
+ * register pair, which matters for BPF_EXIT.
+ */
static void compute_insn_live_regs(struct bpf_verifier_env *env,
struct bpf_insn *insn,
- struct insn_live_regs *info)
+ struct insn_live_regs *info,
+ bool ret_reg_pair)
{
struct bpf_call_summary cs;
const u8 class = BPF_CLASS(insn->code);
@@ -2196,7 +2201,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
break;
case BPF_EXIT:
def = 0;
- use = r0;
+ use = ret_reg_pair ? (r0 | reg64_mask(BPF_REG_2)) : r0;
break;
case BPF_CALL:
def = ALL_CALLER_SAVED_REGS;
@@ -2233,8 +2238,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
struct insn_live_regs *state;
int insn_cnt = env->prog->len;
u64 pos, insn_pos;
- int err = 0, i, j;
- bool changed;
+ int err = 0, i, j, subprog, start, end;
+ bool changed, ret_reg_pair;
/* Use the following algorithm:
* - define the following:
@@ -2261,8 +2266,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
goto out;
}
- for (i = 0; i < insn_cnt; ++i)
- compute_insn_live_regs(env, &insns[i], &state[i]);
+ for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
+ start = env->subprog_info[subprog].start;
+ end = env->subprog_info[subprog + 1].start;
+ ret_reg_pair = bpf_ret_reg_pair(env, subprog);
+
+ for (i = start; i < end; ++i)
+ compute_insn_live_regs(env, &insns[i], &state[i], ret_reg_pair);
+ }
/* Forward pass: resolve stack access through FP-derived pointers */
err = bpf_compute_subprog_arg_access(env);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index add3affc57035..f8294359c85df 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -382,27 +382,75 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)
return aux && aux[subprog].linkage == BTF_FUNC_GLOBAL;
}
-static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+/* Return type of a subprogram, NULL if it cannot be resolved */
+static const struct btf_type *subprog_ret_type(struct bpf_verifier_env *env, int subprog)
{
- const struct btf_type *type, *func, *func_proto;
+ const struct btf_type *func, *func_proto;
const struct btf *btf = env->prog->aux->btf;
u32 btf_id;
+ if (!btf || !env->prog->aux->func_info)
+ return NULL;
+
btf_id = env->prog->aux->func_info[subprog].type_id;
+ /* Both already validated by prepare_btf_func() at prog load. */
func = btf_type_by_id(btf, btf_id);
- if (verifier_bug_if(!func, env, "btf_id %u not found", btf_id))
- return false;
-
func_proto = btf_type_by_id(btf, func->type);
- if (!func_proto)
- return false;
- type = btf_type_skip_modifiers(btf, func_proto->type, NULL);
- if (!type)
- return false;
+ return btf_type_skip_modifiers(btf, func_proto->type, NULL);
+}
+
+static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+{
+ const struct btf_type *type = subprog_ret_type(env, subprog);
- return btf_type_is_void(type);
+ return type && btf_type_is_void(type);
+}
+
+/*
+ * Number of registers holding a function return value: a value of up to 8
+ * bytes is returned in R0, a value of more than 8 bytes and no more than 16
+ * bytes (an __int128 or a struct/union of such size) is returned in the R0:R2
+ * register pair, with R2 holding the upper half.
+ */
+static u32 ret_regs_cnt(u32 size)
+{
+ return size > 8 && size <= 16 ? 2 : 1;
+}
+
+/* Registers holding a function return value, in order. See ret_regs_cnt(). */
+static const int ret_regs[] = { BPF_REG_0, BPF_REG_2 };
+
+/*
+ * Resolve the return convention of every subprogram once, so that
+ * bpf_ret_reg_pair() is a plain flag test on the hot paths that use it.
+ */
+static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+ const struct btf_type *type;
+ int subprog;
+
+ for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
+ type = subprog_ret_type(env, subprog);
+ if (type && (btf_type_is_struct(type) || btf_type_is_scalar(type)))
+ subprog_info(env, subprog)->ret_reg_pair = ret_regs_cnt(type->size) > 1;
+ }
+}
+
+/*
+ * A >8 byte BPF return changes the calling convention to R0:R2, so the
+ * verifier can only allow it while the subprogram's prototype remains
+ * reliable. Once BTF is marked unreliable, reject the feature instead of
+ * silently falling back to R0-only semantics.
+ */
+static bool subprog_ret_pair_unreliable(struct bpf_verifier_env *env, int subprog)
+{
+ struct bpf_prog_aux *aux = env->prog->aux;
+
+ return bpf_ret_reg_pair(env, subprog) &&
+ aux->func_info_aux &&
+ aux->func_info_aux[subprog].unreliable;
}
static const char *subprog_name(const struct bpf_verifier_env *env, int subprog)
@@ -2470,6 +2518,19 @@ int bpf_get_kfunc_addr(const struct bpf_prog *prog, u32 func_id,
return 0;
}
+int bpf_get_kfunc_ret_size(const struct bpf_prog *prog, u32 func_id,
+ u16 btf_fd_idx, u8 *ret_size)
+{
+ const struct bpf_kfunc_desc *desc;
+
+ desc = find_kfunc_desc(prog, func_id, btf_fd_idx);
+ if (!desc)
+ return -EFAULT;
+
+ *ret_size = desc->func_model.ret_size;
+ return 0;
+}
+
#define BPF_FD_SLOT_BTF 1UL
static void fd_slot_set_map(struct bpf_fd_array *slot, struct bpf_map *map)
@@ -2807,6 +2868,19 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
err = btf_distill_func_proto(&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);
if (err)
return err;
+ if (func_model.ret_size > 8) {
+ if (kfunc.flags && (*kfunc.flags & KF_FASTCALL)) {
+ verbose(env,
+ "kfunc %s with >8-byte return is not supported with KF_FASTCALL\n",
+ kfunc.name);
+ return -EOPNOTSUPP;
+ }
+ if (!bpf_jit_supports_kfunc_ret_reg_pair()) {
+ verbose(env, "kfunc %s with >8-byte return is not supported by JIT\n",
+ kfunc.name);
+ return -EOPNOTSUPP;
+ }
+ }
memset(&meta, 0, sizeof(meta));
meta.btf = kfunc.btf;
@@ -9388,6 +9462,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
u16 callee_incoming, stack_arg_cnt;
struct bpf_func_state *caller;
int err, subprog, target_insn;
+ u32 i, nregs;
target_insn = *insn_idx + insn->imm + 1;
subprog = bpf_find_subprog(env, target_insn);
@@ -9399,6 +9474,11 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
err = btf_check_subprog_call(env, subprog, caller->regs);
if (err == -EFAULT)
return err;
+ if (subprog_ret_pair_unreliable(env, subprog)) {
+ verbose(env, "Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n",
+ subprog, subprog_name(env, subprog));
+ return -EINVAL;
+ }
if (bpf_subprog_is_global(env, subprog)) {
const char *sub_name = subprog_name(env, subprog);
@@ -9430,9 +9510,22 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
clear_caller_saved_regs(env, caller->regs);
invalidate_outgoing_stack_args(env, cur_func(env));
- /* All non-void global functions return a 64-bit SCALAR_VALUE. */
+ /*
+ * A non-void global function returns a 64-bit SCALAR_VALUE in
+ * R0, or a >8 byte SCALAR_VALUE in the R0:R2 register pair.
+ */
if (!subprog_returns_void(env, subprog)) {
- mark_reg_unknown(env, caller->regs, BPF_REG_0);
+ nregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;
+ /*
+ * The R0:R2 return convention is only implemented in the
+ * JIT: the interpreter propagates BPF_R0 alone out of a
+ * subprogram, so a caller reading R2 would see a stale
+ * value. Force the JIT once a caller can observe the pair.
+ */
+ if (nregs > 1)
+ env->prog->jit_required = 1;
+ for (i = 0; i < nregs; i++)
+ mark_reg_unknown(env, caller->regs, ret_regs[i]);
}
if (env->subprog_info[subprog].might_throw) {
@@ -9754,10 +9847,19 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
struct bpf_func_state *caller, *callee;
struct bpf_reg_state *r0;
bool in_callback_fn;
+ u32 i, nregs;
int err;
callee = state->frame[state->curframe];
r0 = &callee->regs[BPF_REG_0];
+ if (subprog_ret_pair_unreliable(env, callee->subprogno)) {
+ verbose(env, "Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n",
+ callee->subprogno, subprog_name(env, callee->subprogno));
+ return -EINVAL;
+ }
+ nregs = bpf_ret_reg_pair(env, callee->subprogno) ? 2 : 1;
+ if (nregs > 1)
+ env->prog->jit_required = 1;
if (r0->type == PTR_TO_STACK) {
/* technically it's ok to return caller's stack pointer
* (or caller's caller's pointer) back to the caller,
@@ -9793,8 +9895,23 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
return -EFAULT;
}
} else {
- /* return to the caller whatever r0 had in the callee */
- caller->regs[BPF_REG_0] = *r0;
+ /*
+ * return to the caller whatever the callee had in the
+ * return register(s)
+ */
+ for (i = 0; i < nregs; i++)
+ caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]];
+
+ /*
+ * R2 carries only the upper half of a register pair return
+ * value. A stack pointer must not escape the callee (see the
+ * R0 case above), but there is no need to reject the whole
+ * program for it: hand the caller an uninitialized R2 instead,
+ * so that only a caller actually using the returned pointer
+ * fails.
+ */
+ if (nregs > 1 && caller->regs[BPF_REG_2].type == PTR_TO_STACK)
+ bpf_mark_reg_not_init(env, &caller->regs[BPF_REG_2]);
}
/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,
@@ -10691,6 +10808,19 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
return 0;
}
+/*
+ * Mark the register(s) holding a @size byte kfunc return value as unknown
+ * scalars. Both halves of a register pair are treated the same way.
+ */
+static void mark_kfunc_ret_regs(struct bpf_verifier_env *env,
+ struct bpf_reg_state *regs, u32 size)
+{
+ u32 i, nregs = ret_regs_cnt(size);
+
+ for (i = 0; i < nregs; i++)
+ mark_reg_unknown(env, regs, ret_regs[i]);
+}
+
static bool is_kfunc_acquire(struct bpf_call_arg_meta *meta)
{
return meta->kfunc_flags & KF_ACQUIRE;
@@ -10958,9 +11088,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_
}
/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */
-static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
- const struct btf *btf,
- const struct btf_type *t, int rec)
+bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
+ const struct btf *btf,
+ const struct btf_type *t, int rec)
{
const struct btf_type *member_type;
const struct btf_member *member;
@@ -13167,10 +13297,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
}
if (btf_type_is_scalar(t)) {
- mark_reg_unknown(env, regs, BPF_REG_0);
+ mark_kfunc_ret_regs(env, regs, t->size);
if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))
__mark_reg_const_zero(env, ®s[BPF_REG_0]);
+ } else if (btf_type_is_struct(t)) {
+ /*
+ * The returned struct comes back as raw register bits modeled
+ * as an unknown scalar, so it must contain only scalars:
+ * otherwise a pointer field would be laundered into a scalar
+ * and escape provenance and reference tracking.
+ */
+ if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+ verbose(env,
+ "kernel function %s returns %s %s that is not composed of scalars\n",
+ func_name, btf_type_str(t),
+ btf_name_by_offset(desc_btf, t->name_off));
+ return -EINVAL;
+ }
+ mark_kfunc_ret_regs(env, regs, t->size);
} else if (btf_type_is_ptr(t)) {
ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id);
err = check_special_kfunc(env, &meta, regs, insn_aux, ptr_type, desc_btf);
@@ -16250,6 +16395,11 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)
verbose(env, "callback function not static\n");
return -EINVAL;
}
+ if (bpf_ret_reg_pair(env, subprogno)) {
+ verbose(env,
+ "callback function with >8-byte return value is not supported\n");
+ return -EINVAL;
+ }
dst_reg->type = PTR_TO_FUNC;
dst_reg->subprogno = subprogno;
@@ -16617,37 +16767,61 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
return 0;
}
-static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+static int check_global_ret_scalar_reg(struct bpf_verifier_env *env, u32 regno,
+ bool allow_arena_ptr_return)
{
- struct bpf_reg_state *reg = reg_state(env, BPF_REG_0);
- struct bpf_func_state *cur_frame = cur_func(env);
+ struct bpf_reg_state *reg;
int err;
- if (subprog_returns_void(env, cur_frame->subprogno))
- return 0;
-
- err = check_reg_arg(env, BPF_REG_0, SRC_OP);
+ err = check_reg_arg(env, regno, SRC_OP);
if (err)
return err;
/* Pointers to arena are safe to pass between subprograms. */
- if (is_arena_reg(env, BPF_REG_0))
+ if (allow_arena_ptr_return && is_arena_reg(env, regno))
return 0;
- if (is_pointer_value(env, BPF_REG_0)) {
- verbose(env, "R%d leaks addr as return value\n", BPF_REG_0);
+ if (is_pointer_value(env, regno)) {
+ verbose(env, "R%d leaks addr as return value\n", regno);
return -EACCES;
}
+ reg = reg_state(env, regno);
if (reg->type != SCALAR_VALUE) {
- verbose(env, "At subprogram exit the register R0 is not a scalar value (%s)\n",
- reg_type_str(env, reg->type));
+ verbose(env, "At subprogram exit the register R%d is not a scalar value (%s)\n",
+ regno, reg_type_str(env, reg->type));
return -EINVAL;
}
return 0;
}
+static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+{
+ struct bpf_func_state *cur_frame = cur_func(env);
+ u32 subprog = cur_frame->subprogno;
+ u32 i, nregs;
+ int err;
+
+ if (subprog_returns_void(env, subprog))
+ return 0;
+
+ /*
+ * An arena pointer is only a legitimate return value when it is the
+ * whole of it, that is when it is returned in R0 alone. Both halves of
+ * a register pair carry a piece of a >8 byte scalar, so an arena
+ * pointer in either of them is a leak.
+ */
+ nregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;
+ for (i = 0; i < nregs; i++) {
+ err = check_global_ret_scalar_reg(env, ret_regs[i], nregs == 1);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
/* Bitmask with 1s for all caller saved registers */
#define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1)
@@ -17134,10 +17308,16 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,
*/
if (cur_frame->subprogno &&
!cur_frame->in_async_callback_fn &&
- !cur_frame->in_exception_callback_fn)
+ !cur_frame->in_exception_callback_fn) {
err = check_global_subprog_return_code(env);
- else
+ } else {
+ if (!cur_frame->subprogno && bpf_ret_reg_pair(env, 0)) {
+ verbose(env,
+ "return value larger than 8 bytes is not supported at program exit\n");
+ return -EINVAL;
+ }
err = check_return_code(env, BPF_REG_0, "R0");
+ }
if (err)
return err;
return PROCESS_BPF_EXIT;
@@ -18464,6 +18644,12 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
ret = -EINVAL;
goto out;
}
+ if (bpf_ret_reg_pair(env, subprog)) {
+ verbose(env,
+ "exception cb cannot return value larger than 8 bytes\n");
+ ret = -EINVAL;
+ goto out;
+ }
/* Also ensure the callback only has a single scalar argument. */
if (sub->arg_cnt != 1 || sub->args[0].arg_type != ARG_ANYTHING) {
@@ -19291,6 +19477,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
return -EOPNOTSUPP;
}
+ /*
+ * An extension replaces the target outright, so it has to match
+ * the target's return convention. Its own return value is capped
+ * at 8 bytes (a >8 byte program return is rejected at BPF_EXIT),
+ * so it can never fill the R0:R2 pair the target's callers read.
+ * This cannot be left to btf_check_type_match() above, which
+ * compares return types by btf_type->info only: an int carries no
+ * vlen, so a 16-byte __int128 and an 8-byte long compare equal.
+ */
+ if (prog_extension && tgt_info->fmodel.ret_size > 8) {
+ bpf_log(log,
+ "Cannot replace function %s with a >8 byte return value\n",
+ tname);
+ return -EOPNOTSUPP;
+ }
+
/*
* *.multi programs don't need an address during program
* verification, we just take the module ref if needed.
@@ -20280,6 +20482,9 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
if (ret < 0)
goto skip_full_check;
+ /* must precede the first bpf_ret_reg_pair() user below */
+ bpf_compute_subprog_ret_regs(env);
+
ret = bpf_compute_live_registers(env);
if (ret < 0)
goto skip_full_check;
diff --git a/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c b/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
new file mode 100644
index 0000000000000..8be64d01ff4db
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
@@ -0,0 +1,206 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "aggregate_ret_int128_c.skel.h"
+#include "aggregate_ret_struct_c.skel.h"
+#include "aggregate_ret_union_c.skel.h"
+#include "aggregate_ret_kfunc_c.skel.h"
+#include "aggregate_ret_run.skel.h"
+#include "aggregate_ret_func.skel.h"
+#include "aggregate_ret_kfunc.skel.h"
+
+/*
+ * The bpf_testmod kfuncs returning more than 8 bytes are only built on x86_64
+ * and arm64 (see bpf_testmod.c); everywhere else the tests calling them are
+ * skipped.
+ */
+static bool has_ret_pair_kfuncs(void)
+{
+#if defined(__x86_64__) || defined(__aarch64__)
+ return true;
+#else
+ return false;
+#endif
+}
+
+static void run_prog(struct bpf_program *prog)
+{
+ char buf[64] = {};
+ int err, prog_fd;
+ LIBBPF_OPTS(bpf_test_run_opts, topts,
+ .data_in = buf,
+ .data_size_in = sizeof(buf),
+ .repeat = 1,
+ );
+
+ prog_fd = bpf_program__fd(prog);
+ err = bpf_prog_test_run_opts(prog_fd, &topts);
+ if (!ASSERT_OK(err, "test_run"))
+ return;
+
+ ASSERT_EQ(topts.retval, 0, "aggregate_ret_result");
+}
+
+/*
+ * Run @prog as subtest @name. Where the register-pair return is unsupported
+ * the subtest reports a skip instead, so that the list of subtests does not
+ * depend on the compiler or on the architecture; @prog is unused then and may
+ * be NULL, for a caller that could not even open its object.
+ */
+static void run_subtest(const char *name, struct bpf_program *prog, bool supported)
+{
+ if (!test__start_subtest(name))
+ return;
+
+ if (!supported) {
+ test__skip();
+ return;
+ }
+
+ run_prog(prog);
+}
+
+static void test_int128_c(void)
+{
+ struct aggregate_ret_int128_c *skel;
+
+ skel = aggregate_ret_int128_c__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_int128_c_open_load"))
+ return;
+
+ run_subtest("int128_c", skel->progs.aggregate_ret_int128_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ aggregate_ret_int128_c__destroy(skel);
+}
+
+static void test_struct_c(void)
+{
+ struct aggregate_ret_struct_c *skel;
+
+ skel = aggregate_ret_struct_c__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_struct_c_open_load"))
+ return;
+
+ run_subtest("struct_c", skel->progs.aggregate_ret_struct_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ run_subtest("global_struct_c", skel->progs.aggregate_ret_global_struct_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ aggregate_ret_struct_c__destroy(skel);
+}
+
+static void test_union_c(void)
+{
+ struct aggregate_ret_union_c *skel;
+
+ skel = aggregate_ret_union_c__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_union_c_open_load"))
+ return;
+
+ run_subtest("union_c", skel->progs.aggregate_ret_union_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ aggregate_ret_union_c__destroy(skel);
+}
+
+static void test_kfunc_c(void)
+{
+ struct aggregate_ret_kfunc_c *skel;
+ bool supported;
+ int err;
+
+ skel = aggregate_ret_kfunc_c__open();
+ if (!ASSERT_OK_PTR(skel, "skel_kfunc_c_open"))
+ return;
+
+ supported = skel->rodata->has_reg_pair_ret && has_ret_pair_kfuncs();
+
+ if (supported) {
+ /*
+ * Where the JIT cannot hand the second half of a >8-byte kfunc
+ * return back in R0:R2, bpf_add_kfunc_call() rejects the call
+ * with -EOPNOTSUPP. Asking the kernel keeps this test free of a
+ * list of the JITs that can, which would have to be updated as
+ * the rest of them learn.
+ */
+ err = aggregate_ret_kfunc_c__load(skel);
+ if (err == -EOPNOTSUPP)
+ supported = false;
+ else if (!ASSERT_OK(err, "skel_kfunc_c_load"))
+ goto out;
+ }
+
+ run_subtest("kfunc_int128_c", skel->progs.aggregate_ret_kfunc_int128_c_test,
+ supported);
+
+ run_subtest("kfunc_struct_c", skel->progs.aggregate_ret_kfunc_struct_c_test,
+ supported);
+
+out:
+ aggregate_ret_kfunc_c__destroy(skel);
+}
+
+static void test_run(void)
+{
+ struct aggregate_ret_run *skel;
+ bool kfunc_ok = true;
+ int err;
+
+ /*
+ * Every program in this object shares __kfunc_btf_root(), so where the
+ * testmod kfuncs are absent the object cannot load at all -- including
+ * for the kfunc-free "asm" subtest.
+ */
+ if (!has_ret_pair_kfuncs()) {
+ run_subtest("asm", NULL, false);
+ run_subtest("asm_kfunc", NULL, false);
+ run_subtest("struct", NULL, false);
+ run_subtest("union", NULL, false);
+ return;
+ }
+
+ skel = aggregate_ret_run__open();
+ if (!ASSERT_OK_PTR(skel, "skel_run_open"))
+ return;
+
+ err = aggregate_ret_run__load(skel);
+ if (err == -EOPNOTSUPP) {
+ kfunc_ok = false;
+ aggregate_ret_run__destroy(skel);
+
+ skel = aggregate_ret_run__open();
+ if (!ASSERT_OK_PTR(skel, "skel_run_reopen"))
+ return;
+
+ bpf_program__set_autoload(skel->progs.aggregate_ret_asm_kfunc_test, false);
+ bpf_program__set_autoload(skel->progs.aggregate_ret_struct_test, false);
+ bpf_program__set_autoload(skel->progs.aggregate_ret_union_test, false);
+
+ err = aggregate_ret_run__load(skel);
+ }
+ if (!ASSERT_OK(err, "skel_run_load"))
+ goto out;
+
+ run_subtest("asm", skel->progs.aggregate_ret_asm_test, true);
+ run_subtest("asm_kfunc", skel->progs.aggregate_ret_asm_kfunc_test, kfunc_ok);
+ run_subtest("struct", skel->progs.aggregate_ret_struct_test, kfunc_ok);
+ run_subtest("union", skel->progs.aggregate_ret_union_test, kfunc_ok);
+
+out:
+ aggregate_ret_run__destroy(skel);
+}
+
+void test_aggregate_ret(void)
+{
+ test_int128_c();
+ test_struct_c();
+ test_union_c();
+ test_kfunc_c();
+ test_run();
+
+ RUN_TESTS(aggregate_ret_func);
+ if (has_ret_pair_kfuncs())
+ RUN_TESTS(aggregate_ret_kfunc);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/exceptions.c b/tools/testing/selftests/bpf/prog_tests/exceptions.c
index 3588d6f97fd4e..71d00c568d802 100644
--- a/tools/testing/selftests/bpf/prog_tests/exceptions.c
+++ b/tools/testing/selftests/bpf/prog_tests/exceptions.c
@@ -5,6 +5,7 @@
#include "exceptions.skel.h"
#include "exceptions_ext.skel.h"
#include "exceptions_fail.skel.h"
+#include "exceptions_ret_pair_fail.skel.h"
#include "exceptions_assert.skel.h"
static char log_buf[1024 * 1024];
@@ -12,6 +13,7 @@ static char log_buf[1024 * 1024];
static void test_exceptions_failure(void)
{
RUN_TESTS(exceptions_fail);
+ RUN_TESTS(exceptions_ret_pair_fail);
}
static void test_exceptions_success(void)
diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
index 2523c07a16c65..0b54f911015c5 100644
--- a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
+++ b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
@@ -441,6 +441,19 @@ static void test_func_replace_int_with_void(void)
" doesn't match type INT of global_func2()");
}
+static void test_func_replace_ret_pair(void)
+{
+ const char *msg = "Cannot replace function agg_ret_target_func with a >8 byte return";
+
+ /*
+ * An extension cannot replace a function whose return value comes back
+ * in the R0:R2 pair: the extension's own return is capped at 8 bytes,
+ * so it would leave R2 stale for the target's callers.
+ */
+ test_obj_load_failure_common("freplace_ret_pair.bpf.o",
+ "./aggregate_ret_target.bpf.o", msg);
+}
+
static int find_prog_btf_id(const char *name, __u32 attach_prog_fd)
{
struct bpf_prog_info info = {};
@@ -660,6 +673,8 @@ void serial_test_fexit_bpf2bpf(void)
test_func_replace_progmap();
if (test__start_subtest("freplace_int_with_void"))
test_func_replace_int_with_void();
+ if (test__start_subtest("freplace_ret_pair"))
+ test_func_replace_ret_pair();
if (test__start_subtest("freplace_void"))
test_func_replace_void();
if (test__start_subtest("sleepable_fentry_to_xdp"))
diff --git a/tools/testing/selftests/bpf/prog_tests/timer.c b/tools/testing/selftests/bpf/prog_tests/timer.c
index 09ff21e1ad2f0..593e56d8964ea 100644
--- a/tools/testing/selftests/bpf/prog_tests/timer.c
+++ b/tools/testing/selftests/bpf/prog_tests/timer.c
@@ -6,6 +6,7 @@
#include <sys/syscall.h>
#include "timer.skel.h"
#include "timer_failure.skel.h"
+#include "timer_ret_pair_fail.skel.h"
#include "timer_interrupt.skel.h"
#define NUM_THR 8
@@ -285,6 +286,7 @@ void serial_test_timer(void)
test_timer(timer);
RUN_TESTS(timer_failure);
+ RUN_TESTS(timer_ret_pair_fail);
}
void serial_test_timer_stress(void)
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_func.c b/tools/testing/selftests/bpf/progs/aggregate_ret_func.c
new file mode 100644
index 0000000000000..0bc18450a46ef
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_func.c
@@ -0,0 +1,420 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+typedef unsigned __int128 u128;
+
+__naked u128 global_agg_good(void)
+{
+ asm volatile (
+ "r0 = 0x1234;" /* low 64 bits */
+ "r2 = 0x5678;" /* high 64 bits */
+ "exit;"
+ );
+}
+
+__naked u128 global_agg_bad(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "exit;"
+ );
+}
+
+__naked u128 global_agg_bad_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = r10;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_ret_global(void *ctx)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[global_agg_good];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : __imm(global_agg_good)
+ : "r0", "r1", "r2", "r3", "r4", "r5");
+ if (lo != 0x1234)
+ return 1;
+ if (hi != 0x5678)
+ return 2;
+ return 0;
+}
+
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_global_fail(void)
+{
+ asm volatile (
+ "call %[global_agg_bad];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : __imm(global_agg_bad)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("At subprogram exit the register R2 is not a scalar value")
+__naked int aggregate_ret_global_ptr_fail(void)
+{
+ asm volatile (
+ "call %[global_agg_bad_ptr];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : __imm(global_agg_bad_ptr)
+ : __clobber_all);
+}
+
+static __naked __noinline u128 static_agg_bad_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = r10;" /* stack pointer placed in the second return register */
+ "exit;"
+ );
+}
+
+/*
+ * R2 is caller-saved and only copied from the callee at exit; a PTR_TO_STACK
+ * left in it is turned into an uninitialized R2 in the caller. A caller that
+ * never reads R2 is therefore unaffected and loads fine.
+ */
+SEC("tc")
+__success __retval(0)
+__naked int aggregate_ret_static_ptr_unused(void)
+{
+ asm volatile (
+ "call %[static_agg_bad_ptr];"
+ "r0 = 0;" /* R2 holds a stack pointer but is never read */
+ "exit;"
+ :
+ : __imm(static_agg_bad_ptr)
+ : __clobber_all);
+}
+
+/* But a caller that does read the returned stack pointer is rejected. */
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_static_ptr_read_fail(void)
+{
+ asm volatile (
+ "call %[static_agg_bad_ptr];"
+ "r0 = r2;" /* using the returned stack pointer is rejected */
+ "exit;"
+ :
+ : __imm(static_agg_bad_ptr)
+ : __clobber_all);
+}
+
+static __naked __noinline u128 static_agg_no_r2(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_static_uninit_fail(void)
+{
+ asm volatile (
+ "call %[static_agg_no_r2];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : __imm(static_agg_no_r2)
+ : __clobber_all);
+}
+
+static __naked __noinline u128 static_agg_precise(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 4;" /* second half; its value is made precise below */
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+__log_level(2)
+__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1")
+__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8")
+__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10")
+__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7")
+__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2")
+__msg("mark_precise: frame0: regs=r2 stack= before 12: (95) exit")
+__msg("mark_precise: frame1: regs=r2 stack= before 11: (b7) r2 = 4")
+__naked int aggregate_ret_static_precise(void)
+{
+ asm volatile (
+ "call %[static_agg_precise];"
+ "r6 = r2;" /* derived from the aggregate's second half */
+ "r6 &= 7;" /* keep it in [0, 7] to index the stack */
+ "r1 = r10;"
+ "r1 += -8;"
+ "r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */
+ "r0 = 0;"
+ "*(u8 *)(r1 + 0) = r0;"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(static_agg_precise)
+ : __clobber_all);
+}
+
+SEC("tc")
+__success __retval(0)
+__log_level(2)
+__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1")
+__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8")
+__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10")
+__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7")
+__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2")
+__msg("mark_precise: frame0: regs=r2 stack= before 0: (85) call pc+9")
+__naked int aggregate_ret_global_precise(void)
+{
+ asm volatile (
+ "call %[global_agg_good];"
+ "r6 = r2;" /* derived from the aggregate's second half */
+ "r6 &= 7;" /* keep it in [0, 7] to index the stack */
+ "r1 = r10;"
+ "r1 += -8;"
+ "r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */
+ "r0 = 0;"
+ "*(u8 *)(r1 + 0) = r0;"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_agg_good)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("return value larger than 8 bytes is not supported at program exit")
+__naked u128 aggregate_ret_entry_fail(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ );
+}
+
+#if __clang_major__ >= 23
+
+struct pair {
+ __u64 hi;
+ __u64 lo;
+};
+
+union upair {
+ __u64 halves[2];
+ struct {
+ __u64 lo;
+ __u64 hi;
+ } parts;
+};
+
+/* A by-value struct that smuggles a pointer, which must be rejected. */
+struct with_ptr {
+ void *p;
+ __u64 x;
+};
+
+/* A by-value union that smuggles a pointer, which must be rejected too. */
+union upair_with_ptr {
+ void *p;
+ __u64 halves[2];
+};
+
+/* Global subprogram returning a scalar-only 16-byte struct in R0:R2. */
+__naked struct pair global_ret_struct(void)
+{
+ asm volatile (
+ "r0 = 0x1234;" /* struct's first half */
+ "r2 = 0x5678;" /* struct's second half */
+ "exit;"
+ );
+}
+
+/* Global subprogram returning a scalar-only 16-byte union in R0:R2. */
+__naked union upair global_ret_union(void)
+{
+ asm volatile (
+ "r0 = 0x1234;"
+ "r2 = 0x5678;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_ret_global_struct(void *ctx)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[global_ret_struct];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : __imm(global_ret_struct)
+ : "r0", "r1", "r2", "r3", "r4", "r5");
+ if (lo != 0x1234)
+ return 1;
+ if (hi != 0x5678)
+ return 2;
+ return 0;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_ret_global_union(void *ctx)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[global_ret_union];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : __imm(global_ret_union)
+ : "r0", "r1", "r2", "r3", "r4", "r5");
+ if (lo != 0x1234)
+ return 1;
+ if (hi != 0x5678)
+ return 2;
+ return 0;
+}
+
+__naked struct with_ptr global_ret_struct_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__failure __msg("Global function global_ret_struct_ptr() has unsupported return type")
+__naked int aggregate_ret_global_struct_ptr_fail(void)
+{
+ asm volatile (
+ "call %[global_ret_struct_ptr];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_ret_struct_ptr)
+ : __clobber_all);
+}
+
+__naked union upair_with_ptr global_ret_union_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__failure __msg("Global function global_ret_union_ptr() has unsupported return type")
+__naked int aggregate_ret_global_union_ptr_fail(void)
+{
+ asm volatile (
+ "call %[global_ret_union_ptr];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_ret_union_ptr)
+ : __clobber_all);
+}
+
+#endif /* __clang_major__ >= 23 */
+
+static __naked u128 agg_callee(void)
+{
+ asm volatile (
+ "r0 = 1;"
+ "r2 = 2;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__log_level(2)
+__msg("Live regs before insn:")
+/*
+ * R2 is read at the exit of agg_callee() (insn 5), which returns a pair, but
+ * not at the exit of this program (insn 2), which returns an int.
+ */
+__msg("0: .12345.... (85) call pc+2")
+__msg("1: ..2....... (bf) r0 = r2")
+__msg("2: 0......... (95) exit")
+__msg("3: .......... (b7) r0 = 1")
+__msg("4: 0......... (b7) r2 = 2")
+__msg("5: 0.2....... (95) exit")
+__naked int aggregate_ret_live(void)
+{
+ asm volatile (
+ "call %[agg_callee];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : [agg_callee]"i"(agg_callee)
+ : __clobber_all);
+}
+
+/*
+ * A static subprogram is verified inline, so prepare_func_exit() hands the
+ * caller the callee's actual R0:R2 register state rather than an opaque scalar
+ * pair. A pointer in the returned struct therefore stays tracked and is usable
+ * by the caller, which is why btf_validate_return_type() does not apply the
+ * scalar-only restriction to a local function. Return the context pointer as
+ * the upper half and dereference it in the caller.
+ */
+struct ptr_pair {
+ void *p;
+ __u64 x;
+};
+
+static __naked __noinline struct ptr_pair static_ret_ptr_pair(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = r1;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+__naked int aggregate_ret_static_ptr_pair(void)
+{
+ asm volatile (
+ "call %[static_ret_ptr_pair];"
+ "r1 = *(u32 *)(r2 + 0);" /* deref the returned ctx pointer */
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(static_ret_ptr_pair)
+ : __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c
new file mode 100644
index 0000000000000..f2e09c8be0bed
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+typedef unsigned __int128 u128;
+
+static __noinline u128 make_i128(__u64 a, __u64 b)
+{
+ return ((u128)(a + b) << 64) | (a - b);
+}
+
+SEC("tc")
+int aggregate_ret_int128_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v;
+
+ v = make_i128(a, b);
+ if ((__u64)(v >> 64) != a + b)
+ return 1;
+ if ((__u64)v != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_int128_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
new file mode 100644
index 0000000000000..617724aa70156
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+/*
+ * Reference kfunc addresses to force those BTF to be emitted. Taking the address
+ * (rather than calling) avoids any dependence on the compiler lowering an
+ * __int128 or struct return value, which the BPF backend only supports from
+ * LLVM 23 on.
+ */
+void __kfunc_btf_root(void)
+{
+ asm volatile (""
+ :
+ : "r"(&bpf_kfunc_call_test_i128),
+ "r"(&bpf_kfunc_call_test_ret_fastcall),
+ "r"(&bpf_kfunc_call_test_ret_ptr),
+ "r"(&bpf_kfunc_call_test_ret_ii),
+ "r"(&bpf_kfunc_call_test_ret_big));
+}
+
+/*
+ * bpf_add_kfunc_call() rejects a kfunc returning more than 8 bytes unless the
+ * JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), so a test that has to
+ * get past it is tagged with the architectures implementing it. The two tests
+ * below that are rejected earlier, on KF_FASTCALL or on reading R2 after an
+ * 8-byte struct return, behave the same everywhere and are not tagged.
+ */
+
+SEC("tc")
+__arch_x86_64 __arch_arm64 __arch_riscv64
+__success __retval(0)
+__log_level(2)
+__msg("mark_precise: frame0: last_idx 7 first_idx 0 subseq_idx -1")
+__msg("mark_precise: frame0: regs=r6 stack= before 6: (07) r1 += -8")
+__msg("mark_precise: frame0: regs=r6 stack= before 5: (bf) r1 = r10")
+__msg("mark_precise: frame0: regs=r6 stack= before 4: (57) r6 &= 7")
+__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r6 = r2")
+__msg("mark_precise: frame0: regs=r2 stack= before 2: (85) call bpf_kfunc_call_test_i128")
+__naked int aggregate_ret_kfunc_precise(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 2;"
+ "call %[bpf_kfunc_call_test_i128];"
+ "r6 = r2;" /* second return half */
+ "r6 &= 7;" /* keep it in [0, 7] to index the stack */
+ "r1 = r10;"
+ "r1 += -8;"
+ "r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */
+ "r0 = 0;"
+ "*(u8 *)(r1 + 0) = r0;"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_i128)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("kfunc bpf_kfunc_call_test_ret_fastcall with >8-byte return is not supported with KF_FASTCALL")
+__naked int aggregate_ret_kfunc_fastcall_fail(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 2;"
+ "call %[bpf_kfunc_call_test_ret_fastcall];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_fastcall)
+ : __clobber_all);
+}
+
+SEC("tc")
+__arch_x86_64 __arch_arm64 __arch_riscv64
+__failure __msg("is not composed of scalars")
+__naked int aggregate_ret_kfunc_ptr_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "call %[bpf_kfunc_call_test_ret_ptr];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_ptr)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_kfunc_small_no_r2(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "call %[bpf_kfunc_call_test_ret_ii];"
+ "r0 = r2;" /* R2 is not a return register for a <=8 byte struct */
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_ii)
+ : __clobber_all);
+}
+
+/*
+ * A return value larger than 16 bytes does not fit in R0:R2 and is rejected by
+ * btf_distill_func_proto(), before the KF_FASTCALL and JIT-capability checks,
+ * so this behaves the same on every architecture.
+ */
+SEC("tc")
+__failure __msg("The function bpf_kfunc_call_test_ret_big return type STRUCT is unsupported")
+__naked int aggregate_ret_kfunc_too_big_fail(void)
+{
+ asm volatile (
+ "call %[bpf_kfunc_call_test_ret_big];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_big)
+ : __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c
new file mode 100644
index 0000000000000..fd000620f3141
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+typedef unsigned __int128 u128;
+
+SEC("tc")
+int aggregate_ret_kfunc_int128_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v;
+
+ v = bpf_kfunc_call_test_i128(a, b);
+ if ((__u64)(v >> 64) != a + b)
+ return 1;
+ if ((__u64)v != a - b)
+ return 2;
+
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_ret_pair p;
+
+ p = bpf_kfunc_call_test_ret_pair(a, b);
+ if (p.hi != a + b)
+ return 1;
+ if (p.lo != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_kfunc_int128_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_run.c b/tools/testing/selftests/bpf/progs/aggregate_ret_run.c
new file mode 100644
index 0000000000000..382ef3b90037c
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_run.c
@@ -0,0 +1,178 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+typedef unsigned __int128 u128;
+
+/*
+ * Reference kfunc addresses to force those BTF to be emitted. Taking the address
+ * (rather than calling) avoids any dependence on the compiler lowering an __int128
+ * or struct return value, which the BPF backend only supports from LLVM 23 on.
+ */
+void __kfunc_btf_root(void)
+{
+ asm volatile (""
+ :
+ : "r"(&bpf_kfunc_call_test_i128),
+ "r"(&bpf_kfunc_call_test_ret_pair),
+ "r"(&bpf_kfunc_call_test_ret_li),
+ "r"(&bpf_kfunc_call_test_ret_ii),
+ "r"(&bpf_kfunc_call_test_ret_uu));
+}
+
+#define I128_ASM_LO 0xABCDabcd12345678ULL
+#define I128_ASM_HI 0x1234567890abcdefULL
+
+static __naked __noinline u128 make_i128_asm(void)
+{
+ asm volatile (
+ "r0 = %[lo] ll;" /* low 64 bits */
+ "r2 = %[hi] ll;" /* high 64 bits */
+ "exit;"
+ :
+ : __imm_const(lo, I128_ASM_LO), __imm_const(hi, I128_ASM_HI)
+ );
+}
+
+SEC("tc")
+int aggregate_ret_asm_test(struct __sk_buff *skb)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[callee];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [callee]"i"(make_i128_asm)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != I128_ASM_LO)
+ return 1;
+ if (hi != I128_ASM_HI)
+ return 2;
+
+ return 0;
+}
+
+/*
+ * R0 holds bytes 0..7 of a kfunc return value and R2 bytes 8..15, so where a
+ * member sits inside a register depends on the endianness of the target.
+ * Although arm64 supports both little and big endian, for simplicity, only
+ * do little endian for now..
+ */
+SEC("tc")
+int aggregate_ret_asm_kfunc_test(struct __sk_buff *skb)
+{
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ __u64 a = skb->len;
+ __u64 b = skb->len ^ 0xdeadbeefULL;
+ __u64 lo, hi;
+
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_i128)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (hi != a + b)
+ return 1;
+ if (lo != a - b)
+ return 2;
+#endif
+
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_struct_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len;
+ __u64 b = skb->len ^ 0xdeadbeefULL;
+ __u64 lo, hi;
+
+ /* struct { u64 hi; u64 lo; }: R0 = hi, R2 = lo. */
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_pair)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != a + b)
+ return 1;
+ if (hi != a - b)
+ return 2;
+
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ /* struct { u64 a; int b; }: R0 = a, low 32 bits of R2 = b. */
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_li)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != a)
+ return 3;
+ if ((int)hi != ~(int)b)
+ return 4;
+
+ /* struct { int a; int b; }: 8 bytes, packed into R0; R2 is not used. */
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ : [lo]"=r"(lo)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_ii)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if ((int)lo != (int)a)
+ return 5;
+ if ((int)(lo >> 32) != (int)b)
+ return 6;
+#endif
+
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_union_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len;
+ __u64 b = skb->len ^ 0xdeadbeefULL;
+ __u64 lo, hi;
+
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_uu)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != a + b)
+ return 1;
+ if (hi != a - b)
+ return 2;
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c
new file mode 100644
index 0000000000000..5296e41da3f00
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+struct pair {
+ __u64 hi; /* R0 */
+ __u64 lo; /* R2 */
+};
+
+static __noinline struct pair make_pair(__u64 a, __u64 b)
+{
+ struct pair p = { .hi = a + b, .lo = a - b };
+
+ return p;
+}
+
+SEC("tc")
+int aggregate_ret_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p;
+
+ p = make_pair(a, b);
+ if (p.hi != a + b)
+ return 1;
+ if (p.lo != a - b)
+ return 2;
+
+ return 0;
+}
+
+__noinline struct pair make_pair_global(__u64 a, __u64 b)
+{
+ struct pair p = { .hi = a + b, .lo = a - b };
+
+ return p;
+}
+
+SEC("tc")
+int aggregate_ret_global_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p;
+
+ p = make_pair_global(a, b);
+ if (p.hi != a + b)
+ return 1;
+ if (p.lo != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_struct_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_global_struct_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_target.c b/tools/testing/selftests/bpf/progs/aggregate_ret_target.c
new file mode 100644
index 0000000000000..cffd8d7d3241a
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_target.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+/* freplace target: a global subprogram returning 16 bytes in R0:R2. */
+__naked unsigned __int128 agg_ret_target_func(void)
+{
+ asm volatile (
+ "r0 = 0x1234;"
+ "r2 = 0x5678;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__naked int agg_ret_target(void)
+{
+ asm volatile (
+ "call %[agg_ret_target_func];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(agg_ret_target_func)
+ : __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
new file mode 100644
index 0000000000000..5547fa6cbd495
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+union pair {
+ __u64 halves[2];
+ struct {
+ __u64 lo; /* R0 */
+ __u64 hi; /* R2 */
+ } parts;
+};
+
+static __noinline union pair make_pair(__u64 a, __u64 b)
+{
+ union pair p;
+
+ p.halves[0] = a + b;
+ p.halves[1] = a - b;
+ return p;
+}
+
+SEC("tc")
+int aggregate_ret_union_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ union pair p;
+
+ p = make_pair(a, b);
+ if (p.parts.lo != a + b)
+ return 1;
+ if (p.parts.hi != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_union_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c
new file mode 100644
index 0000000000000..a45db5d9c1d44
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)
+{
+ for (;;)
+ ;
+}
diff --git a/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c
new file mode 100644
index 0000000000000..35506c7c5a916
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer)
+{
+ for (;;)
+ ;
+}
diff --git a/tools/testing/selftests/bpf/progs/exceptions_fail.c b/tools/testing/selftests/bpf/progs/exceptions_fail.c
index ac44d60e50666..9708efb93683b 100644
--- a/tools/testing/selftests/bpf/progs/exceptions_fail.c
+++ b/tools/testing/selftests/bpf/progs/exceptions_fail.c
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)
SEC("?tc")
__exception_cb(exception_cb_bad_ret_type1)
-__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.")
+__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")
int reject_exception_cb_type_1(struct __sk_buff *ctx)
{
bpf_throw(0);
diff --git a/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c
new file mode 100644
index 0000000000000..842f86ad8659e
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_helpers.h>
+
+#include "bpf_misc.h"
+#include "bpf_experimental.h"
+
+__naked __noinline __used
+unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)
+{
+ asm volatile (
+ "r0 = r1;"
+ "r2 = 0;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+SEC("?tc")
+__exception_cb(exception_cb_bad_ret_type3)
+__failure __msg("exception cb cannot return value larger than 8 bytes")
+__btf_func_path("btf__exceptions_ret_pair_fail.bpf.o")
+int reject_exception_cb_ret_pair(void *ctx)
+{
+ bpf_throw(0);
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/freplace_ret_pair.c b/tools/testing/selftests/bpf/progs/freplace_ret_pair.c
new file mode 100644
index 0000000000000..84b701402ca67
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/freplace_ret_pair.c
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+/*
+ * An extension replaces its target outright, so it has to match the target's
+ * return convention. Its own return value is capped at 8 bytes, so it can
+ * never fill the R0:R2 pair that the target's callers read, and the attach is
+ * rejected. btf_check_type_match() cannot catch this: it compares return types
+ * by btf_type->info only, and an int carries no vlen, so the __u64 here and
+ * the target's __int128 compare equal.
+ */
+SEC("freplace/agg_ret_target_func")
+__u64 new_agg_ret_target_func(void)
+{
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c
new file mode 100644
index 0000000000000..29fd294dfd49b
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <linux/bpf.h>
+#include <time.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct elem {
+ struct bpf_timer t;
+};
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, struct elem);
+} timer_map SEC(".maps");
+
+__naked __noinline __used
+static unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ ::: __clobber_all
+ );
+}
+
+SEC("fentry/bpf_fentry_test1")
+__failure __msg("callback function with >8-byte return value is not supported")
+__btf_func_path("btf__timer_ret_pair_fail.bpf.o")
+long BPF_PROG2(test_bad_ret_pair, int, a)
+{
+ int key = 0;
+ struct bpf_timer *timer;
+
+ timer = bpf_map_lookup_elem(&timer_map, &key);
+ if (timer) {
+ bpf_timer_init(timer, &timer_map, CLOCK_BOOTTIME);
+ bpf_timer_set_callback(timer, timer_cb_ret_pair);
+ }
+
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/verifier_arena.c b/tools/testing/selftests/bpf/progs/verifier_arena.c
index b241bbcf54a8a..455b55296f355 100644
--- a/tools/testing/selftests/bpf/progs/verifier_arena.c
+++ b/tools/testing/selftests/bpf/progs/verifier_arena.c
@@ -704,4 +704,42 @@ int check_arena_arg_ret(void *ctx)
return 0;
}
+struct arena_ret_pair {
+ __u64 lo;
+ __u64 hi;
+};
+
+/*
+ * A 16-byte value is returned in the R0:R2 register pair. A global subprogram
+ * may return an arena pointer in R0, but R2 holds the upper half of a scalar
+ * pair, so an arena pointer there is not a valid return value. The ld_imm64 of
+ * the arena map is what links the arena to the program, without which the
+ * addr_space_cast insn is not allowed.
+ */
+__naked struct arena_ret_pair global_ret_arena_ptr_in_r2(void)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r2 = 8192;"
+ "r2 = addr_space_cast(r2, 0x0, 0x1);"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm_addr(arena)
+ : __clobber_all);
+}
+
+SEC("syscall")
+__failure __msg("At subprogram exit the register R2 is not a scalar value (arena)")
+__naked int check_global_ret_arena_ptr_in_r2(void)
+{
+ asm volatile (
+ "call %[global_ret_arena_ptr_in_r2];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_ret_arena_ptr_in_r2)
+ : __clobber_all);
+}
+
char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index a6133f7521f34..9ef95983da3ca 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -939,6 +939,83 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)
return 0;
}
+/*
+ * A kfunc is only usable where the ABI hands its return value back in
+ * registers. s390x, for example, returns a by-value struct or union through a
+ * hidden pointer argument (sret) whatever its size. That pointer shifts every
+ * declared argument by one register, and pahole, which maps parameters to
+ * registers positionally, then skips the function with "unexpected register
+ * usage for parameter". resolve_btfids reports "no BTF func for kfunc" and
+ * leaves the ID at 0, which makes register_btf_kfunc_id_set() fail at module
+ * init, so the module does not load at all.
+ *
+ * Restrict these kfuncs to the architectures where the return value comes back
+ * in registers. A kfunc taking no argument has nothing for the sret pointer to
+ * displace and needs no guard, whatever it returns.
+ */
+#if defined(__x86_64__) || defined(__aarch64__)
+__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b)
+{
+ return (__int128)(((unsigned __int128)(a + b) << 64) | (a - b));
+}
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)
+{
+ struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b };
+
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)
+{
+ struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b };
+
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_li bpf_kfunc_call_test_ret_li(u64 a, int b)
+{
+ struct prog_test_ret_li r = { .a = a, .b = ~b };
+
+ return r;
+}
+
+__bpf_kfunc union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(u64 a, u64 b)
+{
+ union prog_test_ret_uu r;
+
+ r.halves[0] = a + b;
+ r.halves[1] = a - b;
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
+{
+ struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
+
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)
+{
+ struct prog_test_ret_ii r = { .a = a, .b = b };
+
+ return r;
+}
+#endif /* __x86_64__ || __aarch64__ */
+
+/*
+ * Takes no argument on purpose: with no arguments there is nothing for the sret
+ * pointer to displace, so this needs no architecture guard even though it
+ * returns 24 bytes. See the comment on bpf_kfunc_call_test_i128() above.
+ */
+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)
+{
+ struct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };
+
+ return r;
+}
+
__bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
u64 e, u64 f, u64 g, u64 h,
u64 i, u64 j)
@@ -1472,6 +1549,16 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2)
BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
BTF_ID_FLAGS(func, bpf_kfunc_call_test4)
BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
+#if defined(__x86_64__) || defined(__aarch64__)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_fastcall, KF_FASTCALL)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_li)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_uu)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ptr)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
+#endif
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_big)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index c4383acb53c11..755973793e9b5 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -55,6 +55,44 @@ struct prog_test_big_arg {
__u64 b;
};
+/*
+ * A 16-byte struct returned by value from a kfunc: .hi comes back in R0 and
+ * .lo in R2.
+ */
+struct prog_test_ret_pair {
+ __u64 hi;
+ __u64 lo;
+};
+
+struct prog_test_ret_li { /* 16 bytes: R0:R2 */
+ __u64 a;
+ int b;
+};
+
+struct prog_test_ret_ii { /* 8 bytes: R0 only */
+ int a;
+ int b;
+};
+
+union prog_test_ret_uu { /* 16 bytes: R0:R2 */
+ __u64 halves[2];
+ struct {
+ __u64 lo;
+ __u64 hi;
+ } parts;
+};
+
+struct prog_test_ret_ptr { /* 16 bytes: contains a pointer */
+ void *p;
+ __u64 tag;
+};
+
+struct prog_test_ret_big { /* 24 bytes: too large for R0:R2 */
+ __u64 a;
+ __u64 b;
+ __u64 c;
+};
+
struct prog_test_fail1 {
void *p;
int x;
@@ -130,6 +168,16 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
+#ifdef __SIZEOF_INT128__
+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
+#endif
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_li bpf_kfunc_call_test_ret_li(__u64 a, int b) __ksym;
+struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
+union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;
+struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym;
__u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
__u64 e, __u64 f, __u64 g, __u64 h,
__u64 i, __u64 j) __ksym;
|
| 3/2 | 2026/08/11 00:58 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 35622 output: 208 thoughts: 1613 Thoughts: **Fuzzing Target Identification: Extended Kfunc Return Values** Okay, so we have this patch, and it's making a significant change: kfuncs can now return values larger than 8 bytes, up to 16, using the R0:R2 register pair. This touches the BPF verifier and JIT compilers, including x86_64, arm64, and riscv64. Naturally, this is a change that screams for fuzzing. My first thought is to identify the key symbols. The patch adds `bpf_jit_supports_kfunc_ret_reg_pair` to several architectures, and modifies a ton of core BPF logic. I see modifications in the verifier (`kernel/bpf/verifier.c`), BTF (`kernel/bpf/btf.c`), backtrack (`kernel/bpf/backtrack.c`), liveness analysis (`kernel/bpf/liveness.c`), and JIT compilation (`arch/x86/net/bpf_jit_comp.c`), which is great because this means the impact is significant. Now, let's drill down to find the important API entrypoints - the non-static, non-inlined functions that we should focus on. Initially, I identified `bpf_get_kfunc_ret_size`, `__btf_type_is_scalar_struct`, `bpf_jit_supports_kfunc_ret_reg_pair`, `bpf_compute_live_registers`, `bpf_check`, `bpf_check_attach_target`, and `btf_prepare_func_args`. While `bpf_check` is the main verifier entry point and thus a high-value target for generic fuzzing, it might be a bit too broad for the specific changes in this patch. I want to narrow things down. I am going to focus on the functions that are directly tied to the new return value logic: `bpf_get_kfunc_ret_size`, `__btf_type_is_scalar_struct`, and `bpf_jit_supports_kfunc_ret_reg_pair`. Let's see... looking at the code for these, they're all global. Okay, I've got a handle on the key functions, so let's make a shortlist to test. I will include `btf_distill_func_proto` and `btf_prepare_func_args` as well, because they were also modified. The required configurations are `CONFIG_BPF_SYSCALL` and `CONFIG_BPF_JIT`. I am confident in this selection: * `bpf_get_kfunc_ret_size` * `__btf_type_is_scalar_struct` * `bpf_jit_supports_kfunc_ret_reg_pair` * `btf_distill_func_proto` * `btf_prepare_func_args` With that, I am ready to set my fuzzing targets. |
| 4/2 | 2026/08/11 00:59 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"BPF_SYSCALL"
],
"FocusSymbols": [
"bpf_get_kfunc_ret_size",
"__btf_type_is_scalar_struct",
"bpf_jit_supports_kfunc_ret_reg_pair",
"btf_distill_func_proto",
"btf_prepare_func_args"
],
"Reasoning": "The patch introduces support for BPF kfuncs returning values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. This involves significant changes to the BPF verifier (register tracking, liveness, backtracking, BTF validation) and JIT compilers (x86_64, arm64, riscv64). These are core functional changes to the BPF subsystem that are reachable from userspace and should be fuzzed.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"BPF_SYSCALL"
],
"FocusSymbols": [
"bpf_get_kfunc_ret_size",
"__btf_type_is_scalar_struct",
"bpf_jit_supports_kfunc_ret_reg_pair",
"btf_distill_func_proto",
"btf_prepare_func_args"
],
"Reasoning": "The patch introduces support for BPF kfuncs returning values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. This involves significant changes to the BPF verifier (register tracking, liveness, backtracking, BTF validation) and JIT compilers (x86_64, arm64, riscv64). These are core functional changes to the BPF subsystem that are reachable from userspace and should be fuzzed.",
"WorthFuzzing": true
} |
| 5/1 | 2026/08/11 00:59 | action | If |
0mArgs:{
"WorthFuzzing": true
}Results: null |
| 6/2 | 2026/08/11 00:59 | agent | kmsan-evaluator |
0mModel:best-expensive Args: null Results: {
"KMSANReasoning": "The patch series introduces support for BPF kfuncs and subprograms to return values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. The changes are primarily within the BPF verifier (to track and validate the R2 register) and the JIT compilers (to properly move the return values into the expected BPF registers). \n\nFrom an uninitialized memory perspective:\n1. The BPF verifier statically enforces that BPF subprograms fully initialize both R0 and R2 before returning (via `check_reg_arg(..., SRC_OP)`).\n2. If a static subprogram leaks a stack pointer in R2, the verifier safely marks it as uninitialized (`bpf_mark_reg_not_init`), preventing the caller from reading it. This is a static verifier check, not a dynamic kernel memory issue.\n3. The patch does not introduce any new kernel-to-user data structures, complex allocations, or bounds checking logic that could expose uninitialized kernel memory.\n4. The only new kfuncs introduced are for testing purposes (`bpf_testmod`), and their return structures are fully initialized (e.g., using designated initializers which zero-fill padding).\n\nSince the risk of uninitialized memory usage or info-leaks is statically mitigated by the verifier and no new uninitialized kernel memory paths are introduced, a dedicated KMSAN fuzzing session is not required. Standard KASAN and BPF verifier tests are sufficient.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: For your convenience, here is the diff of the changes:
commit e58dc654dc59f196a5aa28b5e90faf612f8b4f83
Author: syz-cluster <triage@syzkaller.com>
Date: Tue Aug 11 00:58:42 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
index 1004eb0bec617..71d3ff8695994 100644
--- a/Documentation/bpf/kfuncs.rst
+++ b/Documentation/bpf/kfuncs.rst
@@ -575,6 +575,68 @@ is also covered by this recovery. A kfunc handed an arena pointer may
therefore access up to ``GUARD_SZ / 2`` past it without bounds-checking
against the arena. Larger accesses must verify the range explicitly.
+2.9 kfunc Return Values
+-----------------------
+
+A kfunc may return a scalar, a pointer, or a small struct or union by
+value. A scalar or pointer of up to 8 bytes is returned in R0, as usual.
+
+A struct or union returned by value must be composed only of scalars
+(recursively), where a scalar is an integer or an enum; arrays of scalars are
+allowed as members. Its bytes are handed back to the program as the raw
+contents of R0 (and R2), so a pointer field would be laundered into a scalar
+and escape the verifier's pointer provenance and reference tracking. A struct
+or union with a pointer member is therefore rejected at load time, and so is
+one with a floating-point member, which the ABI may not return in R0:R2 at
+all.
+
+A kfunc may also return a value larger than 8 bytes and up to 16 bytes -- a
+scalar-only struct or union, or an ``__int128``. Such a value is returned
+in the register pair R0:R2, matching the convention LLVM uses for the BPF
+target: the first 8 bytes in R0 and the second 8 bytes in R2. A struct or
+union of 8 bytes or less is returned in R0 alone.
+
+::
+
+ struct bpf_pair { __u64 a, b; }; /* 16 bytes */
+
+ __bpf_kfunc struct bpf_pair bpf_kfunc_get_pair(void)
+ {
+ struct bpf_pair p = { .a = 1, .b = 2 };
+
+ return p; /* p.a in R0, p.b in R2 */
+ }
+
+Returning a value in the R0:R2 pair requires the JIT to place the second
+half of the return value into R2, which not every architecture supports
+right now. A kfunc with a return value larger than 8 bytes is therefore
+rejected at load time on a JIT that does not advertise this capability (see
+``bpf_jit_supports_kfunc_ret_reg_pair()``), and such a program is never run
+by the interpreter. A return value larger than 16 bytes is not supported.
+
+The same R0:R2 convention applies to a BPF subprogram, global or static,
+that returns an ``__int128`` or a struct or union larger than 8 bytes. Such a
+program also requires the JIT, since the interpreter propagates only R0 out
+of a subprogram. A global subprogram is verified in isolation, so its
+by-value struct or union return is restricted to scalars just like a kfunc's;
+a static subprogram is verified inline and has no such restriction. The main
+program cannot return more than 8 bytes, as its return value is the program's
+exit code.
+
+A global subprogram must leave a scalar in *every* register of the pair, so
+both halves of the returned value have to be assigned. Leaving the upper half
+uninitialized is not merely untidy: the compiler is then free to leave R2
+holding whatever it happened to hold, which for a subprogram taking a pointer
+argument is typically that pointer. Handing the caller an unknown scalar built
+from a pointer is a leak, so the verifier rejects it with::
+
+ At subprogram exit the register R2 is not a scalar value (...)
+
+Initialize the whole return value, for example ``struct pair p = {};``, to
+avoid this. A static subprogram is exempt: it is verified inline, so an
+unassigned R2 is simply passed back to the caller as uninitialized and only a
+caller that reads it fails.
+
.. _BPF_kfunc_lifecycle_expectations:
3. kfunc lifecycle expectations
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index d14d297ebb967..a1febb4c5718c 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2330,6 +2330,11 @@ bool bpf_jit_supports_kfunc_call(void)
return true;
}
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return true;
+}
+
bool bpf_jit_supports_stack_args(void)
{
return true;
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index 6b9972b07c1b0..c8e94d4657e9b 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -2111,6 +2111,11 @@ bool bpf_jit_supports_kfunc_call(void)
return true;
}
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return true;
+}
+
bool bpf_jit_supports_ptr_xchg(void)
{
return true;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 8dddb5d7af21b..01c34114c8502 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2647,6 +2647,22 @@ st: insn_off = insn->off;
return -EINVAL;
if (priv_frame_ptr)
pop_r9(&prog);
+ if (src_reg == BPF_PSEUDO_KFUNC_CALL) {
+ const struct btf_func_model *fm;
+
+ /*
+ * A kfunc returning a >8 byte aggregate hands the
+ * second half back in RDX (the native ABI's second
+ * return reg), but BPF expects it in R0:R2. BPF R0
+ * is RAX (no move needed), while BPF R2 is RSI, so
+ * copy RDX into RSI.
+ */
+ fm = bpf_jit_find_kfunc_model(bpf_prog, insn);
+ if (!fm)
+ return -EFAULT;
+ if (fm->ret_size > 8)
+ emit_mov_reg(&prog, true, BPF_REG_2, BPF_REG_3);
+ }
break;
}
@@ -4137,6 +4153,11 @@ bool bpf_jit_supports_kfunc_call(void)
return true;
}
+bool bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return true;
+}
+
bool bpf_jit_supports_stack_args(void)
{
return true;
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 93f7c2075eeaa..1f516daa41dbf 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -811,6 +811,8 @@ struct bpf_subprog_info {
bool is_async_cb: 1;
bool is_exception_cb: 1;
bool args_cached: 1;
+ /* true if the return value is passed in the R0:R2 register pair */
+ bool ret_reg_pair: 1;
/* true if bpf_fastcall stack region is used by functions that can't be inlined */
bool keep_fastcall_stack: 1;
bool changes_pkt_data: 1;
@@ -1044,6 +1046,16 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
return &env->subprog_info[subprog];
}
+/*
+ * True if @subprog returns its value in the R0:R2 register pair. Cached by
+ * bpf_compute_subprog_ret_regs(), since this is queried on hot paths: at
+ * every subprogram call and at every subprogram exit.
+ */
+static inline bool bpf_ret_reg_pair(struct bpf_verifier_env *env, int subprog)
+{
+ return subprog_info(env, subprog)->ret_reg_pair;
+}
+
struct bpf_call_summary {
u8 num_params;
bool is_void;
@@ -1435,6 +1447,10 @@ int bpf_jmp_offset(struct bpf_insn *insn);
struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);
void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);
bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog);
+int bpf_get_kfunc_ret_size(const struct bpf_prog *prog, u32 func_id,
+ u16 btf_fd_idx, u8 *ret_size);
+bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
+ const struct btf_type *t, int rec);
int bpf_find_subprog(struct bpf_verifier_env *env, int off);
bool bpf_is_throw_kfunc(struct bpf_insn *insn);
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 4edba8182db1b..f3c34fd70f2d2 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1213,6 +1213,7 @@ bool bpf_jit_inlines_helper_call(s32 imm);
bool bpf_jit_supports_subprog_tailcalls(void);
bool bpf_jit_supports_percpu_insn(void);
bool bpf_jit_supports_kfunc_call(void);
+bool bpf_jit_supports_kfunc_ret_reg_pair(void);
bool bpf_jit_supports_stack_args(void);
bool bpf_jit_supports_arena_args(void);
bool bpf_jit_supports_far_kfunc_call(void);
diff --git a/kernel/bpf/backtrack.c b/kernel/bpf/backtrack.c
index 40bd04421a991..fc8ecad6f01b0 100644
--- a/kernel/bpf/backtrack.c
+++ b/kernel/bpf/backtrack.c
@@ -425,6 +425,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
*/
verifier_bug_if(idx + 1 != subseq_idx, env,
"extra insn from subprog");
+ /*
+ * a global subprog returning more than 8 bytes
+ * sets R2 as well. R2 is part of the args mask
+ * checked just below, so it has to be cleared
+ * here rather than next to R0.
+ */
+ if (bt_is_reg_set(bt, BPF_REG_2) &&
+ bpf_ret_reg_pair(env, subprog))
+ bt_clear_reg(bt, BPF_REG_2);
/* r1-r5 are invalidated after subprog call,
* so for global func call it shouldn't be set
* anymore
@@ -508,6 +517,19 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
return -ENOTSUPP;
/* regular helper call sets R0 */
bt_clear_reg(bt, BPF_REG_0);
+ /* a kfunc returning more than 8 bytes also sets R2 */
+ if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL &&
+ bt_is_reg_set(bt, BPF_REG_2)) {
+ u8 ret_size;
+ int err;
+
+ err = bpf_get_kfunc_ret_size(env->prog, insn->imm, insn->off,
+ &ret_size);
+ if (verifier_bug_if(err, env, "no kfunc desc for insn %d", idx))
+ return -EFAULT;
+ if (ret_size > 8)
+ bt_clear_reg(bt, BPF_REG_2);
+ }
if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
/* if backtracking was looking for registers R1-R5
* they should have been found already.
@@ -522,7 +544,30 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
return -EFAULT;
}
} else if (opcode == BPF_EXIT) {
- bool r0_precise;
+ bool from_subprog_call, r0_precise, r2_precise = false;
+
+ /*
+ * BPF_EXIT in subprog or callback always returns
+ * right after the call instruction, so by checking
+ * whether the instruction at subseq_idx-1 is subprog
+ * call or not we can distinguish actual exit from
+ * *subprog* from exit from *callback*. In the former
+ * case, we need to propagate the precision of the
+ * return registers, if necessary. In the latter we
+ * never do that.
+ */
+ from_subprog_call = subseq_idx - 1 >= 0 &&
+ bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]);
+ if (from_subprog_call && bt_is_reg_set(bt, BPF_REG_2)) {
+ struct bpf_subprog_info *callee;
+
+ /* 'idx' is the exit insn, so it is in the callee */
+ callee = bpf_find_containing_subprog(env, idx);
+ if (verifier_bug_if(!callee, env,
+ "no subprog contains exit insn %d", idx))
+ return -EFAULT;
+ r2_precise = bpf_ret_reg_pair(env, callee - env->subprog_info);
+ }
/* Backtracking to a nested function call, 'idx' is a part of
* the inner frame 'subseq_idx' is a part of the outer frame.
@@ -535,23 +580,15 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
if (subseq_idx >= 0 && bpf_calls_callback(env, subseq_idx))
for (i = BPF_REG_1; i <= BPF_REG_5; i++)
bt_clear_reg(bt, i);
+ if (r2_precise)
+ bt_clear_reg(bt, BPF_REG_2);
if (bt_reg_mask(bt) & BPF_REGMASK_ARGS) {
verifier_bug(env, "backtracking exit unexpected regs %x",
bt_reg_mask(bt));
return -EFAULT;
}
- /* BPF_EXIT in subprog or callback always returns
- * right after the call instruction, so by checking
- * whether the instruction at subseq_idx-1 is subprog
- * call or not we can distinguish actual exit from
- * *subprog* from exit from *callback*. In the former
- * case, we need to propagate r0 precision, if
- * necessary. In the former we never do that.
- */
- r0_precise = subseq_idx - 1 >= 0 &&
- bpf_pseudo_call(&env->prog->insnsi[subseq_idx - 1]) &&
- bt_is_reg_set(bt, BPF_REG_0);
+ r0_precise = from_subprog_call && bt_is_reg_set(bt, BPF_REG_0);
bt_clear_reg(bt, BPF_REG_0);
if (bt_subprog_enter(bt))
@@ -559,6 +596,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
if (r0_precise)
bt_set_reg(bt, BPF_REG_0);
+ if (r2_precise)
+ bt_set_reg(bt, BPF_REG_2);
/* r6-r9 and stack slots will stay set in caller frame
* bitmasks until we return back from callee(s)
*/
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 6606187ed4f43..5551abcea1d39 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -7592,7 +7592,12 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,
return -EINVAL;
}
ret = __get_type_size(btf, func->type, &t);
- if (ret < 0 || btf_type_is_struct(t)) {
+ /*
+ * __get_type_size() already restricts a non-negative ret to void, a
+ * pointer, an int, an enum or a struct/union, so only the size is checked
+ * here.
+ */
+ if (ret < 0 || ret > 16) {
bpf_log(log,
"The function %s return type %s is unsupported.\n",
tname, btf_type_str(t));
@@ -7965,7 +7970,7 @@ static int btf_scan_type_tags(struct bpf_verifier_env *env,
/* Check whether the type is a valid return type. */
static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *btf,
- const struct btf_type *t, int subprog)
+ const struct btf_type *t, int subprog, bool is_global)
{
u32 tags = 0;
int err;
@@ -7988,6 +7993,35 @@ static int btf_validate_return_type(struct bpf_verifier_env *env, struct btf *bt
if (btf_type_is_void(t) || btf_type_is_int(t) || btf_is_any_enum(t))
return 0;
+ if (btf_type_is_struct(t) && t->size <= 16) {
+ /*
+ * A >8 byte struct/union is returned in the R0:R2 register pair.
+ * A global function is verified in isolation, so its caller models
+ * the return as an opaque R0:R2 scalar pair; it must therefore
+ * contain only scalars, otherwise a pointer field would be
+ * laundered into a scalar and escape provenance and reference
+ * tracking. That requirement is enforced here: do_check_common()
+ * propagates the error for global functions and for the main
+ * program.
+ *
+ * A local (static) function is verified inline and its R0:R2 are
+ * copied as precise register state (with the JIT forced on when
+ * the pair is consumed), so a pointer field stays tracked and needs
+ * no such restriction. Accepting it here is not by itself what
+ * makes it legal: btf_check_subprog_call() drops any error other
+ * than -EFAULT. What it avoids is needlessly marking the
+ * subprogram's BTF unreliable.
+ *
+ * The main program (subprog 0) takes the scalar-only path as well,
+ * but its return value is the program's exit code, so a >8 byte
+ * return is rejected separately at BPF_EXIT.
+ */
+ bool local_func = subprog && !is_global;
+
+ if (local_func || __btf_type_is_scalar_struct(env, btf, t, 0))
+ return 0;
+ }
+
return -EOPNOTSUPP;
}
@@ -8075,12 +8109,12 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return -EINVAL;
}
- err = btf_validate_return_type(env, btf, t, subprog);
+ err = btf_validate_return_type(env, btf, t, subprog, is_global);
if (err) {
if (is_global) {
bpf_log(log,
- "Global function %s() return value not void or scalar. "
- "Only those are supported.\n",
+ "Global function %s() has unsupported return type. "
+ "Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.\n",
tname);
}
return err;
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index a3e1fae32eace..d98f4220e875e 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3303,6 +3303,11 @@ bool __weak bpf_jit_supports_kfunc_call(void)
return false;
}
+bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
+{
+ return false;
+}
+
bool __weak bpf_jit_supports_stack_args(void)
{
return false;
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index ef9a5a9228872..451edd74fa6f5 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -2062,10 +2062,15 @@ static inline u32 mask_widen(u32 m) { return m | (m << 16); }
static inline u16 mask_lo(u32 m) { return (u16)m; }
static inline u16 mask_hi(u32 m) { return (u16)(m >> 16); }
-/* Compute info->{use,def} fields for the instruction */
+/*
+ * Compute info->{use,def} fields for the instruction. @ret_reg_pair tells
+ * whether the subprogram containing @insn returns its value in the R0:R2
+ * register pair, which matters for BPF_EXIT.
+ */
static void compute_insn_live_regs(struct bpf_verifier_env *env,
struct bpf_insn *insn,
- struct insn_live_regs *info)
+ struct insn_live_regs *info,
+ bool ret_reg_pair)
{
struct bpf_call_summary cs;
const u8 class = BPF_CLASS(insn->code);
@@ -2196,7 +2201,7 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
break;
case BPF_EXIT:
def = 0;
- use = r0;
+ use = ret_reg_pair ? (r0 | reg64_mask(BPF_REG_2)) : r0;
break;
case BPF_CALL:
def = ALL_CALLER_SAVED_REGS;
@@ -2233,8 +2238,8 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
struct insn_live_regs *state;
int insn_cnt = env->prog->len;
u64 pos, insn_pos;
- int err = 0, i, j;
- bool changed;
+ int err = 0, i, j, subprog, start, end;
+ bool changed, ret_reg_pair;
/* Use the following algorithm:
* - define the following:
@@ -2261,8 +2266,14 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
goto out;
}
- for (i = 0; i < insn_cnt; ++i)
- compute_insn_live_regs(env, &insns[i], &state[i]);
+ for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
+ start = env->subprog_info[subprog].start;
+ end = env->subprog_info[subprog + 1].start;
+ ret_reg_pair = bpf_ret_reg_pair(env, subprog);
+
+ for (i = start; i < end; ++i)
+ compute_insn_live_regs(env, &insns[i], &state[i], ret_reg_pair);
+ }
/* Forward pass: resolve stack access through FP-derived pointers */
err = bpf_compute_subprog_arg_access(env);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index add3affc57035..f8294359c85df 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -382,27 +382,75 @@ bool bpf_subprog_is_global(const struct bpf_verifier_env *env, int subprog)
return aux && aux[subprog].linkage == BTF_FUNC_GLOBAL;
}
-static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+/* Return type of a subprogram, NULL if it cannot be resolved */
+static const struct btf_type *subprog_ret_type(struct bpf_verifier_env *env, int subprog)
{
- const struct btf_type *type, *func, *func_proto;
+ const struct btf_type *func, *func_proto;
const struct btf *btf = env->prog->aux->btf;
u32 btf_id;
+ if (!btf || !env->prog->aux->func_info)
+ return NULL;
+
btf_id = env->prog->aux->func_info[subprog].type_id;
+ /* Both already validated by prepare_btf_func() at prog load. */
func = btf_type_by_id(btf, btf_id);
- if (verifier_bug_if(!func, env, "btf_id %u not found", btf_id))
- return false;
-
func_proto = btf_type_by_id(btf, func->type);
- if (!func_proto)
- return false;
- type = btf_type_skip_modifiers(btf, func_proto->type, NULL);
- if (!type)
- return false;
+ return btf_type_skip_modifiers(btf, func_proto->type, NULL);
+}
+
+static bool subprog_returns_void(struct bpf_verifier_env *env, int subprog)
+{
+ const struct btf_type *type = subprog_ret_type(env, subprog);
- return btf_type_is_void(type);
+ return type && btf_type_is_void(type);
+}
+
+/*
+ * Number of registers holding a function return value: a value of up to 8
+ * bytes is returned in R0, a value of more than 8 bytes and no more than 16
+ * bytes (an __int128 or a struct/union of such size) is returned in the R0:R2
+ * register pair, with R2 holding the upper half.
+ */
+static u32 ret_regs_cnt(u32 size)
+{
+ return size > 8 && size <= 16 ? 2 : 1;
+}
+
+/* Registers holding a function return value, in order. See ret_regs_cnt(). */
+static const int ret_regs[] = { BPF_REG_0, BPF_REG_2 };
+
+/*
+ * Resolve the return convention of every subprogram once, so that
+ * bpf_ret_reg_pair() is a plain flag test on the hot paths that use it.
+ */
+static void bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)
+{
+ const struct btf_type *type;
+ int subprog;
+
+ for (subprog = 0; subprog < env->subprog_cnt; subprog++) {
+ type = subprog_ret_type(env, subprog);
+ if (type && (btf_type_is_struct(type) || btf_type_is_scalar(type)))
+ subprog_info(env, subprog)->ret_reg_pair = ret_regs_cnt(type->size) > 1;
+ }
+}
+
+/*
+ * A >8 byte BPF return changes the calling convention to R0:R2, so the
+ * verifier can only allow it while the subprogram's prototype remains
+ * reliable. Once BTF is marked unreliable, reject the feature instead of
+ * silently falling back to R0-only semantics.
+ */
+static bool subprog_ret_pair_unreliable(struct bpf_verifier_env *env, int subprog)
+{
+ struct bpf_prog_aux *aux = env->prog->aux;
+
+ return bpf_ret_reg_pair(env, subprog) &&
+ aux->func_info_aux &&
+ aux->func_info_aux[subprog].unreliable;
}
static const char *subprog_name(const struct bpf_verifier_env *env, int subprog)
@@ -2470,6 +2518,19 @@ int bpf_get_kfunc_addr(const struct bpf_prog *prog, u32 func_id,
return 0;
}
+int bpf_get_kfunc_ret_size(const struct bpf_prog *prog, u32 func_id,
+ u16 btf_fd_idx, u8 *ret_size)
+{
+ const struct bpf_kfunc_desc *desc;
+
+ desc = find_kfunc_desc(prog, func_id, btf_fd_idx);
+ if (!desc)
+ return -EFAULT;
+
+ *ret_size = desc->func_model.ret_size;
+ return 0;
+}
+
#define BPF_FD_SLOT_BTF 1UL
static void fd_slot_set_map(struct bpf_fd_array *slot, struct bpf_map *map)
@@ -2807,6 +2868,19 @@ int bpf_add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, u16 offset)
err = btf_distill_func_proto(&env->log, kfunc.btf, kfunc.proto, kfunc.name, &func_model);
if (err)
return err;
+ if (func_model.ret_size > 8) {
+ if (kfunc.flags && (*kfunc.flags & KF_FASTCALL)) {
+ verbose(env,
+ "kfunc %s with >8-byte return is not supported with KF_FASTCALL\n",
+ kfunc.name);
+ return -EOPNOTSUPP;
+ }
+ if (!bpf_jit_supports_kfunc_ret_reg_pair()) {
+ verbose(env, "kfunc %s with >8-byte return is not supported by JIT\n",
+ kfunc.name);
+ return -EOPNOTSUPP;
+ }
+ }
memset(&meta, 0, sizeof(meta));
meta.btf = kfunc.btf;
@@ -9388,6 +9462,7 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
u16 callee_incoming, stack_arg_cnt;
struct bpf_func_state *caller;
int err, subprog, target_insn;
+ u32 i, nregs;
target_insn = *insn_idx + insn->imm + 1;
subprog = bpf_find_subprog(env, target_insn);
@@ -9399,6 +9474,11 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
err = btf_check_subprog_call(env, subprog, caller->regs);
if (err == -EFAULT)
return err;
+ if (subprog_ret_pair_unreliable(env, subprog)) {
+ verbose(env, "Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n",
+ subprog, subprog_name(env, subprog));
+ return -EINVAL;
+ }
if (bpf_subprog_is_global(env, subprog)) {
const char *sub_name = subprog_name(env, subprog);
@@ -9430,9 +9510,22 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
clear_caller_saved_regs(env, caller->regs);
invalidate_outgoing_stack_args(env, cur_func(env));
- /* All non-void global functions return a 64-bit SCALAR_VALUE. */
+ /*
+ * A non-void global function returns a 64-bit SCALAR_VALUE in
+ * R0, or a >8 byte SCALAR_VALUE in the R0:R2 register pair.
+ */
if (!subprog_returns_void(env, subprog)) {
- mark_reg_unknown(env, caller->regs, BPF_REG_0);
+ nregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;
+ /*
+ * The R0:R2 return convention is only implemented in the
+ * JIT: the interpreter propagates BPF_R0 alone out of a
+ * subprogram, so a caller reading R2 would see a stale
+ * value. Force the JIT once a caller can observe the pair.
+ */
+ if (nregs > 1)
+ env->prog->jit_required = 1;
+ for (i = 0; i < nregs; i++)
+ mark_reg_unknown(env, caller->regs, ret_regs[i]);
}
if (env->subprog_info[subprog].might_throw) {
@@ -9754,10 +9847,19 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
struct bpf_func_state *caller, *callee;
struct bpf_reg_state *r0;
bool in_callback_fn;
+ u32 i, nregs;
int err;
callee = state->frame[state->curframe];
r0 = &callee->regs[BPF_REG_0];
+ if (subprog_ret_pair_unreliable(env, callee->subprogno)) {
+ verbose(env, "Func#%d ('%s') returns >8 bytes, which requires reliable BTF\n",
+ callee->subprogno, subprog_name(env, callee->subprogno));
+ return -EINVAL;
+ }
+ nregs = bpf_ret_reg_pair(env, callee->subprogno) ? 2 : 1;
+ if (nregs > 1)
+ env->prog->jit_required = 1;
if (r0->type == PTR_TO_STACK) {
/* technically it's ok to return caller's stack pointer
* (or caller's caller's pointer) back to the caller,
@@ -9793,8 +9895,23 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
return -EFAULT;
}
} else {
- /* return to the caller whatever r0 had in the callee */
- caller->regs[BPF_REG_0] = *r0;
+ /*
+ * return to the caller whatever the callee had in the
+ * return register(s)
+ */
+ for (i = 0; i < nregs; i++)
+ caller->regs[ret_regs[i]] = callee->regs[ret_regs[i]];
+
+ /*
+ * R2 carries only the upper half of a register pair return
+ * value. A stack pointer must not escape the callee (see the
+ * R0 case above), but there is no need to reject the whole
+ * program for it: hand the caller an uninitialized R2 instead,
+ * so that only a caller actually using the returned pointer
+ * fails.
+ */
+ if (nregs > 1 && caller->regs[BPF_REG_2].type == PTR_TO_STACK)
+ bpf_mark_reg_not_init(env, &caller->regs[BPF_REG_2]);
}
/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,
@@ -10691,6 +10808,19 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
return 0;
}
+/*
+ * Mark the register(s) holding a @size byte kfunc return value as unknown
+ * scalars. Both halves of a register pair are treated the same way.
+ */
+static void mark_kfunc_ret_regs(struct bpf_verifier_env *env,
+ struct bpf_reg_state *regs, u32 size)
+{
+ u32 i, nregs = ret_regs_cnt(size);
+
+ for (i = 0; i < nregs; i++)
+ mark_reg_unknown(env, regs, ret_regs[i]);
+}
+
static bool is_kfunc_acquire(struct bpf_call_arg_meta *meta)
{
return meta->kfunc_flags & KF_ACQUIRE;
@@ -10958,9 +11088,9 @@ static bool is_kfunc_arg_implicit(const struct bpf_call_arg_meta *meta, u32 arg_
}
/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */
-static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
- const struct btf *btf,
- const struct btf_type *t, int rec)
+bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env,
+ const struct btf *btf,
+ const struct btf_type *t, int rec)
{
const struct btf_type *member_type;
const struct btf_member *member;
@@ -13167,10 +13297,25 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
}
if (btf_type_is_scalar(t)) {
- mark_reg_unknown(env, regs, BPF_REG_0);
+ mark_kfunc_ret_regs(env, regs, t->size);
if (meta.btf == btf_vmlinux && (meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
meta.func_id == special_kfunc_list[KF_bpf_res_spin_lock_irqsave]))
__mark_reg_const_zero(env, ®s[BPF_REG_0]);
+ } else if (btf_type_is_struct(t)) {
+ /*
+ * The returned struct comes back as raw register bits modeled
+ * as an unknown scalar, so it must contain only scalars:
+ * otherwise a pointer field would be laundered into a scalar
+ * and escape provenance and reference tracking.
+ */
+ if (!__btf_type_is_scalar_struct(env, desc_btf, t, 0)) {
+ verbose(env,
+ "kernel function %s returns %s %s that is not composed of scalars\n",
+ func_name, btf_type_str(t),
+ btf_name_by_offset(desc_btf, t->name_off));
+ return -EINVAL;
+ }
+ mark_kfunc_ret_regs(env, regs, t->size);
} else if (btf_type_is_ptr(t)) {
ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id);
err = check_special_kfunc(env, &meta, regs, insn_aux, ptr_type, desc_btf);
@@ -16250,6 +16395,11 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)
verbose(env, "callback function not static\n");
return -EINVAL;
}
+ if (bpf_ret_reg_pair(env, subprogno)) {
+ verbose(env,
+ "callback function with >8-byte return value is not supported\n");
+ return -EINVAL;
+ }
dst_reg->type = PTR_TO_FUNC;
dst_reg->subprogno = subprogno;
@@ -16617,37 +16767,61 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
return 0;
}
-static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+static int check_global_ret_scalar_reg(struct bpf_verifier_env *env, u32 regno,
+ bool allow_arena_ptr_return)
{
- struct bpf_reg_state *reg = reg_state(env, BPF_REG_0);
- struct bpf_func_state *cur_frame = cur_func(env);
+ struct bpf_reg_state *reg;
int err;
- if (subprog_returns_void(env, cur_frame->subprogno))
- return 0;
-
- err = check_reg_arg(env, BPF_REG_0, SRC_OP);
+ err = check_reg_arg(env, regno, SRC_OP);
if (err)
return err;
/* Pointers to arena are safe to pass between subprograms. */
- if (is_arena_reg(env, BPF_REG_0))
+ if (allow_arena_ptr_return && is_arena_reg(env, regno))
return 0;
- if (is_pointer_value(env, BPF_REG_0)) {
- verbose(env, "R%d leaks addr as return value\n", BPF_REG_0);
+ if (is_pointer_value(env, regno)) {
+ verbose(env, "R%d leaks addr as return value\n", regno);
return -EACCES;
}
+ reg = reg_state(env, regno);
if (reg->type != SCALAR_VALUE) {
- verbose(env, "At subprogram exit the register R0 is not a scalar value (%s)\n",
- reg_type_str(env, reg->type));
+ verbose(env, "At subprogram exit the register R%d is not a scalar value (%s)\n",
+ regno, reg_type_str(env, reg->type));
return -EINVAL;
}
return 0;
}
+static int check_global_subprog_return_code(struct bpf_verifier_env *env)
+{
+ struct bpf_func_state *cur_frame = cur_func(env);
+ u32 subprog = cur_frame->subprogno;
+ u32 i, nregs;
+ int err;
+
+ if (subprog_returns_void(env, subprog))
+ return 0;
+
+ /*
+ * An arena pointer is only a legitimate return value when it is the
+ * whole of it, that is when it is returned in R0 alone. Both halves of
+ * a register pair carry a piece of a >8 byte scalar, so an arena
+ * pointer in either of them is a leak.
+ */
+ nregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;
+ for (i = 0; i < nregs; i++) {
+ err = check_global_ret_scalar_reg(env, ret_regs[i], nregs == 1);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
/* Bitmask with 1s for all caller saved registers */
#define ALL_CALLER_SAVED_REGS ((1u << CALLER_SAVED_REGS) - 1)
@@ -17134,10 +17308,16 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,
*/
if (cur_frame->subprogno &&
!cur_frame->in_async_callback_fn &&
- !cur_frame->in_exception_callback_fn)
+ !cur_frame->in_exception_callback_fn) {
err = check_global_subprog_return_code(env);
- else
+ } else {
+ if (!cur_frame->subprogno && bpf_ret_reg_pair(env, 0)) {
+ verbose(env,
+ "return value larger than 8 bytes is not supported at program exit\n");
+ return -EINVAL;
+ }
err = check_return_code(env, BPF_REG_0, "R0");
+ }
if (err)
return err;
return PROCESS_BPF_EXIT;
@@ -18464,6 +18644,12 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
ret = -EINVAL;
goto out;
}
+ if (bpf_ret_reg_pair(env, subprog)) {
+ verbose(env,
+ "exception cb cannot return value larger than 8 bytes\n");
+ ret = -EINVAL;
+ goto out;
+ }
/* Also ensure the callback only has a single scalar argument. */
if (sub->arg_cnt != 1 || sub->args[0].arg_type != ARG_ANYTHING) {
@@ -19291,6 +19477,22 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
return -EOPNOTSUPP;
}
+ /*
+ * An extension replaces the target outright, so it has to match
+ * the target's return convention. Its own return value is capped
+ * at 8 bytes (a >8 byte program return is rejected at BPF_EXIT),
+ * so it can never fill the R0:R2 pair the target's callers read.
+ * This cannot be left to btf_check_type_match() above, which
+ * compares return types by btf_type->info only: an int carries no
+ * vlen, so a 16-byte __int128 and an 8-byte long compare equal.
+ */
+ if (prog_extension && tgt_info->fmodel.ret_size > 8) {
+ bpf_log(log,
+ "Cannot replace function %s with a >8 byte return value\n",
+ tname);
+ return -EOPNOTSUPP;
+ }
+
/*
* *.multi programs don't need an address during program
* verification, we just take the module ref if needed.
@@ -20280,6 +20482,9 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
if (ret < 0)
goto skip_full_check;
+ /* must precede the first bpf_ret_reg_pair() user below */
+ bpf_compute_subprog_ret_regs(env);
+
ret = bpf_compute_live_registers(env);
if (ret < 0)
goto skip_full_check;
diff --git a/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c b/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
new file mode 100644
index 0000000000000..8be64d01ff4db
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/aggregate_ret.c
@@ -0,0 +1,206 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include "aggregate_ret_int128_c.skel.h"
+#include "aggregate_ret_struct_c.skel.h"
+#include "aggregate_ret_union_c.skel.h"
+#include "aggregate_ret_kfunc_c.skel.h"
+#include "aggregate_ret_run.skel.h"
+#include "aggregate_ret_func.skel.h"
+#include "aggregate_ret_kfunc.skel.h"
+
+/*
+ * The bpf_testmod kfuncs returning more than 8 bytes are only built on x86_64
+ * and arm64 (see bpf_testmod.c); everywhere else the tests calling them are
+ * skipped.
+ */
+static bool has_ret_pair_kfuncs(void)
+{
+#if defined(__x86_64__) || defined(__aarch64__)
+ return true;
+#else
+ return false;
+#endif
+}
+
+static void run_prog(struct bpf_program *prog)
+{
+ char buf[64] = {};
+ int err, prog_fd;
+ LIBBPF_OPTS(bpf_test_run_opts, topts,
+ .data_in = buf,
+ .data_size_in = sizeof(buf),
+ .repeat = 1,
+ );
+
+ prog_fd = bpf_program__fd(prog);
+ err = bpf_prog_test_run_opts(prog_fd, &topts);
+ if (!ASSERT_OK(err, "test_run"))
+ return;
+
+ ASSERT_EQ(topts.retval, 0, "aggregate_ret_result");
+}
+
+/*
+ * Run @prog as subtest @name. Where the register-pair return is unsupported
+ * the subtest reports a skip instead, so that the list of subtests does not
+ * depend on the compiler or on the architecture; @prog is unused then and may
+ * be NULL, for a caller that could not even open its object.
+ */
+static void run_subtest(const char *name, struct bpf_program *prog, bool supported)
+{
+ if (!test__start_subtest(name))
+ return;
+
+ if (!supported) {
+ test__skip();
+ return;
+ }
+
+ run_prog(prog);
+}
+
+static void test_int128_c(void)
+{
+ struct aggregate_ret_int128_c *skel;
+
+ skel = aggregate_ret_int128_c__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_int128_c_open_load"))
+ return;
+
+ run_subtest("int128_c", skel->progs.aggregate_ret_int128_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ aggregate_ret_int128_c__destroy(skel);
+}
+
+static void test_struct_c(void)
+{
+ struct aggregate_ret_struct_c *skel;
+
+ skel = aggregate_ret_struct_c__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_struct_c_open_load"))
+ return;
+
+ run_subtest("struct_c", skel->progs.aggregate_ret_struct_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ run_subtest("global_struct_c", skel->progs.aggregate_ret_global_struct_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ aggregate_ret_struct_c__destroy(skel);
+}
+
+static void test_union_c(void)
+{
+ struct aggregate_ret_union_c *skel;
+
+ skel = aggregate_ret_union_c__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_union_c_open_load"))
+ return;
+
+ run_subtest("union_c", skel->progs.aggregate_ret_union_c_test,
+ skel->rodata->has_reg_pair_ret);
+
+ aggregate_ret_union_c__destroy(skel);
+}
+
+static void test_kfunc_c(void)
+{
+ struct aggregate_ret_kfunc_c *skel;
+ bool supported;
+ int err;
+
+ skel = aggregate_ret_kfunc_c__open();
+ if (!ASSERT_OK_PTR(skel, "skel_kfunc_c_open"))
+ return;
+
+ supported = skel->rodata->has_reg_pair_ret && has_ret_pair_kfuncs();
+
+ if (supported) {
+ /*
+ * Where the JIT cannot hand the second half of a >8-byte kfunc
+ * return back in R0:R2, bpf_add_kfunc_call() rejects the call
+ * with -EOPNOTSUPP. Asking the kernel keeps this test free of a
+ * list of the JITs that can, which would have to be updated as
+ * the rest of them learn.
+ */
+ err = aggregate_ret_kfunc_c__load(skel);
+ if (err == -EOPNOTSUPP)
+ supported = false;
+ else if (!ASSERT_OK(err, "skel_kfunc_c_load"))
+ goto out;
+ }
+
+ run_subtest("kfunc_int128_c", skel->progs.aggregate_ret_kfunc_int128_c_test,
+ supported);
+
+ run_subtest("kfunc_struct_c", skel->progs.aggregate_ret_kfunc_struct_c_test,
+ supported);
+
+out:
+ aggregate_ret_kfunc_c__destroy(skel);
+}
+
+static void test_run(void)
+{
+ struct aggregate_ret_run *skel;
+ bool kfunc_ok = true;
+ int err;
+
+ /*
+ * Every program in this object shares __kfunc_btf_root(), so where the
+ * testmod kfuncs are absent the object cannot load at all -- including
+ * for the kfunc-free "asm" subtest.
+ */
+ if (!has_ret_pair_kfuncs()) {
+ run_subtest("asm", NULL, false);
+ run_subtest("asm_kfunc", NULL, false);
+ run_subtest("struct", NULL, false);
+ run_subtest("union", NULL, false);
+ return;
+ }
+
+ skel = aggregate_ret_run__open();
+ if (!ASSERT_OK_PTR(skel, "skel_run_open"))
+ return;
+
+ err = aggregate_ret_run__load(skel);
+ if (err == -EOPNOTSUPP) {
+ kfunc_ok = false;
+ aggregate_ret_run__destroy(skel);
+
+ skel = aggregate_ret_run__open();
+ if (!ASSERT_OK_PTR(skel, "skel_run_reopen"))
+ return;
+
+ bpf_program__set_autoload(skel->progs.aggregate_ret_asm_kfunc_test, false);
+ bpf_program__set_autoload(skel->progs.aggregate_ret_struct_test, false);
+ bpf_program__set_autoload(skel->progs.aggregate_ret_union_test, false);
+
+ err = aggregate_ret_run__load(skel);
+ }
+ if (!ASSERT_OK(err, "skel_run_load"))
+ goto out;
+
+ run_subtest("asm", skel->progs.aggregate_ret_asm_test, true);
+ run_subtest("asm_kfunc", skel->progs.aggregate_ret_asm_kfunc_test, kfunc_ok);
+ run_subtest("struct", skel->progs.aggregate_ret_struct_test, kfunc_ok);
+ run_subtest("union", skel->progs.aggregate_ret_union_test, kfunc_ok);
+
+out:
+ aggregate_ret_run__destroy(skel);
+}
+
+void test_aggregate_ret(void)
+{
+ test_int128_c();
+ test_struct_c();
+ test_union_c();
+ test_kfunc_c();
+ test_run();
+
+ RUN_TESTS(aggregate_ret_func);
+ if (has_ret_pair_kfuncs())
+ RUN_TESTS(aggregate_ret_kfunc);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/exceptions.c b/tools/testing/selftests/bpf/prog_tests/exceptions.c
index 3588d6f97fd4e..71d00c568d802 100644
--- a/tools/testing/selftests/bpf/prog_tests/exceptions.c
+++ b/tools/testing/selftests/bpf/prog_tests/exceptions.c
@@ -5,6 +5,7 @@
#include "exceptions.skel.h"
#include "exceptions_ext.skel.h"
#include "exceptions_fail.skel.h"
+#include "exceptions_ret_pair_fail.skel.h"
#include "exceptions_assert.skel.h"
static char log_buf[1024 * 1024];
@@ -12,6 +13,7 @@ static char log_buf[1024 * 1024];
static void test_exceptions_failure(void)
{
RUN_TESTS(exceptions_fail);
+ RUN_TESTS(exceptions_ret_pair_fail);
}
static void test_exceptions_success(void)
diff --git a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
index 2523c07a16c65..0b54f911015c5 100644
--- a/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
+++ b/tools/testing/selftests/bpf/prog_tests/fexit_bpf2bpf.c
@@ -441,6 +441,19 @@ static void test_func_replace_int_with_void(void)
" doesn't match type INT of global_func2()");
}
+static void test_func_replace_ret_pair(void)
+{
+ const char *msg = "Cannot replace function agg_ret_target_func with a >8 byte return";
+
+ /*
+ * An extension cannot replace a function whose return value comes back
+ * in the R0:R2 pair: the extension's own return is capped at 8 bytes,
+ * so it would leave R2 stale for the target's callers.
+ */
+ test_obj_load_failure_common("freplace_ret_pair.bpf.o",
+ "./aggregate_ret_target.bpf.o", msg);
+}
+
static int find_prog_btf_id(const char *name, __u32 attach_prog_fd)
{
struct bpf_prog_info info = {};
@@ -660,6 +673,8 @@ void serial_test_fexit_bpf2bpf(void)
test_func_replace_progmap();
if (test__start_subtest("freplace_int_with_void"))
test_func_replace_int_with_void();
+ if (test__start_subtest("freplace_ret_pair"))
+ test_func_replace_ret_pair();
if (test__start_subtest("freplace_void"))
test_func_replace_void();
if (test__start_subtest("sleepable_fentry_to_xdp"))
diff --git a/tools/testing/selftests/bpf/prog_tests/timer.c b/tools/testing/selftests/bpf/prog_tests/timer.c
index 09ff21e1ad2f0..593e56d8964ea 100644
--- a/tools/testing/selftests/bpf/prog_tests/timer.c
+++ b/tools/testing/selftests/bpf/prog_tests/timer.c
@@ -6,6 +6,7 @@
#include <sys/syscall.h>
#include "timer.skel.h"
#include "timer_failure.skel.h"
+#include "timer_ret_pair_fail.skel.h"
#include "timer_interrupt.skel.h"
#define NUM_THR 8
@@ -285,6 +286,7 @@ void serial_test_timer(void)
test_timer(timer);
RUN_TESTS(timer_failure);
+ RUN_TESTS(timer_ret_pair_fail);
}
void serial_test_timer_stress(void)
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_func.c b/tools/testing/selftests/bpf/progs/aggregate_ret_func.c
new file mode 100644
index 0000000000000..0bc18450a46ef
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_func.c
@@ -0,0 +1,420 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+typedef unsigned __int128 u128;
+
+__naked u128 global_agg_good(void)
+{
+ asm volatile (
+ "r0 = 0x1234;" /* low 64 bits */
+ "r2 = 0x5678;" /* high 64 bits */
+ "exit;"
+ );
+}
+
+__naked u128 global_agg_bad(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "exit;"
+ );
+}
+
+__naked u128 global_agg_bad_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = r10;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_ret_global(void *ctx)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[global_agg_good];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : __imm(global_agg_good)
+ : "r0", "r1", "r2", "r3", "r4", "r5");
+ if (lo != 0x1234)
+ return 1;
+ if (hi != 0x5678)
+ return 2;
+ return 0;
+}
+
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_global_fail(void)
+{
+ asm volatile (
+ "call %[global_agg_bad];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : __imm(global_agg_bad)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("At subprogram exit the register R2 is not a scalar value")
+__naked int aggregate_ret_global_ptr_fail(void)
+{
+ asm volatile (
+ "call %[global_agg_bad_ptr];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : __imm(global_agg_bad_ptr)
+ : __clobber_all);
+}
+
+static __naked __noinline u128 static_agg_bad_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = r10;" /* stack pointer placed in the second return register */
+ "exit;"
+ );
+}
+
+/*
+ * R2 is caller-saved and only copied from the callee at exit; a PTR_TO_STACK
+ * left in it is turned into an uninitialized R2 in the caller. A caller that
+ * never reads R2 is therefore unaffected and loads fine.
+ */
+SEC("tc")
+__success __retval(0)
+__naked int aggregate_ret_static_ptr_unused(void)
+{
+ asm volatile (
+ "call %[static_agg_bad_ptr];"
+ "r0 = 0;" /* R2 holds a stack pointer but is never read */
+ "exit;"
+ :
+ : __imm(static_agg_bad_ptr)
+ : __clobber_all);
+}
+
+/* But a caller that does read the returned stack pointer is rejected. */
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_static_ptr_read_fail(void)
+{
+ asm volatile (
+ "call %[static_agg_bad_ptr];"
+ "r0 = r2;" /* using the returned stack pointer is rejected */
+ "exit;"
+ :
+ : __imm(static_agg_bad_ptr)
+ : __clobber_all);
+}
+
+static __naked __noinline u128 static_agg_no_r2(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_static_uninit_fail(void)
+{
+ asm volatile (
+ "call %[static_agg_no_r2];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : __imm(static_agg_no_r2)
+ : __clobber_all);
+}
+
+static __naked __noinline u128 static_agg_precise(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 4;" /* second half; its value is made precise below */
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+__log_level(2)
+__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1")
+__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8")
+__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10")
+__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7")
+__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2")
+__msg("mark_precise: frame0: regs=r2 stack= before 12: (95) exit")
+__msg("mark_precise: frame1: regs=r2 stack= before 11: (b7) r2 = 4")
+__naked int aggregate_ret_static_precise(void)
+{
+ asm volatile (
+ "call %[static_agg_precise];"
+ "r6 = r2;" /* derived from the aggregate's second half */
+ "r6 &= 7;" /* keep it in [0, 7] to index the stack */
+ "r1 = r10;"
+ "r1 += -8;"
+ "r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */
+ "r0 = 0;"
+ "*(u8 *)(r1 + 0) = r0;"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(static_agg_precise)
+ : __clobber_all);
+}
+
+SEC("tc")
+__success __retval(0)
+__log_level(2)
+__msg("mark_precise: frame0: last_idx 5 first_idx 0 subseq_idx -1")
+__msg("mark_precise: frame0: regs=r6 stack= before 4: (07) r1 += -8")
+__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r1 = r10")
+__msg("mark_precise: frame0: regs=r6 stack= before 2: (57) r6 &= 7")
+__msg("mark_precise: frame0: regs=r6 stack= before 1: (bf) r6 = r2")
+__msg("mark_precise: frame0: regs=r2 stack= before 0: (85) call pc+9")
+__naked int aggregate_ret_global_precise(void)
+{
+ asm volatile (
+ "call %[global_agg_good];"
+ "r6 = r2;" /* derived from the aggregate's second half */
+ "r6 &= 7;" /* keep it in [0, 7] to index the stack */
+ "r1 = r10;"
+ "r1 += -8;"
+ "r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */
+ "r0 = 0;"
+ "*(u8 *)(r1 + 0) = r0;"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_agg_good)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("return value larger than 8 bytes is not supported at program exit")
+__naked u128 aggregate_ret_entry_fail(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ );
+}
+
+#if __clang_major__ >= 23
+
+struct pair {
+ __u64 hi;
+ __u64 lo;
+};
+
+union upair {
+ __u64 halves[2];
+ struct {
+ __u64 lo;
+ __u64 hi;
+ } parts;
+};
+
+/* A by-value struct that smuggles a pointer, which must be rejected. */
+struct with_ptr {
+ void *p;
+ __u64 x;
+};
+
+/* A by-value union that smuggles a pointer, which must be rejected too. */
+union upair_with_ptr {
+ void *p;
+ __u64 halves[2];
+};
+
+/* Global subprogram returning a scalar-only 16-byte struct in R0:R2. */
+__naked struct pair global_ret_struct(void)
+{
+ asm volatile (
+ "r0 = 0x1234;" /* struct's first half */
+ "r2 = 0x5678;" /* struct's second half */
+ "exit;"
+ );
+}
+
+/* Global subprogram returning a scalar-only 16-byte union in R0:R2. */
+__naked union upair global_ret_union(void)
+{
+ asm volatile (
+ "r0 = 0x1234;"
+ "r2 = 0x5678;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_ret_global_struct(void *ctx)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[global_ret_struct];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : __imm(global_ret_struct)
+ : "r0", "r1", "r2", "r3", "r4", "r5");
+ if (lo != 0x1234)
+ return 1;
+ if (hi != 0x5678)
+ return 2;
+ return 0;
+}
+
+SEC("tc")
+__success __retval(0)
+int aggregate_ret_global_union(void *ctx)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[global_ret_union];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : __imm(global_ret_union)
+ : "r0", "r1", "r2", "r3", "r4", "r5");
+ if (lo != 0x1234)
+ return 1;
+ if (hi != 0x5678)
+ return 2;
+ return 0;
+}
+
+__naked struct with_ptr global_ret_struct_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__failure __msg("Global function global_ret_struct_ptr() has unsupported return type")
+__naked int aggregate_ret_global_struct_ptr_fail(void)
+{
+ asm volatile (
+ "call %[global_ret_struct_ptr];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_ret_struct_ptr)
+ : __clobber_all);
+}
+
+__naked union upair_with_ptr global_ret_union_ptr(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__failure __msg("Global function global_ret_union_ptr() has unsupported return type")
+__naked int aggregate_ret_global_union_ptr_fail(void)
+{
+ asm volatile (
+ "call %[global_ret_union_ptr];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_ret_union_ptr)
+ : __clobber_all);
+}
+
+#endif /* __clang_major__ >= 23 */
+
+static __naked u128 agg_callee(void)
+{
+ asm volatile (
+ "r0 = 1;"
+ "r2 = 2;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__log_level(2)
+__msg("Live regs before insn:")
+/*
+ * R2 is read at the exit of agg_callee() (insn 5), which returns a pair, but
+ * not at the exit of this program (insn 2), which returns an int.
+ */
+__msg("0: .12345.... (85) call pc+2")
+__msg("1: ..2....... (bf) r0 = r2")
+__msg("2: 0......... (95) exit")
+__msg("3: .......... (b7) r0 = 1")
+__msg("4: 0......... (b7) r2 = 2")
+__msg("5: 0.2....... (95) exit")
+__naked int aggregate_ret_live(void)
+{
+ asm volatile (
+ "call %[agg_callee];"
+ "r0 = r2;"
+ "exit;"
+ :
+ : [agg_callee]"i"(agg_callee)
+ : __clobber_all);
+}
+
+/*
+ * A static subprogram is verified inline, so prepare_func_exit() hands the
+ * caller the callee's actual R0:R2 register state rather than an opaque scalar
+ * pair. A pointer in the returned struct therefore stays tracked and is usable
+ * by the caller, which is why btf_validate_return_type() does not apply the
+ * scalar-only restriction to a local function. Return the context pointer as
+ * the upper half and dereference it in the caller.
+ */
+struct ptr_pair {
+ void *p;
+ __u64 x;
+};
+
+static __naked __noinline struct ptr_pair static_ret_ptr_pair(void)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = r1;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__success __retval(0)
+__naked int aggregate_ret_static_ptr_pair(void)
+{
+ asm volatile (
+ "call %[static_ret_ptr_pair];"
+ "r1 = *(u32 *)(r2 + 0);" /* deref the returned ctx pointer */
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(static_ret_ptr_pair)
+ : __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c
new file mode 100644
index 0000000000000..f2e09c8be0bed
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_int128_c.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+typedef unsigned __int128 u128;
+
+static __noinline u128 make_i128(__u64 a, __u64 b)
+{
+ return ((u128)(a + b) << 64) | (a - b);
+}
+
+SEC("tc")
+int aggregate_ret_int128_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v;
+
+ v = make_i128(a, b);
+ if ((__u64)(v >> 64) != a + b)
+ return 1;
+ if ((__u64)v != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_int128_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
new file mode 100644
index 0000000000000..617724aa70156
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+/*
+ * Reference kfunc addresses to force those BTF to be emitted. Taking the address
+ * (rather than calling) avoids any dependence on the compiler lowering an
+ * __int128 or struct return value, which the BPF backend only supports from
+ * LLVM 23 on.
+ */
+void __kfunc_btf_root(void)
+{
+ asm volatile (""
+ :
+ : "r"(&bpf_kfunc_call_test_i128),
+ "r"(&bpf_kfunc_call_test_ret_fastcall),
+ "r"(&bpf_kfunc_call_test_ret_ptr),
+ "r"(&bpf_kfunc_call_test_ret_ii),
+ "r"(&bpf_kfunc_call_test_ret_big));
+}
+
+/*
+ * bpf_add_kfunc_call() rejects a kfunc returning more than 8 bytes unless the
+ * JIT advertises bpf_jit_supports_kfunc_ret_reg_pair(), so a test that has to
+ * get past it is tagged with the architectures implementing it. The two tests
+ * below that are rejected earlier, on KF_FASTCALL or on reading R2 after an
+ * 8-byte struct return, behave the same everywhere and are not tagged.
+ */
+
+SEC("tc")
+__arch_x86_64 __arch_arm64 __arch_riscv64
+__success __retval(0)
+__log_level(2)
+__msg("mark_precise: frame0: last_idx 7 first_idx 0 subseq_idx -1")
+__msg("mark_precise: frame0: regs=r6 stack= before 6: (07) r1 += -8")
+__msg("mark_precise: frame0: regs=r6 stack= before 5: (bf) r1 = r10")
+__msg("mark_precise: frame0: regs=r6 stack= before 4: (57) r6 &= 7")
+__msg("mark_precise: frame0: regs=r6 stack= before 3: (bf) r6 = r2")
+__msg("mark_precise: frame0: regs=r2 stack= before 2: (85) call bpf_kfunc_call_test_i128")
+__naked int aggregate_ret_kfunc_precise(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 2;"
+ "call %[bpf_kfunc_call_test_i128];"
+ "r6 = r2;" /* second return half */
+ "r6 &= 7;" /* keep it in [0, 7] to index the stack */
+ "r1 = r10;"
+ "r1 += -8;"
+ "r1 += r6;" /* ptr += scalar marks r6 (hence R2) precise */
+ "r0 = 0;"
+ "*(u8 *)(r1 + 0) = r0;"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_i128)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("kfunc bpf_kfunc_call_test_ret_fastcall with >8-byte return is not supported with KF_FASTCALL")
+__naked int aggregate_ret_kfunc_fastcall_fail(void)
+{
+ asm volatile (
+ "r1 = 1;"
+ "r2 = 2;"
+ "call %[bpf_kfunc_call_test_ret_fastcall];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_fastcall)
+ : __clobber_all);
+}
+
+SEC("tc")
+__arch_x86_64 __arch_arm64 __arch_riscv64
+__failure __msg("is not composed of scalars")
+__naked int aggregate_ret_kfunc_ptr_fail(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "call %[bpf_kfunc_call_test_ret_ptr];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_ptr)
+ : __clobber_all);
+}
+
+SEC("tc")
+__failure __msg("R2 !read_ok")
+__naked int aggregate_ret_kfunc_small_no_r2(void)
+{
+ asm volatile (
+ "r1 = 0;"
+ "r2 = 0;"
+ "call %[bpf_kfunc_call_test_ret_ii];"
+ "r0 = r2;" /* R2 is not a return register for a <=8 byte struct */
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_ii)
+ : __clobber_all);
+}
+
+/*
+ * A return value larger than 16 bytes does not fit in R0:R2 and is rejected by
+ * btf_distill_func_proto(), before the KF_FASTCALL and JIT-capability checks,
+ * so this behaves the same on every architecture.
+ */
+SEC("tc")
+__failure __msg("The function bpf_kfunc_call_test_ret_big return type STRUCT is unsupported")
+__naked int aggregate_ret_kfunc_too_big_fail(void)
+{
+ asm volatile (
+ "call %[bpf_kfunc_call_test_ret_big];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(bpf_kfunc_call_test_ret_big)
+ : __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c
new file mode 100644
index 0000000000000..fd000620f3141
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_kfunc_c.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+typedef unsigned __int128 u128;
+
+SEC("tc")
+int aggregate_ret_kfunc_int128_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ u128 v;
+
+ v = bpf_kfunc_call_test_i128(a, b);
+ if ((__u64)(v >> 64) != a + b)
+ return 1;
+ if ((__u64)v != a - b)
+ return 2;
+
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct prog_test_ret_pair p;
+
+ p = bpf_kfunc_call_test_ret_pair(a, b);
+ if (p.hi != a + b)
+ return 1;
+ if (p.lo != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_kfunc_int128_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_kfunc_struct_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_run.c b/tools/testing/selftests/bpf/progs/aggregate_ret_run.c
new file mode 100644
index 0000000000000..382ef3b90037c
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_run.c
@@ -0,0 +1,178 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+#include "../test_kmods/bpf_testmod_kfunc.h"
+
+typedef unsigned __int128 u128;
+
+/*
+ * Reference kfunc addresses to force those BTF to be emitted. Taking the address
+ * (rather than calling) avoids any dependence on the compiler lowering an __int128
+ * or struct return value, which the BPF backend only supports from LLVM 23 on.
+ */
+void __kfunc_btf_root(void)
+{
+ asm volatile (""
+ :
+ : "r"(&bpf_kfunc_call_test_i128),
+ "r"(&bpf_kfunc_call_test_ret_pair),
+ "r"(&bpf_kfunc_call_test_ret_li),
+ "r"(&bpf_kfunc_call_test_ret_ii),
+ "r"(&bpf_kfunc_call_test_ret_uu));
+}
+
+#define I128_ASM_LO 0xABCDabcd12345678ULL
+#define I128_ASM_HI 0x1234567890abcdefULL
+
+static __naked __noinline u128 make_i128_asm(void)
+{
+ asm volatile (
+ "r0 = %[lo] ll;" /* low 64 bits */
+ "r2 = %[hi] ll;" /* high 64 bits */
+ "exit;"
+ :
+ : __imm_const(lo, I128_ASM_LO), __imm_const(hi, I128_ASM_HI)
+ );
+}
+
+SEC("tc")
+int aggregate_ret_asm_test(struct __sk_buff *skb)
+{
+ __u64 lo, hi;
+
+ asm volatile (
+ "call %[callee];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [callee]"i"(make_i128_asm)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != I128_ASM_LO)
+ return 1;
+ if (hi != I128_ASM_HI)
+ return 2;
+
+ return 0;
+}
+
+/*
+ * R0 holds bytes 0..7 of a kfunc return value and R2 bytes 8..15, so where a
+ * member sits inside a register depends on the endianness of the target.
+ * Although arm64 supports both little and big endian, for simplicity, only
+ * do little endian for now..
+ */
+SEC("tc")
+int aggregate_ret_asm_kfunc_test(struct __sk_buff *skb)
+{
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ __u64 a = skb->len;
+ __u64 b = skb->len ^ 0xdeadbeefULL;
+ __u64 lo, hi;
+
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_i128)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (hi != a + b)
+ return 1;
+ if (lo != a - b)
+ return 2;
+#endif
+
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_struct_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len;
+ __u64 b = skb->len ^ 0xdeadbeefULL;
+ __u64 lo, hi;
+
+ /* struct { u64 hi; u64 lo; }: R0 = hi, R2 = lo. */
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_pair)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != a + b)
+ return 1;
+ if (hi != a - b)
+ return 2;
+
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+ /* struct { u64 a; int b; }: R0 = a, low 32 bits of R2 = b. */
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_li)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != a)
+ return 3;
+ if ((int)hi != ~(int)b)
+ return 4;
+
+ /* struct { int a; int b; }: 8 bytes, packed into R0; R2 is not used. */
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ : [lo]"=r"(lo)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_ii)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if ((int)lo != (int)a)
+ return 5;
+ if ((int)(lo >> 32) != (int)b)
+ return 6;
+#endif
+
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_union_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len;
+ __u64 b = skb->len ^ 0xdeadbeefULL;
+ __u64 lo, hi;
+
+ asm volatile (
+ "r1 = %[a];"
+ "r2 = %[b];"
+ "call %[kfunc];"
+ "%[lo] = r0;"
+ "%[hi] = r2;"
+ : [lo]"=r"(lo), [hi]"=r"(hi)
+ : [a]"r"(a), [b]"r"(b), [kfunc]"i"(bpf_kfunc_call_test_ret_uu)
+ : "r0", "r1", "r2", "r3", "r4", "r5"
+ );
+ if (lo != a + b)
+ return 1;
+ if (hi != a - b)
+ return 2;
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c
new file mode 100644
index 0000000000000..5296e41da3f00
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_struct_c.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+struct pair {
+ __u64 hi; /* R0 */
+ __u64 lo; /* R2 */
+};
+
+static __noinline struct pair make_pair(__u64 a, __u64 b)
+{
+ struct pair p = { .hi = a + b, .lo = a - b };
+
+ return p;
+}
+
+SEC("tc")
+int aggregate_ret_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p;
+
+ p = make_pair(a, b);
+ if (p.hi != a + b)
+ return 1;
+ if (p.lo != a - b)
+ return 2;
+
+ return 0;
+}
+
+__noinline struct pair make_pair_global(__u64 a, __u64 b)
+{
+ struct pair p = { .hi = a + b, .lo = a - b };
+
+ return p;
+}
+
+SEC("tc")
+int aggregate_ret_global_struct_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ struct pair p;
+
+ p = make_pair_global(a, b);
+ if (p.hi != a + b)
+ return 1;
+ if (p.lo != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_struct_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+SEC("tc")
+int aggregate_ret_global_struct_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_target.c b/tools/testing/selftests/bpf/progs/aggregate_ret_target.c
new file mode 100644
index 0000000000000..cffd8d7d3241a
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_target.c
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+/* freplace target: a global subprogram returning 16 bytes in R0:R2. */
+__naked unsigned __int128 agg_ret_target_func(void)
+{
+ asm volatile (
+ "r0 = 0x1234;"
+ "r2 = 0x5678;"
+ "exit;"
+ );
+}
+
+SEC("tc")
+__naked int agg_ret_target(void)
+{
+ asm volatile (
+ "call %[agg_ret_target_func];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(agg_ret_target_func)
+ : __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c b/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
new file mode 100644
index 0000000000000..5547fa6cbd495
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/aggregate_ret_union_c.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+#if defined(__clang_major__) && __clang_major__ >= 23
+
+const volatile bool has_reg_pair_ret = true;
+
+#define MIX_A 0xdeadbeefcafef00dULL
+#define MIX_B 0x0123456789abcdefULL
+
+union pair {
+ __u64 halves[2];
+ struct {
+ __u64 lo; /* R0 */
+ __u64 hi; /* R2 */
+ } parts;
+};
+
+static __noinline union pair make_pair(__u64 a, __u64 b)
+{
+ union pair p;
+
+ p.halves[0] = a + b;
+ p.halves[1] = a - b;
+ return p;
+}
+
+SEC("tc")
+int aggregate_ret_union_c_test(struct __sk_buff *skb)
+{
+ __u64 a = skb->len ^ MIX_A;
+ __u64 b = skb->len ^ MIX_B;
+ union pair p;
+
+ p = make_pair(a, b);
+ if (p.parts.lo != a + b)
+ return 1;
+ if (p.parts.hi != a - b)
+ return 2;
+
+ return 0;
+}
+
+#else
+
+const volatile bool has_reg_pair_ret = false;
+
+SEC("tc")
+int aggregate_ret_union_c_test(struct __sk_buff *skb)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c
new file mode 100644
index 0000000000000..a45db5d9c1d44
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf__exceptions_ret_pair_fail.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)
+{
+ for (;;)
+ ;
+}
diff --git a/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c
new file mode 100644
index 0000000000000..35506c7c5a916
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/btf__timer_ret_pair_fail.c
@@ -0,0 +1,10 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+
+unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer)
+{
+ for (;;)
+ ;
+}
diff --git a/tools/testing/selftests/bpf/progs/exceptions_fail.c b/tools/testing/selftests/bpf/progs/exceptions_fail.c
index ac44d60e50666..9708efb93683b 100644
--- a/tools/testing/selftests/bpf/progs/exceptions_fail.c
+++ b/tools/testing/selftests/bpf/progs/exceptions_fail.c
@@ -60,7 +60,7 @@ __noinline int exception_cb_ok_arg_small(int a)
SEC("?tc")
__exception_cb(exception_cb_bad_ret_type1)
-__failure __msg("Global function exception_cb_bad_ret_type1() return value not void or scalar.")
+__failure __msg("Only void, scalar, or a scalar-only struct/union up to 16 bytes is supported.")
int reject_exception_cb_type_1(struct __sk_buff *ctx)
{
bpf_throw(0);
diff --git a/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c
new file mode 100644
index 0000000000000..842f86ad8659e
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/exceptions_ret_pair_fail.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_tracing.h>
+#include <bpf/bpf_helpers.h>
+
+#include "bpf_misc.h"
+#include "bpf_experimental.h"
+
+__naked __noinline __used
+unsigned __int128 exception_cb_bad_ret_type3(u64 cookie)
+{
+ asm volatile (
+ "r0 = r1;"
+ "r2 = 0;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+SEC("?tc")
+__exception_cb(exception_cb_bad_ret_type3)
+__failure __msg("exception cb cannot return value larger than 8 bytes")
+__btf_func_path("btf__exceptions_ret_pair_fail.bpf.o")
+int reject_exception_cb_ret_pair(void *ctx)
+{
+ bpf_throw(0);
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/freplace_ret_pair.c b/tools/testing/selftests/bpf/progs/freplace_ret_pair.c
new file mode 100644
index 0000000000000..84b701402ca67
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/freplace_ret_pair.c
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+/*
+ * An extension replaces its target outright, so it has to match the target's
+ * return convention. Its own return value is capped at 8 bytes, so it can
+ * never fill the R0:R2 pair that the target's callers read, and the attach is
+ * rejected. btf_check_type_match() cannot catch this: it compares return types
+ * by btf_type->info only, and an int carries no vlen, so the __u64 here and
+ * the target's __int128 compare equal.
+ */
+SEC("freplace/agg_ret_target_func")
+__u64 new_agg_ret_target_func(void)
+{
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c b/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c
new file mode 100644
index 0000000000000..29fd294dfd49b
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/timer_ret_pair_fail.c
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
+
+#include <linux/bpf.h>
+#include <time.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct elem {
+ struct bpf_timer t;
+};
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, struct elem);
+} timer_map SEC(".maps");
+
+__naked __noinline __used
+static unsigned __int128 timer_cb_ret_pair(void *map, int *key, struct bpf_timer *timer)
+{
+ asm volatile (
+ "r0 = 0;"
+ "r2 = 0;"
+ "exit;"
+ ::: __clobber_all
+ );
+}
+
+SEC("fentry/bpf_fentry_test1")
+__failure __msg("callback function with >8-byte return value is not supported")
+__btf_func_path("btf__timer_ret_pair_fail.bpf.o")
+long BPF_PROG2(test_bad_ret_pair, int, a)
+{
+ int key = 0;
+ struct bpf_timer *timer;
+
+ timer = bpf_map_lookup_elem(&timer_map, &key);
+ if (timer) {
+ bpf_timer_init(timer, &timer_map, CLOCK_BOOTTIME);
+ bpf_timer_set_callback(timer, timer_cb_ret_pair);
+ }
+
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/verifier_arena.c b/tools/testing/selftests/bpf/progs/verifier_arena.c
index b241bbcf54a8a..455b55296f355 100644
--- a/tools/testing/selftests/bpf/progs/verifier_arena.c
+++ b/tools/testing/selftests/bpf/progs/verifier_arena.c
@@ -704,4 +704,42 @@ int check_arena_arg_ret(void *ctx)
return 0;
}
+struct arena_ret_pair {
+ __u64 lo;
+ __u64 hi;
+};
+
+/*
+ * A 16-byte value is returned in the R0:R2 register pair. A global subprogram
+ * may return an arena pointer in R0, but R2 holds the upper half of a scalar
+ * pair, so an arena pointer there is not a valid return value. The ld_imm64 of
+ * the arena map is what links the arena to the program, without which the
+ * addr_space_cast insn is not allowed.
+ */
+__naked struct arena_ret_pair global_ret_arena_ptr_in_r2(void)
+{
+ asm volatile (
+ "r1 = %[arena] ll;"
+ "r2 = 8192;"
+ "r2 = addr_space_cast(r2, 0x0, 0x1);"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm_addr(arena)
+ : __clobber_all);
+}
+
+SEC("syscall")
+__failure __msg("At subprogram exit the register R2 is not a scalar value (arena)")
+__naked int check_global_ret_arena_ptr_in_r2(void)
+{
+ asm volatile (
+ "call %[global_ret_arena_ptr_in_r2];"
+ "r0 = 0;"
+ "exit;"
+ :
+ : __imm(global_ret_arena_ptr_in_r2)
+ : __clobber_all);
+}
+
char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index a6133f7521f34..9ef95983da3ca 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -939,6 +939,83 @@ __bpf_kfunc int bpf_kfunc_call_test5(u8 a, u16 b, u32 c)
return 0;
}
+/*
+ * A kfunc is only usable where the ABI hands its return value back in
+ * registers. s390x, for example, returns a by-value struct or union through a
+ * hidden pointer argument (sret) whatever its size. That pointer shifts every
+ * declared argument by one register, and pahole, which maps parameters to
+ * registers positionally, then skips the function with "unexpected register
+ * usage for parameter". resolve_btfids reports "no BTF func for kfunc" and
+ * leaves the ID at 0, which makes register_btf_kfunc_id_set() fail at module
+ * init, so the module does not load at all.
+ *
+ * Restrict these kfuncs to the architectures where the return value comes back
+ * in registers. A kfunc taking no argument has nothing for the sret pointer to
+ * displace and needs no guard, whatever it returns.
+ */
+#if defined(__x86_64__) || defined(__aarch64__)
+__bpf_kfunc __int128 bpf_kfunc_call_test_i128(u64 a, u64 b)
+{
+ return (__int128)(((unsigned __int128)(a + b) << 64) | (a - b));
+}
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(u64 a, u64 b)
+{
+ struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b };
+
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(u64 a, u64 b)
+{
+ struct prog_test_ret_pair r = { .hi = a + b, .lo = a - b };
+
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_li bpf_kfunc_call_test_ret_li(u64 a, int b)
+{
+ struct prog_test_ret_li r = { .a = a, .b = ~b };
+
+ return r;
+}
+
+__bpf_kfunc union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(u64 a, u64 b)
+{
+ union prog_test_ret_uu r;
+
+ r.halves[0] = a + b;
+ r.halves[1] = a - b;
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(u64 tag)
+{
+ struct prog_test_ret_ptr r = { .p = NULL, .tag = tag };
+
+ return r;
+}
+
+__bpf_kfunc struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b)
+{
+ struct prog_test_ret_ii r = { .a = a, .b = b };
+
+ return r;
+}
+#endif /* __x86_64__ || __aarch64__ */
+
+/*
+ * Takes no argument on purpose: with no arguments there is nothing for the sret
+ * pointer to displace, so this needs no architecture guard even though it
+ * returns 24 bytes. See the comment on bpf_kfunc_call_test_i128() above.
+ */
+__bpf_kfunc struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void)
+{
+ struct prog_test_ret_big r = { .a = 1, .b = 2, .c = 3 };
+
+ return r;
+}
+
__bpf_kfunc u64 bpf_kfunc_call_stack_arg(u64 a, u64 b, u64 c, u64 d,
u64 e, u64 f, u64 g, u64 h,
u64 i, u64 j)
@@ -1472,6 +1549,16 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test2)
BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
BTF_ID_FLAGS(func, bpf_kfunc_call_test4)
BTF_ID_FLAGS(func, bpf_kfunc_call_test5)
+#if defined(__x86_64__) || defined(__aarch64__)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_i128)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_pair)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_fastcall, KF_FASTCALL)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_li)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_uu)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ptr)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_ii)
+#endif
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_ret_big)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_ptr)
BTF_ID_FLAGS(func, bpf_kfunc_call_stack_arg_mix)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index c4383acb53c11..755973793e9b5 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -55,6 +55,44 @@ struct prog_test_big_arg {
__u64 b;
};
+/*
+ * A 16-byte struct returned by value from a kfunc: .hi comes back in R0 and
+ * .lo in R2.
+ */
+struct prog_test_ret_pair {
+ __u64 hi;
+ __u64 lo;
+};
+
+struct prog_test_ret_li { /* 16 bytes: R0:R2 */
+ __u64 a;
+ int b;
+};
+
+struct prog_test_ret_ii { /* 8 bytes: R0 only */
+ int a;
+ int b;
+};
+
+union prog_test_ret_uu { /* 16 bytes: R0:R2 */
+ __u64 halves[2];
+ struct {
+ __u64 lo;
+ __u64 hi;
+ } parts;
+};
+
+struct prog_test_ret_ptr { /* 16 bytes: contains a pointer */
+ void *p;
+ __u64 tag;
+};
+
+struct prog_test_ret_big { /* 24 bytes: too large for R0:R2 */
+ __u64 a;
+ __u64 b;
+ __u64 c;
+};
+
struct prog_test_fail1 {
void *p;
int x;
@@ -130,6 +168,16 @@ int bpf_kfunc_call_test2(struct sock *sk, __u32 a, __u32 b) __ksym;
struct sock *bpf_kfunc_call_test3(struct sock *sk) __ksym;
long bpf_kfunc_call_test4(signed char a, short b, int c, long d) __ksym;
int bpf_kfunc_call_test5(__u8 a, __u16 b, __u32 c) __ksym;
+#ifdef __SIZEOF_INT128__
+__int128 bpf_kfunc_call_test_i128(__u64 a, __u64 b) __ksym;
+#endif
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_pair(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_pair bpf_kfunc_call_test_ret_fastcall(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_li bpf_kfunc_call_test_ret_li(__u64 a, int b) __ksym;
+struct prog_test_ret_ii bpf_kfunc_call_test_ret_ii(int a, int b) __ksym;
+union prog_test_ret_uu bpf_kfunc_call_test_ret_uu(__u64 a, __u64 b) __ksym;
+struct prog_test_ret_ptr bpf_kfunc_call_test_ret_ptr(__u64 tag) __ksym;
+struct prog_test_ret_big bpf_kfunc_call_test_ret_big(void) __ksym;
__u64 bpf_kfunc_call_stack_arg(__u64 a, __u64 b, __u64 c, __u64 d,
__u64 e, __u64 f, __u64 g, __u64 h,
__u64 i, __u64 j) __ksym;
|
| 7/3 | 2026/08/11 00:59 | llm | kmsan-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 35619 output: 343 thoughts: 2984 Thoughts: **KMSAN Needs Assessment for BPF 16-Byte Return Support** Okay, here's the deal. This patch series is all about expanding BPF kfunc and subprogram return values from 8 to 16 bytes, using the R0:R2 register pair. My main concern, as always, is whether this introduces a risk of leaking uninitialized kernel memory. First, let's look at the BPF verifier. It’s been modified to handle the R2 register and track these larger return values. Crucially, the verifier explicitly checks that any returned struct or union *only* contains scalars. This prevents the leaking of pointers. It also ensures the *entire* return value is initialized via `check_reg_arg(..., SRC_OP)`. That's good. In the case of static subprograms, the uninitialized R2 is passed to the caller as uninitialized (by design of the verifier) so only BPF programs attempting to use this uninitialized value would error, which is the standard behavior. My worry shifts to: If a kfunc returns a struct by value with padding, can that padding be uninitialized and leaked to the BPF program? Because the return is register-based, the padding bytes *could* contain whatever garbage was in the register or stack. But wait... the verifier models the kfunc return as an unknown scalar, marking the padding as part of this scalar. BPF programs can store it in maps. Now, does KMSAN come into play here? KMSAN detects uses of uninitialized memory *within* the kernel. It's designed to catch memory errors within the kernel code. KMSAN does not fully instrument BPF programs. The test kfuncs being added, `bpf_kfunc_call_test_ret_pair`, etc., do *not* have padding. The other kfuncs are zeroed. So the padding in my test structures, is zeroed. KMSAN may catch uninitialized memory, but my patch itself does not introduce this situation. KMSAN rules say "NeedsKMSAN=true" only if the patch modifies or introduces things like kernel structs to userspace where padding might not be zeroed, or logic that depends on potentially uninitialized variables. This patch doesn't *add* any such struct, and my test code has no padding issue. The verifier changes are key here. `check_global_ret_scalar_reg` checks if the registers are initialized and rejects the BPF program if they are not. In `prepare_func_exit`, if a subprogram returns a stack pointer in R2, the verifier statically marks R2 as uninitialized, so the BPF program cannot access it. In summary, this patch is *mostly* BPF verifier and JIT changes for 16-byte returns. The verifier does static checks. The patch doesn't add any new kernel data structures, complex allocations, or bounds checks that expose uninitialized memory. The only *potential* risk is if a *future* kfunc returned a struct with uninitialized padding, but this patch doesn't introduce any new such kfuncs. Therefore, no, this patch doesn't require a dedicated KMSAN fuzzing session. The BPF verifier's static analysis is sufficient to prevent new kernel info leaks here, and the changes are within the BPF system and verifier, and do not present a potential for uninitialized memory use as the padding is handled correctly with zero initialization. KASAN and standard BPF verifier tests are sufficient. |
| 8/3 | 2026/08/11 00:59 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch series introduces support for BPF kfuncs and subprograms to return values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. The changes are primarily within the BPF verifier (to track and validate the R2 register) and the JIT compilers (to properly move the return values into the expected BPF registers). \n\nFrom an uninitialized memory perspective:\n1. The BPF verifier statically enforces that BPF subprograms fully initialize both R0 and R2 before returning (via `check_reg_arg(..., SRC_OP)`).\n2. If a static subprogram leaks a stack pointer in R2, the verifier safely marks it as uninitialized (`bpf_mark_reg_not_init`), preventing the caller from reading it. This is a static verifier check, not a dynamic kernel memory issue.\n3. The patch does not introduce any new kernel-to-user data structures, complex allocations, or bounds checking logic that could expose uninitialized kernel memory.\n4. The only new kfuncs introduced are for testing purposes (`bpf_testmod`), and their return structures are fully initialized (e.g., using designated initializers which zero-fill padding).\n\nSince the risk of uninitialized memory usage or info-leaks is statically mitigated by the verifier and no new uninitialized kernel memory paths are introduced, a dedicated KMSAN fuzzing session is not required. Standard KASAN and BPF verifier tests are sufficient.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch series introduces support for BPF kfuncs and subprograms to return values larger than 8 bytes (up to 16 bytes) in the R0:R2 register pair. The changes are primarily within the BPF verifier (to track and validate the R2 register) and the JIT compilers (to properly move the return values into the expected BPF registers). \n\nFrom an uninitialized memory perspective:\n1. The BPF verifier statically enforces that BPF subprograms fully initialize both R0 and R2 before returning (via `check_reg_arg(..., SRC_OP)`).\n2. If a static subprogram leaks a stack pointer in R2, the verifier safely marks it as uninitialized (`bpf_mark_reg_not_init`), preventing the caller from reading it. This is a static verifier check, not a dynamic kernel memory issue.\n3. The patch does not introduce any new kernel-to-user data structures, complex allocations, or bounds checking logic that could expose uninitialized kernel memory.\n4. The only new kfuncs introduced are for testing purposes (`bpf_testmod`), and their return structures are fully initialized (e.g., using designated initializers which zero-fill padding).\n\nSince the risk of uninitialized memory usage or info-leaks is statically mitigated by the verifier and no new uninitialized kernel memory paths are introduced, a dedicated KMSAN fuzzing session is not required. Standard KASAN and BPF verifier tests are sufficient.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|