AI Trajectory


Trajectory: (Expand/Collapse All)
Seq Timestamp Type Name Duration
0/0 2026/10/20 09:15 flow patch-triage
5m Args:
null

Results:
{
  "EnableConfigs": [
    "BPF_SYSCALL",
    "BPF_JIT"
  ],
  "FocusSymbols": [
    "bpf_unwind_target",
    "check_unwind_insns",
    "add_unwind_exentry",
    "emit_unwind_ret_addr",
    "bpf_jit_supports_unwind",
    "bpf_check_attach_target"
  ],
  "KMSANReasoning": "The patch series introduces support for BPF exception unwinding (`unwind_goto` and `unwind_exit`) across the BPF verifier, core runtime, and JIT compilers (x86_64 and arm64).\n\n1. Kernel-to-user info-leak assessment:\nThe only user-facing data transfer touched is in `bpf_insn_prepare_dump()`, where instructions are dumped to user space via `bpf_prog_get_info_by_fd()`. The instruction buffer is allocated using `kmemdup` from the existing `prog-\u003einsnsi` stream, and the patch merely adjusts handling of `BPF_JCOND` so that `unwind_goto` instructions are preserved rather than mistaken for `BPF_CALL_ARGS`. No uninitialized memory, padding, or uninitialized fields are exposed to user space.\n\n2. Uninitialized memory reads / branching:\n- In the JIT compilers (e.g. `add_unwind_exentry()`), all fields of `struct exception_table_entry` (`insn`, `fixup`, `data`) are fully initialized on creation.\n- In the verifier and runtime data structures, new fields such as `unwound` in `struct bpf_verifier_state` and `can_unwind` in `struct bpf_subprog_info` and `struct bpf_prog_aux` are allocated via `kzalloc`/`kzalloc_obj`, properly copied in `bpf_copy_verifier_state()`, and evaluated in `states_equal()`.\n- All newly introduced functions (`add_unwind_exentry()`, `bpf_unwind_target()`, `emit_unwind_ret_addr()`, `check_unwind_insns()`) explicitly initialize their local variables and arguments.\n\n3. KASAN vs KMSAN applicability:\nThe primary risks associated with this patch series involve JIT code generation correctness, return address manipulation on the stack frame, out-of-bounds extable indexing, or verifier branch simulation logic. These issues manifest as out-of-bounds accesses, stack corruption, or kernel panics/crashes, which are covered by KASAN and standard kernel debug facilities. There are no uninitialized memory read risks or info-leaks that specifically necessitate KMSAN fuzzing.",
  "NeedsKMSAN": false,
  "Reasoning": "The patch introduces support for BPF unwinding (BPF_UNWIND_GOTO and BPF_UNWIND_EXIT) in the BPF core verifier and the x86 JIT compiler. It adds new verification logic for unwind instructions, generates JIT code and exception table entries for unwind paths, and introduces a runtime unwind resolution function (bpf_unwind_target) containing runtime assertions (WARN_ON_ONCE). This code is fully reachable in standard QEMU amd64 environments via the bpf() syscall (e.g. BPF_PROG_LOAD and BPF_PROG_TEST_RUN), making it prime for fuzzing.",
  "WorthFuzzing": true
}

1/1 2026/10/20 09:15 action read-patch-diff
0m Args:
null

Results:
{
  "PatchDiff": "commit 880657c9fd431b4343c8c6dcc4398fc89e69a930\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate:   Tue Oct 20 09:14:59 2026 +0000\n\n    syz-cluster: applied patch under review\n\ndiff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h\nindex b2fe6e6dcf447..50029dd9baa63 100644\n--- a/arch/arm64/net/bpf_jit.h\n+++ b/arch/arm64/net/bpf_jit.h\n@@ -316,6 +316,8 @@\n \n #define A64_PACIASP A64_HINT(AARCH64_INSN_HINT_PACIASP)\n #define A64_AUTIASP A64_HINT(AARCH64_INSN_HINT_AUTIASP)\n+#define A64_PACIA1716 A64_HINT(AARCH64_INSN_HINT_PACIA_1716)\n+#define A64_AUTIA1716 A64_HINT(AARCH64_INSN_HINT_AUTIA_1716)\n \n /* BTI */\n #define A64_BTI_C  A64_HINT(AARCH64_INSN_HINT_BTIC)\ndiff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c\nindex 1d5f4cf88d331..45ed69b83265d 100644\n--- a/arch/arm64/net/bpf_jit_comp.c\n+++ b/arch/arm64/net/bpf_jit_comp.c\n@@ -11,6 +11,7 @@\n #include \u003clinux/bitfield.h\u003e\n #include \u003clinux/bpf.h\u003e\n #include \u003clinux/cfi.h\u003e\n+#include \u003clinux/extable.h\u003e\n #include \u003clinux/filter.h\u003e\n #include \u003clinux/memory.h\u003e\n #include \u003clinux/printk.h\u003e\n@@ -1221,6 +1222,85 @@ static int add_exception_handler(const struct bpf_insn *insn,\n \treturn 0;\n }\n \n+/*\n+ * There is no branch for unwind_goto, only an extable entry\n+ *   { .insn = ret_addr - 1, .fixup = pad - ret_addr }\n+ * where ret_addr is the return address of the call in front of it. unwind_exit\n+ * replaces LR in its frame record with bpf_unwind_target() of it.\n+ * ret_addr - 1, because what follows the call can have an entry of its own.\n+ * No insn starts there, so a fault cannot hit the entry.\n+ * 'bl foo; mov x8, x0' is in front of unwind_goto. The pad doesn't miss the mov:\n+ * r0 is x8 and the epilogue of the callee copies it to x0 and leaves x8 as it is.\n+ */\n+static int add_unwind_exentry(struct jit_ctx *ctx, int pad)\n+{\n+\tint ret_addr = ctx-\u003eidx - 1;\n+\tstruct exception_table_entry *ex;\n+\toff_t ins_offset;\n+\n+\tif (!ctx-\u003eimage)\n+\t\t/* First pass */\n+\t\treturn 0;\n+\n+\tif (!ctx-\u003eprog-\u003eaux-\u003eextable ||\n+\t    WARN_ON_ONCE(ctx-\u003eexentry_idx \u003e= ctx-\u003eprog-\u003eaux-\u003enum_exentries))\n+\t\treturn -EINVAL;\n+\n+\tex = \u0026ctx-\u003eprog-\u003eaux-\u003eextable[ctx-\u003eexentry_idx];\n+\tins_offset = (long)\u0026ctx-\u003ero_image[ret_addr] - 1 - (long)\u0026ex-\u003einsn;\n+\tif (WARN_ON_ONCE(ins_offset \u003e= 0 || ins_offset \u003c INT_MIN))\n+\t\treturn -ERANGE;\n+\n+\t/* switch ex to rw buffer for writing */\n+\tex = (void *)ctx-\u003eimage + ((void *)ex - (void *)ctx-\u003ero_image);\n+\tex-\u003einsn = ins_offset;\n+\tex-\u003efixup = (ctx-\u003eoffset[pad] - ret_addr) * AARCH64_INSN_SIZE;\n+\tex-\u003etype = EX_TYPE_BPF;\n+\n+\tctx-\u003eexentry_idx++;\n+\treturn 0;\n+}\n+\n+static u64 bpf_unwind_target(u64 ret_addr)\n+{\n+\tconst struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);\n+\n+\t/* The verifier made sure that unwind_goto follows the call */\n+\tif (WARN_ON_ONCE(!e))\n+\t\treturn ret_addr;\n+\treturn ret_addr + e-\u003efixup;\n+}\n+\n+/*\n+ * unwind_exit: fix LR in the frame record. It's signed with SP at the entry of\n+ * the prog, which is FP + 16. The x16/x17 forms are hints like PACIASP: nops\n+ * without pointer auth. r0 and r2 are returned.\n+ */\n+static void emit_unwind_ret_addr(struct jit_ctx *ctx)\n+{\n+\tconst bool ptr_auth = IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL);\n+\tconst u8 r0 = bpf2a64[BPF_REG_0];\n+\tconst u8 r2 = bpf2a64[BPF_REG_2];\n+\n+\temit(A64_PUSH(r0, r2, A64_SP), ctx);\n+\n+\temit(A64_LDR64I(A64_R(17), A64_FP, 8), ctx);\n+\tif (ptr_auth) {\n+\t\temit(A64_ADD_I(1, A64_R(16), A64_FP, 16), ctx);\n+\t\temit(A64_AUTIA1716, ctx);\n+\t}\n+\temit(A64_MOV(1, A64_R(0), A64_R(17)), ctx);\n+\temit_call((u64)bpf_unwind_target, ctx);\n+\temit(A64_MOV(1, A64_R(17), A64_R(0)), ctx);\n+\tif (ptr_auth) {\n+\t\temit(A64_ADD_I(1, A64_R(16), A64_FP, 16), ctx);\n+\t\temit(A64_PACIA1716, ctx);\n+\t}\n+\temit(A64_STR64I(A64_R(17), A64_FP, 8), ctx);\n+\n+\temit(A64_POP(r0, r2, A64_SP), ctx);\n+}\n+\n static const struct bpf_jit_arg_abi arm64_arg_abi = {\n \t.nr_arg_regs\t\t= 8,\n \t.even_reg_align\t\t= true,\n@@ -1806,8 +1886,16 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn\n \t\tif (emit_bpf_tail_call(ctx))\n \t\t\treturn -EFAULT;\n \t\tbreak;\n+\t/* unwind_goto. The verifier expanded may_goto. */\n+\tcase BPF_JMP | BPF_JCOND:\n+\t\tret = add_unwind_exentry(ctx, i + 1 + off);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t\tbreak;\n \t/* function return */\n \tcase BPF_JMP | BPF_EXIT:\n+\t\tif (bpf_is_unwind_exit(insn) \u0026\u0026 bpf_prog_can_unwind(ctx-\u003eprog))\n+\t\t\temit_unwind_ret_addr(ctx);\n \t\t/* Optimization: when last instruction is EXIT,\n \t\t   simply fallthrough to epilogue. */\n \t\tif (i == ctx-\u003eprog-\u003elen - 1)\n@@ -2466,6 +2554,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)\n \treturn true;\n }\n \n+bool bpf_jit_supports_unwind(void)\n+{\n+\treturn true;\n+}\n+\n const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)\n {\n \treturn \u0026arm64_arg_abi;\ndiff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c\nindex 72a2a0a89e033..c583eed4e9549 100644\n--- a/arch/x86/net/bpf_jit_comp.c\n+++ b/arch/x86/net/bpf_jit_comp.c\n@@ -15,6 +15,7 @@\n #include \u003clinux/memory.h\u003e\n #include \u003clinux/sort.h\u003e\n #include \u003clinux/execmem.h\u003e\n+#include \u003clinux/extable.h\u003e\n #include \u003casm/extable.h\u003e\n #include \u003casm/ftrace.h\u003e\n #include \u003casm/set_memory.h\u003e\n@@ -2511,6 +2512,67 @@ static void adjust_cmp_imm(u32 *op, s32 *imm)\n \t}\n }\n \n+/*\n+ * There is no branch for unwind_goto, only an extable entry\n+ *   { .insn = ret_addr - 1, .fixup = target - ret_addr }\n+ * where ret_addr is the return address of the call in front of it. unwind_exit\n+ * replaces the return address of its frame with bpf_unwind_target() of it.\n+ * ret_addr - 1, because what follows the call can have an entry of its own.\n+ * No insn starts there, so a fault cannot hit the entry.\n+ */\n+static int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,\n+\t\t\t      int ret_addr, int target)\n+{\n+\tstruct exception_table_entry *ex;\n+\ts64 delta;\n+\n+\tif (!prog-\u003eaux-\u003eextable)\n+\t\treturn 0;\n+\tif (*excnt \u003e= prog-\u003eaux-\u003enum_exentries) {\n+\t\tpr_err(\"unwind extable bug\\n\");\n+\t\treturn -EFAULT;\n+\t}\n+\tex = \u0026prog-\u003eaux-\u003eextable[(*excnt)++];\n+\tdelta = image + ret_addr - 1 - (u8 *)\u0026ex-\u003einsn;\n+\tif (!is_simm32(delta)) {\n+\t\tpr_err(\"extable-\u003einsn doesn't fit into 32-bit\\n\");\n+\t\treturn -EFAULT;\n+\t}\n+\t/* switch ex to rw buffer for writes */\n+\tex = (void *)rw_image + ((void *)ex - (void *)image);\n+\tex-\u003einsn = delta;\n+\tex-\u003efixup = target - ret_addr;\n+\tex-\u003edata = EX_TYPE_BPF;\n+\treturn 0;\n+}\n+\n+static u64 bpf_unwind_target(u64 ret_addr)\n+{\n+\tconst struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);\n+\n+\t/* The verifier made sure that unwind_goto follows the call */\n+\tif (WARN_ON_ONCE(!e))\n+\t\treturn ret_addr;\n+\treturn ret_addr + e-\u003efixup;\n+}\n+\n+/* unwind_exit: fix the return address of this frame. r0 and r2 are returned. */\n+static int emit_unwind_ret_addr(u8 **pprog, u8 *ip)\n+{\n+\tu8 *prog = *pprog, *start = prog;\n+\n+\tEMIT2(0x50, 0x56);\t\t\t/* push rax; push rsi */\n+\tEMIT4(0x48, 0x8B, 0x7D, 0x08);\t\t/* mov rdi, [rbp + 8] */\n+\tip += prog - start;\n+\tip += x86_call_depth_emit_accounting(\u0026prog, bpf_unwind_target, ip);\n+\tif (emit_call(\u0026prog, bpf_unwind_target, ip))\n+\t\treturn -EINVAL;\n+\tEMIT4(0x48, 0x89, 0x45, 0x08);\t\t/* mov [rbp + 8], rax */\n+\tEMIT2(0x5E, 0x58);\t\t\t/* pop rsi; pop rax */\n+\t*pprog = prog;\n+\treturn 0;\n+}\n+\n static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,\n \t\t  u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)\n {\n@@ -3493,6 +3555,34 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *\n \t\t\t\tpop_r9(\u0026prog);\n \t\t\tbreak;\n \n+\t\t\t/* unwind_goto. The verifier expanded may_goto. */\n+\t\tcase BPF_JMP | BPF_JCOND:\n+\t\t\tif (!priv_frame_ptr) {\n+\t\t\t\terr = add_unwind_exentry(bpf_prog, image, rw_image, \u0026excnt,\n+\t\t\t\t\t\t\t addrs[i - 1], addrs[i + insn-\u003eoff]);\n+\t\t\t\tif (err)\n+\t\t\t\t\treturn err;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\t/*\n+\t\t\t * 'pop r9' after the call has to be done on the way to\n+\t\t\t * the pad too:\n+\t\t\t *   call foo\n+\t\t\t *   pop r9      \u003c- exit returns here\n+\t\t\t *   jmp 1f\n+\t\t\t *   pop r9      \u003c- unwind_exit returns here\n+\t\t\t *   jmp pad\n+\t\t\t * 1:\n+\t\t\t */\n+\t\t\terr = add_unwind_exentry(bpf_prog, image, rw_image, \u0026excnt,\n+\t\t\t\t\t\t addrs[i - 1] - 2, addrs[i - 1] + 2);\n+\t\t\tif (err)\n+\t\t\t\treturn err;\n+\t\t\tEMIT2(0xEB, 7);\n+\t\t\tpop_r9(\u0026prog);\n+\t\t\tEMIT1_off32(0xE9, addrs[i + insn-\u003eoff] - (addrs[i - 1] + 9));\n+\t\t\tbreak;\n+\n \t\tcase BPF_JMP | BPF_TAIL_CALL:\n \t\t\tif (imm32)\n \t\t\t\temit_bpf_tail_call_direct(bpf_prog,\n@@ -3774,13 +3864,16 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *\n \t\t\tbreak;\n \n \t\tcase BPF_JMP | BPF_EXIT:\n+\t\t\tif (src_reg == BPF_UNWIND_EXIT \u0026\u0026 bpf_prog_can_unwind(bpf_prog) \u0026\u0026\n+\t\t\t    emit_unwind_ret_addr(\u0026prog, ip))\n+\t\t\t\treturn -EINVAL;\n \t\t\tif (seen_exit) {\n \t\t\t\tjmp_offset = ctx-\u003ecleanup_addr - addrs[i];\n \t\t\t\tgoto emit_jmp;\n \t\t\t}\n \t\t\tseen_exit = true;\n \t\t\t/* Update cleanup_addr */\n-\t\t\tctx-\u003ecleanup_addr = proglen;\n+\t\t\tctx-\u003ecleanup_addr = proglen + (prog - temp);\n \t\t\tif (bpf_prog_was_classic(bpf_prog) \u0026\u0026\n \t\t\t    !ns_capable_noaudit(\u0026init_user_ns, CAP_SYS_ADMIN)) {\n \t\t\t\tif (emit_spectre_bhb_barrier(\u0026prog, ip, bpf_prog))\n@@ -5027,6 +5120,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)\n \treturn true;\n }\n \n+bool bpf_jit_supports_unwind(void)\n+{\n+\treturn true;\n+}\n+\n const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)\n {\n \treturn \u0026x86_arg_abi;\ndiff --git a/include/linux/bpf.h b/include/linux/bpf.h\nindex 7750df9734942..580f89c450b55 100644\n--- a/include/linux/bpf.h\n+++ b/include/linux/bpf.h\n@@ -1897,6 +1897,7 @@ struct bpf_prog_aux {\n \tbool changes_pkt_data;\n \tbool has_prologue;\n \tbool might_sleep;\n+\tbool can_unwind;\n \tbool kprobe_write_ctx;\n \t/* has insns that work with the arena base, see bpf_prog_has_arena_insns() */\n \tbool uses_arena_insns;\ndiff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h\nindex 54481da4ed2d2..6606b341351eb 100644\n--- a/include/linux/bpf_verifier.h\n+++ b/include/linux/bpf_verifier.h\n@@ -532,6 +532,8 @@ struct bpf_verifier_state {\n \n \tbool speculative;\n \tbool in_sleepable;\n+\t/* The callee left with unwind_exit. Next insn is unwind_goto. */\n+\tbool unwound;\n \n \t/*\n \t * First and last insn idx of this verifier state.\n@@ -957,6 +959,7 @@ struct bpf_subprog_info {\n \ts16 fastcall_stack_off;\n \tbool has_tail_call: 1;\n \tbool might_throw: 1;\n+\tbool can_unwind: 1; /* has unwind_exit */\n \tbool tail_call_reachable: 1;\n \tbool has_ld_abs: 1;\n \tbool is_cb: 1;\ndiff --git a/include/linux/filter.h b/include/linux/filter.h\nindex 6f4c9bd024a0b..fccd8db336470 100644\n--- a/include/linux/filter.h\n+++ b/include/linux/filter.h\n@@ -254,6 +254,25 @@ static inline bool insn_is_mov_percpu_addr(const struct bpf_insn *insn)\n \t\t.off   = 0,\t\t\t\t\t\\\n \t\t.imm   = 1 })\n \n+static inline bool bpf_is_unwind_goto(const struct bpf_insn *insn)\n+{\n+\treturn insn-\u003ecode == (BPF_JMP | BPF_JCOND) \u0026\u0026 insn-\u003esrc_reg == BPF_UNWIND_GOTO;\n+}\n+\n+static inline bool bpf_is_unwind_exit(const struct bpf_insn *insn)\n+{\n+\treturn insn-\u003ecode == (BPF_JMP | BPF_EXIT) \u0026\u0026 insn-\u003esrc_reg == BPF_UNWIND_EXIT;\n+}\n+\n+/*\n+ * The caller of the main prog doesn't tell unwind_exit from exit. An extension\n+ * is different: its caller is the caller of the subprog that it replaced.\n+ */\n+static inline bool bpf_prog_can_unwind(const struct bpf_prog *prog)\n+{\n+\treturn bpf_is_subprog(prog) || prog-\u003etype == BPF_PROG_TYPE_EXT;\n+}\n+\n static inline bool insn_is_zext(const struct bpf_insn *insn)\n {\n \treturn insn-\u003ecode == (BPF_ALU | BPF_MOV | BPF_X) \u0026\u0026 insn-\u003eimm == 1;\n@@ -1257,6 +1276,7 @@ bool bpf_jit_supports_percpu_insn(void);\n bool bpf_jit_supports_kfunc_call(void);\n bool bpf_jit_supports_callx(void);\n bool bpf_jit_supports_kfunc_ret_reg_pair(void);\n+bool bpf_jit_supports_unwind(void);\n bool bpf_jit_supports_stack_args(void);\n bool bpf_jit_supports_arena_args(void);\n bool bpf_jit_supports_far_kfunc_call(void);\ndiff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h\nindex 6f70db7515dd4..6efe0ca036248 100644\n--- a/include/uapi/linux/bpf.h\n+++ b/include/uapi/linux/bpf.h\n@@ -56,8 +56,19 @@\n \n enum bpf_cond_pseudo_jmp {\n \tBPF_MAY_GOTO = 0,\n+\t/*\n+\t * Follows a call of a bpf subprog. Taken when the subprog left with\n+\t * BPF_EXIT that has src_reg == BPF_UNWIND_EXIT.\n+\t */\n+\tBPF_UNWIND_GOTO = 1,\n };\n \n+/*\n+ * When bpf_exit-\u003esrc_reg == BPF_UNWIND_EXIT, it's the same as bpf_exit, except\n+ * that BPF_UNWIND_GOTO after the call is taken.\n+ */\n+#define BPF_UNWIND_EXIT\t1\n+\n /* Register numbers */\n enum {\n \tBPF_REG_0 = 0,\ndiff --git a/kernel/bpf/const_fold.c b/kernel/bpf/const_fold.c\nindex 8dd8c5958eb85..f9edfde965dc7 100644\n--- a/kernel/bpf/const_fold.c\n+++ b/kernel/bpf/const_fold.c\n@@ -357,7 +357,7 @@ int bpf_prune_dead_branches(struct bpf_verifier_env *env)\n \n \t\tif (!bpf_insn_is_cond_jump(insn-\u003ecode))\n \t\t\tcontinue;\n-\t\tif (bpf_is_may_goto_insn(insn))\n+\t\tif (BPF_OP(insn-\u003ecode) == BPF_JCOND)\n \t\t\tcontinue;\n \n \t\tif (!(aux-\u003econst_reg_mask \u0026 BIT(insn-\u003edst_reg)))\ndiff --git a/kernel/bpf/core.c b/kernel/bpf/core.c\nindex fc22d66d194a9..3030beb531d99 100644\n--- a/kernel/bpf/core.c\n+++ b/kernel/bpf/core.c\n@@ -3399,6 +3399,12 @@ bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)\n \treturn false;\n }\n \n+/* unwind_goto and unwind_exit */\n+bool __weak bpf_jit_supports_unwind(void)\n+{\n+\treturn false;\n+}\n+\n /*\n  * How this arch places a by-value kfunc argument, or NULL for one that has\n  * not opted in and so only takes an argument of a single eightbyte, which\ndiff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c\nindex 36d3228d77454..889fc074496f1 100644\n--- a/kernel/bpf/disasm.c\n+++ b/kernel/bpf/disasm.c\n@@ -375,11 +375,16 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,\n \t\t\t   insn-\u003esrc_reg == BPF_MAY_GOTO) {\n \t\t\tverbose(cbs-\u003eprivate_data, \"(%02x) may_goto pc%+d\",\n \t\t\t\tinsn-\u003ecode, insn-\u003eoff);\n+\t\t} else if (insn-\u003ecode == (BPF_JMP | BPF_JCOND) \u0026\u0026\n+\t\t\t   insn-\u003esrc_reg == BPF_UNWIND_GOTO) {\n+\t\t\tverbose(cbs-\u003eprivate_data, \"(%02x) unwind_goto pc%+d\",\n+\t\t\t\tinsn-\u003ecode, insn-\u003eoff);\n \t\t} else if (insn-\u003ecode == (BPF_JMP32 | BPF_JA)) {\n \t\t\tverbose(cbs-\u003eprivate_data, \"(%02x) gotol pc%+d\",\n \t\t\t\tinsn-\u003ecode, insn-\u003eimm);\n \t\t} else if (insn-\u003ecode == (BPF_JMP | BPF_EXIT)) {\n-\t\t\tverbose(cbs-\u003eprivate_data, \"(%02x) exit\", insn-\u003ecode);\n+\t\t\tverbose(cbs-\u003eprivate_data, \"(%02x) %s\", insn-\u003ecode,\n+\t\t\t\tinsn-\u003esrc_reg == BPF_UNWIND_EXIT ? \"unwind_exit\" : \"exit\");\n \t\t} else if (BPF_SRC(insn-\u003ecode) == BPF_X) {\n \t\t\tverbose(cbs-\u003eprivate_data,\n \t\t\t\t\"(%02x) if %c%d %s %c%d goto pc%+d\",\ndiff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c\nindex 409b9104c723e..9ae3a81604483 100644\n--- a/kernel/bpf/fixups.c\n+++ b/kernel/bpf/fixups.c\n@@ -666,6 +666,10 @@ void bpf_opt_hard_wire_dead_code_branches(struct bpf_verifier_env *env)\n \t\tif (!bpf_insn_is_cond_jump(insn-\u003ecode))\n \t\t\tcontinue;\n \n+\t\t/* unwind_exit of a callee that never exits needs it too */\n+\t\tif (bpf_is_unwind_goto(insn) \u0026\u0026 aux_data[i + 1 + insn-\u003eoff].seen)\n+\t\t\tcontinue;\n+\n \t\tif (!aux_data[i + 1].seen)\n \t\t\tja.off = insn-\u003eoff;\n \t\telse if (!aux_data[i + 1 + insn-\u003eoff].seen)\n@@ -1467,12 +1471,15 @@ static int jit_subprogs(struct bpf_verifier_env *env)\n \t\t\tif (BPF_CLASS(insn-\u003ecode) == BPF_STX \u0026\u0026\n \t\t\t     BPF_MODE(insn-\u003ecode) == BPF_PROBE_ATOMIC)\n \t\t\t\tnum_exentries++;\n+\t\t\tif (bpf_is_unwind_goto(insn))\n+\t\t\t\tnum_exentries++;\n \t\t}\n \t\tfunc[i]-\u003eaux-\u003enum_exentries = num_exentries;\n \t\tfunc[i]-\u003eaux-\u003etail_call_reachable = env-\u003esubprog_info[i].tail_call_reachable;\n \t\tfunc[i]-\u003eaux-\u003eexception_cb = env-\u003esubprog_info[i].is_exception_cb;\n \t\tfunc[i]-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[i].changes_pkt_data;\n \t\tfunc[i]-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[i].might_sleep;\n+\t\tfunc[i]-\u003eaux-\u003ecan_unwind = env-\u003esubprog_info[i].can_unwind;\n \t\tfunc[i]-\u003eaux-\u003etoken = prog-\u003eaux-\u003etoken;\n \t\tif (!i)\n \t\t\tfunc[i]-\u003eaux-\u003eexception_boundary = env-\u003eseen_exception;\ndiff --git a/kernel/bpf/states.c b/kernel/bpf/states.c\nindex b3d250caaaa65..d7df6632a5c47 100644\n--- a/kernel/bpf/states.c\n+++ b/kernel/bpf/states.c\n@@ -1046,6 +1046,9 @@ static bool states_equal(struct bpf_verifier_env *env,\n \tif (old-\u003ein_sleepable != cur-\u003ein_sleepable)\n \t\treturn false;\n \n+\tif (old-\u003eunwound != cur-\u003eunwound)\n+\t\treturn false;\n+\n \tif (!refsafe(old, cur, \u0026env-\u003eidmap_scratch))\n \t\treturn false;\n \ndiff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c\nindex 849bc9136299d..90bc7230afa8a 100644\n--- a/kernel/bpf/syscall.c\n+++ b/kernel/bpf/syscall.c\n@@ -5087,6 +5087,12 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,\n \t\t\tinsns[i].imm = BPF_FUNC_tail_call;\n \t\t\t/* fall-through */\n \t\t}\n+\t\t/*\n+\t\t * BPF_JCOND is the same opcode as BPF_CALL_ARGS. The latter is for\n+\t\t * the interpreter, and a prog with unwind_goto is always JITed.\n+\t\t */\n+\t\tif (prog-\u003ejited \u0026\u0026 bpf_is_unwind_goto(\u0026insns[i]))\n+\t\t\tcontinue;\n \t\tif (code == (BPF_JMP | BPF_CALL) ||\n \t\t    code == (BPF_JMP | BPF_CALL_ARGS)) {\n \t\t\t/* Restore the legacy xlated dump layout. */\ndiff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c\nindex 01434e1d23b59..6c56675c3298d 100644\n--- a/kernel/bpf/verifier.c\n+++ b/kernel/bpf/verifier.c\n@@ -1769,6 +1769,7 @@ int bpf_copy_verifier_state(struct bpf_verifier_state *dst_state,\n \t\treturn err;\n \tdst_state-\u003especulative = src-\u003especulative;\n \tdst_state-\u003ein_sleepable = src-\u003ein_sleepable;\n+\tdst_state-\u003eunwound = src-\u003eunwound;\n \tdst_state-\u003ecurframe = src-\u003ecurframe;\n \tdst_state-\u003ebranches = src-\u003ebranches;\n \tdst_state-\u003eparent = src-\u003eparent;\n@@ -3430,6 +3431,46 @@ static int next_callx_callee(struct bpf_verifier_env *env, int caller, int from)\n \treturn bit \u003c end ? bit - caller * cnt : -1;\n }\n \n+/*\n+ * 'call foo; unwind_goto pad' goes to the pad when foo left with unwind_exit.\n+ * JITs don't emit a branch for it. unwind_exit finds the pad by its return\n+ * address. So unwind_goto has to follow the call and nothing may jump to it.\n+ */\n+static int check_unwind_insns(struct bpf_verifier_env *env)\n+{\n+\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n+\tbool seen = false;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c env-\u003eprog-\u003elen; i++, insn++) {\n+\t\tif (bpf_is_unwind_exit(insn)) {\n+\t\t\tbpf_find_containing_subprog(env, i)-\u003ecan_unwind = true;\n+\t\t\tseen = true;\n+\t\t}\n+\t\tif (!bpf_is_unwind_goto(insn))\n+\t\t\tcontinue;\n+\t\tseen = true;\n+\t\tif (!i || !(bpf_pseudo_call(insn - 1) || bpf_is_callx(insn - 1))) {\n+\t\t\tverbose(env, \"unwind_goto at insn %d doesn't follow a call of a subprog\\n\",\n+\t\t\t\ti);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tif (bpf_is_jump_target(env, i)) {\n+\t\t\tverbose(env, \"jump to unwind_goto at insn %d\\n\", i);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t}\n+\tif (!seen)\n+\t\treturn 0;\n+\tif (!bpf_jit_supports_unwind()) {\n+\t\tverbose(env, \"JIT does not support unwinding\\n\");\n+\t\treturn -ENOTSUPP;\n+\t}\n+\tenv-\u003eprog-\u003ejit_required = 1;\n+\tenv-\u003eprog-\u003eaux-\u003ecan_unwind = env-\u003esubprog_info[0].can_unwind;\n+\treturn 0;\n+}\n+\n /*\n  * Sort subprogs in topological order so that leaf subprogs come first and\n  * their callers come later. This is a DFS post-order traversal of the call\n@@ -11584,6 +11625,19 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n \t\t\t}\n \t\t}\n \n+\t\tif (env-\u003esubprog_info[subprog].can_unwind) {\n+\t\t\tstruct bpf_verifier_state *branch;\n+\n+\t\t\tif (!bpf_is_unwind_goto(insn + 1)) {\n+\t\t\t\tverbose(env, \"no unwind_goto after the call of %s()\\n\", sub_name);\n+\t\t\t\treturn -EINVAL;\n+\t\t\t}\n+\t\t\tbranch = push_stack(env, *insn_idx + 1, *insn_idx, false);\n+\t\t\tif (IS_ERR(branch))\n+\t\t\t\treturn PTR_ERR(branch);\n+\t\t\tbranch-\u003eunwound = true;\n+\t\t}\n+\n \t\tif (env-\u003esubprog_info[subprog].might_throw) {\n \t\t\tstruct bpf_verifier_state *branch;\n \n@@ -12057,6 +12111,19 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)\n \t}\n \n \tcaller = state-\u003eframe[state-\u003ecurframe - 1];\n+\tif (bpf_is_unwind_exit(\u0026env-\u003eprog-\u003einsnsi[*insn_idx])) {\n+\t\t/* the helper or the kfunc that called it doesn't expect that */\n+\t\tif (callee-\u003ein_callback_fn) {\n+\t\t\tverbose(env, \"callback cannot unwind\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tif (!bpf_is_unwind_goto(\u0026env-\u003eprog-\u003einsnsi[callee-\u003ecallsite + 1])) {\n+\t\t\tverbose(env, \"unwind_exit, but no unwind_goto after the call at insn %d\\n\",\n+\t\t\t\tcallee-\u003ecallsite);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tstate-\u003eunwound = true;\n+\t}\n \tif (callee-\u003ein_callback_fn) {\n \t\tif (r0-\u003etype != SCALAR_VALUE) {\n \t\t\tverbose(env, \"R0 not a scalar value\\n\");\n@@ -18759,6 +18826,23 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,\n \t\treturn -EINVAL;\n \t}\n \n+\tif (bpf_is_unwind_goto(insn)) {\n+\t\tif (!env-\u003ecur_state-\u003eunwound)\n+\t\t\treturn 0;\n+\t\tenv-\u003ecur_state-\u003eunwound = false;\n+\t\t/*\n+\t\t * unwind_exit changed its return address, so the return is\n+\t\t * mispredicted and what follows unwind_goto runs speculatively.\n+\t\t */\n+\t\tif (!env-\u003ebypass_spec_v1) {\n+\t\t\terr = sanitize_speculative_path(env, NULL, *insn_idx + 1, *insn_idx);\n+\t\t\tif (err)\n+\t\t\t\treturn err;\n+\t\t}\n+\t\t*insn_idx += insn-\u003eoff;\n+\t\treturn 0;\n+\t}\n+\n \tif (opcode == BPF_JCOND) {\n \t\tstruct bpf_verifier_state *cur_st = env-\u003ecur_state, *queued_st, *prev_st;\n \t\tint idx = *insn_idx;\n@@ -19986,6 +20070,12 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,\n \t * Exit from async callback implies return code restrictions\n \t * that depend on async scheduling mechanism.\n \t */\n+\tif ((cur_frame-\u003ein_async_callback_fn || cur_frame-\u003ein_exception_callback_fn) \u0026\u0026\n+\t    bpf_is_unwind_exit(\u0026env-\u003eprog-\u003einsnsi[env-\u003einsn_idx])) {\n+\t\tverbose(env, \"callback cannot unwind\\n\");\n+\t\treturn -EINVAL;\n+\t}\n+\n \tif (cur_frame-\u003esubprogno \u0026\u0026\n \t    !cur_frame-\u003ein_async_callback_fn \u0026\u0026\n \t    !cur_frame-\u003ein_exception_callback_fn)\n@@ -21223,14 +21313,14 @@ static int check_jmp_fields(struct bpf_verifier_env *env, struct bpf_insn *insn)\n \t\treturn 0;\n \tcase BPF_EXIT:\n \t\tif (BPF_SRC(insn-\u003ecode) != BPF_K || insn-\u003eimm != 0 ||\n-\t\t    insn-\u003esrc_reg != BPF_REG_0 || insn-\u003edst_reg != BPF_REG_0 ||\n+\t\t    insn-\u003esrc_reg \u003e BPF_UNWIND_EXIT || insn-\u003edst_reg != BPF_REG_0 ||\n \t\t    class == BPF_JMP32) {\n \t\t\tverbose(env, \"BPF_EXIT uses reserved fields\\n\");\n \t\t\treturn -EINVAL;\n \t\t}\n \t\treturn 0;\n \tcase BPF_JCOND:\n-\t\tif (insn-\u003ecode != (BPF_JMP | BPF_JCOND) || insn-\u003esrc_reg != BPF_MAY_GOTO ||\n+\t\tif (insn-\u003ecode != (BPF_JMP | BPF_JCOND) || insn-\u003esrc_reg \u003e BPF_UNWIND_GOTO ||\n \t\t    insn-\u003edst_reg || insn-\u003eimm) {\n \t\t\tverbose(env, \"invalid may_goto imm %d\\n\", insn-\u003eimm);\n \t\t\treturn -EINVAL;\n@@ -22436,6 +22526,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,\n \t\tstruct bpf_prog_aux *aux = tgt_prog-\u003eaux;\n \t\tbool tgt_changes_pkt_data;\n \t\tbool tgt_might_sleep;\n+\t\tbool tgt_can_unwind;\n \n \t\tif (bpf_prog_is_dev_bound(prog-\u003eaux) \u0026\u0026\n \t\t    !bpf_prog_dev_bound_match(prog, tgt_prog)) {\n@@ -22464,6 +22555,16 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,\n \t\t\t\tprog_extension ? \"Extension\" : \"Tracing\");\n \t\t\treturn -EINVAL;\n \t\t}\n+\t\t/* unwind_exit of the main prog is exit, unless it's an extension */\n+\t\ttgt_can_unwind = (subprog || tgt_prog-\u003etype == BPF_PROG_TYPE_EXT) \u0026\u0026\n+\t\t\t\t (aux-\u003efunc ? aux-\u003efunc[subprog]-\u003eaux-\u003ecan_unwind\n+\t\t\t\t\t    : aux-\u003ecan_unwind);\n+\t\t/* unwind_exit would return to the trampoline that called the target */\n+\t\tif (tgt_can_unwind \u0026\u0026 !prog_extension \u0026\u0026\n+\t\t    prog-\u003eexpected_attach_type != BPF_TRACE_FENTRY) {\n+\t\t\tbpf_log(log, \"Only fentry can attach to %s, which can unwind\\n\", tname);\n+\t\t\treturn -EINVAL;\n+\t\t}\n \t\tconservative = aux-\u003efunc_info_aux[subprog].unreliable;\n \t\tif (prog_extension) {\n \t\t\tif (aux-\u003efunc_info_aux[subprog].linkage != BTF_FUNC_GLOBAL ||\n@@ -22511,6 +22612,12 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,\n \t\t\t\t\t\"Extension program may sleep, while original does not\\n\");\n \t\t\t\treturn -EINVAL;\n \t\t\t}\n+\t\t\t/* There is no unwind_goto after the calls of the original */\n+\t\t\tif (prog-\u003eaux-\u003ecan_unwind \u0026\u0026 !tgt_can_unwind) {\n+\t\t\t\tbpf_log(log,\n+\t\t\t\t\t\"Extension program can unwind, while original does not\\n\");\n+\t\t\t\treturn -EINVAL;\n+\t\t\t}\n \t\t}\n \t\tif (!tgt_prog-\u003ejited) {\n \t\t\tbpf_log(log, \"Can attach to only JITed progs\\n\");\n@@ -23845,6 +23952,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n \tif (ret \u003c 0)\n \t\tgoto skip_full_check;\n \n+\tret = check_unwind_insns(env);\n+\tif (ret \u003c 0)\n+\t\tgoto skip_full_check;\n+\n \tret = bpf_compute_postorder(env);\n \tif (ret \u003c 0)\n \t\tgoto skip_full_check;\ndiff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h\nindex 6f70db7515dd4..6efe0ca036248 100644\n--- a/tools/include/uapi/linux/bpf.h\n+++ b/tools/include/uapi/linux/bpf.h\n@@ -56,8 +56,19 @@\n \n enum bpf_cond_pseudo_jmp {\n \tBPF_MAY_GOTO = 0,\n+\t/*\n+\t * Follows a call of a bpf subprog. Taken when the subprog left with\n+\t * BPF_EXIT that has src_reg == BPF_UNWIND_EXIT.\n+\t */\n+\tBPF_UNWIND_GOTO = 1,\n };\n \n+/*\n+ * When bpf_exit-\u003esrc_reg == BPF_UNWIND_EXIT, it's the same as bpf_exit, except\n+ * that BPF_UNWIND_GOTO after the call is taken.\n+ */\n+#define BPF_UNWIND_EXIT\t1\n+\n /* Register numbers */\n enum {\n \tBPF_REG_0 = 0,\ndiff --git a/tools/testing/selftests/bpf/Makefile.skel b/tools/testing/selftests/bpf/Makefile.skel\nindex 06e297a17156c..6570c48f92b01 100644\n--- a/tools/testing/selftests/bpf/Makefile.skel\n+++ b/tools/testing/selftests/bpf/Makefile.skel\n@@ -153,4 +153,15 @@ $(RUST_OBJS): $(RDIR)/%.bpf.o: progs/%.rs $(RUST_CORE) | $(RDIR)\n endif\n endif\n \n+# Programs in C++. No skeletons: gcc doesn't build them.\n+ifeq ($(BPF_CC),$(CLANG))\n+CXX_OBJS := $(patsubst progs/%.cpp,$(RDIR)/%.bpf.o,$(wildcard progs/*.cpp))\n+BPF_OBJS += $(CXX_OBJS)\n+\n+$(CXX_OBJS): $(RDIR)/%.bpf.o: progs/%.cpp | $(RDIR)\n+\t$(call msg,$(BPF_CC_MSG),$(BINARY),$@)\n+\t$(Q)$(BPF_CC) -x c++ -g -Wall -Werror -fexceptions -funwind-tables -fno-rtti\t\\\n+\t\t$(BPF_CC_FLAGS) -c $\u003c -o $@ $(call skip_on_fail,BPF)\n+endif\n+\n endif # BPF_CC\ndiff --git a/tools/testing/selftests/bpf/prog_tests/unwind_attach.c b/tools/testing/selftests/bpf/prog_tests/unwind_attach.c\nnew file mode 100644\nindex 0000000000000..92eab5b77f62a\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/prog_tests/unwind_attach.c\n@@ -0,0 +1,111 @@\n+// SPDX-License-Identifier: GPL-2.0\n+#include \u003ctest_progs.h\u003e\n+#include \"verifier_unwind.skel.h\"\n+#include \"unwind_attach.skel.h\"\n+\n+#ifndef ENOTSUPP\n+#define ENOTSUPP 524\n+#endif\n+\n+static char log_buf[16 * 1024];\n+\n+static int run(int fd)\n+{\n+\tchar data[64] = {};\n+\tLIBBPF_OPTS(bpf_test_run_opts, opts,\n+\t\t.data_in = data,\n+\t\t.data_size_in = sizeof(data),\n+\t\t.repeat = 1,\n+\t);\n+\n+\tASSERT_OK(bpf_prog_test_run_opts(fd, \u0026opts), \"test_run\");\n+\treturn opts.retval;\n+}\n+\n+/*\n+ * msg: why it cannot be loaded. Otherwise retval is what the target returns\n+ * with the prog attached.\n+ */\n+static void attach(int tgt_fd, const char *func, const char *name, const char *msg, int retval)\n+{\n+\tstruct bpf_link *link = NULL;\n+\tstruct unwind_attach *skel;\n+\tstruct bpf_program *prog;\n+\tint err;\n+\n+\tskel = unwind_attach__open();\n+\tif (!ASSERT_OK_PTR(skel, \"unwind_attach__open\"))\n+\t\treturn;\n+\n+\tprog = bpf_object__find_program_by_name(skel-\u003eobj, name);\n+\tif (!ASSERT_OK_PTR(prog, name))\n+\t\tgoto out;\n+\tbpf_program__set_autoload(prog, true);\n+\tlog_buf[0] = 0;\n+\tbpf_program__set_log_buf(prog, log_buf, sizeof(log_buf));\n+\terr = bpf_program__set_attach_target(prog, tgt_fd, func);\n+\tif (!ASSERT_OK(err, \"set_attach_target\"))\n+\t\tgoto out;\n+\n+\terr = unwind_attach__load(skel);\n+\tif (msg) {\n+\t\tASSERT_ERR(err, name);\n+\t\tASSERT_HAS_SUBSTR(log_buf, msg, name);\n+\t\tgoto out;\n+\t}\n+\tif (!ASSERT_OK(err, name))\n+\t\tgoto out;\n+\n+\tlink = bpf_program__attach(prog);\n+\tif (!ASSERT_OK_PTR(link, \"attach\"))\n+\t\tgoto out;\n+\tASSERT_EQ(run(tgt_fd), retval, \"retval\");\n+\tif (prog == skel-\u003eprogs.fentry_unwind)\n+\t\tASSERT_EQ(skel-\u003ebss-\u003efentry_cnt, 1, \"fentry_cnt\");\n+out:\n+\tbpf_link__destroy(link);\n+\tunwind_attach__destroy(skel);\n+}\n+\n+void test_unwind_attach(void)\n+{\n+\tstruct verifier_unwind *tgt;\n+\tstruct bpf_program *prog;\n+\tint tgt_fd, plain_fd, err;\n+\n+\ttgt = verifier_unwind__open();\n+\tif (!ASSERT_OK_PTR(tgt, \"verifier_unwind__open\"))\n+\t\treturn;\n+\tbpf_object__for_each_program(prog, tgt-\u003eobj)\n+\t\tbpf_program__set_autoload(prog, prog == tgt-\u003eprogs.calls_global_unwinds ||\n+\t\t\t\t\t\tprog == tgt-\u003eprogs.calls_global_never_unwinds);\n+\terr = verifier_unwind__load(tgt);\n+\tif (err == -ENOTSUPP) {\n+\t\tprintf(\"%s:SKIP:JIT does not support unwinding\\n\", __func__);\n+\t\ttest__skip();\n+\t\tgoto out;\n+\t}\n+\tif (!ASSERT_OK(err, \"verifier_unwind__load\"))\n+\t\tgoto out;\n+\t/*\n+\t * It calls global_unwinds(), which unwinds with r0 = 5, and returns r0 + 10\n+\t * from the pad.\n+\t */\n+\ttgt_fd = bpf_program__fd(tgt-\u003eprogs.calls_global_unwinds);\n+\tplain_fd = bpf_program__fd(tgt-\u003eprogs.calls_global_never_unwinds);\n+\tASSERT_EQ(run(tgt_fd), 15, \"retval\");\n+\n+\tattach(tgt_fd, \"global_unwinds\", \"fentry_unwind\", NULL, 15);\n+\t/* unwind_exit would return to the trampoline */\n+\tattach(tgt_fd, \"global_unwinds\", \"fexit_unwind\",\n+\t       \"Only fentry can attach to global_unwinds\", 0);\n+\t/* An extension leaves either way, like what it replaced */\n+\tattach(tgt_fd, \"global_unwinds\", \"freplace_returns\", NULL, 9);\n+\tattach(tgt_fd, \"global_unwinds\", \"freplace_unwinds\", NULL, 13);\n+\t/* There is no unwind_goto after the call */\n+\tattach(plain_fd, \"global_never_unwinds\", \"freplace_unwinds\",\n+\t       \"Extension program can unwind, while original does not\", 0);\n+\tASSERT_EQ(run(tgt_fd), 15, \"retval\");\n+out:\n+\tverifier_unwind__destroy(tgt);\n+}\ndiff --git a/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c b/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c\nnew file mode 100644\nindex 0000000000000..23e9c7566563d\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c\n@@ -0,0 +1,98 @@\n+// SPDX-License-Identifier: GPL-2.0\n+#include \u003ctest_progs.h\u003e\n+#include \u003cbpf/btf.h\u003e\n+\n+#ifndef ENOTSUPP\n+#define ENOTSUPP 524\n+#endif\n+\n+/* The progs are under #ifdef __BPF_FEATURE_UNWIND */\n+static bool llvm_lowers_unwinding(const char *file)\n+{\n+\tstruct btf *btf = btf__parse(file, NULL);\n+\tbool ret;\n+\n+\tret = btf \u0026\u0026 btf__find_by_name_kind(btf, \"catches\", BTF_KIND_FUNC) \u003e 0;\n+\tbtf__free(btf);\n+\treturn ret;\n+}\n+\n+static void run(struct bpf_object *obj, const char *name, int len, int retval, int dtors)\n+{\n+\tstruct bpf_program *prog = bpf_object__find_program_by_name(obj, name);\n+\tstruct bpf_map *map = bpf_object__find_map_by_name(obj, \".data.dtors\");\n+\tchar data[256] = {};\n+\tLIBBPF_OPTS(bpf_test_run_opts, opts,\n+\t\t.data_in = data,\n+\t\t.data_size_in = len,\n+\t\t.repeat = 1,\n+\t);\n+\tint key = 0, val = 0;\n+\n+\tif (!ASSERT_OK_PTR(prog, name) || !ASSERT_OK_PTR(map, \"dtors\"))\n+\t\treturn;\n+\tASSERT_OK(bpf_map__update_elem(map, \u0026key, sizeof(key), \u0026val, sizeof(val), 0), \"clear\");\n+\tASSERT_OK(bpf_prog_test_run_opts(bpf_program__fd(prog), \u0026opts), \"test_run\");\n+\tASSERT_EQ(opts.retval, retval, \"retval\");\n+\tASSERT_OK(bpf_map__lookup_elem(map, \u0026key, sizeof(key), \u0026val, sizeof(val), 0), \"lookup\");\n+\tASSERT_EQ(val, dtors, \"dtors\");\n+}\n+\n+void test_unwind_cxx(void)\n+{\n+\tconst char *file = \"./unwind_cxx.bpf.o\";\n+\tstruct bpf_object *obj;\n+\tstruct bpf_program *prog;\n+\tint err = -EINVAL;\n+\n+\tif (access(file, R_OK)) {\n+\t\tprintf(\"%s:SKIP:gcc doesn't build %s\\n\", __func__, file);\n+\t\ttest__skip();\n+\t\treturn;\n+\t}\n+\tif (!llvm_lowers_unwinding(file)) {\n+\t\tprintf(\"%s:SKIP:llvm doesn't lower unwinding\\n\", __func__);\n+\t\ttest__skip();\n+\t\treturn;\n+\t}\n+\tobj = bpf_object__open_file(file, NULL);\n+\tif (!ASSERT_OK_PTR(obj, \"open\"))\n+\t\treturn;\n+\tprog = bpf_object__find_program_by_name(obj, \"catches_aggregate\");\n+\tif (!ASSERT_OK_PTR(prog, \"catches_aggregate\"))\n+\t\tgoto out;\n+\tbpf_program__set_autoload(prog, false);\n+\terr = bpf_object__load(obj);\n+\tif (err == -ENOTSUPP) {\n+\t\tprintf(\"%s:SKIP:JIT does not support unwinding\\n\", __func__);\n+\t\ttest__skip();\n+\t\tgoto out;\n+\t}\n+\tif (!ASSERT_OK(err, \"load\"))\n+\t\tgoto out;\n+\n+\t/* inner's destructor, then middle's */\n+\trun(obj, \"catches\", 64, 65, 12);\n+\trun(obj, \"catches\", 200, 42, 12);\n+\trun(obj, \"does_not_catch\", 64, 65, 12);\n+\trun(obj, \"does_not_catch\", 200, 0, 12);\n+out:\n+\tbpf_object__close(obj);\n+\tif (err)\n+\t\treturn;\n+\n+\tif (!test__start_subtest(\"aggregate_return\"))\n+\t\treturn;\n+\tobj = bpf_object__open_file(file, NULL);\n+\tif (!ASSERT_OK_PTR(obj, \"open_aggregate\"))\n+\t\treturn;\n+\tbpf_object__for_each_program(prog, obj)\n+\t\tbpf_program__set_autoload(prog,\n+\t\t\t\t\t strcmp(bpf_program__name(prog), \"catches_aggregate\") == 0);\n+\tif (!ASSERT_OK(bpf_object__load(obj), \"load_aggregate\"))\n+\t\tgoto out_aggregate;\n+\trun(obj, \"catches_aggregate\", 64, 71, 0);\n+\trun(obj, \"catches_aggregate\", 200, 42, 0);\n+out_aggregate:\n+\tbpf_object__close(obj);\n+}\ndiff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c\nindex 6df440d826750..04740ba37e918 100644\n--- a/tools/testing/selftests/bpf/prog_tests/verifier.c\n+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c\n@@ -130,6 +130,7 @@\n #include \"verifier_typedef.skel.h\"\n #include \"verifier_uninit.skel.h\"\n #include \"verifier_unpriv.skel.h\"\n+#include \"verifier_unwind.skel.h\"\n #include \"verifier_unpriv_perf.skel.h\"\n #include \"verifier_value_adj_spill.skel.h\"\n #include \"verifier_value.skel.h\"\n@@ -320,6 +321,7 @@ void test_verifier_tailcall_jit(void)         { RUN(verifier_tailcall_jit); }\n void test_verifier_typedef(void)              { RUN(verifier_typedef); }\n void test_verifier_uninit(void)               { RUN(verifier_uninit); }\n void test_verifier_unpriv(void)               { RUN(verifier_unpriv); }\n+void test_verifier_unwind(void)               { RUN(verifier_unwind); }\n void test_verifier_unpriv_perf(void)          { RUN(verifier_unpriv_perf); }\n void test_verifier_value_adj_spill(void)      { RUN(verifier_value_adj_spill); }\n void test_verifier_value(void)                { RUN(verifier_value); }\ndiff --git a/tools/testing/selftests/bpf/progs/unwind_attach.c b/tools/testing/selftests/bpf/progs/unwind_attach.c\nnew file mode 100644\nindex 0000000000000..df73c26c1ff46\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/unwind_attach.c\n@@ -0,0 +1,43 @@\n+// SPDX-License-Identifier: GPL-2.0\n+#include \u003clinux/bpf.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \u003cbpf/bpf_tracing.h\u003e\n+#include \"../../../include/linux/filter.h\"\n+#include \"bpf_misc.h\"\n+\n+/* The targets are global_unwinds() and global_never_unwinds() in verifier_unwind.c */\n+\n+int fentry_cnt;\n+\n+SEC(\"?fentry/global_unwinds\")\n+int BPF_PROG(fentry_unwind)\n+{\n+\tfentry_cnt++;\n+\treturn 0;\n+}\n+\n+SEC(\"?fexit/global_unwinds\")\n+int BPF_PROG(fexit_unwind)\n+{\n+\treturn 0;\n+}\n+\n+SEC(\"?freplace/global_unwinds\")\n+unsigned long freplace_returns(void)\n+{\n+\treturn 9;\n+}\n+\n+SEC(\"?freplace/global_unwinds\")\n+__naked unsigned long freplace_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 3;\"\n+\t\t\".8byte %[unwind_exit];\"\n+\t\t:\n+\t\t: __imm_insn(unwind_exit,\n+\t\t\t     BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT, 0, 0))\n+\t);\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/unwind_cxx.cpp b/tools/testing/selftests/bpf/progs/unwind_cxx.cpp\nnew file mode 100644\nindex 0000000000000..3a113ffea29ca\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/unwind_cxx.cpp\n@@ -0,0 +1,150 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * C++ exceptions. llvm lowers unwinding to two insns:\n+ *   invoke foo unwind pad    call foo; unwind_goto pad\n+ *   resume                   r0 = 0; unwind_exit\n+ * so 'throw' is a jump or an exit, 'catch (...)' and the destructors that run on\n+ * the way are ordinary blocks.\n+ *\n+ * vmlinux.h and bpf_helpers.h are not C++.\n+ */\n+#define SEC(name) __attribute__((section(name), used))\n+#define __ksym __attribute__((section(\".ksyms\")))\n+#define __noinline __attribute__((noinline))\n+#define __always_inline inline __attribute__((always_inline))\n+\n+#ifdef __BPF_FEATURE_UNWIND\n+\n+struct __sk_buff {\n+\tunsigned int len;\n+};\n+\n+extern \"C\" {\n+void bpf_rcu_read_lock(void) noexcept __ksym;\n+void bpf_rcu_read_unlock(void) noexcept __ksym;\n+void bpf_preempt_disable(void) noexcept __ksym;\n+void bpf_preempt_enable(void) noexcept __ksym;\n+\n+/* Starts an unwind. Not a kfunc: llvm lowers the call and it doesn't come back. */\n+int _Unwind_RaiseException(void *exc);\n+\n+/* All that is left of the C++ runtime. Nobody looks at the exception object. */\n+static char exc_obj[16];\n+\n+__always_inline void *__cxa_allocate_exception(unsigned long size) noexcept\n+{\n+\treturn exc_obj;\n+}\n+\n+__always_inline void __cxa_throw(void *exc, void *tinfo, void (*dtor)(void *))\n+{\n+\t_Unwind_RaiseException(exc);\n+}\n+\n+__always_inline void *__cxa_begin_catch(void *exc) noexcept\n+{\n+\treturn exc;\n+}\n+\n+__always_inline void __cxa_end_catch(void) noexcept\n+{\n+}\n+}\n+\n+/* One decimal digit per destructor, in the order they ran */\n+volatile int dtors SEC(\".data.dtors\");\n+\n+/* The verifier insists that both are undone on every path out of the prog */\n+struct rcu_guard {\n+\t__always_inline rcu_guard()\n+\t{\n+\t\tbpf_rcu_read_lock();\n+\t}\n+\t__always_inline ~rcu_guard()\n+\t{\n+\t\tbpf_rcu_read_unlock();\n+\t\tdtors = dtors * 10 + 1;\n+\t}\n+};\n+\n+struct plain_rcu_guard {\n+\t__always_inline plain_rcu_guard()\n+\t{\n+\t\tbpf_rcu_read_lock();\n+\t}\n+\t__always_inline ~plain_rcu_guard()\n+\t{\n+\t\tbpf_rcu_read_unlock();\n+\t}\n+};\n+\n+struct preempt_guard {\n+\t__always_inline preempt_guard()\n+\t{\n+\t\tbpf_preempt_disable();\n+\t}\n+\t__always_inline ~preempt_guard()\n+\t{\n+\t\tbpf_preempt_enable();\n+\t\tdtors = dtors * 10 + 2;\n+\t}\n+};\n+\n+static __noinline int inner(int len)\n+{\n+\trcu_guard guard;\n+\n+\tif (len \u003e 100)\n+\t\tthrow 1;\n+\treturn len;\n+}\n+\n+static __noinline int middle(int len)\n+{\n+\tpreempt_guard guard;\n+\n+\treturn inner(len) + 1;\n+}\n+\n+using u128 = unsigned __int128;\n+\n+/* Global subprogs returning 16 bytes use R0:R2, including unwind exits. */\n+extern \"C\" __noinline u128 aggregate_return(int len)\n+{\n+\tplain_rcu_guard guard;\n+\n+\tif (len \u003e 100)\n+\t\tthrow 1;\n+\treturn (static_cast\u003cu128\u003e(len) \u003c\u003c 64) | 7;\n+}\n+\n+extern \"C\" SEC(\"tc\") int catches(struct __sk_buff *skb)\n+{\n+\ttry {\n+\t\treturn middle(skb-\u003elen);\n+\t} catch (...) {\n+\t\treturn 42;\n+\t}\n+}\n+\n+/* The exception leaves the prog: it returns 0 */\n+extern \"C\" SEC(\"tc\") int does_not_catch(struct __sk_buff *skb)\n+{\n+\treturn middle(skb-\u003elen);\n+}\n+\n+extern \"C\" SEC(\"tc\") int catches_aggregate(struct __sk_buff *skb)\n+{\n+\ttry {\n+\t\tu128 value = aggregate_return(skb-\u003elen);\n+\n+\t\treturn static_cast\u003cunsigned long\u003e(value) +\n+\t\t       static_cast\u003cunsigned long\u003e(value \u003e\u003e 64);\n+\t} catch (...) {\n+\t\treturn 42;\n+\t}\n+}\n+\n+#endif\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/progs/verifier_unwind.c b/tools/testing/selftests/bpf/progs/verifier_unwind.c\nnew file mode 100644\nindex 0000000000000..dbe7b5d91acd4\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/verifier_unwind.c\n@@ -0,0 +1,1182 @@\n+// SPDX-License-Identifier: GPL-2.0\n+#include \u003clinux/bpf.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \"../../../include/linux/filter.h\"\n+#include \"bpf_misc.h\"\n+\n+/*\n+ * llvm lowers unwinding to two insns:\n+ *   invoke foo unwind pad    call foo; unwind_goto pad\n+ *   resume                   unwind_exit\n+ * The tests spell them out as raw insns, so they don't depend on llvm version.\n+ * The argument of unwind_goto() is the number of insns to skip.\n+ */\n+#define unwind_goto(off) \".8byte %[insn_unwind_goto_\" #off \"];\"\n+#define unwind_exit \".8byte %[insn_unwind_exit];\"\n+\n+#define __imm_unwind_goto(off) \\\n+\t__imm_insn(insn_unwind_goto_##off, \\\n+\t\t   BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO, off, 0))\n+#define __imm_unwind_exit \\\n+\t__imm_insn(insn_unwind_exit, \\\n+\t\t   BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT, 0, 0))\n+\n+extern void bpf_rcu_read_lock(void) __ksym;\n+extern void bpf_rcu_read_unlock(void) __ksym;\n+extern void bpf_throw(__u64 cookie) __ksym;\n+\n+struct {\n+\t__uint(type, BPF_MAP_TYPE_PROG_ARRAY);\n+\t__uint(max_entries, 1);\n+\t__uint(key_size, sizeof(int));\n+\t__uint(value_size, sizeof(int));\n+} prog_array SEC(\".maps\");\n+\n+struct timer_elem {\n+\tstruct bpf_timer t;\n+};\n+\n+struct {\n+\t__uint(type, BPF_MAP_TYPE_ARRAY);\n+\t__uint(max_entries, 1);\n+\t__type(key, int);\n+\t__type(value, struct timer_elem);\n+} timer_map SEC(\".maps\");\n+\n+/* The verifier doesn't know what is in there */\n+unsigned long zero;\n+unsigned long one = 1;\n+\n+/*\n+ * The verifier follows a static subprog, so it knows which way it leaves when\n+ * it knows the argument. To have both ways at run time the tests pass skb-\u003elen,\n+ * which is 64, or skb-\u003elen - 64.\n+ *\n+ * r1 == 0: returns 7, otherwise unwinds with r0 = 5\n+ */\n+__naked __noinline __used\n+static unsigned long may_unwind(void)\n+{\n+\tasm volatile (\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\t\"r0 = 5;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t);\n+}\n+\n+/* owns nothing: its pad only carries on */\n+__naked __noinline __used\n+static unsigned long passes_on(void)\n+{\n+\tasm volatile (\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(2)\n+\t\t\"r0 += 1;\"\n+\t\t\"exit;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_goto(2),\n+\t\t  __imm_unwind_exit\n+\t);\n+}\n+\n+__naked __noinline __used\n+static unsigned long always_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 5;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t);\n+}\n+\n+__naked __noinline __used\n+static unsigned long never_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t);\n+}\n+\n+/* Both can unwind. This one returns 7. */\n+__naked __noinline __used\n+unsigned long global_returns(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = %[zero] ll;\"\n+\t\t\"r1 = *(u64 *)(r1 + 0);\"\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\t\"r0 = 5;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_addr(zero),\n+\t\t  __imm_unwind_exit\n+\t);\n+}\n+\n+/* This one unwinds with r0 = 5 */\n+__naked __noinline __used\n+unsigned long global_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = %[one] ll;\"\n+\t\t\"r1 = *(u64 *)(r1 + 0);\"\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\t\"r0 = 5;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_addr(one),\n+\t\t  __imm_unwind_exit\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: the callee returns\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int callee_returns(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"r1 -= 64;\"\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: the callee unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(42)\n+__naked int callee_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: return through two frames\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(8)\n+__naked int two_frames_return(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"r1 -= 64;\"\n+\t\t\"call passes_on;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* unwind_exit returns r0 like exit does */\n+SEC(\"tc\")\n+__description(\"unwind: unwind through two frames, r0 reaches the pad\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(15)\n+__naked int two_frames_unwind(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call passes_on;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 += 10;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: r6-r9 and the stack are intact in the pad\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(1111)\n+__naked int pad_sees_callee_saved(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"r6 = 1;\"\n+\t\t\"r7 = 10;\"\n+\t\t\"r8 = 100;\"\n+\t\t\"r9 = 400;\"\n+\t\t\"*(u64 *)(r10 - 8) = 600;\"\n+\t\t\"call passes_on;\"\n+\t\tunwind_goto(2)\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t\"r0 = r6;\"\n+\t\t\"r0 += r7;\"\n+\t\t\"r0 += r8;\"\n+\t\t\"r0 += r9;\"\n+\t\t\"r1 = *(u64 *)(r10 - 8);\"\n+\t\t\"r0 += r1;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(2)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: r1-r5 are not readable in the pad\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"R3 !read_ok\")\n+__naked int pad_reads_r3(void)\n+{\n+\tasm volatile (\n+\t\t\"r3 = 0;\"\n+\t\t\"call always_unwinds;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = r3;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unwind_exit of the main prog is exit\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(3)\n+__naked int main_prog_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 3;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unwind_exit of the main prog needs r0\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"R0 !read_ok\")\n+__naked int main_prog_unwinds_no_r0(void)\n+{\n+\tasm volatile (\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* What follows unwind_goto is dead code. unwind_exit still has to find the pad. */\n+SEC(\"tc\")\n+__description(\"unwind: the callee never returns\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(42)\n+__xlated(\"0: call pc+3\")\n+__xlated(\"1: unwind_goto pc+0\")\n+__xlated(\"2: r0 = 42\")\n+__xlated(\"3: exit\")\n+__naked int callee_never_returns(void)\n+{\n+\tasm volatile (\n+\t\t\"call always_unwinds;\"\n+\t\tunwind_goto(2)\n+\t\t\"r0 = 1;\"\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(2)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* The pad is dead code and goes away together with unwind_goto */\n+SEC(\"tc\")\n+__description(\"unwind: the callee never unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__xlated(\"0: call pc+1\")\n+__xlated(\"1: exit\")\n+__xlated(\"2: r0 = 7\")\n+__xlated(\"3: exit\")\n+__naked int callee_never_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"call never_unwinds;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = *(u64 *)(r0 + 0);\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: the pad drops what the frame owns\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(42)\n+__naked int pad_unlocks(void)\n+{\n+\tasm volatile (\n+\t\t\"r6 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call %[bpf_rcu_read_lock];\"\n+\t\t\"r1 = r6;\"\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(3)\n+\t\t\"call %[bpf_rcu_read_unlock];\"\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t\"call %[bpf_rcu_read_unlock];\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm(bpf_rcu_read_lock),\n+\t\t  __imm(bpf_rcu_read_unlock),\n+\t\t  __imm_unwind_goto(3)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: the pad is checked like any other code\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"in main prog cannot be used inside bpf_rcu_read_lock-ed region\")\n+__naked int pad_does_not_unlock(void)\n+{\n+\tasm volatile (\n+\t\t\"r6 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call %[bpf_rcu_read_lock];\"\n+\t\t\"r1 = r6;\"\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(3)\n+\t\t\"call %[bpf_rcu_read_unlock];\"\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm(bpf_rcu_read_lock),\n+\t\t  __imm(bpf_rcu_read_unlock),\n+\t\t  __imm_unwind_goto(3)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* Leaves either way with the same registers */\n+__naked __noinline __used\n+static unsigned long same_regs(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 0;\"\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t);\n+}\n+\n+/*\n+ * After the call the two states differ only in the way the callee left.\n+ * The second one must not be pruned, or nobody looks at the pad.\n+ */\n+SEC(\"tc\")\n+__description(\"unwind: the way the callee left is a part of the state\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"R0 invalid mem access 'scalar'\")\n+__flag(BPF_F_TEST_STATE_FREQ)\n+__naked int no_pruning_after_the_call(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call same_regs;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = *(u64 *)(r0 + 0);\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* Returns ctx, or unwinds with a number in r0 */\n+__naked __noinline __used\n+static unsigned long ctx_or_unwind(void)\n+{\n+\tasm volatile (\n+\t\t\"r2 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"if r2 \u003e 100 goto 1f;\"\n+\t\t\"r0 = r1;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\t\"r0 = 0;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_exit\n+\t);\n+}\n+\n+/*\n+ * unwind_exit changes its return address, so the cpu mispredicts the return and\n+ * runs what follows unwind_goto with r0 of unwind_exit.\n+ */\n+SEC(\"socket\")\n+__description(\"unwind: what follows unwind_goto runs speculatively\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(50) /* 64 without the ethernet header */\n+__success_unpriv __retval_unpriv(50)\n+__caps_unpriv(CAP_BPF)\n+#ifdef SPEC_V1\n+__xlated_unpriv(\"unwind_goto pc+{{.*}}\")\n+__xlated_unpriv(\"nospec\") /* inserted to prevent `R0 invalid mem access 'scalar'` */\n+__xlated_unpriv(\"r0 = *(u32 *)(r0 +{{.*}})\")\n+#endif\n+__naked int speculative_fall_through(void)\n+{\n+\tasm volatile (\n+\t\t\"call ctx_or_unwind;\"\n+\t\tunwind_goto(2)\n+\t\t\"r0 = *(u32 *)(r0 + %[len]);\"\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(2)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: a global subprog returns\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int calls_global_returns(void)\n+{\n+\tasm volatile (\n+\t\t\"call global_returns;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: a global subprog unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(15)\n+__naked int calls_global_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"call global_unwinds;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 += 10;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* The verifier doesn't look inside */\n+SEC(\"tc\")\n+__description(\"unwind: the pad of a call of a global subprog is always checked\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"R0 invalid mem access 'scalar'\")\n+__naked int global_pad_is_checked(void)\n+{\n+\tasm volatile (\n+\t\t\"call global_returns;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = *(u64 *)(r0 + 0);\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+__naked __noinline __used\n+unsigned long global_never_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: a global subprog that cannot unwind\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int calls_global_never_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"call global_never_unwinds;\"\n+\t\t\"exit;\"\n+\t\t::: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: no unwind_goto after a call of a global subprog\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"no unwind_goto after the call of global_returns()\")\n+__naked int global_no_unwind_goto(void)\n+{\n+\tasm volatile (\n+\t\t\"call global_returns;\"\n+\t\t\"exit;\"\n+\t\t::: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: no unwind_goto after a call of a static subprog\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"unwind_exit, but no unwind_goto after the call at insn 1\")\n+__naked int static_no_unwind_goto(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call may_unwind;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len))\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* The verifier sees that it doesn't get to unwind_exit */\n+SEC(\"tc\")\n+__description(\"unwind: no unwind_goto is fine when the callee doesn't unwind\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int static_does_not_get_there(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = 0;\"\n+\t\t\"call may_unwind;\"\n+\t\t\"exit;\"\n+\t\t::: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: callx, the callee returns\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int callx_returns(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"r1 -= 64;\"\n+\t\t\"r6 = %[may_unwind] ll;\"\n+\t\t\"callx r6;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_addr(may_unwind),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: callx, the callee unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(42)\n+__naked int callx_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"r6 = %[may_unwind] ll;\"\n+\t\t\"callx r6;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_addr(may_unwind),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unwind_goto after a plain insn\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"unwind_goto at insn 1 doesn't follow a call of a subprog\")\n+__naked int unwind_goto_after_mov(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 0;\"\n+\t\tunwind_goto(0)\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(0)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unwind_goto is the first insn\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"unwind_goto at insn 0 doesn't follow a call of a subprog\")\n+__naked int unwind_goto_first(void)\n+{\n+\tasm volatile (\n+\t\tunwind_goto(0)\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(0)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unwind_goto after a call of a helper\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"unwind_goto at insn 1 doesn't follow a call of a subprog\")\n+__naked int unwind_goto_after_helper(void)\n+{\n+\tasm volatile (\n+\t\t\"call %[bpf_get_prandom_u32];\"\n+\t\tunwind_goto(0)\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm(bpf_get_prandom_u32),\n+\t\t  __imm_unwind_goto(0)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unwind_goto after a call of a kfunc\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"unwind_goto at insn 1 doesn't follow a call of a subprog\")\n+__naked int unwind_goto_after_kfunc(void)\n+{\n+\tasm volatile (\n+\t\t\"call %[bpf_rcu_read_lock];\"\n+\t\tunwind_goto(0)\n+\t\t\"call %[bpf_rcu_read_unlock];\"\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm(bpf_rcu_read_lock),\n+\t\t  __imm(bpf_rcu_read_unlock),\n+\t\t  __imm_unwind_goto(0)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: jump to unwind_goto\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"jump to unwind_goto at insn 3\")\n+__naked int jump_to_unwind_goto(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 0;\"\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"call never_unwinds;\"\n+\t\"1:\"\n+\t\tunwind_goto(0)\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(0)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: reserved fields of unwind_goto\")\n+__failure __msg(\"invalid may_goto imm 1\")\n+__naked int unwind_goto_reserved(void)\n+{\n+\tasm volatile (\n+\t\t\"call never_unwinds;\"\n+\t\t\".8byte %[bad];\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO, 0, 1))\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unknown kind of jcond\")\n+__failure __msg(\"invalid may_goto imm 0\")\n+__naked int jcond_unknown(void)\n+{\n+\tasm volatile (\n+\t\t\"call never_unwinds;\"\n+\t\t\".8byte %[bad];\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO + 1, 0, 0))\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: unknown kind of exit\")\n+__failure __msg(\"BPF_EXIT uses reserved fields\")\n+__naked int exit_unknown(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 0;\"\n+\t\t\".8byte %[bad];\"\n+\t\t:\n+\t\t: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT + 1, 0, 0))\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* r1 is the index: unwinds in the second iteration */\n+__naked __noinline __used\n+static unsigned long cb_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r0 = 0;\"\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: callback unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"callback cannot unwind\")\n+__naked int callback_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = 3;\"\n+\t\t\"r2 = %[cb_unwinds];\"\n+\t\t\"r3 = 0;\"\n+\t\t\"r4 = 0;\"\n+\t\t\"call %[bpf_loop];\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_ptr(cb_unwinds),\n+\t\t  __imm(bpf_loop)\n+\t\t: __clobber_common\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: async callback unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__failure __msg(\"callback cannot unwind\")\n+int async_callback_unwinds(void *ctx)\n+{\n+\tstruct timer_elem *val;\n+\tint key = 0;\n+\n+\tval = bpf_map_lookup_elem(\u0026timer_map, \u0026key);\n+\tif (!val)\n+\t\treturn 0;\n+\n+\tbpf_timer_init(\u0026val-\u003et, \u0026timer_map, 0);\n+\tbpf_timer_set_callback(\u0026val-\u003et, (void *)always_unwinds);\n+\treturn 0;\n+}\n+\n+/* Not naked: the argument has to have a name in BTF */\n+__noinline int exception_cb_unwinds(__u64 cookie)\n+{\n+\tif (cookie)\n+\t\tasm volatile (\"r0 = 0;\" unwind_exit :: __imm_unwind_exit : \"r0\");\n+\treturn 0;\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: exception callback unwinds\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__attribute__((btf_decl_tag(\"exception_callback:exception_cb_unwinds\")))\n+__failure __msg(\"callback cannot unwind\")\n+int exception_callback_unwinds(struct __sk_buff *skb)\n+{\n+\tbpf_throw(skb-\u003elen);\n+\treturn 0;\n+}\n+\n+/* What it calls can unwind as long as it doesn't let it out */\n+__naked __noinline __used\n+static unsigned long cb_catches(void)\n+{\n+\tasm volatile (\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(0)\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(0)\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: callback catches\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(3)\n+__naked int callback_catches(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = 3;\"\n+\t\t\"r2 = %[cb_catches];\"\n+\t\t\"r3 = 0;\"\n+\t\t\"r4 = 0;\"\n+\t\t\"call %[bpf_loop];\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_ptr(cb_catches),\n+\t\t  __imm(bpf_loop)\n+\t\t: __clobber_common\n+\t);\n+}\n+\n+/* Both leave the subprog like exit does */\n+__naked __noinline __used\n+static unsigned long does_tail_call(void)\n+{\n+\tasm volatile (\n+\t\t\"r6 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"r2 = %[prog_array] ll;\"\n+\t\t\"r3 = 0;\"\n+\t\t\"call %[bpf_tail_call];\"\n+\t\t\"r0 = 7;\"\n+\t\t\"if r6 == 64 goto 1f;\"\n+\t\tunwind_exit\n+\t\"1:\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm(bpf_tail_call),\n+\t\t  __imm_addr(prog_array),\n+\t\t  __imm_unwind_exit\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: tail call in a subprog that can unwind\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int tail_call_in_subprog(void)\n+{\n+\tasm volatile (\n+\t\t\"call does_tail_call;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* The packet is shorter than that: ld_abs leaves the subprog with r0 = 0 */\n+__naked __noinline __used\n+static unsigned long does_ld_abs(void)\n+{\n+\tasm volatile (\n+\t\t\"r6 = r1;\"\n+\t\t\"r0 = *(u8 *)skb[1000];\"\n+\t\t\"r0 = 5;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t);\n+}\n+\n+SEC(\"tc\")\n+__description(\"unwind: ld_abs in a subprog that can unwind\")\n+__arch_x86_64\n+__arch_arm64\n+__load_if_JITed()\n+__success __retval(7)\n+__naked int ld_abs_in_subprog(void)\n+{\n+\tasm volatile (\n+\t\t\"call does_ld_abs;\"\n+\t\tunwind_goto(2)\n+\t\t\"r0 += 7;\"\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_unwind_goto(2)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/*\n+ * JITs don't emit anything for unwind_goto. It's an extable entry, where\n+ * unwind_exit looks its return address up.\n+ */\n+SEC(\"tc\")\n+__description(\"unwind: jit\")\n+__success __retval(42)\n+__load_if_JITed()\n+__arch_x86_64\n+__jited(\"\tcallq\t0x{{.*}}\")\n+__jited(\"\tmovl\t$0x7, %eax\")\n+__jited(\"...\")\n+__jited(\"\tmovl\t$0x5, %eax\")\n+__jited(\"\tpushq\t%rax\")\n+__jited(\"\tpushq\t%rsi\")\n+__jited(\"\tmovq\t0x8(%rbp), %rdi\")\n+__jited(\"...\") /* call depth accounting */\n+__jited(\"\tcallq\t0x{{.*}}\")\n+__jited(\"\tmovq\t%rax, 0x8(%rbp)\")\n+__jited(\"\tpopq\t%rsi\")\n+__jited(\"\tpopq\t%rax\")\n+__arch_arm64\n+__jited(\"\tbl\t{{.*}}\")\n+__jited(\"\tmov\tx8, x0\")\n+__jited(\"\tmov\tx8, #0x7\")\n+__jited(\"...\")\n+__jited(\"\tmov\tx8, #0x5\")\n+__jited(\"\tstp\tx8, x1, [sp, #-0x10]!\")\n+__jited(\"\tldr\tx17, [x29, #0x8]\")\n+__jited(\"...\")\n+__jited(\"\tstr\tx17, [x29, #0x8]\")\n+__jited(\"\tldp\tx8, x1, [sp], #0x10\")\n+__naked int jit(void)\n+{\n+\tasm volatile (\n+\t\t\"r1 = *(u32 *)(r1 + %[len]);\"\n+\t\t\"call may_unwind;\"\n+\t\tunwind_goto(2)\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t\t\"r0 = 42;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_const(len, offsetof(struct __sk_buff, len)),\n+\t\t  __imm_unwind_goto(2)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/* r1 == 0: returns 7, otherwise unwinds with r0 = 5 */\n+__naked __noinline __used\n+static unsigned long may_unwind_big_stack(void)\n+{\n+\tasm volatile (\n+\t\t\"*(u64 *)(r10 - 200) = r1;\"\n+\t\t\"if r1 != 0 goto 1f;\"\n+\t\t\"r0 = 7;\"\n+\t\t\"exit;\"\n+\t\"1:\"\n+\t\t\"r0 = 5;\"\n+\t\tunwind_exit\n+\t\t:\n+\t\t: __imm_unwind_exit\n+\t);\n+}\n+\n+/*\n+ * x86-64 keeps the pointer to the private stack in r9 and saves it around a call.\n+ * The value that is read back in the pad is there only when r9 is restored.\n+ */\n+SEC(\"raw_tp\")\n+__description(\"unwind: private stack, the callee unwinds\")\n+__success __retval(47)\n+__load_if_JITed()\n+__arch_x86_64\n+__jited(\"\tpushq\t%r9\")\n+__jited(\"...\") /* call depth accounting */\n+__jited(\"\tcallq\t0x{{.*}}\")\n+__jited(\"\tpopq\t%r9\")\n+__jited(\"\tjmp\tL{{.*}}\")\n+__jited(\"\tpopq\t%r9\")\n+__jited(\"\tjmp\tL{{.*}}\")\n+__arch_arm64\n+__naked int private_stack_unwinds(void)\n+{\n+\tasm volatile (\n+\t\t\"*(u64 *)(r10 - 200) = 42;\"\n+\t\t\"r1 = %[one] ll;\"\n+\t\t\"r1 = *(u64 *)(r1 + 0);\"\n+\t\t\"call may_unwind_big_stack;\"\n+\t\tunwind_goto(1)\n+\t\t\"exit;\"\n+\t\t\"r1 = *(u64 *)(r10 - 200);\"\n+\t\t\"r0 += r1;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_addr(one),\n+\t\t  __imm_unwind_goto(1)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+SEC(\"raw_tp\")\n+__description(\"unwind: private stack, the callee returns\")\n+__success __retval(49)\n+__load_if_JITed()\n+__arch_x86_64\n+__arch_arm64\n+__naked int private_stack_returns(void)\n+{\n+\tasm volatile (\n+\t\t\"*(u64 *)(r10 - 200) = 42;\"\n+\t\t\"r1 = %[zero] ll;\"\n+\t\t\"r1 = *(u64 *)(r1 + 0);\"\n+\t\t\"call may_unwind_big_stack;\"\n+\t\tunwind_goto(3)\n+\t\t\"r1 = *(u64 *)(r10 - 200);\"\n+\t\t\"r0 += r1;\"\n+\t\t\"exit;\"\n+\t\t\"r0 = 0;\"\n+\t\t\"exit;\"\n+\t\t:\n+\t\t: __imm_addr(zero),\n+\t\t  __imm_unwind_goto(3)\n+\t\t: __clobber_all\n+\t);\n+}\n+\n+/*\n+ * BTF FUNC records are not generated for kfuncs referenced\n+ * from inline assembly. These records are necessary for\n+ * libbpf to link the program. The function below is a hack\n+ * to ensure that BTF FUNC records are generated.\n+ */\n+void __kfunc_btf_root(void)\n+{\n+\tbpf_rcu_read_lock();\n+\tbpf_rcu_read_unlock();\n+}\n+\n+char _license[] SEC(\"license\") = \"GPL\";\ndiff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c\nindex 18d47e9254007..824e5912ebcb8 100644\n--- a/tools/testing/selftests/bpf/test_loader.c\n+++ b/tools/testing/selftests/bpf/test_loader.c\n@@ -1546,7 +1546,8 @@ void run_subtest(struct test_loader *tester,\n \t\t}\n \n \t\terr = do_prog_test_run(bpf_program__fd(tprog), \u0026retval,\n-\t\t\t\t       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ? true : false,\n+\t\t\t\t       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ||\n+\t\t\t\t       bpf_program__type(tprog) == BPF_PROG_TYPE_RAW_TRACEPOINT,\n \t\t\t\t       spec-\u003elinear_sz);\n \t\tif (!err \u0026\u0026 retval != subspec-\u003eretval \u0026\u0026 subspec-\u003eretval != POINTER_VALUE) {\n \t\t\tPRINT_FAIL(\"Unexpected retval: %d != %d\\n\", retval, subspec-\u003eretval);\n"
}

2/1 2026/10/20 09:15 agent patch-evaluator
3m Model:
core

Args:
null

Results:
{
  "EnableConfigs": [
    "BPF_SYSCALL",
    "BPF_JIT"
  ],
  "FocusSymbols": [
    "bpf_unwind_target",
    "check_unwind_insns",
    "add_unwind_exentry",
    "emit_unwind_ret_addr",
    "bpf_jit_supports_unwind",
    "bpf_check_attach_target"
  ],
  "Reasoning": "The patch introduces support for BPF unwinding (BPF_UNWIND_GOTO and BPF_UNWIND_EXIT) in the BPF core verifier and the x86 JIT compiler. It adds new verification logic for unwind instructions, generates JIT code and exception table entries for unwind paths, and introduces a runtime unwind resolution function (bpf_unwind_target) containing runtime assertions (WARN_ON_ONCE). This code is fully reachable in standard QEMU amd64 environments via the bpf() syscall (e.g. BPF_PROG_LOAD and BPF_PROG_TEST_RUN), making it prime for fuzzing.",
  "WorthFuzzing": true
}

Instruction:
You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.

IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.

================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.

- REACHABILITY IS THE PRIMARY GATE:
  Fuzzing can only discover bugs in code that can actually execute in standard virtualized
  environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
  If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
  regardless of whether it adds assertions or complex logic.

- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
  Patch authors routinely label changes as "cleanups", "refactorings", or state
  "No functional change intended". Do NOT take these claims at face value.
  Code refactorings that rearrange logic, introduce helper functions, or alter state management
  in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
  If reachable executable code is modified or refactored, it MUST be fuzzed.

- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
  When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
  BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
  Even if the author believes the invariant always holds, fuzzing is essential to verify whether
  an unusual sequence of operations can violate it.

================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:

- Non-kernel and non-executable changes:
  * Modifications to Documentation/, comments, or spelling fixes.
  * User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
    that do not affect the compiled kernel image (vmlinux) or kernel modules.
  * Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
    that do not alter control flow or data structures.
  * Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
  * Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
    ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
  * Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
    physical PCI hot-unplug or manual sysfs driver unbinding.
  * Code paths exclusive to architectures other than the target architecture.

================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
  * Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
    BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
  * Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
  * Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
  * Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).

================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:

- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
  Do NOT list generic, ubiquitous functions called by almost every program in the corpus
  (including, but not limited to: general memory allocators and deallocators, page fault
  and trap handlers, or core synchronization primitives; this is not an exhaustive list).
  Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
  which severely dilutes fuzzing effort away from the actual changes.

- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
  List functions that specifically implement the logic being added or altered, or direct API entrypoints
  for the subsystem feature under review.

- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
  Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
  distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
  If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
  functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).

================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.


Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.

Prompt:
Target architecture: amd64

For your convenience, here is the diff of the changes:
commit 880657c9fd431b4343c8c6dcc4398fc89e69a930
Author: syz-cluster <triage@syzkaller.com>
Date:   Tue Oct 20 09:14:59 2026 +0000

    syz-cluster: applied patch under review

diff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h
index b2fe6e6dcf447..50029dd9baa63 100644
--- a/arch/arm64/net/bpf_jit.h
+++ b/arch/arm64/net/bpf_jit.h
@@ -316,6 +316,8 @@
 
 #define A64_PACIASP A64_HINT(AARCH64_INSN_HINT_PACIASP)
 #define A64_AUTIASP A64_HINT(AARCH64_INSN_HINT_AUTIASP)
+#define A64_PACIA1716 A64_HINT(AARCH64_INSN_HINT_PACIA_1716)
+#define A64_AUTIA1716 A64_HINT(AARCH64_INSN_HINT_AUTIA_1716)
 
 /* BTI */
 #define A64_BTI_C  A64_HINT(AARCH64_INSN_HINT_BTIC)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 1d5f4cf88d331..45ed69b83265d 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -11,6 +11,7 @@
 #include <linux/bitfield.h>
 #include <linux/bpf.h>
 #include <linux/cfi.h>
+#include <linux/extable.h>
 #include <linux/filter.h>
 #include <linux/memory.h>
 #include <linux/printk.h>
@@ -1221,6 +1222,85 @@ static int add_exception_handler(const struct bpf_insn *insn,
 	return 0;
 }
 
+/*
+ * There is no branch for unwind_goto, only an extable entry
+ *   { .insn = ret_addr - 1, .fixup = pad - ret_addr }
+ * where ret_addr is the return address of the call in front of it. unwind_exit
+ * replaces LR in its frame record with bpf_unwind_target() of it.
+ * ret_addr - 1, because what follows the call can have an entry of its own.
+ * No insn starts there, so a fault cannot hit the entry.
+ * 'bl foo; mov x8, x0' is in front of unwind_goto. The pad doesn't miss the mov:
+ * r0 is x8 and the epilogue of the callee copies it to x0 and leaves x8 as it is.
+ */
+static int add_unwind_exentry(struct jit_ctx *ctx, int pad)
+{
+	int ret_addr = ctx->idx - 1;
+	struct exception_table_entry *ex;
+	off_t ins_offset;
+
+	if (!ctx->image)
+		/* First pass */
+		return 0;
+
+	if (!ctx->prog->aux->extable ||
+	    WARN_ON_ONCE(ctx->exentry_idx >= ctx->prog->aux->num_exentries))
+		return -EINVAL;
+
+	ex = &ctx->prog->aux->extable[ctx->exentry_idx];
+	ins_offset = (long)&ctx->ro_image[ret_addr] - 1 - (long)&ex->insn;
+	if (WARN_ON_ONCE(ins_offset >= 0 || ins_offset < INT_MIN))
+		return -ERANGE;
+
+	/* switch ex to rw buffer for writing */
+	ex = (void *)ctx->image + ((void *)ex - (void *)ctx->ro_image);
+	ex->insn = ins_offset;
+	ex->fixup = (ctx->offset[pad] - ret_addr) * AARCH64_INSN_SIZE;
+	ex->type = EX_TYPE_BPF;
+
+	ctx->exentry_idx++;
+	return 0;
+}
+
+static u64 bpf_unwind_target(u64 ret_addr)
+{
+	const struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);
+
+	/* The verifier made sure that unwind_goto follows the call */
+	if (WARN_ON_ONCE(!e))
+		return ret_addr;
+	return ret_addr + e->fixup;
+}
+
+/*
+ * unwind_exit: fix LR in the frame record. It's signed with SP at the entry of
+ * the prog, which is FP + 16. The x16/x17 forms are hints like PACIASP: nops
+ * without pointer auth. r0 and r2 are returned.
+ */
+static void emit_unwind_ret_addr(struct jit_ctx *ctx)
+{
+	const bool ptr_auth = IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL);
+	const u8 r0 = bpf2a64[BPF_REG_0];
+	const u8 r2 = bpf2a64[BPF_REG_2];
+
+	emit(A64_PUSH(r0, r2, A64_SP), ctx);
+
+	emit(A64_LDR64I(A64_R(17), A64_FP, 8), ctx);
+	if (ptr_auth) {
+		emit(A64_ADD_I(1, A64_R(16), A64_FP, 16), ctx);
+		emit(A64_AUTIA1716, ctx);
+	}
+	emit(A64_MOV(1, A64_R(0), A64_R(17)), ctx);
+	emit_call((u64)bpf_unwind_target, ctx);
+	emit(A64_MOV(1, A64_R(17), A64_R(0)), ctx);
+	if (ptr_auth) {
+		emit(A64_ADD_I(1, A64_R(16), A64_FP, 16), ctx);
+		emit(A64_PACIA1716, ctx);
+	}
+	emit(A64_STR64I(A64_R(17), A64_FP, 8), ctx);
+
+	emit(A64_POP(r0, r2, A64_SP), ctx);
+}
+
 static const struct bpf_jit_arg_abi arm64_arg_abi = {
 	.nr_arg_regs		= 8,
 	.even_reg_align		= true,
@@ -1806,8 +1886,16 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 		if (emit_bpf_tail_call(ctx))
 			return -EFAULT;
 		break;
+	/* unwind_goto. The verifier expanded may_goto. */
+	case BPF_JMP | BPF_JCOND:
+		ret = add_unwind_exentry(ctx, i + 1 + off);
+		if (ret)
+			return ret;
+		break;
 	/* function return */
 	case BPF_JMP | BPF_EXIT:
+		if (bpf_is_unwind_exit(insn) && bpf_prog_can_unwind(ctx->prog))
+			emit_unwind_ret_addr(ctx);
 		/* Optimization: when last instruction is EXIT,
 		   simply fallthrough to epilogue. */
 		if (i == ctx->prog->len - 1)
@@ -2466,6 +2554,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+bool bpf_jit_supports_unwind(void)
+{
+	return true;
+}
+
 const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
 {
 	return &arm64_arg_abi;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72a2a0a89e033..c583eed4e9549 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -15,6 +15,7 @@
 #include <linux/memory.h>
 #include <linux/sort.h>
 #include <linux/execmem.h>
+#include <linux/extable.h>
 #include <asm/extable.h>
 #include <asm/ftrace.h>
 #include <asm/set_memory.h>
@@ -2511,6 +2512,67 @@ static void adjust_cmp_imm(u32 *op, s32 *imm)
 	}
 }
 
+/*
+ * There is no branch for unwind_goto, only an extable entry
+ *   { .insn = ret_addr - 1, .fixup = target - ret_addr }
+ * where ret_addr is the return address of the call in front of it. unwind_exit
+ * replaces the return address of its frame with bpf_unwind_target() of it.
+ * ret_addr - 1, because what follows the call can have an entry of its own.
+ * No insn starts there, so a fault cannot hit the entry.
+ */
+static int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,
+			      int ret_addr, int target)
+{
+	struct exception_table_entry *ex;
+	s64 delta;
+
+	if (!prog->aux->extable)
+		return 0;
+	if (*excnt >= prog->aux->num_exentries) {
+		pr_err("unwind extable bug\n");
+		return -EFAULT;
+	}
+	ex = &prog->aux->extable[(*excnt)++];
+	delta = image + ret_addr - 1 - (u8 *)&ex->insn;
+	if (!is_simm32(delta)) {
+		pr_err("extable->insn doesn't fit into 32-bit\n");
+		return -EFAULT;
+	}
+	/* switch ex to rw buffer for writes */
+	ex = (void *)rw_image + ((void *)ex - (void *)image);
+	ex->insn = delta;
+	ex->fixup = target - ret_addr;
+	ex->data = EX_TYPE_BPF;
+	return 0;
+}
+
+static u64 bpf_unwind_target(u64 ret_addr)
+{
+	const struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);
+
+	/* The verifier made sure that unwind_goto follows the call */
+	if (WARN_ON_ONCE(!e))
+		return ret_addr;
+	return ret_addr + e->fixup;
+}
+
+/* unwind_exit: fix the return address of this frame. r0 and r2 are returned. */
+static int emit_unwind_ret_addr(u8 **pprog, u8 *ip)
+{
+	u8 *prog = *pprog, *start = prog;
+
+	EMIT2(0x50, 0x56);			/* push rax; push rsi */
+	EMIT4(0x48, 0x8B, 0x7D, 0x08);		/* mov rdi, [rbp + 8] */
+	ip += prog - start;
+	ip += x86_call_depth_emit_accounting(&prog, bpf_unwind_target, ip);
+	if (emit_call(&prog, bpf_unwind_target, ip))
+		return -EINVAL;
+	EMIT4(0x48, 0x89, 0x45, 0x08);		/* mov [rbp + 8], rax */
+	EMIT2(0x5E, 0x58);			/* pop rsi; pop rax */
+	*pprog = prog;
+	return 0;
+}
+
 static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
 		  u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
 {
@@ -3493,6 +3555,34 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 				pop_r9(&prog);
 			break;
 
+			/* unwind_goto. The verifier expanded may_goto. */
+		case BPF_JMP | BPF_JCOND:
+			if (!priv_frame_ptr) {
+				err = add_unwind_exentry(bpf_prog, image, rw_image, &excnt,
+							 addrs[i - 1], addrs[i + insn->off]);
+				if (err)
+					return err;
+				break;
+			}
+			/*
+			 * 'pop r9' after the call has to be done on the way to
+			 * the pad too:
+			 *   call foo
+			 *   pop r9      <- exit returns here
+			 *   jmp 1f
+			 *   pop r9      <- unwind_exit returns here
+			 *   jmp pad
+			 * 1:
+			 */
+			err = add_unwind_exentry(bpf_prog, image, rw_image, &excnt,
+						 addrs[i - 1] - 2, addrs[i - 1] + 2);
+			if (err)
+				return err;
+			EMIT2(0xEB, 7);
+			pop_r9(&prog);
+			EMIT1_off32(0xE9, addrs[i + insn->off] - (addrs[i - 1] + 9));
+			break;
+
 		case BPF_JMP | BPF_TAIL_CALL:
 			if (imm32)
 				emit_bpf_tail_call_direct(bpf_prog,
@@ -3774,13 +3864,16 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 			break;
 
 		case BPF_JMP | BPF_EXIT:
+			if (src_reg == BPF_UNWIND_EXIT && bpf_prog_can_unwind(bpf_prog) &&
+			    emit_unwind_ret_addr(&prog, ip))
+				return -EINVAL;
 			if (seen_exit) {
 				jmp_offset = ctx->cleanup_addr - addrs[i];
 				goto emit_jmp;
 			}
 			seen_exit = true;
 			/* Update cleanup_addr */
-			ctx->cleanup_addr = proglen;
+			ctx->cleanup_addr = proglen + (prog - temp);
 			if (bpf_prog_was_classic(bpf_prog) &&
 			    !ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
 				if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
@@ -5027,6 +5120,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+bool bpf_jit_supports_unwind(void)
+{
+	return true;
+}
+
 const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
 {
 	return &x86_arg_abi;
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 7750df9734942..580f89c450b55 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1897,6 +1897,7 @@ struct bpf_prog_aux {
 	bool changes_pkt_data;
 	bool has_prologue;
 	bool might_sleep;
+	bool can_unwind;
 	bool kprobe_write_ctx;
 	/* has insns that work with the arena base, see bpf_prog_has_arena_insns() */
 	bool uses_arena_insns;
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 54481da4ed2d2..6606b341351eb 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -532,6 +532,8 @@ struct bpf_verifier_state {
 
 	bool speculative;
 	bool in_sleepable;
+	/* The callee left with unwind_exit. Next insn is unwind_goto. */
+	bool unwound;
 
 	/*
 	 * First and last insn idx of this verifier state.
@@ -957,6 +959,7 @@ struct bpf_subprog_info {
 	s16 fastcall_stack_off;
 	bool has_tail_call: 1;
 	bool might_throw: 1;
+	bool can_unwind: 1; /* has unwind_exit */
 	bool tail_call_reachable: 1;
 	bool has_ld_abs: 1;
 	bool is_cb: 1;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 6f4c9bd024a0b..fccd8db336470 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -254,6 +254,25 @@ static inline bool insn_is_mov_percpu_addr(const struct bpf_insn *insn)
 		.off   = 0,					\
 		.imm   = 1 })
 
+static inline bool bpf_is_unwind_goto(const struct bpf_insn *insn)
+{
+	return insn->code == (BPF_JMP | BPF_JCOND) && insn->src_reg == BPF_UNWIND_GOTO;
+}
+
+static inline bool bpf_is_unwind_exit(const struct bpf_insn *insn)
+{
+	return insn->code == (BPF_JMP | BPF_EXIT) && insn->src_reg == BPF_UNWIND_EXIT;
+}
+
+/*
+ * The caller of the main prog doesn't tell unwind_exit from exit. An extension
+ * is different: its caller is the caller of the subprog that it replaced.
+ */
+static inline bool bpf_prog_can_unwind(const struct bpf_prog *prog)
+{
+	return bpf_is_subprog(prog) || prog->type == BPF_PROG_TYPE_EXT;
+}
+
 static inline bool insn_is_zext(const struct bpf_insn *insn)
 {
 	return insn->code == (BPF_ALU | BPF_MOV | BPF_X) && insn->imm == 1;
@@ -1257,6 +1276,7 @@ bool bpf_jit_supports_percpu_insn(void);
 bool bpf_jit_supports_kfunc_call(void);
 bool bpf_jit_supports_callx(void);
 bool bpf_jit_supports_kfunc_ret_reg_pair(void);
+bool bpf_jit_supports_unwind(void);
 bool bpf_jit_supports_stack_args(void);
 bool bpf_jit_supports_arena_args(void);
 bool bpf_jit_supports_far_kfunc_call(void);
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 6f70db7515dd4..6efe0ca036248 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -56,8 +56,19 @@
 
 enum bpf_cond_pseudo_jmp {
 	BPF_MAY_GOTO = 0,
+	/*
+	 * Follows a call of a bpf subprog. Taken when the subprog left with
+	 * BPF_EXIT that has src_reg == BPF_UNWIND_EXIT.
+	 */
+	BPF_UNWIND_GOTO = 1,
 };
 
+/*
+ * When bpf_exit->src_reg == BPF_UNWIND_EXIT, it's the same as bpf_exit, except
+ * that BPF_UNWIND_GOTO after the call is taken.
+ */
+#define BPF_UNWIND_EXIT	1
+
 /* Register numbers */
 enum {
 	BPF_REG_0 = 0,
diff --git a/kernel/bpf/const_fold.c b/kernel/bpf/const_fold.c
index 8dd8c5958eb85..f9edfde965dc7 100644
--- a/kernel/bpf/const_fold.c
+++ b/kernel/bpf/const_fold.c
@@ -357,7 +357,7 @@ int bpf_prune_dead_branches(struct bpf_verifier_env *env)
 
 		if (!bpf_insn_is_cond_jump(insn->code))
 			continue;
-		if (bpf_is_may_goto_insn(insn))
+		if (BPF_OP(insn->code) == BPF_JCOND)
 			continue;
 
 		if (!(aux->const_reg_mask & BIT(insn->dst_reg)))
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fc22d66d194a9..3030beb531d99 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3399,6 +3399,12 @@ bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return false;
 }
 
+/* unwind_goto and unwind_exit */
+bool __weak bpf_jit_supports_unwind(void)
+{
+	return false;
+}
+
 /*
  * How this arch places a by-value kfunc argument, or NULL for one that has
  * not opted in and so only takes an argument of a single eightbyte, which
diff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c
index 36d3228d77454..889fc074496f1 100644
--- a/kernel/bpf/disasm.c
+++ b/kernel/bpf/disasm.c
@@ -375,11 +375,16 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 			   insn->src_reg == BPF_MAY_GOTO) {
 			verbose(cbs->private_data, "(%02x) may_goto pc%+d",
 				insn->code, insn->off);
+		} else if (insn->code == (BPF_JMP | BPF_JCOND) &&
+			   insn->src_reg == BPF_UNWIND_GOTO) {
+			verbose(cbs->private_data, "(%02x) unwind_goto pc%+d",
+				insn->code, insn->off);
 		} else if (insn->code == (BPF_JMP32 | BPF_JA)) {
 			verbose(cbs->private_data, "(%02x) gotol pc%+d",
 				insn->code, insn->imm);
 		} else if (insn->code == (BPF_JMP | BPF_EXIT)) {
-			verbose(cbs->private_data, "(%02x) exit", insn->code);
+			verbose(cbs->private_data, "(%02x) %s", insn->code,
+				insn->src_reg == BPF_UNWIND_EXIT ? "unwind_exit" : "exit");
 		} else if (BPF_SRC(insn->code) == BPF_X) {
 			verbose(cbs->private_data,
 				"(%02x) if %c%d %s %c%d goto pc%+d",
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 409b9104c723e..9ae3a81604483 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -666,6 +666,10 @@ void bpf_opt_hard_wire_dead_code_branches(struct bpf_verifier_env *env)
 		if (!bpf_insn_is_cond_jump(insn->code))
 			continue;
 
+		/* unwind_exit of a callee that never exits needs it too */
+		if (bpf_is_unwind_goto(insn) && aux_data[i + 1 + insn->off].seen)
+			continue;
+
 		if (!aux_data[i + 1].seen)
 			ja.off = insn->off;
 		else if (!aux_data[i + 1 + insn->off].seen)
@@ -1467,12 +1471,15 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 			if (BPF_CLASS(insn->code) == BPF_STX &&
 			     BPF_MODE(insn->code) == BPF_PROBE_ATOMIC)
 				num_exentries++;
+			if (bpf_is_unwind_goto(insn))
+				num_exentries++;
 		}
 		func[i]->aux->num_exentries = num_exentries;
 		func[i]->aux->tail_call_reachable = env->subprog_info[i].tail_call_reachable;
 		func[i]->aux->exception_cb = env->subprog_info[i].is_exception_cb;
 		func[i]->aux->changes_pkt_data = env->subprog_info[i].changes_pkt_data;
 		func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
+		func[i]->aux->can_unwind = env->subprog_info[i].can_unwind;
 		func[i]->aux->token = prog->aux->token;
 		if (!i)
 			func[i]->aux->exception_boundary = env->seen_exception;
diff --git a/kernel/bpf/states.c b/kernel/bpf/states.c
index b3d250caaaa65..d7df6632a5c47 100644
--- a/kernel/bpf/states.c
+++ b/kernel/bpf/states.c
@@ -1046,6 +1046,9 @@ static bool states_equal(struct bpf_verifier_env *env,
 	if (old->in_sleepable != cur->in_sleepable)
 		return false;
 
+	if (old->unwound != cur->unwound)
+		return false;
+
 	if (!refsafe(old, cur, &env->idmap_scratch))
 		return false;
 
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 849bc9136299d..90bc7230afa8a 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -5087,6 +5087,12 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,
 			insns[i].imm = BPF_FUNC_tail_call;
 			/* fall-through */
 		}
+		/*
+		 * BPF_JCOND is the same opcode as BPF_CALL_ARGS. The latter is for
+		 * the interpreter, and a prog with unwind_goto is always JITed.
+		 */
+		if (prog->jited && bpf_is_unwind_goto(&insns[i]))
+			continue;
 		if (code == (BPF_JMP | BPF_CALL) ||
 		    code == (BPF_JMP | BPF_CALL_ARGS)) {
 			/* Restore the legacy xlated dump layout. */
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 01434e1d23b59..6c56675c3298d 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -1769,6 +1769,7 @@ int bpf_copy_verifier_state(struct bpf_verifier_state *dst_state,
 		return err;
 	dst_state->speculative = src->speculative;
 	dst_state->in_sleepable = src->in_sleepable;
+	dst_state->unwound = src->unwound;
 	dst_state->curframe = src->curframe;
 	dst_state->branches = src->branches;
 	dst_state->parent = src->parent;
@@ -3430,6 +3431,46 @@ static int next_callx_callee(struct bpf_verifier_env *env, int caller, int from)
 	return bit < end ? bit - caller * cnt : -1;
 }
 
+/*
+ * 'call foo; unwind_goto pad' goes to the pad when foo left with unwind_exit.
+ * JITs don't emit a branch for it. unwind_exit finds the pad by its return
+ * address. So unwind_goto has to follow the call and nothing may jump to it.
+ */
+static int check_unwind_insns(struct bpf_verifier_env *env)
+{
+	struct bpf_insn *insn = env->prog->insnsi;
+	bool seen = false;
+	int i;
+
+	for (i = 0; i < env->prog->len; i++, insn++) {
+		if (bpf_is_unwind_exit(insn)) {
+			bpf_find_containing_subprog(env, i)->can_unwind = true;
+			seen = true;
+		}
+		if (!bpf_is_unwind_goto(insn))
+			continue;
+		seen = true;
+		if (!i || !(bpf_pseudo_call(insn - 1) || bpf_is_callx(insn - 1))) {
+			verbose(env, "unwind_goto at insn %d doesn't follow a call of a subprog\n",
+				i);
+			return -EINVAL;
+		}
+		if (bpf_is_jump_target(env, i)) {
+			verbose(env, "jump to unwind_goto at insn %d\n", i);
+			return -EINVAL;
+		}
+	}
+	if (!seen)
+		return 0;
+	if (!bpf_jit_supports_unwind()) {
+		verbose(env, "JIT does not support unwinding\n");
+		return -ENOTSUPP;
+	}
+	env->prog->jit_required = 1;
+	env->prog->aux->can_unwind = env->subprog_info[0].can_unwind;
+	return 0;
+}
+
 /*
  * Sort subprogs in topological order so that leaf subprogs come first and
  * their callers come later. This is a DFS post-order traversal of the call
@@ -11584,6 +11625,19 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			}
 		}
 
+		if (env->subprog_info[subprog].can_unwind) {
+			struct bpf_verifier_state *branch;
+
+			if (!bpf_is_unwind_goto(insn + 1)) {
+				verbose(env, "no unwind_goto after the call of %s()\n", sub_name);
+				return -EINVAL;
+			}
+			branch = push_stack(env, *insn_idx + 1, *insn_idx, false);
+			if (IS_ERR(branch))
+				return PTR_ERR(branch);
+			branch->unwound = true;
+		}
+
 		if (env->subprog_info[subprog].might_throw) {
 			struct bpf_verifier_state *branch;
 
@@ -12057,6 +12111,19 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
 	}
 
 	caller = state->frame[state->curframe - 1];
+	if (bpf_is_unwind_exit(&env->prog->insnsi[*insn_idx])) {
+		/* the helper or the kfunc that called it doesn't expect that */
+		if (callee->in_callback_fn) {
+			verbose(env, "callback cannot unwind\n");
+			return -EINVAL;
+		}
+		if (!bpf_is_unwind_goto(&env->prog->insnsi[callee->callsite + 1])) {
+			verbose(env, "unwind_exit, but no unwind_goto after the call at insn %d\n",
+				callee->callsite);
+			return -EINVAL;
+		}
+		state->unwound = true;
+	}
 	if (callee->in_callback_fn) {
 		if (r0->type != SCALAR_VALUE) {
 			verbose(env, "R0 not a scalar value\n");
@@ -18759,6 +18826,23 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 		return -EINVAL;
 	}
 
+	if (bpf_is_unwind_goto(insn)) {
+		if (!env->cur_state->unwound)
+			return 0;
+		env->cur_state->unwound = false;
+		/*
+		 * unwind_exit changed its return address, so the return is
+		 * mispredicted and what follows unwind_goto runs speculatively.
+		 */
+		if (!env->bypass_spec_v1) {
+			err = sanitize_speculative_path(env, NULL, *insn_idx + 1, *insn_idx);
+			if (err)
+				return err;
+		}
+		*insn_idx += insn->off;
+		return 0;
+	}
+
 	if (opcode == BPF_JCOND) {
 		struct bpf_verifier_state *cur_st = env->cur_state, *queued_st, *prev_st;
 		int idx = *insn_idx;
@@ -19986,6 +20070,12 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,
 	 * Exit from async callback implies return code restrictions
 	 * that depend on async scheduling mechanism.
 	 */
+	if ((cur_frame->in_async_callback_fn || cur_frame->in_exception_callback_fn) &&
+	    bpf_is_unwind_exit(&env->prog->insnsi[env->insn_idx])) {
+		verbose(env, "callback cannot unwind\n");
+		return -EINVAL;
+	}
+
 	if (cur_frame->subprogno &&
 	    !cur_frame->in_async_callback_fn &&
 	    !cur_frame->in_exception_callback_fn)
@@ -21223,14 +21313,14 @@ static int check_jmp_fields(struct bpf_verifier_env *env, struct bpf_insn *insn)
 		return 0;
 	case BPF_EXIT:
 		if (BPF_SRC(insn->code) != BPF_K || insn->imm != 0 ||
-		    insn->src_reg != BPF_REG_0 || insn->dst_reg != BPF_REG_0 ||
+		    insn->src_reg > BPF_UNWIND_EXIT || insn->dst_reg != BPF_REG_0 ||
 		    class == BPF_JMP32) {
 			verbose(env, "BPF_EXIT uses reserved fields\n");
 			return -EINVAL;
 		}
 		return 0;
 	case BPF_JCOND:
-		if (insn->code != (BPF_JMP | BPF_JCOND) || insn->src_reg != BPF_MAY_GOTO ||
+		if (insn->code != (BPF_JMP | BPF_JCOND) || insn->src_reg > BPF_UNWIND_GOTO ||
 		    insn->dst_reg || insn->imm) {
 			verbose(env, "invalid may_goto imm %d\n", insn->imm);
 			return -EINVAL;
@@ -22436,6 +22526,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 		struct bpf_prog_aux *aux = tgt_prog->aux;
 		bool tgt_changes_pkt_data;
 		bool tgt_might_sleep;
+		bool tgt_can_unwind;
 
 		if (bpf_prog_is_dev_bound(prog->aux) &&
 		    !bpf_prog_dev_bound_match(prog, tgt_prog)) {
@@ -22464,6 +22555,16 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 				prog_extension ? "Extension" : "Tracing");
 			return -EINVAL;
 		}
+		/* unwind_exit of the main prog is exit, unless it's an extension */
+		tgt_can_unwind = (subprog || tgt_prog->type == BPF_PROG_TYPE_EXT) &&
+				 (aux->func ? aux->func[subprog]->aux->can_unwind
+					    : aux->can_unwind);
+		/* unwind_exit would return to the trampoline that called the target */
+		if (tgt_can_unwind && !prog_extension &&
+		    prog->expected_attach_type != BPF_TRACE_FENTRY) {
+			bpf_log(log, "Only fentry can attach to %s, which can unwind\n", tname);
+			return -EINVAL;
+		}
 		conservative = aux->func_info_aux[subprog].unreliable;
 		if (prog_extension) {
 			if (aux->func_info_aux[subprog].linkage != BTF_FUNC_GLOBAL ||
@@ -22511,6 +22612,12 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 					"Extension program may sleep, while original does not\n");
 				return -EINVAL;
 			}
+			/* There is no unwind_goto after the calls of the original */
+			if (prog->aux->can_unwind && !tgt_can_unwind) {
+				bpf_log(log,
+					"Extension program can unwind, while original does not\n");
+				return -EINVAL;
+			}
 		}
 		if (!tgt_prog->jited) {
 			bpf_log(log, "Can attach to only JITed progs\n");
@@ -23845,6 +23952,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
 	if (ret < 0)
 		goto skip_full_check;
 
+	ret = check_unwind_insns(env);
+	if (ret < 0)
+		goto skip_full_check;
+
 	ret = bpf_compute_postorder(env);
 	if (ret < 0)
 		goto skip_full_check;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 6f70db7515dd4..6efe0ca036248 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -56,8 +56,19 @@
 
 enum bpf_cond_pseudo_jmp {
 	BPF_MAY_GOTO = 0,
+	/*
+	 * Follows a call of a bpf subprog. Taken when the subprog left with
+	 * BPF_EXIT that has src_reg == BPF_UNWIND_EXIT.
+	 */
+	BPF_UNWIND_GOTO = 1,
 };
 
+/*
+ * When bpf_exit->src_reg == BPF_UNWIND_EXIT, it's the same as bpf_exit, except
+ * that BPF_UNWIND_GOTO after the call is taken.
+ */
+#define BPF_UNWIND_EXIT	1
+
 /* Register numbers */
 enum {
 	BPF_REG_0 = 0,
diff --git a/tools/testing/selftests/bpf/Makefile.skel b/tools/testing/selftests/bpf/Makefile.skel
index 06e297a17156c..6570c48f92b01 100644
--- a/tools/testing/selftests/bpf/Makefile.skel
+++ b/tools/testing/selftests/bpf/Makefile.skel
@@ -153,4 +153,15 @@ $(RUST_OBJS): $(RDIR)/%.bpf.o: progs/%.rs $(RUST_CORE) | $(RDIR)
 endif
 endif
 
+# Programs in C++. No skeletons: gcc doesn't build them.
+ifeq ($(BPF_CC),$(CLANG))
+CXX_OBJS := $(patsubst progs/%.cpp,$(RDIR)/%.bpf.o,$(wildcard progs/*.cpp))
+BPF_OBJS += $(CXX_OBJS)
+
+$(CXX_OBJS): $(RDIR)/%.bpf.o: progs/%.cpp | $(RDIR)
+	$(call msg,$(BPF_CC_MSG),$(BINARY),$@)
+	$(Q)$(BPF_CC) -x c++ -g -Wall -Werror -fexceptions -funwind-tables -fno-rtti	\
+		$(BPF_CC_FLAGS) -c $< -o $@ $(call skip_on_fail,BPF)
+endif
+
 endif # BPF_CC
diff --git a/tools/testing/selftests/bpf/prog_tests/unwind_attach.c b/tools/testing/selftests/bpf/prog_tests/unwind_attach.c
new file mode 100644
index 0000000000000..92eab5b77f62a
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/unwind_attach.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include "verifier_unwind.skel.h"
+#include "unwind_attach.skel.h"
+
+#ifndef ENOTSUPP
+#define ENOTSUPP 524
+#endif
+
+static char log_buf[16 * 1024];
+
+static int run(int fd)
+{
+	char data[64] = {};
+	LIBBPF_OPTS(bpf_test_run_opts, opts,
+		.data_in = data,
+		.data_size_in = sizeof(data),
+		.repeat = 1,
+	);
+
+	ASSERT_OK(bpf_prog_test_run_opts(fd, &opts), "test_run");
+	return opts.retval;
+}
+
+/*
+ * msg: why it cannot be loaded. Otherwise retval is what the target returns
+ * with the prog attached.
+ */
+static void attach(int tgt_fd, const char *func, const char *name, const char *msg, int retval)
+{
+	struct bpf_link *link = NULL;
+	struct unwind_attach *skel;
+	struct bpf_program *prog;
+	int err;
+
+	skel = unwind_attach__open();
+	if (!ASSERT_OK_PTR(skel, "unwind_attach__open"))
+		return;
+
+	prog = bpf_object__find_program_by_name(skel->obj, name);
+	if (!ASSERT_OK_PTR(prog, name))
+		goto out;
+	bpf_program__set_autoload(prog, true);
+	log_buf[0] = 0;
+	bpf_program__set_log_buf(prog, log_buf, sizeof(log_buf));
+	err = bpf_program__set_attach_target(prog, tgt_fd, func);
+	if (!ASSERT_OK(err, "set_attach_target"))
+		goto out;
+
+	err = unwind_attach__load(skel);
+	if (msg) {
+		ASSERT_ERR(err, name);
+		ASSERT_HAS_SUBSTR(log_buf, msg, name);
+		goto out;
+	}
+	if (!ASSERT_OK(err, name))
+		goto out;
+
+	link = bpf_program__attach(prog);
+	if (!ASSERT_OK_PTR(link, "attach"))
+		goto out;
+	ASSERT_EQ(run(tgt_fd), retval, "retval");
+	if (prog == skel->progs.fentry_unwind)
+		ASSERT_EQ(skel->bss->fentry_cnt, 1, "fentry_cnt");
+out:
+	bpf_link__destroy(link);
+	unwind_attach__destroy(skel);
+}
+
+void test_unwind_attach(void)
+{
+	struct verifier_unwind *tgt;
+	struct bpf_program *prog;
+	int tgt_fd, plain_fd, err;
+
+	tgt = verifier_unwind__open();
+	if (!ASSERT_OK_PTR(tgt, "verifier_unwind__open"))
+		return;
+	bpf_object__for_each_program(prog, tgt->obj)
+		bpf_program__set_autoload(prog, prog == tgt->progs.calls_global_unwinds ||
+						prog == tgt->progs.calls_global_never_unwinds);
+	err = verifier_unwind__load(tgt);
+	if (err == -ENOTSUPP) {
+		printf("%s:SKIP:JIT does not support unwinding\n", __func__);
+		test__skip();
+		goto out;
+	}
+	if (!ASSERT_OK(err, "verifier_unwind__load"))
+		goto out;
+	/*
+	 * It calls global_unwinds(), which unwinds with r0 = 5, and returns r0 + 10
+	 * from the pad.
+	 */
+	tgt_fd = bpf_program__fd(tgt->progs.calls_global_unwinds);
+	plain_fd = bpf_program__fd(tgt->progs.calls_global_never_unwinds);
+	ASSERT_EQ(run(tgt_fd), 15, "retval");
+
+	attach(tgt_fd, "global_unwinds", "fentry_unwind", NULL, 15);
+	/* unwind_exit would return to the trampoline */
+	attach(tgt_fd, "global_unwinds", "fexit_unwind",
+	       "Only fentry can attach to global_unwinds", 0);
+	/* An extension leaves either way, like what it replaced */
+	attach(tgt_fd, "global_unwinds", "freplace_returns", NULL, 9);
+	attach(tgt_fd, "global_unwinds", "freplace_unwinds", NULL, 13);
+	/* There is no unwind_goto after the call */
+	attach(plain_fd, "global_never_unwinds", "freplace_unwinds",
+	       "Extension program can unwind, while original does not", 0);
+	ASSERT_EQ(run(tgt_fd), 15, "retval");
+out:
+	verifier_unwind__destroy(tgt);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c b/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c
new file mode 100644
index 0000000000000..23e9c7566563d
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include <bpf/btf.h>
+
+#ifndef ENOTSUPP
+#define ENOTSUPP 524
+#endif
+
+/* The progs are under #ifdef __BPF_FEATURE_UNWIND */
+static bool llvm_lowers_unwinding(const char *file)
+{
+	struct btf *btf = btf__parse(file, NULL);
+	bool ret;
+
+	ret = btf && btf__find_by_name_kind(btf, "catches", BTF_KIND_FUNC) > 0;
+	btf__free(btf);
+	return ret;
+}
+
+static void run(struct bpf_object *obj, const char *name, int len, int retval, int dtors)
+{
+	struct bpf_program *prog = bpf_object__find_program_by_name(obj, name);
+	struct bpf_map *map = bpf_object__find_map_by_name(obj, ".data.dtors");
+	char data[256] = {};
+	LIBBPF_OPTS(bpf_test_run_opts, opts,
+		.data_in = data,
+		.data_size_in = len,
+		.repeat = 1,
+	);
+	int key = 0, val = 0;
+
+	if (!ASSERT_OK_PTR(prog, name) || !ASSERT_OK_PTR(map, "dtors"))
+		return;
+	ASSERT_OK(bpf_map__update_elem(map, &key, sizeof(key), &val, sizeof(val), 0), "clear");
+	ASSERT_OK(bpf_prog_test_run_opts(bpf_program__fd(prog), &opts), "test_run");
+	ASSERT_EQ(opts.retval, retval, "retval");
+	ASSERT_OK(bpf_map__lookup_elem(map, &key, sizeof(key), &val, sizeof(val), 0), "lookup");
+	ASSERT_EQ(val, dtors, "dtors");
+}
+
+void test_unwind_cxx(void)
+{
+	const char *file = "./unwind_cxx.bpf.o";
+	struct bpf_object *obj;
+	struct bpf_program *prog;
+	int err = -EINVAL;
+
+	if (access(file, R_OK)) {
+		printf("%s:SKIP:gcc doesn't build %s\n", __func__, file);
+		test__skip();
+		return;
+	}
+	if (!llvm_lowers_unwinding(file)) {
+		printf("%s:SKIP:llvm doesn't lower unwinding\n", __func__);
+		test__skip();
+		return;
+	}
+	obj = bpf_object__open_file(file, NULL);
+	if (!ASSERT_OK_PTR(obj, "open"))
+		return;
+	prog = bpf_object__find_program_by_name(obj, "catches_aggregate");
+	if (!ASSERT_OK_PTR(prog, "catches_aggregate"))
+		goto out;
+	bpf_program__set_autoload(prog, false);
+	err = bpf_object__load(obj);
+	if (err == -ENOTSUPP) {
+		printf("%s:SKIP:JIT does not support unwinding\n", __func__);
+		test__skip();
+		goto out;
+	}
+	if (!ASSERT_OK(err, "load"))
+		goto out;
+
+	/* inner's destructor, then middle's */
+	run(obj, "catches", 64, 65, 12);
+	run(obj, "catches", 200, 42, 12);
+	run(obj, "does_not_catch", 64, 65, 12);
+	run(obj, "does_not_catch", 200, 0, 12);
+out:
+	bpf_object__close(obj);
+	if (err)
+		return;
+
+	if (!test__start_subtest("aggregate_return"))
+		return;
+	obj = bpf_object__open_file(file, NULL);
+	if (!ASSERT_OK_PTR(obj, "open_aggregate"))
+		return;
+	bpf_object__for_each_program(prog, obj)
+		bpf_program__set_autoload(prog,
+					 strcmp(bpf_program__name(prog), "catches_aggregate") == 0);
+	if (!ASSERT_OK(bpf_object__load(obj), "load_aggregate"))
+		goto out_aggregate;
+	run(obj, "catches_aggregate", 64, 71, 0);
+	run(obj, "catches_aggregate", 200, 42, 0);
+out_aggregate:
+	bpf_object__close(obj);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index 6df440d826750..04740ba37e918 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -130,6 +130,7 @@
 #include "verifier_typedef.skel.h"
 #include "verifier_uninit.skel.h"
 #include "verifier_unpriv.skel.h"
+#include "verifier_unwind.skel.h"
 #include "verifier_unpriv_perf.skel.h"
 #include "verifier_value_adj_spill.skel.h"
 #include "verifier_value.skel.h"
@@ -320,6 +321,7 @@ void test_verifier_tailcall_jit(void)         { RUN(verifier_tailcall_jit); }
 void test_verifier_typedef(void)              { RUN(verifier_typedef); }
 void test_verifier_uninit(void)               { RUN(verifier_uninit); }
 void test_verifier_unpriv(void)               { RUN(verifier_unpriv); }
+void test_verifier_unwind(void)               { RUN(verifier_unwind); }
 void test_verifier_unpriv_perf(void)          { RUN(verifier_unpriv_perf); }
 void test_verifier_value_adj_spill(void)      { RUN(verifier_value_adj_spill); }
 void test_verifier_value(void)                { RUN(verifier_value); }
diff --git a/tools/testing/selftests/bpf/progs/unwind_attach.c b/tools/testing/selftests/bpf/progs/unwind_attach.c
new file mode 100644
index 0000000000000..df73c26c1ff46
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/unwind_attach.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "../../../include/linux/filter.h"
+#include "bpf_misc.h"
+
+/* The targets are global_unwinds() and global_never_unwinds() in verifier_unwind.c */
+
+int fentry_cnt;
+
+SEC("?fentry/global_unwinds")
+int BPF_PROG(fentry_unwind)
+{
+	fentry_cnt++;
+	return 0;
+}
+
+SEC("?fexit/global_unwinds")
+int BPF_PROG(fexit_unwind)
+{
+	return 0;
+}
+
+SEC("?freplace/global_unwinds")
+unsigned long freplace_returns(void)
+{
+	return 9;
+}
+
+SEC("?freplace/global_unwinds")
+__naked unsigned long freplace_unwinds(void)
+{
+	asm volatile (
+		"r0 = 3;"
+		".8byte %[unwind_exit];"
+		:
+		: __imm_insn(unwind_exit,
+			     BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT, 0, 0))
+	);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/unwind_cxx.cpp b/tools/testing/selftests/bpf/progs/unwind_cxx.cpp
new file mode 100644
index 0000000000000..3a113ffea29ca
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/unwind_cxx.cpp
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * C++ exceptions. llvm lowers unwinding to two insns:
+ *   invoke foo unwind pad    call foo; unwind_goto pad
+ *   resume                   r0 = 0; unwind_exit
+ * so 'throw' is a jump or an exit, 'catch (...)' and the destructors that run on
+ * the way are ordinary blocks.
+ *
+ * vmlinux.h and bpf_helpers.h are not C++.
+ */
+#define SEC(name) __attribute__((section(name), used))
+#define __ksym __attribute__((section(".ksyms")))
+#define __noinline __attribute__((noinline))
+#define __always_inline inline __attribute__((always_inline))
+
+#ifdef __BPF_FEATURE_UNWIND
+
+struct __sk_buff {
+	unsigned int len;
+};
+
+extern "C" {
+void bpf_rcu_read_lock(void) noexcept __ksym;
+void bpf_rcu_read_unlock(void) noexcept __ksym;
+void bpf_preempt_disable(void) noexcept __ksym;
+void bpf_preempt_enable(void) noexcept __ksym;
+
+/* Starts an unwind. Not a kfunc: llvm lowers the call and it doesn't come back. */
+int _Unwind_RaiseException(void *exc);
+
+/* All that is left of the C++ runtime. Nobody looks at the exception object. */
+static char exc_obj[16];
+
+__always_inline void *__cxa_allocate_exception(unsigned long size) noexcept
+{
+	return exc_obj;
+}
+
+__always_inline void __cxa_throw(void *exc, void *tinfo, void (*dtor)(void *))
+{
+	_Unwind_RaiseException(exc);
+}
+
+__always_inline void *__cxa_begin_catch(void *exc) noexcept
+{
+	return exc;
+}
+
+__always_inline void __cxa_end_catch(void) noexcept
+{
+}
+}
+
+/* One decimal digit per destructor, in the order they ran */
+volatile int dtors SEC(".data.dtors");
+
+/* The verifier insists that both are undone on every path out of the prog */
+struct rcu_guard {
+	__always_inline rcu_guard()
+	{
+		bpf_rcu_read_lock();
+	}
+	__always_inline ~rcu_guard()
+	{
+		bpf_rcu_read_unlock();
+		dtors = dtors * 10 + 1;
+	}
+};
+
+struct plain_rcu_guard {
+	__always_inline plain_rcu_guard()
+	{
+		bpf_rcu_read_lock();
+	}
+	__always_inline ~plain_rcu_guard()
+	{
+		bpf_rcu_read_unlock();
+	}
+};
+
+struct preempt_guard {
+	__always_inline preempt_guard()
+	{
+		bpf_preempt_disable();
+	}
+	__always_inline ~preempt_guard()
+	{
+		bpf_preempt_enable();
+		dtors = dtors * 10 + 2;
+	}
+};
+
+static __noinline int inner(int len)
+{
+	rcu_guard guard;
+
+	if (len > 100)
+		throw 1;
+	return len;
+}
+
+static __noinline int middle(int len)
+{
+	preempt_guard guard;
+
+	return inner(len) + 1;
+}
+
+using u128 = unsigned __int128;
+
+/* Global subprogs returning 16 bytes use R0:R2, including unwind exits. */
+extern "C" __noinline u128 aggregate_return(int len)
+{
+	plain_rcu_guard guard;
+
+	if (len > 100)
+		throw 1;
+	return (static_cast<u128>(len) << 64) | 7;
+}
+
+extern "C" SEC("tc") int catches(struct __sk_buff *skb)
+{
+	try {
+		return middle(skb->len);
+	} catch (...) {
+		return 42;
+	}
+}
+
+/* The exception leaves the prog: it returns 0 */
+extern "C" SEC("tc") int does_not_catch(struct __sk_buff *skb)
+{
+	return middle(skb->len);
+}
+
+extern "C" SEC("tc") int catches_aggregate(struct __sk_buff *skb)
+{
+	try {
+		u128 value = aggregate_return(skb->len);
+
+		return static_cast<unsigned long>(value) +
+		       static_cast<unsigned long>(value >> 64);
+	} catch (...) {
+		return 42;
+	}
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/verifier_unwind.c b/tools/testing/selftests/bpf/progs/verifier_unwind.c
new file mode 100644
index 0000000000000..dbe7b5d91acd4
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_unwind.c
@@ -0,0 +1,1182 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "../../../include/linux/filter.h"
+#include "bpf_misc.h"
+
+/*
+ * llvm lowers unwinding to two insns:
+ *   invoke foo unwind pad    call foo; unwind_goto pad
+ *   resume                   unwind_exit
+ * The tests spell them out as raw insns, so they don't depend on llvm version.
+ * The argument of unwind_goto() is the number of insns to skip.
+ */
+#define unwind_goto(off) ".8byte %[insn_unwind_goto_" #off "];"
+#define unwind_exit ".8byte %[insn_unwind_exit];"
+
+#define __imm_unwind_goto(off) \
+	__imm_insn(insn_unwind_goto_##off, \
+		   BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO, off, 0))
+#define __imm_unwind_exit \
+	__imm_insn(insn_unwind_exit, \
+		   BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT, 0, 0))
+
+extern void bpf_rcu_read_lock(void) __ksym;
+extern void bpf_rcu_read_unlock(void) __ksym;
+extern void bpf_throw(__u64 cookie) __ksym;
+
+struct {
+	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+	__uint(max_entries, 1);
+	__uint(key_size, sizeof(int));
+	__uint(value_size, sizeof(int));
+} prog_array SEC(".maps");
+
+struct timer_elem {
+	struct bpf_timer t;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__uint(max_entries, 1);
+	__type(key, int);
+	__type(value, struct timer_elem);
+} timer_map SEC(".maps");
+
+/* The verifier doesn't know what is in there */
+unsigned long zero;
+unsigned long one = 1;
+
+/*
+ * The verifier follows a static subprog, so it knows which way it leaves when
+ * it knows the argument. To have both ways at run time the tests pass skb->len,
+ * which is 64, or skb->len - 64.
+ *
+ * r1 == 0: returns 7, otherwise unwinds with r0 = 5
+ */
+__naked __noinline __used
+static unsigned long may_unwind(void)
+{
+	asm volatile (
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+/* owns nothing: its pad only carries on */
+__naked __noinline __used
+static unsigned long passes_on(void)
+{
+	asm volatile (
+		"call may_unwind;"
+		unwind_goto(2)
+		"r0 += 1;"
+		"exit;"
+		unwind_exit
+		:
+		: __imm_unwind_goto(2),
+		  __imm_unwind_exit
+	);
+}
+
+__naked __noinline __used
+static unsigned long always_unwinds(void)
+{
+	asm volatile (
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+__naked __noinline __used
+static unsigned long never_unwinds(void)
+{
+	asm volatile (
+		"r0 = 7;"
+		"exit;"
+	);
+}
+
+/* Both can unwind. This one returns 7. */
+__naked __noinline __used
+unsigned long global_returns(void)
+{
+	asm volatile (
+		"r1 = %[zero] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_addr(zero),
+		  __imm_unwind_exit
+	);
+}
+
+/* This one unwinds with r0 = 5 */
+__naked __noinline __used
+unsigned long global_unwinds(void)
+{
+	asm volatile (
+		"r1 = %[one] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_addr(one),
+		  __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: the callee returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int callee_returns(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r1 -= 64;"
+		"call may_unwind;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: the callee unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__naked int callee_unwinds(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call may_unwind;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: return through two frames")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(8)
+__naked int two_frames_return(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r1 -= 64;"
+		"call passes_on;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* unwind_exit returns r0 like exit does */
+SEC("tc")
+__description("unwind: unwind through two frames, r0 reaches the pad")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(15)
+__naked int two_frames_unwind(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call passes_on;"
+		unwind_goto(1)
+		"exit;"
+		"r0 += 10;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: r6-r9 and the stack are intact in the pad")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(1111)
+__naked int pad_sees_callee_saved(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r6 = 1;"
+		"r7 = 10;"
+		"r8 = 100;"
+		"r9 = 400;"
+		"*(u64 *)(r10 - 8) = 600;"
+		"call passes_on;"
+		unwind_goto(2)
+		"r0 = 0;"
+		"exit;"
+		"r0 = r6;"
+		"r0 += r7;"
+		"r0 += r8;"
+		"r0 += r9;"
+		"r1 = *(u64 *)(r10 - 8);"
+		"r0 += r1;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: r1-r5 are not readable in the pad")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R3 !read_ok")
+__naked int pad_reads_r3(void)
+{
+	asm volatile (
+		"r3 = 0;"
+		"call always_unwinds;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = r3;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_exit of the main prog is exit")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(3)
+__naked int main_prog_unwinds(void)
+{
+	asm volatile (
+		"r0 = 3;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_exit of the main prog needs r0")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R0 !read_ok")
+__naked int main_prog_unwinds_no_r0(void)
+{
+	asm volatile (
+		unwind_exit
+		:
+		: __imm_unwind_exit
+		: __clobber_all
+	);
+}
+
+/* What follows unwind_goto is dead code. unwind_exit still has to find the pad. */
+SEC("tc")
+__description("unwind: the callee never returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__xlated("0: call pc+3")
+__xlated("1: unwind_goto pc+0")
+__xlated("2: r0 = 42")
+__xlated("3: exit")
+__naked int callee_never_returns(void)
+{
+	asm volatile (
+		"call always_unwinds;"
+		unwind_goto(2)
+		"r0 = 1;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+/* The pad is dead code and goes away together with unwind_goto */
+SEC("tc")
+__description("unwind: the callee never unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__xlated("0: call pc+1")
+__xlated("1: exit")
+__xlated("2: r0 = 7")
+__xlated("3: exit")
+__naked int callee_never_unwinds(void)
+{
+	asm volatile (
+		"call never_unwinds;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = *(u64 *)(r0 + 0);"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: the pad drops what the frame owns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__naked int pad_unlocks(void)
+{
+	asm volatile (
+		"r6 = *(u32 *)(r1 + %[len]);"
+		"call %[bpf_rcu_read_lock];"
+		"r1 = r6;"
+		"call may_unwind;"
+		unwind_goto(3)
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 0;"
+		"exit;"
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm(bpf_rcu_read_lock),
+		  __imm(bpf_rcu_read_unlock),
+		  __imm_unwind_goto(3)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: the pad is checked like any other code")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("in main prog cannot be used inside bpf_rcu_read_lock-ed region")
+__naked int pad_does_not_unlock(void)
+{
+	asm volatile (
+		"r6 = *(u32 *)(r1 + %[len]);"
+		"call %[bpf_rcu_read_lock];"
+		"r1 = r6;"
+		"call may_unwind;"
+		unwind_goto(3)
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 0;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm(bpf_rcu_read_lock),
+		  __imm(bpf_rcu_read_unlock),
+		  __imm_unwind_goto(3)
+		: __clobber_all
+	);
+}
+
+/* Leaves either way with the same registers */
+__naked __noinline __used
+static unsigned long same_regs(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"if r1 != 0 goto 1f;"
+		"exit;"
+	"1:"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+/*
+ * After the call the two states differ only in the way the callee left.
+ * The second one must not be pruned, or nobody looks at the pad.
+ */
+SEC("tc")
+__description("unwind: the way the callee left is a part of the state")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R0 invalid mem access 'scalar'")
+__flag(BPF_F_TEST_STATE_FREQ)
+__naked int no_pruning_after_the_call(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call same_regs;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = *(u64 *)(r0 + 0);"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* Returns ctx, or unwinds with a number in r0 */
+__naked __noinline __used
+static unsigned long ctx_or_unwind(void)
+{
+	asm volatile (
+		"r2 = *(u32 *)(r1 + %[len]);"
+		"if r2 > 100 goto 1f;"
+		"r0 = r1;"
+		"exit;"
+	"1:"
+		"r0 = 0;"
+		unwind_exit
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_exit
+	);
+}
+
+/*
+ * unwind_exit changes its return address, so the cpu mispredicts the return and
+ * runs what follows unwind_goto with r0 of unwind_exit.
+ */
+SEC("socket")
+__description("unwind: what follows unwind_goto runs speculatively")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(50) /* 64 without the ethernet header */
+__success_unpriv __retval_unpriv(50)
+__caps_unpriv(CAP_BPF)
+#ifdef SPEC_V1
+__xlated_unpriv("unwind_goto pc+{{.*}}")
+__xlated_unpriv("nospec") /* inserted to prevent `R0 invalid mem access 'scalar'` */
+__xlated_unpriv("r0 = *(u32 *)(r0 +{{.*}})")
+#endif
+__naked int speculative_fall_through(void)
+{
+	asm volatile (
+		"call ctx_or_unwind;"
+		unwind_goto(2)
+		"r0 = *(u32 *)(r0 + %[len]);"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: a global subprog returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int calls_global_returns(void)
+{
+	asm volatile (
+		"call global_returns;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: a global subprog unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(15)
+__naked int calls_global_unwinds(void)
+{
+	asm volatile (
+		"call global_unwinds;"
+		unwind_goto(1)
+		"exit;"
+		"r0 += 10;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* The verifier doesn't look inside */
+SEC("tc")
+__description("unwind: the pad of a call of a global subprog is always checked")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R0 invalid mem access 'scalar'")
+__naked int global_pad_is_checked(void)
+{
+	asm volatile (
+		"call global_returns;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = *(u64 *)(r0 + 0);"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+__naked __noinline __used
+unsigned long global_never_unwinds(void)
+{
+	asm volatile (
+		"r0 = 7;"
+		"exit;"
+	);
+}
+
+SEC("tc")
+__description("unwind: a global subprog that cannot unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int calls_global_never_unwinds(void)
+{
+	asm volatile (
+		"call global_never_unwinds;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: no unwind_goto after a call of a global subprog")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("no unwind_goto after the call of global_returns()")
+__naked int global_no_unwind_goto(void)
+{
+	asm volatile (
+		"call global_returns;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: no unwind_goto after a call of a static subprog")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_exit, but no unwind_goto after the call at insn 1")
+__naked int static_no_unwind_goto(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call may_unwind;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len))
+		: __clobber_all
+	);
+}
+
+/* The verifier sees that it doesn't get to unwind_exit */
+SEC("tc")
+__description("unwind: no unwind_goto is fine when the callee doesn't unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int static_does_not_get_there(void)
+{
+	asm volatile (
+		"r1 = 0;"
+		"call may_unwind;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: callx, the callee returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int callx_returns(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r1 -= 64;"
+		"r6 = %[may_unwind] ll;"
+		"callx r6;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_addr(may_unwind),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: callx, the callee unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__naked int callx_unwinds(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r6 = %[may_unwind] ll;"
+		"callx r6;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_addr(may_unwind),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto after a plain insn")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 1 doesn't follow a call of a subprog")
+__naked int unwind_goto_after_mov(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		unwind_goto(0)
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto is the first insn")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 0 doesn't follow a call of a subprog")
+__naked int unwind_goto_first(void)
+{
+	asm volatile (
+		unwind_goto(0)
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto after a call of a helper")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 1 doesn't follow a call of a subprog")
+__naked int unwind_goto_after_helper(void)
+{
+	asm volatile (
+		"call %[bpf_get_prandom_u32];"
+		unwind_goto(0)
+		"exit;"
+		:
+		: __imm(bpf_get_prandom_u32),
+		  __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto after a call of a kfunc")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 1 doesn't follow a call of a subprog")
+__naked int unwind_goto_after_kfunc(void)
+{
+	asm volatile (
+		"call %[bpf_rcu_read_lock];"
+		unwind_goto(0)
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm(bpf_rcu_read_lock),
+		  __imm(bpf_rcu_read_unlock),
+		  __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: jump to unwind_goto")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("jump to unwind_goto at insn 3")
+__naked int jump_to_unwind_goto(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"if r1 != 0 goto 1f;"
+		"call never_unwinds;"
+	"1:"
+		unwind_goto(0)
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: reserved fields of unwind_goto")
+__failure __msg("invalid may_goto imm 1")
+__naked int unwind_goto_reserved(void)
+{
+	asm volatile (
+		"call never_unwinds;"
+		".8byte %[bad];"
+		"exit;"
+		:
+		: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO, 0, 1))
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unknown kind of jcond")
+__failure __msg("invalid may_goto imm 0")
+__naked int jcond_unknown(void)
+{
+	asm volatile (
+		"call never_unwinds;"
+		".8byte %[bad];"
+		"exit;"
+		:
+		: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO + 1, 0, 0))
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unknown kind of exit")
+__failure __msg("BPF_EXIT uses reserved fields")
+__naked int exit_unknown(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		".8byte %[bad];"
+		:
+		: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT + 1, 0, 0))
+		: __clobber_all
+	);
+}
+
+/* r1 is the index: unwinds in the second iteration */
+__naked __noinline __used
+static unsigned long cb_unwinds(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"if r1 != 0 goto 1f;"
+		"exit;"
+	"1:"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: callback unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("callback cannot unwind")
+__naked int callback_unwinds(void)
+{
+	asm volatile (
+		"r1 = 3;"
+		"r2 = %[cb_unwinds];"
+		"r3 = 0;"
+		"r4 = 0;"
+		"call %[bpf_loop];"
+		"exit;"
+		:
+		: __imm_ptr(cb_unwinds),
+		  __imm(bpf_loop)
+		: __clobber_common
+	);
+}
+
+SEC("tc")
+__description("unwind: async callback unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("callback cannot unwind")
+int async_callback_unwinds(void *ctx)
+{
+	struct timer_elem *val;
+	int key = 0;
+
+	val = bpf_map_lookup_elem(&timer_map, &key);
+	if (!val)
+		return 0;
+
+	bpf_timer_init(&val->t, &timer_map, 0);
+	bpf_timer_set_callback(&val->t, (void *)always_unwinds);
+	return 0;
+}
+
+/* Not naked: the argument has to have a name in BTF */
+__noinline int exception_cb_unwinds(__u64 cookie)
+{
+	if (cookie)
+		asm volatile ("r0 = 0;" unwind_exit :: __imm_unwind_exit : "r0");
+	return 0;
+}
+
+SEC("tc")
+__description("unwind: exception callback unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__attribute__((btf_decl_tag("exception_callback:exception_cb_unwinds")))
+__failure __msg("callback cannot unwind")
+int exception_callback_unwinds(struct __sk_buff *skb)
+{
+	bpf_throw(skb->len);
+	return 0;
+}
+
+/* What it calls can unwind as long as it doesn't let it out */
+__naked __noinline __used
+static unsigned long cb_catches(void)
+{
+	asm volatile (
+		"call may_unwind;"
+		unwind_goto(0)
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+	);
+}
+
+SEC("tc")
+__description("unwind: callback catches")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(3)
+__naked int callback_catches(void)
+{
+	asm volatile (
+		"r1 = 3;"
+		"r2 = %[cb_catches];"
+		"r3 = 0;"
+		"r4 = 0;"
+		"call %[bpf_loop];"
+		"exit;"
+		:
+		: __imm_ptr(cb_catches),
+		  __imm(bpf_loop)
+		: __clobber_common
+	);
+}
+
+/* Both leave the subprog like exit does */
+__naked __noinline __used
+static unsigned long does_tail_call(void)
+{
+	asm volatile (
+		"r6 = *(u32 *)(r1 + %[len]);"
+		"r2 = %[prog_array] ll;"
+		"r3 = 0;"
+		"call %[bpf_tail_call];"
+		"r0 = 7;"
+		"if r6 == 64 goto 1f;"
+		unwind_exit
+	"1:"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm(bpf_tail_call),
+		  __imm_addr(prog_array),
+		  __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: tail call in a subprog that can unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int tail_call_in_subprog(void)
+{
+	asm volatile (
+		"call does_tail_call;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* The packet is shorter than that: ld_abs leaves the subprog with r0 = 0 */
+__naked __noinline __used
+static unsigned long does_ld_abs(void)
+{
+	asm volatile (
+		"r6 = r1;"
+		"r0 = *(u8 *)skb[1000];"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: ld_abs in a subprog that can unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int ld_abs_in_subprog(void)
+{
+	asm volatile (
+		"call does_ld_abs;"
+		unwind_goto(2)
+		"r0 += 7;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+/*
+ * JITs don't emit anything for unwind_goto. It's an extable entry, where
+ * unwind_exit looks its return address up.
+ */
+SEC("tc")
+__description("unwind: jit")
+__success __retval(42)
+__load_if_JITed()
+__arch_x86_64
+__jited("	callq	0x{{.*}}")
+__jited("	movl	$0x7, %eax")
+__jited("...")
+__jited("	movl	$0x5, %eax")
+__jited("	pushq	%rax")
+__jited("	pushq	%rsi")
+__jited("	movq	0x8(%rbp), %rdi")
+__jited("...") /* call depth accounting */
+__jited("	callq	0x{{.*}}")
+__jited("	movq	%rax, 0x8(%rbp)")
+__jited("	popq	%rsi")
+__jited("	popq	%rax")
+__arch_arm64
+__jited("	bl	{{.*}}")
+__jited("	mov	x8, x0")
+__jited("	mov	x8, #0x7")
+__jited("...")
+__jited("	mov	x8, #0x5")
+__jited("	stp	x8, x1, [sp, #-0x10]!")
+__jited("	ldr	x17, [x29, #0x8]")
+__jited("...")
+__jited("	str	x17, [x29, #0x8]")
+__jited("	ldp	x8, x1, [sp], #0x10")
+__naked int jit(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call may_unwind;"
+		unwind_goto(2)
+		"r0 = 7;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+/* r1 == 0: returns 7, otherwise unwinds with r0 = 5 */
+__naked __noinline __used
+static unsigned long may_unwind_big_stack(void)
+{
+	asm volatile (
+		"*(u64 *)(r10 - 200) = r1;"
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+/*
+ * x86-64 keeps the pointer to the private stack in r9 and saves it around a call.
+ * The value that is read back in the pad is there only when r9 is restored.
+ */
+SEC("raw_tp")
+__description("unwind: private stack, the callee unwinds")
+__success __retval(47)
+__load_if_JITed()
+__arch_x86_64
+__jited("	pushq	%r9")
+__jited("...") /* call depth accounting */
+__jited("	callq	0x{{.*}}")
+__jited("	popq	%r9")
+__jited("	jmp	L{{.*}}")
+__jited("	popq	%r9")
+__jited("	jmp	L{{.*}}")
+__arch_arm64
+__naked int private_stack_unwinds(void)
+{
+	asm volatile (
+		"*(u64 *)(r10 - 200) = 42;"
+		"r1 = %[one] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"call may_unwind_big_stack;"
+		unwind_goto(1)
+		"exit;"
+		"r1 = *(u64 *)(r10 - 200);"
+		"r0 += r1;"
+		"exit;"
+		:
+		: __imm_addr(one),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("raw_tp")
+__description("unwind: private stack, the callee returns")
+__success __retval(49)
+__load_if_JITed()
+__arch_x86_64
+__arch_arm64
+__naked int private_stack_returns(void)
+{
+	asm volatile (
+		"*(u64 *)(r10 - 200) = 42;"
+		"r1 = %[zero] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"call may_unwind_big_stack;"
+		unwind_goto(3)
+		"r1 = *(u64 *)(r10 - 200);"
+		"r0 += r1;"
+		"exit;"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_addr(zero),
+		  __imm_unwind_goto(3)
+		: __clobber_all
+	);
+}
+
+/*
+ * BTF FUNC records are not generated for kfuncs referenced
+ * from inline assembly. These records are necessary for
+ * libbpf to link the program. The function below is a hack
+ * to ensure that BTF FUNC records are generated.
+ */
+void __kfunc_btf_root(void)
+{
+	bpf_rcu_read_lock();
+	bpf_rcu_read_unlock();
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c
index 18d47e9254007..824e5912ebcb8 100644
--- a/tools/testing/selftests/bpf/test_loader.c
+++ b/tools/testing/selftests/bpf/test_loader.c
@@ -1546,7 +1546,8 @@ void run_subtest(struct test_loader *tester,
 		}
 
 		err = do_prog_test_run(bpf_program__fd(tprog), &retval,
-				       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ? true : false,
+				       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ||
+				       bpf_program__type(tprog) == BPF_PROG_TYPE_RAW_TRACEPOINT,
 				       spec->linear_sz);
 		if (!err && retval != subspec->retval && subspec->retval != POINTER_VALUE) {
 			PRINT_FAIL("Unexpected retval: %d != %d\n", retval, subspec->retval);

3/2 2026/10/20 09:15 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 31079
							output: 85
							thoughts: 1239
						

Thoughts:
**BPF Unwinding Patch Analysis - AMD64 Focus**

Okay, let's dive into this patch series. My initial impression is that this is significant, especially considering the target architecture is amd64. Given my expertise, the addition of unwinding support to BPF immediately flags as interesting. The changes are widespread, hitting several core areas: `arch/arm64/net/bpf_jit.h`, `arch/arm64/net/bpf_jit_comp.c`, `arch/x86/net/bpf_jit_comp.c`, and core BPF infrastructure files, including the verifier. Because we are targeting amd64, the x86_64 JIT is directly in the crosshairs.

My triage philosophy dictates that modifications to BPF are prime candidates for fuzzing, and this patch absolutely introduces new execution paths. Syzkaller's existing BPF fuzzing capabilities via `bpf` syscalls confirm this. "WorthFuzzing" is essentially a given here.

Now, let's zero in on FocusSymbols and EnableConfigs. I need to avoid generic functions and target the specific logic being added or altered. I am going to have to do a deep dive to work through all of these changes.

Looking at the modified functions, `arch/x86/net/bpf_jit_comp.c` has several static functions (like `add_unwind_exentry`, `bpf_unwind_target`, `emit_unwind_ret_addr`, `do_jit`, but also has the non-static `bpf_jit_supports_unwind`) related to the JIT compiler. The verifier has the most changes, with all static functions. `kernel/bpf/const_fold.c` has `bpf_prune_dead_branches`, `kernel/bpf/fixups.c` has `bpf_opt_hard_wire_dead_code_branches` and `jit_subprogs`, `kernel/bpf/states.c` has `states_equal`, `kernel/bpf/syscall.c` has `bpf_insn_prepare_dump` and `kernel/bpf/core.c` has `bpf_jit_supports_unwind`.

My next step is to quickly determine the visibility of these functions. Are any exported or non-static? Are they called from elsewhere? I will need to check this quickly to get the full scope of the impact of the changes. I will use grepper or read-file to determine the full scope of how the symbols are used and exported.



4/2 2026/10/20 09:15 tool grepper
0m Args:
{
  "Expression": "bpf_check_attach_target|bpf_check|bpf_prune_dead_branches|bpf_opt_hard_wire_dead_code_branches|bpf_jit_supports_unwind",
  "PathPrefixes": [
    "kernel/bpf/",
    "arch/x86/net/"
  ]
}

Results:
{
  "Output": "\nFull output is too long, showing 200 out of 261 lines.\nUse more precise expression or PathPrefixes if possible.\n\nNumber of matching lines per file (13 files in total):\narch/x86/net/bpf_jit_comp.c:1\narch/x86/net/bpf_timed_may_goto.S:1\nkernel/bpf/bpf_iter.c:1\nkernel/bpf/btf.c:1\nkernel/bpf/cfg.c:1\nkernel/bpf/cgroup.c:1\nkernel/bpf/check_btf.c:5\nkernel/bpf/const_fold.c:1\nkernel/bpf/core.c:5\nkernel/bpf/fixups.c:2\nkernel/bpf/syscall.c:10\nkernel/bpf/trampoline.c:2\nkernel/bpf/verifier.c:17\n\narch/x86/net/bpf_jit_comp.c=5118=bool bpf_jit_supports_kfunc_ret_reg_pair(void)\n--\narch/x86/net/bpf_jit_comp.c-5122-\narch/x86/net/bpf_jit_comp.c:5123:bool bpf_jit_supports_unwind(void)\narch/x86/net/bpf_jit_comp.c-5124-{\n--\narch/x86/net/bpf_timed_may_goto.S=11=SYM_FUNC_START(arch_bpf_timed_may_goto)\n--\narch/x86/net/bpf_timed_may_goto.S-33-\tCALL_DEPTH_ACCOUNT\narch/x86/net/bpf_timed_may_goto.S:34:\tcall bpf_check_timed_may_goto\narch/x86/net/bpf_timed_may_goto.S-35-\n--\nkernel/bpf/bpf_iter.c=530=int bpf_iter_link_attach(const union bpf_attr *attr, bpfptr_t uattr,\n--\nkernel/bpf/bpf_iter.c-551-\tif (!bpfptr_is_null(ulinfo)) {\nkernel/bpf/bpf_iter.c:552:\t\terr = bpf_check_uarg_tail_zero(ulinfo, sizeof(linfo),\nkernel/bpf/bpf_iter.c-553-\t\t\t\t\t       linfo_len);\n--\nkernel/bpf/btf.c=7438=bool btf_ctx_access(int off, int size, enum bpf_access_type type,\n--\nkernel/bpf/btf.c-7622-\t\t\treturn false;\nkernel/bpf/btf.c:7623:\t\t/* Checked by bpf_check_attach_target */\nkernel/bpf/btf.c-7624-\t\ttname += sizeof(\"btf_trace_\") - 1;\n--\nkernel/bpf/cfg.c=542=static int visit_insn(int t, struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-674- */\nkernel/bpf/cfg.c:675:int bpf_check_cfg(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-676-{\n--\nkernel/bpf/cgroup.c=1953=EXPORT_SYMBOL(__cgroup_bpf_run_filter_sock_ops);\nkernel/bpf/cgroup.c-1954-\nkernel/bpf/cgroup.c:1955:int __cgroup_bpf_check_dev_permission(short dev_type, u32 major, u32 minor,\nkernel/bpf/cgroup.c-1956-\t\t\t\t      short access, enum cgroup_bpf_attach_type atype)\n--\nkernel/bpf/check_btf.c=31=static int prepare_btf_func(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-70-\tfor (i = 0; i \u003c nfuncs; i++) {\nkernel/bpf/check_btf.c:71:\t\tret = bpf_check_uarg_tail_zero(urecord, krec_size, urec_size);\nkernel/bpf/check_btf.c-72-\t\tif (ret) {\n--\nkernel/bpf/check_btf.c=213=static int check_btf_line(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-253-\tfor (i = 0; i \u003c nr_linfo; i++) {\nkernel/bpf/check_btf.c:254:\t\terr = bpf_check_uarg_tail_zero(ulinfo, expected_size, rec_size);\nkernel/bpf/check_btf.c-255-\t\tif (err) {\n--\nkernel/bpf/check_btf.c-340-\nkernel/bpf/check_btf.c:341:int bpf_check_core_relo(struct bpf_verifier_env *env,\nkernel/bpf/check_btf.c-342-\t\t\tconst union bpf_attr *attr,\n--\nkernel/bpf/check_btf.c-376-\t\t/* future proofing when sizeof(bpf_core_relo) changes */\nkernel/bpf/check_btf.c:377:\t\terr = bpf_check_uarg_tail_zero(u_core_relo, expected_size, rec_size);\nkernel/bpf/check_btf.c-378-\t\tif (err) {\n--\nkernel/bpf/check_btf.c=410=int bpf_prepare_btf_info(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-437-\nkernel/bpf/check_btf.c:438:int bpf_check_btf_info(struct bpf_verifier_env *env,\nkernel/bpf/check_btf.c-439-\t\t       const union bpf_attr *attr,\n--\nkernel/bpf/const_fold.c=319=static int eval_const_branch(u8 opcode, u64 dst_val, u64 src_val)\n--\nkernel/bpf/const_fold.c-342- */\nkernel/bpf/const_fold.c:343:int bpf_prune_dead_branches(struct bpf_verifier_env *env)\nkernel/bpf/const_fold.c-344-{\n--\nkernel/bpf/core.c-16- * Andi Kleen - Fix a few bad bugs and races.\nkernel/bpf/core.c:17: * Kris Katterjohn - Added many additional checks in bpf_check_classic()\nkernel/bpf/core.c-18- */\n--\nkernel/bpf/core.c=2581=bool bpf_prog_map_compatible(struct bpf_map *map, const struct bpf_prog *fp)\n--\nkernel/bpf/core.c-2593-\nkernel/bpf/core.c:2594:static int bpf_check_tail_call(const struct bpf_prog *fp)\nkernel/bpf/core.c-2595-{\n--\nkernel/bpf/core.c=2731=struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,\n--\nkernel/bpf/core.c-2784-\t */\nkernel/bpf/core.c:2785:\t*err = bpf_check_tail_call(fp);\nkernel/bpf/core.c-2786-\n--\nkernel/bpf/core.c=3397=bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)\n--\nkernel/bpf/core.c-3402-/* unwind_goto and unwind_exit */\nkernel/bpf/core.c:3403:bool __weak bpf_jit_supports_unwind(void)\nkernel/bpf/core.c-3404-{\n--\nkernel/bpf/core.c=3620=static noinline void bpf_prog_report_may_goto_violation(void)\n--\nkernel/bpf/core.c-3635-\nkernel/bpf/core.c:3636:u64 bpf_check_timed_may_goto(struct bpf_timed_may_goto *p)\nkernel/bpf/core.c-3637-{\n--\nkernel/bpf/fixups.c=643=bool bpf_insn_is_cond_jump(u8 code)\n--\nkernel/bpf/fixups.c-656-\nkernel/bpf/fixups.c:657:void bpf_opt_hard_wire_dead_code_branches(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-658-{\n--\nkernel/bpf/fixups.c=920=int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-978-\t\t\tif (env-\u003eprog-\u003etype == BPF_PROG_TYPE_EXT \u0026\u0026\nkernel/bpf/fixups.c:979:\t\t\t    bpf_check_ext_prologue(\u0026env-\u003elog,\nkernel/bpf/fixups.c-980-\t\t\t\t\t\t   env-\u003eprog-\u003eaux-\u003edst_prog,\n--\nkernel/bpf/syscall.c=71=static const struct bpf_map_ops * const bpf_map_types[] = {\n--\nkernel/bpf/syscall.c-90- */\nkernel/bpf/syscall.c:91:int bpf_check_uarg_tail_zero(bpfptr_t uaddr,\nkernel/bpf/syscall.c-92-\t\t\t     size_t expected_size,\n--\nkernel/bpf/syscall.c=2952=static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_attr *attr_log)\n--\nkernel/bpf/syscall.c-3175-\t/* run eBPF verifier */\nkernel/bpf/syscall.c:3176:\terr = bpf_check(\u0026prog, attr, uattr, attr_log);\nkernel/bpf/syscall.c-3177-\tif (err \u003c 0)\n--\nkernel/bpf/syscall.c=3677=static int bpf_tracing_prog_attach(struct bpf_prog *prog,\n--\nkernel/bpf/syscall.c-3809-\nkernel/bpf/syscall.c:3810:\t\terr = bpf_check_attach_target(NULL, prog, tgt_prog, btf_id,\nkernel/bpf/syscall.c-3811-\t\t\t\t\t      \u0026tgt_info);\n--\nkernel/bpf/syscall.c=5165=static int bpf_prog_get_info_by_fd(struct file *file,\n--\nkernel/bpf/syscall.c-5179-\tlen = offsetofend(struct bpf_prog_info, attach_btf_id);\nkernel/bpf/syscall.c:5180:\terr = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), len, info_len);\nkernel/bpf/syscall.c-5181-\tif (err)\n--\nkernel/bpf/syscall.c=5455=static int bpf_map_get_info_by_fd(struct file *file,\n--\nkernel/bpf/syscall.c-5465-\tlen = offsetofend(struct bpf_map_info, hash_size);\nkernel/bpf/syscall.c:5466:\terr = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), len, info_len);\nkernel/bpf/syscall.c-5467-\tif (err)\n--\nkernel/bpf/syscall.c=5530=static int bpf_btf_get_info_by_fd(struct file *file,\n--\nkernel/bpf/syscall.c-5538-\nkernel/bpf/syscall.c:5539:\terr = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), sizeof(*uinfo), info_len);\nkernel/bpf/syscall.c-5540-\tif (err)\n--\nkernel/bpf/syscall.c=5546=static int bpf_link_get_info_by_fd(struct file *file,\n--\nkernel/bpf/syscall.c-5556-\nkernel/bpf/syscall.c:5557:\terr = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), sizeof(info), info_len);\nkernel/bpf/syscall.c-5558-\tif (err)\n--\nkernel/bpf/syscall.c=5588=static int token_get_info_by_fd(struct file *file,\n--\nkernel/bpf/syscall.c-5596-\nkernel/bpf/syscall.c:5597:\terr = bpf_check_uarg_tail_zero(USER_BPFPTR(uinfo), sizeof(*uinfo), info_len);\nkernel/bpf/syscall.c-5598-\tif (err)\n--\nkernel/bpf/syscall.c=6400=static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size,\n--\nkernel/bpf/syscall.c-6408-\nkernel/bpf/syscall.c:6409:\terr = bpf_check_uarg_tail_zero(uattr, sizeof(attr), size);\nkernel/bpf/syscall.c-6410-\tif (err)\n--\nkernel/bpf/syscall.c-6420-\tif (cmd \u0026 BPF_COMMON_ATTRS) {\nkernel/bpf/syscall.c:6421:\t\terr = bpf_check_uarg_tail_zero(uattr_common,\nkernel/bpf/syscall.c-6422-\t\t\t\t\t       offsetofend(struct bpf_common_attr, log_true_size),\n--\nkernel/bpf/trampoline.c=1176=int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog,\n--\nkernel/bpf/trampoline.c-1186-\nkernel/bpf/trampoline.c:1187:\terr = bpf_check_attach_target(NULL, prog, NULL,\nkernel/bpf/trampoline.c-1188-\t\t\t\t      prog-\u003eaux-\u003eattach_btf_id,\n--\nkernel/bpf/trampoline.c=1726=int bpf_trampoline_multi_attach(struct bpf_prog *prog, u32 *ids,\n--\nkernel/bpf/trampoline.c-1739-\nkernel/bpf/trampoline.c:1740:\t\terr = bpf_check_attach_btf_id_multi(btf, prog, ids[i], \u0026tgt_info);\nkernel/bpf/trampoline.c-1741-\t\tif (err)\n--\nkernel/bpf/verifier.c=61=static bool bpf_global_percpu_ma_set;\nkernel/bpf/verifier.c-62-\nkernel/bpf/verifier.c:63:/* bpf_check() is a static code analyzer that walks eBPF program\nkernel/bpf/verifier.c-64- * instruction by instruction and updates register/stack state.\n--\nkernel/bpf/verifier.c=3439=static int check_unwind_insns(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-3464-\t\treturn 0;\nkernel/bpf/verifier.c:3465:\tif (!bpf_jit_supports_unwind()) {\nkernel/bpf/verifier.c-3466-\t\tverbose(env, \"JIT does not support unwinding\\n\");\n--\nkernel/bpf/verifier.c=14862=static int process_kf_arg_ptr_to_rbtree_node(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-14873- * LSM hooks and iters (both sleepable and non-sleepable) are safe.\nkernel/bpf/verifier.c:14874: * Any sleepable progs are also safe since bpf_check_attach_target() enforce\nkernel/bpf/verifier.c-14875- * them can only be attached to some specific hook points.\n--\nkernel/bpf/verifier.c=22437=static int find_tgt_subprog(const struct bpf_prog *tgt_prog, u32 btf_id)\n--\nkernel/bpf/verifier.c-22453- */\nkernel/bpf/verifier.c:22454:int bpf_check_ext_prologue(struct bpf_verifier_log *log,\nkernel/bpf/verifier.c-22455-\t\t\t   const struct bpf_prog *tgt_prog, u32 btf_id)\n--\n\n"
}

5/2 2026/10/20 09:15 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 37396
							output: 45
							thoughts: 619
						

Thoughts:
**Analyzing BPF Verifier Symbol Coverage for Fuzzing Efficiency**

Okay, let's dive into this BPF coverage issue and optimize our fuzzing efforts. First, I need to check `bpf_check_attach_target` in `kernel/bpf/verifier.c`. Good, it's non-static, so we're starting on a solid track. Then, let's quickly review other relevant functions: `bpf_prune_dead_branches`, `bpf_opt_hard_wire_dead_code_branches` and `bpf_check`.  Alright, they are non-static as well.

Now, a key question arises: Is `bpf_check` *ubiquitous*? If so, we're likely wasting our focus.  Let's circle back to the FocusSymbols rules.  The guidelines are clear: avoid ubiquitous functions because they dilute the fuzzing effort. We want *feature-specific* logic and API entrypoints. That's the name of the game.

Okay, let's see if we can find something that's *not* ubiquitous. Time to check the static functions in `kernel/bpf/verifier.c`.  Ah, `check_unwind_insns` is static. This triggers a question: Are static functions symbols in vmlinux if kallsyms or debug symbols are enabled? We need to keep this in mind.

Wait a second, let me re-read the syzkaller symbol coverage instructions carefully. It emphasizes handling *static inline* functions in headers, because they lack distinct symbol addresses in vmlinux. Crucially, they lack distinct symbol addresses. If the changes are in static inline helpers, we need to find non-static caller functions.

But hold on! The instruction specifically talks about static inline functions in headers, which lack symbols. But static functions in .c files DO have symbols! Okay, even if they might be inlined, it's still worth checking what functions in .c files were modified or added.

Now, I'll shift gears and look at `arch/x86/net/bpf_jit_comp.c`. Let's view the modified code there. This is where the action is, right?



6/2 2026/10/20 09:15 tool read-file
0m Args:
{
  "File": "arch/x86/net/bpf_jit_comp.c",
  "FirstLine": 2520,
  "LineCount": 70
}

Results:
{
  "Contents": "2520:\t * ret_addr - 1, because what follows the call can have an entry of its own.\n2521:\t * No insn starts there, so a fault cannot hit the entry.\n2522:\t */\n2523:\tstatic int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,\n2524:\t\t\t\t      int ret_addr, int target)\n2525:\t{\n2526:\t\tstruct exception_table_entry *ex;\n2527:\t\ts64 delta;\n2528:\t\n2529:\t\tif (!prog-\u003eaux-\u003eextable)\n2530:\t\t\treturn 0;\n2531:\t\tif (*excnt \u003e= prog-\u003eaux-\u003enum_exentries) {\n2532:\t\t\tpr_err(\"unwind extable bug\\n\");\n2533:\t\t\treturn -EFAULT;\n2534:\t\t}\n2535:\t\tex = \u0026prog-\u003eaux-\u003eextable[(*excnt)++];\n2536:\t\tdelta = image + ret_addr - 1 - (u8 *)\u0026ex-\u003einsn;\n2537:\t\tif (!is_simm32(delta)) {\n2538:\t\t\tpr_err(\"extable-\u003einsn doesn't fit into 32-bit\\n\");\n2539:\t\t\treturn -EFAULT;\n2540:\t\t}\n2541:\t\t/* switch ex to rw buffer for writes */\n2542:\t\tex = (void *)rw_image + ((void *)ex - (void *)image);\n2543:\t\tex-\u003einsn = delta;\n2544:\t\tex-\u003efixup = target - ret_addr;\n2545:\t\tex-\u003edata = EX_TYPE_BPF;\n2546:\t\treturn 0;\n2547:\t}\n2548:\t\n2549:\tstatic u64 bpf_unwind_target(u64 ret_addr)\n2550:\t{\n2551:\t\tconst struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);\n2552:\t\n2553:\t\t/* The verifier made sure that unwind_goto follows the call */\n2554:\t\tif (WARN_ON_ONCE(!e))\n2555:\t\t\treturn ret_addr;\n2556:\t\treturn ret_addr + e-\u003efixup;\n2557:\t}\n2558:\t\n2559:\t/* unwind_exit: fix the return address of this frame. r0 and r2 are returned. */\n2560:\tstatic int emit_unwind_ret_addr(u8 **pprog, u8 *ip)\n2561:\t{\n2562:\t\tu8 *prog = *pprog, *start = prog;\n2563:\t\n2564:\t\tEMIT2(0x50, 0x56);\t\t\t/* push rax; push rsi */\n2565:\t\tEMIT4(0x48, 0x8B, 0x7D, 0x08);\t\t/* mov rdi, [rbp + 8] */\n2566:\t\tip += prog - start;\n2567:\t\tip += x86_call_depth_emit_accounting(\u0026prog, bpf_unwind_target, ip);\n2568:\t\tif (emit_call(\u0026prog, bpf_unwind_target, ip))\n2569:\t\t\treturn -EINVAL;\n2570:\t\tEMIT4(0x48, 0x89, 0x45, 0x08);\t\t/* mov [rbp + 8], rax */\n2571:\t\tEMIT2(0x5E, 0x58);\t\t\t/* pop rsi; pop rax */\n2572:\t\t*pprog = prog;\n2573:\t\treturn 0;\n2574:\t}\n2575:\t\n2576:\tstatic int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,\n2577:\t\t\t  u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)\n2578:\t{\n2579:\t\tbool tail_call_reachable = bpf_prog-\u003eaux-\u003etail_call_reachable;\n2580:\t\tstruct bpf_insn *insn = bpf_prog-\u003einsnsi;\n2581:\t\tbool callee_regs_used[4] = {};\n2582:\t\tint insn_cnt = bpf_prog-\u003elen;\n2583:\t\tbool seen_exit = false;\n2584:\t\tu8 temp[BPF_MAX_INSN_SIZE + BPF_INSN_SAFETY];\n2585:\t\tvoid __percpu *priv_frame_ptr = NULL;\n2586:\t\tu16 out_stack_arg_cnt, outgoing_rsp;\n2587:\t\tu64 arena_vm_start = 0, user_vm_start = 0;\n2588:\t\tvoid __percpu *priv_stack_ptr;\n2589:\t\tint i, excnt = 0;\n"
}

7/2 2026/10/20 09:15 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 39888
							output: 37
							thoughts: 545
						

Thoughts:
**BPF Unwinding and Potential Fuzzing Targets**

Okay, let's dive into this `bpf_unwind_target` function. I see it's right there in `arch/x86/net/bpf_jit_comp.c`. The key is how it's *called*: `emit_call(&prog, bpf_unwind_target, ip);`. That's a *direct* function address being passed, a real kernel function that JIT-compiled BPF code will call during execution. So, when the JIT'd program hits an `unwind_exit`, it emits a call to `bpf_unwind_target`.

Now, looking at the code for `bpf_unwind_target` itself:

```c
static u64 bpf_unwind_target(u64 ret_addr)
{
	const struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);

	/* The verifier made sure that unwind_goto follows the call */
	if (WARN_ON_ONCE(!e))
		return ret_addr;
	return ret_addr + e->fixup;
}
```

...that `WARN_ON_ONCE(!e)` line jumps out.  Precisely the sort of thing they tell us to be wary of: "NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED"!  This is a runtime check within a function *directly* called during the execution of JIT'd BPF code that has `unwind_exit`.  This makes it a prime candidate for fuzzing. We need to verify that these invariants hold under all possible execution paths.

Let's quickly check whether `bpf_unwind_target` can be inlined... it is passed as a function pointer to `emit_call`, so its address is taken! It cannot be fully inlined and will exist as a function symbol.

Beyond that, I should probably also look at the other functions in `arch/x86/net/bpf_jit_comp.c`, such as `add_unwind_exentry`, `emit_unwind_ret_addr`, and `bpf_jit_supports_unwind` to see if there are other potential weak spots.

Finally, I need to understand the verifier's role here, so I'll also check out `kernel/bpf/verifier.c`.  I'll focus around line 3439, looking at `check_unwind_insns`, and the surrounding code, specifically things like `check_cond_jmp_op`, `check_func_call`, etc.  The verifier is supposed to ensure that `unwind_goto` instructions follow the call. I will try to follow the control and data flow of the unwinding logic to identify potential points of vulnerability.



8/2 2026/10/20 09:15 tool read-file
0m Args:
{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 3430,
  "LineCount": 45
}

Results:
{
  "Contents": "3430:\t\tbit = find_next_bit(env-\u003ecallx_edges, end, caller * cnt + from);\n3431:\t\treturn bit \u003c end ? bit - caller * cnt : -1;\n3432:\t}\n3433:\t\n3434:\t/*\n3435:\t * 'call foo; unwind_goto pad' goes to the pad when foo left with unwind_exit.\n3436:\t * JITs don't emit a branch for it. unwind_exit finds the pad by its return\n3437:\t * address. So unwind_goto has to follow the call and nothing may jump to it.\n3438:\t */\n3439:\tstatic int check_unwind_insns(struct bpf_verifier_env *env)\n3440:\t{\n3441:\t\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n3442:\t\tbool seen = false;\n3443:\t\tint i;\n3444:\t\n3445:\t\tfor (i = 0; i \u003c env-\u003eprog-\u003elen; i++, insn++) {\n3446:\t\t\tif (bpf_is_unwind_exit(insn)) {\n3447:\t\t\t\tbpf_find_containing_subprog(env, i)-\u003ecan_unwind = true;\n3448:\t\t\t\tseen = true;\n3449:\t\t\t}\n3450:\t\t\tif (!bpf_is_unwind_goto(insn))\n3451:\t\t\t\tcontinue;\n3452:\t\t\tseen = true;\n3453:\t\t\tif (!i || !(bpf_pseudo_call(insn - 1) || bpf_is_callx(insn - 1))) {\n3454:\t\t\t\tverbose(env, \"unwind_goto at insn %d doesn't follow a call of a subprog\\n\",\n3455:\t\t\t\t\ti);\n3456:\t\t\t\treturn -EINVAL;\n3457:\t\t\t}\n3458:\t\t\tif (bpf_is_jump_target(env, i)) {\n3459:\t\t\t\tverbose(env, \"jump to unwind_goto at insn %d\\n\", i);\n3460:\t\t\t\treturn -EINVAL;\n3461:\t\t\t}\n3462:\t\t}\n3463:\t\tif (!seen)\n3464:\t\t\treturn 0;\n3465:\t\tif (!bpf_jit_supports_unwind()) {\n3466:\t\t\tverbose(env, \"JIT does not support unwinding\\n\");\n3467:\t\t\treturn -ENOTSUPP;\n3468:\t\t}\n3469:\t\tenv-\u003eprog-\u003ejit_required = 1;\n3470:\t\tenv-\u003eprog-\u003eaux-\u003ecan_unwind = env-\u003esubprog_info[0].can_unwind;\n3471:\t\treturn 0;\n3472:\t}\n3473:\t\n3474:\t/*\n"
}

9/2 2026/10/20 09:15 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 41859
							output: 59
							thoughts: 298
						

Thoughts:
**My Analysis of `check_unwind_insns`'s Dependencies**

Okay, so `check_unwind_insns` is a pretty fundamental piece of the BPF verifier. Let's trace its usage. I know `bpf_check` calls it, which is the heart of program verification. My immediate goal is to understand its scope and the surrounding control flow.

Let's see: In `kernel/bpf/verifier.c`, it's directly called by `check_func_call`, `prepare_func_exit`, `check_cond_jmp_op`, `process_bpf_exit_full`, `check_jmp_fields`, `bpf_check_attach_target`, and, of course, `bpf_check` itself. Clearly, it's deeply interwoven into the core verification logic.

Interestingly, in `kernel/bpf/const_fold.c`, `bpf_prune_dead_branches` also seems to use it, which is related to constant folding and dead code elimination – it's crucial for optimization. In `kernel/bpf/fixups.c`, it's indirectly related to optimization too, via `bpf_opt_hard_wire_dead_code_branches` and `jit_subprogs`.

Then, we've got some related functions in `arch/x86/net/bpf_jit_comp.c`: `add_unwind_exentry`, `bpf_unwind_target`, `emit_unwind_ret_addr`, `bpf_jit_supports_unwind`, and `do_jit`. These functions are interesting because they imply a connection to the JIT compiler, likely for generating unwind information. Since BPF JIT compiles programs into machine code for speed, these functions are essential for managing stack unwinding and exception handling within the JIT'd BPF programs.

Next step: I should verify if the `BPF_JIT` config is required for these parts. I suspect it's integral to the JIT compilation flow. Now, let me check the Kconfig options, to better understand which components are optional and how they relate to `check_unwind_insns`.



10/2 2026/10/20 09:15 tool grepper
0m Args:
{
  "Expression": "config BPF_JIT",
  "PathPrefixes": [
    "init/Kconfig",
    "arch/x86/Kconfig",
    "kernel/bpf/Kconfig",
    "net/Kconfig"
  ]
}

Results:
{
  "Output": "kernel/bpf/Kconfig=31=config BPF_SYSCALL\n--\nkernel/bpf/Kconfig-45-\nkernel/bpf/Kconfig:46:config BPF_JIT\nkernel/bpf/Kconfig-47-\tbool \"Enable BPF Just In Time compiler\"\n--\nkernel/bpf/Kconfig-61-\nkernel/bpf/Kconfig:62:config BPF_JIT_ALWAYS_ON\nkernel/bpf/Kconfig-63-\tbool \"Permanently enable BPF JIT and remove BPF interpreter\"\n--\nkernel/bpf/Kconfig-72-\nkernel/bpf/Kconfig:73:config BPF_JIT_DEFAULT_ON\nkernel/bpf/Kconfig-74-\tdef_bool ARCH_WANT_DEFAULT_BPF_JIT || BPF_JIT_ALWAYS_ON\n--\nkernel/bpf/Kconfig=105=config BPF_LSM\n--\nkernel/bpf/Kconfig-116-\nkernel/bpf/Kconfig:117:config BPF_JIT_KASAN\nkernel/bpf/Kconfig-118-\tbool\n"
}

11/2 2026/10/20 09:15 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 42990
							output: 34
							thoughts: 23
						

Thoughts:
Here's a summary of my thought process, expert to expert:

**Navigating the BPF Kconfig Landscape**

Okay, so I need to dive into the kernel configuration for BPF. My initial instinct is to go straight to `kernel/bpf/Kconfig`. That's where all the fundamental BPF-related configuration options are likely defined, naturally. Let's see... specifically, the area around lines 31 to 65. That range *should* cover the core feature flags that I'm interested in. I'm guessing that's where I'll find the primary knobs for enabling and disabling various BPF subsystems or features. That should give me the starting point for whatever I'm troubleshooting. Now, to the source...



12/2 2026/10/20 09:16 tool read-file
0m Args:
{
  "File": "kernel/bpf/Kconfig",
  "FirstLine": 30,
  "LineCount": 45
}

Results:
{
  "Contents": "  30:\t\n  31:\tconfig BPF_SYSCALL\n  32:\t\tbool \"Enable bpf() system call\"\n  33:\t\tselect BPF\n  34:\t\tselect IRQ_WORK\n  35:\t\tselect NEED_TASKS_RCU\n  36:\t\tselect TASKS_TRACE_RCU\n  37:\t\tselect BINARY_PRINTF\n  38:\t\tselect NET_SOCK_MSG if NET\n  39:\t\tselect NET_XGRESS if NET\n  40:\t\tselect PAGE_POOL if NET\n  41:\t\tdefault n\n  42:\t\thelp\n  43:\t\t  Enable the bpf() system call that allows to manipulate BPF programs\n  44:\t\t  and maps via file descriptors.\n  45:\t\n  46:\tconfig BPF_JIT\n  47:\t\tbool \"Enable BPF Just In Time compiler\"\n  48:\t\tdepends on BPF\n  49:\t\tdepends on HAVE_CBPF_JIT || HAVE_EBPF_JIT\n  50:\t\tselect EXECMEM\n  51:\t\thelp\n  52:\t\t  BPF programs are normally handled by a BPF interpreter. This option\n  53:\t\t  allows the kernel to generate native code when a program is loaded\n  54:\t\t  into the kernel. This will significantly speed-up processing of BPF\n  55:\t\t  programs.\n  56:\t\n  57:\t\t  Note, an admin should enable this feature changing:\n  58:\t\t  /proc/sys/net/core/bpf_jit_enable\n  59:\t\t  /proc/sys/net/core/bpf_jit_harden   (optional)\n  60:\t\t  /proc/sys/net/core/bpf_jit_kallsyms (optional)\n  61:\t\n  62:\tconfig BPF_JIT_ALWAYS_ON\n  63:\t\tbool \"Permanently enable BPF JIT and remove BPF interpreter\"\n  64:\t\tdepends on BPF_SYSCALL \u0026\u0026 HAVE_EBPF_JIT \u0026\u0026 BPF_JIT\n  65:\t\thelp\n  66:\t\t  Enables BPF JIT and removes BPF interpreter to avoid speculative\n  67:\t\t  execution of BPF instructions by the interpreter.\n  68:\t\n  69:\t\t  When CONFIG_BPF_JIT_ALWAYS_ON is enabled, /proc/sys/net/core/bpf_jit_enable\n  70:\t\t  is permanently set to 1 and setting any other value than that will\n  71:\t\t  return failure.\n  72:\t\n  73:\tconfig BPF_JIT_DEFAULT_ON\n  74:\t\tdef_bool ARCH_WANT_DEFAULT_BPF_JIT || BPF_JIT_ALWAYS_ON\n"
}

13/2 2026/10/20 09:16 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 43900
							output: 35
							thoughts: 173
						

Thoughts:
Here's my understanding of the situation, framed as my own thought process:

**Unwinding and JIT Dependencies**

Okay, so I'm seeing that both `BPF_SYSCALL` and `BPF_JIT` are required for some unwinding functionality. This makes sense from a high-level perspective, since JIT compilation is likely needed to support complex features like unwinding in BPF.

Looking more closely at `check_unwind_insns`, I see the critical check:
```c
	if (!bpf_jit_supports_unwind()) {
		verbose(env, "JIT does not support unwinding\n");
		return -ENOTSUPP;
	}
	env->prog->jit_required = 1;
```
It's clear that if the JIT isn't enabled (`BPF_JIT` isn't set, or `bpf_jit_supports_unwind()` returns false on a specific architecture), any program relying on unwinding will *fail* verification. This is important to remember because it means that there might be implicit dependencies that we should keep in mind. We need to be careful with architectures where the JIT might not support unwind.

I need to understand how the X86 JIT is being built in relation to this requirement, so let's check out `arch/x86/net/bpf_jit_comp.c`, and more importantly, how the `arch/x86/net/Makefile` handles it.



14/2 2026/10/20 09:16 tool grepper
0m Args:
{
  "Expression": "bpf_jit_comp",
  "PathPrefixes": [
    "arch/x86/net/"
  ]
}

Results:
{
  "Output": "arch/x86/net/Makefile=6=ifeq ($(CONFIG_X86_32),y)\narch/x86/net/Makefile:7:        obj-$(CONFIG_BPF_JIT) += bpf_jit_comp32.o\narch/x86/net/Makefile-8-else\narch/x86/net/Makefile:9:        obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o bpf_timed_may_goto.o\narch/x86/net/Makefile-10-endif\n"
}

15/2 2026/10/20 09:16 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 44585
							output: 47
							thoughts: 2261
						

Thoughts:
**Focusing on BPF Unwinding for Fuzzing**

Alright, let's nail down these FocusSymbols for BPF unwinding. First, the basics: `CONFIG_BPF_SYSCALL` and `CONFIG_BPF_JIT` are givens. I'm on amd64, so `CONFIG_X86_64` is relevant. Also, since `BPF_SYSCALL` selects `CONFIG_BPF`, I'm good there.

Now, the juicy stuff. Based on the code, unwinding is tied to subprograms and extension programs, so I've looked at `bpf_prog_can_unwind`, as well as `bpf_is_unwind_exit` and `bpf_is_unwind_goto`.

My task is to identify key symbols for fuzzing. The goal is to target specific feature logic, not ubiquitous functions. I need to avoid common functions like general allocators, deallocators, or trap handlers. I need to avoid `bpf_check` as well, since that's called for every BPF program.

So, let's review the changes. I've got a list of modified or added functions.

Specifically, the following were added or modified in the patch:

1.  `bpf_unwind_target`: This one is critical; it's *specific* to this patch, non-inlined (its address is passed to `emit_call`), and executed when an unwound frame exits! I need to keep in mind, even though this is static, it *is* available.
2.  `add_unwind_exentry`: Added in this patch and constructs the exception table entry for `unwind_goto`.
3.  `emit_unwind_ret_addr`: Added in this patch and emits x86 instructions for `unwind_exit`.
4.  `bpf_jit_supports_unwind`: Added in this patch, this should be targeted for fuzzing.
5.  `check_unwind_insns`: Added to check unwind instructions.
6.  `bpf_check_attach_target`: Modified to check `tgt_can_unwind`. Should consider this for fuzzing since it is not ubiquitous.
7.  `prepare_func_exit`: Modified to handle `bpf_is_unwind_exit`.
8.  `check_cond_jmp_op`: Modified to handle `bpf_is_unwind_goto`.
9.  `check_func_call`: Modified to handle `can_unwind` and branch verification.
10. `bpf_opt_hard_wire_dead_code_branches`: Modified to handle `bpf_is_unwind_goto`.
11. `jit_subprogs`: Modified to count `bpf_is_unwind_goto` and propagate `can_unwind`.
12. `bpf_prune_dead_branches`: Modified for `BPF_OP(insn->code) == BPF_JCOND`.

Let's focus on:
-   `bpf_unwind_target`
-   `check_unwind_insns`
-   `add_unwind_exentry`
-   `emit_unwind_ret_addr`
-   `bpf_jit_supports_unwind`
-   `bpf_check_attach_target`
-   `bpf_opt_hard_wire_dead_code_branches`
-   `bpf_prune_dead_branches`

I'll keep a close eye on whether `check_unwind_insns` is inlined, and if so, I might need to choose other targets.



16/2 2026/10/20 09:16 tool grepper
0m Args:
{
  "Expression": "(static )?int check_cond_jmp_op|check_func_call",
  "PathPrefixes": [
    "kernel/bpf/verifier.c"
  ]
}

Results:
{
  "Output": "kernel/bpf/verifier.c=11488=static int check_static_func_call(struct bpf_verifier_env *env, int subprog,\n--\nkernel/bpf/verifier.c-11532-\nkernel/bpf/verifier.c:11533:static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\nkernel/bpf/verifier.c-11534-\t\t\t   int *insn_idx)\n--\nkernel/bpf/verifier.c-11669- */\nkernel/bpf/verifier.c:11670:static int check_func_callx(struct bpf_verifier_env *env, struct bpf_insn *insn,\nkernel/bpf/verifier.c-11671-\t\t\t    int *insn_idx)\n--\nkernel/bpf/verifier.c=18752=static void sync_linked_regs(struct bpf_verifier_env *env, struct bpf_verifier_state *vstate,\n--\nkernel/bpf/verifier.c-18807-\nkernel/bpf/verifier.c:18808:static int check_cond_jmp_op(struct bpf_verifier_env *env,\nkernel/bpf/verifier.c-18809-\t\t\t     struct bpf_insn *insn, int *insn_idx)\n--\nkernel/bpf/verifier.c=20215=static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)\n--\nkernel/bpf/verifier.c-20289-\t\t\tif (bpf_is_callx(insn))\nkernel/bpf/verifier.c:20290:\t\t\t\treturn check_func_callx(env, insn, \u0026env-\u003einsn_idx);\nkernel/bpf/verifier.c-20291-\t\t\tif (insn-\u003esrc_reg == BPF_PSEUDO_CALL)\nkernel/bpf/verifier.c:20292:\t\t\t\treturn check_func_call(env, insn, \u0026env-\u003einsn_idx);\nkernel/bpf/verifier.c-20293-\t\t\tif (insn-\u003esrc_reg == BPF_PSEUDO_KFUNC_CALL)\n"
}

17/2 2026/10/20 09:16 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 48302
							output: 47
							thoughts: 1050
						

Thoughts:
**Analysis of BPF Unwind Feature for Fuzzing**

Alright, let's break this down. My goal is to determine if this BPF unwind functionality is a good target for fuzzing. I'll start by tracing the code path.

First, I need to understand the key components. I see `add_unwind_exentry` and `emit_unwind_ret_addr` are static within `arch/x86/net/bpf_jit_comp.c`, which is the x86 BPF JIT compiler. `bpf_unwind_target` is also defined there, along with the global `bpf_jit_supports_unwind`. The important thing here is the interaction between the JIT compilation and the exception handling.

Next, I need to find the definition of `search_bpf_extables`. This is the core to lookup the unwind pad. Okay, it's in `kernel/bpf/core.c`. It's used by `bpf_unwind_target` to look up the exception table entry when an unwind is triggered. So, an unwind exit triggers a lookup using `search_bpf_extables`.

Now, the crucial question: can userspace trigger this? Yes! BPF programs containing `BPF_UNWIND_GOTO` and `BPF_UNWIND_EXIT` instructions, are loaded via `bpf(BPF_PROG_LOAD, ...)` in userspace. The verifier checks these unwind instructions, and if valid, the JIT compiler kicks in (`do_jit`, `add_unwind_exentry`, `emit_unwind_ret_addr`). The compiled program runs, and during execution, `bpf_unwind_target()` is called. This can lead to assertion triggers or crashes, if extables or return addresses are wrong, due to `WARN_ON_ONCE(!e)`.

This touches the core BPF subsystem, the verifier, the x86 JIT compiler, and the exception table handling. And, critically, new assertions have been added.

Given all this, the verdict is clear: `WorthFuzzing: true`. The critical path is reachable from userspace, specifically through the `bpf()` syscall in a standard QEMU environment on x86_64. Newly added or modified assertions are perfect targets for fuzzing.

Now, let's determine the necessary configurations. `BPF_SYSCALL` is essential, as it enables the `bpf()` syscall and the verifier. Also, `BPF_JIT` is required, which enables the BPF JIT compiler. The verifier uses `bpf_jit_supports_unwind()`, which returns true only if the JIT compiler is enabled. If JIT is not enabled, the weak implementation returns false, and `check_unwind_insns` returns `-ENOTSUPP`. `BPF_JIT` also depends on `BPF` and `HAVE_EBPF_JIT`.

I need to confirm there are no other hidden dependencies by grepping for specific unwind functions. I don't see any further config dependencies, so `BPF_SYSCALL` and `BPF_JIT` appear to be sufficient.



18/2 2026/10/20 09:17 tool grepper
0m Args:
{
  "Expression": "bpf_is_unwind",
  "PathPrefixes": [
    "kernel/bpf/",
    "include/",
    "arch/x86/"
  ]
}

Results:
{
  "Output": "include/linux/filter.h=207=static inline bool insn_is_mov_percpu_addr(const struct bpf_insn *insn)\n--\ninclude/linux/filter.h-256-\ninclude/linux/filter.h:257:static inline bool bpf_is_unwind_goto(const struct bpf_insn *insn)\ninclude/linux/filter.h-258-{\n--\ninclude/linux/filter.h-261-\ninclude/linux/filter.h:262:static inline bool bpf_is_unwind_exit(const struct bpf_insn *insn)\ninclude/linux/filter.h-263-{\n--\nkernel/bpf/fixups.c=657=void bpf_opt_hard_wire_dead_code_branches(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-669-\t\t/* unwind_exit of a callee that never exits needs it too */\nkernel/bpf/fixups.c:670:\t\tif (bpf_is_unwind_goto(insn) \u0026\u0026 aux_data[i + 1 + insn-\u003eoff].seen)\nkernel/bpf/fixups.c-671-\t\t\tcontinue;\n--\nkernel/bpf/fixups.c=1345=static int jit_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1473-\t\t\t\tnum_exentries++;\nkernel/bpf/fixups.c:1474:\t\t\tif (bpf_is_unwind_goto(insn))\nkernel/bpf/fixups.c-1475-\t\t\t\tnum_exentries++;\n--\nkernel/bpf/syscall.c=5067=static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,\n--\nkernel/bpf/syscall.c-5093-\t\t */\nkernel/bpf/syscall.c:5094:\t\tif (prog-\u003ejited \u0026\u0026 bpf_is_unwind_goto(\u0026insns[i]))\nkernel/bpf/syscall.c-5095-\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=3439=static int check_unwind_insns(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-3445-\tfor (i = 0; i \u003c env-\u003eprog-\u003elen; i++, insn++) {\nkernel/bpf/verifier.c:3446:\t\tif (bpf_is_unwind_exit(insn)) {\nkernel/bpf/verifier.c-3447-\t\t\tbpf_find_containing_subprog(env, i)-\u003ecan_unwind = true;\n--\nkernel/bpf/verifier.c-3449-\t\t}\nkernel/bpf/verifier.c:3450:\t\tif (!bpf_is_unwind_goto(insn))\nkernel/bpf/verifier.c-3451-\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=11533=static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-11630-\nkernel/bpf/verifier.c:11631:\t\t\tif (!bpf_is_unwind_goto(insn + 1)) {\nkernel/bpf/verifier.c-11632-\t\t\t\tverbose(env, \"no unwind_goto after the call of %s()\\n\", sub_name);\n--\nkernel/bpf/verifier.c=12088=static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)\n--\nkernel/bpf/verifier.c-12113-\tcaller = state-\u003eframe[state-\u003ecurframe - 1];\nkernel/bpf/verifier.c:12114:\tif (bpf_is_unwind_exit(\u0026env-\u003eprog-\u003einsnsi[*insn_idx])) {\nkernel/bpf/verifier.c-12115-\t\t/* the helper or the kfunc that called it doesn't expect that */\n--\nkernel/bpf/verifier.c-12119-\t\t}\nkernel/bpf/verifier.c:12120:\t\tif (!bpf_is_unwind_goto(\u0026env-\u003eprog-\u003einsnsi[callee-\u003ecallsite + 1])) {\nkernel/bpf/verifier.c-12121-\t\t\tverbose(env, \"unwind_exit, but no unwind_goto after the call at insn %d\\n\",\n--\nkernel/bpf/verifier.c=18808=static int check_cond_jmp_op(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-18828-\nkernel/bpf/verifier.c:18829:\tif (bpf_is_unwind_goto(insn)) {\nkernel/bpf/verifier.c-18830-\t\tif (!env-\u003ecur_state-\u003eunwound)\n--\nkernel/bpf/verifier.c=20024=static int process_bpf_exit_full(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-20073-\tif ((cur_frame-\u003ein_async_callback_fn || cur_frame-\u003ein_exception_callback_fn) \u0026\u0026\nkernel/bpf/verifier.c:20074:\t    bpf_is_unwind_exit(\u0026env-\u003eprog-\u003einsnsi[env-\u003einsn_idx])) {\nkernel/bpf/verifier.c-20075-\t\tverbose(env, \"callback cannot unwind\\n\");\n"
}

19/2 2026/10/20 09:17 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 51547
							output: 35
							thoughts: 650
						

Thoughts:
**Analyzing BPF Unwinding Related Code**

Okay, let's dive into this. First, I need to pinpoint the key symbols related to BPF unwinding. I'll start with `FocusSymbols`. It looks like we've got:

*   `bpf_unwind_target`: This seems to be the runtime function called by the JITted code to handle `unwind_exit`. I see a `WARN_ON_ONCE` in there, which is interesting.
*   `check_unwind_insns`: This is the verification function that validates `unwind_goto` and `unwind_exit` instructions. That makes sense.
*   `add_unwind_exentry`: This generates exception table entries specifically for `unwind_goto`.
*   `emit_unwind_ret_addr`: This is responsible for generating the actual machine code to handle the `unwind_exit` return address rewrite in the JIT.
*   `bpf_jit_supports_unwind`: This checks whether JIT unwinding is even supported in the first place, found in both x86 and core BPF code.
*   `bpf_check_attach_target`: This is more about validating unwind constraints for tracing or extension programs.

Now, a quick sanity check to see if any of these are more general. Nope, they all look pretty focused on the BPF unwinding feature itself, including related attach target checks. They aren't generic system components like allocators, locking primitives, or syscall handlers.

Next, I need to check the relevant configuration options. `EnableConfigs` gives me a starting point: `BPF_SYSCALL` and `BPF_JIT`. I wonder if there's any dependency on, say, the `NET` config? `BPF_SYSCALL` seems independent, but the BPF JIT code on x86 is in `arch/x86/net/bpf_jit_comp.c`. That's a strong hint. The JIT code is conditioned on `CONFIG_BPF_JIT`.

Let me confirm the relationship. I'll look at `arch/x86/net/Makefile`:

```makefile
obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o bpf_timed_may_goto.o
```

Ah, so it's directly included in the `arch/x86/net/` build based on that config. But how does that whole directory get built? I should peek into `arch/x86/Makefile` to understand the bigger picture.



20/2 2026/10/20 09:17 tool grepper
0m Error:
no matches

Args:
{
  "Expression": "arch/x86/net",
  "PathPrefixes": [
    "arch/x86/Makefile"
  ]
}

Results:
{
  "error": "no matches"
}

21/2 2026/10/20 09:17 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 52812
							output: 30
							thoughts: 18
						

Thoughts:
Okay, let's dive into this. Here's my summary, presented as my own thought process, given my expertise:

**Examining Kbuild Inclusion of arch/x86/net**

Alright, time to get to the core of the matter. I need to understand how the `arch/x86/net` directory is incorporated into the Kbuild system for this architecture. This isn't just a casual glance; I need to understand the *mechanics* of its inclusion. I'm thinking about the implications for build configurations, dependencies, and any potential platform-specific optimizations that might be in play. I'll be looking for specific Kconfig symbols, makefile directives (like `obj-y`, `obj-m`, `subdir-y`), and any conditional inclusions that determine whether the code within `arch/x86/net` is built and linked into the kernel image. The goal is to build an accurate picture of the build process.



22/2 2026/10/20 09:17 tool grepper
0m Error:
no matches

Args:
{
  "Expression": "net/",
  "PathPrefixes": [
    "arch/x86/Makefile"
  ]
}

Results:
{
  "error": "no matches"
}

23/2 2026/10/20 09:17 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 53077
							output: 28
							thoughts: 14
						

Thoughts:
Okay, here's my take on those thoughts, tailored for an expert audience and presented in the first person:

**Navigating the X86 Kernel Build System**

Alright, time to get to work. First things first, let's dive into the core of this build process, and, from my experience, the place to start is always the `arch/x86/Kbuild` file. I know the x86 architecture like the back of my hand, and Kbuild is the beating heart of how we configure and build for it. Understanding this file is key to understanding the whole build process in this particular context. This is where the magic happens and where I'm going to find the rules and dependencies that will steer me right. It's the starting point for me to trace how things are put together.



24/2 2026/10/20 09:17 tool grepper
0m Args:
{
  "Expression": "net",
  "PathPrefixes": [
    "arch/x86/"
  ]
}

Results:
{
  "Output": "\nFull output is too long, showing 200 out of 530 lines.\nUse more precise expression or PathPrefixes if possible.\n\nNumber of matching lines per file (76 files in total):\narch/x86/Kbuild:1\narch/x86/Kconfig:4\narch/x86/Kconfig.cpu:1\narch/x86/crypto/blowfish-x86_64-asm_64.S:1\narch/x86/crypto/blowfish_glue.c:1\narch/x86/crypto/camellia-aesni-avx-asm_64.S:1\narch/x86/crypto/camellia-x86_64-asm_64.S:1\narch/x86/crypto/camellia_aesni_avx2_glue.c:1\narch/x86/crypto/camellia_glue.c:1\narch/x86/crypto/cast5-avx-x86_64-asm_64.S:1\narch/x86/crypto/glue_helper-asm-avx2.S:1\narch/x86/crypto/serpent-avx2-asm_64.S:1\narch/x86/crypto/serpent-sse2-i586-asm_32.S:1\narch/x86/crypto/serpent-sse2-x86_64-asm_64.S:1\narch/x86/crypto/serpent_avx2_glue.c:1\narch/x86/crypto/serpent_sse2_glue.c:1\narch/x86/crypto/twofish-i586-asm_32.S:1\narch/x86/crypto/twofish-x86_64-asm_64-3way.S:1\narch/x86/crypto/twofish-x86_64-asm_64.S:1\narch/x86/crypto/twofish_glue.c:1\narch/x86/crypto/twofish_glue_3way.c:1\narch/x86/entry/entry_64.S:1\narch/x86/entry/vdso/common/vclock_gettime.c:1\narch/x86/entry/vdso/vdso32/system_call.S:1\narch/x86/entry/vsyscall/vsyscall_64.c:1\narch/x86/include/asm/apm.h:1\narch/x86/include/asm/checksum_32.h:1\narch/x86/include/asm/checksum_64.h:3\narch/x86/include/asm/cpuid/leaf_types.h:2\narch/x86/include/asm/mach_timer.h:1\narch/x86/include/asm/mach_traps.h:1\narch/x86/include/asm/mpspec.h:1\narch/x86/include/asm/processor.h:1\narch/x86/include/asm/uv/uv_hub.h:1\narch/x86/include/asm/vdso/gettimeofday.h:1\narch/x86/include/asm/vmx.h:3\narch/x86/include/uapi/asm/bootparam.h:1\narch/x86/include/uapi/asm/vmx.h:3\narch/x86/kernel/amd_gart_64.c:1\narch/x86/kernel/apic/apic.c:2\narch/x86/kernel/apm_32.c:3\narch/x86/kernel/cpu/intel.c:1\narch/x86/kernel/cpu/mce/dev-mcelog.c:1\narch/x86/kernel/cpu/resctrl/pseudo_lock.c:1\narch/x86/kernel/hw_breakpoint.c:1\narch/x86/kernel/traps.c:1\narch/x86/kvm/emulate.c:3\narch/x86/kvm/hyperv.c:5\narch/x86/kvm/hyperv.h:5\narch/x86/kvm/lapic.c:2\n... and 26 more files\n\narch/x86/Kbuild=32=obj-y += platform/\narch/x86/Kbuild:33:obj-y += net/\narch/x86/Kbuild-34-\n--\narch/x86/Kconfig=633=config X86_INTEL_MID\n--\narch/x86/Kconfig-644-\t  Select to build a kernel capable of supporting 64-bit Intel MID\narch/x86/Kconfig:645:\t  (Mobile Internet Device) platform systems which do not have\narch/x86/Kconfig-646-\t  the PCI legacy interfaces.\n--\narch/x86/Kconfig=1885=config EFI\n--\narch/x86/Kconfig-1897-\t  In addition, you should use the latest ELILO loader available\narch/x86/Kconfig:1898:\t  at \u003chttp://elilo.sourceforge.net\u003e in order to take advantage\narch/x86/Kconfig-1899-\t  of EFI runtime services. However, even with this option, the\n--\narch/x86/Kconfig=3048=config NET5501\narch/x86/Kconfig:3049:\tbool \"Soekris Engineering net5501 System Support (LEDS, GPIO, etc)\"\narch/x86/Kconfig-3050-\tdepends on GPIO_CS5535=y\n--\narch/x86/Kconfig-3052-\thelp\narch/x86/Kconfig:3053:\t  This option enables system support for the Soekris Engineering net5501.\narch/x86/Kconfig-3054-\n--\narch/x86/Kconfig.cpu=197=config CC_HAS_MARCH_NATIVE\n--\narch/x86/Kconfig.cpu-200-\t# LLVM 18 has an easily triggered internal compiler error in core\narch/x86/Kconfig.cpu:201:\t# networking code with '-march=native' on certain systems:\narch/x86/Kconfig.cpu-202-\t# https://github.com/llvm/llvm-project/issues/72026\n--\narch/x86/crypto/blowfish-x86_64-asm_64.S-4- *\narch/x86/crypto/blowfish-x86_64-asm_64.S:5: * Copyright (C) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/blowfish-x86_64-asm_64.S-6- */\n--\narch/x86/crypto/blowfish_glue.c-4- *\narch/x86/crypto/blowfish_glue.c:5: * Copyright (c) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/blowfish_glue.c-6- *\n--\narch/x86/crypto/camellia-aesni-avx-asm_64.S-14- * Version licensed under 2-clause BSD License is available at:\narch/x86/crypto/camellia-aesni-avx-asm_64.S:15: *\thttp://koti.mbnet.fi/axh/crypto/camellia-BSD-1.2.0-aesni1.tar.xz\narch/x86/crypto/camellia-aesni-avx-asm_64.S-16- */\n--\narch/x86/crypto/camellia-x86_64-asm_64.S-4- *\narch/x86/crypto/camellia-x86_64-asm_64.S:5: * Copyright (C) 2012 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/camellia-x86_64-asm_64.S-6- */\n--\narch/x86/crypto/camellia_aesni_avx2_glue.c-4- *\narch/x86/crypto/camellia_aesni_avx2_glue.c:5: * Copyright © 2013 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/camellia_aesni_avx2_glue.c-6- */\n--\narch/x86/crypto/camellia_glue.c-4- *\narch/x86/crypto/camellia_glue.c:5: * Copyright (c) 2012 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/camellia_glue.c-6- *\n--\narch/x86/crypto/cast5-avx-x86_64-asm_64.S-7- *\narch/x86/crypto/cast5-avx-x86_64-asm_64.S:8: * Copyright © 2012 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/cast5-avx-x86_64-asm_64.S-9- */\n--\narch/x86/crypto/glue_helper-asm-avx2.S-4- *\narch/x86/crypto/glue_helper-asm-avx2.S:5: * Copyright © 2012-2013 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/glue_helper-asm-avx2.S-6- */\n--\narch/x86/crypto/serpent-avx2-asm_64.S-4- *\narch/x86/crypto/serpent-avx2-asm_64.S:5: * Copyright © 2012-2013 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/serpent-avx2-asm_64.S-6- *\n--\narch/x86/crypto/serpent-sse2-i586-asm_32.S-4- *\narch/x86/crypto/serpent-sse2-i586-asm_32.S:5: * Copyright (C) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/serpent-sse2-i586-asm_32.S-6- *\n--\narch/x86/crypto/serpent-sse2-x86_64-asm_64.S-4- *\narch/x86/crypto/serpent-sse2-x86_64-asm_64.S:5: * Copyright (C) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/serpent-sse2-x86_64-asm_64.S-6- *\n--\narch/x86/crypto/serpent_avx2_glue.c-4- *\narch/x86/crypto/serpent_avx2_glue.c:5: * Copyright © 2012-2013 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/serpent_avx2_glue.c-6- */\n--\narch/x86/crypto/serpent_sse2_glue.c-4- *\narch/x86/crypto/serpent_sse2_glue.c:5: * Copyright (c) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/serpent_sse2_glue.c-6- *\n--\narch/x86/crypto/twofish-i586-asm_32.S-2-/***************************************************************************\narch/x86/crypto/twofish-i586-asm_32.S:3:*   Copyright (C) 2006 by Joachim Fritschi, \u003cjfritschi@freenet.de\u003e        *\narch/x86/crypto/twofish-i586-asm_32.S-4-*                                                                         *\n--\narch/x86/crypto/twofish-x86_64-asm_64-3way.S-4- *\narch/x86/crypto/twofish-x86_64-asm_64-3way.S:5: * Copyright (C) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/twofish-x86_64-asm_64-3way.S-6- */\n--\narch/x86/crypto/twofish-x86_64-asm_64.S-2-/***************************************************************************\narch/x86/crypto/twofish-x86_64-asm_64.S:3:*   Copyright (C) 2006 by Joachim Fritschi, \u003cjfritschi@freenet.de\u003e        *\narch/x86/crypto/twofish-x86_64-asm_64.S-4-*                                                                         *\n--\narch/x86/crypto/twofish_glue.c-8- * Ported to the kerneli patch by Marc Mutz \u003cMarc@Mutz.com\u003e\narch/x86/crypto/twofish_glue.c:9: * Ported to CryptoAPI by Colin Slater \u003choho@tacomeat.net\u003e\narch/x86/crypto/twofish_glue.c-10- *\n--\narch/x86/crypto/twofish_glue_3way.c-4- *\narch/x86/crypto/twofish_glue_3way.c:5: * Copyright (c) 2011 Jussi Kivilinna \u003cjussi.kivilinna@mbnet.fi\u003e\narch/x86/crypto/twofish_glue_3way.c-6- */\n--\narch/x86/entry/entry_64.S=662=native_irq_return_ldt:\n--\narch/x86/entry/entry_64.S-708-\t * the same page.  Set up RSP so that RSP[31:16] contains the\narch/x86/entry/entry_64.S:709:\t * respective 16 bits of the /userspace/ RSP and RSP nonetheless\narch/x86/entry/entry_64.S-710-\t * still points to an RO alias of the ESPFIX stack.\n--\narch/x86/entry/vdso/common/vclock_gettime.c-7- *\narch/x86/entry/vdso/common/vclock_gettime.c:8: * 32 Bit compat layer by Stefani Seibold \u003cstefani@seibold.net\u003e\narch/x86/entry/vdso/common/vclock_gettime.c-9- *  sponsored by Rohde \u0026 Schwarz GmbH \u0026 Co. KG Munich/Germany\n--\narch/x86/entry/vdso/vdso32/system_call.S=15=__kernel_vsyscall:\n--\narch/x86/entry/vdso/vdso32/system_call.S-40-\t * sequence.  That is severely broken for a number of reasons (ask\narch/x86/entry/vdso/vdso32/system_call.S:41:\t * anyone with an AMD CPU, for example).  Nonetheless, we try to keep\narch/x86/entry/vdso/vdso32/system_call.S-42-\t * it working approximately as well as it ever worked.\n--\narch/x86/entry/vsyscall/vsyscall_64.c-2-/*\narch/x86/entry/vsyscall/vsyscall_64.c:3: * Copyright (c) 2012-2014 Andy Lutomirski \u003cluto@amacapital.net\u003e\narch/x86/entry/vsyscall/vsyscall_64.c-4- *\n--\narch/x86/include/asm/apm.h-3- *  Machine specific APM BIOS functions for generic.\narch/x86/include/asm/apm.h:4: *  Split out from apm.c by Osamu Tomita \u003ctomita@cinet.co.jp\u003e\narch/x86/include/asm/apm.h-5- */\n--\narch/x86/include/asm/checksum_32.h=44=static inline __wsum csum_and_copy_from_user(const void __user *src,\n--\narch/x86/include/asm/checksum_32.h-61- *\narch/x86/include/asm/checksum_32.h:62: *\tBy Jorge Cwik \u003cjorge@laser.satlink.net\u003e, adapted for linux by\narch/x86/include/asm/checksum_32.h-63- *\tArnt Gulbrandsen.\n--\narch/x86/include/asm/checksum_64.h=25=static inline __sum16 csum_fold(__wsum sum)\n--\narch/x86/include/asm/checksum_64.h-38- *\narch/x86/include/asm/checksum_64.h:39: *\tBy Jorge Cwik \u003cjorge@laser.satlink.net\u003e, adapted for linux by\narch/x86/include/asm/checksum_64.h-40- *\tArnt Gulbrandsen.\n--\narch/x86/include/asm/checksum_64.h=117=static inline __sum16 csum_tcpudp_magic(__be32 saddr, __be32 daddr,\n--\narch/x86/include/asm/checksum_64.h-124-/**\narch/x86/include/asm/checksum_64.h:125: * csum_partial - Compute an internet checksum.\narch/x86/include/asm/checksum_64.h-126- * @buff: buffer to be checksummed\n--\narch/x86/include/asm/checksum_64.h-129- *\narch/x86/include/asm/checksum_64.h:130: * Returns: the 32bit unfolded internet checksum of the buffer.\narch/x86/include/asm/checksum_64.h-131- * Before filling it in it needs to be csum_fold()'ed.\n--\narch/x86/include/asm/cpuid/leaf_types.h=265=struct leaf_0x7_0 {\n--\narch/x86/include/asm/cpuid/leaf_types.h-312-\t\tvpclmulqdq\t\t\t:  1, // VPCLMULQDQ 256-bit instruction\narch/x86/include/asm/cpuid/leaf_types.h:313:\t\tavx512_vnni\t\t\t:  1, // Vector neural network instructions\narch/x86/include/asm/cpuid/leaf_types.h-314-\t\tavx512_bitalg\t\t\t:  1, // AVX-512 bitwise algorithms\n--\narch/x86/include/asm/cpuid/leaf_types.h-332-\t\tsgx_keys\t\t\t:  1, // Intel SGX attestation services\narch/x86/include/asm/cpuid/leaf_types.h:333:\t\tavx512_4vnniw\t\t\t:  1, // AVX-512 neural network instructions\narch/x86/include/asm/cpuid/leaf_types.h-334-\t\tavx512_4fmaps\t\t\t:  1, // AVX-512 multiply accumulation single precision\n--\narch/x86/include/asm/mach_timer.h-3- *  Machine specific calibrate_tsc() for generic.\narch/x86/include/asm/mach_timer.h:4: *  Split out from timer_tsc.c by Osamu Tomita \u003ctomita@cinet.co.jp\u003e\narch/x86/include/asm/mach_timer.h-5- */\n--\narch/x86/include/asm/mach_traps.h-3- *  Machine specific NMI handling for generic.\narch/x86/include/asm/mach_traps.h:4: *  Split out from traps.c by Osamu Tomita \u003ctomita@cinet.co.jp\u003e\narch/x86/include/asm/mach_traps.h-5- */\n--\narch/x86/include/asm/mpspec.h=11=extern int pic_mode;\n--\narch/x86/include/asm/mpspec.h-16- * Summit or generic (i.e. installer) kernels need lots of bus entries.\narch/x86/include/asm/mpspec.h:17: * Maximum 256 PCI busses, plus 1 ISA bus in each of 4 cabinets.\narch/x86/include/asm/mpspec.h-18- */\n--\narch/x86/include/asm/processor.h=11=struct vm86;\n--\narch/x86/include/asm/processor.h-43- *\narch/x86/include/asm/processor.h:44: * Based on this we disable the IP header alignment in network drivers.\narch/x86/include/asm/processor.h-45- */\n--\narch/x86/include/asm/uv/uv_hub.h-40- *\narch/x86/include/asm/uv/uv_hub.h:41: *\tNASID   - network ID of a router, Mbrick or Cbrick. Nasid values of\narch/x86/include/asm/uv/uv_hub.h-42- *\t\t  routers always have low bit of 1, C/MBricks have low bit\n--\narch/x86/include/asm/vdso/gettimeofday.h-6- * Copyright 2006 Andi Kleen, SUSE Labs.\narch/x86/include/asm/vdso/gettimeofday.h:7: * 32 Bit compat layer by Stefani Seibold \u003cstefani@seibold.net\u003e\narch/x86/include/asm/vdso/gettimeofday.h-8- *  sponsored by Rohde \u0026 Schwarz GmbH \u0026 Co. KG Munich/Germany\n--\narch/x86/include/asm/vmx.h-6- * A few random additions are:\narch/x86/include/asm/vmx.h:7: * Copyright (C) 2006 Qumranet\narch/x86/include/asm/vmx.h:8: *    Avi Kivity \u003cavi@qumranet.com\u003e\narch/x86/include/asm/vmx.h:9: *    Yaniv Kamay \u003cyaniv@qumranet.com\u003e\narch/x86/include/asm/vmx.h-10- */\n--\narch/x86/include/uapi/asm/bootparam.h=116=struct boot_params {\n--\narch/x86/include/uapi/asm/bootparam.h-197- * \tcurrently supported through this PV boot path.\narch/x86/include/uapi/asm/bootparam.h:198: * @X86_SUBARCH_INTEL_MID: Used for Intel MID (Mobile Internet Device) platform\narch/x86/include/uapi/asm/bootparam.h-199- *\tsystems which do not have the PCI legacy interfaces.\n--\narch/x86/include/uapi/asm/vmx.h-19- * A few random additions are:\narch/x86/include/uapi/asm/vmx.h:20: * Copyright (C) 2006 Qumranet\n\n"
}

25/2 2026/10/20 09:17 llm patch-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 59352
							output: 253
							thoughts: 673
						

Thoughts:
**Analyzing BPF Unwind Support for Fuzzing**

Okay, so I'm looking at this patch and tracing the dependencies. `arch/x86/Kbuild` includes `net/`, and specifically, `arch/x86/net/Makefile` reveals that `bpf_jit_comp.o` depends on `CONFIG_BPF_JIT`. This means `arch/x86/net/` is always traversed, and that the BPF JIT compilation is relevant. So, definitely need to make sure `BPF_SYSCALL` and `BPF_JIT` are set.

Now, let's see what files were modified to get a better scope. I see changes in `arch/x86/net/bpf_jit_comp.c`, specifically functions like `add_unwind_exentry`, `bpf_unwind_target`, and related functions related to unwinding, but also related modifications in the verifier (`kernel/bpf/verifier.c`), fixups (`kernel/bpf/fixups.c`) and constant folding (`kernel/bpf/const_fold.c`).

I need to make sure I'm targeting the right symbols. The goal is to identify unique and important features that are related to the unwind functionality and avoid the obvious core functions like `bpf_check`. So, I've selected `bpf_unwind_target`, `check_unwind_insns`, `add_unwind_exentry`, `emit_unwind_ret_addr`, `bpf_jit_supports_unwind`, and `bpf_check_attach_target` as the key ones to focus on. These are specifically tied to the new unwinding support.

The reasoning here is that the patch introduces support for BPF unwinding with `BPF_UNWIND_GOTO` and `BPF_UNWIND_EXIT`, which modifies the verifier, fixups, constant folding, and the JIT compilation. This includes exception handling and runtime target resolution. Given that userspace can trigger these through the `bpf()` syscall in virtualized x86_64 environments, this *screams* for fuzzing.

Alright, let's call the `set-results` tool now.



26/2 2026/10/20 09:18 tool set-results
0m Args:
{
  "EnableConfigs": [
    "BPF_SYSCALL",
    "BPF_JIT"
  ],
  "FocusSymbols": [
    "bpf_unwind_target",
    "check_unwind_insns",
    "add_unwind_exentry",
    "emit_unwind_ret_addr",
    "bpf_jit_supports_unwind",
    "bpf_check_attach_target"
  ],
  "Reasoning": "The patch introduces support for BPF unwinding (BPF_UNWIND_GOTO and BPF_UNWIND_EXIT) in the BPF core verifier and the x86 JIT compiler. It adds new verification logic for unwind instructions, generates JIT code and exception table entries for unwind paths, and introduces a runtime unwind resolution function (bpf_unwind_target) containing runtime assertions (WARN_ON_ONCE). This code is fully reachable in standard QEMU amd64 environments via the bpf() syscall (e.g. BPF_PROG_LOAD and BPF_PROG_TEST_RUN), making it prime for fuzzing.",
  "WorthFuzzing": true
}

Results:
{
  "EnableConfigs": [
    "BPF_SYSCALL",
    "BPF_JIT"
  ],
  "FocusSymbols": [
    "bpf_unwind_target",
    "check_unwind_insns",
    "add_unwind_exentry",
    "emit_unwind_ret_addr",
    "bpf_jit_supports_unwind",
    "bpf_check_attach_target"
  ],
  "Reasoning": "The patch introduces support for BPF unwinding (BPF_UNWIND_GOTO and BPF_UNWIND_EXIT) in the BPF core verifier and the x86 JIT compiler. It adds new verification logic for unwind instructions, generates JIT code and exception table entries for unwind paths, and introduces a runtime unwind resolution function (bpf_unwind_target) containing runtime assertions (WARN_ON_ONCE). This code is fully reachable in standard QEMU amd64 environments via the bpf() syscall (e.g. BPF_PROG_LOAD and BPF_PROG_TEST_RUN), making it prime for fuzzing.",
  "WorthFuzzing": true
}

27/1 2026/10/20 09:18 action If
2m Args:
{
  "WorthFuzzing": true
}

Results:
null

28/2 2026/10/20 09:18 agent kmsan-evaluator
2m Model:
core

Args:
null

Results:
{
  "KMSANReasoning": "The patch series introduces support for BPF exception unwinding (`unwind_goto` and `unwind_exit`) across the BPF verifier, core runtime, and JIT compilers (x86_64 and arm64).\n\n1. Kernel-to-user info-leak assessment:\nThe only user-facing data transfer touched is in `bpf_insn_prepare_dump()`, where instructions are dumped to user space via `bpf_prog_get_info_by_fd()`. The instruction buffer is allocated using `kmemdup` from the existing `prog-\u003einsnsi` stream, and the patch merely adjusts handling of `BPF_JCOND` so that `unwind_goto` instructions are preserved rather than mistaken for `BPF_CALL_ARGS`. No uninitialized memory, padding, or uninitialized fields are exposed to user space.\n\n2. Uninitialized memory reads / branching:\n- In the JIT compilers (e.g. `add_unwind_exentry()`), all fields of `struct exception_table_entry` (`insn`, `fixup`, `data`) are fully initialized on creation.\n- In the verifier and runtime data structures, new fields such as `unwound` in `struct bpf_verifier_state` and `can_unwind` in `struct bpf_subprog_info` and `struct bpf_prog_aux` are allocated via `kzalloc`/`kzalloc_obj`, properly copied in `bpf_copy_verifier_state()`, and evaluated in `states_equal()`.\n- All newly introduced functions (`add_unwind_exentry()`, `bpf_unwind_target()`, `emit_unwind_ret_addr()`, `check_unwind_insns()`) explicitly initialize their local variables and arguments.\n\n3. KASAN vs KMSAN applicability:\nThe primary risks associated with this patch series involve JIT code generation correctness, return address manipulation on the stack frame, out-of-bounds extable indexing, or verifier branch simulation logic. These issues manifest as out-of-bounds accesses, stack corruption, or kernel panics/crashes, which are covered by KASAN and standard kernel debug facilities. There are no uninitialized memory read risks or info-leaks that specifically necessitate KMSAN fuzzing.",
  "NeedsKMSAN": false
}

Instruction:
You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.

CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
  a comprehensive suite of debugging tools and sanitizers, including KASAN
  (out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
  (locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
  or page allocations) and kernel-to-user memory info-leaks.

Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.

Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.

Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
   attributes, ioctl output arguments, socket options, or BPF buffers) where fields
   or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
   or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
   could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
   uninitialized bytes of existing buffers.

Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
  dereferences, locking deadlocks, or use-after-free bugs (these are already caught
  by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
  or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.

Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.


Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.

Prompt:
Target architecture: amd64

For your convenience, here is the diff of the changes:
commit 880657c9fd431b4343c8c6dcc4398fc89e69a930
Author: syz-cluster <triage@syzkaller.com>
Date:   Tue Oct 20 09:14:59 2026 +0000

    syz-cluster: applied patch under review

diff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h
index b2fe6e6dcf447..50029dd9baa63 100644
--- a/arch/arm64/net/bpf_jit.h
+++ b/arch/arm64/net/bpf_jit.h
@@ -316,6 +316,8 @@
 
 #define A64_PACIASP A64_HINT(AARCH64_INSN_HINT_PACIASP)
 #define A64_AUTIASP A64_HINT(AARCH64_INSN_HINT_AUTIASP)
+#define A64_PACIA1716 A64_HINT(AARCH64_INSN_HINT_PACIA_1716)
+#define A64_AUTIA1716 A64_HINT(AARCH64_INSN_HINT_AUTIA_1716)
 
 /* BTI */
 #define A64_BTI_C  A64_HINT(AARCH64_INSN_HINT_BTIC)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 1d5f4cf88d331..45ed69b83265d 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -11,6 +11,7 @@
 #include <linux/bitfield.h>
 #include <linux/bpf.h>
 #include <linux/cfi.h>
+#include <linux/extable.h>
 #include <linux/filter.h>
 #include <linux/memory.h>
 #include <linux/printk.h>
@@ -1221,6 +1222,85 @@ static int add_exception_handler(const struct bpf_insn *insn,
 	return 0;
 }
 
+/*
+ * There is no branch for unwind_goto, only an extable entry
+ *   { .insn = ret_addr - 1, .fixup = pad - ret_addr }
+ * where ret_addr is the return address of the call in front of it. unwind_exit
+ * replaces LR in its frame record with bpf_unwind_target() of it.
+ * ret_addr - 1, because what follows the call can have an entry of its own.
+ * No insn starts there, so a fault cannot hit the entry.
+ * 'bl foo; mov x8, x0' is in front of unwind_goto. The pad doesn't miss the mov:
+ * r0 is x8 and the epilogue of the callee copies it to x0 and leaves x8 as it is.
+ */
+static int add_unwind_exentry(struct jit_ctx *ctx, int pad)
+{
+	int ret_addr = ctx->idx - 1;
+	struct exception_table_entry *ex;
+	off_t ins_offset;
+
+	if (!ctx->image)
+		/* First pass */
+		return 0;
+
+	if (!ctx->prog->aux->extable ||
+	    WARN_ON_ONCE(ctx->exentry_idx >= ctx->prog->aux->num_exentries))
+		return -EINVAL;
+
+	ex = &ctx->prog->aux->extable[ctx->exentry_idx];
+	ins_offset = (long)&ctx->ro_image[ret_addr] - 1 - (long)&ex->insn;
+	if (WARN_ON_ONCE(ins_offset >= 0 || ins_offset < INT_MIN))
+		return -ERANGE;
+
+	/* switch ex to rw buffer for writing */
+	ex = (void *)ctx->image + ((void *)ex - (void *)ctx->ro_image);
+	ex->insn = ins_offset;
+	ex->fixup = (ctx->offset[pad] - ret_addr) * AARCH64_INSN_SIZE;
+	ex->type = EX_TYPE_BPF;
+
+	ctx->exentry_idx++;
+	return 0;
+}
+
+static u64 bpf_unwind_target(u64 ret_addr)
+{
+	const struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);
+
+	/* The verifier made sure that unwind_goto follows the call */
+	if (WARN_ON_ONCE(!e))
+		return ret_addr;
+	return ret_addr + e->fixup;
+}
+
+/*
+ * unwind_exit: fix LR in the frame record. It's signed with SP at the entry of
+ * the prog, which is FP + 16. The x16/x17 forms are hints like PACIASP: nops
+ * without pointer auth. r0 and r2 are returned.
+ */
+static void emit_unwind_ret_addr(struct jit_ctx *ctx)
+{
+	const bool ptr_auth = IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL);
+	const u8 r0 = bpf2a64[BPF_REG_0];
+	const u8 r2 = bpf2a64[BPF_REG_2];
+
+	emit(A64_PUSH(r0, r2, A64_SP), ctx);
+
+	emit(A64_LDR64I(A64_R(17), A64_FP, 8), ctx);
+	if (ptr_auth) {
+		emit(A64_ADD_I(1, A64_R(16), A64_FP, 16), ctx);
+		emit(A64_AUTIA1716, ctx);
+	}
+	emit(A64_MOV(1, A64_R(0), A64_R(17)), ctx);
+	emit_call((u64)bpf_unwind_target, ctx);
+	emit(A64_MOV(1, A64_R(17), A64_R(0)), ctx);
+	if (ptr_auth) {
+		emit(A64_ADD_I(1, A64_R(16), A64_FP, 16), ctx);
+		emit(A64_PACIA1716, ctx);
+	}
+	emit(A64_STR64I(A64_R(17), A64_FP, 8), ctx);
+
+	emit(A64_POP(r0, r2, A64_SP), ctx);
+}
+
 static const struct bpf_jit_arg_abi arm64_arg_abi = {
 	.nr_arg_regs		= 8,
 	.even_reg_align		= true,
@@ -1806,8 +1886,16 @@ static int build_insn(const struct bpf_verifier_env *env, const struct bpf_insn
 		if (emit_bpf_tail_call(ctx))
 			return -EFAULT;
 		break;
+	/* unwind_goto. The verifier expanded may_goto. */
+	case BPF_JMP | BPF_JCOND:
+		ret = add_unwind_exentry(ctx, i + 1 + off);
+		if (ret)
+			return ret;
+		break;
 	/* function return */
 	case BPF_JMP | BPF_EXIT:
+		if (bpf_is_unwind_exit(insn) && bpf_prog_can_unwind(ctx->prog))
+			emit_unwind_ret_addr(ctx);
 		/* Optimization: when last instruction is EXIT,
 		   simply fallthrough to epilogue. */
 		if (i == ctx->prog->len - 1)
@@ -2466,6 +2554,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+bool bpf_jit_supports_unwind(void)
+{
+	return true;
+}
+
 const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
 {
 	return &arm64_arg_abi;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 72a2a0a89e033..c583eed4e9549 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -15,6 +15,7 @@
 #include <linux/memory.h>
 #include <linux/sort.h>
 #include <linux/execmem.h>
+#include <linux/extable.h>
 #include <asm/extable.h>
 #include <asm/ftrace.h>
 #include <asm/set_memory.h>
@@ -2511,6 +2512,67 @@ static void adjust_cmp_imm(u32 *op, s32 *imm)
 	}
 }
 
+/*
+ * There is no branch for unwind_goto, only an extable entry
+ *   { .insn = ret_addr - 1, .fixup = target - ret_addr }
+ * where ret_addr is the return address of the call in front of it. unwind_exit
+ * replaces the return address of its frame with bpf_unwind_target() of it.
+ * ret_addr - 1, because what follows the call can have an entry of its own.
+ * No insn starts there, so a fault cannot hit the entry.
+ */
+static int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,
+			      int ret_addr, int target)
+{
+	struct exception_table_entry *ex;
+	s64 delta;
+
+	if (!prog->aux->extable)
+		return 0;
+	if (*excnt >= prog->aux->num_exentries) {
+		pr_err("unwind extable bug\n");
+		return -EFAULT;
+	}
+	ex = &prog->aux->extable[(*excnt)++];
+	delta = image + ret_addr - 1 - (u8 *)&ex->insn;
+	if (!is_simm32(delta)) {
+		pr_err("extable->insn doesn't fit into 32-bit\n");
+		return -EFAULT;
+	}
+	/* switch ex to rw buffer for writes */
+	ex = (void *)rw_image + ((void *)ex - (void *)image);
+	ex->insn = delta;
+	ex->fixup = target - ret_addr;
+	ex->data = EX_TYPE_BPF;
+	return 0;
+}
+
+static u64 bpf_unwind_target(u64 ret_addr)
+{
+	const struct exception_table_entry *e = search_bpf_extables(ret_addr - 1);
+
+	/* The verifier made sure that unwind_goto follows the call */
+	if (WARN_ON_ONCE(!e))
+		return ret_addr;
+	return ret_addr + e->fixup;
+}
+
+/* unwind_exit: fix the return address of this frame. r0 and r2 are returned. */
+static int emit_unwind_ret_addr(u8 **pprog, u8 *ip)
+{
+	u8 *prog = *pprog, *start = prog;
+
+	EMIT2(0x50, 0x56);			/* push rax; push rsi */
+	EMIT4(0x48, 0x8B, 0x7D, 0x08);		/* mov rdi, [rbp + 8] */
+	ip += prog - start;
+	ip += x86_call_depth_emit_accounting(&prog, bpf_unwind_target, ip);
+	if (emit_call(&prog, bpf_unwind_target, ip))
+		return -EINVAL;
+	EMIT4(0x48, 0x89, 0x45, 0x08);		/* mov [rbp + 8], rax */
+	EMIT2(0x5E, 0x58);			/* pop rsi; pop rax */
+	*pprog = prog;
+	return 0;
+}
+
 static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,
 		  u8 *rw_image, int oldproglen, struct jit_context *ctx, bool jmp_padding)
 {
@@ -3493,6 +3555,34 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 				pop_r9(&prog);
 			break;
 
+			/* unwind_goto. The verifier expanded may_goto. */
+		case BPF_JMP | BPF_JCOND:
+			if (!priv_frame_ptr) {
+				err = add_unwind_exentry(bpf_prog, image, rw_image, &excnt,
+							 addrs[i - 1], addrs[i + insn->off]);
+				if (err)
+					return err;
+				break;
+			}
+			/*
+			 * 'pop r9' after the call has to be done on the way to
+			 * the pad too:
+			 *   call foo
+			 *   pop r9      <- exit returns here
+			 *   jmp 1f
+			 *   pop r9      <- unwind_exit returns here
+			 *   jmp pad
+			 * 1:
+			 */
+			err = add_unwind_exentry(bpf_prog, image, rw_image, &excnt,
+						 addrs[i - 1] - 2, addrs[i - 1] + 2);
+			if (err)
+				return err;
+			EMIT2(0xEB, 7);
+			pop_r9(&prog);
+			EMIT1_off32(0xE9, addrs[i + insn->off] - (addrs[i - 1] + 9));
+			break;
+
 		case BPF_JMP | BPF_TAIL_CALL:
 			if (imm32)
 				emit_bpf_tail_call_direct(bpf_prog,
@@ -3774,13 +3864,16 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 			break;
 
 		case BPF_JMP | BPF_EXIT:
+			if (src_reg == BPF_UNWIND_EXIT && bpf_prog_can_unwind(bpf_prog) &&
+			    emit_unwind_ret_addr(&prog, ip))
+				return -EINVAL;
 			if (seen_exit) {
 				jmp_offset = ctx->cleanup_addr - addrs[i];
 				goto emit_jmp;
 			}
 			seen_exit = true;
 			/* Update cleanup_addr */
-			ctx->cleanup_addr = proglen;
+			ctx->cleanup_addr = proglen + (prog - temp);
 			if (bpf_prog_was_classic(bpf_prog) &&
 			    !ns_capable_noaudit(&init_user_ns, CAP_SYS_ADMIN)) {
 				if (emit_spectre_bhb_barrier(&prog, ip, bpf_prog))
@@ -5027,6 +5120,11 @@ bool bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return true;
 }
 
+bool bpf_jit_supports_unwind(void)
+{
+	return true;
+}
+
 const struct bpf_jit_arg_abi *bpf_jit_arg_abi(void)
 {
 	return &x86_arg_abi;
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 7750df9734942..580f89c450b55 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1897,6 +1897,7 @@ struct bpf_prog_aux {
 	bool changes_pkt_data;
 	bool has_prologue;
 	bool might_sleep;
+	bool can_unwind;
 	bool kprobe_write_ctx;
 	/* has insns that work with the arena base, see bpf_prog_has_arena_insns() */
 	bool uses_arena_insns;
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 54481da4ed2d2..6606b341351eb 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -532,6 +532,8 @@ struct bpf_verifier_state {
 
 	bool speculative;
 	bool in_sleepable;
+	/* The callee left with unwind_exit. Next insn is unwind_goto. */
+	bool unwound;
 
 	/*
 	 * First and last insn idx of this verifier state.
@@ -957,6 +959,7 @@ struct bpf_subprog_info {
 	s16 fastcall_stack_off;
 	bool has_tail_call: 1;
 	bool might_throw: 1;
+	bool can_unwind: 1; /* has unwind_exit */
 	bool tail_call_reachable: 1;
 	bool has_ld_abs: 1;
 	bool is_cb: 1;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 6f4c9bd024a0b..fccd8db336470 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -254,6 +254,25 @@ static inline bool insn_is_mov_percpu_addr(const struct bpf_insn *insn)
 		.off   = 0,					\
 		.imm   = 1 })
 
+static inline bool bpf_is_unwind_goto(const struct bpf_insn *insn)
+{
+	return insn->code == (BPF_JMP | BPF_JCOND) && insn->src_reg == BPF_UNWIND_GOTO;
+}
+
+static inline bool bpf_is_unwind_exit(const struct bpf_insn *insn)
+{
+	return insn->code == (BPF_JMP | BPF_EXIT) && insn->src_reg == BPF_UNWIND_EXIT;
+}
+
+/*
+ * The caller of the main prog doesn't tell unwind_exit from exit. An extension
+ * is different: its caller is the caller of the subprog that it replaced.
+ */
+static inline bool bpf_prog_can_unwind(const struct bpf_prog *prog)
+{
+	return bpf_is_subprog(prog) || prog->type == BPF_PROG_TYPE_EXT;
+}
+
 static inline bool insn_is_zext(const struct bpf_insn *insn)
 {
 	return insn->code == (BPF_ALU | BPF_MOV | BPF_X) && insn->imm == 1;
@@ -1257,6 +1276,7 @@ bool bpf_jit_supports_percpu_insn(void);
 bool bpf_jit_supports_kfunc_call(void);
 bool bpf_jit_supports_callx(void);
 bool bpf_jit_supports_kfunc_ret_reg_pair(void);
+bool bpf_jit_supports_unwind(void);
 bool bpf_jit_supports_stack_args(void);
 bool bpf_jit_supports_arena_args(void);
 bool bpf_jit_supports_far_kfunc_call(void);
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 6f70db7515dd4..6efe0ca036248 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -56,8 +56,19 @@
 
 enum bpf_cond_pseudo_jmp {
 	BPF_MAY_GOTO = 0,
+	/*
+	 * Follows a call of a bpf subprog. Taken when the subprog left with
+	 * BPF_EXIT that has src_reg == BPF_UNWIND_EXIT.
+	 */
+	BPF_UNWIND_GOTO = 1,
 };
 
+/*
+ * When bpf_exit->src_reg == BPF_UNWIND_EXIT, it's the same as bpf_exit, except
+ * that BPF_UNWIND_GOTO after the call is taken.
+ */
+#define BPF_UNWIND_EXIT	1
+
 /* Register numbers */
 enum {
 	BPF_REG_0 = 0,
diff --git a/kernel/bpf/const_fold.c b/kernel/bpf/const_fold.c
index 8dd8c5958eb85..f9edfde965dc7 100644
--- a/kernel/bpf/const_fold.c
+++ b/kernel/bpf/const_fold.c
@@ -357,7 +357,7 @@ int bpf_prune_dead_branches(struct bpf_verifier_env *env)
 
 		if (!bpf_insn_is_cond_jump(insn->code))
 			continue;
-		if (bpf_is_may_goto_insn(insn))
+		if (BPF_OP(insn->code) == BPF_JCOND)
 			continue;
 
 		if (!(aux->const_reg_mask & BIT(insn->dst_reg)))
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fc22d66d194a9..3030beb531d99 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3399,6 +3399,12 @@ bool __weak bpf_jit_supports_kfunc_ret_reg_pair(void)
 	return false;
 }
 
+/* unwind_goto and unwind_exit */
+bool __weak bpf_jit_supports_unwind(void)
+{
+	return false;
+}
+
 /*
  * How this arch places a by-value kfunc argument, or NULL for one that has
  * not opted in and so only takes an argument of a single eightbyte, which
diff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c
index 36d3228d77454..889fc074496f1 100644
--- a/kernel/bpf/disasm.c
+++ b/kernel/bpf/disasm.c
@@ -375,11 +375,16 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 			   insn->src_reg == BPF_MAY_GOTO) {
 			verbose(cbs->private_data, "(%02x) may_goto pc%+d",
 				insn->code, insn->off);
+		} else if (insn->code == (BPF_JMP | BPF_JCOND) &&
+			   insn->src_reg == BPF_UNWIND_GOTO) {
+			verbose(cbs->private_data, "(%02x) unwind_goto pc%+d",
+				insn->code, insn->off);
 		} else if (insn->code == (BPF_JMP32 | BPF_JA)) {
 			verbose(cbs->private_data, "(%02x) gotol pc%+d",
 				insn->code, insn->imm);
 		} else if (insn->code == (BPF_JMP | BPF_EXIT)) {
-			verbose(cbs->private_data, "(%02x) exit", insn->code);
+			verbose(cbs->private_data, "(%02x) %s", insn->code,
+				insn->src_reg == BPF_UNWIND_EXIT ? "unwind_exit" : "exit");
 		} else if (BPF_SRC(insn->code) == BPF_X) {
 			verbose(cbs->private_data,
 				"(%02x) if %c%d %s %c%d goto pc%+d",
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 409b9104c723e..9ae3a81604483 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -666,6 +666,10 @@ void bpf_opt_hard_wire_dead_code_branches(struct bpf_verifier_env *env)
 		if (!bpf_insn_is_cond_jump(insn->code))
 			continue;
 
+		/* unwind_exit of a callee that never exits needs it too */
+		if (bpf_is_unwind_goto(insn) && aux_data[i + 1 + insn->off].seen)
+			continue;
+
 		if (!aux_data[i + 1].seen)
 			ja.off = insn->off;
 		else if (!aux_data[i + 1 + insn->off].seen)
@@ -1467,12 +1471,15 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 			if (BPF_CLASS(insn->code) == BPF_STX &&
 			     BPF_MODE(insn->code) == BPF_PROBE_ATOMIC)
 				num_exentries++;
+			if (bpf_is_unwind_goto(insn))
+				num_exentries++;
 		}
 		func[i]->aux->num_exentries = num_exentries;
 		func[i]->aux->tail_call_reachable = env->subprog_info[i].tail_call_reachable;
 		func[i]->aux->exception_cb = env->subprog_info[i].is_exception_cb;
 		func[i]->aux->changes_pkt_data = env->subprog_info[i].changes_pkt_data;
 		func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
+		func[i]->aux->can_unwind = env->subprog_info[i].can_unwind;
 		func[i]->aux->token = prog->aux->token;
 		if (!i)
 			func[i]->aux->exception_boundary = env->seen_exception;
diff --git a/kernel/bpf/states.c b/kernel/bpf/states.c
index b3d250caaaa65..d7df6632a5c47 100644
--- a/kernel/bpf/states.c
+++ b/kernel/bpf/states.c
@@ -1046,6 +1046,9 @@ static bool states_equal(struct bpf_verifier_env *env,
 	if (old->in_sleepable != cur->in_sleepable)
 		return false;
 
+	if (old->unwound != cur->unwound)
+		return false;
+
 	if (!refsafe(old, cur, &env->idmap_scratch))
 		return false;
 
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 849bc9136299d..90bc7230afa8a 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -5087,6 +5087,12 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,
 			insns[i].imm = BPF_FUNC_tail_call;
 			/* fall-through */
 		}
+		/*
+		 * BPF_JCOND is the same opcode as BPF_CALL_ARGS. The latter is for
+		 * the interpreter, and a prog with unwind_goto is always JITed.
+		 */
+		if (prog->jited && bpf_is_unwind_goto(&insns[i]))
+			continue;
 		if (code == (BPF_JMP | BPF_CALL) ||
 		    code == (BPF_JMP | BPF_CALL_ARGS)) {
 			/* Restore the legacy xlated dump layout. */
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 01434e1d23b59..6c56675c3298d 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -1769,6 +1769,7 @@ int bpf_copy_verifier_state(struct bpf_verifier_state *dst_state,
 		return err;
 	dst_state->speculative = src->speculative;
 	dst_state->in_sleepable = src->in_sleepable;
+	dst_state->unwound = src->unwound;
 	dst_state->curframe = src->curframe;
 	dst_state->branches = src->branches;
 	dst_state->parent = src->parent;
@@ -3430,6 +3431,46 @@ static int next_callx_callee(struct bpf_verifier_env *env, int caller, int from)
 	return bit < end ? bit - caller * cnt : -1;
 }
 
+/*
+ * 'call foo; unwind_goto pad' goes to the pad when foo left with unwind_exit.
+ * JITs don't emit a branch for it. unwind_exit finds the pad by its return
+ * address. So unwind_goto has to follow the call and nothing may jump to it.
+ */
+static int check_unwind_insns(struct bpf_verifier_env *env)
+{
+	struct bpf_insn *insn = env->prog->insnsi;
+	bool seen = false;
+	int i;
+
+	for (i = 0; i < env->prog->len; i++, insn++) {
+		if (bpf_is_unwind_exit(insn)) {
+			bpf_find_containing_subprog(env, i)->can_unwind = true;
+			seen = true;
+		}
+		if (!bpf_is_unwind_goto(insn))
+			continue;
+		seen = true;
+		if (!i || !(bpf_pseudo_call(insn - 1) || bpf_is_callx(insn - 1))) {
+			verbose(env, "unwind_goto at insn %d doesn't follow a call of a subprog\n",
+				i);
+			return -EINVAL;
+		}
+		if (bpf_is_jump_target(env, i)) {
+			verbose(env, "jump to unwind_goto at insn %d\n", i);
+			return -EINVAL;
+		}
+	}
+	if (!seen)
+		return 0;
+	if (!bpf_jit_supports_unwind()) {
+		verbose(env, "JIT does not support unwinding\n");
+		return -ENOTSUPP;
+	}
+	env->prog->jit_required = 1;
+	env->prog->aux->can_unwind = env->subprog_info[0].can_unwind;
+	return 0;
+}
+
 /*
  * Sort subprogs in topological order so that leaf subprogs come first and
  * their callers come later. This is a DFS post-order traversal of the call
@@ -11584,6 +11625,19 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			}
 		}
 
+		if (env->subprog_info[subprog].can_unwind) {
+			struct bpf_verifier_state *branch;
+
+			if (!bpf_is_unwind_goto(insn + 1)) {
+				verbose(env, "no unwind_goto after the call of %s()\n", sub_name);
+				return -EINVAL;
+			}
+			branch = push_stack(env, *insn_idx + 1, *insn_idx, false);
+			if (IS_ERR(branch))
+				return PTR_ERR(branch);
+			branch->unwound = true;
+		}
+
 		if (env->subprog_info[subprog].might_throw) {
 			struct bpf_verifier_state *branch;
 
@@ -12057,6 +12111,19 @@ static int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)
 	}
 
 	caller = state->frame[state->curframe - 1];
+	if (bpf_is_unwind_exit(&env->prog->insnsi[*insn_idx])) {
+		/* the helper or the kfunc that called it doesn't expect that */
+		if (callee->in_callback_fn) {
+			verbose(env, "callback cannot unwind\n");
+			return -EINVAL;
+		}
+		if (!bpf_is_unwind_goto(&env->prog->insnsi[callee->callsite + 1])) {
+			verbose(env, "unwind_exit, but no unwind_goto after the call at insn %d\n",
+				callee->callsite);
+			return -EINVAL;
+		}
+		state->unwound = true;
+	}
 	if (callee->in_callback_fn) {
 		if (r0->type != SCALAR_VALUE) {
 			verbose(env, "R0 not a scalar value\n");
@@ -18759,6 +18826,23 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env,
 		return -EINVAL;
 	}
 
+	if (bpf_is_unwind_goto(insn)) {
+		if (!env->cur_state->unwound)
+			return 0;
+		env->cur_state->unwound = false;
+		/*
+		 * unwind_exit changed its return address, so the return is
+		 * mispredicted and what follows unwind_goto runs speculatively.
+		 */
+		if (!env->bypass_spec_v1) {
+			err = sanitize_speculative_path(env, NULL, *insn_idx + 1, *insn_idx);
+			if (err)
+				return err;
+		}
+		*insn_idx += insn->off;
+		return 0;
+	}
+
 	if (opcode == BPF_JCOND) {
 		struct bpf_verifier_state *cur_st = env->cur_state, *queued_st, *prev_st;
 		int idx = *insn_idx;
@@ -19986,6 +20070,12 @@ static int process_bpf_exit_full(struct bpf_verifier_env *env,
 	 * Exit from async callback implies return code restrictions
 	 * that depend on async scheduling mechanism.
 	 */
+	if ((cur_frame->in_async_callback_fn || cur_frame->in_exception_callback_fn) &&
+	    bpf_is_unwind_exit(&env->prog->insnsi[env->insn_idx])) {
+		verbose(env, "callback cannot unwind\n");
+		return -EINVAL;
+	}
+
 	if (cur_frame->subprogno &&
 	    !cur_frame->in_async_callback_fn &&
 	    !cur_frame->in_exception_callback_fn)
@@ -21223,14 +21313,14 @@ static int check_jmp_fields(struct bpf_verifier_env *env, struct bpf_insn *insn)
 		return 0;
 	case BPF_EXIT:
 		if (BPF_SRC(insn->code) != BPF_K || insn->imm != 0 ||
-		    insn->src_reg != BPF_REG_0 || insn->dst_reg != BPF_REG_0 ||
+		    insn->src_reg > BPF_UNWIND_EXIT || insn->dst_reg != BPF_REG_0 ||
 		    class == BPF_JMP32) {
 			verbose(env, "BPF_EXIT uses reserved fields\n");
 			return -EINVAL;
 		}
 		return 0;
 	case BPF_JCOND:
-		if (insn->code != (BPF_JMP | BPF_JCOND) || insn->src_reg != BPF_MAY_GOTO ||
+		if (insn->code != (BPF_JMP | BPF_JCOND) || insn->src_reg > BPF_UNWIND_GOTO ||
 		    insn->dst_reg || insn->imm) {
 			verbose(env, "invalid may_goto imm %d\n", insn->imm);
 			return -EINVAL;
@@ -22436,6 +22526,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 		struct bpf_prog_aux *aux = tgt_prog->aux;
 		bool tgt_changes_pkt_data;
 		bool tgt_might_sleep;
+		bool tgt_can_unwind;
 
 		if (bpf_prog_is_dev_bound(prog->aux) &&
 		    !bpf_prog_dev_bound_match(prog, tgt_prog)) {
@@ -22464,6 +22555,16 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 				prog_extension ? "Extension" : "Tracing");
 			return -EINVAL;
 		}
+		/* unwind_exit of the main prog is exit, unless it's an extension */
+		tgt_can_unwind = (subprog || tgt_prog->type == BPF_PROG_TYPE_EXT) &&
+				 (aux->func ? aux->func[subprog]->aux->can_unwind
+					    : aux->can_unwind);
+		/* unwind_exit would return to the trampoline that called the target */
+		if (tgt_can_unwind && !prog_extension &&
+		    prog->expected_attach_type != BPF_TRACE_FENTRY) {
+			bpf_log(log, "Only fentry can attach to %s, which can unwind\n", tname);
+			return -EINVAL;
+		}
 		conservative = aux->func_info_aux[subprog].unreliable;
 		if (prog_extension) {
 			if (aux->func_info_aux[subprog].linkage != BTF_FUNC_GLOBAL ||
@@ -22511,6 +22612,12 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
 					"Extension program may sleep, while original does not\n");
 				return -EINVAL;
 			}
+			/* There is no unwind_goto after the calls of the original */
+			if (prog->aux->can_unwind && !tgt_can_unwind) {
+				bpf_log(log,
+					"Extension program can unwind, while original does not\n");
+				return -EINVAL;
+			}
 		}
 		if (!tgt_prog->jited) {
 			bpf_log(log, "Can attach to only JITed progs\n");
@@ -23845,6 +23952,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
 	if (ret < 0)
 		goto skip_full_check;
 
+	ret = check_unwind_insns(env);
+	if (ret < 0)
+		goto skip_full_check;
+
 	ret = bpf_compute_postorder(env);
 	if (ret < 0)
 		goto skip_full_check;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 6f70db7515dd4..6efe0ca036248 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -56,8 +56,19 @@
 
 enum bpf_cond_pseudo_jmp {
 	BPF_MAY_GOTO = 0,
+	/*
+	 * Follows a call of a bpf subprog. Taken when the subprog left with
+	 * BPF_EXIT that has src_reg == BPF_UNWIND_EXIT.
+	 */
+	BPF_UNWIND_GOTO = 1,
 };
 
+/*
+ * When bpf_exit->src_reg == BPF_UNWIND_EXIT, it's the same as bpf_exit, except
+ * that BPF_UNWIND_GOTO after the call is taken.
+ */
+#define BPF_UNWIND_EXIT	1
+
 /* Register numbers */
 enum {
 	BPF_REG_0 = 0,
diff --git a/tools/testing/selftests/bpf/Makefile.skel b/tools/testing/selftests/bpf/Makefile.skel
index 06e297a17156c..6570c48f92b01 100644
--- a/tools/testing/selftests/bpf/Makefile.skel
+++ b/tools/testing/selftests/bpf/Makefile.skel
@@ -153,4 +153,15 @@ $(RUST_OBJS): $(RDIR)/%.bpf.o: progs/%.rs $(RUST_CORE) | $(RDIR)
 endif
 endif
 
+# Programs in C++. No skeletons: gcc doesn't build them.
+ifeq ($(BPF_CC),$(CLANG))
+CXX_OBJS := $(patsubst progs/%.cpp,$(RDIR)/%.bpf.o,$(wildcard progs/*.cpp))
+BPF_OBJS += $(CXX_OBJS)
+
+$(CXX_OBJS): $(RDIR)/%.bpf.o: progs/%.cpp | $(RDIR)
+	$(call msg,$(BPF_CC_MSG),$(BINARY),$@)
+	$(Q)$(BPF_CC) -x c++ -g -Wall -Werror -fexceptions -funwind-tables -fno-rtti	\
+		$(BPF_CC_FLAGS) -c $< -o $@ $(call skip_on_fail,BPF)
+endif
+
 endif # BPF_CC
diff --git a/tools/testing/selftests/bpf/prog_tests/unwind_attach.c b/tools/testing/selftests/bpf/prog_tests/unwind_attach.c
new file mode 100644
index 0000000000000..92eab5b77f62a
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/unwind_attach.c
@@ -0,0 +1,111 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include "verifier_unwind.skel.h"
+#include "unwind_attach.skel.h"
+
+#ifndef ENOTSUPP
+#define ENOTSUPP 524
+#endif
+
+static char log_buf[16 * 1024];
+
+static int run(int fd)
+{
+	char data[64] = {};
+	LIBBPF_OPTS(bpf_test_run_opts, opts,
+		.data_in = data,
+		.data_size_in = sizeof(data),
+		.repeat = 1,
+	);
+
+	ASSERT_OK(bpf_prog_test_run_opts(fd, &opts), "test_run");
+	return opts.retval;
+}
+
+/*
+ * msg: why it cannot be loaded. Otherwise retval is what the target returns
+ * with the prog attached.
+ */
+static void attach(int tgt_fd, const char *func, const char *name, const char *msg, int retval)
+{
+	struct bpf_link *link = NULL;
+	struct unwind_attach *skel;
+	struct bpf_program *prog;
+	int err;
+
+	skel = unwind_attach__open();
+	if (!ASSERT_OK_PTR(skel, "unwind_attach__open"))
+		return;
+
+	prog = bpf_object__find_program_by_name(skel->obj, name);
+	if (!ASSERT_OK_PTR(prog, name))
+		goto out;
+	bpf_program__set_autoload(prog, true);
+	log_buf[0] = 0;
+	bpf_program__set_log_buf(prog, log_buf, sizeof(log_buf));
+	err = bpf_program__set_attach_target(prog, tgt_fd, func);
+	if (!ASSERT_OK(err, "set_attach_target"))
+		goto out;
+
+	err = unwind_attach__load(skel);
+	if (msg) {
+		ASSERT_ERR(err, name);
+		ASSERT_HAS_SUBSTR(log_buf, msg, name);
+		goto out;
+	}
+	if (!ASSERT_OK(err, name))
+		goto out;
+
+	link = bpf_program__attach(prog);
+	if (!ASSERT_OK_PTR(link, "attach"))
+		goto out;
+	ASSERT_EQ(run(tgt_fd), retval, "retval");
+	if (prog == skel->progs.fentry_unwind)
+		ASSERT_EQ(skel->bss->fentry_cnt, 1, "fentry_cnt");
+out:
+	bpf_link__destroy(link);
+	unwind_attach__destroy(skel);
+}
+
+void test_unwind_attach(void)
+{
+	struct verifier_unwind *tgt;
+	struct bpf_program *prog;
+	int tgt_fd, plain_fd, err;
+
+	tgt = verifier_unwind__open();
+	if (!ASSERT_OK_PTR(tgt, "verifier_unwind__open"))
+		return;
+	bpf_object__for_each_program(prog, tgt->obj)
+		bpf_program__set_autoload(prog, prog == tgt->progs.calls_global_unwinds ||
+						prog == tgt->progs.calls_global_never_unwinds);
+	err = verifier_unwind__load(tgt);
+	if (err == -ENOTSUPP) {
+		printf("%s:SKIP:JIT does not support unwinding\n", __func__);
+		test__skip();
+		goto out;
+	}
+	if (!ASSERT_OK(err, "verifier_unwind__load"))
+		goto out;
+	/*
+	 * It calls global_unwinds(), which unwinds with r0 = 5, and returns r0 + 10
+	 * from the pad.
+	 */
+	tgt_fd = bpf_program__fd(tgt->progs.calls_global_unwinds);
+	plain_fd = bpf_program__fd(tgt->progs.calls_global_never_unwinds);
+	ASSERT_EQ(run(tgt_fd), 15, "retval");
+
+	attach(tgt_fd, "global_unwinds", "fentry_unwind", NULL, 15);
+	/* unwind_exit would return to the trampoline */
+	attach(tgt_fd, "global_unwinds", "fexit_unwind",
+	       "Only fentry can attach to global_unwinds", 0);
+	/* An extension leaves either way, like what it replaced */
+	attach(tgt_fd, "global_unwinds", "freplace_returns", NULL, 9);
+	attach(tgt_fd, "global_unwinds", "freplace_unwinds", NULL, 13);
+	/* There is no unwind_goto after the call */
+	attach(plain_fd, "global_never_unwinds", "freplace_unwinds",
+	       "Extension program can unwind, while original does not", 0);
+	ASSERT_EQ(run(tgt_fd), 15, "retval");
+out:
+	verifier_unwind__destroy(tgt);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c b/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c
new file mode 100644
index 0000000000000..23e9c7566563d
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/unwind_cxx.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include <bpf/btf.h>
+
+#ifndef ENOTSUPP
+#define ENOTSUPP 524
+#endif
+
+/* The progs are under #ifdef __BPF_FEATURE_UNWIND */
+static bool llvm_lowers_unwinding(const char *file)
+{
+	struct btf *btf = btf__parse(file, NULL);
+	bool ret;
+
+	ret = btf && btf__find_by_name_kind(btf, "catches", BTF_KIND_FUNC) > 0;
+	btf__free(btf);
+	return ret;
+}
+
+static void run(struct bpf_object *obj, const char *name, int len, int retval, int dtors)
+{
+	struct bpf_program *prog = bpf_object__find_program_by_name(obj, name);
+	struct bpf_map *map = bpf_object__find_map_by_name(obj, ".data.dtors");
+	char data[256] = {};
+	LIBBPF_OPTS(bpf_test_run_opts, opts,
+		.data_in = data,
+		.data_size_in = len,
+		.repeat = 1,
+	);
+	int key = 0, val = 0;
+
+	if (!ASSERT_OK_PTR(prog, name) || !ASSERT_OK_PTR(map, "dtors"))
+		return;
+	ASSERT_OK(bpf_map__update_elem(map, &key, sizeof(key), &val, sizeof(val), 0), "clear");
+	ASSERT_OK(bpf_prog_test_run_opts(bpf_program__fd(prog), &opts), "test_run");
+	ASSERT_EQ(opts.retval, retval, "retval");
+	ASSERT_OK(bpf_map__lookup_elem(map, &key, sizeof(key), &val, sizeof(val), 0), "lookup");
+	ASSERT_EQ(val, dtors, "dtors");
+}
+
+void test_unwind_cxx(void)
+{
+	const char *file = "./unwind_cxx.bpf.o";
+	struct bpf_object *obj;
+	struct bpf_program *prog;
+	int err = -EINVAL;
+
+	if (access(file, R_OK)) {
+		printf("%s:SKIP:gcc doesn't build %s\n", __func__, file);
+		test__skip();
+		return;
+	}
+	if (!llvm_lowers_unwinding(file)) {
+		printf("%s:SKIP:llvm doesn't lower unwinding\n", __func__);
+		test__skip();
+		return;
+	}
+	obj = bpf_object__open_file(file, NULL);
+	if (!ASSERT_OK_PTR(obj, "open"))
+		return;
+	prog = bpf_object__find_program_by_name(obj, "catches_aggregate");
+	if (!ASSERT_OK_PTR(prog, "catches_aggregate"))
+		goto out;
+	bpf_program__set_autoload(prog, false);
+	err = bpf_object__load(obj);
+	if (err == -ENOTSUPP) {
+		printf("%s:SKIP:JIT does not support unwinding\n", __func__);
+		test__skip();
+		goto out;
+	}
+	if (!ASSERT_OK(err, "load"))
+		goto out;
+
+	/* inner's destructor, then middle's */
+	run(obj, "catches", 64, 65, 12);
+	run(obj, "catches", 200, 42, 12);
+	run(obj, "does_not_catch", 64, 65, 12);
+	run(obj, "does_not_catch", 200, 0, 12);
+out:
+	bpf_object__close(obj);
+	if (err)
+		return;
+
+	if (!test__start_subtest("aggregate_return"))
+		return;
+	obj = bpf_object__open_file(file, NULL);
+	if (!ASSERT_OK_PTR(obj, "open_aggregate"))
+		return;
+	bpf_object__for_each_program(prog, obj)
+		bpf_program__set_autoload(prog,
+					 strcmp(bpf_program__name(prog), "catches_aggregate") == 0);
+	if (!ASSERT_OK(bpf_object__load(obj), "load_aggregate"))
+		goto out_aggregate;
+	run(obj, "catches_aggregate", 64, 71, 0);
+	run(obj, "catches_aggregate", 200, 42, 0);
+out_aggregate:
+	bpf_object__close(obj);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index 6df440d826750..04740ba37e918 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -130,6 +130,7 @@
 #include "verifier_typedef.skel.h"
 #include "verifier_uninit.skel.h"
 #include "verifier_unpriv.skel.h"
+#include "verifier_unwind.skel.h"
 #include "verifier_unpriv_perf.skel.h"
 #include "verifier_value_adj_spill.skel.h"
 #include "verifier_value.skel.h"
@@ -320,6 +321,7 @@ void test_verifier_tailcall_jit(void)         { RUN(verifier_tailcall_jit); }
 void test_verifier_typedef(void)              { RUN(verifier_typedef); }
 void test_verifier_uninit(void)               { RUN(verifier_uninit); }
 void test_verifier_unpriv(void)               { RUN(verifier_unpriv); }
+void test_verifier_unwind(void)               { RUN(verifier_unwind); }
 void test_verifier_unpriv_perf(void)          { RUN(verifier_unpriv_perf); }
 void test_verifier_value_adj_spill(void)      { RUN(verifier_value_adj_spill); }
 void test_verifier_value(void)                { RUN(verifier_value); }
diff --git a/tools/testing/selftests/bpf/progs/unwind_attach.c b/tools/testing/selftests/bpf/progs/unwind_attach.c
new file mode 100644
index 0000000000000..df73c26c1ff46
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/unwind_attach.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "../../../include/linux/filter.h"
+#include "bpf_misc.h"
+
+/* The targets are global_unwinds() and global_never_unwinds() in verifier_unwind.c */
+
+int fentry_cnt;
+
+SEC("?fentry/global_unwinds")
+int BPF_PROG(fentry_unwind)
+{
+	fentry_cnt++;
+	return 0;
+}
+
+SEC("?fexit/global_unwinds")
+int BPF_PROG(fexit_unwind)
+{
+	return 0;
+}
+
+SEC("?freplace/global_unwinds")
+unsigned long freplace_returns(void)
+{
+	return 9;
+}
+
+SEC("?freplace/global_unwinds")
+__naked unsigned long freplace_unwinds(void)
+{
+	asm volatile (
+		"r0 = 3;"
+		".8byte %[unwind_exit];"
+		:
+		: __imm_insn(unwind_exit,
+			     BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT, 0, 0))
+	);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/unwind_cxx.cpp b/tools/testing/selftests/bpf/progs/unwind_cxx.cpp
new file mode 100644
index 0000000000000..3a113ffea29ca
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/unwind_cxx.cpp
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * C++ exceptions. llvm lowers unwinding to two insns:
+ *   invoke foo unwind pad    call foo; unwind_goto pad
+ *   resume                   r0 = 0; unwind_exit
+ * so 'throw' is a jump or an exit, 'catch (...)' and the destructors that run on
+ * the way are ordinary blocks.
+ *
+ * vmlinux.h and bpf_helpers.h are not C++.
+ */
+#define SEC(name) __attribute__((section(name), used))
+#define __ksym __attribute__((section(".ksyms")))
+#define __noinline __attribute__((noinline))
+#define __always_inline inline __attribute__((always_inline))
+
+#ifdef __BPF_FEATURE_UNWIND
+
+struct __sk_buff {
+	unsigned int len;
+};
+
+extern "C" {
+void bpf_rcu_read_lock(void) noexcept __ksym;
+void bpf_rcu_read_unlock(void) noexcept __ksym;
+void bpf_preempt_disable(void) noexcept __ksym;
+void bpf_preempt_enable(void) noexcept __ksym;
+
+/* Starts an unwind. Not a kfunc: llvm lowers the call and it doesn't come back. */
+int _Unwind_RaiseException(void *exc);
+
+/* All that is left of the C++ runtime. Nobody looks at the exception object. */
+static char exc_obj[16];
+
+__always_inline void *__cxa_allocate_exception(unsigned long size) noexcept
+{
+	return exc_obj;
+}
+
+__always_inline void __cxa_throw(void *exc, void *tinfo, void (*dtor)(void *))
+{
+	_Unwind_RaiseException(exc);
+}
+
+__always_inline void *__cxa_begin_catch(void *exc) noexcept
+{
+	return exc;
+}
+
+__always_inline void __cxa_end_catch(void) noexcept
+{
+}
+}
+
+/* One decimal digit per destructor, in the order they ran */
+volatile int dtors SEC(".data.dtors");
+
+/* The verifier insists that both are undone on every path out of the prog */
+struct rcu_guard {
+	__always_inline rcu_guard()
+	{
+		bpf_rcu_read_lock();
+	}
+	__always_inline ~rcu_guard()
+	{
+		bpf_rcu_read_unlock();
+		dtors = dtors * 10 + 1;
+	}
+};
+
+struct plain_rcu_guard {
+	__always_inline plain_rcu_guard()
+	{
+		bpf_rcu_read_lock();
+	}
+	__always_inline ~plain_rcu_guard()
+	{
+		bpf_rcu_read_unlock();
+	}
+};
+
+struct preempt_guard {
+	__always_inline preempt_guard()
+	{
+		bpf_preempt_disable();
+	}
+	__always_inline ~preempt_guard()
+	{
+		bpf_preempt_enable();
+		dtors = dtors * 10 + 2;
+	}
+};
+
+static __noinline int inner(int len)
+{
+	rcu_guard guard;
+
+	if (len > 100)
+		throw 1;
+	return len;
+}
+
+static __noinline int middle(int len)
+{
+	preempt_guard guard;
+
+	return inner(len) + 1;
+}
+
+using u128 = unsigned __int128;
+
+/* Global subprogs returning 16 bytes use R0:R2, including unwind exits. */
+extern "C" __noinline u128 aggregate_return(int len)
+{
+	plain_rcu_guard guard;
+
+	if (len > 100)
+		throw 1;
+	return (static_cast<u128>(len) << 64) | 7;
+}
+
+extern "C" SEC("tc") int catches(struct __sk_buff *skb)
+{
+	try {
+		return middle(skb->len);
+	} catch (...) {
+		return 42;
+	}
+}
+
+/* The exception leaves the prog: it returns 0 */
+extern "C" SEC("tc") int does_not_catch(struct __sk_buff *skb)
+{
+	return middle(skb->len);
+}
+
+extern "C" SEC("tc") int catches_aggregate(struct __sk_buff *skb)
+{
+	try {
+		u128 value = aggregate_return(skb->len);
+
+		return static_cast<unsigned long>(value) +
+		       static_cast<unsigned long>(value >> 64);
+	} catch (...) {
+		return 42;
+	}
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/verifier_unwind.c b/tools/testing/selftests/bpf/progs/verifier_unwind.c
new file mode 100644
index 0000000000000..dbe7b5d91acd4
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_unwind.c
@@ -0,0 +1,1182 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "../../../include/linux/filter.h"
+#include "bpf_misc.h"
+
+/*
+ * llvm lowers unwinding to two insns:
+ *   invoke foo unwind pad    call foo; unwind_goto pad
+ *   resume                   unwind_exit
+ * The tests spell them out as raw insns, so they don't depend on llvm version.
+ * The argument of unwind_goto() is the number of insns to skip.
+ */
+#define unwind_goto(off) ".8byte %[insn_unwind_goto_" #off "];"
+#define unwind_exit ".8byte %[insn_unwind_exit];"
+
+#define __imm_unwind_goto(off) \
+	__imm_insn(insn_unwind_goto_##off, \
+		   BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO, off, 0))
+#define __imm_unwind_exit \
+	__imm_insn(insn_unwind_exit, \
+		   BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT, 0, 0))
+
+extern void bpf_rcu_read_lock(void) __ksym;
+extern void bpf_rcu_read_unlock(void) __ksym;
+extern void bpf_throw(__u64 cookie) __ksym;
+
+struct {
+	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+	__uint(max_entries, 1);
+	__uint(key_size, sizeof(int));
+	__uint(value_size, sizeof(int));
+} prog_array SEC(".maps");
+
+struct timer_elem {
+	struct bpf_timer t;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARRAY);
+	__uint(max_entries, 1);
+	__type(key, int);
+	__type(value, struct timer_elem);
+} timer_map SEC(".maps");
+
+/* The verifier doesn't know what is in there */
+unsigned long zero;
+unsigned long one = 1;
+
+/*
+ * The verifier follows a static subprog, so it knows which way it leaves when
+ * it knows the argument. To have both ways at run time the tests pass skb->len,
+ * which is 64, or skb->len - 64.
+ *
+ * r1 == 0: returns 7, otherwise unwinds with r0 = 5
+ */
+__naked __noinline __used
+static unsigned long may_unwind(void)
+{
+	asm volatile (
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+/* owns nothing: its pad only carries on */
+__naked __noinline __used
+static unsigned long passes_on(void)
+{
+	asm volatile (
+		"call may_unwind;"
+		unwind_goto(2)
+		"r0 += 1;"
+		"exit;"
+		unwind_exit
+		:
+		: __imm_unwind_goto(2),
+		  __imm_unwind_exit
+	);
+}
+
+__naked __noinline __used
+static unsigned long always_unwinds(void)
+{
+	asm volatile (
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+__naked __noinline __used
+static unsigned long never_unwinds(void)
+{
+	asm volatile (
+		"r0 = 7;"
+		"exit;"
+	);
+}
+
+/* Both can unwind. This one returns 7. */
+__naked __noinline __used
+unsigned long global_returns(void)
+{
+	asm volatile (
+		"r1 = %[zero] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_addr(zero),
+		  __imm_unwind_exit
+	);
+}
+
+/* This one unwinds with r0 = 5 */
+__naked __noinline __used
+unsigned long global_unwinds(void)
+{
+	asm volatile (
+		"r1 = %[one] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_addr(one),
+		  __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: the callee returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int callee_returns(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r1 -= 64;"
+		"call may_unwind;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: the callee unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__naked int callee_unwinds(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call may_unwind;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: return through two frames")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(8)
+__naked int two_frames_return(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r1 -= 64;"
+		"call passes_on;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* unwind_exit returns r0 like exit does */
+SEC("tc")
+__description("unwind: unwind through two frames, r0 reaches the pad")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(15)
+__naked int two_frames_unwind(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call passes_on;"
+		unwind_goto(1)
+		"exit;"
+		"r0 += 10;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: r6-r9 and the stack are intact in the pad")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(1111)
+__naked int pad_sees_callee_saved(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r6 = 1;"
+		"r7 = 10;"
+		"r8 = 100;"
+		"r9 = 400;"
+		"*(u64 *)(r10 - 8) = 600;"
+		"call passes_on;"
+		unwind_goto(2)
+		"r0 = 0;"
+		"exit;"
+		"r0 = r6;"
+		"r0 += r7;"
+		"r0 += r8;"
+		"r0 += r9;"
+		"r1 = *(u64 *)(r10 - 8);"
+		"r0 += r1;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: r1-r5 are not readable in the pad")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R3 !read_ok")
+__naked int pad_reads_r3(void)
+{
+	asm volatile (
+		"r3 = 0;"
+		"call always_unwinds;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = r3;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_exit of the main prog is exit")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(3)
+__naked int main_prog_unwinds(void)
+{
+	asm volatile (
+		"r0 = 3;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_exit of the main prog needs r0")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R0 !read_ok")
+__naked int main_prog_unwinds_no_r0(void)
+{
+	asm volatile (
+		unwind_exit
+		:
+		: __imm_unwind_exit
+		: __clobber_all
+	);
+}
+
+/* What follows unwind_goto is dead code. unwind_exit still has to find the pad. */
+SEC("tc")
+__description("unwind: the callee never returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__xlated("0: call pc+3")
+__xlated("1: unwind_goto pc+0")
+__xlated("2: r0 = 42")
+__xlated("3: exit")
+__naked int callee_never_returns(void)
+{
+	asm volatile (
+		"call always_unwinds;"
+		unwind_goto(2)
+		"r0 = 1;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+/* The pad is dead code and goes away together with unwind_goto */
+SEC("tc")
+__description("unwind: the callee never unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__xlated("0: call pc+1")
+__xlated("1: exit")
+__xlated("2: r0 = 7")
+__xlated("3: exit")
+__naked int callee_never_unwinds(void)
+{
+	asm volatile (
+		"call never_unwinds;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = *(u64 *)(r0 + 0);"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: the pad drops what the frame owns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__naked int pad_unlocks(void)
+{
+	asm volatile (
+		"r6 = *(u32 *)(r1 + %[len]);"
+		"call %[bpf_rcu_read_lock];"
+		"r1 = r6;"
+		"call may_unwind;"
+		unwind_goto(3)
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 0;"
+		"exit;"
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm(bpf_rcu_read_lock),
+		  __imm(bpf_rcu_read_unlock),
+		  __imm_unwind_goto(3)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: the pad is checked like any other code")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("in main prog cannot be used inside bpf_rcu_read_lock-ed region")
+__naked int pad_does_not_unlock(void)
+{
+	asm volatile (
+		"r6 = *(u32 *)(r1 + %[len]);"
+		"call %[bpf_rcu_read_lock];"
+		"r1 = r6;"
+		"call may_unwind;"
+		unwind_goto(3)
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 0;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm(bpf_rcu_read_lock),
+		  __imm(bpf_rcu_read_unlock),
+		  __imm_unwind_goto(3)
+		: __clobber_all
+	);
+}
+
+/* Leaves either way with the same registers */
+__naked __noinline __used
+static unsigned long same_regs(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"if r1 != 0 goto 1f;"
+		"exit;"
+	"1:"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+/*
+ * After the call the two states differ only in the way the callee left.
+ * The second one must not be pruned, or nobody looks at the pad.
+ */
+SEC("tc")
+__description("unwind: the way the callee left is a part of the state")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R0 invalid mem access 'scalar'")
+__flag(BPF_F_TEST_STATE_FREQ)
+__naked int no_pruning_after_the_call(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call same_regs;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = *(u64 *)(r0 + 0);"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* Returns ctx, or unwinds with a number in r0 */
+__naked __noinline __used
+static unsigned long ctx_or_unwind(void)
+{
+	asm volatile (
+		"r2 = *(u32 *)(r1 + %[len]);"
+		"if r2 > 100 goto 1f;"
+		"r0 = r1;"
+		"exit;"
+	"1:"
+		"r0 = 0;"
+		unwind_exit
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_exit
+	);
+}
+
+/*
+ * unwind_exit changes its return address, so the cpu mispredicts the return and
+ * runs what follows unwind_goto with r0 of unwind_exit.
+ */
+SEC("socket")
+__description("unwind: what follows unwind_goto runs speculatively")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(50) /* 64 without the ethernet header */
+__success_unpriv __retval_unpriv(50)
+__caps_unpriv(CAP_BPF)
+#ifdef SPEC_V1
+__xlated_unpriv("unwind_goto pc+{{.*}}")
+__xlated_unpriv("nospec") /* inserted to prevent `R0 invalid mem access 'scalar'` */
+__xlated_unpriv("r0 = *(u32 *)(r0 +{{.*}})")
+#endif
+__naked int speculative_fall_through(void)
+{
+	asm volatile (
+		"call ctx_or_unwind;"
+		unwind_goto(2)
+		"r0 = *(u32 *)(r0 + %[len]);"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: a global subprog returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int calls_global_returns(void)
+{
+	asm volatile (
+		"call global_returns;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: a global subprog unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(15)
+__naked int calls_global_unwinds(void)
+{
+	asm volatile (
+		"call global_unwinds;"
+		unwind_goto(1)
+		"exit;"
+		"r0 += 10;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* The verifier doesn't look inside */
+SEC("tc")
+__description("unwind: the pad of a call of a global subprog is always checked")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("R0 invalid mem access 'scalar'")
+__naked int global_pad_is_checked(void)
+{
+	asm volatile (
+		"call global_returns;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = *(u64 *)(r0 + 0);"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+__naked __noinline __used
+unsigned long global_never_unwinds(void)
+{
+	asm volatile (
+		"r0 = 7;"
+		"exit;"
+	);
+}
+
+SEC("tc")
+__description("unwind: a global subprog that cannot unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int calls_global_never_unwinds(void)
+{
+	asm volatile (
+		"call global_never_unwinds;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: no unwind_goto after a call of a global subprog")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("no unwind_goto after the call of global_returns()")
+__naked int global_no_unwind_goto(void)
+{
+	asm volatile (
+		"call global_returns;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: no unwind_goto after a call of a static subprog")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_exit, but no unwind_goto after the call at insn 1")
+__naked int static_no_unwind_goto(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call may_unwind;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len))
+		: __clobber_all
+	);
+}
+
+/* The verifier sees that it doesn't get to unwind_exit */
+SEC("tc")
+__description("unwind: no unwind_goto is fine when the callee doesn't unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int static_does_not_get_there(void)
+{
+	asm volatile (
+		"r1 = 0;"
+		"call may_unwind;"
+		"exit;"
+		::: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: callx, the callee returns")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int callx_returns(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r1 -= 64;"
+		"r6 = %[may_unwind] ll;"
+		"callx r6;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_addr(may_unwind),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: callx, the callee unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(42)
+__naked int callx_unwinds(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"r6 = %[may_unwind] ll;"
+		"callx r6;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_addr(may_unwind),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto after a plain insn")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 1 doesn't follow a call of a subprog")
+__naked int unwind_goto_after_mov(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		unwind_goto(0)
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto is the first insn")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 0 doesn't follow a call of a subprog")
+__naked int unwind_goto_first(void)
+{
+	asm volatile (
+		unwind_goto(0)
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto after a call of a helper")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 1 doesn't follow a call of a subprog")
+__naked int unwind_goto_after_helper(void)
+{
+	asm volatile (
+		"call %[bpf_get_prandom_u32];"
+		unwind_goto(0)
+		"exit;"
+		:
+		: __imm(bpf_get_prandom_u32),
+		  __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unwind_goto after a call of a kfunc")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("unwind_goto at insn 1 doesn't follow a call of a subprog")
+__naked int unwind_goto_after_kfunc(void)
+{
+	asm volatile (
+		"call %[bpf_rcu_read_lock];"
+		unwind_goto(0)
+		"call %[bpf_rcu_read_unlock];"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm(bpf_rcu_read_lock),
+		  __imm(bpf_rcu_read_unlock),
+		  __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: jump to unwind_goto")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("jump to unwind_goto at insn 3")
+__naked int jump_to_unwind_goto(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"if r1 != 0 goto 1f;"
+		"call never_unwinds;"
+	"1:"
+		unwind_goto(0)
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: reserved fields of unwind_goto")
+__failure __msg("invalid may_goto imm 1")
+__naked int unwind_goto_reserved(void)
+{
+	asm volatile (
+		"call never_unwinds;"
+		".8byte %[bad];"
+		"exit;"
+		:
+		: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO, 0, 1))
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unknown kind of jcond")
+__failure __msg("invalid may_goto imm 0")
+__naked int jcond_unknown(void)
+{
+	asm volatile (
+		"call never_unwinds;"
+		".8byte %[bad];"
+		"exit;"
+		:
+		: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, BPF_UNWIND_GOTO + 1, 0, 0))
+		: __clobber_all
+	);
+}
+
+SEC("tc")
+__description("unwind: unknown kind of exit")
+__failure __msg("BPF_EXIT uses reserved fields")
+__naked int exit_unknown(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		".8byte %[bad];"
+		:
+		: __imm_insn(bad, BPF_RAW_INSN(BPF_JMP | BPF_EXIT, 0, BPF_UNWIND_EXIT + 1, 0, 0))
+		: __clobber_all
+	);
+}
+
+/* r1 is the index: unwinds in the second iteration */
+__naked __noinline __used
+static unsigned long cb_unwinds(void)
+{
+	asm volatile (
+		"r0 = 0;"
+		"if r1 != 0 goto 1f;"
+		"exit;"
+	"1:"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: callback unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("callback cannot unwind")
+__naked int callback_unwinds(void)
+{
+	asm volatile (
+		"r1 = 3;"
+		"r2 = %[cb_unwinds];"
+		"r3 = 0;"
+		"r4 = 0;"
+		"call %[bpf_loop];"
+		"exit;"
+		:
+		: __imm_ptr(cb_unwinds),
+		  __imm(bpf_loop)
+		: __clobber_common
+	);
+}
+
+SEC("tc")
+__description("unwind: async callback unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__failure __msg("callback cannot unwind")
+int async_callback_unwinds(void *ctx)
+{
+	struct timer_elem *val;
+	int key = 0;
+
+	val = bpf_map_lookup_elem(&timer_map, &key);
+	if (!val)
+		return 0;
+
+	bpf_timer_init(&val->t, &timer_map, 0);
+	bpf_timer_set_callback(&val->t, (void *)always_unwinds);
+	return 0;
+}
+
+/* Not naked: the argument has to have a name in BTF */
+__noinline int exception_cb_unwinds(__u64 cookie)
+{
+	if (cookie)
+		asm volatile ("r0 = 0;" unwind_exit :: __imm_unwind_exit : "r0");
+	return 0;
+}
+
+SEC("tc")
+__description("unwind: exception callback unwinds")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__attribute__((btf_decl_tag("exception_callback:exception_cb_unwinds")))
+__failure __msg("callback cannot unwind")
+int exception_callback_unwinds(struct __sk_buff *skb)
+{
+	bpf_throw(skb->len);
+	return 0;
+}
+
+/* What it calls can unwind as long as it doesn't let it out */
+__naked __noinline __used
+static unsigned long cb_catches(void)
+{
+	asm volatile (
+		"call may_unwind;"
+		unwind_goto(0)
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_unwind_goto(0)
+	);
+}
+
+SEC("tc")
+__description("unwind: callback catches")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(3)
+__naked int callback_catches(void)
+{
+	asm volatile (
+		"r1 = 3;"
+		"r2 = %[cb_catches];"
+		"r3 = 0;"
+		"r4 = 0;"
+		"call %[bpf_loop];"
+		"exit;"
+		:
+		: __imm_ptr(cb_catches),
+		  __imm(bpf_loop)
+		: __clobber_common
+	);
+}
+
+/* Both leave the subprog like exit does */
+__naked __noinline __used
+static unsigned long does_tail_call(void)
+{
+	asm volatile (
+		"r6 = *(u32 *)(r1 + %[len]);"
+		"r2 = %[prog_array] ll;"
+		"r3 = 0;"
+		"call %[bpf_tail_call];"
+		"r0 = 7;"
+		"if r6 == 64 goto 1f;"
+		unwind_exit
+	"1:"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm(bpf_tail_call),
+		  __imm_addr(prog_array),
+		  __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: tail call in a subprog that can unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int tail_call_in_subprog(void)
+{
+	asm volatile (
+		"call does_tail_call;"
+		unwind_goto(1)
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+/* The packet is shorter than that: ld_abs leaves the subprog with r0 = 0 */
+__naked __noinline __used
+static unsigned long does_ld_abs(void)
+{
+	asm volatile (
+		"r6 = r1;"
+		"r0 = *(u8 *)skb[1000];"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+SEC("tc")
+__description("unwind: ld_abs in a subprog that can unwind")
+__arch_x86_64
+__arch_arm64
+__load_if_JITed()
+__success __retval(7)
+__naked int ld_abs_in_subprog(void)
+{
+	asm volatile (
+		"call does_ld_abs;"
+		unwind_goto(2)
+		"r0 += 7;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+/*
+ * JITs don't emit anything for unwind_goto. It's an extable entry, where
+ * unwind_exit looks its return address up.
+ */
+SEC("tc")
+__description("unwind: jit")
+__success __retval(42)
+__load_if_JITed()
+__arch_x86_64
+__jited("	callq	0x{{.*}}")
+__jited("	movl	$0x7, %eax")
+__jited("...")
+__jited("	movl	$0x5, %eax")
+__jited("	pushq	%rax")
+__jited("	pushq	%rsi")
+__jited("	movq	0x8(%rbp), %rdi")
+__jited("...") /* call depth accounting */
+__jited("	callq	0x{{.*}}")
+__jited("	movq	%rax, 0x8(%rbp)")
+__jited("	popq	%rsi")
+__jited("	popq	%rax")
+__arch_arm64
+__jited("	bl	{{.*}}")
+__jited("	mov	x8, x0")
+__jited("	mov	x8, #0x7")
+__jited("...")
+__jited("	mov	x8, #0x5")
+__jited("	stp	x8, x1, [sp, #-0x10]!")
+__jited("	ldr	x17, [x29, #0x8]")
+__jited("...")
+__jited("	str	x17, [x29, #0x8]")
+__jited("	ldp	x8, x1, [sp], #0x10")
+__naked int jit(void)
+{
+	asm volatile (
+		"r1 = *(u32 *)(r1 + %[len]);"
+		"call may_unwind;"
+		unwind_goto(2)
+		"r0 = 7;"
+		"exit;"
+		"r0 = 42;"
+		"exit;"
+		:
+		: __imm_const(len, offsetof(struct __sk_buff, len)),
+		  __imm_unwind_goto(2)
+		: __clobber_all
+	);
+}
+
+/* r1 == 0: returns 7, otherwise unwinds with r0 = 5 */
+__naked __noinline __used
+static unsigned long may_unwind_big_stack(void)
+{
+	asm volatile (
+		"*(u64 *)(r10 - 200) = r1;"
+		"if r1 != 0 goto 1f;"
+		"r0 = 7;"
+		"exit;"
+	"1:"
+		"r0 = 5;"
+		unwind_exit
+		:
+		: __imm_unwind_exit
+	);
+}
+
+/*
+ * x86-64 keeps the pointer to the private stack in r9 and saves it around a call.
+ * The value that is read back in the pad is there only when r9 is restored.
+ */
+SEC("raw_tp")
+__description("unwind: private stack, the callee unwinds")
+__success __retval(47)
+__load_if_JITed()
+__arch_x86_64
+__jited("	pushq	%r9")
+__jited("...") /* call depth accounting */
+__jited("	callq	0x{{.*}}")
+__jited("	popq	%r9")
+__jited("	jmp	L{{.*}}")
+__jited("	popq	%r9")
+__jited("	jmp	L{{.*}}")
+__arch_arm64
+__naked int private_stack_unwinds(void)
+{
+	asm volatile (
+		"*(u64 *)(r10 - 200) = 42;"
+		"r1 = %[one] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"call may_unwind_big_stack;"
+		unwind_goto(1)
+		"exit;"
+		"r1 = *(u64 *)(r10 - 200);"
+		"r0 += r1;"
+		"exit;"
+		:
+		: __imm_addr(one),
+		  __imm_unwind_goto(1)
+		: __clobber_all
+	);
+}
+
+SEC("raw_tp")
+__description("unwind: private stack, the callee returns")
+__success __retval(49)
+__load_if_JITed()
+__arch_x86_64
+__arch_arm64
+__naked int private_stack_returns(void)
+{
+	asm volatile (
+		"*(u64 *)(r10 - 200) = 42;"
+		"r1 = %[zero] ll;"
+		"r1 = *(u64 *)(r1 + 0);"
+		"call may_unwind_big_stack;"
+		unwind_goto(3)
+		"r1 = *(u64 *)(r10 - 200);"
+		"r0 += r1;"
+		"exit;"
+		"r0 = 0;"
+		"exit;"
+		:
+		: __imm_addr(zero),
+		  __imm_unwind_goto(3)
+		: __clobber_all
+	);
+}
+
+/*
+ * BTF FUNC records are not generated for kfuncs referenced
+ * from inline assembly. These records are necessary for
+ * libbpf to link the program. The function below is a hack
+ * to ensure that BTF FUNC records are generated.
+ */
+void __kfunc_btf_root(void)
+{
+	bpf_rcu_read_lock();
+	bpf_rcu_read_unlock();
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_loader.c b/tools/testing/selftests/bpf/test_loader.c
index 18d47e9254007..824e5912ebcb8 100644
--- a/tools/testing/selftests/bpf/test_loader.c
+++ b/tools/testing/selftests/bpf/test_loader.c
@@ -1546,7 +1546,8 @@ void run_subtest(struct test_loader *tester,
 		}
 
 		err = do_prog_test_run(bpf_program__fd(tprog), &retval,
-				       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ? true : false,
+				       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ||
+				       bpf_program__type(tprog) == BPF_PROG_TYPE_RAW_TRACEPOINT,
 				       spec->linear_sz);
 		if (!err && retval != subspec->retval && subspec->retval != POINTER_VALUE) {
 			PRINT_FAIL("Unexpected retval: %d != %d\n", retval, subspec->retval);

29/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 30291
							output: 38
							thoughts: 1553
						

Thoughts:
**BPF Unwind Patch: KMSAN Evaluation**

Okay, let's break this down. My primary concern is whether this patch series necessitates a dedicated KMSAN fuzzing session, especially given the target architecture is amd64. I need to determine if it introduces any uninitialized memory usage that KASAN or other standard bug detectors wouldn't catch.

First, I see that this patch introduces unwind functionality to the BPF verifier and JIT, somewhat mirroring C++ exception handling. Specifically, we're talking about `unwind_goto` and `unwind_exit` instructions, and they're going to use an exception table for these unwind paths. The JIT code is being updated to support these unwind features. In essence, `BPF_UNWIND_EXIT` modifies the return address on the stack to jump to a landing pad specified by `unwind_goto`. The JIT adds exception table entries (`EX_TYPE_BPF`), and the verifier tracks and validates these unwind paths.

So, the key here is what this unwind functionality actually *does*. Does it introduce new ways for the kernel to expose uninitialized memory to user space, which is what KMSAN is really good at catching? Remember, KMSAN is for detecting uninitialized memory reads and leaks to user space, something KASAN *doesn't* always catch. KASAN is still good for the usual suspects: OOB, UAF, double free, and memory corruption.

The rule of thumb is: no KMSAN if the patch primarily risks issues that KASAN/LOCKDEP already handle. I need to be sure that the patch either:
1.  Doesn't touch kernel structures exposed to user space, especially if there's structure padding.
2.  Doesn't introduce logic that uses uninitialized variables/struct fields.
3.  Doesn't allocate or initialize complex data structures in such a way that uninitialized fields could be read later.
4.  Doesn't introduce uninitialized bytes due to bounds checks/length logic.

And I need to greenlight KMSAN *only* if I see the code potentially leaking uninitialized memory.

Now, let's dive back in and examine these changes specifically looking for potential leak points. I was specifically looking at `bpf_insn_prepare_dump` in `kernel/bpf/syscall.c` because it is related to dumping instructions. In `bpf_insn_prepare_dump`, I see a section related to unwinding. What's happening here? Essentially, it is related to dumping the `BPF` instructions, and it looks like it is skipping the dumping of the instruction if it's the `unwind_goto` instruction. So, it's not looking like a KMSAN issue... but, I need to check further.



30/3 2026/10/20 09:18 tool grepper
0m Args:
{
  "Expression": "bpf_insn_prepare_dump",
  "PathPrefixes": [
    "kernel/bpf/syscall.c"
  ]
}

Results:
{
  "Output": "kernel/bpf/syscall.c=5044=static void prepare_dump_pseudo_call(struct bpf_insn *insn)\n--\nkernel/bpf/syscall.c-5066-\nkernel/bpf/syscall.c:5067:static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,\nkernel/bpf/syscall.c-5068-\t\t\t\t\t      const struct cred *f_cred)\n--\nkernel/bpf/syscall.c=5165=static int bpf_prog_get_info_by_fd(struct file *file,\n--\nkernel/bpf/syscall.c-5246-\t\tif (!prog-\u003eblinded || bpf_dump_raw_ok(file-\u003ef_cred)) {\nkernel/bpf/syscall.c:5247:\t\t\tinsns_sanitized = bpf_insn_prepare_dump(prog, file-\u003ef_cred);\nkernel/bpf/syscall.c-5248-\t\t\tif (!insns_sanitized)\n"
}

31/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 32751
							output: 37
							thoughts: 15
						

Thoughts:
Okay, here's my take on those thoughts, tailored for an expert audience:

**Analyzing `bpf_insn_prepare_dump`**

Alright, let's dive into `bpf_insn_prepare_dump`. I'm guessing this function is central to how we're visualizing or debugging BPF instructions, probably for a tool or within the kernel itself. My initial thought is it's likely responsible for taking a raw BPF instruction, or a series of them, and preparing them for human consumption – meaning formatting them into a readable form.

I'd expect it to handle things like resolving any immediate values, symbolic offsets, and perhaps even converting register numbers into something more meaningful, like the register's purpose. It also might be doing some pre-calculation of, say, instruction length, to determine the layout. It's almost certainly involved in handling opcode decoding to make the printout as understandable as possible. In other words, this function is the critical step of bridging the gap between raw BPF opcodes and a human-readable representation.



32/3 2026/10/20 09:18 tool read-file
0m Args:
{
  "File": "kernel/bpf/syscall.c",
  "FirstLine": 5067,
  "LineCount": 40
}

Results:
{
  "Contents": "5067:\tstatic struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog,\n5068:\t\t\t\t\t\t      const struct cred *f_cred)\n5069:\t{\n5070:\t\tconst struct bpf_map *map;\n5071:\t\tstruct bpf_insn *insns;\n5072:\t\tu32 off, type;\n5073:\t\tu64 imm;\n5074:\t\tu8 code;\n5075:\t\tint i;\n5076:\t\n5077:\t\tinsns = kmemdup(prog-\u003einsnsi, bpf_prog_insn_size(prog),\n5078:\t\t\t\tGFP_USER);\n5079:\t\tif (!insns)\n5080:\t\t\treturn insns;\n5081:\t\n5082:\t\tfor (i = 0; i \u003c prog-\u003elen; i++) {\n5083:\t\t\tcode = insns[i].code;\n5084:\t\n5085:\t\t\tif (code == (BPF_JMP | BPF_TAIL_CALL)) {\n5086:\t\t\t\tinsns[i].code = BPF_JMP | BPF_CALL;\n5087:\t\t\t\tinsns[i].imm = BPF_FUNC_tail_call;\n5088:\t\t\t\t/* fall-through */\n5089:\t\t\t}\n5090:\t\t\t/*\n5091:\t\t\t * BPF_JCOND is the same opcode as BPF_CALL_ARGS. The latter is for\n5092:\t\t\t * the interpreter, and a prog with unwind_goto is always JITed.\n5093:\t\t\t */\n5094:\t\t\tif (prog-\u003ejited \u0026\u0026 bpf_is_unwind_goto(\u0026insns[i]))\n5095:\t\t\t\tcontinue;\n5096:\t\t\tif (code == (BPF_JMP | BPF_CALL) ||\n5097:\t\t\t    code == (BPF_JMP | BPF_CALL_ARGS)) {\n5098:\t\t\t\t/* Restore the legacy xlated dump layout. */\n5099:\t\t\t\tif (insns[i].src_reg == BPF_PSEUDO_CALL)\n5100:\t\t\t\t\tprepare_dump_pseudo_call(\u0026insns[i]);\n5101:\t\t\t\tif (code == (BPF_JMP | BPF_CALL_ARGS))\n5102:\t\t\t\t\tinsns[i].code = BPF_JMP | BPF_CALL;\n5103:\t\t\t\tif (!bpf_dump_raw_ok(f_cred))\n5104:\t\t\t\t\tinsns[i].imm = 0;\n5105:\t\t\t\tcontinue;\n5106:\t\t\t}\n"
}

33/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 33725
							output: 49
							thoughts: 447
						

Thoughts:
**My Analysis of the BPF Unwind Issue**

Okay, so I'm looking at this patch and my initial thought is, `kmemdup(prog->insnsi, ...)` is duplicating existing instructions. The patch simply adds a check, `if (prog->jited && bpf_is_unwind_goto(&insns[i])) continue;`, to prevent it from being treated as `BPF_CALL_ARGS`. The buffer has already been allocated and initialized from `prog->insnsi`, so that part seems straightforward.

Now, I need to understand how the unwind extable entries are handled, specifically the X86 implementation in `add_unwind_exentry`. This looks interesting:

```c
static int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,
			      int ret_addr, int target)
{
	struct exception_table_entry *ex;
	s64 delta;

	if (!prog->aux->extable)
		return 0;
	if (*excnt >= prog->aux->num_exentries) {
		pr_err("unwind extable bug\n");
		return -EFAULT;
	}
	ex = &prog->aux->extable[(*excnt)++];
	delta = image + ret_addr - 1 - (u8 *)&ex->insn;
	if (!is_simm32(delta)) {
		pr_err("extable->insn doesn't fit into 32-bit\n");
		return -EFAULT;
	}
	/* switch ex to rw buffer for writes */
	ex = (void *)rw_image + ((void *)ex - (void *)image);
	ex->insn = delta;
	ex->fixup = target - ret_addr;
	ex->data = EX_TYPE_BPF;
	return 0;
}
```

Wait a second... I need to back up a bit and make sure I fully understand what's going on with `prog->aux->extable`. Where is this allocated in the first place? I need to track down the allocation of `prog->aux->extable` and ensure the `exception_table_entry` struct itself doesn't have any padding or uninitialized fields. That could introduce subtle vulnerabilities if we're not careful. Let's do a quick search in the codebase for where `aux->extable` is actually allocated.



34/3 2026/10/20 09:18 tool grepper
0m Args:
{
  "Expression": "aux-\u003eextable",
  "PathPrefixes": [
    "arch/x86/net/bpf_jit_comp.c",
    "kernel/bpf/"
  ]
}

Results:
{
  "Output": "arch/x86/net/bpf_jit_comp.c=1961=static int emit_arena_exentry(struct bpf_prog *bpf_prog, u8 *image, u8 *rw_image,\n--\narch/x86/net/bpf_jit_comp.c-1967-\narch/x86/net/bpf_jit_comp.c:1968:\tif (!bpf_prog-\u003eaux-\u003eextable)\narch/x86/net/bpf_jit_comp.c-1969-\t\treturn 0;\n--\narch/x86/net/bpf_jit_comp.c-1974-\t}\narch/x86/net/bpf_jit_comp.c:1975:\tex = \u0026bpf_prog-\u003eaux-\u003eextable[(*excnt)++];\narch/x86/net/bpf_jit_comp.c-1976-\n--\narch/x86/net/bpf_jit_comp.c=2523=static int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,\n--\narch/x86/net/bpf_jit_comp.c-2528-\narch/x86/net/bpf_jit_comp.c:2529:\tif (!prog-\u003eaux-\u003eextable)\narch/x86/net/bpf_jit_comp.c-2530-\t\treturn 0;\n--\narch/x86/net/bpf_jit_comp.c-2534-\t}\narch/x86/net/bpf_jit_comp.c:2535:\tex = \u0026prog-\u003eaux-\u003eextable[(*excnt)++];\narch/x86/net/bpf_jit_comp.c-2536-\tdelta = image + ret_addr - 1 - (u8 *)\u0026ex-\u003einsn;\n--\narch/x86/net/bpf_jit_comp.c=2576=static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,\n--\narch/x86/net/bpf_jit_comp.c-3321-\narch/x86/net/bpf_jit_comp.c:3322:\t\t\t\tif (!bpf_prog-\u003eaux-\u003eextable)\narch/x86/net/bpf_jit_comp.c-3323-\t\t\t\t\tbreak;\n--\narch/x86/net/bpf_jit_comp.c-3328-\t\t\t\t}\narch/x86/net/bpf_jit_comp.c:3329:\t\t\t\tex = \u0026bpf_prog-\u003eaux-\u003eextable[excnt++];\narch/x86/net/bpf_jit_comp.c-3330-\n--\narch/x86/net/bpf_jit_comp.c=4893=struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)\n--\narch/x86/net/bpf_jit_comp.c-5031-\t\t\t\tgoto out_addrs;\narch/x86/net/bpf_jit_comp.c:5032:\t\t\tprog-\u003eaux-\u003eextable = (void *) image + roundup(proglen, align);\narch/x86/net/bpf_jit_comp.c-5033-\t\t}\n--\nkernel/bpf/core.c=789=const struct exception_table_entry *search_bpf_extables(unsigned long addr)\n--\nkernel/bpf/core.c-800-\nkernel/bpf/core.c:801:\te = search_extable(prog-\u003eaux-\u003eextable, prog-\u003eaux-\u003enum_exentries, addr);\nkernel/bpf/core.c-802-out:\n--\nkernel/bpf/fixups.c=1345=static int jit_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1586-\tprog-\u003ejited_len = func[0]-\u003ejited_len;\nkernel/bpf/fixups.c:1587:\tprog-\u003eaux-\u003eextable = func[0]-\u003eaux-\u003eextable;\nkernel/bpf/fixups.c-1588-\tprog-\u003eaux-\u003enum_exentries = func[0]-\u003eaux-\u003enum_exentries;\n"
}

35/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 35930
							output: 45
							thoughts: 24
						

Thoughts:
Here's my thought process, as if I were considering this myself:

**Investigating `aux->extable` Allocation in `bpf_int_jit_compile`**

Okay, I need to understand exactly how `aux->extable` is allocated within `bpf_int_jit_compile`. This is a crucial detail, especially when dealing with potential issues related to memory management or JIT compilation. I want to make sure I'm fully aware of how this extension table is being handled. I'll need to dig into the function's implementation, and specifically focus on the code that deals with that `aux` structure and its `extable` member. I'll be looking for `malloc`, `kzalloc`, or any allocation calls, as well as the related deallocation to ensure no leaks or corruption could occur. I'm going to look for any nuances to ensure the table is properly sized and initialized for its intended purpose. I expect there is some allocation and initialization involved with managing extended program data. I need to fully grok this code.



36/3 2026/10/20 09:18 tool read-file
0m Args:
{
  "File": "arch/x86/net/bpf_jit_comp.c",
  "FirstLine": 5015,
  "LineCount": 35
}

Results:
{
  "Contents": "5015:\t\t\t\t * insns that access kernel memory via \"pointer to BTF type\".\n5016:\t\t\t\t * The verifier changed their opcode from LDX|MEM|size\n5017:\t\t\t\t * to LDX|PROBE_MEM|size to make JITing easier. Arena\n5018:\t\t\t\t * accesses add one each, and an arena fetching AND/OR/XOR\n5019:\t\t\t\t * one more, see the adjustment of num_exentries above.\n5020:\t\t\t\t */\n5021:\t\t\t\tu32 align = __alignof__(struct exception_table_entry);\n5022:\t\t\t\tu32 extable_size = prog-\u003eaux-\u003enum_exentries *\n5023:\t\t\t\t\tsizeof(struct exception_table_entry);\n5024:\t\n5025:\t\t\t\t/* allocate module memory for x86 insns and extable */\n5026:\t\t\t\theader = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,\n5027:\t\t\t\t\t\t\t\t   \u0026image, align, \u0026rw_header, \u0026rw_image,\n5028:\t\t\t\t\t\t\t\t   jit_fill_hole,\n5029:\t\t\t\t\t\t\t\t   bpf_prog_was_classic(prog));\n5030:\t\t\t\tif (!header)\n5031:\t\t\t\t\tgoto out_addrs;\n5032:\t\t\t\tprog-\u003eaux-\u003eextable = (void *) image + roundup(proglen, align);\n5033:\t\t\t}\n5034:\t\t\toldproglen = proglen;\n5035:\t\t\tcond_resched();\n5036:\t\t}\n5037:\t\n5038:\t\tif (bpf_jit_enable \u003e 1)\n5039:\t\t\tbpf_jit_dump(prog-\u003elen, proglen, pass + 1, rw_image);\n5040:\t\n5041:\t\tif (image) {\n5042:\t\t\tif (!prog-\u003eis_func || extra_pass) {\n5043:\t\t\t\t/*\n5044:\t\t\t\t * bpf_jit_binary_pack_finalize fails in two scenarios:\n5045:\t\t\t\t *   1) header is not pointing to proper module memory;\n5046:\t\t\t\t *   2) the arch doesn't support bpf_arch_text_copy().\n5047:\t\t\t\t *\n5048:\t\t\t\t * Both cases are serious bugs and justify WARN_ON.\n5049:\t\t\t\t */\n"
}

37/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 36837
							output: 41
							thoughts: 61
						

Thoughts:
Here's my summary, as though I'm thinking through this process:

**Diving into the BPF JIT Allocation**

Alright, so `bpf_jit_binary_pack_alloc` is initializing the memory it's setting up, and it looks like it's using `jit_fill_hole` to do that. Specifically, I see it's likely filling this allocated memory with `INT3` instructions, or `0xcc` bytes, if we're on an x86 architecture. That's a classic breakpoint instruction, used to signal a deliberate stop in execution for debugging purposes. Now, this is interesting, but not the core of what I was looking at.

The more pressing question right now is how `sort_extable` gets called. I remember that from the other day, and want to know what leads to it. So, let me quickly trace where that function is used within this section of code. I need to understand its context within this memory initialization process and the bigger picture of the JIT compilation. Let me check the code.



38/3 2026/10/20 09:18 tool grepper
0m Error:
no matches

Args:
{
  "Expression": "sort_extable",
  "PathPrefixes": [
    "arch/x86/net/bpf_jit_comp.c"
  ]
}

Results:
{
  "error": "no matches"
}

39/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 37181
							output: 41
							thoughts: 30
						

Thoughts:
Here's my thought process, as it is:

**Targeting `num_exentries` in `arch/x86/net/bpf_jit_comp.c`**

Okay, so I need to find something in the code, and my initial approach is to go straight to the source. The file `arch/x86/net/bpf_jit_comp.c` seems like the logical place to start digging into the x86-specific BPF JIT compilation. My plan is straightforward: let's use `grep` to quickly search within that file for the variable `num_exentries`. This direct search should quickly show me where it's defined, used, and, hopefully, give me some context surrounding it. It's a pragmatic, targeted approach; I'm not going to wander aimlessly when a specific search term can likely give me the information I need, fast. This kind of focused investigation is usually the most efficient starting point when you know the element's name.



40/3 2026/10/20 09:18 tool grepper
0m Args:
{
  "Expression": "num_exentries",
  "PathPrefixes": [
    "arch/x86/net/bpf_jit_comp.c"
  ]
}

Results:
{
  "Output": "arch/x86/net/bpf_jit_comp.c=1961=static int emit_arena_exentry(struct bpf_prog *bpf_prog, u8 *image, u8 *rw_image,\n--\narch/x86/net/bpf_jit_comp.c-1970-\narch/x86/net/bpf_jit_comp.c:1971:\tif (*excnt \u003e= bpf_prog-\u003eaux-\u003enum_exentries) {\narch/x86/net/bpf_jit_comp.c-1972-\t\tpr_err(\"arena extable bug\\n\");\n--\narch/x86/net/bpf_jit_comp.c=2523=static int add_unwind_exentry(struct bpf_prog *prog, u8 *image, u8 *rw_image, int *excnt,\n--\narch/x86/net/bpf_jit_comp.c-2530-\t\treturn 0;\narch/x86/net/bpf_jit_comp.c:2531:\tif (*excnt \u003e= prog-\u003eaux-\u003enum_exentries) {\narch/x86/net/bpf_jit_comp.c-2532-\t\tpr_err(\"unwind extable bug\\n\");\n--\narch/x86/net/bpf_jit_comp.c=2576=static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *addrs, u8 *image,\n--\narch/x86/net/bpf_jit_comp.c-3324-\narch/x86/net/bpf_jit_comp.c:3325:\t\t\t\tif (excnt \u003e= bpf_prog-\u003eaux-\u003enum_exentries) {\narch/x86/net/bpf_jit_comp.c-3326-\t\t\t\t\tpr_err(\"ex gen bug\\n\");\n--\narch/x86/net/bpf_jit_comp.c-3935-\narch/x86/net/bpf_jit_comp.c:3936:\tif (image \u0026\u0026 excnt != bpf_prog-\u003eaux-\u003enum_exentries) {\narch/x86/net/bpf_jit_comp.c-3937-\t\tpr_err(\"extable is not populated\\n\");\n--\narch/x86/net/bpf_jit_comp.c=4893=struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)\n--\narch/x86/net/bpf_jit_comp.c-4959-\t\t    is_atomic_fetch_op(insn))\narch/x86/net/bpf_jit_comp.c:4960:\t\t\tprog-\u003eaux-\u003enum_exentries++;\narch/x86/net/bpf_jit_comp.c-4961-\t}\n--\narch/x86/net/bpf_jit_comp.c-5018-\t\t\t * accesses add one each, and an arena fetching AND/OR/XOR\narch/x86/net/bpf_jit_comp.c:5019:\t\t\t * one more, see the adjustment of num_exentries above.\narch/x86/net/bpf_jit_comp.c-5020-\t\t\t */\narch/x86/net/bpf_jit_comp.c-5021-\t\t\tu32 align = __alignof__(struct exception_table_entry);\narch/x86/net/bpf_jit_comp.c:5022:\t\t\tu32 extable_size = prog-\u003eaux-\u003enum_exentries *\narch/x86/net/bpf_jit_comp.c-5023-\t\t\t\tsizeof(struct exception_table_entry);\n"
}

41/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 38443
							output: 42
							thoughts: 100
						

Thoughts:
Here's my thought process, as if I were considering this code:

**Analyzing the Unwind Entry Modification**

Alright, let's break this down. We're in `add_unwind_exentry`, which likely means we're dealing with exception handling and unwind information. I see we're modifying an `ex` entry, which I assume is an entry in the exception table.

First, the pointer arithmetic. We're essentially adjusting the `ex` pointer to be relative to `rw_image`. Then, `delta` is assigned to `ex->insn`, and `target - ret_addr` is assigned to `ex->fixup`. Finally, `ex->data` is set to `EX_TYPE_BPF`. This seems to be setting up an exception table entry to handle a BPF-related unwind operation. The `delta` likely indicates the offset of the instruction within a BPF program, and the `target - ret_addr` is probably calculating the necessary fixup for the return address. But I need to refresh my memory on the structure of these table entries. Specifically I need to check what `struct exception_table_entry` actually looks like on x86. I need to make sure I am understanding how the kernel represents its exception tables, and how the values in these table entries map to their actual use. Time to consult the definition.



42/3 2026/10/20 09:18 tool grepper
0m Args:
{
  "Expression": "struct exception_table_entry \\{",
  "PathPrefixes": [
    "arch/x86/include/asm/extable.h"
  ]
}

Results:
{
  "Output": "arch/x86/include/asm/extable.h-22-\narch/x86/include/asm/extable.h:23:struct exception_table_entry {\narch/x86/include/asm/extable.h-24-\tint insn, fixup, data;\n"
}

43/3 2026/10/20 09:18 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 38962
							output: 38
							thoughts: 1264
						

Thoughts:
**Unwinding BPF Subprograms: An Investigation**

Okay, so I'm looking at how unwinding is implemented for BPF subprograms, specifically focusing on x86_64. My initial thought is to understand the structure of the exception table entry, how the return address is modified, and how the verifier ensures the unwound path is valid.

First, I see the structure `exception_table_entry` on x86 has `insn`, `fixup`, and `data` fields. These are all explicitly assigned – nothing unexpected there. But, I noticed a discrepancy when I saw the struct definition on arm64 - are we dealing with cross-compilation here? It seems not - the target architecture is clearly x86_64.

Now, focusing on the unwinding logic itself, the crux is that a subprogram, exiting via `unwind_exit`, returns not to the instruction *after* the `call`, but to a designated "landing pad" specified by `unwind_goto`.

In the JIT'd x86_64 code, I'm observing a conditional jump (`BPF_JMP | BPF_JCOND`). If there's no private frame pointer, an `add_unwind_exentry` is called. If there is, it calls `add_unwind_exentry` with a slightly different offset, emits a jump instruction, and then a jump to the unwind target. Also, when encountering `BPF_JMP | BPF_EXIT` and `src_reg` is `BPF_UNWIND_EXIT`, I find the function `emit_unwind_ret_addr` being used.

`emit_unwind_ret_addr` is key. It pushes `rax` and `rsi` to preserve them, moves the return address from `rbp+8` to `rdi`, and then calls `bpf_unwind_target`. `bpf_unwind_target` is what replaces the original return address on the stack. It uses `search_bpf_extables` to look up the new target based on the old one, and then adds a `fixup` value to compute the actual landing pad address. Then, the `[rbp+8]` slot (the return address) is overwritten with the landing pad address!  The code then executes the subprogram epilogue (`leave; ret`), which pops `rbp` and then jumps to this new return address! Slick.

Next, I need to see how the verifier ensures the correctness of the unwound path. Specifically, I'm digging into `check_func_call`. The verifier checks that if a subprogram can unwind, there *must* be an `unwind_goto` instruction after the call. I see that the verifier does a `push_stack` when unwinding. The crucial point is: what is the state of registers and memory when we reach the `unwind_goto`? Specifically, the `push_stack` call is what I need to understand, which means I'll need to study the code in verifier.c. I'm focusing on the registers, and state changes that are relevant to this unwound path. I'll need to examine what happens to `branch`! Let's go spelunking in the verifier code...



44/3 2026/10/20 09:19 tool read-file
0m Args:
{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 11615,
  "LineCount": 45
}

Results:
{
  "Contents": "11615:\t\t\t */\n11616:\t\t\tif (!returns_void) {\n11617:\t\t\t\tnregs = bpf_ret_reg_pair(env, subprog) ? 2 : 1;\n11618:\t\t\t\tmark_reg_unknown(env, caller-\u003eregs, ret_regs[0]);\n11619:\t\t\t\tbpf_diag_mod_end(env);\n11620:\t\t\t\tfor (i = 1; i \u003c nregs; i++) {\n11621:\t\t\t\t\tbpf_diag_mod_begin(env, \u0026caller-\u003eregs[ret_regs[i]], NULL,\n11622:\t\t\t\t\t\t\t   BPF_DIAG_MOD_WRITE);\n11623:\t\t\t\t\tmark_reg_unknown(env, caller-\u003eregs, ret_regs[i]);\n11624:\t\t\t\t\tbpf_diag_mod_end(env);\n11625:\t\t\t\t}\n11626:\t\t\t}\n11627:\t\n11628:\t\t\tif (env-\u003esubprog_info[subprog].can_unwind) {\n11629:\t\t\t\tstruct bpf_verifier_state *branch;\n11630:\t\n11631:\t\t\t\tif (!bpf_is_unwind_goto(insn + 1)) {\n11632:\t\t\t\t\tverbose(env, \"no unwind_goto after the call of %s()\\n\", sub_name);\n11633:\t\t\t\t\treturn -EINVAL;\n11634:\t\t\t\t}\n11635:\t\t\t\tbranch = push_stack(env, *insn_idx + 1, *insn_idx, false);\n11636:\t\t\t\tif (IS_ERR(branch))\n11637:\t\t\t\t\treturn PTR_ERR(branch);\n11638:\t\t\t\tbranch-\u003eunwound = true;\n11639:\t\t\t}\n11640:\t\n11641:\t\t\tif (env-\u003esubprog_info[subprog].might_throw) {\n11642:\t\t\t\tstruct bpf_verifier_state *branch;\n11643:\t\n11644:\t\t\t\tbranch = push_stack(env, *insn_idx + 1, *insn_idx, false);\n11645:\t\t\t\tif (IS_ERR(branch)) {\n11646:\t\t\t\t\tverbose(env, \"failed to push state for global subprog exception path\\n\");\n11647:\t\t\t\t\treturn PTR_ERR(branch);\n11648:\t\t\t\t}\n11649:\t\t\t\treturn process_bpf_exit_full(env, NULL, true);\n11650:\t\t\t}\n11651:\t\n11652:\t\t\t/* continue with next insn after call */\n11653:\t\t\treturn 0;\n11654:\t\t}\n11655:\t\n11656:\t\treturn check_static_func_call(env, subprog, insn_idx);\n11657:\t}\n11658:\t\n11659:\t/*\n"
}

45/3 2026/10/20 09:19 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 41753
							output: 38
							thoughts: 380
						

Thoughts:
**Analyzing Function Exit Handling**

Okay, so I'm looking at how function exits are handled, specifically within `check_func_call`. It seems the logic in `check_func_call` prioritizes global function calls, because `check_static_func_call` is at the end of the check. That makes sense, but it immediately got me thinking about static function calls. Where does the static subprog exiting actually come into play?

I need to understand the exit procedure. Let's dig into the code, and trace that functionality in `prepare_func_exit`. Okay, here's the relevant section from the patch. I see some new logic here, specifically the checks related to `bpf_is_unwind_exit`. So, it looks like when a static subprog exits, `prepare_func_exit` is called. The new logic is catching some errors related to unwind exits. If a helper or kfunc doesn't expect the unwind, or if there's an `unwind_exit` without a corresponding `unwind_goto`, it's flagged as an error. And this will set the `state->unwound` flag. Got it.



46/3 2026/10/20 09:19 tool read-file
0m Args:
{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 12045,
  "LineCount": 45
}

Results:
{
  "Contents": "12045:\t{\n12046:\t\tstruct bpf_verifier_state *state = env-\u003ecur_state;\n12047:\t\tint frame;\n12048:\t\n12049:\t\tfor (frame = state-\u003ecurframe; frame \u003e= 0; frame--)\n12050:\t\t\taccount_processed_insns(env, state-\u003eframe[frame],\n12051:\t\t\t\t\t\tframe ? state-\u003eframe[frame - 1] : NULL);\n12052:\t}\n12053:\t\n12054:\t/*\n12055:\t * Are we currently verifying the callback for an rbtree kfunc that must\n12056:\t * be called with a lock held, or one of that callback's subprogs? If so,\n12057:\t * no need to complain about an unreleased lock.\n12058:\t */\n12059:\tstatic bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env)\n12060:\t{\n12061:\t\tstruct bpf_verifier_state *state = env-\u003ecur_state;\n12062:\t\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n12063:\t\tstruct bpf_func_state *callee;\n12064:\t\tint kfunc_btf_id;\n12065:\t\tu32 frame;\n12066:\t\n12067:\t\tfor (frame = state-\u003ecurframe; frame; frame--) {\n12068:\t\t\tcallee = state-\u003eframe[frame];\n12069:\t\t\tif (!callee-\u003ein_callback_fn)\n12070:\t\t\t\tcontinue;\n12071:\t\n12072:\t\t\tkfunc_btf_id = insn[callee-\u003ecallsite].imm;\n12073:\t\t\tif (is_rbtree_lock_required_kfunc(kfunc_btf_id))\n12074:\t\t\t\treturn true;\n12075:\t\t}\n12076:\t\n12077:\t\treturn false;\n12078:\t}\n12079:\t\n12080:\tstatic bool retval_range_within(struct bpf_retval_range range, const struct bpf_reg_state *reg)\n12081:\t{\n12082:\t\tif (range.return_32bit)\n12083:\t\t\treturn range.minval \u003c= reg_s32_min(reg) \u0026\u0026 reg_s32_max(reg) \u003c= range.maxval;\n12084:\t\telse\n12085:\t\t\treturn range.minval \u003c= reg_smin(reg) \u0026\u0026 reg_smax(reg) \u003c= range.maxval;\n12086:\t}\n12087:\t\n12088:\tstatic int prepare_func_exit(struct bpf_verifier_env *env, int *insn_idx)\n12089:\t{\n"
}

47/3 2026/10/20 09:19 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 43205
							output: 38
							thoughts: 14
						

Thoughts:
Okay, here's my interpretation of that thought process, assuming I'm already deep in the code as an expert:

**Diving Deeper into Function Exit Handling**

Alright, I'm thinking, "Okay, we're at the point where a function is about to exit. Let's not make assumptions here. I need to understand *precisely* what's going on within that `prepare_func_exit` function. I've got a mental model of the typical execution flow, but it's time to drill down and confirm all my assumptions. Is there any subtle edge cases in that part of the function that I'm missing? There’s no point in guessing. It's time to read the code, line by line if need be, in order to get the full picture. My focus now is on what happens *before* the function truly returns, any cleanup, memory management, logging, error handling, etc. I need to make sure this section behaves as I expect. Let's see what's what!



48/3 2026/10/20 09:19 tool read-file
0m Args:
{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 12090,
  "LineCount": 55
}

Results:
{
  "Contents": "12090:\t\tstruct bpf_verifier_state *state = env-\u003ecur_state, *prev_st;\n12091:\t\tstruct bpf_func_state *caller, *callee;\n12092:\t\tstruct bpf_reg_state *r0;\n12093:\t\tbool in_callback_fn;\n12094:\t\tu32 i, nregs;\n12095:\t\tint err;\n12096:\t\n12097:\t\tcallee = state-\u003eframe[state-\u003ecurframe];\n12098:\t\tr0 = \u0026callee-\u003eregs[BPF_REG_0];\n12099:\t\tnregs = bpf_ret_reg_pair(env, callee-\u003esubprogno) ? 2 : 1;\n12100:\t\tfor (i = 0; i \u003c nregs; i++) {\n12101:\t\t\tif (callee-\u003eregs[ret_regs[i]].type != PTR_TO_STACK)\n12102:\t\t\t\tcontinue;\n12103:\t\t\t/* technically it's ok to return caller's stack pointer\n12104:\t\t\t * (or caller's caller's pointer) back to the caller,\n12105:\t\t\t * since these pointers are valid. Only current stack\n12106:\t\t\t * pointer will be invalid as soon as function exits,\n12107:\t\t\t * but let's be conservative\n12108:\t\t\t */\n12109:\t\t\tverbose(env, \"cannot return stack pointer to the caller\\n\");\n12110:\t\t\treturn -EINVAL;\n12111:\t\t}\n12112:\t\n12113:\t\tcaller = state-\u003eframe[state-\u003ecurframe - 1];\n12114:\t\tif (bpf_is_unwind_exit(\u0026env-\u003eprog-\u003einsnsi[*insn_idx])) {\n12115:\t\t\t/* the helper or the kfunc that called it doesn't expect that */\n12116:\t\t\tif (callee-\u003ein_callback_fn) {\n12117:\t\t\t\tverbose(env, \"callback cannot unwind\\n\");\n12118:\t\t\t\treturn -EINVAL;\n12119:\t\t\t}\n12120:\t\t\tif (!bpf_is_unwind_goto(\u0026env-\u003eprog-\u003einsnsi[callee-\u003ecallsite + 1])) {\n12121:\t\t\t\tverbose(env, \"unwind_exit, but no unwind_goto after the call at insn %d\\n\",\n12122:\t\t\t\t\tcallee-\u003ecallsite);\n12123:\t\t\t\treturn -EINVAL;\n12124:\t\t\t}\n12125:\t\t\tstate-\u003eunwound = true;\n12126:\t\t}\n12127:\t\tif (callee-\u003ein_callback_fn) {\n12128:\t\t\tif (r0-\u003etype != SCALAR_VALUE) {\n12129:\t\t\t\tverbose(env, \"R0 not a scalar value\\n\");\n12130:\t\t\t\treturn -EACCES;\n12131:\t\t\t}\n12132:\t\n12133:\t\t\t/* we are going to rely on register's precise value */\n12134:\t\t\terr = mark_chain_precision(env, BPF_REG_0);\n12135:\t\t\tif (err)\n12136:\t\t\t\treturn err;\n12137:\t\n12138:\t\t\t/* enforce R0 return value range, and bpf_callback_t returns 64bit */\n12139:\t\t\tif (!retval_range_within(callee-\u003ecallback_ret_range, r0)) {\n12140:\t\t\t\tverbose_invalid_scalar(env, r0, callee-\u003ecallback_ret_range,\n12141:\t\t\t\t\t\t       \"At callback return\", \"R0\");\n12142:\t\t\t\treturn -EINVAL;\n12143:\t\t\t}\n12144:\t\t\tif (!bpf_calls_callback(env, callee-\u003ecallsite)) {\n"
}

49/3 2026/10/20 09:19 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 44507
							output: 38
							thoughts: 22
						

Thoughts:
Here's a summary of my thinking, presented as if it were my thought process:

**Diving into `prepare_func_exit` After the Callee Exits**

Okay, I'm ready to dig into the next step, specifically, what happens in `prepare_func_exit` now that `callee` has completed its execution. This is a crucial area for understanding the control flow and state management. I need to understand how the system transitions back to the caller, what cleanup is performed, and how return values (if any) are handled. I'm especially interested in the mechanics of stack unwinding (if applicable, depending on the architecture and language) and ensuring the integrity of the calling context. I'll need to look closely at registers, memory, and any relevant data structures modified during the callee's execution. Ultimately, I want to fully trace the control path to know how the program resumes.



50/3 2026/10/20 09:19 tool read-file
0m Args:
{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 12145,
  "LineCount": 45
}

Results:
{
  "Contents": "12145:\t\t\t\tverifier_bug(env, \"in callback at %d, callsite %d !calls_callback\",\n12146:\t\t\t\t\t     *insn_idx, callee-\u003ecallsite);\n12147:\t\t\t\treturn -EFAULT;\n12148:\t\t\t}\n12149:\t\t} else {\n12150:\t\t\t/*\n12151:\t\t\t * return to the caller whatever the callee had in the\n12152:\t\t\t * return register(s)\n12153:\t\t\t */\n12154:\t\t\tfor (i = 0; i \u003c nregs; i++) {\n12155:\t\t\t\tu32 regno = ret_regs[i];\n12156:\t\n12157:\t\t\t\tbpf_diag_mod_begin(env, \u0026caller-\u003eregs[regno], \u0026callee-\u003eregs[regno],\n12158:\t\t\t\t\t\t   BPF_DIAG_MOD_WRITE);\n12159:\t\t\t\tcaller-\u003eregs[regno] = callee-\u003eregs[regno];\n12160:\t\t\t\tbpf_diag_mod_end(env);\n12161:\t\t\t}\n12162:\t\t}\n12163:\t\n12164:\t\t/* Invalidate callee-local dynptrs and their slices before the frame goes away. */\n12165:\t\terr = destroy_dynptrs_in_stack_slots(env, callee, 0,\n12166:\t\t\t\t\t\t     callee-\u003eallocated_stack / BPF_REG_SIZE - 1);\n12167:\t\tif (err)\n12168:\t\t\treturn err;\n12169:\t\n12170:\t\t/* for callbacks like bpf_loop or bpf_for_each_map_elem go back to callsite,\n12171:\t\t * there function call logic would reschedule callback visit. If iteration\n12172:\t\t * converges is_state_visited() would prune that visit eventually.\n12173:\t\t */\n12174:\t\tin_callback_fn = callee-\u003ein_callback_fn;\n12175:\t\tif (in_callback_fn)\n12176:\t\t\t*insn_idx = callee-\u003ecallsite;\n12177:\t\telse\n12178:\t\t\t*insn_idx = callee-\u003ecallsite + 1;\n12179:\t\n12180:\t\tif (env-\u003elog.level \u0026 BPF_LOG_LEVEL) {\n12181:\t\t\tverbose(env, \"returning from callee:\\n\");\n12182:\t\t\tprint_verifier_state(env, state, callee-\u003eframeno, true);\n12183:\t\t\tverbose(env, \"to caller at %d:\\n\", *insn_idx);\n12184:\t\t\tprint_verifier_state(env, state, caller-\u003eframeno, true);\n12185:\t\t}\n12186:\t\taccount_processed_insns(env, callee, caller);\n12187:\t\t/* clear everything in the callee. In case of exceptional exits using\n12188:\t\t * bpf_throw, this will be done by copy_verifier_state for extra frames. */\n12189:\t\tfree_func_state(callee);\n"
}

51/3 2026/10/20 09:19 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 45602
							output: 35
							thoughts: 1057
						

Thoughts:
**Analyzing the Unwind Goto and KMSAN Implications**

Okay, so I'm looking closely at how `check_cond_jmp_op` handles `bpf_is_unwind_goto`. If the unwind happens, the verifier sets `env->cur_state->unwound` to `true`.  Then it's immediately set back to `false`, after sanitizing the speculative path and jumping to the pad. This makes sense; it's cleaning up after the `unwind_exit`. The test case demonstrates that after the unwind, R1-R5 are marked as not readable because they are caller-saved and their values are not guaranteed. R0 is either transferred, or unknown in global calls.

Now, I need to consider Kernel Memory Sanitization (KMSAN). I need to determine if there are any uninitialized kernel memory issues. I'll go through the criteria for triggering KMSAN:

1.  **Kernel structures to userspace:** My initial assessment is that this patch doesn't introduce any new kernel structures copied to user space.  `bpf_insn_prepare_dump` already exists. It seems to just bypass modifying `unwind_goto` instructions.

2.  **Conditional logic and uninitialized variables:** I need to make sure this isn't triggered by this patch.
    I've reviewed `add_unwind_exentry`, `bpf_unwind_target`, and `emit_unwind_ret_addr` in both x86 and arm64 JIT compilation. Local variables appear to be initialized. In the verifier (`check_unwind_insns`), I see that `seen` is initialized, and `i` is an integer.

    But, wait a minute!  The `bpf_verifier_state` struct now includes a `unwound` boolean.  I need to check where `bpf_verifier_state` is allocated and copied, to see if that `unwound` field is initialized correctly. So I need to go hunt for where `bpf_verifier_state` is allocated and copied. Let's see... the code shows it's copied in `bpf_copy_verifier_state`, and it's being copied, with the `unwound` member copied as well.



52/3 2026/10/20 09:19 tool grepper
0m Args:
{
  "Expression": "kzalloc.*bpf_verifier_state",
  "PathPrefixes": [
    "kernel/bpf/"
  ]
}

Results:
{
  "Output": "kernel/bpf/states.c=1296=int bpf_split_cur_state(struct bpf_verifier_env *env)\n--\nkernel/bpf/states.c-1304-\thead = bpf_explored_state(env, insn_idx);\nkernel/bpf/states.c:1305:\tnew_sl = kzalloc_obj(struct bpf_verifier_state_list, GFP_KERNEL_ACCOUNT);\nkernel/bpf/states.c-1306-\tif (!new_sl)\n--\nkernel/bpf/verifier.c=21754=static int do_check_common(struct bpf_verifier_env *env, int subprog,\n--\nkernel/bpf/verifier.c-21768-\nkernel/bpf/verifier.c:21769:\tstate = kzalloc_obj(struct bpf_verifier_state, GFP_KERNEL_ACCOUNT);\nkernel/bpf/verifier.c-21770-\tif (!state)\n"
}

53/3 2026/10/20 09:19 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 47442
							output: 33
							thoughts: 42
						

Thoughts:
Okay, let's dive into this `push_stack` function. I need to understand exactly how stack states are handled within it. My immediate thought is, "Where does the memory allocation happen?" I'll need to trace this back. The most likely place to start looking is the `push_stack` function itself, which I know resides in `kernel/bpf/verifier.c`. Let me open that file and see what's going on within `push_stack`. I'll need to examine the code carefully to pin down the allocation strategy used for the stack states. I'm looking for clues about memory management and initialization, especially how the verifier creates and manages these critical stack structures.



54/3 2026/10/20 09:19 tool grepper
0m Args:
{
  "Expression": "push_stack\\(",
  "PathPrefixes": [
    "kernel/bpf/verifier.c"
  ]
}

Results:
{
  "Output": "kernel/bpf/verifier.c=1866=static bool error_recoverable_with_nospec(int err)\n--\nkernel/bpf/verifier.c-1877-\nkernel/bpf/verifier.c:1878:static struct bpf_verifier_state *push_stack(struct bpf_verifier_env *env,\nkernel/bpf/verifier.c-1879-\t\t\t\t\t     int insn_idx, int prev_insn_idx,\n--\nkernel/bpf/verifier.c=2552=static void init_func_state(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-2564-\nkernel/bpf/verifier.c:2565:/* Similar to push_stack(), but for async callbacks */\nkernel/bpf/verifier.c-2566-static struct bpf_verifier_state *push_async_cb(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-2589-\t}\nkernel/bpf/verifier.c:2590:\t/* Unlike push_stack() do not bpf_copy_verifier_state().\nkernel/bpf/verifier.c-2591-\t * The caller state doesn't matter.\n--\nkernel/bpf/verifier.c=7111=static int check_func_ptr_read(struct bpf_verifier_env *env, struct bpf_reg_state *reg, int off,\n--\nkernel/bpf/verifier.c-7166-\tfor (i = 0; i \u003c n - 1; i++) {\nkernel/bpf/verifier.c:7167:\t\tbranch = push_stack(env, env-\u003einsn_idx + 1, env-\u003einsn_idx,\nkernel/bpf/verifier.c-7168-\t\t\t\t    env-\u003ecur_state-\u003especulative);\n--\nkernel/bpf/verifier.c=8925=static int process_iter_next_call(struct bpf_verifier_env *env, int insn_idx,\n--\nkernel/bpf/verifier.c-8957-\t\t/* branch out active iter state */\nkernel/bpf/verifier.c:8958:\t\tqueued_st = push_stack(env, insn_idx + 1, insn_idx, false);\nkernel/bpf/verifier.c-8959-\t\tif (IS_ERR(queued_st))\n--\nkernel/bpf/verifier.c=11400=static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-11464-\t */\nkernel/bpf/verifier.c:11465:\tcallback_state = push_stack(env, env-\u003esubprog_info[subprog].start, insn_idx, false);\nkernel/bpf/verifier.c-11466-\tif (IS_ERR(callback_state))\n--\nkernel/bpf/verifier.c=11533=static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-11634-\t\t\t}\nkernel/bpf/verifier.c:11635:\t\t\tbranch = push_stack(env, *insn_idx + 1, *insn_idx, false);\nkernel/bpf/verifier.c-11636-\t\t\tif (IS_ERR(branch))\n--\nkernel/bpf/verifier.c-11643-\nkernel/bpf/verifier.c:11644:\t\t\tbranch = push_stack(env, *insn_idx + 1, *insn_idx, false);\nkernel/bpf/verifier.c-11645-\t\t\tif (IS_ERR(branch)) {\n--\nkernel/bpf/verifier.c=12609=static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-13037-\t\t */\nkernel/bpf/verifier.c:13038:\t\tbranch = push_stack(env, env-\u003einsn_idx + 1, env-\u003einsn_idx, false);\nkernel/bpf/verifier.c-13039-\t\tif (IS_ERR(branch))\n--\nkernel/bpf/verifier.c-13115-\t\t\tmark_reg_scratched(env, BPF_REG_0);\nkernel/bpf/verifier.c:13116:\t\t\tbranch = push_stack(env, env-\u003einsn_idx + 1, env-\u003einsn_idx, false);\nkernel/bpf/verifier.c-13117-\t\t\tif (IS_ERR(branch))\n--\nkernel/bpf/verifier.c=15396=static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-15445-\nkernel/bpf/verifier.c:15446:\t\tbranch = push_stack(env, env-\u003einsn_idx + 1, env-\u003einsn_idx, false);\nkernel/bpf/verifier.c-15447-\t\tif (IS_ERR(branch)) {\n--\nkernel/bpf/verifier.c=16036=static int sanitize_speculative_path(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-16042-\nkernel/bpf/verifier.c:16043:\tbranch = push_stack(env, next_idx, curr_idx, true);\nkernel/bpf/verifier.c-16044-\tif (!IS_ERR(branch) \u0026\u0026 insn) {\n--\nkernel/bpf/verifier.c=17277=static int maybe_fork_scalars(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-17290-\nkernel/bpf/verifier.c:17291:\tbranch = push_stack(env, env-\u003einsn_idx, env-\u003einsn_idx, false);\nkernel/bpf/verifier.c-17292-\tif (IS_ERR(branch))\n--\nkernel/bpf/verifier.c=18808=static int check_cond_jmp_op(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-18852-\t\t/* branch out 'fallthrough' insn as a new state to explore */\nkernel/bpf/verifier.c:18853:\t\tqueued_st = push_stack(env, idx + 1, idx, false);\nkernel/bpf/verifier.c-18854-\t\tif (IS_ERR(queued_st))\n--\nkernel/bpf/verifier.c-18976-\nkernel/bpf/verifier.c:18977:\tother_branch = push_stack(env, *insn_idx + insn-\u003eoff + 1, *insn_idx, false);\nkernel/bpf/verifier.c-18978-\tif (IS_ERR(other_branch))\n--\nkernel/bpf/verifier.c=19220=static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-19296-\t\tmark_reg_scratched(env, BPF_REG_0);\nkernel/bpf/verifier.c:19297:\t\tbranch = push_stack(env, env-\u003einsn_idx + 1, env-\u003einsn_idx, false);\nkernel/bpf/verifier.c-19298-\t\tif (IS_ERR(branch))\n--\nkernel/bpf/verifier.c=20137=static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-20204-\t\tmark_indirect_target(env, env-\u003egotox_tmp_buf-\u003eitems[i]);\nkernel/bpf/verifier.c:20205:\t\tother_branch = push_stack(env, env-\u003egotox_tmp_buf-\u003eitems[i],\nkernel/bpf/verifier.c-20206-\t\t\t\t\t  env-\u003einsn_idx, env-\u003ecur_state-\u003especulative);\n"
}

55/3 2026/10/20 09:19 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 49799
							output: 37
							thoughts: 11
						

Thoughts:
Okay, let's dive into this. Here's my understanding, presented as a first-person thought process, keeping in mind my expertise:

**Checking the `push_stack` Implementation**

Alright, time to critically examine the `push_stack` function. I need to be meticulous here. I'm going to quickly review the core logic, looking for potential areas of concern, such as:

*   **Memory Management:** How is memory being allocated and deallocated for the stack's underlying data structure? Is there a risk of memory leaks, buffer overflows, or incorrect deallocation if an element isn't handled correctly during a push operation or stack expansion? That's always my first concern.
*   **Data Structures Used:** What is the underlying data structure used to implement this stack? Is it an array, a linked list, or something else? I need to quickly assess how efficiently this structure handles insertion at the "top" of the stack. Are we looking at O(1) time complexity, or something less optimal, such as O(n)?
*   **Error Handling:** What's the error handling like? Does it account for full stacks, invalid data, or other edge cases during the push operation? Is it robust enough to deal with memory allocation failures?
*   **Concurrency:** If this is a multi-threaded environment, I need to check how thread-safe the implementation is. Are there proper locking mechanisms to protect against race conditions? If not, that's a serious red flag.
*   **Edge Cases:** I'm also going to think about those obscure edge cases – what happens with empty stacks, large element sizes, or other unusual inputs? Need to be sure all assumptions are correct and tested.

I'll be specifically looking to make sure the implementation is aligned with best practices, maintainable, and performs according to what the requirements say.



56/3 2026/10/20 09:20 tool read-file
0m Args:
{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 1878,
  "LineCount": 40
}

Results:
{
  "Contents": "1878:\tstatic struct bpf_verifier_state *push_stack(struct bpf_verifier_env *env,\n1879:\t\t\t\t\t\t     int insn_idx, int prev_insn_idx,\n1880:\t\t\t\t\t\t     bool speculative)\n1881:\t{\n1882:\t\tstruct bpf_verifier_state *cur = env-\u003ecur_state;\n1883:\t\tstruct bpf_verifier_stack_elem *elem;\n1884:\t\tint err;\n1885:\t\n1886:\t\telem = kzalloc_obj(struct bpf_verifier_stack_elem, GFP_KERNEL_ACCOUNT);\n1887:\t\tif (!elem)\n1888:\t\t\treturn ERR_PTR(-ENOMEM);\n1889:\t\n1890:\t\telem-\u003einsn_idx = insn_idx;\n1891:\t\telem-\u003eprev_insn_idx = prev_insn_idx;\n1892:\t\telem-\u003enext = env-\u003ehead;\n1893:\t\telem-\u003elog_pos = env-\u003elog.end_pos;\n1894:\t\telem-\u003ediag_log_pos = bpf_diag_event_log_save(env);\n1895:\t\tenv-\u003ehead = elem;\n1896:\t\tenv-\u003estack_size++;\n1897:\t\terr = bpf_copy_verifier_state(\u0026elem-\u003est, cur);\n1898:\t\tif (err)\n1899:\t\t\treturn ERR_PTR(-ENOMEM);\n1900:\t\telem-\u003est.speculative |= speculative;\n1901:\t\tif (env-\u003estack_size \u003e BPF_COMPLEXITY_LIMIT_JMP_SEQ) {\n1902:\t\t\tverbose(env, \"The sequence of %d jumps is too complex.\\n\",\n1903:\t\t\t\tenv-\u003estack_size);\n1904:\t\t\treturn ERR_PTR(-E2BIG);\n1905:\t\t}\n1906:\t\tif (elem-\u003est.parent) {\n1907:\t\t\t++elem-\u003est.parent-\u003ebranches;\n1908:\t\t\t/* WARN_ON(branches \u003e 2) technically makes sense here,\n1909:\t\t\t * but\n1910:\t\t\t * 1. speculative states will bump 'branches' for non-branch\n1911:\t\t\t * instructions\n1912:\t\t\t * 2. is_state_visited() heuristics may decide not to create\n1913:\t\t\t * a new state for a sequence of branches and all such current\n1914:\t\t\t * and cloned states will be pointing to a single parent state\n1915:\t\t\t * which might have large 'branches' count.\n1916:\t\t\t */\n1917:\t\t}\n"
}

57/3 2026/10/20 09:20 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 50927
							output: 37
							thoughts: 67
						

Thoughts:
Here's my summary, as I see it:

**Examining BPF State Handling**

Okay, so I'm looking at how we're managing the BPF state, specifically in the context of splitting. Let me break this down. First, the `push_stack` function is using `kzalloc_obj` for allocation, which is standard practice in these scenarios to ensure that the memory is zeroed out. Then, it uses `bpf_copy_verifier_state` to perform a copy. The important detail here is that the function copies `unwound = src->unwound;`. This seems critical because the `unwound` field clearly holds some kind of state related to the unwinding process.

Now, let's turn our attention to `bpf_split_cur_state`. This is where the splitting logic is happening, so it's the right place to look. I need to dive deeper into this function to understand precisely *how* the state is being split and how the `unwound` value is handled during that process. The copy itself is a critical part, but splitting implies that there are *multiple* states being managed, and it's essential to understand how the `unwound` value interacts with these separate states. I'll need to trace the flow within `bpf_split_cur_state` to make sure the copy operation is consistent with the desired semantics.



58/3 2026/10/20 09:20 tool read-file
0m Args:
{
  "File": "kernel/bpf/states.c",
  "FirstLine": 1300,
  "LineCount": 30
}

Results:
{
  "Contents": "1300:\t\tstruct list_head *head;\n1301:\t\tint insn_idx = cur-\u003einsn_idx;\n1302:\t\tint err;\n1303:\t\n1304:\t\thead = bpf_explored_state(env, insn_idx);\n1305:\t\tnew_sl = kzalloc_obj(struct bpf_verifier_state_list, GFP_KERNEL_ACCOUNT);\n1306:\t\tif (!new_sl)\n1307:\t\t\treturn -ENOMEM;\n1308:\t\tenv-\u003etotal_states++;\n1309:\t\tenv-\u003eexplored_states_size++;\n1310:\t\tupdate_peak_states(env);\n1311:\t\tenv-\u003eprev_jmps_processed = env-\u003ejmps_processed;\n1312:\t\tenv-\u003eprev_insn_processed = env-\u003einsn_processed;\n1313:\t\n1314:\t\t/* forget precise markings we inherited, see __mark_chain_precision */\n1315:\t\tif (env-\u003ebpf_capable)\n1316:\t\t\tmark_all_scalars_imprecise(env, cur);\n1317:\t\n1318:\t\tbpf_clear_singular_ids(env, cur);\n1319:\t\n1320:\t\t/* add new state to the head of linked list */\n1321:\t\tnew = \u0026new_sl-\u003estate;\n1322:\t\terr = bpf_copy_verifier_state(new, cur);\n1323:\t\tif (err) {\n1324:\t\t\tbpf_free_verifier_state(new, false);\n1325:\t\t\tkfree(new_sl);\n1326:\t\t\treturn err;\n1327:\t\t}\n1328:\t\tnew-\u003einsn_idx = insn_idx;\n1329:\t\tverifier_bug_if(new-\u003ebranches != 1, env,\n"
}

59/3 2026/10/20 09:20 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 51821
							output: 40
							thoughts: 282
						

Thoughts:
**My Analysis of BPF Verifier State Equality and Unwinding**

Okay, so I'm digging into how the BPF verifier handles state equality, specifically around unwinding, and I'm seeing a change in `kernel/bpf/states.c`. The `states_equal` function now includes a check for `unwound`. This is important because it directly impacts whether the verifier considers two states identical. Since `kzalloc` is used for allocation, copying, and comparison, it is imperative to thoroughly comprehend these states.

Now, let's look at `bpf_prog_aux` for deeper context. Looking in `include/linux/bpf.h`, I find `bool can_unwind;`, indicating unwinding capability is part of the program's auxiliary information. I see that `prog->aux` is allocated with `kzalloc` which means it is initialized with zero which is important for comparing states.

Further examination of `jit_subprogs` reveals that `func[i]->aux->can_unwind = env->subprog_info[i].can_unwind;`. This means that information about unwinding is being propagated from the environment (`env`) to the program's auxiliary data. I see that `env->subprog_info` is allocated with `kvcalloc` in `alloc_subprog_info`, so I will check that now.

This suggests that the verifier is now explicitly tracking whether a BPF program or sub-program is considered "unwound," and this is critical for state equality determination. My next step is to understand what signals are involved in setting this flag. I'm going to look at the specifics of the `alloc_subprog_info` to get a grasp of its role in this system.



60/3 2026/10/20 09:20 tool grepper
0m Args:
{
  "Expression": "alloc_subprog_info|subprog_info",
  "PathPrefixes": [
    "kernel/bpf/verifier.c"
  ]
}

Results:
{
  "Output": "\nFull output is too long, showing 200 out of 296 lines.\nUse more precise expression or PathPrefixes if possible.\n\nNumber of matching lines per file (1 files in total):\nkernel/bpf/verifier.c:65\n\nkernel/bpf/verifier.c=440=static int bpf_compute_subprog_ret_regs(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-463-\t\tif (ret_regs_cnt(size) \u003e 1) {\nkernel/bpf/verifier.c:464:\t\t\tsubprog_info(env, subprog)-\u003eret_reg_pair = true;\nkernel/bpf/verifier.c-465-\t\t\tenv-\u003eprog-\u003ejit_required = 1;\n--\nkernel/bpf/verifier.c=483=void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/verifier.c-484-{\nkernel/bpf/verifier.c:485:\tstruct bpf_subprog_info *info = subprog_info(env, subprog);\nkernel/bpf/verifier.c-486-\n--\nkernel/bpf/verifier.c=492=static bool subprog_is_exc_cb(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/verifier.c-493-{\nkernel/bpf/verifier.c:494:\treturn subprog_info(env, subprog)-\u003eis_exception_cb;\nkernel/bpf/verifier.c-495-}\n--\nkernel/bpf/verifier.c=1466=static int resize_reference_state(struct bpf_verifier_state *state, size_t n)\n--\nkernel/bpf/verifier.c-1477-/* Possibly update state-\u003eallocated_stack to be at least size bytes. Also\nkernel/bpf/verifier.c:1478: * possibly update the function's high-water mark in its bpf_subprog_info.\nkernel/bpf/verifier.c-1479- */\nkernel/bpf/verifier.c=1480=int bpf_grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state *state, int size)\n--\nkernel/bpf/verifier.c-1497-\t/* update known max for given subprogram */\nkernel/bpf/verifier.c:1498:\tif (env-\u003esubprog_info[state-\u003esubprogno].stack_depth \u003c size)\nkernel/bpf/verifier.c:1499:\t\tenv-\u003esubprog_info[state-\u003esubprogno].stack_depth = size;\nkernel/bpf/verifier.c-1500-\n--\nkernel/bpf/verifier.c=2608=static int cmp_subprogs(const void *a, const void *b)\nkernel/bpf/verifier.c-2609-{\nkernel/bpf/verifier.c:2610:\treturn ((struct bpf_subprog_info *)a)-\u003estart -\nkernel/bpf/verifier.c:2611:\t       ((struct bpf_subprog_info *)b)-\u003estart;\nkernel/bpf/verifier.c-2612-}\n--\nkernel/bpf/verifier.c-2614-/* Find subprogram that contains instruction at 'off' */\nkernel/bpf/verifier.c:2615:struct bpf_subprog_info *bpf_find_containing_subprog(struct bpf_verifier_env *env, int off)\nkernel/bpf/verifier.c-2616-{\nkernel/bpf/verifier.c:2617:\tstruct bpf_subprog_info *vals = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2618-\tint l, r, m;\n--\nkernel/bpf/verifier.c=2636=int bpf_find_subprog(struct bpf_verifier_env *env, int off)\nkernel/bpf/verifier.c-2637-{\nkernel/bpf/verifier.c:2638:\tstruct bpf_subprog_info *p;\nkernel/bpf/verifier.c-2639-\n--\nkernel/bpf/verifier.c-2642-\t\treturn -ENOENT;\nkernel/bpf/verifier.c:2643:\treturn p - env-\u003esubprog_info;\nkernel/bpf/verifier.c-2644-}\n--\nkernel/bpf/verifier.c=2646=static int add_subprog(struct bpf_verifier_env *env, int off)\n--\nkernel/bpf/verifier.c-2662-\t/* determine subprog starts. The end is one before the next starts */\nkernel/bpf/verifier.c:2663:\tenv-\u003esubprog_info[env-\u003esubprog_cnt].exit_idx = U32_MAX;\nkernel/bpf/verifier.c:2664:\tenv-\u003esubprog_info[env-\u003esubprog_cnt++].start = off;\nkernel/bpf/verifier.c:2665:\tsort(env-\u003esubprog_info, env-\u003esubprog_cnt,\nkernel/bpf/verifier.c:2666:\t     sizeof(env-\u003esubprog_info[0]), cmp_subprogs, NULL);\nkernel/bpf/verifier.c-2667-\treturn env-\u003esubprog_cnt - 1;\n--\nkernel/bpf/verifier.c=3198=static int add_subprogs(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-3199-{\nkernel/bpf/verifier.c:3200:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3201-\tint i, ret, insn_cnt = env-\u003eprog-\u003elen, ex_cb_insn;\n--\nkernel/bpf/verifier.c-3263-\t\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/verifier.c:3264:\t\t\tif (env-\u003esubprog_info[i].start != ex_cb_insn)\nkernel/bpf/verifier.c-3265-\t\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=3310=static void find_subprog_properties(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-3311-{\nkernel/bpf/verifier.c:3312:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3313-\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=3338=static int check_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-3340-\tint i, subprog_start, subprog_end, off, cur_subprog = 0;\nkernel/bpf/verifier.c:3341:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3342-\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=3439=static int check_unwind_insns(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-3469-\tenv-\u003eprog-\u003ejit_required = 1;\nkernel/bpf/verifier.c:3470:\tenv-\u003eprog-\u003eaux-\u003ecan_unwind = env-\u003esubprog_info[0].can_unwind;\nkernel/bpf/verifier.c-3471-\treturn 0;\n--\nkernel/bpf/verifier.c=3485=static int sort_subprogs_topo(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-3486-{\nkernel/bpf/verifier.c:3487:\tstruct bpf_subprog_info *si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3488-\tint *insn_postorder = env-\u003ecfg.insn_postorder;\n--\nkernel/bpf/verifier.c=3874=static void check_fastcall_stack_contract(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-3876-{\nkernel/bpf/verifier.c:3877:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-3878-\tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/verifier.c=4593=static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_state *state,\n--\nkernel/bpf/verifier.c-4596-\tint max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;\nkernel/bpf/verifier.c:4597:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4598-\tint spi = -off / BPF_REG_SIZE - 1;\n--\nkernel/bpf/verifier.c=4636=static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_state *state,\n--\nkernel/bpf/verifier.c-4638-{\nkernel/bpf/verifier.c:4639:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4640-\tstruct bpf_verifier_state *vstate = env-\u003ecur_state;\n--\nkernel/bpf/verifier.c=5756=static int check_max_stack_depth_subprog(struct bpf_verifier_env *env, int idx,\n--\nkernel/bpf/verifier.c-5759-{\nkernel/bpf/verifier.c:5760:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-5761-\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=6006=static int check_max_stack_depth(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-6009-\tstruct bpf_subprog_call_depth_info *dinfo;\nkernel/bpf/verifier.c:6010:\tstruct bpf_subprog_info *si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-6011-\tbool priv_stack_supported;\n--\nkernel/bpf/verifier.c=6447=void bpf_adjust_func_ptrs_after_remove(struct bpf_verifier_env *env, u32 off, u32 len)\n--\nkernel/bpf/verifier.c-6461-\t\tsubprog = bpf_find_subprog(env, p-\u003exlated_off);\nkernel/bpf/verifier.c:6462:\t\tif (subprog \u003e 0 \u0026\u0026 env-\u003esubprog_info[subprog + 1].start \u003e off + len)\nkernel/bpf/verifier.c-6463-\t\t\tp-\u003exlated_off = off;\n--\nkernel/bpf/verifier.c=11238=static int setup_func_entry(struct bpf_verifier_env *env, int subprog, int callsite,\n--\nkernel/bpf/verifier.c-11285-\nkernel/bpf/verifier.c:11286:static void gen_subprog_arg_proto(const struct bpf_subprog_info *sub, const struct btf *btf,\nkernel/bpf/verifier.c-11287-\t\t\t\t  const struct btf_type *func_proto, struct bpf_func_proto *proto)\n--\nkernel/bpf/verifier.c=11318=static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog, struct btf *btf)\nkernel/bpf/verifier.c-11319-{\nkernel/bpf/verifier.c:11320:\tstruct bpf_subprog_info *sub = subprog_info(env, subprog);\nkernel/bpf/verifier.c-11321-\tstruct bpf_func_state *caller = cur_func(env);\n--\nkernel/bpf/verifier.c=11400=static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-11416-\t */\nkernel/bpf/verifier.c:11417:\tenv-\u003esubprog_info[subprog].is_cb = true;\nkernel/bpf/verifier.c-11418-\tif (bpf_pseudo_kfunc_call(insn) \u0026\u0026\n--\nkernel/bpf/verifier.c-11433-\t\t/* there is no real recursion here. timer and workqueue callbacks are async */\nkernel/bpf/verifier.c:11434:\t\tenv-\u003esubprog_info[subprog].is_async_cb = true;\nkernel/bpf/verifier.c:11435:\t\tasync_cb = push_async_cb(env, env-\u003esubprog_info[subprog].start,\nkernel/bpf/verifier.c-11436-\t\t\t\t\t insn_idx, subprog,\n--\nkernel/bpf/verifier.c-11464-\t */\nkernel/bpf/verifier.c:11465:\tcallback_state = push_stack(env, env-\u003esubprog_info[subprog].start, insn_idx, false);\nkernel/bpf/verifier.c-11466-\tif (IS_ERR(callback_state))\n--\nkernel/bpf/verifier.c=11488=static int check_static_func_call(struct bpf_verifier_env *env, int subprog,\n--\nkernel/bpf/verifier.c-11491-\tstruct bpf_verifier_state *state = env-\u003ecur_state;\nkernel/bpf/verifier.c:11492:\tstruct bpf_subprog_info *caller_info;\nkernel/bpf/verifier.c-11493-\tu16 callee_incoming, stack_arg_cnt;\n--\nkernel/bpf/verifier.c-11502-\t */\nkernel/bpf/verifier.c:11503:\tcaller_info = \u0026env-\u003esubprog_info[caller-\u003esubprogno];\nkernel/bpf/verifier.c:11504:\tcallee_incoming = bpf_in_stack_arg_cnt(\u0026env-\u003esubprog_info[subprog]);\nkernel/bpf/verifier.c-11505-\tstack_arg_cnt = bpf_in_stack_arg_cnt(caller_info) + callee_incoming;\n--\nkernel/bpf/verifier.c-11520-\t/* and go analyze first insn of the callee */\nkernel/bpf/verifier.c:11521:\t*insn_idx = env-\u003esubprog_info[subprog].start - 1;\nkernel/bpf/verifier.c-11522-\n--\nkernel/bpf/verifier.c=11533=static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-11565-\nkernel/bpf/verifier.c:11566:\t\tif (env-\u003esubprog_info[subprog].might_sleep \u0026\u0026 !in_sleepable_context(env)) {\nkernel/bpf/verifier.c-11567-\t\t\tconst char *suggestion;\n--\nkernel/bpf/verifier.c-11595-\t\treturns_void = subprog_returns_void(env, subprog);\nkernel/bpf/verifier.c:11596:\t\tif (env-\u003esubprog_info[subprog].changes_pkt_data)\nkernel/bpf/verifier.c-11597-\t\t\tclear_all_pkt_pointers(env);\n--\nkernel/bpf/verifier.c-11607-\t\t */\nkernel/bpf/verifier.c:11608:\t\tif (!env-\u003esubprog_info[subprog].changes_pkt_data \u0026\u0026\nkernel/bpf/verifier.c-11609-\t\t    (state-\u003eframe[0]-\u003esubprogno || holds_pkt_pointers(env)))\n--\nkernel/bpf/verifier.c-11627-\nkernel/bpf/verifier.c:11628:\t\tif (env-\u003esubprog_info[subprog].can_unwind) {\nkernel/bpf/verifier.c-11629-\t\t\tstruct bpf_verifier_state *branch;\n--\nkernel/bpf/verifier.c-11640-\nkernel/bpf/verifier.c:11641:\t\tif (env-\u003esubprog_info[subprog].might_throw) {\nkernel/bpf/verifier.c-11642-\t\t\tstruct bpf_verifier_state *branch;\n--\nkernel/bpf/verifier.c=12018=static void account_processed_insn(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-12023-\tframe-\u003einsns_subtotal++;\nkernel/bpf/verifier.c:12024:\tenv-\u003esubprog_info[frame-\u003esubprogno].insns_self++;\nkernel/bpf/verifier.c-12025-}\n--\nkernel/bpf/verifier.c=12027=static void account_processed_insns(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-12037-\nkernel/bpf/verifier.c:12038:\tenv-\u003esubprog_info[callee-\u003esubprogno].insns_total += insns;\nkernel/bpf/verifier.c-12039-\tif (caller)\n--\n\n"
}

61/3 2026/10/20 09:20 llm kmsan-evaluator
0m Model:
gemini-3.8-flash

Args:
null

Results:
null

Tokens:
							input: 57196
							output: 528
							thoughts: 3038
						

Thoughts:
**BPF Unwinding and the Need for KMSAN**

Okay, so I've been digging into this patch, and it's all about adding BPF unwinding support – essentially, enabling a way for BPF programs to handle exceptions and return from nested subprograms in a controlled way, like a landing pad. The patch involves modifications across the board: the BPF verifier, the JIT compilers (x86 and ARM64), and the instruction dumping mechanism.

My primary concern, of course, is memory safety and information leaks, so I need to go through everything with a fine-toothed comb. First, there's the interaction with userspace. The most relevant part there is the `bpf_insn_prepare_dump` function. It's used when dumping BPF program instructions via `bpf_prog_get_info_by_fd`. But I'm satisfied – the memory that gets sent to userspace is a `kmemdup` of existing instructions. No new uninitialized data is introduced.

Next, exception table entries. The JIT compilers are responsible for populating these. After reviewing the related code, I can confidently state that the JIT compiles zeroed/hole filled memory, and every field is initialized before they're looked up.

Then, the verifier state tracking: new fields like `unwound` and `can_unwind` are added to existing structures (`bpf_verifier_state`, `bpf_subprog_info`, and `bpf_prog_aux`). These are allocated using `kzalloc`/`kzalloc_obj`, and correctly copied in the relevant functions (`bpf_copy_verifier_state`, `states_equal`). So, I think it is safe to say no issues here.

Now, for those new functions I'm really concerned with, like `add_unwind_exentry`, `bpf_unwind_target`, `emit_unwind_ret_addr`, and the code in the verifier. All of the local variables are initialized, so I can rest easy.

Finally, the KMSAN question. Could any of the bugs introduced here be caught ONLY by KMSAN and not by other means? The most likely sources of potential bugs here are JIT compilation errors (e.g., generating incorrect exception table entries), control-flow problems, stack frame corruption, or issues within the verifier logic itself. However, these types of bugs would be detectable by KASAN (e.g., for out-of-bounds memory accesses), standard kernel assertions (like WARN_ON or BUG_ON), or generic execution faults. KMSAN specifically targets reads of uninitialized memory and information leaks to user space. Since those issues are not present here, I believe we don't need a dedicated KMSAN fuzzing session. This patch is safe.



62/3 2026/10/20 09:20 tool set-results
0m Args:
{
  "KMSANReasoning": "The patch series introduces support for BPF exception unwinding (`unwind_goto` and `unwind_exit`) across the BPF verifier, core runtime, and JIT compilers (x86_64 and arm64).\n\n1. Kernel-to-user info-leak assessment:\nThe only user-facing data transfer touched is in `bpf_insn_prepare_dump()`, where instructions are dumped to user space via `bpf_prog_get_info_by_fd()`. The instruction buffer is allocated using `kmemdup` from the existing `prog-\u003einsnsi` stream, and the patch merely adjusts handling of `BPF_JCOND` so that `unwind_goto` instructions are preserved rather than mistaken for `BPF_CALL_ARGS`. No uninitialized memory, padding, or uninitialized fields are exposed to user space.\n\n2. Uninitialized memory reads / branching:\n- In the JIT compilers (e.g. `add_unwind_exentry()`), all fields of `struct exception_table_entry` (`insn`, `fixup`, `data`) are fully initialized on creation.\n- In the verifier and runtime data structures, new fields such as `unwound` in `struct bpf_verifier_state` and `can_unwind` in `struct bpf_subprog_info` and `struct bpf_prog_aux` are allocated via `kzalloc`/`kzalloc_obj`, properly copied in `bpf_copy_verifier_state()`, and evaluated in `states_equal()`.\n- All newly introduced functions (`add_unwind_exentry()`, `bpf_unwind_target()`, `emit_unwind_ret_addr()`, `check_unwind_insns()`) explicitly initialize their local variables and arguments.\n\n3. KASAN vs KMSAN applicability:\nThe primary risks associated with this patch series involve JIT code generation correctness, return address manipulation on the stack frame, out-of-bounds extable indexing, or verifier branch simulation logic. These issues manifest as out-of-bounds accesses, stack corruption, or kernel panics/crashes, which are covered by KASAN and standard kernel debug facilities. There are no uninitialized memory read risks or info-leaks that specifically necessitate KMSAN fuzzing.",
  "NeedsKMSAN": false
}

Results:
{
  "KMSANReasoning": "The patch series introduces support for BPF exception unwinding (`unwind_goto` and `unwind_exit`) across the BPF verifier, core runtime, and JIT compilers (x86_64 and arm64).\n\n1. Kernel-to-user info-leak assessment:\nThe only user-facing data transfer touched is in `bpf_insn_prepare_dump()`, where instructions are dumped to user space via `bpf_prog_get_info_by_fd()`. The instruction buffer is allocated using `kmemdup` from the existing `prog-\u003einsnsi` stream, and the patch merely adjusts handling of `BPF_JCOND` so that `unwind_goto` instructions are preserved rather than mistaken for `BPF_CALL_ARGS`. No uninitialized memory, padding, or uninitialized fields are exposed to user space.\n\n2. Uninitialized memory reads / branching:\n- In the JIT compilers (e.g. `add_unwind_exentry()`), all fields of `struct exception_table_entry` (`insn`, `fixup`, `data`) are fully initialized on creation.\n- In the verifier and runtime data structures, new fields such as `unwound` in `struct bpf_verifier_state` and `can_unwind` in `struct bpf_subprog_info` and `struct bpf_prog_aux` are allocated via `kzalloc`/`kzalloc_obj`, properly copied in `bpf_copy_verifier_state()`, and evaluated in `states_equal()`.\n- All newly introduced functions (`add_unwind_exentry()`, `bpf_unwind_target()`, `emit_unwind_ret_addr()`, `check_unwind_insns()`) explicitly initialize their local variables and arguments.\n\n3. KASAN vs KMSAN applicability:\nThe primary risks associated with this patch series involve JIT code generation correctness, return address manipulation on the stack frame, out-of-bounds extable indexing, or verifier branch simulation logic. These issues manifest as out-of-bounds accesses, stack corruption, or kernel panics/crashes, which are covered by KASAN and standard kernel debug facilities. There are no uninitialized memory read risks or info-leaks that specifically necessitate KMSAN fuzzing.",
  "NeedsKMSAN": false
}

LLM Calls Summary:
Total Calls Total Tokens Avg Tokens Total Duration (Seconds) Avg Duration (Seconds)
Tool Calls Summary:
Total Calls Total Duration (Seconds) Avg Duration (Seconds)