| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/09/14 21:21 | flow | patch-triage |
5mArgs:null Results: {
"EnableConfigs": [
"BPF_SYSCALL",
"BPF_JIT"
],
"FocusSymbols": [
"bpf_check_cfg",
"compute_subprog_jts",
"bpf_free_subprog_jts",
"bpf_compute_scc",
"check_indirect_jump",
"bpf_insn_successors",
"bpf_insn_array_adjust",
"bpf_insn_array_init",
"bpf_patch_insn_data",
"bpf_opt_remove_nops"
],
"KMSANReasoning": "The patch refactors BPF indirect jump (gotox) handling and jump table verification in the BPF verifier:\n1. It consolidates jump tables to be per-subprogram (`subprog-\u003ejt`) rather than per-instruction in `insn_aux_data`, cleans up their lifecycle in `bpf_free_subprog_jts()`, and prevents gotox instructions from jumping outside their containing subprogram.\n2. In `bpf_compute_scc()`, it tracks the successor traversal resume position with `dfs_pos`, which is allocated using `kvcalloc` (zero-initialized) and freed with `kvfree`.\n3. In `bpf_insn_array.c`, `bpf_insn_array_init()` explicitly zeroes `jitted_off` and `ips[i]` to clear stale values on reuse.\n4. In `fixups.c`, it distinguishes prepend vs append instruction patching for adjusting indirect jump target flags and avoids removing NOPs if they are indirect jump targets.\n\nAll newly allocated buffers and modified structures (such as `dfs_pos`, `env-\u003ecfg`, and `env-\u003esubprog_info`) are allocated using zeroing allocators (`kvcalloc`, `kvzalloc_obj`), explicitly cleared, or initialized prior to use. Any potential bugs involving buffer bounds (e.g. `succ-\u003eitems`, `jt-\u003eitems`, or `subprog_info` indices) or pointer lifecycles would be out-of-bounds accesses or use-after-free bugs detected by standard KASAN. There is no risk of uninitialized memory reads or information disclosure to userspace, so a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false,
"Reasoning": "The patch refactors indirect jump (gotox) and jump table (BPF_MAP_TYPE_INSN_ARRAY) verification in the BPF verifier. It restructures CFG verification by computing jump tables per subprogram rather than per instruction, introduces an indirect jump edge complexity limit, updates DFS tracking in SCC computation, modifies instruction successor resolution for liveness tracking, and updates instruction patching to distinguish prepending versus appending while protecting indirect jump targets from dead code/NOP elimination. These paths are directly reachable from userspace via the bpf() syscall and warrant fuzzing to uncover potential verifier bypasses, out-of-bounds access, or verifier panics.",
"WorthFuzzing": true
} |
| 1/1 | 2026/09/14 21:21 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit 22a0a01defffa4374bd99018ad18557be4b6f8a5\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Mon Sep 14 21:21:27 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/include/linux/bpf.h b/include/linux/bpf.h\nindex e57af902560c3..8115f307444ad 100644\n--- a/include/linux/bpf.h\n+++ b/include/linux/bpf.h\n@@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);\n int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);\n int bpf_insn_array_ready(struct bpf_map *map);\n void bpf_insn_array_release(struct bpf_map *map);\n-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);\n+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);\n void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);\n \n #ifdef CONFIG_BPF_SYSCALL\ndiff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h\nindex 36b65797877d0..754e24d41f47f 100644\n--- a/include/linux/bpf_verifier.h\n+++ b/include/linux/bpf_verifier.h\n@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {\n \t\t/* remember the offset of node field within type to rewrite */\n \t\tu64 insert_off;\n \t};\n-\tstruct bpf_iarray *jt;\t/* jump table for gotox or bpf_tailcall call instruction */\n \tstruct btf_struct_meta *kptr_struct_meta;\n \tu64 map_key_state; /* constant (32 bit) key tracking for maps */\n \tint ctx_field_size; /* the ctx field size for load insn, maybe 0 */\n@@ -805,6 +804,7 @@ struct bpf_subprog_info {\n \tu32 linfo_idx; /* The idx to the main_prog-\u003eaux-\u003elinfo */\n \tu32 postorder_start; /* The idx to the env-\u003ecfg.insn_postorder */\n \tu32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */\n+\tstruct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */\n \tu16 stack_depth; /* max. stack depth used by this function */\n \tu16 stack_extra;\n \tu32 insns_total;\n@@ -977,6 +977,8 @@ struct bpf_verifier_env {\n \t\tint cur_stack;\n \t\t/* current position in the insn_postorder vector */\n \t\tint cur_postorder;\n+\t\tu32 gotox_edges;\n+\t\tbool subprog_jts_ready;\n \t} cfg;\n \tstruct backtrack_state bt;\n \tstruct bpf_jmp_history_entry *cur_hist_ent;\n@@ -1501,6 +1503,7 @@ bool bpf_is_throw_kfunc(struct bpf_insn *insn);\n int bpf_compute_const_regs(struct bpf_verifier_env *env);\n int bpf_prune_dead_branches(struct bpf_verifier_env *env);\n int bpf_check_cfg(struct bpf_verifier_env *env);\n+void bpf_free_subprog_jts(struct bpf_verifier_env *env);\n int bpf_compute_postorder(struct bpf_verifier_env *env);\n int bpf_compute_scc(struct bpf_verifier_env *env);\n \n@@ -1702,7 +1705,6 @@ struct bpf_kfunc_desc_tab {\n };\n \n /* Functions exported from verifier.c, used by fixups.c */\n-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len);\n void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog);\n bool bpf_allow_tail_call_in_subprogs(struct bpf_verifier_env *env);\n bool bpf_verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm);\ndiff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c\nindex a2f84afe6f7c6..350ecea8f1e85 100644\n--- a/kernel/bpf/bpf_insn_array.c\n+++ b/kernel/bpf/bpf_insn_array.c\n@@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)\n \t\treturn -EBUSY;\n \n \t/*\n-\t * Reset all the map indexes to the original values. This is needed,\n-\t * e.g., when a replay of verification with different log level should\n-\t * be performed.\n+\t * Reset the map to its pre-verification state. The xlated and jitted\n+\t * offsets and the jitted target pointers are recomputed by the verifier\n+\t * and the JIT for this program, so any values left by a previous owner\n+\t * must be cleared here.\n \t */\n-\tfor (i = 0; i \u003c map-\u003emax_entries; i++)\n+\tfor (i = 0; i \u003c map-\u003emax_entries; i++) {\n \t\tvalues[i].xlated_off = values[i].orig_off;\n+\t\tvalues[i].jitted_off = 0;\n+\t\tinsn_array-\u003eips[i] = 0;\n+\t}\n \n \treturn 0;\n }\n@@ -231,7 +235,7 @@ void bpf_insn_array_release(struct bpf_map *map)\n \tatomic_set(\u0026insn_array-\u003eused, 0);\n }\n \n-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)\n+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)\n {\n \tstruct bpf_insn_array *insn_array = cast_insn_array(map);\n \tint i;\n@@ -240,7 +244,7 @@ void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)\n \t\treturn;\n \n \tfor (i = 0; i \u003c map-\u003emax_entries; i++) {\n-\t\tif (insn_array-\u003evalues[i].xlated_off \u003c= off)\n+\t\tif (insn_array-\u003evalues[i].xlated_off \u003c first)\n \t\t\tcontinue;\n \t\tif (insn_array-\u003evalues[i].xlated_off == INSN_DELETED)\n \t\t\tcontinue;\ndiff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c\nindex 842c7d1eabccc..5a91d73d3e3fe 100644\n--- a/kernel/bpf/cfg.c\n+++ b/kernel/bpf/cfg.c\n@@ -9,6 +9,8 @@\n \n #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)\n \n+#define BPF_MAX_GOTOX_EDGES\tBPF_COMPLEXITY_LIMIT_INSNS\n+\n /* non-recursive DFS pseudo code\n * 1 procedure DFS-iterative(G,v):\n * 2 label v as discovered\n@@ -284,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)\n }\n \n /*\n- * Find and collect all maps which fit in the subprog. Return the result as one\n- * combined jump table in jt-\u003eitems (allocated with kvcalloc)\n+ * Collect the jump table of every subprogram that has one, as the combined\n+ * table of all maps whose targets land inside that subprogram. All gotox\n+ * instructions of a subprogram share the same table, so this is done in a\n+ * single pass over the maps rather than once per gotox.\n */\n-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,\n-\t\t\t\t\t int subprog_start, int subprog_end)\n+static int compute_subprog_jts(struct bpf_verifier_env *env)\n {\n-\tstruct bpf_iarray *jt = NULL;\n+\tstruct bpf_subprog_info *subprog;\n+\tstruct bpf_iarray *jt, *jt_cur;\n \tstruct bpf_map *map;\n-\tstruct bpf_iarray *jt_cur;\n+\tu32 old_cnt;\n \tint i;\n \n \tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++) {\n@@ -303,73 +307,84 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,\n \t\tmap = env-\u003einsn_array_maps[i];\n \n \t\tjt_cur = jt_from_map(map);\n-\t\tif (IS_ERR(jt_cur)) {\n-\t\t\tkvfree(jt);\n-\t\t\treturn jt_cur;\n+\t\tif (IS_ERR(jt_cur))\n+\t\t\treturn PTR_ERR(jt_cur);\n+\n+\t\tsubprog = bpf_find_containing_subprog(env, jt_cur-\u003eitems[0]);\n+\t\tif (!subprog) {\n+\t\t\tkvfree(jt_cur);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (jt_cur-\u003eitems[jt_cur-\u003ecnt - 1] \u003e= (subprog + 1)-\u003estart) {\n+\t\t\tverbose(env, \"jump table of subprog starting at %u spans multiple subprogs\\n\",\n+\t\t\t\tsubprog-\u003estart);\n+\t\t\tbpf_diag_program_structure(env, subprog-\u003estart, \"jump table spans subprograms\",\n+\t\t\t\t\"Keep every entry of a jump table inside one subprogram.\",\n+\t\t\t\t\"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.\",\n+\t\t\t\tsubprog-\u003estart, (subprog + 1)-\u003estart);\n+\t\t\tkvfree(jt_cur);\n+\t\t\treturn -EINVAL;\n \t\t}\n \n-\t\t/*\n-\t\t * This is enough to check one element. The full table is\n-\t\t * checked to fit inside the subprog later in create_jt()\n-\t\t */\n-\t\tif (jt_cur-\u003eitems[0] \u003e= subprog_start \u0026\u0026 jt_cur-\u003eitems[0] \u003c subprog_end) {\n-\t\t\tu32 old_cnt = jt ? jt-\u003ecnt : 0;\n-\t\t\tjt = bpf_iarray_realloc(jt, old_cnt + jt_cur-\u003ecnt);\n-\t\t\tif (!jt) {\n-\t\t\t\tkvfree(jt_cur);\n-\t\t\t\treturn ERR_PTR(-ENOMEM);\n-\t\t\t}\n-\t\t\tmemcpy(jt-\u003eitems + old_cnt, jt_cur-\u003eitems, jt_cur-\u003ecnt \u003c\u003c 2);\n+\t\told_cnt = subprog-\u003ejt ? subprog-\u003ejt-\u003ecnt : 0;\n+\t\tjt = bpf_iarray_realloc(subprog-\u003ejt, old_cnt + jt_cur-\u003ecnt);\n+\t\tif (!jt) {\n+\t\t\tsubprog-\u003ejt = NULL;\n+\t\t\tkvfree(jt_cur);\n+\t\t\treturn -ENOMEM;\n \t\t}\n+\t\tmemcpy(jt-\u003eitems + old_cnt, jt_cur-\u003eitems, jt_cur-\u003ecnt \u003c\u003c 2);\n+\t\tsubprog-\u003ejt = jt;\n \n \t\tkvfree(jt_cur);\n \t}\n \n-\tif (!jt) {\n-\t\tverbose(env, \"no jump tables found for subprog starting at %u\\n\", subprog_start);\n-\t\tbpf_diag_program_structure(\n-\t\t\tenv, subprog_start, \"missing jump table\",\n-\t\t\t\"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.\",\n-\t\t\t\"No jump table was found for the subprogram that starts at instruction %u.\",\n-\t\t\tsubprog_start);\n-\t\treturn ERR_PTR(-EINVAL);\n+\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\n+\t\tjt = env-\u003esubprog_info[i].jt;\n+\t\tif (jt)\n+\t\t\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\n \t}\n \n-\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\n-\treturn jt;\n+\tenv-\u003ecfg.subprog_jts_ready = true;\n+\treturn 0;\n+}\n+\n+void bpf_free_subprog_jts(struct bpf_verifier_env *env)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(env-\u003esubprog_info); i++) {\n+\t\tkvfree(env-\u003esubprog_info[i].jt);\n+\t\tenv-\u003esubprog_info[i].jt = NULL;\n+\t}\n+\tenv-\u003ecfg.subprog_jts_ready = false;\n }\n \n static struct bpf_iarray *\n-create_jt(int t, struct bpf_verifier_env *env)\n+subprog_jt(int t, struct bpf_verifier_env *env)\n {\n \tstruct bpf_subprog_info *subprog;\n-\tint subprog_start, subprog_end;\n-\tstruct bpf_iarray *jt;\n-\tint i;\n+\tint subprog_start, err;\n+\n+\tif (!env-\u003ecfg.subprog_jts_ready) {\n+\t\terr = compute_subprog_jts(env);\n+\t\tif (err)\n+\t\t\treturn ERR_PTR(err);\n+\t}\n \n \tsubprog = bpf_find_containing_subprog(env, t);\n \tsubprog_start = subprog-\u003estart;\n-\tsubprog_end = (subprog + 1)-\u003estart;\n-\tjt = jt_from_subprog(env, subprog_start, subprog_end);\n-\tif (IS_ERR(jt))\n-\t\treturn jt;\n \n-\t/* Check that the every element of the jump table fits within the given subprogram */\n-\tfor (i = 0; i \u003c jt-\u003ecnt; i++) {\n-\t\tif (jt-\u003eitems[i] \u003c subprog_start || jt-\u003eitems[i] \u003e= subprog_end) {\n-\t\t\tverbose(env, \"jump table for insn %d points outside of the subprog [%u,%u]\\n\",\n-\t\t\t\t\tt, subprog_start, subprog_end);\n-\t\t\tbpf_diag_program_structure(\n-\t\t\t\tenv, t, \"jump table target out of range\",\n-\t\t\t\t\"Keep every jump-table target inside the same subprogram.\",\n-\t\t\t\t\"The jump table for instruction %d points outside subprogram range [%u,%u).\",\n-\t\t\t\tt, subprog_start, subprog_end);\n-\t\t\tkvfree(jt);\n-\t\t\treturn ERR_PTR(-EINVAL);\n-\t\t}\n+\tif (!subprog-\u003ejt) {\n+\t\tverbose(env, \"no jump tables found for subprog starting at %u\\n\", subprog_start);\n+\t\tbpf_diag_program_structure(env, subprog_start, \"missing jump table\",\n+\t\t\t\"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.\",\n+\t\t\t\"No jump table was found for the subprogram that starts at instruction %u.\",\n+\t\t\tsubprog_start);\n+\t\treturn ERR_PTR(-EINVAL);\n \t}\n \n-\treturn jt;\n+\treturn subprog-\u003ejt;\n }\n \n /* \"conditional jump with N edges\" */\n@@ -381,13 +396,24 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)\n \tstruct bpf_iarray *jt;\n \tint i, w;\n \n-\tjt = env-\u003einsn_aux_data[t].jt;\n-\tif (!jt) {\n-\t\tjt = create_jt(t, env);\n-\t\tif (IS_ERR(jt))\n-\t\t\treturn PTR_ERR(jt);\n-\n-\t\tenv-\u003einsn_aux_data[t].jt = jt;\n+\tjt = subprog_jt(t, env);\n+\tif (IS_ERR(jt))\n+\t\treturn PTR_ERR(jt);\n+\n+\tif (!(insn_state[t] \u0026 BRANCH)) {\n+\t\tinsn_state[t] |= BRANCH;\n+\n+\t\tif (check_add_overflow(env-\u003ecfg.gotox_edges, jt-\u003ecnt,\n+\t\t\t\t \u0026env-\u003ecfg.gotox_edges) ||\n+\t\t env-\u003ecfg.gotox_edges \u003e BPF_MAX_GOTOX_EDGES) {\n+\t\t\tverbose(env, \"number of indirect jump edges in the program exceeds %u\\n\",\n+\t\t\t\tBPF_MAX_GOTOX_EDGES);\n+\t\t\tbpf_diag_program_structure(env, t, \"too many indirect jump edges\",\n+\t\t\t\t\"Reduce the number of indirect jumps, or the number of distinct targets they can reach.\",\n+\t\t\t\t\"The program has more than %u indirect jump edges in total, counted over every gotox instruction.\",\n+\t\t\t\tBPF_MAX_GOTOX_EDGES);\n+\t\t\treturn -E2BIG;\n+\t\t}\n \t}\n \n \tmark_prune_point(env, t);\n@@ -421,30 +447,6 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)\n \treturn keep_exploring ? KEEP_EXPLORING : DONE_EXPLORING;\n }\n \n-/*\n- * Instructions that can abnormally return from a subprog (tail_call\n- * upon success, ld_{abs,ind} upon load failure) have a hidden exit\n- * that the verifier must account for.\n- */\n-static int visit_abnormal_return_insn(struct bpf_verifier_env *env, int t)\n-{\n-\tstruct bpf_subprog_info *subprog;\n-\tstruct bpf_iarray *jt;\n-\n-\tif (env-\u003einsn_aux_data[t].jt)\n-\t\treturn 0;\n-\n-\tjt = bpf_iarray_realloc(NULL, 2);\n-\tif (!jt)\n-\t\treturn -ENOMEM;\n-\n-\tsubprog = bpf_find_containing_subprog(env, t);\n-\tjt-\u003eitems[0] = t + 1;\n-\tjt-\u003eitems[1] = subprog-\u003eexit_idx;\n-\tenv-\u003einsn_aux_data[t].jt = jt;\n-\treturn 0;\n-}\n-\n /* Visits the instruction at index t and returns one of the following:\n * \u003c 0 - an error occurred\n * DONE_EXPLORING - the instruction was fully explored\n@@ -461,13 +463,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)\n \t/* All non-branch instructions have a single fall-through edge. */\n \tif (BPF_CLASS(insn-\u003ecode) != BPF_JMP \u0026\u0026\n \t BPF_CLASS(insn-\u003ecode) != BPF_JMP32) {\n-\t\tif (BPF_CLASS(insn-\u003ecode) == BPF_LD \u0026\u0026\n-\t\t (BPF_MODE(insn-\u003ecode) == BPF_ABS ||\n-\t\t BPF_MODE(insn-\u003ecode) == BPF_IND)) {\n-\t\t\tret = visit_abnormal_return_insn(env, t);\n-\t\t\tif (ret)\n-\t\t\t\treturn ret;\n-\t\t}\n \t\tinsn_sz = bpf_is_ldimm64(insn) ? 2 : 1;\n \t\treturn push_insn(t, t + insn_sz, FALLTHROUGH, env);\n \t}\n@@ -512,11 +507,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)\n \t\t\t\tmark_subprog_might_sleep(env, t);\n \t\t\tif (bpf_helper_changes_pkt_data(insn-\u003eimm))\n \t\t\t\tmark_subprog_changes_pkt_data(env, t);\n-\t\t\tif (insn-\u003eimm == BPF_FUNC_tail_call) {\n-\t\t\t\tret = visit_abnormal_return_insn(env, t);\n-\t\t\t\tif (ret)\n-\t\t\t\t\treturn ret;\n-\t\t\t}\n \t\t} else if (insn-\u003esrc_reg == BPF_PSEUDO_KFUNC_CALL) {\n \t\t\tstruct bpf_call_arg_meta meta;\n \n@@ -749,7 +739,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n \tconst u32 insn_cnt = env-\u003eprog-\u003elen;\n \tint stack_sz, dfs_sz, err = 0;\n-\tu32 *stack, *pre, *low, *dfs;\n+\tu32 *stack, *pre, *low, *dfs, *dfs_pos;\n \tu32 i, j, t, w;\n \tu32 next_preorder_num;\n \tu32 next_scc_id;\n@@ -762,13 +752,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \t * - 'stack' accumulates vertices in DFS order, see invariant comment below;\n \t * - 'pre[t] == p' =\u003e preorder number of vertex 't' is 'p';\n \t * - 'low[t] == n' =\u003e smallest preorder number of the vertex reachable from 't' is 'n';\n-\t * - 'dfs' DFS traversal stack, used to emulate explicit recursion.\n+\t * - 'dfs' DFS traversal stack, used to emulate explicit recursion;\n+\t * - 'dfs_pos[k] == j' =\u003e the frame 'dfs[k]' resumes visiting its\n+\t * successors at index 'j'.\n \t */\n \tstack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tpre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tlow = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tdfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);\n-\tif (!stack || !pre || !low || !dfs) {\n+\tdfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);\n+\tif (!stack || !pre || !low || !dfs || !dfs_pos) {\n \t\terr = -ENOMEM;\n \t\tgoto exit;\n \t}\n@@ -851,6 +844,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \t\tstack_sz = 0;\n \t\tdfs_sz = 1;\n \t\tdfs[0] = i;\n+\t\tdfs_pos[0] = 0;\n dfs_continue:\n \t\twhile (dfs_sz) {\n \t\t\tw = dfs[dfs_sz - 1];\n@@ -860,13 +854,22 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \t\t\t\tnext_preorder_num++;\n \t\t\t\tstack[stack_sz++] = w;\n \t\t\t}\n-\t\t\t/* Visit 'w' successors */\n+\t\t\t/* Visit remaining 'w' successors */\n \t\t\tsucc = bpf_insn_successors(env, w);\n-\t\t\tfor (j = 0; j \u003c succ-\u003ecnt; ++j) {\n+\t\t\tfor (j = dfs_pos[dfs_sz - 1]; j \u003c succ-\u003ecnt; ++j) {\n \t\t\t\tif (pre[succ-\u003eitems[j]]) {\n \t\t\t\t\tlow[w] = min(low[w], low[succ-\u003eitems[j]]);\n \t\t\t\t} else {\n-\t\t\t\t\tdfs[dfs_sz++] = succ-\u003eitems[j];\n+\t\t\t\t\t/*\n+\t\t\t\t\t * Once DFS for succ-\u003eitems[j] is complete,\n+\t\t\t\t\t * pre[succ-\u003eitems[j]] is non-zero, hence\n+\t\t\t\t\t * resuming at 'j' allows to follow the\n+\t\t\t\t\t * low[w] = min(...) update branch above.\n+\t\t\t\t\t */\n+\t\t\t\t\tdfs_pos[dfs_sz - 1] = j;\n+\t\t\t\t\tdfs_pos[dfs_sz] = 0;\n+\t\t\t\t\tdfs[dfs_sz] = succ-\u003eitems[j];\n+\t\t\t\t\tdfs_sz++;\n \t\t\t\t\tgoto dfs_continue;\n \t\t\t\t}\n \t\t\t}\n@@ -916,5 +919,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \tkvfree(pre);\n \tkvfree(low);\n \tkvfree(dfs);\n+\tkvfree(dfs_pos);\n \treturn err;\n }\ndiff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c\nindex 52d3cec336727..e132bd0600c7e 100644\n--- a/kernel/bpf/fixups.c\n+++ b/kernel/bpf/fixups.c\n@@ -213,12 +213,18 @@ static int get_callee_stack_depth(struct bpf_verifier_env *env,\n }\n #endif\n \n+enum bpf_patch_mode {\n+\tBPF_APPEND,\n+\tBPF_PREPEND,\n+};\n+\n /* single env-\u003eprog-\u003einsni[off] instruction was replaced with the range\n * insni[off, off + cnt). Adjust corresponding insn_aux_data by copying\n * [0, off) and [off, end) to new locations, so the patched range stays zero\n */\n static void adjust_insn_aux_data(struct bpf_verifier_env *env,\n-\t\t\t\t struct bpf_prog *new_prog, u32 off, u32 cnt)\n+\t\t\t\t struct bpf_prog *new_prog, u32 off, u32 cnt,\n+\t\t\t\t enum bpf_patch_mode mode)\n {\n \tstruct bpf_insn_aux_data *data = env-\u003einsn_aux_data;\n \tstruct bpf_insn *insn = new_prog-\u003einsnsi;\n@@ -251,9 +257,10 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,\n \t * new instructions by the above memmove and memset, but the indirect jump target is\n \t * actually the first instruction, so move it back. This also matches with the behavior\n \t * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new\n-\t * instruction.\n+\t * instruction. For BPF_PREPEND the original instruction is the last one, so the flag\n+\t * already sits where needed.\n \t */\n-\tif (data[off + cnt - 1].indirect_target) {\n+\tif (mode == BPF_APPEND \u0026\u0026 data[off + cnt - 1].indirect_target) {\n \t\tdata[off].indirect_target = 1;\n \t\tdata[off + cnt - 1].indirect_target = 0;\n \t}\n@@ -273,7 +280,7 @@ static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len\n \t}\n }\n \n-static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)\n+static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 first, u32 len)\n {\n \tint i;\n \n@@ -281,7 +288,7 @@ static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)\n \t\treturn;\n \n \tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++)\n-\t\tbpf_insn_array_adjust(env-\u003einsn_array_maps[i], off, len);\n+\t\tbpf_insn_array_adjust(env-\u003einsn_array_maps[i], first, len);\n }\n \n static void adjust_insn_arrays_after_remove(struct bpf_verifier_env *env, u32 off, u32 len)\n@@ -306,8 +313,9 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)\n \t}\n }\n \n-struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,\n-\t\t\t\t const struct bpf_insn *patch, u32 len)\n+static struct bpf_prog *__bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,\n+\t\t\t\t\t const struct bpf_insn *patch, u32 len,\n+\t\t\t\t\t enum bpf_patch_mode mode)\n {\n \tstruct bpf_prog *new_prog;\n \tstruct bpf_insn_aux_data *new_data = NULL;\n@@ -331,13 +339,19 @@ struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,\n \t\t\t\tenv-\u003einsn_aux_data[off].orig_idx);\n \t\treturn NULL;\n \t}\n-\tadjust_insn_aux_data(env, new_prog, off, len);\n+\tadjust_insn_aux_data(env, new_prog, off, len, mode);\n \tadjust_subprog_starts(env, off, len);\n-\tadjust_insn_arrays(env, off, len);\n+\tadjust_insn_arrays(env, mode == BPF_PREPEND ? off : off + 1, len);\n \tadjust_poke_descs(new_prog, off, len);\n \treturn new_prog;\n }\n \n+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,\n+\t\t\t\t const struct bpf_insn *patch, u32 len)\n+{\n+\treturn __bpf_patch_insn_data(env, off, patch, len, BPF_APPEND);\n+}\n+\n /*\n * For all jmp insns in a given 'prog' that point to 'tgt_idx' insn adjust the\n * jump offset by 'delta'.\n@@ -500,23 +514,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,\n \treturn 0;\n }\n \n-/*\n- * Clean up dynamically allocated fields of aux data for instructions [start, ...]\n- */\n-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)\n-{\n-\tstruct bpf_insn_aux_data *aux_data = env-\u003einsn_aux_data;\n-\tint end = start + len;\n-\tint i;\n-\n-\tfor (i = start; i \u003c end; i++) {\n-\t\tif (aux_data[i].jt) {\n-\t\t\tkvfree(aux_data[i].jt);\n-\t\t\taux_data[i].jt = NULL;\n-\t\t}\n-\t}\n-}\n-\n static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)\n {\n \tstruct bpf_insn_aux_data *aux_data = env-\u003einsn_aux_data;\n@@ -526,8 +523,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)\n \tif (bpf_prog_is_offloaded(env-\u003eprog-\u003eaux))\n \t\tbpf_prog_offload_remove_insns(env, off, cnt);\n \n-\tbpf_clear_insn_aux_data(env, off, cnt);\n-\n \terr = bpf_remove_insns(env-\u003eprog, off, cnt);\n \tif (err)\n \t\treturn err;\n@@ -618,6 +613,7 @@ int bpf_opt_remove_dead_code(struct bpf_verifier_env *env)\n \n int bpf_opt_remove_nops(struct bpf_verifier_env *env)\n {\n+\tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n \tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n \tint insn_cnt = env-\u003eprog-\u003elen;\n \tbool is_may_goto_0, is_ja;\n@@ -629,6 +625,8 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)\n \n \t\tif (!is_may_goto_0 \u0026\u0026 !is_ja)\n \t\t\tcontinue;\n+\t\tif (aux[i].indirect_target)\n+\t\t\tcontinue;\n \n \t\terr = verifier_remove_insns(env, i, 1);\n \t\tif (err)\n@@ -771,7 +769,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)\n \t\t\tinsn_buf[cnt++] = BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1,\n \t\t\t\t\t\t -subprogs[0].stack_depth);\n \t\t\tinsn_buf[cnt++] = env-\u003eprog-\u003einsnsi[0];\n-\t\t\tnew_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);\n+\t\t\tnew_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,\n+\t\t\t\t\t\t\t BPF_PREPEND);\n \t\t\tif (!new_prog)\n \t\t\t\treturn -ENOMEM;\n \t\t\tenv-\u003eprog = new_prog;\n@@ -794,7 +793,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)\n \t\t\tverifier_bug(env, \"prologue is too long\");\n \t\t\treturn -EFAULT;\n \t\t} else if (cnt) {\n-\t\t\tnew_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);\n+\t\t\tnew_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,\n+\t\t\t\t\t\t\t BPF_PREPEND);\n \t\t\tif (!new_prog)\n \t\t\t\treturn -ENOMEM;\n \n@@ -2459,7 +2459,8 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env)\n \t\t/* Copy first actual insn to preserve it */\n \t\tinsn_buf[cnt++] = env-\u003eprog-\u003einsnsi[subprog_start];\n \n-\t\tnew_prog = bpf_patch_insn_data(env, subprog_start, insn_buf, cnt);\n+\t\tnew_prog = __bpf_patch_insn_data(env, subprog_start, insn_buf, cnt,\n+\t\t\t\t\t\t BPF_PREPEND);\n \t\tif (!new_prog)\n \t\t\treturn -ENOMEM;\n \t\tenv-\u003eprog = prog = new_prog;\ndiff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c\nindex 301fc60bddc43..08592fdb55103 100644\n--- a/kernel/bpf/liveness.c\n+++ b/kernel/bpf/liveness.c\n@@ -248,9 +248,12 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)\n \tstruct bpf_iarray *succ, *jt;\n \tint insn_sz;\n \n-\tjt = env-\u003einsn_aux_data[idx].jt;\n-\tif (unlikely(jt))\n-\t\treturn jt;\n+\t/* All gotox of a subprogram share its jump table, see compute_subprog_jts() */\n+\tif (unlikely(insn_is_gotox(insn))) {\n+\t\tjt = bpf_find_containing_subprog(env, idx)-\u003ejt;\n+\t\tif (jt)\n+\t\t\treturn jt;\n+\t}\n \n \t/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */\n \tsucc = env-\u003esucc;\n@@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)\n \tif (opcode_info-\u003ecan_jump)\n \t\tsucc-\u003eitems[succ-\u003ecnt++] = idx + bpf_jmp_offset(insn) + 1;\n \n+\t/*\n+\t * tail_call upon success and ld_{abs,ind} upon load failure return\n+\t * from the subprogram, which is a hidden edge to its exit.\n+\t */\n+\tif (unlikely((bpf_helper_call(insn) \u0026\u0026 insn-\u003eimm == BPF_FUNC_tail_call) ||\n+\t\t (BPF_CLASS(insn-\u003ecode) == BPF_LD \u0026\u0026\n+\t\t (BPF_MODE(insn-\u003ecode) == BPF_ABS || BPF_MODE(insn-\u003ecode) == BPF_IND))))\n+\t\tsucc-\u003eitems[succ-\u003ecnt++] = bpf_find_containing_subprog(env, idx)-\u003eexit_idx;\n+\n \treturn succ;\n }\n \ndiff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c\nindex ddba53eaa3331..dc3375fcfab9d 100644\n--- a/kernel/bpf/verifier.c\n+++ b/kernel/bpf/verifier.c\n@@ -18191,11 +18191,43 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,\n \treturn 0;\n }\n \n+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */\n+static bool jt_contains(const struct bpf_iarray *jt, u32 target)\n+{\n+\tint l = 0, r = jt-\u003ecnt - 1, m;\n+\n+\twhile (l \u003c= r) {\n+\t\tm = l + (r - l) / 2;\n+\t\tif (jt-\u003eitems[m] == target)\n+\t\t\treturn true;\n+\t\tif (jt-\u003eitems[m] \u003c target)\n+\t\t\tl = m + 1;\n+\t\telse\n+\t\t\tr = m - 1;\n+\t}\n+\treturn false;\n+}\n+\n+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,\n+\t\t\t\t u32 subprog_start, u32 subprog_end)\n+{\n+\tverbose(env, \"indirect jump from insn %d to %u leaves the subprog [%u,%u)\\n\",\n+\t\t env-\u003einsn_idx, target, subprog_start, subprog_end);\n+\tbpf_diag_program_structure(env, env-\u003einsn_idx, \"indirect jump leaves subprogram\",\n+\t\t\"Keep every reachable jump-table target inside the subprogram of the indirect jump.\",\n+\t\t\"Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).\",\n+\t\tenv-\u003einsn_idx, target, subprog_start, subprog_end);\n+\treturn -EINVAL;\n+}\n+\n /* gotox *dst_reg */\n static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n {\n \tstruct bpf_verifier_state *other_branch;\n+\tstruct bpf_subprog_info *subprog;\n+\tu32 subprog_start, subprog_end;\n \tstruct bpf_reg_state *dst_reg;\n+\tstruct bpf_iarray *jt;\n \tstruct bpf_map *map;\n \tu32 min_index, max_index;\n \tint err = 0;\n@@ -18238,6 +18270,29 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in\n \t\treturn -EINVAL;\n \t}\n \n+\tsubprog = bpf_find_containing_subprog(env, env-\u003einsn_idx);\n+\tif (verifier_bug_if(!subprog, env, \"no subprog contains insn %d\", env-\u003einsn_idx))\n+\t\treturn -EFAULT;\n+\tsubprog_start = subprog-\u003estart;\n+\tsubprog_end = (subprog + 1)-\u003estart;\n+\n+\tjt = subprog-\u003ejt;\n+\tif (verifier_bug_if(!jt, env, \"no jump table for insn %d\", env-\u003einsn_idx))\n+\t\treturn -EFAULT;\n+\n+\tfor (i = 0; i \u003c n; i++) {\n+\t\tu32 target = env-\u003egotox_tmp_buf-\u003eitems[i];\n+\n+\t\tif (target \u003c subprog_start || target \u003e= subprog_end)\n+\t\t\treturn reject_gotox_out_of_subprog(env, target, subprog_start,\n+\t\t\t\t\t\t\t subprog_end);\n+\t\t/* Maps are confined to a subprog, see compute_subprog_jts() */\n+\t\tif (verifier_bug_if(!jt_contains(jt, target), env,\n+\t\t\t\t \"insn %d target %u is not in the jump table of its subprog\",\n+\t\t\t\t env-\u003einsn_idx, target))\n+\t\t\treturn -EFAULT;\n+\t}\n+\n \tfor (i = 0; i \u003c n - 1; i++) {\n \t\tmark_indirect_target(env, env-\u003egotox_tmp_buf-\u003eitems[i]);\n \t\tother_branch = push_stack(env, env-\u003egotox_tmp_buf-\u003eitems[i],\n@@ -21477,8 +21532,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n \trelease_maps(env);\n \trelease_btfs(env);\n err_free_env:\n-\tif (env-\u003einsn_aux_data)\n-\t\tbpf_clear_insn_aux_data(env, 0, env-\u003einsn_aux_data_len);\n+\tbpf_free_subprog_jts(env);\n \tvfree(env-\u003einsn_aux_data);\n \tkvfree(env-\u003efd_array);\n \tbpf_stack_liveness_free(env);\ndiff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\nindex 0222a9a5d0761..f102435675c2b 100644\n--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\n+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\n@@ -1,6 +1,7 @@\n // SPDX-License-Identifier: GPL-2.0\n \n #include \u003cbpf/bpf.h\u003e\n+#include \u003cbpf/btf.h\u003e\n #include \u003ctest_progs.h\u003e\n \n #if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)\n@@ -453,43 +454,1083 @@ static void check_bpf_no_lookup(void)\n \tclose(map_fd);\n }\n \n+#define GOTOX_CNT_AT_LIMIT\t1000\n+#define GOTOX_LOG_SZ\t\t(256 * 1024)\n+\n+static const char gotox_limit_msg[] =\n+\t\"number of indirect jump edges in the program exceeds\";\n+\n+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)\n+{\n+\t/* the run of gotox itself, plus the insn right after it */\n+\tconst __u32 jt_cnt = gotox_cnt + 1;\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c jt_cnt; i++) {\n+\t\tval.orig_off = first_gotox + i;\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,\n+\t\t\t\t int *fd_array, __u32 fd_array_cnt, char *log,\n+\t\t\t\t int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)\n+{\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tint prog_fd;\n+\n+\tlog[0] = 0;\n+\topts.fd_array = fd_array;\n+\topts.fd_array_cnt = fd_array_cnt;\n+\topts.log_buf = log;\n+\topts.log_size = GOTOX_LOG_SZ;\n+\topts.log_level = 1;\n+\tif (fi_cnt) {\n+\t\topts.prog_btf_fd = btf_fd;\n+\t\topts.func_info = fi;\n+\t\topts.func_info_cnt = fi_cnt;\n+\t\topts.func_info_rec_size = sizeof(*fi);\n+\t}\n+\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", insns, insn_cnt, \u0026opts);\n+\tif (prog_fd \u003e= 0) {\n+\t\tclose(prog_fd);\n+\t\treturn 0;\n+\t}\n+\treturn prog_fd;\n+}\n+\n+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,\n+\t\t\t int *fd_array, __u32 fd_array_cnt, char *log)\n+{\n+\treturn gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,\n+\t\t\t\t -1, NULL, 0);\n+}\n+\n+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */\n+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)\n+{\n+\t__u32 i;\n+\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);\n+\tfor (i = 1; i \u003c= gotox_cnt; i++)\n+\t\tinsns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+}\n+\n+static void check_gotox_limit_hit(const char *log, int err)\n+{\n+\tASSERT_EQ(err, -E2BIG, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, gotox_limit_msg, \"verifier log\");\n+}\n+\n+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)\n+{\n+\tconst __u32 insn_cnt = gotox_cnt + 3;\n+\tstruct bpf_insn *insns;\n+\tbool attempted = false;\n+\tint map_fd;\n+\n+\tinsns = calloc(insn_cnt, sizeof(*insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\treturn false;\n+\n+\tgotox_run_fill(insns, gotox_cnt);\n+\tinsns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tmap_fd = gotox_jt_create(1, gotox_cnt);\n+\tif (map_fd \u003c 0)\n+\t\tgoto free_insns;\n+\n+\t*err = gotox_prog_load(insns, insn_cnt, \u0026map_fd, 1, log);\n+\tclose(map_fd);\n+\tattempted = true;\n+free_insns:\n+\tfree(insns);\n+\treturn attempted;\n+}\n+\n+/*\n+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt\n+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.\n+ */\n+static void check_too_many_gotox_edges(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tif (try_load_gotox_prog(gotox_cnt, log, \u0026err))\n+\t\tcheck_gotox_limit_hit(log, err);\n+\n+\tfree(log);\n+}\n+\n+/*\n+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump\n+ * table holds the starts of the blocks that follow plus the exit block,\n+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits\n+ * exactly at the limit and must still load.\n+ */\n+#define GOTOX_BLOCK_SZ\t\t4\n+\n+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)\n+{\n+\tstruct bpf_insn *at;\n+\t__u32 k;\n+\n+\tfor (k = 0; k \u003c gotox_cnt; k++) {\n+\t\tat = insns + k * GOTOX_BLOCK_SZ;\n+\n+\t\t/* r1 = \u0026jt[0], by index 0 into fd_array */\n+\t\tat[0] = (struct bpf_insn) {\n+\t\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t\t.dst_reg = BPF_REG_1,\n+\t\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t\t.imm = 0,\n+\t\t};\n+\t\tat[1] = (struct bpf_insn) { .imm = 0 };\n+\t\tat[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);\n+\t\tat[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\t}\n+\n+\tinsns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();\n+}\n+\n+static int gotox_chain_jt_create(__u32 gotox_cnt)\n+{\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c gotox_cnt; i++) {\n+\t\tval.orig_off = (i + 1) * GOTOX_BLOCK_SZ;\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+static void check_gotox_edges_at_limit(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;\n+\tconst __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;\n+\tstruct bpf_insn *insns;\n+\tchar *log;\n+\tint map_fd, err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tinsns = calloc(insn_cnt, sizeof(*insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\tgoto free_log;\n+\n+\tgotox_chain_fill(insns, gotox_cnt);\n+\n+\tmap_fd = gotox_chain_jt_create(gotox_cnt);\n+\tif (map_fd \u003c 0)\n+\t\tgoto free_insns;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, \u0026map_fd, 1, log);\n+\tclose(map_fd);\n+\n+\tif (!ASSERT_OK(err, \"program at the edge limit should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+free_insns:\n+\tfree(insns);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_edges_across_subprogs(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;\n+\tconst __u32 sub_start = gotox_cnt + 3;\n+\tconst __u32 insn_cnt = 2 * (gotox_cnt + 3);\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_insn *insns;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tinsns = calloc(insn_cnt, sizeof(*insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\tgoto free_log;\n+\n+\tgotox_run_fill(insns, gotox_cnt);\n+\tinsns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,\n+\t\t\t\t\t BPF_PSEUDO_CALL, 0,\n+\t\t\t\t\t sub_start - (gotox_cnt + 1) - 1);\n+\tinsns[gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tgotox_run_fill(insns + sub_start, gotox_cnt);\n+\tinsns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tmap_fd[0] = gotox_jt_create(1, gotox_cnt);\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_insns;\n+\tmap_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);\n+\tcheck_gotox_limit_hit(log, err);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_insns:\n+\tfree(insns);\n+free_log:\n+\tfree(log);\n+}\n+\n+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)\n+{\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c cnt; i++) {\n+\t\tval.orig_off = offs[i];\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+#define GOTOX_SUB_START\t\t4\n+#define GOTOX_MAIN_TGT\t\t2\n+#define GOTOX_SUB_TGT\t\t8\n+#define GOTOX_TWO_INSN_CNT\t10\n+\n+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_SUB_START - 1 - 1);\n+\tinsns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_EXIT_INSN();\n+\n+\t/* r1 = \u0026jt[0], by index 'jt_idx' into fd_array */\n+\tinsns[GOTOX_SUB_START] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = jt_idx,\n+\t};\n+\tinsns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);\n+\tinsns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[9] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_target_other_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 7 to 2 leaves the subprog [4,10)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_jt_per_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 1, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, 0, \"bpf(BPF_PROG_LOAD)\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_FWD_GOTOX\t\t11\n+#define GOTOX_FWD_OWN_TGT\t12\n+#define GOTOX_FWD_SUB_START\t14\n+#define GOTOX_FWD_INSN_CNT\t16\n+\n+static void gotox_from_main_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_FWD_SUB_START - 1 - 1);\n+\tinsns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,\n+\t\t\t offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);\n+\n+\t/* r1 = \u0026jt_leaves[0], by index 1 into fd_array */\n+\tinsns[4] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 1,\n+\t};\n+\tinsns[5] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\tinsns[7] = BPF_JMP_A(3);\n+\n+\t/* r1 = \u0026jt_own[0], by index 0 into fd_array */\n+\tinsns[8] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[9] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\n+\tinsns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[13] = BPF_EXIT_INSN();\n+\tinsns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[15] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_target_subprog_from_main(void)\n+{\n+\tconst __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };\n+\tconst __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 11 to 14 leaves the subprog [0,14)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/*\n+ * The only map of the subprog holding the gotox reaches past that subprog, so\n+ * the subprog is left without a jump table at all.\n+ */\n+static void check_gotox_jt_spans_subprogs(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };\n+\tconst __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"jump table of subprog starting at 0 spans multiple subprogs\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/*\n+ * The subprog holding the gotox has a well formed jump table of its own and\n+ * also collects a map that reaches past its end. The spanning map is still\n+ * rejected, even though the subprog is not left without a table.\n+ */\n+static void check_gotox_jt_spans_with_own_table(void)\n+{\n+\tconst __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };\n+\tconst __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"jump table of subprog starting at 0 spans multiple subprogs\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_SLICE_SUB_START\t6\n+#define GOTOX_SLICE_GOTOX\t14\n+#define GOTOX_SLICE_SUB_TGT\t15\n+#define GOTOX_SLICE_INSN_CNT\t17\n+\n+static void gotox_slice_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_SLICE_SUB_START - 1 - 1);\n+\tinsns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[5] = BPF_EXIT_INSN();\n+\n+\tinsns[GOTOX_SLICE_SUB_START] =\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,\n+\t\t\t offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);\n+\tinsns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);\n+\tinsns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);\n+\n+\t/* r1 = \u0026jt_main[0], by index 0 into fd_array */\n+\tinsns[10] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[11] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);\n+\tinsns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\n+\tinsns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[16] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_index_slice_other_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { 2, 3, 4 };\n+\tconst __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_SLICE_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_slice_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 14 to 3 leaves the subprog [6,17)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,\n+\t\t\t struct bpf_func_info *fi, struct btf **pbtf)\n+{\n+\tint int_id, proto_id, id;\n+\tstruct btf *btf;\n+\tchar name[24];\n+\t__u32 i;\n+\n+\tbtf = btf__new_empty();\n+\tif (!ASSERT_OK_PTR(btf, \"btf__new_empty\"))\n+\t\treturn -1;\n+\n+\tint_id = btf__add_int(btf, \"int\", 4, BTF_INT_SIGNED);\n+\tif (!ASSERT_GT(int_id, 0, \"btf__add_int\"))\n+\t\tgoto err;\n+\n+\tproto_id = btf__add_func_proto(btf, int_id);\n+\tif (!ASSERT_GT(proto_id, 0, \"btf__add_func_proto\"))\n+\t\tgoto err;\n+\n+\tfor (i = 0; i \u003c cnt; i++) {\n+\t\tsnprintf(name, sizeof(name), \"gotox_f%u\", i);\n+\t\tid = btf__add_func(btf, name, linkage[i], proto_id);\n+\t\tif (!ASSERT_GT(id, 0, \"btf__add_func\"))\n+\t\t\tgoto err;\n+\t\tfi[i].insn_off = starts[i];\n+\t\tfi[i].type_id = id;\n+\t}\n+\n+\tif (!ASSERT_OK(btf__load_into_kernel(btf), \"btf__load_into_kernel\"))\n+\t\tgoto err;\n+\n+\t*pbtf = btf;\n+\treturn btf__fd(btf);\n+err:\n+\tbtf__free(btf);\n+\treturn -1;\n+}\n+\n+static void check_gotox_target_other_global_subprog(void)\n+{\n+\tconst __u32 starts[] = { 0, GOTOX_SUB_START };\n+\tconst __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_func_info fi[2];\n+\tstruct btf *btf = NULL;\n+\tint btf_fd;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 0);\n+\n+\tbtf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, \u0026btf);\n+\tif (btf_fd \u003c 0)\n+\t\tgoto free_log;\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_btf;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,\n+\t\t\t\t btf_fd, fi, ARRAY_SIZE(fi));\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 7 to 2 leaves the subprog [4,10)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_btf:\n+\tbtf__free(btf);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_CB_MAIN_TGT\t6\n+#define GOTOX_CB_START\t\t8\n+#define GOTOX_CB_GOTOX\t\t11\n+#define GOTOX_CB_TGT\t\t12\n+#define GOTOX_CB_INSN_CNT\t14\n+\n+static void gotox_callback_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);\n+\t/* r2 = \u0026callback */\n+\tinsns[1] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_2,\n+\t\t.src_reg = BPF_PSEUDO_FUNC,\n+\t\t.imm = GOTOX_CB_START - 1 - 1,\n+\t};\n+\tinsns[2] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);\n+\tinsns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);\n+\tinsns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);\n+\tinsns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[7] = BPF_EXIT_INSN();\n+\n+\t/* r1 = \u0026jt_main[0], by index 0 into fd_array */\n+\tinsns[GOTOX_CB_START] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[9] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\tinsns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[13] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_callback_leaves_subprog(void)\n+{\n+\tconst __u32 starts[] = { 0, GOTOX_CB_START };\n+\tconst __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };\n+\tconst __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };\n+\tconst __u32 jt_cb[] = { GOTOX_CB_TGT };\n+\tstruct bpf_insn insns[GOTOX_CB_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_func_info fi[2];\n+\tstruct btf *btf = NULL;\n+\tint btf_fd;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_callback_fill(insns);\n+\n+\tbtf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, \u0026btf);\n+\tif (btf_fd \u003c 0)\n+\t\tgoto free_log;\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_btf;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,\n+\t\t\t\t btf_fd, fi, ARRAY_SIZE(fi));\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 11 to 6 leaves the subprog [8,14)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_btf:\n+\tbtf__free(btf);\n+free_log:\n+\tfree(log);\n+}\n+\n static void check_bpf_side(void)\n {\n \tcheck_bpf_no_lookup();\n }\n \n+static void check_gotox_target_nop(void)\n+{\n+\tstruct bpf_insn insns[] = {\n+\t\t/* r1 = \u0026jt[0] */\n+\t\tBPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),\n+\t\tBPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),\n+\t\tBPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */\n+\t\tBPF_JMP_IMM(BPF_JA, 0, 0, 0),\t\t/* insn 4: the nop target */\n+\t\tBPF_MOV64_IMM(BPF_REG_0, XDP_PASS),\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tstruct bpf_insn_array_value val = {};\n+\tint prog_fd = -1, map_fd;\n+\t__u32 key = 0;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn;\n+\n+\tval.orig_off = 4;\n+\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026key, \u0026val, 0), 0, \"bpf_map_update_elem\"))\n+\t\tgoto cleanup;\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto cleanup;\n+\n+\topts.fd_array = \u0026map_fd;\n+\topts.fd_array_cnt = 1;\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", insns, ARRAY_SIZE(insns), \u0026opts);\n+\tif (!ASSERT_GE(prog_fd, 0, \"bpf(BPF_PROG_LOAD)\"))\n+\t\tgoto cleanup;\n+\n+\tif (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, \u0026key, \u0026val), 0, \"bpf_map_lookup_elem\"))\n+\t\tgoto cleanup;\n+\tASSERT_NEQ(val.xlated_off, (__u32)-1, \"gotox target must not be INSN_DELETED\");\n+\tASSERT_NEQ(val.jitted_off, 0, \"gotox target must have a jitted address\");\n+cleanup:\n+\tif (prog_fd \u003e= 0)\n+\t\tclose(prog_fd);\n+\tclose(map_fd);\n+}\n+\n+static void check_insn_array_stale_reuse(void)\n+{\n+\t/* helper of a different prog type, to set the prog array's owner */\n+\tstruct bpf_insn sf[] = {\n+\t\tBPF_MOV64_IMM(BPF_REG_0, 0),\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\t/*\n+\t * prog1 binds the insn array (entry points at a real insn, so the JIT\n+\t * fills it in) and tail calls a prog array whose owner is now a\n+\t * different type, which fails in the post-JIT compatibility check.\n+\t */\n+\tstruct bpf_insn prog1[] = {\n+\t\tBPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */\n+\t\tBPF_MOV64_IMM(BPF_REG_3, 0),\t\t\t /* r3 = index */\n+\t\tBPF_EMIT_CALL(BPF_FUNC_tail_call),\n+\t\tBPF_MOV64_IMM(BPF_REG_0, XDP_PASS),\t\t /* insn 4: real entry */\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\t/*\n+\t * prog2 reuses the insn array as a tracker. Its entry points at a nop\n+\t * that the nop pass removes, so the entry becomes INSN_DELETED and the\n+\t * JIT skips it -- exactly the slot a stale pointer would linger in.\n+\t */\n+\tstruct bpf_insn prog2[] = {\n+\t\tBPF_MOV64_IMM(BPF_REG_0, XDP_PASS),\n+\t\tBPF_JMP_IMM(BPF_JA, 0, 0, 0),\t\t/* nop */\n+\t\tBPF_MOV64_IMM(BPF_REG_0, XDP_PASS),\n+\t\tBPF_JMP_IMM(BPF_JA, 0, 0, 0),\t\t/* nop */\n+\t\tBPF_JMP_IMM(BPF_JA, 0, 0, 0),\t\t/* insn 4: tracked nop */\n+\t\tBPF_MOV64_IMM(BPF_REG_0, XDP_PASS),\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tint jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;\n+\tstruct bpf_insn_array_value val = {};\n+\tint fd_array[2];\n+\t__u32 key = 0;\n+\n+\tjt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);\n+\tif (!ASSERT_GE(jt_fd, 0, \"insn_array create\"))\n+\t\treturn;\n+\tval.orig_off = 4;\n+\tif (!ASSERT_EQ(bpf_map_update_elem(jt_fd, \u0026key, \u0026val, 0), 0, \"insn_array update\"))\n+\t\tgoto cleanup;\n+\tif (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, \"insn_array freeze\"))\n+\t\tgoto cleanup;\n+\n+\tpa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, \"pa\", 4, 4, 1, NULL);\n+\tif (!ASSERT_GE(pa_fd, 0, \"prog_array create\"))\n+\t\tgoto cleanup;\n+\tsf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, \"GPL\", sf, ARRAY_SIZE(sf), NULL);\n+\tif (!ASSERT_GE(sf_fd, 0, \"owner prog load\"))\n+\t\tgoto cleanup;\n+\t/* insert a non-XDP prog to pin the prog array owner to another type */\n+\tif (!ASSERT_EQ(bpf_map_update_elem(pa_fd, \u0026key, \u0026sf_fd, 0), 0, \"prog_array update\"))\n+\t\tgoto cleanup;\n+\n+\tfd_array[0] = jt_fd;\n+\tfd_array[1] = pa_fd;\n+\topts.fd_array = fd_array;\n+\topts.fd_array_cnt = 2;\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", prog1, ARRAY_SIZE(prog1), \u0026opts);\n+\tif (!ASSERT_LT(prog_fd, 0, \"prog1 must fail the post-JIT tail call check\"))\n+\t\tgoto cleanup;\n+\n+\t/* prog1 reached the JIT before failing, so the entry is now populated */\n+\tif (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, \u0026key, \u0026val), 0, \"lookup after prog1\"))\n+\t\tgoto cleanup;\n+\tif (!ASSERT_NEQ(val.jitted_off, 0, \"prog1 should have filled the jitted address\"))\n+\t\tgoto cleanup;\n+\n+\topts.fd_array_cnt = 1;\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", prog2, ARRAY_SIZE(prog2), \u0026opts);\n+\tif (!ASSERT_GE(prog_fd, 0, \"prog2 reuse load\"))\n+\t\tgoto cleanup;\n+\n+\tif (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, \u0026key, \u0026val), 0, \"lookup after prog2\"))\n+\t\tgoto cleanup;\n+\tASSERT_EQ(val.xlated_off, (__u32)-1, \"reused entry should be INSN_DELETED\");\n+\tASSERT_EQ(val.jitted_off, 0, \"stale jitted address must be cleared on reuse\");\n+cleanup:\n+\tif (prog_fd \u003e= 0)\n+\t\tclose(prog_fd);\n+\tif (sf_fd \u003e= 0)\n+\t\tclose(sf_fd);\n+\tif (pa_fd \u003e= 0)\n+\t\tclose(pa_fd);\n+\tif (jt_fd \u003e= 0)\n+\t\tclose(jt_fd);\n+}\n+\n+static void check_gotox_target_prologue_shift(void)\n+{\n+\tstruct bpf_insn insns[] = {\n+\t\t/* insn 0: gotox target and subprog start */\n+\t\tBPF_MOV64_IMM(BPF_REG_0, 0),\n+\t\t/* may_goto +4 -\u003e exit block, bounds the loop */\n+\t\tBPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),\n+\t\t/* r1 = \u0026jt[0] (insns 2 and 3) */\n+\t\tBPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),\n+\t\tBPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */\n+\t\t/* insn 5: gotox r1 -\u003e insn 0 */\n+\t\tBPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),\n+\t\tBPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tstruct bpf_insn_array_value val = {};\n+\tint prog_fd = -1, map_fd;\n+\t__u32 key = 0;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn;\n+\n+\tval.orig_off = 0;\n+\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026key, \u0026val, 0), 0, \"bpf_map_update_elem\"))\n+\t\tgoto cleanup;\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto cleanup;\n+\n+\topts.fd_array = \u0026map_fd;\n+\topts.fd_array_cnt = 1;\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", insns, ARRAY_SIZE(insns), \u0026opts);\n+\tif (!ASSERT_GE(prog_fd, 0, \"bpf(BPF_PROG_LOAD)\"))\n+\t\tgoto cleanup;\n+\n+\tif (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, \u0026key, \u0026val), 0, \"bpf_map_lookup_elem\"))\n+\t\tgoto cleanup;\n+\tASSERT_NEQ(val.xlated_off, 0, \"gotox target retargeted past prepend\");\n+\tASSERT_NEQ(val.xlated_off, (__u32)-1, \"gotox target not INSN_DELETED\");\n+\tASSERT_NEQ(val.jitted_off, 0, \"gotox target has a jitted address\");\n+cleanup:\n+\tif (prog_fd \u003e= 0)\n+\t\tclose(prog_fd);\n+\tclose(map_fd);\n+}\n+\n+static void check_gotox_target_ctx_prologue_shift(void)\n+{\n+\tstruct bpf_insn insns[] = {\n+\t\t/* insn 0: gotox target and subprog start */\n+\t\tBPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),\n+\t\tBPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */\n+\t\tBPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),\n+\t\tBPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),\n+\t\tBPF_MOV64_REG(BPF_REG_4, BPF_REG_2),\n+\t\tBPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),\n+\t\tBPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),\n+\t\t/* insn 9: direct packet write -\u003e tc unclone prologue prepended at insn 0 */\n+\t\tBPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),\n+\t\t/* r5 = \u0026jt[0] (insns 10 and 11) */\n+\t\tBPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),\n+\t\tBPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */\n+\t\t/* insn 13: gotox r5 -\u003e insn 0 */\n+\t\tBPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),\n+\t\tBPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tstruct bpf_insn_array_value val = {};\n+\tint prog_fd = -1, map_fd;\n+\t__u32 key = 0;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn;\n+\n+\tval.orig_off = 0;\n+\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026key, \u0026val, 0), 0, \"bpf_map_update_elem\"))\n+\t\tgoto cleanup;\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto cleanup;\n+\n+\topts.fd_array = \u0026map_fd;\n+\topts.fd_array_cnt = 1;\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, \"GPL\", insns,\n+\t\t\t\tARRAY_SIZE(insns), \u0026opts);\n+\tif (!ASSERT_GE(prog_fd, 0, \"bpf(BPF_PROG_LOAD)\"))\n+\t\tgoto cleanup;\n+\n+\tif (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, \u0026key, \u0026val), 0, \"bpf_map_lookup_elem\"))\n+\t\tgoto cleanup;\n+\tASSERT_NEQ(val.xlated_off, 0, \"gotox target retargeted past prepend\");\n+\tASSERT_NEQ(val.xlated_off, (__u32)-1, \"gotox target not INSN_DELETED\");\n+\tASSERT_NEQ(val.jitted_off, 0, \"gotox target has a jitted address\");\n+cleanup:\n+\tif (prog_fd \u003e= 0)\n+\t\tclose(prog_fd);\n+\tclose(map_fd);\n+}\n+\n static void __test_bpf_insn_array(void)\n {\n \t/* Test if offsets are adjusted properly */\n-\n \tif (test__start_subtest(\"one2one\"))\n \t\tcheck_one_to_one_mapping();\n-\n \tif (test__start_subtest(\"simple\"))\n \t\tcheck_simple();\n-\n \tif (test__start_subtest(\"deletions\"))\n \t\tcheck_deletions();\n-\n \tif (test__start_subtest(\"deletions-with-functions\"))\n \t\tcheck_deletions_with_functions();\n-\n \tif (test__start_subtest(\"blindness\"))\n \t\tcheck_blindness();\n-\n \t/* Check all kinds of operations and related restrictions */\n-\n \tif (test__start_subtest(\"incorrect-index\"))\n \t\tcheck_incorrect_index();\n-\n \tif (test__start_subtest(\"load-unfrozen-map\"))\n \t\tcheck_load_unfrozen_map();\n-\n \tif (test__start_subtest(\"no-map-reuse\"))\n \t\tcheck_no_map_reuse();\n-\n \tif (test__start_subtest(\"bpf-side-ops\"))\n \t\tcheck_bpf_side();\n+\tif (test__start_subtest(\"too-many-gotox-edges\"))\n+\t\tcheck_too_many_gotox_edges();\n+\tif (test__start_subtest(\"gotox-edges-at-limit\"))\n+\t\tcheck_gotox_edges_at_limit();\n+\tif (test__start_subtest(\"gotox-edges-across-subprogs\"))\n+\t\tcheck_gotox_edges_across_subprogs();\n+\tif (test__start_subtest(\"gotox-jt-spans-subprogs\"))\n+\t\tcheck_gotox_jt_spans_subprogs();\n+\tif (test__start_subtest(\"gotox-jt-spans-with-own-table\"))\n+\t\tcheck_gotox_jt_spans_with_own_table();\n+\tif (test__start_subtest(\"gotox-target-other-subprog\"))\n+\t\tcheck_gotox_target_other_subprog();\n+\tif (test__start_subtest(\"gotox-jt-per-subprog\"))\n+\t\tcheck_gotox_jt_per_subprog();\n+\tif (test__start_subtest(\"gotox-target-subprog-from-main\"))\n+\t\tcheck_gotox_target_subprog_from_main();\n+\tif (test__start_subtest(\"gotox-index-slice-other-subprog\"))\n+\t\tcheck_gotox_index_slice_other_subprog();\n+\tif (test__start_subtest(\"gotox-target-other-global-subprog\"))\n+\t\tcheck_gotox_target_other_global_subprog();\n+\tif (test__start_subtest(\"gotox-callback-leaves-subprog\"))\n+\t\tcheck_gotox_callback_leaves_subprog();\n+\tif (test__start_subtest(\"gotox-target-nop\"))\n+\t\tcheck_gotox_target_nop();\n+\tif (test__start_subtest(\"insn-array-stale-reuse\"))\n+\t\tcheck_insn_array_stale_reuse();\n+\tif (test__start_subtest(\"gotox-target-prologue-shift\"))\n+\t\tcheck_gotox_target_prologue_shift();\n+\tif (test__start_subtest(\"gotox-target-ctx-prologue-shift\"))\n+\t\tcheck_gotox_target_ctx_prologue_shift();\n }\n #else\n static void __test_bpf_insn_array(void)\ndiff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c\nindex 0e27c2c79c57e..4931a77ba0b26 100644\n--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c\n+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c\n@@ -366,7 +366,7 @@ __used static int test_subprog(void)\n }\n \n SEC(\"socket\")\n-__failure __msg(\"jump table for insn 4 points outside of the subprog [0,10]\")\n+__failure __msg(\"jump table of subprog starting at 0 spans multiple subprogs\")\n __naked void jump_table_outside_subprog(void)\n {\n \tasm volatile (\"\t\t\t\t\t\t\\\n"
} |
| 2/1 | 2026/09/14 21:21 | agent | patch-evaluator |
1mModel:core Args: null Results: {
"EnableConfigs": [
"BPF_SYSCALL",
"BPF_JIT"
],
"FocusSymbols": [
"bpf_check_cfg",
"compute_subprog_jts",
"bpf_free_subprog_jts",
"bpf_compute_scc",
"check_indirect_jump",
"bpf_insn_successors",
"bpf_insn_array_adjust",
"bpf_insn_array_init",
"bpf_patch_insn_data",
"bpf_opt_remove_nops"
],
"Reasoning": "The patch refactors indirect jump (gotox) and jump table (BPF_MAP_TYPE_INSN_ARRAY) verification in the BPF verifier. It restructures CFG verification by computing jump tables per subprogram rather than per instruction, introduces an indirect jump edge complexity limit, updates DFS tracking in SCC computation, modifies instruction successor resolution for liveness tracking, and updates instruction patching to distinguish prepending versus appending while protecting indirect jump targets from dead code/NOP elimination. These paths are directly reachable from userspace via the bpf() syscall and warrant fuzzing to uncover potential verifier bypasses, out-of-bounds access, or verifier panics.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit 22a0a01defffa4374bd99018ad18557be4b6f8a5
Author: syz-cluster <triage@syzkaller.com>
Date: Mon Sep 14 21:21:27 2026 +0000
syz-cluster: applied patch under review
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e57af902560c3..8115f307444ad 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);
int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
int bpf_insn_array_ready(struct bpf_map *map);
void bpf_insn_array_release(struct bpf_map *map);
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);
void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);
#ifdef CONFIG_BPF_SYSCALL
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 36b65797877d0..754e24d41f47f 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
/* remember the offset of node field within type to rewrite */
u64 insert_off;
};
- struct bpf_iarray *jt; /* jump table for gotox or bpf_tailcall call instruction */
struct btf_struct_meta *kptr_struct_meta;
u64 map_key_state; /* constant (32 bit) key tracking for maps */
int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
@@ -805,6 +804,7 @@ struct bpf_subprog_info {
u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+ struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
u16 stack_depth; /* max. stack depth used by this function */
u16 stack_extra;
u32 insns_total;
@@ -977,6 +977,8 @@ struct bpf_verifier_env {
int cur_stack;
/* current position in the insn_postorder vector */
int cur_postorder;
+ u32 gotox_edges;
+ bool subprog_jts_ready;
} cfg;
struct backtrack_state bt;
struct bpf_jmp_history_entry *cur_hist_ent;
@@ -1501,6 +1503,7 @@ bool bpf_is_throw_kfunc(struct bpf_insn *insn);
int bpf_compute_const_regs(struct bpf_verifier_env *env);
int bpf_prune_dead_branches(struct bpf_verifier_env *env);
int bpf_check_cfg(struct bpf_verifier_env *env);
+void bpf_free_subprog_jts(struct bpf_verifier_env *env);
int bpf_compute_postorder(struct bpf_verifier_env *env);
int bpf_compute_scc(struct bpf_verifier_env *env);
@@ -1702,7 +1705,6 @@ struct bpf_kfunc_desc_tab {
};
/* Functions exported from verifier.c, used by fixups.c */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len);
void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog);
bool bpf_allow_tail_call_in_subprogs(struct bpf_verifier_env *env);
bool bpf_verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm);
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index a2f84afe6f7c6..350ecea8f1e85 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
return -EBUSY;
/*
- * Reset all the map indexes to the original values. This is needed,
- * e.g., when a replay of verification with different log level should
- * be performed.
+ * Reset the map to its pre-verification state. The xlated and jitted
+ * offsets and the jitted target pointers are recomputed by the verifier
+ * and the JIT for this program, so any values left by a previous owner
+ * must be cleared here.
*/
- for (i = 0; i < map->max_entries; i++)
+ for (i = 0; i < map->max_entries; i++) {
values[i].xlated_off = values[i].orig_off;
+ values[i].jitted_off = 0;
+ insn_array->ips[i] = 0;
+ }
return 0;
}
@@ -231,7 +235,7 @@ void bpf_insn_array_release(struct bpf_map *map)
atomic_set(&insn_array->used, 0);
}
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)
{
struct bpf_insn_array *insn_array = cast_insn_array(map);
int i;
@@ -240,7 +244,7 @@ void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
return;
for (i = 0; i < map->max_entries; i++) {
- if (insn_array->values[i].xlated_off <= off)
+ if (insn_array->values[i].xlated_off < first)
continue;
if (insn_array->values[i].xlated_off == INSN_DELETED)
continue;
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 842c7d1eabccc..5a91d73d3e3fe 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -9,6 +9,8 @@
#define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
+#define BPF_MAX_GOTOX_EDGES BPF_COMPLEXITY_LIMIT_INSNS
+
/* non-recursive DFS pseudo code
* 1 procedure DFS-iterative(G,v):
* 2 label v as discovered
@@ -284,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
}
/*
- * Find and collect all maps which fit in the subprog. Return the result as one
- * combined jump table in jt->items (allocated with kvcalloc)
+ * Collect the jump table of every subprogram that has one, as the combined
+ * table of all maps whose targets land inside that subprogram. All gotox
+ * instructions of a subprogram share the same table, so this is done in a
+ * single pass over the maps rather than once per gotox.
*/
-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
- int subprog_start, int subprog_end)
+static int compute_subprog_jts(struct bpf_verifier_env *env)
{
- struct bpf_iarray *jt = NULL;
+ struct bpf_subprog_info *subprog;
+ struct bpf_iarray *jt, *jt_cur;
struct bpf_map *map;
- struct bpf_iarray *jt_cur;
+ u32 old_cnt;
int i;
for (i = 0; i < env->insn_array_map_cnt; i++) {
@@ -303,73 +307,84 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
map = env->insn_array_maps[i];
jt_cur = jt_from_map(map);
- if (IS_ERR(jt_cur)) {
- kvfree(jt);
- return jt_cur;
+ if (IS_ERR(jt_cur))
+ return PTR_ERR(jt_cur);
+
+ subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
+ if (!subprog) {
+ kvfree(jt_cur);
+ continue;
+ }
+ if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) {
+ verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n",
+ subprog->start);
+ bpf_diag_program_structure(env, subprog->start, "jump table spans subprograms",
+ "Keep every entry of a jump table inside one subprogram.",
+ "A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.",
+ subprog->start, (subprog + 1)->start);
+ kvfree(jt_cur);
+ return -EINVAL;
}
- /*
- * This is enough to check one element. The full table is
- * checked to fit inside the subprog later in create_jt()
- */
- if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) {
- u32 old_cnt = jt ? jt->cnt : 0;
- jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt);
- if (!jt) {
- kvfree(jt_cur);
- return ERR_PTR(-ENOMEM);
- }
- memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+ old_cnt = subprog->jt ? subprog->jt->cnt : 0;
+ jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
+ if (!jt) {
+ subprog->jt = NULL;
+ kvfree(jt_cur);
+ return -ENOMEM;
}
+ memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+ subprog->jt = jt;
kvfree(jt_cur);
}
- if (!jt) {
- verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
- bpf_diag_program_structure(
- env, subprog_start, "missing jump table",
- "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
- "No jump table was found for the subprogram that starts at instruction %u.",
- subprog_start);
- return ERR_PTR(-EINVAL);
+ for (i = 0; i < env->subprog_cnt; i++) {
+ jt = env->subprog_info[i].jt;
+ if (jt)
+ jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
}
- jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
- return jt;
+ env->cfg.subprog_jts_ready = true;
+ return 0;
+}
+
+void bpf_free_subprog_jts(struct bpf_verifier_env *env)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
+ kvfree(env->subprog_info[i].jt);
+ env->subprog_info[i].jt = NULL;
+ }
+ env->cfg.subprog_jts_ready = false;
}
static struct bpf_iarray *
-create_jt(int t, struct bpf_verifier_env *env)
+subprog_jt(int t, struct bpf_verifier_env *env)
{
struct bpf_subprog_info *subprog;
- int subprog_start, subprog_end;
- struct bpf_iarray *jt;
- int i;
+ int subprog_start, err;
+
+ if (!env->cfg.subprog_jts_ready) {
+ err = compute_subprog_jts(env);
+ if (err)
+ return ERR_PTR(err);
+ }
subprog = bpf_find_containing_subprog(env, t);
subprog_start = subprog->start;
- subprog_end = (subprog + 1)->start;
- jt = jt_from_subprog(env, subprog_start, subprog_end);
- if (IS_ERR(jt))
- return jt;
- /* Check that the every element of the jump table fits within the given subprogram */
- for (i = 0; i < jt->cnt; i++) {
- if (jt->items[i] < subprog_start || jt->items[i] >= subprog_end) {
- verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n",
- t, subprog_start, subprog_end);
- bpf_diag_program_structure(
- env, t, "jump table target out of range",
- "Keep every jump-table target inside the same subprogram.",
- "The jump table for instruction %d points outside subprogram range [%u,%u).",
- t, subprog_start, subprog_end);
- kvfree(jt);
- return ERR_PTR(-EINVAL);
- }
+ if (!subprog->jt) {
+ verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
+ bpf_diag_program_structure(env, subprog_start, "missing jump table",
+ "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
+ "No jump table was found for the subprogram that starts at instruction %u.",
+ subprog_start);
+ return ERR_PTR(-EINVAL);
}
- return jt;
+ return subprog->jt;
}
/* "conditional jump with N edges" */
@@ -381,13 +396,24 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
struct bpf_iarray *jt;
int i, w;
- jt = env->insn_aux_data[t].jt;
- if (!jt) {
- jt = create_jt(t, env);
- if (IS_ERR(jt))
- return PTR_ERR(jt);
-
- env->insn_aux_data[t].jt = jt;
+ jt = subprog_jt(t, env);
+ if (IS_ERR(jt))
+ return PTR_ERR(jt);
+
+ if (!(insn_state[t] & BRANCH)) {
+ insn_state[t] |= BRANCH;
+
+ if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+ &env->cfg.gotox_edges) ||
+ env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+ verbose(env, "number of indirect jump edges in the program exceeds %u\n",
+ BPF_MAX_GOTOX_EDGES);
+ bpf_diag_program_structure(env, t, "too many indirect jump edges",
+ "Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+ "The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+ BPF_MAX_GOTOX_EDGES);
+ return -E2BIG;
+ }
}
mark_prune_point(env, t);
@@ -421,30 +447,6 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
return keep_exploring ? KEEP_EXPLORING : DONE_EXPLORING;
}
-/*
- * Instructions that can abnormally return from a subprog (tail_call
- * upon success, ld_{abs,ind} upon load failure) have a hidden exit
- * that the verifier must account for.
- */
-static int visit_abnormal_return_insn(struct bpf_verifier_env *env, int t)
-{
- struct bpf_subprog_info *subprog;
- struct bpf_iarray *jt;
-
- if (env->insn_aux_data[t].jt)
- return 0;
-
- jt = bpf_iarray_realloc(NULL, 2);
- if (!jt)
- return -ENOMEM;
-
- subprog = bpf_find_containing_subprog(env, t);
- jt->items[0] = t + 1;
- jt->items[1] = subprog->exit_idx;
- env->insn_aux_data[t].jt = jt;
- return 0;
-}
-
/* Visits the instruction at index t and returns one of the following:
* < 0 - an error occurred
* DONE_EXPLORING - the instruction was fully explored
@@ -461,13 +463,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
/* All non-branch instructions have a single fall-through edge. */
if (BPF_CLASS(insn->code) != BPF_JMP &&
BPF_CLASS(insn->code) != BPF_JMP32) {
- if (BPF_CLASS(insn->code) == BPF_LD &&
- (BPF_MODE(insn->code) == BPF_ABS ||
- BPF_MODE(insn->code) == BPF_IND)) {
- ret = visit_abnormal_return_insn(env, t);
- if (ret)
- return ret;
- }
insn_sz = bpf_is_ldimm64(insn) ? 2 : 1;
return push_insn(t, t + insn_sz, FALLTHROUGH, env);
}
@@ -512,11 +507,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
mark_subprog_might_sleep(env, t);
if (bpf_helper_changes_pkt_data(insn->imm))
mark_subprog_changes_pkt_data(env, t);
- if (insn->imm == BPF_FUNC_tail_call) {
- ret = visit_abnormal_return_insn(env, t);
- if (ret)
- return ret;
- }
} else if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
struct bpf_call_arg_meta meta;
@@ -749,7 +739,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
struct bpf_insn_aux_data *aux = env->insn_aux_data;
const u32 insn_cnt = env->prog->len;
int stack_sz, dfs_sz, err = 0;
- u32 *stack, *pre, *low, *dfs;
+ u32 *stack, *pre, *low, *dfs, *dfs_pos;
u32 i, j, t, w;
u32 next_preorder_num;
u32 next_scc_id;
@@ -762,13 +752,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
* - 'stack' accumulates vertices in DFS order, see invariant comment below;
* - 'pre[t] == p' => preorder number of vertex 't' is 'p';
* - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n';
- * - 'dfs' DFS traversal stack, used to emulate explicit recursion.
+ * - 'dfs' DFS traversal stack, used to emulate explicit recursion;
+ * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its
+ * successors at index 'j'.
*/
stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
dfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);
- if (!stack || !pre || !low || !dfs) {
+ dfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);
+ if (!stack || !pre || !low || !dfs || !dfs_pos) {
err = -ENOMEM;
goto exit;
}
@@ -851,6 +844,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
stack_sz = 0;
dfs_sz = 1;
dfs[0] = i;
+ dfs_pos[0] = 0;
dfs_continue:
while (dfs_sz) {
w = dfs[dfs_sz - 1];
@@ -860,13 +854,22 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
next_preorder_num++;
stack[stack_sz++] = w;
}
- /* Visit 'w' successors */
+ /* Visit remaining 'w' successors */
succ = bpf_insn_successors(env, w);
- for (j = 0; j < succ->cnt; ++j) {
+ for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) {
if (pre[succ->items[j]]) {
low[w] = min(low[w], low[succ->items[j]]);
} else {
- dfs[dfs_sz++] = succ->items[j];
+ /*
+ * Once DFS for succ->items[j] is complete,
+ * pre[succ->items[j]] is non-zero, hence
+ * resuming at 'j' allows to follow the
+ * low[w] = min(...) update branch above.
+ */
+ dfs_pos[dfs_sz - 1] = j;
+ dfs_pos[dfs_sz] = 0;
+ dfs[dfs_sz] = succ->items[j];
+ dfs_sz++;
goto dfs_continue;
}
}
@@ -916,5 +919,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
kvfree(pre);
kvfree(low);
kvfree(dfs);
+ kvfree(dfs_pos);
return err;
}
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 52d3cec336727..e132bd0600c7e 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -213,12 +213,18 @@ static int get_callee_stack_depth(struct bpf_verifier_env *env,
}
#endif
+enum bpf_patch_mode {
+ BPF_APPEND,
+ BPF_PREPEND,
+};
+
/* single env->prog->insni[off] instruction was replaced with the range
* insni[off, off + cnt). Adjust corresponding insn_aux_data by copying
* [0, off) and [off, end) to new locations, so the patched range stays zero
*/
static void adjust_insn_aux_data(struct bpf_verifier_env *env,
- struct bpf_prog *new_prog, u32 off, u32 cnt)
+ struct bpf_prog *new_prog, u32 off, u32 cnt,
+ enum bpf_patch_mode mode)
{
struct bpf_insn_aux_data *data = env->insn_aux_data;
struct bpf_insn *insn = new_prog->insnsi;
@@ -251,9 +257,10 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
* new instructions by the above memmove and memset, but the indirect jump target is
* actually the first instruction, so move it back. This also matches with the behavior
* of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
- * instruction.
+ * instruction. For BPF_PREPEND the original instruction is the last one, so the flag
+ * already sits where needed.
*/
- if (data[off + cnt - 1].indirect_target) {
+ if (mode == BPF_APPEND && data[off + cnt - 1].indirect_target) {
data[off].indirect_target = 1;
data[off + cnt - 1].indirect_target = 0;
}
@@ -273,7 +280,7 @@ static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len
}
}
-static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)
+static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 first, u32 len)
{
int i;
@@ -281,7 +288,7 @@ static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)
return;
for (i = 0; i < env->insn_array_map_cnt; i++)
- bpf_insn_array_adjust(env->insn_array_maps[i], off, len);
+ bpf_insn_array_adjust(env->insn_array_maps[i], first, len);
}
static void adjust_insn_arrays_after_remove(struct bpf_verifier_env *env, u32 off, u32 len)
@@ -306,8 +313,9 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+static struct bpf_prog *__bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len,
+ enum bpf_patch_mode mode)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -331,13 +339,19 @@ struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
env->insn_aux_data[off].orig_idx);
return NULL;
}
- adjust_insn_aux_data(env, new_prog, off, len);
+ adjust_insn_aux_data(env, new_prog, off, len, mode);
adjust_subprog_starts(env, off, len);
- adjust_insn_arrays(env, off, len);
+ adjust_insn_arrays(env, mode == BPF_PREPEND ? off : off + 1, len);
adjust_poke_descs(new_prog, off, len);
return new_prog;
}
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return __bpf_patch_insn_data(env, off, patch, len, BPF_APPEND);
+}
+
/*
* For all jmp insns in a given 'prog' that point to 'tgt_idx' insn adjust the
* jump offset by 'delta'.
@@ -500,23 +514,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
return 0;
}
-/*
- * Clean up dynamically allocated fields of aux data for instructions [start, ...]
- */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
-{
- struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
- int end = start + len;
- int i;
-
- for (i = start; i < end; i++) {
- if (aux_data[i].jt) {
- kvfree(aux_data[i].jt);
- aux_data[i].jt = NULL;
- }
- }
-}
-
static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
{
struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
@@ -526,8 +523,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
if (bpf_prog_is_offloaded(env->prog->aux))
bpf_prog_offload_remove_insns(env, off, cnt);
- bpf_clear_insn_aux_data(env, off, cnt);
-
err = bpf_remove_insns(env->prog, off, cnt);
if (err)
return err;
@@ -618,6 +613,7 @@ int bpf_opt_remove_dead_code(struct bpf_verifier_env *env)
int bpf_opt_remove_nops(struct bpf_verifier_env *env)
{
+ struct bpf_insn_aux_data *aux = env->insn_aux_data;
struct bpf_insn *insn = env->prog->insnsi;
int insn_cnt = env->prog->len;
bool is_may_goto_0, is_ja;
@@ -629,6 +625,8 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)
if (!is_may_goto_0 && !is_ja)
continue;
+ if (aux[i].indirect_target)
+ continue;
err = verifier_remove_insns(env, i, 1);
if (err)
@@ -771,7 +769,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
insn_buf[cnt++] = BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1,
-subprogs[0].stack_depth);
insn_buf[cnt++] = env->prog->insnsi[0];
- new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+ new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+ BPF_PREPEND);
if (!new_prog)
return -ENOMEM;
env->prog = new_prog;
@@ -794,7 +793,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
verifier_bug(env, "prologue is too long");
return -EFAULT;
} else if (cnt) {
- new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+ new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+ BPF_PREPEND);
if (!new_prog)
return -ENOMEM;
@@ -2459,7 +2459,8 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env)
/* Copy first actual insn to preserve it */
insn_buf[cnt++] = env->prog->insnsi[subprog_start];
- new_prog = bpf_patch_insn_data(env, subprog_start, insn_buf, cnt);
+ new_prog = __bpf_patch_insn_data(env, subprog_start, insn_buf, cnt,
+ BPF_PREPEND);
if (!new_prog)
return -ENOMEM;
env->prog = prog = new_prog;
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 301fc60bddc43..08592fdb55103 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -248,9 +248,12 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
struct bpf_iarray *succ, *jt;
int insn_sz;
- jt = env->insn_aux_data[idx].jt;
- if (unlikely(jt))
- return jt;
+ /* All gotox of a subprogram share its jump table, see compute_subprog_jts() */
+ if (unlikely(insn_is_gotox(insn))) {
+ jt = bpf_find_containing_subprog(env, idx)->jt;
+ if (jt)
+ return jt;
+ }
/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */
succ = env->succ;
@@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
if (opcode_info->can_jump)
succ->items[succ->cnt++] = idx + bpf_jmp_offset(insn) + 1;
+ /*
+ * tail_call upon success and ld_{abs,ind} upon load failure return
+ * from the subprogram, which is a hidden edge to its exit.
+ */
+ if (unlikely((bpf_helper_call(insn) && insn->imm == BPF_FUNC_tail_call) ||
+ (BPF_CLASS(insn->code) == BPF_LD &&
+ (BPF_MODE(insn->code) == BPF_ABS || BPF_MODE(insn->code) == BPF_IND))))
+ succ->items[succ->cnt++] = bpf_find_containing_subprog(env, idx)->exit_idx;
+
return succ;
}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index ddba53eaa3331..dc3375fcfab9d 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18191,11 +18191,43 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,
return 0;
}
+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */
+static bool jt_contains(const struct bpf_iarray *jt, u32 target)
+{
+ int l = 0, r = jt->cnt - 1, m;
+
+ while (l <= r) {
+ m = l + (r - l) / 2;
+ if (jt->items[m] == target)
+ return true;
+ if (jt->items[m] < target)
+ l = m + 1;
+ else
+ r = m - 1;
+ }
+ return false;
+}
+
+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,
+ u32 subprog_start, u32 subprog_end)
+{
+ verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n",
+ env->insn_idx, target, subprog_start, subprog_end);
+ bpf_diag_program_structure(env, env->insn_idx, "indirect jump leaves subprogram",
+ "Keep every reachable jump-table target inside the subprogram of the indirect jump.",
+ "Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).",
+ env->insn_idx, target, subprog_start, subprog_end);
+ return -EINVAL;
+}
+
/* gotox *dst_reg */
static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)
{
struct bpf_verifier_state *other_branch;
+ struct bpf_subprog_info *subprog;
+ u32 subprog_start, subprog_end;
struct bpf_reg_state *dst_reg;
+ struct bpf_iarray *jt;
struct bpf_map *map;
u32 min_index, max_index;
int err = 0;
@@ -18238,6 +18270,29 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
return -EINVAL;
}
+ subprog = bpf_find_containing_subprog(env, env->insn_idx);
+ if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
+ return -EFAULT;
+ subprog_start = subprog->start;
+ subprog_end = (subprog + 1)->start;
+
+ jt = subprog->jt;
+ if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
+ return -EFAULT;
+
+ for (i = 0; i < n; i++) {
+ u32 target = env->gotox_tmp_buf->items[i];
+
+ if (target < subprog_start || target >= subprog_end)
+ return reject_gotox_out_of_subprog(env, target, subprog_start,
+ subprog_end);
+ /* Maps are confined to a subprog, see compute_subprog_jts() */
+ if (verifier_bug_if(!jt_contains(jt, target), env,
+ "insn %d target %u is not in the jump table of its subprog",
+ env->insn_idx, target))
+ return -EFAULT;
+ }
+
for (i = 0; i < n - 1; i++) {
mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
@@ -21477,8 +21532,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
release_maps(env);
release_btfs(env);
err_free_env:
- if (env->insn_aux_data)
- bpf_clear_insn_aux_data(env, 0, env->insn_aux_data_len);
+ bpf_free_subprog_jts(env);
vfree(env->insn_aux_data);
kvfree(env->fd_array);
bpf_stack_liveness_free(env);
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 0222a9a5d0761..f102435675c2b 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
#include <bpf/bpf.h>
+#include <bpf/btf.h>
#include <test_progs.h>
#if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)
@@ -453,43 +454,1083 @@ static void check_bpf_no_lookup(void)
close(map_fd);
}
+#define GOTOX_CNT_AT_LIMIT 1000
+#define GOTOX_LOG_SZ (256 * 1024)
+
+static const char gotox_limit_msg[] =
+ "number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+ /* the run of gotox itself, plus the insn right after it */
+ const __u32 jt_cnt = gotox_cnt + 1;
+ struct bpf_insn_array_value val = {};
+ int map_fd;
+ __u32 i;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return map_fd;
+
+ for (i = 0; i < jt_cnt; i++) {
+ val.orig_off = first_gotox + i;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+ "bpf_map_update_elem"))
+ goto err;
+ }
+
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto err;
+
+ return map_fd;
+err:
+ close(map_fd);
+ return -1;
+}
+
+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
+ int *fd_array, __u32 fd_array_cnt, char *log,
+ int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
+{
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ int prog_fd;
+
+ log[0] = 0;
+ opts.fd_array = fd_array;
+ opts.fd_array_cnt = fd_array_cnt;
+ opts.log_buf = log;
+ opts.log_size = GOTOX_LOG_SZ;
+ opts.log_level = 1;
+ if (fi_cnt) {
+ opts.prog_btf_fd = btf_fd;
+ opts.func_info = fi;
+ opts.func_info_cnt = fi_cnt;
+ opts.func_info_rec_size = sizeof(*fi);
+ }
+
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+ if (prog_fd >= 0) {
+ close(prog_fd);
+ return 0;
+ }
+ return prog_fd;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+ int *fd_array, __u32 fd_array_cnt, char *log)
+{
+ return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,
+ -1, NULL, 0);
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+ __u32 i;
+
+ insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+ for (i = 1; i <= gotox_cnt; i++)
+ insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+ ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+ const __u32 insn_cnt = gotox_cnt + 3;
+ struct bpf_insn *insns;
+ bool attempted = false;
+ int map_fd;
+
+ insns = calloc(insn_cnt, sizeof(*insns));
+ if (!ASSERT_OK_PTR(insns, "calloc insns"))
+ return false;
+
+ gotox_run_fill(insns, gotox_cnt);
+ insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+ map_fd = gotox_jt_create(1, gotox_cnt);
+ if (map_fd < 0)
+ goto free_insns;
+
+ *err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+ close(map_fd);
+ attempted = true;
+free_insns:
+ free(insns);
+ return attempted;
+}
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+ const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ if (try_load_gotox_prog(gotox_cnt, log, &err))
+ check_gotox_limit_hit(log, err);
+
+ free(log);
+}
+
+/*
+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
+ * table holds the starts of the blocks that follow plus the exit block,
+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
+ * exactly at the limit and must still load.
+ */
+#define GOTOX_BLOCK_SZ 4
+
+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+ struct bpf_insn *at;
+ __u32 k;
+
+ for (k = 0; k < gotox_cnt; k++) {
+ at = insns + k * GOTOX_BLOCK_SZ;
+
+ /* r1 = &jt[0], by index 0 into fd_array */
+ at[0] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ at[1] = (struct bpf_insn) { .imm = 0 };
+ at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
+ at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ }
+
+ insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
+}
+
+static int gotox_chain_jt_create(__u32 gotox_cnt)
+{
+ struct bpf_insn_array_value val = {};
+ int map_fd;
+ __u32 i;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return map_fd;
+
+ for (i = 0; i < gotox_cnt; i++) {
+ val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+ "bpf_map_update_elem"))
+ goto err;
+ }
+
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto err;
+
+ return map_fd;
+err:
+ close(map_fd);
+ return -1;
+}
+
+static void check_gotox_edges_at_limit(void)
+{
+ const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+ const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
+ struct bpf_insn *insns;
+ char *log;
+ int map_fd, err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ insns = calloc(insn_cnt, sizeof(*insns));
+ if (!ASSERT_OK_PTR(insns, "calloc insns"))
+ goto free_log;
+
+ gotox_chain_fill(insns, gotox_cnt);
+
+ map_fd = gotox_chain_jt_create(gotox_cnt);
+ if (map_fd < 0)
+ goto free_insns;
+
+ err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+ close(map_fd);
+
+ if (!ASSERT_OK(err, "program at the edge limit should load"))
+ fprintf(stderr, "verifier log: %s\n", log);
+
+free_insns:
+ free(insns);
+free_log:
+ free(log);
+}
+
+static void check_gotox_edges_across_subprogs(void)
+{
+ const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
+ const __u32 sub_start = gotox_cnt + 3;
+ const __u32 insn_cnt = 2 * (gotox_cnt + 3);
+ int map_fd[2] = { -1, -1 };
+ struct bpf_insn *insns;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ insns = calloc(insn_cnt, sizeof(*insns));
+ if (!ASSERT_OK_PTR(insns, "calloc insns"))
+ goto free_log;
+
+ gotox_run_fill(insns, gotox_cnt);
+ insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
+ BPF_PSEUDO_CALL, 0,
+ sub_start - (gotox_cnt + 1) - 1);
+ insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+ gotox_run_fill(insns + sub_start, gotox_cnt);
+ insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
+
+ map_fd[0] = gotox_jt_create(1, gotox_cnt);
+ if (map_fd[0] < 0)
+ goto free_insns;
+ map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
+ check_gotox_limit_hit(log, err);
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_insns:
+ free(insns);
+free_log:
+ free(log);
+}
+
+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
+{
+ struct bpf_insn_array_value val = {};
+ int map_fd;
+ __u32 i;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return map_fd;
+
+ for (i = 0; i < cnt; i++) {
+ val.orig_off = offs[i];
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+ "bpf_map_update_elem"))
+ goto err;
+ }
+
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto err;
+
+ return map_fd;
+err:
+ close(map_fd);
+ return -1;
+}
+
+#define GOTOX_SUB_START 4
+#define GOTOX_MAIN_TGT 2
+#define GOTOX_SUB_TGT 8
+#define GOTOX_TWO_INSN_CNT 10
+
+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)
+{
+ insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+ GOTOX_SUB_START - 1 - 1);
+ insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[3] = BPF_EXIT_INSN();
+
+ /* r1 = &jt[0], by index 'jt_idx' into fd_array */
+ insns[GOTOX_SUB_START] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = jt_idx,
+ };
+ insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };
+ insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);
+ insns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[9] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_other_subprog(void)
+{
+ const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+ const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+ struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_two_subprogs_fill(insns, 0, 0);
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+static void check_gotox_jt_per_subprog(void)
+{
+ const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+ const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+ struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_two_subprogs_fill(insns, 1, 0);
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+#define GOTOX_FWD_GOTOX 11
+#define GOTOX_FWD_OWN_TGT 12
+#define GOTOX_FWD_SUB_START 14
+#define GOTOX_FWD_INSN_CNT 16
+
+static void gotox_from_main_fill(struct bpf_insn *insns)
+{
+ insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);
+ insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+ GOTOX_FWD_SUB_START - 1 - 1);
+ insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,
+ offsetof(struct xdp_md, ingress_ifindex));
+ insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);
+
+ /* r1 = &jt_leaves[0], by index 1 into fd_array */
+ insns[4] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 1,
+ };
+ insns[5] = (struct bpf_insn) { .imm = 0 };
+ insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+ insns[7] = BPF_JMP_A(3);
+
+ /* r1 = &jt_own[0], by index 0 into fd_array */
+ insns[8] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ insns[9] = (struct bpf_insn) { .imm = 0 };
+ insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+ insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[13] = BPF_EXIT_INSN();
+ insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[15] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_subprog_from_main(void)
+{
+ const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+ const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+ struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_from_main_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+/*
+ * The only map of the subprog holding the gotox reaches past that subprog, so
+ * the subprog is left without a jump table at all.
+ */
+static void check_gotox_jt_spans_subprogs(void)
+{
+ const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+ const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+ struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_from_main_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+/*
+ * The subprog holding the gotox has a well formed jump table of its own and
+ * also collects a map that reaches past its end. The spanning map is still
+ * rejected, even though the subprog is not left without a table.
+ */
+static void check_gotox_jt_spans_with_own_table(void)
+{
+ const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+ const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+ struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_from_main_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+#define GOTOX_SLICE_SUB_START 6
+#define GOTOX_SLICE_GOTOX 14
+#define GOTOX_SLICE_SUB_TGT 15
+#define GOTOX_SLICE_INSN_CNT 17
+
+static void gotox_slice_fill(struct bpf_insn *insns)
+{
+ insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+ GOTOX_SLICE_SUB_START - 1 - 1);
+ insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[5] = BPF_EXIT_INSN();
+
+ insns[GOTOX_SLICE_SUB_START] =
+ BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+ offsetof(struct xdp_md, ingress_ifindex));
+ insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);
+ insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);
+ insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+
+ /* r1 = &jt_main[0], by index 0 into fd_array */
+ insns[10] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ insns[11] = (struct bpf_insn) { .imm = 0 };
+ insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);
+ insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+ insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[16] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_index_slice_other_subprog(void)
+{
+ const __u32 jt_main[] = { 2, 3, 4 };
+ const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };
+ struct bpf_insn insns[GOTOX_SLICE_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_slice_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,
+ struct bpf_func_info *fi, struct btf **pbtf)
+{
+ int int_id, proto_id, id;
+ struct btf *btf;
+ char name[24];
+ __u32 i;
+
+ btf = btf__new_empty();
+ if (!ASSERT_OK_PTR(btf, "btf__new_empty"))
+ return -1;
+
+ int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED);
+ if (!ASSERT_GT(int_id, 0, "btf__add_int"))
+ goto err;
+
+ proto_id = btf__add_func_proto(btf, int_id);
+ if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto"))
+ goto err;
+
+ for (i = 0; i < cnt; i++) {
+ snprintf(name, sizeof(name), "gotox_f%u", i);
+ id = btf__add_func(btf, name, linkage[i], proto_id);
+ if (!ASSERT_GT(id, 0, "btf__add_func"))
+ goto err;
+ fi[i].insn_off = starts[i];
+ fi[i].type_id = id;
+ }
+
+ if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel"))
+ goto err;
+
+ *pbtf = btf;
+ return btf__fd(btf);
+err:
+ btf__free(btf);
+ return -1;
+}
+
+static void check_gotox_target_other_global_subprog(void)
+{
+ const __u32 starts[] = { 0, GOTOX_SUB_START };
+ const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };
+ const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+ const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+ struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ struct bpf_func_info fi[2];
+ struct btf *btf = NULL;
+ int btf_fd;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_two_subprogs_fill(insns, 0, 0);
+
+ btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+ if (btf_fd < 0)
+ goto free_log;
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_btf;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+ btf_fd, fi, ARRAY_SIZE(fi));
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_btf:
+ btf__free(btf);
+free_log:
+ free(log);
+}
+
+#define GOTOX_CB_MAIN_TGT 6
+#define GOTOX_CB_START 8
+#define GOTOX_CB_GOTOX 11
+#define GOTOX_CB_TGT 12
+#define GOTOX_CB_INSN_CNT 14
+
+static void gotox_callback_fill(struct bpf_insn *insns)
+{
+ insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);
+ /* r2 = &callback */
+ insns[1] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_2,
+ .src_reg = BPF_PSEUDO_FUNC,
+ .imm = GOTOX_CB_START - 1 - 1,
+ };
+ insns[2] = (struct bpf_insn) { .imm = 0 };
+ insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);
+ insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);
+ insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);
+ insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[7] = BPF_EXIT_INSN();
+
+ /* r1 = &jt_main[0], by index 0 into fd_array */
+ insns[GOTOX_CB_START] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ insns[9] = (struct bpf_insn) { .imm = 0 };
+ insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+ insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[13] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_callback_leaves_subprog(void)
+{
+ const __u32 starts[] = { 0, GOTOX_CB_START };
+ const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };
+ const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };
+ const __u32 jt_cb[] = { GOTOX_CB_TGT };
+ struct bpf_insn insns[GOTOX_CB_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ struct bpf_func_info fi[2];
+ struct btf *btf = NULL;
+ int btf_fd;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_callback_fill(insns);
+
+ btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+ if (btf_fd < 0)
+ goto free_log;
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_btf;
+ map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+ btf_fd, fi, ARRAY_SIZE(fi));
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_btf:
+ btf__free(btf);
+free_log:
+ free(log);
+}
+
static void check_bpf_side(void)
{
check_bpf_no_lookup();
}
+static void check_gotox_target_nop(void)
+{
+ struct bpf_insn insns[] = {
+ /* r1 = &jt[0] */
+ BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* insn 4: the nop target */
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ struct bpf_insn_array_value val = {};
+ int prog_fd = -1, map_fd;
+ __u32 key = 0;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return;
+
+ val.orig_off = 4;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto cleanup;
+
+ opts.fd_array = &map_fd;
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+ goto cleanup;
+ ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target must not be INSN_DELETED");
+ ASSERT_NEQ(val.jitted_off, 0, "gotox target must have a jitted address");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ close(map_fd);
+}
+
+static void check_insn_array_stale_reuse(void)
+{
+ /* helper of a different prog type, to set the prog array's owner */
+ struct bpf_insn sf[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ };
+ /*
+ * prog1 binds the insn array (entry points at a real insn, so the JIT
+ * fills it in) and tail calls a prog array whose owner is now a
+ * different type, which fails in the post-JIT compatibility check.
+ */
+ struct bpf_insn prog1[] = {
+ BPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */
+ BPF_MOV64_IMM(BPF_REG_3, 0), /* r3 = index */
+ BPF_EMIT_CALL(BPF_FUNC_tail_call),
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 4: real entry */
+ BPF_EXIT_INSN(),
+ };
+ /*
+ * prog2 reuses the insn array as a tracker. Its entry points at a nop
+ * that the nop pass removes, so the entry becomes INSN_DELETED and the
+ * JIT skips it -- exactly the slot a stale pointer would linger in.
+ */
+ struct bpf_insn prog2[] = {
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* insn 4: tracked nop */
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ int jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;
+ struct bpf_insn_array_value val = {};
+ int fd_array[2];
+ __u32 key = 0;
+
+ jt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(jt_fd, 0, "insn_array create"))
+ return;
+ val.orig_off = 4;
+ if (!ASSERT_EQ(bpf_map_update_elem(jt_fd, &key, &val, 0), 0, "insn_array update"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, "insn_array freeze"))
+ goto cleanup;
+
+ pa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, "pa", 4, 4, 1, NULL);
+ if (!ASSERT_GE(pa_fd, 0, "prog_array create"))
+ goto cleanup;
+ sf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", sf, ARRAY_SIZE(sf), NULL);
+ if (!ASSERT_GE(sf_fd, 0, "owner prog load"))
+ goto cleanup;
+ /* insert a non-XDP prog to pin the prog array owner to another type */
+ if (!ASSERT_EQ(bpf_map_update_elem(pa_fd, &key, &sf_fd, 0), 0, "prog_array update"))
+ goto cleanup;
+
+ fd_array[0] = jt_fd;
+ fd_array[1] = pa_fd;
+ opts.fd_array = fd_array;
+ opts.fd_array_cnt = 2;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog1, ARRAY_SIZE(prog1), &opts);
+ if (!ASSERT_LT(prog_fd, 0, "prog1 must fail the post-JIT tail call check"))
+ goto cleanup;
+
+ /* prog1 reached the JIT before failing, so the entry is now populated */
+ if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog1"))
+ goto cleanup;
+ if (!ASSERT_NEQ(val.jitted_off, 0, "prog1 should have filled the jitted address"))
+ goto cleanup;
+
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog2, ARRAY_SIZE(prog2), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "prog2 reuse load"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog2"))
+ goto cleanup;
+ ASSERT_EQ(val.xlated_off, (__u32)-1, "reused entry should be INSN_DELETED");
+ ASSERT_EQ(val.jitted_off, 0, "stale jitted address must be cleared on reuse");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ if (sf_fd >= 0)
+ close(sf_fd);
+ if (pa_fd >= 0)
+ close(pa_fd);
+ if (jt_fd >= 0)
+ close(jt_fd);
+}
+
+static void check_gotox_target_prologue_shift(void)
+{
+ struct bpf_insn insns[] = {
+ /* insn 0: gotox target and subprog start */
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ /* may_goto +4 -> exit block, bounds the loop */
+ BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),
+ /* r1 = &jt[0] (insns 2 and 3) */
+ BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */
+ /* insn 5: gotox r1 -> insn 0 */
+ BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ struct bpf_insn_array_value val = {};
+ int prog_fd = -1, map_fd;
+ __u32 key = 0;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return;
+
+ val.orig_off = 0;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto cleanup;
+
+ opts.fd_array = &map_fd;
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+ goto cleanup;
+ ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+ ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+ ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ close(map_fd);
+}
+
+static void check_gotox_target_ctx_prologue_shift(void)
+{
+ struct bpf_insn insns[] = {
+ /* insn 0: gotox target and subprog start */
+ BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */
+ BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),
+ BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),
+ BPF_MOV64_REG(BPF_REG_4, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),
+ /* insn 9: direct packet write -> tc unclone prologue prepended at insn 0 */
+ BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),
+ /* r5 = &jt[0] (insns 10 and 11) */
+ BPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */
+ /* insn 13: gotox r5 -> insn 0 */
+ BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),
+ BPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ struct bpf_insn_array_value val = {};
+ int prog_fd = -1, map_fd;
+ __u32 key = 0;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return;
+
+ val.orig_off = 0;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto cleanup;
+
+ opts.fd_array = &map_fd;
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns,
+ ARRAY_SIZE(insns), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+ goto cleanup;
+ ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+ ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+ ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ close(map_fd);
+}
+
static void __test_bpf_insn_array(void)
{
/* Test if offsets are adjusted properly */
-
if (test__start_subtest("one2one"))
check_one_to_one_mapping();
-
if (test__start_subtest("simple"))
check_simple();
-
if (test__start_subtest("deletions"))
check_deletions();
-
if (test__start_subtest("deletions-with-functions"))
check_deletions_with_functions();
-
if (test__start_subtest("blindness"))
check_blindness();
-
/* Check all kinds of operations and related restrictions */
-
if (test__start_subtest("incorrect-index"))
check_incorrect_index();
-
if (test__start_subtest("load-unfrozen-map"))
check_load_unfrozen_map();
-
if (test__start_subtest("no-map-reuse"))
check_no_map_reuse();
-
if (test__start_subtest("bpf-side-ops"))
check_bpf_side();
+ if (test__start_subtest("too-many-gotox-edges"))
+ check_too_many_gotox_edges();
+ if (test__start_subtest("gotox-edges-at-limit"))
+ check_gotox_edges_at_limit();
+ if (test__start_subtest("gotox-edges-across-subprogs"))
+ check_gotox_edges_across_subprogs();
+ if (test__start_subtest("gotox-jt-spans-subprogs"))
+ check_gotox_jt_spans_subprogs();
+ if (test__start_subtest("gotox-jt-spans-with-own-table"))
+ check_gotox_jt_spans_with_own_table();
+ if (test__start_subtest("gotox-target-other-subprog"))
+ check_gotox_target_other_subprog();
+ if (test__start_subtest("gotox-jt-per-subprog"))
+ check_gotox_jt_per_subprog();
+ if (test__start_subtest("gotox-target-subprog-from-main"))
+ check_gotox_target_subprog_from_main();
+ if (test__start_subtest("gotox-index-slice-other-subprog"))
+ check_gotox_index_slice_other_subprog();
+ if (test__start_subtest("gotox-target-other-global-subprog"))
+ check_gotox_target_other_global_subprog();
+ if (test__start_subtest("gotox-callback-leaves-subprog"))
+ check_gotox_callback_leaves_subprog();
+ if (test__start_subtest("gotox-target-nop"))
+ check_gotox_target_nop();
+ if (test__start_subtest("insn-array-stale-reuse"))
+ check_insn_array_stale_reuse();
+ if (test__start_subtest("gotox-target-prologue-shift"))
+ check_gotox_target_prologue_shift();
+ if (test__start_subtest("gotox-target-ctx-prologue-shift"))
+ check_gotox_target_ctx_prologue_shift();
}
#else
static void __test_bpf_insn_array(void)
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index 0e27c2c79c57e..4931a77ba0b26 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -366,7 +366,7 @@ __used static int test_subprog(void)
}
SEC("socket")
-__failure __msg("jump table for insn 4 points outside of the subprog [0,10]")
+__failure __msg("jump table of subprog starting at 0 spans multiple subprogs")
__naked void jump_table_outside_subprog(void)
{
asm volatile (" \
|
| 3/2 | 2026/09/14 21:21 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 30218 output: 31 thoughts: 479 Thoughts: **My Initial Assessment of the BPF Patch Series** Okay, so I've been asked to evaluate this patch series in the kernel workspace. My initial thought process is to quickly get a handle on what's being touched, and whether this warrants a closer look from a fuzzing perspective. First things first, I see that this patch modifies quite a few files: `include/linux/bpf.h`, `include/linux/bpf_verifier.h`, and a good chunk of the core BPF subsystem files within `kernel/bpf/` – `bpf_insn_array.c`, `cfg.c`, `fixups.c`, `liveness.c`, and `verifier.c` as well as changes to the self-tests in `tools/testing/selftests/bpf/...`. That immediately flags BPF as a core subsystem – it's reachable via the `bpf` syscall in standard QEMU/x86_64 environments, according to the instructions. This pretty much confirms that `WorthFuzzing` is going to be `true`. We're definitely looking at code that's worth the investment in further analysis. Now, my next step is to drill down and extract some key details. I need to define the outputs: 1. `WorthFuzzing`: Definitely `true`, given the core BPF involvement. 2. `FocusSymbols`: I'll need to identify specific functions that are added, modified (especially if they're not ubiquitous), or if there are any direct API entry points that are now reachable. This is where I'll use `git` and `grep` to quickly go through the patches and figure out what the new functions are and what calls them. 3. `EnableConfigs`: I need to determine what kernel configurations might be required to enable the new functionality or to exercise the modified code paths. More `git` and `grep` work should reveal these. 4. `Reasoning`: I'll formulate a concise explanation of why this patch series warrants fuzzing, likely highlighting the core BPF modifications and any new attack surface that's been exposed. I'll start with the git inspections, and see what the code has to offer! |
| 4/2 | 2026/09/14 21:21 | tool | grepper |
0mArgs:{
"Expression": "bpf_insn_array",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/Makefile=11=obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o bloom_filter.o\nkernel/bpf/Makefile:12:obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o bpf_insn_array.o\nkernel/bpf/Makefile-13-obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o\n--\nkernel/bpf/bpf_insn_array.c-5-\nkernel/bpf/bpf_insn_array.c:6:struct bpf_insn_array {\nkernel/bpf/bpf_insn_array.c-7-\tstruct bpf_map map;\n--\nkernel/bpf/bpf_insn_array.c-9-\tlong *ips;\nkernel/bpf/bpf_insn_array.c:10:\tDECLARE_FLEX_ARRAY(struct bpf_insn_array_value, values);\nkernel/bpf/bpf_insn_array.c-11-};\n--\nkernel/bpf/bpf_insn_array.c-13-#define cast_insn_array(MAP_PTR) \\\nkernel/bpf/bpf_insn_array.c:14:\tcontainer_of((MAP_PTR), struct bpf_insn_array, map)\nkernel/bpf/bpf_insn_array.c-15-\n--\nkernel/bpf/bpf_insn_array.c=18=static inline u64 insn_array_alloc_size(u32 max_entries)\nkernel/bpf/bpf_insn_array.c-19-{\nkernel/bpf/bpf_insn_array.c:20:\tconst u64 base_size = sizeof(struct bpf_insn_array);\nkernel/bpf/bpf_insn_array.c:21:\tconst u64 entry_size = sizeof(struct bpf_insn_array_value);\nkernel/bpf/bpf_insn_array.c-22-\n--\nkernel/bpf/bpf_insn_array.c=26=static int insn_array_alloc_check(union bpf_attr *attr)\nkernel/bpf/bpf_insn_array.c-27-{\nkernel/bpf/bpf_insn_array.c:28:\tu32 value_size = sizeof(struct bpf_insn_array_value);\nkernel/bpf/bpf_insn_array.c-29-\n--\nkernel/bpf/bpf_insn_array.c=37=static void insn_array_free(struct bpf_map *map)\nkernel/bpf/bpf_insn_array.c-38-{\nkernel/bpf/bpf_insn_array.c:39:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-40-\n--\nkernel/bpf/bpf_insn_array.c=44=static struct bpf_map *insn_array_alloc(union bpf_attr *attr)\n--\nkernel/bpf/bpf_insn_array.c-46-\tu64 size = insn_array_alloc_size(attr-\u003emax_entries);\nkernel/bpf/bpf_insn_array.c:47:\tstruct bpf_insn_array *insn_array;\nkernel/bpf/bpf_insn_array.c-48-\n--\nkernel/bpf/bpf_insn_array.c=64=static void *insn_array_lookup_elem(struct bpf_map *map, void *key)\nkernel/bpf/bpf_insn_array.c-65-{\nkernel/bpf/bpf_insn_array.c:66:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-67-\tu32 index = *(u32 *)key;\n--\nkernel/bpf/bpf_insn_array.c=75=static long insn_array_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags)\nkernel/bpf/bpf_insn_array.c-76-{\nkernel/bpf/bpf_insn_array.c:77:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-78-\tu32 index = *(u32 *)key;\nkernel/bpf/bpf_insn_array.c:79:\tstruct bpf_insn_array_value val = {};\nkernel/bpf/bpf_insn_array.c-80-\n--\nkernel/bpf/bpf_insn_array.c=120=static int insn_array_map_direct_value_addr(const struct bpf_map *map, u64 *imm, u32 off)\nkernel/bpf/bpf_insn_array.c-121-{\nkernel/bpf/bpf_insn_array.c:122:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-123-\n--\nkernel/bpf/bpf_insn_array.c-133-\nkernel/bpf/bpf_insn_array.c:134:BTF_ID_LIST_SINGLE(insn_array_btf_ids, struct, bpf_insn_array)\nkernel/bpf/bpf_insn_array.c-135-\n--\nkernel/bpf/bpf_insn_array.c=157=static bool is_insn_array(const struct bpf_map *map)\n--\nkernel/bpf/bpf_insn_array.c-161-\nkernel/bpf/bpf_insn_array.c:162:static inline bool valid_offsets(const struct bpf_insn_array *insn_array,\nkernel/bpf/bpf_insn_array.c-163-\t\t\t\t const struct bpf_prog *prog)\n--\nkernel/bpf/bpf_insn_array.c-182-\nkernel/bpf/bpf_insn_array.c:183:int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)\nkernel/bpf/bpf_insn_array.c-184-{\nkernel/bpf/bpf_insn_array.c:185:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c:186:\tstruct bpf_insn_array_value *values = insn_array-\u003evalues;\nkernel/bpf/bpf_insn_array.c-187-\tint i;\n--\nkernel/bpf/bpf_insn_array.c-215-\nkernel/bpf/bpf_insn_array.c:216:int bpf_insn_array_ready(struct bpf_map *map)\nkernel/bpf/bpf_insn_array.c-217-{\nkernel/bpf/bpf_insn_array.c:218:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-219-\tint i;\n--\nkernel/bpf/bpf_insn_array.c-230-\nkernel/bpf/bpf_insn_array.c:231:void bpf_insn_array_release(struct bpf_map *map)\nkernel/bpf/bpf_insn_array.c-232-{\nkernel/bpf/bpf_insn_array.c:233:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-234-\n--\nkernel/bpf/bpf_insn_array.c-237-\nkernel/bpf/bpf_insn_array.c:238:void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)\nkernel/bpf/bpf_insn_array.c-239-{\nkernel/bpf/bpf_insn_array.c:240:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-241-\tint i;\n--\nkernel/bpf/bpf_insn_array.c-254-\nkernel/bpf/bpf_insn_array.c:255:void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len)\nkernel/bpf/bpf_insn_array.c-256-{\nkernel/bpf/bpf_insn_array.c:257:\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c-258-\tint i;\n--\nkernel/bpf/bpf_insn_array.c=278=void bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image)\nkernel/bpf/bpf_insn_array.c-279-{\nkernel/bpf/bpf_insn_array.c:280:\tstruct bpf_insn_array *insn_array;\nkernel/bpf/bpf_insn_array.c-281-\tstruct bpf_map *map;\n--\nkernel/bpf/cfg.c=204=static int copy_insn_array(struct bpf_map *map, u32 start, u32 end, u32 *items)\nkernel/bpf/cfg.c-205-{\nkernel/bpf/cfg.c:206:\tstruct bpf_insn_array_value *value;\nkernel/bpf/cfg.c-207-\tu32 i;\n--\nkernel/bpf/fixups.c=225=static void adjust_insn_aux_data(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-258-\t * actually the first instruction, so move it back. This also matches with the behavior\nkernel/bpf/fixups.c:259:\t * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new\nkernel/bpf/fixups.c-260-\t * instruction. For BPF_PREPEND the original instruction is the last one, so the flag\n--\nkernel/bpf/fixups.c=283=static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 first, u32 len)\n--\nkernel/bpf/fixups.c-290-\tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++)\nkernel/bpf/fixups.c:291:\t\tbpf_insn_array_adjust(env-\u003einsn_array_maps[i], first, len);\nkernel/bpf/fixups.c-292-}\n--\nkernel/bpf/fixups.c=294=static void adjust_insn_arrays_after_remove(struct bpf_verifier_env *env, u32 off, u32 len)\n--\nkernel/bpf/fixups.c-298-\tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++)\nkernel/bpf/fixups.c:299:\t\tbpf_insn_array_adjust_after_remove(env-\u003einsn_array_maps[i], off, len);\nkernel/bpf/fixups.c-300-}\n--\nkernel/bpf/syscall.c=2892=static int bpf_prog_mark_insn_arrays_ready(struct bpf_prog *prog)\n--\nkernel/bpf/syscall.c-2900-\nkernel/bpf/syscall.c:2901:\t\terr = bpf_insn_array_ready(prog-\u003eaux-\u003eused_maps[i]);\nkernel/bpf/syscall.c-2902-\t\tif (err)\n--\nkernel/bpf/verifier.c=4675=static int check_map_kptr_access(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-4747- * Return the size of the memory region accessible from a pointer to map value.\nkernel/bpf/verifier.c:4748: * For INSN_ARRAY maps whole bpf_insn_array-\u003eips array is accessible.\nkernel/bpf/verifier.c-4749- */\n--\nkernel/bpf/verifier.c=18953=static int __add_used_map(struct bpf_verifier_env *env, struct bpf_map *map)\n--\nkernel/bpf/verifier.c-18990-\tif (map-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY) {\nkernel/bpf/verifier.c:18991:\t\terr = bpf_insn_array_init(map, env-\u003eprog);\nkernel/bpf/verifier.c-18992-\t\tif (err) {\n--\nkernel/bpf/verifier.c=19427=static void release_insn_arrays(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-19431-\tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++)\nkernel/bpf/verifier.c:19432:\t\tbpf_insn_array_release(env-\u003einsn_array_maps[i]);\nkernel/bpf/verifier.c-19433-}\n"
} |
| 5/2 | 2026/09/14 21:21 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 34882 output: 35 thoughts: 1391 Thoughts: **Analyzing the BPF Code Changes** Okay, let's dive into these BPF changes. The first thing I'm doing is a detailed scan of this diff. I don't have a direct Git tool at hand, but this provided context is enough to get a grasp on things. My initial impression is that this is a significant refactoring, particularly around indirect jumps (`gotox`) and subprograms. It looks like they're reworking how jump tables (`jt`) are handled, aiming for a more structured and efficient approach. I am particularly curious to understand the reason behind moving `struct bpf_iarray *jt;` from `struct bpf_insn_aux_data` to `struct bpf_subprog_info`. That strikes me as a significant architectural shift. I see they've introduced the concept of a combined jump table for subprograms, which makes sense. Using `compute_subprog_jts` to gather all of the per-subprogram jump tables into a unified jump table. They are also adding bounds checking for these jump tables with a `subprog_jt` function, which is critical for security. The introduction of `u32 gotox_edges` and `bool subprog_jts_ready` within the `bpf_verifier_env` suggests they are tracking and managing edge counts and jump table readiness more explicitly. The `BPF_MAX_GOTOX_EDGES` limit is a good addition to prevent potential resource exhaustion. This has the feeling of a more robust way to model and verify the control flow. The `bpf_insn_array_adjust` changes look pretty straightforward – they appear to be addressing an off-by-one error or a subtle index issue. The update to `bpf_compute_scc` to add `dfs_pos` to track DFS traversal positions suggests they've been dealing with potential recursion problems or complexities in graph traversal. The changes in `fixups.c` are interesting. They are introducing `enum bpf_patch_mode` (append/prepend) which signals that there are new modes of patch manipulation. They're refining how instruction data is manipulated during the patch process. I need to fully grok the use of the different patching modes, particularly with regard to how they relate to the `indirect_target` flag. I am curious about the implications for performance. The adjustments to the liveness analysis (`liveness.c`) with respect to `gotox` instructions and subprogram jump tables also make sense in light of the previous updates. The use of the subprogram-specific jump table is a good example of the modularity of the design. Finally, in `verifier.c`, the new `jt_contains` function with binary search is a performance improvement for checking jump table targets, and the `reject_gotox_out_of_subprog` checks further reinforce the security improvements and explicit bounds-checking approach. Replacing `bpf_clear_insn_aux_data` with `bpf_free_subprog_jts` is a clear indication that jump tables are the focus of memory management for indirect jumps. Overall, it looks like a concerted effort to improve BPF's indirect jump capabilities, security, and verification robustness. I'll need to dig into the Kconfig options to see how these features are enabled, but this seems like a solid step forward for BPF's control-flow handling. |
| 6/2 | 2026/09/14 21:21 | tool | grepper |
0mArgs:{
"Expression": "BPF_MAP_TYPE_INSN_ARRAY",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/bpf_insn_array.c=157=static bool is_insn_array(const struct bpf_map *map)\nkernel/bpf/bpf_insn_array.c-158-{\nkernel/bpf/bpf_insn_array.c:159:\treturn map-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY;\nkernel/bpf/bpf_insn_array.c-160-}\n--\nkernel/bpf/syscall.c=1382=static int map_create_alloc(union bpf_attr *attr, bpfptr_t uattr, struct bpf_verifier_log *log,\n--\nkernel/bpf/syscall.c-1516-\tcase BPF_MAP_TYPE_ARENA:\nkernel/bpf/syscall.c:1517:\tcase BPF_MAP_TYPE_INSN_ARRAY:\nkernel/bpf/syscall.c-1518-\t\tif (!bpf_token_capable(token, CAP_BPF))\n--\nkernel/bpf/syscall.c=2892=static int bpf_prog_mark_insn_arrays_ready(struct bpf_prog *prog)\n--\nkernel/bpf/syscall.c-2897-\tfor (i = 0; i \u003c prog-\u003eaux-\u003eused_map_cnt; i++) {\nkernel/bpf/syscall.c:2898:\t\tif (prog-\u003eaux-\u003eused_maps[i]-\u003emap_type != BPF_MAP_TYPE_INSN_ARRAY)\nkernel/bpf/syscall.c-2899-\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=4750=static u32 map_mem_size(const struct bpf_map *map)\nkernel/bpf/verifier.c-4751-{\nkernel/bpf/verifier.c:4752:\tif (map-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY)\nkernel/bpf/verifier.c-4753-\t\treturn map-\u003emax_entries * sizeof(long);\n--\nkernel/bpf/verifier.c=5213=static int check_ptr_alignment(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-5234-\t\tpointer_desc = \"value \";\nkernel/bpf/verifier.c:5235:\t\tif (reg-\u003emap_ptr-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY)\nkernel/bpf/verifier.c-5236-\t\t\tstrict = true;\n--\nkernel/bpf/verifier.c=5822=int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val,\n--\nkernel/bpf/verifier.c-5828-\nkernel/bpf/verifier.c:5829:\tif (map-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY || map-\u003emap_type == BPF_MAP_TYPE_PERCPU_ARRAY)\nkernel/bpf/verifier.c-5830-\t\treturn -EINVAL;\n--\nkernel/bpf/verifier.c=6333=static int check_map_mem_read(struct bpf_verifier_env *env, struct bpf_reg_state *reg, int off,\n--\nkernel/bpf/verifier.c-6340-\tswitch (map-\u003emap_type) {\nkernel/bpf/verifier.c:6341:\tcase BPF_MAP_TYPE_INSN_ARRAY:\nkernel/bpf/verifier.c-6342-\t\tif (bpf_size != BPF_DW) {\n--\nkernel/bpf/verifier.c=8575=static int check_arg_const_str(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-8586-\nkernel/bpf/verifier.c:8587:\tif (map-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY) {\nkernel/bpf/verifier.c-8588-\t\tverbose(env, \"%s points to insn_array map which cannot be used as const string\\n\",\n--\nkernel/bpf/verifier.c=9005=static int check_map_func_compatibility(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-9132-\t\tbreak;\nkernel/bpf/verifier.c:9133:\tcase BPF_MAP_TYPE_INSN_ARRAY:\nkernel/bpf/verifier.c-9134-\t\tgoto error;\n--\nkernel/bpf/verifier.c=17177=static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-17270-\t\tdst_reg-\u003emap_ptr = map;\nkernel/bpf/verifier.c:17271:\t\tWARN_ON_ONCE(map-\u003emap_type != BPF_MAP_TYPE_INSN_ARRAY \u0026\u0026\nkernel/bpf/verifier.c-17272-\t\t\t map-\u003emax_entries != 1);\n--\nkernel/bpf/verifier.c=18224=static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-18247-\nkernel/bpf/verifier.c:18248:\tif (verifier_bug_if(map-\u003emap_type != BPF_MAP_TYPE_INSN_ARRAY, env,\nkernel/bpf/verifier.c-18249-\t\t\t \"R%d has incorrect map type %d\", insn-\u003edst_reg, map-\u003emap_type))\n--\nkernel/bpf/verifier.c=18843=static int check_map_prog_compatibility(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-18909-\t\tcase BPF_MAP_TYPE_ARENA:\nkernel/bpf/verifier.c:18910:\t\tcase BPF_MAP_TYPE_INSN_ARRAY:\nkernel/bpf/verifier.c-18911-\t\tcase BPF_MAP_TYPE_PROG_ARRAY:\n--\nkernel/bpf/verifier.c=18953=static int __add_used_map(struct bpf_verifier_env *env, struct bpf_map *map)\n--\nkernel/bpf/verifier.c-18989-\nkernel/bpf/verifier.c:18990:\tif (map-\u003emap_type == BPF_MAP_TYPE_INSN_ARRAY) {\nkernel/bpf/verifier.c-18991-\t\terr = bpf_insn_array_init(map, env-\u003eprog);\n"
} |
| 7/2 | 2026/09/14 21:21 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 38751 output: 46 thoughts: 51 Thoughts: Okay, let's dive into this. **Investigating the BPF Compilation Configuration** Right, so I need to figure out which configuration options in the kernel enable those critical BPF source files: `cfg.c`, `fixups.c`, `liveness.c`, `verifier.c`, and `bpf_insn_array.c`. My immediate thought is to check the `kernel/bpf/Makefile` file. That's the logical place to start digging for the build-time dependencies and conditional compilation flags related to BPF functionality. Let's see what reveals itself. |
| 8/2 | 2026/09/14 21:21 | tool | grepper |
0mArgs:{
"Expression": "cfg\\.o|fixups\\.o|bpf_insn_array\\.o|verifier\\.o",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/Makefile=7=CFLAGS_core.o += -Wno-override-init $(cflags-nogcse-yy)\nkernel/bpf/Makefile-8-\nkernel/bpf/Makefile:9:obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o cnum.o log.o token.o liveness.o const_fold.o diagnostics.o\nkernel/bpf/Makefile-10-obj-$(CONFIG_BPF_SYSCALL) += bpf_iter.o map_iter.o task_iter.o prog_iter.o link_iter.o\nkernel/bpf/Makefile=11=obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o bloom_filter.o\nkernel/bpf/Makefile:12:obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o bpf_insn_array.o\nkernel/bpf/Makefile-13-obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o\nkernel/bpf/Makefile:14:obj-$(CONFIG_BPF_SYSCALL) += fixups.o cfg.o states.o backtrack.o check_btf.o\nkernel/bpf/Makefile-15-obj-${CONFIG_BPF_LSM}\t += bpf_inode_storage.o\n"
} |
| 9/2 | 2026/09/14 21:21 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 39372 output: 31 thoughts: 39 Thoughts: Okay, here's my take on those thoughts, framed from an expert's perspective: **Kernel Configuration Dependencies: BPF and Syscalls** Right, so everything they're talking about hinges on `CONFIG_BPF_SYSCALL`. That's the core enabler. I need to make *absolutely* sure, before I go any further, that I've accounted for any indirect dependencies. A quick dive into the Kconfig files related to `BPF_SYSCALL` is in order. It’s not enough to just see that the primary option is enabled, I've seen too many subtle interactions to take anything for granted. There could be other options, even seemingly unrelated ones, that subtly affect the behavior or availability of BPF syscalls based on this setting. It's time to map out the complete dependency tree. |
| 10/2 | 2026/09/14 21:21 | tool | grepper |
0mError:no matches Args: {
"Expression": "config BPF_SYSCALL",
"PathPrefix": "init/Kconfig"
}Results: {
"error": "no matches"
} |
| 11/2 | 2026/09/14 21:21 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 39634 output: 21 thoughts: 19 Thoughts: Here's my thought process, as if I were thinking it: **Locating the BPF_SYSCALL Configuration** Okay, so I need to track down the definition of `BPF_SYSCALL` within the kernel's Kconfig system. This likely means I'm looking to understand a compile-time option related to BPF syscall handling, or perhaps just to verify its current state or location. The Kconfig files are the go-to place for these kinds of details. My initial thought is to probably start with a `grep` or similar search. I need to figure out where that specific symbol is defined. Is it in the top-level `Kconfig` or within a more specific subsystem's configuration file? I'll probably start broad and refine my search as I go, maybe use some command line wizardry to pinpoint the precise file. It will depend on what I know about the kernel generally but the kconfig system is quite well organized so I am not too worried. I need to find the config option, understand its dependencies, and see its default and current state. This could be useful for debugging or figuring out what needs to be enabled for BPF syscall functionality to work correctly in a specific environment. |
| 12/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "config BPF_SYSCALL"
}Results: {
"Output": "kernel/bpf/Kconfig=25=menu \"BPF subsystem\"\nkernel/bpf/Kconfig-26-\nkernel/bpf/Kconfig:27:config BPF_SYSCALL\nkernel/bpf/Kconfig-28-\tbool \"Enable bpf() system call\"\n"
} |
| 13/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 40013 output: 34 thoughts: 22 Thoughts: Here's my take on those lines from `kernel/bpf/Kconfig`, as I see it: **Examining the BPF Kconfig Section** Alright, let's dive into lines 25-55 of the `kernel/bpf/Kconfig` file. I'm focusing on the configuration options related to BPF, naturally. My expectation is that these lines define the knobs and levers we, as developers, can use to enable, disable, and fine-tune the BPF subsystem within the kernel. I'm looking for the key components. I immediately want to see the main switches: are we talking about core BPF functionality, perhaps including things like basic instruction sets, and verifier? Or are there any experimental features? I'm specifically expecting to see the dependencies each option may have on other kernel features or hardware architectures. I need to understand the constraints. Any features built on top of the base BPF, like extended programs, JIT compilation, or specific helpers, are also of interest. I'm also looking for sensible defaults and any dependencies on other kernel subsystems or features. This file should be the definitive source of truth for the configurability of BPF within this kernel build. Understanding this is key to being able to correctly target my development with any extensions or new functionality I might want to introduce. |
| 14/2 | 2026/09/14 21:22 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/Kconfig",
"FirstLine": 25,
"LineCount": 35
}Results: {
"Contents": " 25:\tmenu \"BPF subsystem\"\n 26:\t\n 27:\tconfig BPF_SYSCALL\n 28:\t\tbool \"Enable bpf() system call\"\n 29:\t\tselect BPF\n 30:\t\tselect IRQ_WORK\n 31:\t\tselect NEED_TASKS_RCU\n 32:\t\tselect TASKS_TRACE_RCU\n 33:\t\tselect BINARY_PRINTF\n 34:\t\tselect NET_SOCK_MSG if NET\n 35:\t\tselect NET_XGRESS if NET\n 36:\t\tselect PAGE_POOL if NET\n 37:\t\tdefault n\n 38:\t\thelp\n 39:\t\t Enable the bpf() system call that allows to manipulate BPF programs\n 40:\t\t and maps via file descriptors.\n 41:\t\n 42:\tconfig BPF_JIT\n 43:\t\tbool \"Enable BPF Just In Time compiler\"\n 44:\t\tdepends on BPF\n 45:\t\tdepends on HAVE_CBPF_JIT || HAVE_EBPF_JIT\n 46:\t\tselect EXECMEM\n 47:\t\thelp\n 48:\t\t BPF programs are normally handled by a BPF interpreter. This option\n 49:\t\t allows the kernel to generate native code when a program is loaded\n 50:\t\t into the kernel. This will significantly speed-up processing of BPF\n 51:\t\t programs.\n 52:\t\n 53:\t\t Note, an admin should enable this feature changing:\n 54:\t\t /proc/sys/net/core/bpf_jit_enable\n 55:\t\t /proc/sys/net/core/bpf_jit_harden (optional)\n 56:\t\t /proc/sys/net/core/bpf_jit_kallsyms (optional)\n 57:\t\n 58:\tconfig BPF_JIT_ALWAYS_ON\n 59:\t\tbool \"Permanently enable BPF JIT and remove BPF interpreter\"\n"
} |
| 15/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 40854 output: 29 thoughts: 82 Thoughts: **My BPF Configuration Audit** Okay, so I'm looking at `BPF_SYSCALL`. No direct dependencies – it implicitly includes the basics like BPF. But, you know, I always dig deeper. I need to make sure I haven't missed a subtle dependency lurking in the shadows. Let's see... Are there any *other* configuration options that might influence this, directly or indirectly? I'm specifically curious about `bpf_insn_array` or `gotox`. Could either of these be contingent on something else – maybe `BPF_JIT`? Or something even more obscure? I need a complete picture here. Alright, time to get into the code. I need to trace where `insn_array` and `gotox` are actually *used*. References are the key. That'll reveal if there's some conditional logic being employed or a feature interaction I haven't accounted for. Gotta make sure everything is rock-solid. |
| 16/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "insn_is_gotox",
"PathPrefix": "include/"
}Results: {
"Output": "include/linux/bpf_verifier.h=1466=static inline bool bpf_stack_narrow_access_ok(int off, int fill_size, int spill_size)\n--\ninclude/linux/bpf_verifier.h-1474-\ninclude/linux/bpf_verifier.h:1475:static inline bool insn_is_gotox(struct bpf_insn *insn)\ninclude/linux/bpf_verifier.h-1476-{\n"
} |
| 17/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 41305 output: 26 thoughts: 258 Thoughts: **Analyzing the `bpf_insn_array_adjust` Change** Okay, let's dive into this. My first thought is to verify the declaration of `bpf_insn_array_adjust`, which is in `include/linux/bpf.h`. The diff shows a simple name change in the arguments to the function: the second argument, `off`, has been renamed to `first`. No functional changes seem apparent from this snippet. However, I need to understand the implications of this change within the broader context of the BPF subsystem. Next, I need to determine if any broader effects may take place because of this change. Specifically, I need to check how `bpf_prog_update_insn_ptrs` and/or the JIT compiler interact with this. My approach will be to grep within the `arch/x86` directory. |
| 18/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "bpf_prog_update_insn_ptrs"
}Results: {
"Output": "arch/arm64/net/bpf_jit_comp.c=2158=struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)\n--\narch/arm64/net/bpf_jit_comp.c-2349-\t\t/*\narch/arm64/net/bpf_jit_comp.c:2350:\t\t * The bpf_prog_update_insn_ptrs function expects offsets to\narch/arm64/net/bpf_jit_comp.c-2351-\t\t * point to the first byte of the jitted instruction (unlike\n--\narch/arm64/net/bpf_jit_comp.c-2354-\t\t */\narch/arm64/net/bpf_jit_comp.c:2355:\t\tbpf_prog_update_insn_ptrs(prog, ctx.offset, ctx.ro_image);\narch/arm64/net/bpf_jit_comp.c-2356-out_off:\n--\narch/powerpc/net/bpf_jit_comp.c=165=struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *fp)\n--\narch/powerpc/net/bpf_jit_comp.c-355-\t\t */\narch/powerpc/net/bpf_jit_comp.c:356:\t\tbpf_prog_update_insn_ptrs(fp, addrs,\narch/powerpc/net/bpf_jit_comp.c-357-\t\t\t\t(void *)fimage + FUNCTION_DESCR_SIZE);\n--\narch/x86/net/bpf_jit_comp.c=3964=struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)\n--\narch/x86/net/bpf_jit_comp.c-4119-\t\t/*\narch/x86/net/bpf_jit_comp.c:4120:\t\t * The bpf_prog_update_insn_ptrs function expects addrs to\narch/x86/net/bpf_jit_comp.c-4121-\t\t * point to the first byte of the jitted instruction (unlike\n--\narch/x86/net/bpf_jit_comp.c-4124-\t\t */\narch/x86/net/bpf_jit_comp.c:4125:\t\tbpf_prog_update_insn_ptrs(prog, addrs, image);\narch/x86/net/bpf_jit_comp.c-4126-\n--\ninclude/linux/bpf.h=4169=void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);\n--\ninclude/linux/bpf.h-4171-#ifdef CONFIG_BPF_SYSCALL\ninclude/linux/bpf.h:4172:void bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image);\ninclude/linux/bpf.h-4173-#else\ninclude/linux/bpf.h=4174=static inline void\ninclude/linux/bpf.h:4175:bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image)\ninclude/linux/bpf.h-4176-{\n--\nkernel/bpf/bpf_insn_array.c=255=void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len)\n--\nkernel/bpf/bpf_insn_array.c-277- */\nkernel/bpf/bpf_insn_array.c:278:void bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image)\nkernel/bpf/bpf_insn_array.c-279-{\n"
} |
| 19/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 42835 output: 31 thoughts: 1161 Thoughts: **Fuzzing Target Identification: A BPF Verifier Deep Dive** Okay, so I've got this patch in front of me, and my initial thought is, "This screams fuzzing target!" The core of this patch seems to be fundamentally about enhancing the BPF verifier's abilities. While `BPF_JIT` is also associated, the main work done is in `BPF_SYSCALL`, which is standard for BPF. I'm focusing on that, with an optional eye on `BPF_JIT`. I started by looking at all the modified and introduced functions. `bpf_insn_array_init` and `bpf_insn_array_adjust` have been tweaked within `kernel/bpf/bpf_insn_array.c`. This suggests some changes in how we're dealing with instruction arrays. Then, in `kernel/bpf/cfg.c`, the introduction of `compute_subprog_jts`, `bpf_free_subprog_jts`, and `subprog_jt`, along with modifications to `visit_gotox_insn`, and `bpf_compute_scc`, really caught my eye. This all points to a significant overhaul in Control Flow Graph (CFG) analysis and jump table computation on a per subprogram basis. I've also noticed a refactoring of the DFS recursion within Tarjan's SCC algorithm. `kernel/bpf/fixups.c` shows changes to how instructions are patched or fixed. This includes a new `enum bpf_patch_mode`, helper function `__bpf_patch_insn_data`, and modifications to `bpf_opt_remove_nops`, `bpf_convert_ctx_accesses`, and `bpf_do_misc_fixups`, all of which involve `BPF_PREPEND` patching. This looks like a new way to adjust or prepend instructions. In `kernel/bpf/liveness.c`, I saw `bpf_insn_successors` dealing with subprogram jump tables, which could introduce new ways to trigger unexpected behavior. Finally, `kernel/bpf/verifier.c` has these new helpers, `jt_contains`, and `reject_gotox_out_of_subprog`. The modifications to `check_indirect_jump`, especially with the checking of subprogram boundaries and the presence in the subprog jt, are very interesting. The change to `bpf_check` to call `bpf_free_subprog_jts` for cleanup solidifies this. My conclusion? This is absolutely worth fuzzing! My reasoning: This patch is reworking the BPF verifier's core logic: CFG analysis, jump table (specifically `BPF_MAP_TYPE_INSN_ARRAY`) handling, instruction patching, SCC computation, and the verification of indirect jumps (`gotox`) and data structure updates. It touches critical execution paths invoked directly via `bpf(BPF_PROG_LOAD)` and `bpf(BPF_MAP_CREATE)`. Any bug here could be catastrophic: memory corruption, arbitrary code execution, or bypass of verification. Now, which symbols should I put into the `FocusSymbols`? I'm looking for specifics, nothing too general. I've noted down the following for consideration: `bpf_insn_array_init`, `bpf_insn_array_adjust`, and `compute_subprog_jts`. But `compute_subprog_jts` is static, so I need to check whether it's visible or if calling functions are better, and I'll see if it's static and non-inlined. I'll need to check the callers of both `compute_subprog_jts` and `subprog_jt` in `kernel/bpf/cfg.c` to see if those are viable, as well. |
| 20/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "compute_subprog_jts",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/cfg.c=261=static struct bpf_iarray *jt_from_map(struct bpf_map *map)\n--\nkernel/bpf/cfg.c-293- */\nkernel/bpf/cfg.c:294:static int compute_subprog_jts(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-295-{\n--\nkernel/bpf/cfg.c=364=subprog_jt(int t, struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-369-\tif (!env-\u003ecfg.subprog_jts_ready) {\nkernel/bpf/cfg.c:370:\t\terr = compute_subprog_jts(env);\nkernel/bpf/cfg.c-371-\t\tif (err)\n--\nkernel/bpf/liveness.c=218=bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)\n--\nkernel/bpf/liveness.c-250-\nkernel/bpf/liveness.c:251:\t/* All gotox of a subprogram share its jump table, see compute_subprog_jts() */\nkernel/bpf/liveness.c-252-\tif (unlikely(insn_is_gotox(insn))) {\n--\nkernel/bpf/verifier.c=18224=static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-18288-\t\t\t\t\t\t\t subprog_end);\nkernel/bpf/verifier.c:18289:\t\t/* Maps are confined to a subprog, see compute_subprog_jts() */\nkernel/bpf/verifier.c-18290-\t\tif (verifier_bug_if(!jt_contains(jt, target), env,\n"
} |
| 21/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 45365 output: 30 thoughts: 74 Thoughts: Okay, let's break this down. **CFG Analysis Process** Right, so I need to understand how this CFG analysis works. It looks like `bpf_check_cfg` in `cfg.c` is the primary function for this process. It seems to be orchestrating everything, handling the core CFG analysis. Specifically, I need to figure out how it handles the jumps; it looks like `visit_gotox_insn` is relevant here, along with `visit_insn`, etc.. The next logical step is to pinpoint the exact definition and implementation details of `visit_gotox_insn` and `bpf_check_cfg` to further understanding of how the control flow graph is constructed and validated. |
| 22/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "bpf_check_cfg",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/cfg.c=455=static int visit_insn(int t, struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-579- */\nkernel/bpf/cfg.c:580:int bpf_check_cfg(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-581-{\n--\nkernel/bpf/verifier.c=21213=int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-21358-\nkernel/bpf/verifier.c:21359:\tret = bpf_check_cfg(env);\nkernel/bpf/verifier.c-21360-\tif (ret \u003c 0)\n"
} |
| 23/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 45853 output: 30 thoughts: 295 Thoughts: **My Assessment of BPF Verifier Functions** Okay, let's take a look at these functions. I need to get a handle on the scope and visibility of these BPF verifier functions, especially concerning whether they're static or non-static, i.e., exported. First, I'll review these particular functions and note their accessibility. I see that `bpf_free_subprog_jts` is definitely non-static; it's exported and visible across different BPF verifier files. Then I see that `bpf_compute_scc`, found in `cfg.c`, is also non-static. On the other hand, `check_indirect_jump` appears to be defined as static within `verifier.c`. Now, I also see that `bpf_insn_array_init` and `bpf_insn_array_adjust`, which live in `bpf_insn_array.c`, are both non-static. Finally, `bpf_patch_insn_data` and `bpf_opt_remove_nops` located in `fixups.c`, are also non-static functions. Now, let's turn our attention to `bpf_insn_successors`. I'm going to look at that one in `kernel/bpf/liveness.c`. Specifically, I am interested in `struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)`. I need to figure out if that particular function is also exported or non-static. |
| 24/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "bpf_insn_successors",
"PathPrefix": "include/"
}Results: {
"Output": "include/linux/bpf_verifier.h=618=struct bpf_map_ptr_state {\n--\ninclude/linux/bpf_verifier.h-634- * An array of BPF instructions.\ninclude/linux/bpf_verifier.h:635: * Primary usage: return value of bpf_insn_successors.\ninclude/linux/bpf_verifier.h-636- */\n--\ninclude/linux/bpf_verifier.h=1496=int bpf_jmp_offset(struct bpf_insn *insn);\ninclude/linux/bpf_verifier.h:1497:struct bpf_iarray *bpf_insn_successors(struct bpf_verifier_env *env, u32 idx);\ninclude/linux/bpf_verifier.h-1498-void bpf_fmt_stack_mask(char *buf, ssize_t buf_sz, u64 stack_mask);\n"
} |
| 25/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 46769 output: 29 thoughts: 102 Thoughts: **Investigating bpf_insn_successors** Okay, so `bpf_insn_successors` is confirmed as a non-static symbol. My immediate next step is to examine the `check_indirect_jump` function. This feels like the right path, as indirect jumps are often where this kind of issue manifests. Now, according to the code, `check_indirect_jump` is *static* in `kernel/bpf/verifier.c`. However, just because it's static doesn't mean it's entirely inaccessible. We have to consider its callers, as they might be the key to understanding the issue. I need to trace back from `check_indirect_jump` to its higher-level functions, namely, `do_check` or `do_check_subprogs`. I'll need to analyze the calls to `check_indirect_jump` within those functions to fully understand the context. This feels like the right direction to proceed. |
| 26/2 | 2026/09/14 21:22 | tool | grepper |
0mArgs:{
"Expression": "check_indirect_jump",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/verifier.c=18211=static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,\n--\nkernel/bpf/verifier.c-18223-/* gotox *dst_reg */\nkernel/bpf/verifier.c:18224:static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\nkernel/bpf/verifier.c-18225-{\n--\nkernel/bpf/verifier.c=18308=static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)\n--\nkernel/bpf/verifier.c-18384-\t\t\tif (BPF_SRC(insn-\u003ecode) == BPF_X)\nkernel/bpf/verifier.c:18385:\t\t\t\treturn check_indirect_jump(env, insn);\nkernel/bpf/verifier.c-18386-\n"
} |
| 27/2 | 2026/09/14 21:22 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 47379 output: 31 thoughts: 780 Thoughts: **My Analysis of BPF Verifier Functionality** Okay, so I'm looking at this `check_indirect_jump` function. It's static within `verifier.c` and called by `do_check_insn`. On x86_64, non-inlined static functions like this are visible in `kallsyms` as `t check_indirect_jump`. Given its approximate 80-line size, it's very likely the compiler isn't inlining it. That's good information to start with. Now, I need to identify the relevant code, based on the FocusSymbols guidelines. The instructions are: "List functions that specifically implement the logic being added or altered, or direct API entrypoints for the subsystem feature under review." So, the ones I've selected, from my knowledge of the codebase, are: * `bpf_check_cfg`: This is the crucial entry point for Control Flow Graph (CFG) analysis of BPF programs, and it is going to run `subprog_jt`, `compute_subprog_jts`, and `visit_gotox_insn`, which is a likely candidate. * `compute_subprog_jts`: This function is directly related to computing jump tables for each subprogram. * `bpf_free_subprog_jts`: And this is its cleanup function, so makes sense to include it too. * `bpf_compute_scc`: The SCC (Strongly Connected Components) computation algorithm in the CFG, which was refactored with `dfs_pos`. * `check_indirect_jump`: This is the function itself, so it's a key part of the logic. * `bpf_insn_successors`: This computes the instruction successors in the CFG and liveness analysis. It's been altered for `gotox` and abnormal returns, which are key to this feature. * `bpf_insn_array_adjust`: This adjusts jump table offsets when instructions are patched. * `bpf_insn_array_init`: This initializes or resets the instruction array map. * `bpf_patch_insn_data`: This patches BPF instructions and has been altered in the logic, specifically regarding prepending versus appending. * `bpf_opt_remove_nops`: And this BPF optimization, which removes NOP instructions, has been modified to protect indirect jump targets. Next, I need to think about the configuration options that relate to these. `BPF_SYSCALL` is a given; it's required for BPF syscalls, maps, and the verifier itself. Now, what about `BPF_JIT`? Does `bpf_insn_array` depend on JIT functionality? I'll need to check `bpf_insn_array.c` or other files in the relevant area to determine that. This should give me a complete picture. |
| 28/2 | 2026/09/14 21:23 | tool | grepper |
0mArgs:{
"Expression": "CONFIG_BPF_JIT",
"PathPrefix": "kernel/bpf"
}Results: {
"Output": "kernel/bpf/Kconfig=58=config BPF_JIT_ALWAYS_ON\n--\nkernel/bpf/Kconfig-64-\nkernel/bpf/Kconfig:65:\t When CONFIG_BPF_JIT_ALWAYS_ON is enabled, /proc/sys/net/core/bpf_jit_enable\nkernel/bpf/Kconfig-66-\t is permanently set to 1 and setting any other value than that will\n--\nkernel/bpf/Makefile=2=obj-y := core.o\nkernel/bpf/Makefile:3:ifneq ($(CONFIG_BPF_JIT_ALWAYS_ON),y)\nkernel/bpf/Makefile-4-# ___bpf_prog_run() needs GCSE disabled on x86; see 3193c0836f203 for details\n--\nkernel/bpf/Makefile=16=obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o\nkernel/bpf/Makefile:17:obj-$(CONFIG_BPF_JIT) += trampoline.o\nkernel/bpf/Makefile-18-obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o rqspinlock.o stream.o\n--\nkernel/bpf/Makefile=21=endif\nkernel/bpf/Makefile:22:obj-$(CONFIG_BPF_JIT) += dispatcher.o\nkernel/bpf/Makefile-23-ifeq ($(CONFIG_NET),y)\n--\nkernel/bpf/Makefile=42=endif\nkernel/bpf/Makefile:43:ifeq ($(CONFIG_BPF_JIT),y)\nkernel/bpf/Makefile-44-obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o\n--\nkernel/bpf/btf.c=9857=bool btf_type_ids_nocast_alias(struct bpf_verifier_log *log,\n--\nkernel/bpf/btf.c-9911-\nkernel/bpf/btf.c:9912:#ifdef CONFIG_BPF_JIT\nkernel/bpf/btf.c-9913-static int\n--\nkernel/bpf/core.c=100=struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flags)\n--\nkernel/bpf/core.c-128-\tfp-\u003ejit_requested = ebpf_jit_enabled();\nkernel/bpf/core.c:129:\tfp-\u003ejit_required = IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON);\nkernel/bpf/core.c-130-\tfp-\u003eblinding_requested = bpf_jit_blinding_enabled(fp);\n--\nkernel/bpf/core.c=541=void bpf_prog_kallsyms_del_all(struct bpf_prog *fp)\n--\nkernel/bpf/core.c-546-\nkernel/bpf/core.c:547:#ifdef CONFIG_BPF_JIT\nkernel/bpf/core.c-548-/* All BPF JIT sysctl knobs here. */\nkernel/bpf/core.c:549:int bpf_jit_enable __read_mostly = IS_BUILTIN(CONFIG_BPF_JIT_DEFAULT_ON);\nkernel/bpf/core.c:550:int bpf_jit_kallsyms __read_mostly = IS_BUILTIN(CONFIG_BPF_JIT_DEFAULT_ON);\nkernel/bpf/core.c-551-int bpf_jit_harden __read_mostly;\n--\nkernel/bpf/core.c=1650=u16 bpf_out_stack_arg_cnt(const struct bpf_verifier_env *env, const struct bpf_prog *prog)\n--\nkernel/bpf/core.c-1658-}\nkernel/bpf/core.c:1659:#endif /* CONFIG_BPF_JIT */\nkernel/bpf/core.c-1660-\n--\nkernel/bpf/core.c=1818=bool bpf_opcode_in_insntable(u8 code)\n--\nkernel/bpf/core.c-1840-\nkernel/bpf/core.c:1841:#ifndef CONFIG_BPF_JIT_ALWAYS_ON\nkernel/bpf/core.c-1842-/* Absolute value of s32 without undefined behavior for S32_MIN */\n--\nkernel/bpf/core.c=2605=static bool bpf_prog_select_interpreter(struct bpf_prog *fp)\n--\nkernel/bpf/core.c-2607-\tbool select_interpreter = false;\nkernel/bpf/core.c:2608:#ifndef CONFIG_BPF_JIT_ALWAYS_ON\nkernel/bpf/core.c-2609-\tu32 stack_depth = max_t(u32, fp-\u003eaux-\u003estack_depth, 1);\n--\nkernel/bpf/core.c=2628=static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struct bpf_prog *prog)\nkernel/bpf/core.c-2629-{\nkernel/bpf/core.c:2630:#ifdef CONFIG_BPF_JIT\nkernel/bpf/core.c-2631-\tstruct bpf_prog *orig_prog;\n--\nkernel/bpf/fixups.c=188=static int add_kfunc_in_insns(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-202-\nkernel/bpf/fixups.c:203:#ifndef CONFIG_BPF_JIT_ALWAYS_ON\nkernel/bpf/fixups.c-204-static int get_callee_stack_depth(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c=1410=int bpf_fixup_call_args(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-1411-{\nkernel/bpf/fixups.c:1412:#ifndef CONFIG_BPF_JIT_ALWAYS_ON\nkernel/bpf/fixups.c-1413-\tstruct bpf_prog *prog = env-\u003eprog;\n--\nkernel/bpf/fixups.c-1438-\t}\nkernel/bpf/fixups.c:1439:#ifndef CONFIG_BPF_JIT_ALWAYS_ON\nkernel/bpf/fixups.c-1440-\tif (prog-\u003ejit_required) {\n--\nkernel/bpf/syscall.c=6526=int kern_sys_bpf(int cmd, union bpf_attr *attr, unsigned int size)\n--\nkernel/bpf/syscall.c-6531-\tswitch (cmd) {\nkernel/bpf/syscall.c:6532:#ifdef CONFIG_BPF_JIT /* __bpf_prog_enter_sleepable used by trampoline and JIT */\nkernel/bpf/syscall.c-6533-\tcase BPF_PROG_TEST_RUN:\n--\nkernel/bpf/verifier.c=4594=BTF_ID(struct, cgroup)\nkernel/bpf/verifier.c-4595-#endif\nkernel/bpf/verifier.c:4596:#ifdef CONFIG_BPF_JIT\nkernel/bpf/verifier.c-4597-BTF_ID(struct, bpf_cpumask)\n"
} |
| 29/2 | 2026/09/14 21:23 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 50741 output: 292 thoughts: 534 Thoughts: **My Triage Thoughts on the BPF JIT Compilation Patch** Okay, let's break this down. I'm focusing on `bpf_prog_update_insn_ptrs` in `arch/x86/net/bpf_jit_comp.c`. It's crucial because `bpf_int_jit_compile` calls it during the JIT process. I see that `bpf_insn_array_init` is resetting `values[i].jitted_off` and `insn_array->ips[i]`, which immediately flags this as something to dig deeper into. Given this, `BPF_SYSCALL` and `BPF_JIT` are definitely relevant configurations for testing this area. Now, let's confirm the triage aspects. 1. **WorthFuzzing:** Absolutely, yes. The patch touches the verifier, and the indirect jump logic is always a juicy target. 2. **Reasoning:** This patch is refactoring indirect jumps (`gotox`), jump tables (`BPF_MAP_TYPE_INSN_ARRAY`) and modifying CFG verification which is a critical area. It’s making changes to the control flow graph verification (`compute_subprog_jts`, `bpf_check_cfg`, `visit_gotox_insn`), adding indirect jump edge accounting (`BPF_MAX_GOTOX_EDGES`), updating SCC computation recursion tracking, adjusting instruction successor analysis (`bpf_insn_successors`), and introduces prepend/append distinction for instruction patching (`__bpf_patch_insn_data`) while protecting indirect jump targets from NOP elimination. These code paths are directly exposed to user space through `bpf` syscall (`BPF_PROG_LOAD` and `BPF_MAP_CREATE`), so we *need* to fuzz here to catch any verifier bypasses, crashes, or assertions. 3. **FocusSymbols:** I want to put specific emphasis on these functions: `bpf_check_cfg`, `compute_subprog_jts`, `bpf_free_subprog_jts`, `bpf_compute_scc`, `check_indirect_jump`, `bpf_insn_successors`, `bpf_insn_array_adjust`, `bpf_insn_array_init`, `bpf_patch_insn_data`, and `bpf_opt_remove_nops`. They are the core logic touched by the patch. 4. **EnableConfigs:** We absolutely need `BPF_SYSCALL` and `BPF_JIT` enabled. Alright, that covers it. Time to call `set-results`. |
| 30/2 | 2026/09/14 21:23 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"BPF_SYSCALL",
"BPF_JIT"
],
"FocusSymbols": [
"bpf_check_cfg",
"compute_subprog_jts",
"bpf_free_subprog_jts",
"bpf_compute_scc",
"check_indirect_jump",
"bpf_insn_successors",
"bpf_insn_array_adjust",
"bpf_insn_array_init",
"bpf_patch_insn_data",
"bpf_opt_remove_nops"
],
"Reasoning": "The patch refactors indirect jump (gotox) and jump table (BPF_MAP_TYPE_INSN_ARRAY) verification in the BPF verifier. It restructures CFG verification by computing jump tables per subprogram rather than per instruction, introduces an indirect jump edge complexity limit, updates DFS tracking in SCC computation, modifies instruction successor resolution for liveness tracking, and updates instruction patching to distinguish prepending versus appending while protecting indirect jump targets from dead code/NOP elimination. These paths are directly reachable from userspace via the bpf() syscall and warrant fuzzing to uncover potential verifier bypasses, out-of-bounds access, or verifier panics.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"BPF_SYSCALL",
"BPF_JIT"
],
"FocusSymbols": [
"bpf_check_cfg",
"compute_subprog_jts",
"bpf_free_subprog_jts",
"bpf_compute_scc",
"check_indirect_jump",
"bpf_insn_successors",
"bpf_insn_array_adjust",
"bpf_insn_array_init",
"bpf_patch_insn_data",
"bpf_opt_remove_nops"
],
"Reasoning": "The patch refactors indirect jump (gotox) and jump table (BPF_MAP_TYPE_INSN_ARRAY) verification in the BPF verifier. It restructures CFG verification by computing jump tables per subprogram rather than per instruction, introduces an indirect jump edge complexity limit, updates DFS tracking in SCC computation, modifies instruction successor resolution for liveness tracking, and updates instruction patching to distinguish prepending versus appending while protecting indirect jump targets from dead code/NOP elimination. These paths are directly reachable from userspace via the bpf() syscall and warrant fuzzing to uncover potential verifier bypasses, out-of-bounds access, or verifier panics.",
"WorthFuzzing": true
} |
| 31/1 | 2026/09/14 21:23 | action | If |
3mArgs:{
"WorthFuzzing": true
}Results: null |
| 32/2 | 2026/09/14 21:23 | agent | kmsan-evaluator |
3mModel:core Args: null Results: {
"KMSANReasoning": "The patch refactors BPF indirect jump (gotox) handling and jump table verification in the BPF verifier:\n1. It consolidates jump tables to be per-subprogram (`subprog-\u003ejt`) rather than per-instruction in `insn_aux_data`, cleans up their lifecycle in `bpf_free_subprog_jts()`, and prevents gotox instructions from jumping outside their containing subprogram.\n2. In `bpf_compute_scc()`, it tracks the successor traversal resume position with `dfs_pos`, which is allocated using `kvcalloc` (zero-initialized) and freed with `kvfree`.\n3. In `bpf_insn_array.c`, `bpf_insn_array_init()` explicitly zeroes `jitted_off` and `ips[i]` to clear stale values on reuse.\n4. In `fixups.c`, it distinguishes prepend vs append instruction patching for adjusting indirect jump target flags and avoids removing NOPs if they are indirect jump targets.\n\nAll newly allocated buffers and modified structures (such as `dfs_pos`, `env-\u003ecfg`, and `env-\u003esubprog_info`) are allocated using zeroing allocators (`kvcalloc`, `kvzalloc_obj`), explicitly cleared, or initialized prior to use. Any potential bugs involving buffer bounds (e.g. `succ-\u003eitems`, `jt-\u003eitems`, or `subprog_info` indices) or pointer lifecycles would be out-of-bounds accesses or use-after-free bugs detected by standard KASAN. There is no risk of uninitialized memory reads or information disclosure to userspace, so a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit 22a0a01defffa4374bd99018ad18557be4b6f8a5
Author: syz-cluster <triage@syzkaller.com>
Date: Mon Sep 14 21:21:27 2026 +0000
syz-cluster: applied patch under review
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index e57af902560c3..8115f307444ad 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);
int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
int bpf_insn_array_ready(struct bpf_map *map);
void bpf_insn_array_release(struct bpf_map *map);
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);
void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);
#ifdef CONFIG_BPF_SYSCALL
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 36b65797877d0..754e24d41f47f 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
/* remember the offset of node field within type to rewrite */
u64 insert_off;
};
- struct bpf_iarray *jt; /* jump table for gotox or bpf_tailcall call instruction */
struct btf_struct_meta *kptr_struct_meta;
u64 map_key_state; /* constant (32 bit) key tracking for maps */
int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
@@ -805,6 +804,7 @@ struct bpf_subprog_info {
u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+ struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
u16 stack_depth; /* max. stack depth used by this function */
u16 stack_extra;
u32 insns_total;
@@ -977,6 +977,8 @@ struct bpf_verifier_env {
int cur_stack;
/* current position in the insn_postorder vector */
int cur_postorder;
+ u32 gotox_edges;
+ bool subprog_jts_ready;
} cfg;
struct backtrack_state bt;
struct bpf_jmp_history_entry *cur_hist_ent;
@@ -1501,6 +1503,7 @@ bool bpf_is_throw_kfunc(struct bpf_insn *insn);
int bpf_compute_const_regs(struct bpf_verifier_env *env);
int bpf_prune_dead_branches(struct bpf_verifier_env *env);
int bpf_check_cfg(struct bpf_verifier_env *env);
+void bpf_free_subprog_jts(struct bpf_verifier_env *env);
int bpf_compute_postorder(struct bpf_verifier_env *env);
int bpf_compute_scc(struct bpf_verifier_env *env);
@@ -1702,7 +1705,6 @@ struct bpf_kfunc_desc_tab {
};
/* Functions exported from verifier.c, used by fixups.c */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len);
void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog);
bool bpf_allow_tail_call_in_subprogs(struct bpf_verifier_env *env);
bool bpf_verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm);
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index a2f84afe6f7c6..350ecea8f1e85 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
return -EBUSY;
/*
- * Reset all the map indexes to the original values. This is needed,
- * e.g., when a replay of verification with different log level should
- * be performed.
+ * Reset the map to its pre-verification state. The xlated and jitted
+ * offsets and the jitted target pointers are recomputed by the verifier
+ * and the JIT for this program, so any values left by a previous owner
+ * must be cleared here.
*/
- for (i = 0; i < map->max_entries; i++)
+ for (i = 0; i < map->max_entries; i++) {
values[i].xlated_off = values[i].orig_off;
+ values[i].jitted_off = 0;
+ insn_array->ips[i] = 0;
+ }
return 0;
}
@@ -231,7 +235,7 @@ void bpf_insn_array_release(struct bpf_map *map)
atomic_set(&insn_array->used, 0);
}
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)
{
struct bpf_insn_array *insn_array = cast_insn_array(map);
int i;
@@ -240,7 +244,7 @@ void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
return;
for (i = 0; i < map->max_entries; i++) {
- if (insn_array->values[i].xlated_off <= off)
+ if (insn_array->values[i].xlated_off < first)
continue;
if (insn_array->values[i].xlated_off == INSN_DELETED)
continue;
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 842c7d1eabccc..5a91d73d3e3fe 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -9,6 +9,8 @@
#define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
+#define BPF_MAX_GOTOX_EDGES BPF_COMPLEXITY_LIMIT_INSNS
+
/* non-recursive DFS pseudo code
* 1 procedure DFS-iterative(G,v):
* 2 label v as discovered
@@ -284,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
}
/*
- * Find and collect all maps which fit in the subprog. Return the result as one
- * combined jump table in jt->items (allocated with kvcalloc)
+ * Collect the jump table of every subprogram that has one, as the combined
+ * table of all maps whose targets land inside that subprogram. All gotox
+ * instructions of a subprogram share the same table, so this is done in a
+ * single pass over the maps rather than once per gotox.
*/
-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
- int subprog_start, int subprog_end)
+static int compute_subprog_jts(struct bpf_verifier_env *env)
{
- struct bpf_iarray *jt = NULL;
+ struct bpf_subprog_info *subprog;
+ struct bpf_iarray *jt, *jt_cur;
struct bpf_map *map;
- struct bpf_iarray *jt_cur;
+ u32 old_cnt;
int i;
for (i = 0; i < env->insn_array_map_cnt; i++) {
@@ -303,73 +307,84 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
map = env->insn_array_maps[i];
jt_cur = jt_from_map(map);
- if (IS_ERR(jt_cur)) {
- kvfree(jt);
- return jt_cur;
+ if (IS_ERR(jt_cur))
+ return PTR_ERR(jt_cur);
+
+ subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
+ if (!subprog) {
+ kvfree(jt_cur);
+ continue;
+ }
+ if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) {
+ verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n",
+ subprog->start);
+ bpf_diag_program_structure(env, subprog->start, "jump table spans subprograms",
+ "Keep every entry of a jump table inside one subprogram.",
+ "A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.",
+ subprog->start, (subprog + 1)->start);
+ kvfree(jt_cur);
+ return -EINVAL;
}
- /*
- * This is enough to check one element. The full table is
- * checked to fit inside the subprog later in create_jt()
- */
- if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) {
- u32 old_cnt = jt ? jt->cnt : 0;
- jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt);
- if (!jt) {
- kvfree(jt_cur);
- return ERR_PTR(-ENOMEM);
- }
- memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+ old_cnt = subprog->jt ? subprog->jt->cnt : 0;
+ jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
+ if (!jt) {
+ subprog->jt = NULL;
+ kvfree(jt_cur);
+ return -ENOMEM;
}
+ memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+ subprog->jt = jt;
kvfree(jt_cur);
}
- if (!jt) {
- verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
- bpf_diag_program_structure(
- env, subprog_start, "missing jump table",
- "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
- "No jump table was found for the subprogram that starts at instruction %u.",
- subprog_start);
- return ERR_PTR(-EINVAL);
+ for (i = 0; i < env->subprog_cnt; i++) {
+ jt = env->subprog_info[i].jt;
+ if (jt)
+ jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
}
- jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
- return jt;
+ env->cfg.subprog_jts_ready = true;
+ return 0;
+}
+
+void bpf_free_subprog_jts(struct bpf_verifier_env *env)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
+ kvfree(env->subprog_info[i].jt);
+ env->subprog_info[i].jt = NULL;
+ }
+ env->cfg.subprog_jts_ready = false;
}
static struct bpf_iarray *
-create_jt(int t, struct bpf_verifier_env *env)
+subprog_jt(int t, struct bpf_verifier_env *env)
{
struct bpf_subprog_info *subprog;
- int subprog_start, subprog_end;
- struct bpf_iarray *jt;
- int i;
+ int subprog_start, err;
+
+ if (!env->cfg.subprog_jts_ready) {
+ err = compute_subprog_jts(env);
+ if (err)
+ return ERR_PTR(err);
+ }
subprog = bpf_find_containing_subprog(env, t);
subprog_start = subprog->start;
- subprog_end = (subprog + 1)->start;
- jt = jt_from_subprog(env, subprog_start, subprog_end);
- if (IS_ERR(jt))
- return jt;
- /* Check that the every element of the jump table fits within the given subprogram */
- for (i = 0; i < jt->cnt; i++) {
- if (jt->items[i] < subprog_start || jt->items[i] >= subprog_end) {
- verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n",
- t, subprog_start, subprog_end);
- bpf_diag_program_structure(
- env, t, "jump table target out of range",
- "Keep every jump-table target inside the same subprogram.",
- "The jump table for instruction %d points outside subprogram range [%u,%u).",
- t, subprog_start, subprog_end);
- kvfree(jt);
- return ERR_PTR(-EINVAL);
- }
+ if (!subprog->jt) {
+ verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
+ bpf_diag_program_structure(env, subprog_start, "missing jump table",
+ "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
+ "No jump table was found for the subprogram that starts at instruction %u.",
+ subprog_start);
+ return ERR_PTR(-EINVAL);
}
- return jt;
+ return subprog->jt;
}
/* "conditional jump with N edges" */
@@ -381,13 +396,24 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
struct bpf_iarray *jt;
int i, w;
- jt = env->insn_aux_data[t].jt;
- if (!jt) {
- jt = create_jt(t, env);
- if (IS_ERR(jt))
- return PTR_ERR(jt);
-
- env->insn_aux_data[t].jt = jt;
+ jt = subprog_jt(t, env);
+ if (IS_ERR(jt))
+ return PTR_ERR(jt);
+
+ if (!(insn_state[t] & BRANCH)) {
+ insn_state[t] |= BRANCH;
+
+ if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+ &env->cfg.gotox_edges) ||
+ env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+ verbose(env, "number of indirect jump edges in the program exceeds %u\n",
+ BPF_MAX_GOTOX_EDGES);
+ bpf_diag_program_structure(env, t, "too many indirect jump edges",
+ "Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+ "The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+ BPF_MAX_GOTOX_EDGES);
+ return -E2BIG;
+ }
}
mark_prune_point(env, t);
@@ -421,30 +447,6 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
return keep_exploring ? KEEP_EXPLORING : DONE_EXPLORING;
}
-/*
- * Instructions that can abnormally return from a subprog (tail_call
- * upon success, ld_{abs,ind} upon load failure) have a hidden exit
- * that the verifier must account for.
- */
-static int visit_abnormal_return_insn(struct bpf_verifier_env *env, int t)
-{
- struct bpf_subprog_info *subprog;
- struct bpf_iarray *jt;
-
- if (env->insn_aux_data[t].jt)
- return 0;
-
- jt = bpf_iarray_realloc(NULL, 2);
- if (!jt)
- return -ENOMEM;
-
- subprog = bpf_find_containing_subprog(env, t);
- jt->items[0] = t + 1;
- jt->items[1] = subprog->exit_idx;
- env->insn_aux_data[t].jt = jt;
- return 0;
-}
-
/* Visits the instruction at index t and returns one of the following:
* < 0 - an error occurred
* DONE_EXPLORING - the instruction was fully explored
@@ -461,13 +463,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
/* All non-branch instructions have a single fall-through edge. */
if (BPF_CLASS(insn->code) != BPF_JMP &&
BPF_CLASS(insn->code) != BPF_JMP32) {
- if (BPF_CLASS(insn->code) == BPF_LD &&
- (BPF_MODE(insn->code) == BPF_ABS ||
- BPF_MODE(insn->code) == BPF_IND)) {
- ret = visit_abnormal_return_insn(env, t);
- if (ret)
- return ret;
- }
insn_sz = bpf_is_ldimm64(insn) ? 2 : 1;
return push_insn(t, t + insn_sz, FALLTHROUGH, env);
}
@@ -512,11 +507,6 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
mark_subprog_might_sleep(env, t);
if (bpf_helper_changes_pkt_data(insn->imm))
mark_subprog_changes_pkt_data(env, t);
- if (insn->imm == BPF_FUNC_tail_call) {
- ret = visit_abnormal_return_insn(env, t);
- if (ret)
- return ret;
- }
} else if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
struct bpf_call_arg_meta meta;
@@ -749,7 +739,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
struct bpf_insn_aux_data *aux = env->insn_aux_data;
const u32 insn_cnt = env->prog->len;
int stack_sz, dfs_sz, err = 0;
- u32 *stack, *pre, *low, *dfs;
+ u32 *stack, *pre, *low, *dfs, *dfs_pos;
u32 i, j, t, w;
u32 next_preorder_num;
u32 next_scc_id;
@@ -762,13 +752,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
* - 'stack' accumulates vertices in DFS order, see invariant comment below;
* - 'pre[t] == p' => preorder number of vertex 't' is 'p';
* - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n';
- * - 'dfs' DFS traversal stack, used to emulate explicit recursion.
+ * - 'dfs' DFS traversal stack, used to emulate explicit recursion;
+ * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its
+ * successors at index 'j'.
*/
stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
dfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);
- if (!stack || !pre || !low || !dfs) {
+ dfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);
+ if (!stack || !pre || !low || !dfs || !dfs_pos) {
err = -ENOMEM;
goto exit;
}
@@ -851,6 +844,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
stack_sz = 0;
dfs_sz = 1;
dfs[0] = i;
+ dfs_pos[0] = 0;
dfs_continue:
while (dfs_sz) {
w = dfs[dfs_sz - 1];
@@ -860,13 +854,22 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
next_preorder_num++;
stack[stack_sz++] = w;
}
- /* Visit 'w' successors */
+ /* Visit remaining 'w' successors */
succ = bpf_insn_successors(env, w);
- for (j = 0; j < succ->cnt; ++j) {
+ for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) {
if (pre[succ->items[j]]) {
low[w] = min(low[w], low[succ->items[j]]);
} else {
- dfs[dfs_sz++] = succ->items[j];
+ /*
+ * Once DFS for succ->items[j] is complete,
+ * pre[succ->items[j]] is non-zero, hence
+ * resuming at 'j' allows to follow the
+ * low[w] = min(...) update branch above.
+ */
+ dfs_pos[dfs_sz - 1] = j;
+ dfs_pos[dfs_sz] = 0;
+ dfs[dfs_sz] = succ->items[j];
+ dfs_sz++;
goto dfs_continue;
}
}
@@ -916,5 +919,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
kvfree(pre);
kvfree(low);
kvfree(dfs);
+ kvfree(dfs_pos);
return err;
}
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 52d3cec336727..e132bd0600c7e 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -213,12 +213,18 @@ static int get_callee_stack_depth(struct bpf_verifier_env *env,
}
#endif
+enum bpf_patch_mode {
+ BPF_APPEND,
+ BPF_PREPEND,
+};
+
/* single env->prog->insni[off] instruction was replaced with the range
* insni[off, off + cnt). Adjust corresponding insn_aux_data by copying
* [0, off) and [off, end) to new locations, so the patched range stays zero
*/
static void adjust_insn_aux_data(struct bpf_verifier_env *env,
- struct bpf_prog *new_prog, u32 off, u32 cnt)
+ struct bpf_prog *new_prog, u32 off, u32 cnt,
+ enum bpf_patch_mode mode)
{
struct bpf_insn_aux_data *data = env->insn_aux_data;
struct bpf_insn *insn = new_prog->insnsi;
@@ -251,9 +257,10 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
* new instructions by the above memmove and memset, but the indirect jump target is
* actually the first instruction, so move it back. This also matches with the behavior
* of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new
- * instruction.
+ * instruction. For BPF_PREPEND the original instruction is the last one, so the flag
+ * already sits where needed.
*/
- if (data[off + cnt - 1].indirect_target) {
+ if (mode == BPF_APPEND && data[off + cnt - 1].indirect_target) {
data[off].indirect_target = 1;
data[off + cnt - 1].indirect_target = 0;
}
@@ -273,7 +280,7 @@ static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len
}
}
-static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)
+static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 first, u32 len)
{
int i;
@@ -281,7 +288,7 @@ static void adjust_insn_arrays(struct bpf_verifier_env *env, u32 off, u32 len)
return;
for (i = 0; i < env->insn_array_map_cnt; i++)
- bpf_insn_array_adjust(env->insn_array_maps[i], off, len);
+ bpf_insn_array_adjust(env->insn_array_maps[i], first, len);
}
static void adjust_insn_arrays_after_remove(struct bpf_verifier_env *env, u32 off, u32 len)
@@ -306,8 +313,9 @@ static void adjust_poke_descs(struct bpf_prog *prog, u32 off, u32 len)
}
}
-struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
- const struct bpf_insn *patch, u32 len)
+static struct bpf_prog *__bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len,
+ enum bpf_patch_mode mode)
{
struct bpf_prog *new_prog;
struct bpf_insn_aux_data *new_data = NULL;
@@ -331,13 +339,19 @@ struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
env->insn_aux_data[off].orig_idx);
return NULL;
}
- adjust_insn_aux_data(env, new_prog, off, len);
+ adjust_insn_aux_data(env, new_prog, off, len, mode);
adjust_subprog_starts(env, off, len);
- adjust_insn_arrays(env, off, len);
+ adjust_insn_arrays(env, mode == BPF_PREPEND ? off : off + 1, len);
adjust_poke_descs(new_prog, off, len);
return new_prog;
}
+struct bpf_prog *bpf_patch_insn_data(struct bpf_verifier_env *env, u32 off,
+ const struct bpf_insn *patch, u32 len)
+{
+ return __bpf_patch_insn_data(env, off, patch, len, BPF_APPEND);
+}
+
/*
* For all jmp insns in a given 'prog' that point to 'tgt_idx' insn adjust the
* jump offset by 'delta'.
@@ -500,23 +514,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
return 0;
}
-/*
- * Clean up dynamically allocated fields of aux data for instructions [start, ...]
- */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
-{
- struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
- int end = start + len;
- int i;
-
- for (i = start; i < end; i++) {
- if (aux_data[i].jt) {
- kvfree(aux_data[i].jt);
- aux_data[i].jt = NULL;
- }
- }
-}
-
static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
{
struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
@@ -526,8 +523,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
if (bpf_prog_is_offloaded(env->prog->aux))
bpf_prog_offload_remove_insns(env, off, cnt);
- bpf_clear_insn_aux_data(env, off, cnt);
-
err = bpf_remove_insns(env->prog, off, cnt);
if (err)
return err;
@@ -618,6 +613,7 @@ int bpf_opt_remove_dead_code(struct bpf_verifier_env *env)
int bpf_opt_remove_nops(struct bpf_verifier_env *env)
{
+ struct bpf_insn_aux_data *aux = env->insn_aux_data;
struct bpf_insn *insn = env->prog->insnsi;
int insn_cnt = env->prog->len;
bool is_may_goto_0, is_ja;
@@ -629,6 +625,8 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)
if (!is_may_goto_0 && !is_ja)
continue;
+ if (aux[i].indirect_target)
+ continue;
err = verifier_remove_insns(env, i, 1);
if (err)
@@ -771,7 +769,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
insn_buf[cnt++] = BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1,
-subprogs[0].stack_depth);
insn_buf[cnt++] = env->prog->insnsi[0];
- new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+ new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+ BPF_PREPEND);
if (!new_prog)
return -ENOMEM;
env->prog = new_prog;
@@ -794,7 +793,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
verifier_bug(env, "prologue is too long");
return -EFAULT;
} else if (cnt) {
- new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+ new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+ BPF_PREPEND);
if (!new_prog)
return -ENOMEM;
@@ -2459,7 +2459,8 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env)
/* Copy first actual insn to preserve it */
insn_buf[cnt++] = env->prog->insnsi[subprog_start];
- new_prog = bpf_patch_insn_data(env, subprog_start, insn_buf, cnt);
+ new_prog = __bpf_patch_insn_data(env, subprog_start, insn_buf, cnt,
+ BPF_PREPEND);
if (!new_prog)
return -ENOMEM;
env->prog = prog = new_prog;
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 301fc60bddc43..08592fdb55103 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -248,9 +248,12 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
struct bpf_iarray *succ, *jt;
int insn_sz;
- jt = env->insn_aux_data[idx].jt;
- if (unlikely(jt))
- return jt;
+ /* All gotox of a subprogram share its jump table, see compute_subprog_jts() */
+ if (unlikely(insn_is_gotox(insn))) {
+ jt = bpf_find_containing_subprog(env, idx)->jt;
+ if (jt)
+ return jt;
+ }
/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */
succ = env->succ;
@@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
if (opcode_info->can_jump)
succ->items[succ->cnt++] = idx + bpf_jmp_offset(insn) + 1;
+ /*
+ * tail_call upon success and ld_{abs,ind} upon load failure return
+ * from the subprogram, which is a hidden edge to its exit.
+ */
+ if (unlikely((bpf_helper_call(insn) && insn->imm == BPF_FUNC_tail_call) ||
+ (BPF_CLASS(insn->code) == BPF_LD &&
+ (BPF_MODE(insn->code) == BPF_ABS || BPF_MODE(insn->code) == BPF_IND))))
+ succ->items[succ->cnt++] = bpf_find_containing_subprog(env, idx)->exit_idx;
+
return succ;
}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index ddba53eaa3331..dc3375fcfab9d 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18191,11 +18191,43 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,
return 0;
}
+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */
+static bool jt_contains(const struct bpf_iarray *jt, u32 target)
+{
+ int l = 0, r = jt->cnt - 1, m;
+
+ while (l <= r) {
+ m = l + (r - l) / 2;
+ if (jt->items[m] == target)
+ return true;
+ if (jt->items[m] < target)
+ l = m + 1;
+ else
+ r = m - 1;
+ }
+ return false;
+}
+
+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,
+ u32 subprog_start, u32 subprog_end)
+{
+ verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n",
+ env->insn_idx, target, subprog_start, subprog_end);
+ bpf_diag_program_structure(env, env->insn_idx, "indirect jump leaves subprogram",
+ "Keep every reachable jump-table target inside the subprogram of the indirect jump.",
+ "Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).",
+ env->insn_idx, target, subprog_start, subprog_end);
+ return -EINVAL;
+}
+
/* gotox *dst_reg */
static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)
{
struct bpf_verifier_state *other_branch;
+ struct bpf_subprog_info *subprog;
+ u32 subprog_start, subprog_end;
struct bpf_reg_state *dst_reg;
+ struct bpf_iarray *jt;
struct bpf_map *map;
u32 min_index, max_index;
int err = 0;
@@ -18238,6 +18270,29 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
return -EINVAL;
}
+ subprog = bpf_find_containing_subprog(env, env->insn_idx);
+ if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
+ return -EFAULT;
+ subprog_start = subprog->start;
+ subprog_end = (subprog + 1)->start;
+
+ jt = subprog->jt;
+ if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
+ return -EFAULT;
+
+ for (i = 0; i < n; i++) {
+ u32 target = env->gotox_tmp_buf->items[i];
+
+ if (target < subprog_start || target >= subprog_end)
+ return reject_gotox_out_of_subprog(env, target, subprog_start,
+ subprog_end);
+ /* Maps are confined to a subprog, see compute_subprog_jts() */
+ if (verifier_bug_if(!jt_contains(jt, target), env,
+ "insn %d target %u is not in the jump table of its subprog",
+ env->insn_idx, target))
+ return -EFAULT;
+ }
+
for (i = 0; i < n - 1; i++) {
mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
@@ -21477,8 +21532,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
release_maps(env);
release_btfs(env);
err_free_env:
- if (env->insn_aux_data)
- bpf_clear_insn_aux_data(env, 0, env->insn_aux_data_len);
+ bpf_free_subprog_jts(env);
vfree(env->insn_aux_data);
kvfree(env->fd_array);
bpf_stack_liveness_free(env);
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 0222a9a5d0761..f102435675c2b 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
#include <bpf/bpf.h>
+#include <bpf/btf.h>
#include <test_progs.h>
#if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)
@@ -453,43 +454,1083 @@ static void check_bpf_no_lookup(void)
close(map_fd);
}
+#define GOTOX_CNT_AT_LIMIT 1000
+#define GOTOX_LOG_SZ (256 * 1024)
+
+static const char gotox_limit_msg[] =
+ "number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+ /* the run of gotox itself, plus the insn right after it */
+ const __u32 jt_cnt = gotox_cnt + 1;
+ struct bpf_insn_array_value val = {};
+ int map_fd;
+ __u32 i;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return map_fd;
+
+ for (i = 0; i < jt_cnt; i++) {
+ val.orig_off = first_gotox + i;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+ "bpf_map_update_elem"))
+ goto err;
+ }
+
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto err;
+
+ return map_fd;
+err:
+ close(map_fd);
+ return -1;
+}
+
+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
+ int *fd_array, __u32 fd_array_cnt, char *log,
+ int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
+{
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ int prog_fd;
+
+ log[0] = 0;
+ opts.fd_array = fd_array;
+ opts.fd_array_cnt = fd_array_cnt;
+ opts.log_buf = log;
+ opts.log_size = GOTOX_LOG_SZ;
+ opts.log_level = 1;
+ if (fi_cnt) {
+ opts.prog_btf_fd = btf_fd;
+ opts.func_info = fi;
+ opts.func_info_cnt = fi_cnt;
+ opts.func_info_rec_size = sizeof(*fi);
+ }
+
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+ if (prog_fd >= 0) {
+ close(prog_fd);
+ return 0;
+ }
+ return prog_fd;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+ int *fd_array, __u32 fd_array_cnt, char *log)
+{
+ return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,
+ -1, NULL, 0);
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+ __u32 i;
+
+ insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+ for (i = 1; i <= gotox_cnt; i++)
+ insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+ ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+ const __u32 insn_cnt = gotox_cnt + 3;
+ struct bpf_insn *insns;
+ bool attempted = false;
+ int map_fd;
+
+ insns = calloc(insn_cnt, sizeof(*insns));
+ if (!ASSERT_OK_PTR(insns, "calloc insns"))
+ return false;
+
+ gotox_run_fill(insns, gotox_cnt);
+ insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+ map_fd = gotox_jt_create(1, gotox_cnt);
+ if (map_fd < 0)
+ goto free_insns;
+
+ *err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+ close(map_fd);
+ attempted = true;
+free_insns:
+ free(insns);
+ return attempted;
+}
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+ const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ if (try_load_gotox_prog(gotox_cnt, log, &err))
+ check_gotox_limit_hit(log, err);
+
+ free(log);
+}
+
+/*
+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
+ * table holds the starts of the blocks that follow plus the exit block,
+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
+ * exactly at the limit and must still load.
+ */
+#define GOTOX_BLOCK_SZ 4
+
+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+ struct bpf_insn *at;
+ __u32 k;
+
+ for (k = 0; k < gotox_cnt; k++) {
+ at = insns + k * GOTOX_BLOCK_SZ;
+
+ /* r1 = &jt[0], by index 0 into fd_array */
+ at[0] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ at[1] = (struct bpf_insn) { .imm = 0 };
+ at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
+ at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ }
+
+ insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
+}
+
+static int gotox_chain_jt_create(__u32 gotox_cnt)
+{
+ struct bpf_insn_array_value val = {};
+ int map_fd;
+ __u32 i;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return map_fd;
+
+ for (i = 0; i < gotox_cnt; i++) {
+ val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+ "bpf_map_update_elem"))
+ goto err;
+ }
+
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto err;
+
+ return map_fd;
+err:
+ close(map_fd);
+ return -1;
+}
+
+static void check_gotox_edges_at_limit(void)
+{
+ const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+ const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
+ struct bpf_insn *insns;
+ char *log;
+ int map_fd, err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ insns = calloc(insn_cnt, sizeof(*insns));
+ if (!ASSERT_OK_PTR(insns, "calloc insns"))
+ goto free_log;
+
+ gotox_chain_fill(insns, gotox_cnt);
+
+ map_fd = gotox_chain_jt_create(gotox_cnt);
+ if (map_fd < 0)
+ goto free_insns;
+
+ err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+ close(map_fd);
+
+ if (!ASSERT_OK(err, "program at the edge limit should load"))
+ fprintf(stderr, "verifier log: %s\n", log);
+
+free_insns:
+ free(insns);
+free_log:
+ free(log);
+}
+
+static void check_gotox_edges_across_subprogs(void)
+{
+ const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
+ const __u32 sub_start = gotox_cnt + 3;
+ const __u32 insn_cnt = 2 * (gotox_cnt + 3);
+ int map_fd[2] = { -1, -1 };
+ struct bpf_insn *insns;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ insns = calloc(insn_cnt, sizeof(*insns));
+ if (!ASSERT_OK_PTR(insns, "calloc insns"))
+ goto free_log;
+
+ gotox_run_fill(insns, gotox_cnt);
+ insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
+ BPF_PSEUDO_CALL, 0,
+ sub_start - (gotox_cnt + 1) - 1);
+ insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+ gotox_run_fill(insns + sub_start, gotox_cnt);
+ insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
+
+ map_fd[0] = gotox_jt_create(1, gotox_cnt);
+ if (map_fd[0] < 0)
+ goto free_insns;
+ map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
+ check_gotox_limit_hit(log, err);
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_insns:
+ free(insns);
+free_log:
+ free(log);
+}
+
+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
+{
+ struct bpf_insn_array_value val = {};
+ int map_fd;
+ __u32 i;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return map_fd;
+
+ for (i = 0; i < cnt; i++) {
+ val.orig_off = offs[i];
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+ "bpf_map_update_elem"))
+ goto err;
+ }
+
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto err;
+
+ return map_fd;
+err:
+ close(map_fd);
+ return -1;
+}
+
+#define GOTOX_SUB_START 4
+#define GOTOX_MAIN_TGT 2
+#define GOTOX_SUB_TGT 8
+#define GOTOX_TWO_INSN_CNT 10
+
+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)
+{
+ insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+ GOTOX_SUB_START - 1 - 1);
+ insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[3] = BPF_EXIT_INSN();
+
+ /* r1 = &jt[0], by index 'jt_idx' into fd_array */
+ insns[GOTOX_SUB_START] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = jt_idx,
+ };
+ insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };
+ insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);
+ insns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[9] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_other_subprog(void)
+{
+ const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+ const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+ struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_two_subprogs_fill(insns, 0, 0);
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+static void check_gotox_jt_per_subprog(void)
+{
+ const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+ const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+ struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_two_subprogs_fill(insns, 1, 0);
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+#define GOTOX_FWD_GOTOX 11
+#define GOTOX_FWD_OWN_TGT 12
+#define GOTOX_FWD_SUB_START 14
+#define GOTOX_FWD_INSN_CNT 16
+
+static void gotox_from_main_fill(struct bpf_insn *insns)
+{
+ insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);
+ insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+ GOTOX_FWD_SUB_START - 1 - 1);
+ insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,
+ offsetof(struct xdp_md, ingress_ifindex));
+ insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);
+
+ /* r1 = &jt_leaves[0], by index 1 into fd_array */
+ insns[4] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 1,
+ };
+ insns[5] = (struct bpf_insn) { .imm = 0 };
+ insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+ insns[7] = BPF_JMP_A(3);
+
+ /* r1 = &jt_own[0], by index 0 into fd_array */
+ insns[8] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ insns[9] = (struct bpf_insn) { .imm = 0 };
+ insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+ insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[13] = BPF_EXIT_INSN();
+ insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[15] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_subprog_from_main(void)
+{
+ const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+ const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+ struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_from_main_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+/*
+ * The only map of the subprog holding the gotox reaches past that subprog, so
+ * the subprog is left without a jump table at all.
+ */
+static void check_gotox_jt_spans_subprogs(void)
+{
+ const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+ const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+ struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_from_main_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+/*
+ * The subprog holding the gotox has a well formed jump table of its own and
+ * also collects a map that reaches past its end. The spanning map is still
+ * rejected, even though the subprog is not left without a table.
+ */
+static void check_gotox_jt_spans_with_own_table(void)
+{
+ const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+ const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+ struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_from_main_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+#define GOTOX_SLICE_SUB_START 6
+#define GOTOX_SLICE_GOTOX 14
+#define GOTOX_SLICE_SUB_TGT 15
+#define GOTOX_SLICE_INSN_CNT 17
+
+static void gotox_slice_fill(struct bpf_insn *insns)
+{
+ insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+ GOTOX_SLICE_SUB_START - 1 - 1);
+ insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[5] = BPF_EXIT_INSN();
+
+ insns[GOTOX_SLICE_SUB_START] =
+ BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+ offsetof(struct xdp_md, ingress_ifindex));
+ insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);
+ insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);
+ insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+
+ /* r1 = &jt_main[0], by index 0 into fd_array */
+ insns[10] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ insns[11] = (struct bpf_insn) { .imm = 0 };
+ insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);
+ insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+ insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+ insns[16] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_index_slice_other_subprog(void)
+{
+ const __u32 jt_main[] = { 2, 3, 4 };
+ const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };
+ struct bpf_insn insns[GOTOX_SLICE_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_slice_fill(insns);
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_log;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_log:
+ free(log);
+}
+
+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,
+ struct bpf_func_info *fi, struct btf **pbtf)
+{
+ int int_id, proto_id, id;
+ struct btf *btf;
+ char name[24];
+ __u32 i;
+
+ btf = btf__new_empty();
+ if (!ASSERT_OK_PTR(btf, "btf__new_empty"))
+ return -1;
+
+ int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED);
+ if (!ASSERT_GT(int_id, 0, "btf__add_int"))
+ goto err;
+
+ proto_id = btf__add_func_proto(btf, int_id);
+ if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto"))
+ goto err;
+
+ for (i = 0; i < cnt; i++) {
+ snprintf(name, sizeof(name), "gotox_f%u", i);
+ id = btf__add_func(btf, name, linkage[i], proto_id);
+ if (!ASSERT_GT(id, 0, "btf__add_func"))
+ goto err;
+ fi[i].insn_off = starts[i];
+ fi[i].type_id = id;
+ }
+
+ if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel"))
+ goto err;
+
+ *pbtf = btf;
+ return btf__fd(btf);
+err:
+ btf__free(btf);
+ return -1;
+}
+
+static void check_gotox_target_other_global_subprog(void)
+{
+ const __u32 starts[] = { 0, GOTOX_SUB_START };
+ const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };
+ const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+ const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+ struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ struct bpf_func_info fi[2];
+ struct btf *btf = NULL;
+ int btf_fd;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_two_subprogs_fill(insns, 0, 0);
+
+ btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+ if (btf_fd < 0)
+ goto free_log;
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_btf;
+ map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+ btf_fd, fi, ARRAY_SIZE(fi));
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_btf:
+ btf__free(btf);
+free_log:
+ free(log);
+}
+
+#define GOTOX_CB_MAIN_TGT 6
+#define GOTOX_CB_START 8
+#define GOTOX_CB_GOTOX 11
+#define GOTOX_CB_TGT 12
+#define GOTOX_CB_INSN_CNT 14
+
+static void gotox_callback_fill(struct bpf_insn *insns)
+{
+ insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);
+ /* r2 = &callback */
+ insns[1] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_2,
+ .src_reg = BPF_PSEUDO_FUNC,
+ .imm = GOTOX_CB_START - 1 - 1,
+ };
+ insns[2] = (struct bpf_insn) { .imm = 0 };
+ insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);
+ insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);
+ insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);
+ insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[7] = BPF_EXIT_INSN();
+
+ /* r1 = &jt_main[0], by index 0 into fd_array */
+ insns[GOTOX_CB_START] = (struct bpf_insn) {
+ .code = BPF_LD | BPF_DW | BPF_IMM,
+ .dst_reg = BPF_REG_1,
+ .src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+ .imm = 0,
+ };
+ insns[9] = (struct bpf_insn) { .imm = 0 };
+ insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+ insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+ insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+ insns[13] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_callback_leaves_subprog(void)
+{
+ const __u32 starts[] = { 0, GOTOX_CB_START };
+ const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };
+ const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };
+ const __u32 jt_cb[] = { GOTOX_CB_TGT };
+ struct bpf_insn insns[GOTOX_CB_INSN_CNT];
+ int map_fd[2] = { -1, -1 };
+ struct bpf_func_info fi[2];
+ struct btf *btf = NULL;
+ int btf_fd;
+ char *log;
+ int err;
+
+ log = calloc(1, GOTOX_LOG_SZ);
+ if (!ASSERT_OK_PTR(log, "calloc log"))
+ return;
+
+ gotox_callback_fill(insns);
+
+ btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+ if (btf_fd < 0)
+ goto free_log;
+
+ map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+ if (map_fd[0] < 0)
+ goto free_btf;
+ map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));
+ if (map_fd[1] < 0)
+ goto close_maps;
+
+ err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+ btf_fd, fi, ARRAY_SIZE(fi));
+ ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+ ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)",
+ "verifier log");
+
+close_maps:
+ close(map_fd[0]);
+ close(map_fd[1]);
+free_btf:
+ btf__free(btf);
+free_log:
+ free(log);
+}
+
static void check_bpf_side(void)
{
check_bpf_no_lookup();
}
+static void check_gotox_target_nop(void)
+{
+ struct bpf_insn insns[] = {
+ /* r1 = &jt[0] */
+ BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* insn 4: the nop target */
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ struct bpf_insn_array_value val = {};
+ int prog_fd = -1, map_fd;
+ __u32 key = 0;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return;
+
+ val.orig_off = 4;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto cleanup;
+
+ opts.fd_array = &map_fd;
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+ goto cleanup;
+ ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target must not be INSN_DELETED");
+ ASSERT_NEQ(val.jitted_off, 0, "gotox target must have a jitted address");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ close(map_fd);
+}
+
+static void check_insn_array_stale_reuse(void)
+{
+ /* helper of a different prog type, to set the prog array's owner */
+ struct bpf_insn sf[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ };
+ /*
+ * prog1 binds the insn array (entry points at a real insn, so the JIT
+ * fills it in) and tail calls a prog array whose owner is now a
+ * different type, which fails in the post-JIT compatibility check.
+ */
+ struct bpf_insn prog1[] = {
+ BPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */
+ BPF_MOV64_IMM(BPF_REG_3, 0), /* r3 = index */
+ BPF_EMIT_CALL(BPF_FUNC_tail_call),
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 4: real entry */
+ BPF_EXIT_INSN(),
+ };
+ /*
+ * prog2 reuses the insn array as a tracker. Its entry points at a nop
+ * that the nop pass removes, so the entry becomes INSN_DELETED and the
+ * JIT skips it -- exactly the slot a stale pointer would linger in.
+ */
+ struct bpf_insn prog2[] = {
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* insn 4: tracked nop */
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ int jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;
+ struct bpf_insn_array_value val = {};
+ int fd_array[2];
+ __u32 key = 0;
+
+ jt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(jt_fd, 0, "insn_array create"))
+ return;
+ val.orig_off = 4;
+ if (!ASSERT_EQ(bpf_map_update_elem(jt_fd, &key, &val, 0), 0, "insn_array update"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, "insn_array freeze"))
+ goto cleanup;
+
+ pa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, "pa", 4, 4, 1, NULL);
+ if (!ASSERT_GE(pa_fd, 0, "prog_array create"))
+ goto cleanup;
+ sf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", sf, ARRAY_SIZE(sf), NULL);
+ if (!ASSERT_GE(sf_fd, 0, "owner prog load"))
+ goto cleanup;
+ /* insert a non-XDP prog to pin the prog array owner to another type */
+ if (!ASSERT_EQ(bpf_map_update_elem(pa_fd, &key, &sf_fd, 0), 0, "prog_array update"))
+ goto cleanup;
+
+ fd_array[0] = jt_fd;
+ fd_array[1] = pa_fd;
+ opts.fd_array = fd_array;
+ opts.fd_array_cnt = 2;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog1, ARRAY_SIZE(prog1), &opts);
+ if (!ASSERT_LT(prog_fd, 0, "prog1 must fail the post-JIT tail call check"))
+ goto cleanup;
+
+ /* prog1 reached the JIT before failing, so the entry is now populated */
+ if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog1"))
+ goto cleanup;
+ if (!ASSERT_NEQ(val.jitted_off, 0, "prog1 should have filled the jitted address"))
+ goto cleanup;
+
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog2, ARRAY_SIZE(prog2), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "prog2 reuse load"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog2"))
+ goto cleanup;
+ ASSERT_EQ(val.xlated_off, (__u32)-1, "reused entry should be INSN_DELETED");
+ ASSERT_EQ(val.jitted_off, 0, "stale jitted address must be cleared on reuse");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ if (sf_fd >= 0)
+ close(sf_fd);
+ if (pa_fd >= 0)
+ close(pa_fd);
+ if (jt_fd >= 0)
+ close(jt_fd);
+}
+
+static void check_gotox_target_prologue_shift(void)
+{
+ struct bpf_insn insns[] = {
+ /* insn 0: gotox target and subprog start */
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ /* may_goto +4 -> exit block, bounds the loop */
+ BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),
+ /* r1 = &jt[0] (insns 2 and 3) */
+ BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */
+ /* insn 5: gotox r1 -> insn 0 */
+ BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),
+ BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ struct bpf_insn_array_value val = {};
+ int prog_fd = -1, map_fd;
+ __u32 key = 0;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return;
+
+ val.orig_off = 0;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto cleanup;
+
+ opts.fd_array = &map_fd;
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+ goto cleanup;
+ ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+ ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+ ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ close(map_fd);
+}
+
+static void check_gotox_target_ctx_prologue_shift(void)
+{
+ struct bpf_insn insns[] = {
+ /* insn 0: gotox target and subprog start */
+ BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),
+ BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */
+ BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),
+ BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),
+ BPF_MOV64_REG(BPF_REG_4, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),
+ /* insn 9: direct packet write -> tc unclone prologue prepended at insn 0 */
+ BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),
+ /* r5 = &jt[0] (insns 10 and 11) */
+ BPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */
+ /* insn 13: gotox r5 -> insn 0 */
+ BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),
+ BPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts);
+ struct bpf_insn_array_value val = {};
+ int prog_fd = -1, map_fd;
+ __u32 key = 0;
+
+ map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+ if (!ASSERT_GE(map_fd, 0, "map_create"))
+ return;
+
+ val.orig_off = 0;
+ if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+ goto cleanup;
+ if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+ goto cleanup;
+
+ opts.fd_array = &map_fd;
+ opts.fd_array_cnt = 1;
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns,
+ ARRAY_SIZE(insns), &opts);
+ if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+ goto cleanup;
+
+ if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+ goto cleanup;
+ ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+ ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+ ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+ if (prog_fd >= 0)
+ close(prog_fd);
+ close(map_fd);
+}
+
static void __test_bpf_insn_array(void)
{
/* Test if offsets are adjusted properly */
-
if (test__start_subtest("one2one"))
check_one_to_one_mapping();
-
if (test__start_subtest("simple"))
check_simple();
-
if (test__start_subtest("deletions"))
check_deletions();
-
if (test__start_subtest("deletions-with-functions"))
check_deletions_with_functions();
-
if (test__start_subtest("blindness"))
check_blindness();
-
/* Check all kinds of operations and related restrictions */
-
if (test__start_subtest("incorrect-index"))
check_incorrect_index();
-
if (test__start_subtest("load-unfrozen-map"))
check_load_unfrozen_map();
-
if (test__start_subtest("no-map-reuse"))
check_no_map_reuse();
-
if (test__start_subtest("bpf-side-ops"))
check_bpf_side();
+ if (test__start_subtest("too-many-gotox-edges"))
+ check_too_many_gotox_edges();
+ if (test__start_subtest("gotox-edges-at-limit"))
+ check_gotox_edges_at_limit();
+ if (test__start_subtest("gotox-edges-across-subprogs"))
+ check_gotox_edges_across_subprogs();
+ if (test__start_subtest("gotox-jt-spans-subprogs"))
+ check_gotox_jt_spans_subprogs();
+ if (test__start_subtest("gotox-jt-spans-with-own-table"))
+ check_gotox_jt_spans_with_own_table();
+ if (test__start_subtest("gotox-target-other-subprog"))
+ check_gotox_target_other_subprog();
+ if (test__start_subtest("gotox-jt-per-subprog"))
+ check_gotox_jt_per_subprog();
+ if (test__start_subtest("gotox-target-subprog-from-main"))
+ check_gotox_target_subprog_from_main();
+ if (test__start_subtest("gotox-index-slice-other-subprog"))
+ check_gotox_index_slice_other_subprog();
+ if (test__start_subtest("gotox-target-other-global-subprog"))
+ check_gotox_target_other_global_subprog();
+ if (test__start_subtest("gotox-callback-leaves-subprog"))
+ check_gotox_callback_leaves_subprog();
+ if (test__start_subtest("gotox-target-nop"))
+ check_gotox_target_nop();
+ if (test__start_subtest("insn-array-stale-reuse"))
+ check_insn_array_stale_reuse();
+ if (test__start_subtest("gotox-target-prologue-shift"))
+ check_gotox_target_prologue_shift();
+ if (test__start_subtest("gotox-target-ctx-prologue-shift"))
+ check_gotox_target_ctx_prologue_shift();
}
#else
static void __test_bpf_insn_array(void)
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index 0e27c2c79c57e..4931a77ba0b26 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -366,7 +366,7 @@ __used static int test_subprog(void)
}
SEC("socket")
-__failure __msg("jump table for insn 4 points outside of the subprog [0,10]")
+__failure __msg("jump table of subprog starting at 0 spans multiple subprogs")
__naked void jump_table_outside_subprog(void)
{
asm volatile (" \
|
| 33/3 | 2026/09/14 21:23 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 29430 output: 22 thoughts: 924 Thoughts: **KMSAN Fuzzing Evaluation for BPF Patch Series** Okay, let's break this down. My goal is to determine if this BPF patch series warrants a dedicated KMSAN fuzzing session, beyond the standard KASAN checks we're already running. The rule is clear: KMSAN is expensive, and *only* if the changes specifically introduce or expose uninitialized memory risks that KMSAN uniquely catches do we enable it. KASAN, LOCKDEP, and other bug detectors are our first line of defense. I need to meticulously examine this patch series and the surrounding code, looking for those specific uninitialized memory scenarios. Specifically, I'm focusing on the following areas: 1. **Kernel structures passed to userspace:** Are we sending structures out that might have uninitialized fields (copy\_to\_user, put\_user, netlink, ioctl, socket options, BPF buffers)? 2. **Conditional logic and uninitialized variables:** Does the patch introduce branching or decisions based on potentially uninitialized variables or struct fields? 3. **Complex data structure initialization:** Are we allocating and initializing complex data structures where uninitialized fields could be read later in reachable code paths? 4. **Bounds checks and lengths:** Are we messing with bounds checks in a way that could allow access to uninitialized bytes of existing buffers? If the answer to these questions is no, and the patch primarily introduces risks already covered by KASAN (OOB, overflows, NULL dereferences, UAF, locking issues), then KMSAN is unnecessary. Similarly, if all touched structures are fully zeroed/initialized before reading/copying, we're good. Now, let's dive into the patch itself. The files touched are: `include/linux/bpf.h`, `include/linux/bpf_verifier.h`, `kernel/bpf/bpf_insn_array.c`, `kernel/bpf/cfg.c`, `kernel/bpf/fixups.c`, `kernel/bpf/liveness.c`, `kernel/bpf/verifier.c`, and some selftests. First, `bpf_insn_array.c`: The patch resets `jitted_off` and `ips[i]` to zero during map re-initialization. This is good; it's explicitly addressing stale pointers or offsets from reuse. It's a clean-up, but it's not immediately screaming "uninitialized memory vulnerability." Second, `bpf_insn_array_adjust`: The parameter rename and the change to the comparison condition seem straightforward. The changes appear to be more focused on correcting an offset, not adding an uninitialized memory risk. Third, `cfg.c`: Jump tables are being moved from per-instruction to per-subprogram. The patch reallocates and copies the jump table. I need to understand this realloc. I need to verify that the copying/reallocation handles initialization correctly and doesn't introduce any gaps where uninitialized memory could be exposed. Time to consult `bpf_iarray_realloc`'s implementation to see how it works and what `grepper` suggests. I'll focus on how the new jump tables are managed, how the reallocations are handled, and how memory is initialized during that process. Let's make sure nothing is missed! |
| 34/3 | 2026/09/14 21:23 | tool | grepper |
0mArgs:{
"Expression": "bpf_iarray_realloc"
}Results: {
"Output": "include/linux/bpf_verifier.h=1624=bool bpf_is_kfunc_pkt_changing(struct bpf_call_arg_meta *meta);\ninclude/linux/bpf_verifier.h:1625:struct bpf_iarray *bpf_iarray_realloc(struct bpf_iarray *old, size_t n_elem);\ninclude/linux/bpf_verifier.h-1626-int bpf_copy_insn_array_uniq(struct bpf_map *map, u32 start, u32 end, u32 *off);\n--\nkernel/bpf/cfg.c=163=static int visit_func_call_insn(int t, struct bpf_insn *insns,\n--\nkernel/bpf/cfg.c-187-\nkernel/bpf/cfg.c:188:struct bpf_iarray *bpf_iarray_realloc(struct bpf_iarray *old, size_t n_elem)\nkernel/bpf/cfg.c-189-{\n--\nkernel/bpf/cfg.c=261=static struct bpf_iarray *jt_from_map(struct bpf_map *map)\n--\nkernel/bpf/cfg.c-266-\nkernel/bpf/cfg.c:267:\tjt = bpf_iarray_realloc(NULL, map-\u003emax_entries);\nkernel/bpf/cfg.c-268-\tif (!jt)\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-329-\t\told_cnt = subprog-\u003ejt ? subprog-\u003ejt-\u003ecnt : 0;\nkernel/bpf/cfg.c:330:\t\tjt = bpf_iarray_realloc(subprog-\u003ejt, old_cnt + jt_cur-\u003ecnt);\nkernel/bpf/cfg.c-331-\t\tif (!jt) {\n--\nkernel/bpf/verifier.c=18224=static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-18257-\tif (!env-\u003egotox_tmp_buf || env-\u003egotox_tmp_buf-\u003ecnt \u003c max_index - min_index + 1) {\nkernel/bpf/verifier.c:18258:\t\tenv-\u003egotox_tmp_buf = bpf_iarray_realloc(env-\u003egotox_tmp_buf,\nkernel/bpf/verifier.c-18259-\t\t\t\t\t\t max_index - min_index + 1);\n--\nkernel/bpf/verifier.c=21213=int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-21289-\t\tenv-\u003einsn_aux_data[i].orig_idx = i;\nkernel/bpf/verifier.c:21290:\tenv-\u003esucc = bpf_iarray_realloc(NULL, 2);\nkernel/bpf/verifier.c-21291-\tif (!env-\u003esucc)\n"
} |
| 35/3 | 2026/09/14 21:23 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 31917 output: 36 thoughts: 25 Thoughts: Here's my summary, as though I'm reading and thinking about the code: **Diving into `bpf_iarray_realloc`** Okay, time to dig into `bpf_iarray_realloc` within `kernel/bpf/cfg.c`. I need to refresh my understanding of how BPF instruction arrays are managed during compilation. I recall that this function is crucial for handling situations where the initial allocation isn't sufficient to accommodate the generated BPF code. We're talking dynamic resizing, which, in a kernel context, is always a sensitive operation. I'm expecting to see careful checks for various error conditions – memory allocation failures being the obvious one. I'll be looking to understand the mechanics of moving data from the old array to the new, larger array. Are they using `memcpy` or some other more optimized kernel function? What considerations are made for potential alignment requirements? I also want to see how they handle the old array after the reallocation. Is it freed immediately, or is there a deferred freeing mechanism to avoid potential race conditions? This is where the compiler's safety and efficiency are most clearly expressed. I'm eager to see how the developers have balanced these competing demands in this area. It's often the little details in functions like this that reveal the true complexity and elegance of the underlying system. Let's get to it! |
| 36/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/cfg.c",
"FirstLine": 185,
"LineCount": 40
}Results: {
"Contents": " 185:\t\treturn ret;\n 186:\t}\n 187:\t\n 188:\tstruct bpf_iarray *bpf_iarray_realloc(struct bpf_iarray *old, size_t n_elem)\n 189:\t{\n 190:\t\tsize_t new_size = sizeof(struct bpf_iarray) + n_elem * sizeof(old-\u003eitems[0]);\n 191:\t\tstruct bpf_iarray *new;\n 192:\t\n 193:\t\tnew = kvrealloc(old, new_size, GFP_KERNEL_ACCOUNT);\n 194:\t\tif (!new) {\n 195:\t\t\t/* this is what callers always want, so simplify the call site */\n 196:\t\t\tkvfree(old);\n 197:\t\t\treturn NULL;\n 198:\t\t}\n 199:\t\n 200:\t\tnew-\u003ecnt = n_elem;\n 201:\t\treturn new;\n 202:\t}\n 203:\t\n 204:\tstatic int copy_insn_array(struct bpf_map *map, u32 start, u32 end, u32 *items)\n 205:\t{\n 206:\t\tstruct bpf_insn_array_value *value;\n 207:\t\tu32 i;\n 208:\t\n 209:\t\tfor (i = start; i \u003c= end; i++) {\n 210:\t\t\tvalue = map-\u003eops-\u003emap_lookup_elem(map, \u0026i);\n 211:\t\t\t/*\n 212:\t\t\t * map_lookup_elem of an array map will never return an error,\n 213:\t\t\t * but not checking it makes some static analysers to worry\n 214:\t\t\t */\n 215:\t\t\tif (IS_ERR(value))\n 216:\t\t\t\treturn PTR_ERR(value);\n 217:\t\t\telse if (!value)\n 218:\t\t\t\treturn -EINVAL;\n 219:\t\t\titems[i - start] = value-\u003exlated_off;\n 220:\t\t}\n 221:\t\treturn 0;\n 222:\t}\n 223:\t\n 224:\tstatic int cmp_ptr_to_u32(const void *a, const void *b)\n"
} |
| 37/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 32874 output: 22 thoughts: 278 Thoughts: **Analyzing the bpf_iarray_realloc Implementation and Usage** Okay, so let's break this down. First, `bpf_iarray_realloc` uses `kvrealloc` under the hood, allocating memory with `GFP_KERNEL_ACCOUNT`. That's standard and expected. Now, in the context of `compute_subprog_jts`, I'm seeing how it's used. Specifically, I'm focusing on that `bpf_iarray_realloc` call inside `compute_subprog_jts`. It looks like we're dynamically growing a jump table, `subprog->jt`. The code is adding `jt_cur->cnt` to the existing count, `old_cnt`. That `bpf_iarray_realloc` call is crucial; the code correctly handles a failure. It sets `subprog->jt` to `NULL` if `realloc` fails. Excellent, that’s good defensive programming – freeing the old allocation is handled by `bpf_iarray_realloc`, and no memory leak. Next, a `memcpy` copies the new data into the reallocated array, starting at the correct offset. Then we have this call to `sort_insn_array_uniq` which sorts and de-duplicates the items. I need to make sure `sort_insn_array_uniq` is doing its job correctly – especially in relation to the `memcpy` and reallocation. I'm going to quickly review that function now. |
| 38/3 | 2026/09/14 21:24 | tool | grepper |
0mArgs:{
"Expression": "sort_insn_array_uniq"
}Results: {
"Output": "kernel/bpf/cfg.c=224=static int cmp_ptr_to_u32(const void *a, const void *b)\n--\nkernel/bpf/cfg.c-228-\nkernel/bpf/cfg.c:229:static int sort_insn_array_uniq(u32 *items, int cnt)\nkernel/bpf/cfg.c-230-{\n--\nkernel/bpf/cfg.c=246=int bpf_copy_insn_array_uniq(struct bpf_map *map, u32 start, u32 end, u32 *off)\n--\nkernel/bpf/cfg.c-254-\nkernel/bpf/cfg.c:255:\treturn sort_insn_array_uniq(off, n);\nkernel/bpf/cfg.c-256-}\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-344-\t\tif (jt)\nkernel/bpf/cfg.c:345:\t\t\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\nkernel/bpf/cfg.c-346-\t}\n--\nkernel/bpf/verifier.c=18161=static int indirect_jump_min_max_index(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-18193-\nkernel/bpf/verifier.c:18194:/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */\nkernel/bpf/verifier.c-18195-static bool jt_contains(const struct bpf_iarray *jt, u32 target)\n"
} |
| 39/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 33963 output: 36 thoughts: 31 Thoughts: Here's my take on the thought process, as requested: **Diving into `sort_insn_array_uniq`** Okay, time to zero in on `sort_insn_array_uniq`. Specifically, I want to look at what's happening around line 229 in `kernel/bpf/cfg.c`. I'm guessing there's likely some kind of instruction array sorting or unique element handling going on there, but I need to understand the specifics within that context. I need to refresh my memory on the surrounding code, and then I can deduce what's actually being done here. From there, I need to understand why this function is being called and whether it's working properly for this configuration. |
| 40/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/cfg.c",
"FirstLine": 225,
"LineCount": 35
}Results: {
"Contents": " 225:\t{\n 226:\t\treturn *(u32 *)a - *(u32 *)b;\n 227:\t}\n 228:\t\n 229:\tstatic int sort_insn_array_uniq(u32 *items, int cnt)\n 230:\t{\n 231:\t\tint unique = 1;\n 232:\t\tint i;\n 233:\t\n 234:\t\tsort(items, cnt, sizeof(items[0]), cmp_ptr_to_u32, NULL);\n 235:\t\n 236:\t\tfor (i = 1; i \u003c cnt; i++)\n 237:\t\t\tif (items[i] != items[unique - 1])\n 238:\t\t\t\titems[unique++] = items[i];\n 239:\t\n 240:\t\treturn unique;\n 241:\t}\n 242:\t\n 243:\t/*\n 244:\t * sort_unique({map[start], ..., map[end]}) into off\n 245:\t */\n 246:\tint bpf_copy_insn_array_uniq(struct bpf_map *map, u32 start, u32 end, u32 *off)\n 247:\t{\n 248:\t\tu32 n = end - start + 1;\n 249:\t\tint err;\n 250:\t\n 251:\t\terr = copy_insn_array(map, start, end, off);\n 252:\t\tif (err)\n 253:\t\t\treturn err;\n 254:\t\n 255:\t\treturn sort_insn_array_uniq(off, n);\n 256:\t}\n 257:\t\n 258:\t/*\n 259:\t * Copy all unique offsets from the map\n"
} |
| 41/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 34675 output: 36 thoughts: 22 Thoughts: Here's a summary of my thought process, as if I were the one thinking it: **Diving into the BPF CFG Implementation** Okay, time to get my hands dirty. I need to take a look at the code responsible for the control flow graph generation within the BPF kernel. My intuition tells me the area that handles the specific block-linking logic might be in the 270-300 line range within `kernel/bpf/cfg.c`. I need to understand how the CFG is built. |
| 42/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/cfg.c",
"FirstLine": 260,
"LineCount": 40
}Results: {
"Contents": " 260:\t */\n 261:\tstatic struct bpf_iarray *jt_from_map(struct bpf_map *map)\n 262:\t{\n 263:\t\tstruct bpf_iarray *jt;\n 264:\t\tint err;\n 265:\t\tint n;\n 266:\t\n 267:\t\tjt = bpf_iarray_realloc(NULL, map-\u003emax_entries);\n 268:\t\tif (!jt)\n 269:\t\t\treturn ERR_PTR(-ENOMEM);\n 270:\t\n 271:\t\tn = bpf_copy_insn_array_uniq(map, 0, map-\u003emax_entries - 1, jt-\u003eitems);\n 272:\t\tif (n \u003c 0) {\n 273:\t\t\terr = n;\n 274:\t\t\tgoto err_free;\n 275:\t\t}\n 276:\t\tif (n == 0) {\n 277:\t\t\terr = -EINVAL;\n 278:\t\t\tgoto err_free;\n 279:\t\t}\n 280:\t\tjt-\u003ecnt = n;\n 281:\t\treturn jt;\n 282:\t\n 283:\terr_free:\n 284:\t\tkvfree(jt);\n 285:\t\treturn ERR_PTR(err);\n 286:\t}\n 287:\t\n 288:\t/*\n 289:\t * Collect the jump table of every subprogram that has one, as the combined\n 290:\t * table of all maps whose targets land inside that subprogram. All gotox\n 291:\t * instructions of a subprogram share the same table, so this is done in a\n 292:\t * single pass over the maps rather than once per gotox.\n 293:\t */\n 294:\tstatic int compute_subprog_jts(struct bpf_verifier_env *env)\n 295:\t{\n 296:\t\tstruct bpf_subprog_info *subprog;\n 297:\t\tstruct bpf_iarray *jt, *jt_cur;\n 298:\t\tstruct bpf_map *map;\n 299:\t\tu32 old_cnt;\n"
} |
| 43/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 35434 output: 37 thoughts: 801 Thoughts: **Analyzing BPF Control Flow and Patching** Alright, let's dive into this BPF code. I'm focusing on the improvements in strongly connected component (SCC) computation within the verifier. I see we're looking at `bpf_compute_scc` in `kernel/bpf/cfg.c`. The first thing that jumps out is the use of `kvcalloc` for the `dfs_pos` array. This is good; it ensures zero-initialization, which is crucial for the algorithm's correctness. We also free it with `kvfree` later on, as expected. Looking at the DFS traversal logic, the introduction of `dfs_pos` is clever. It's essentially an optimization to avoid re-scanning successors of a node that we've already visited within a single DFS search. The use of the `goto dfs_continue` is efficient, avoiding recursion, and `dfs_pos` allows us to resume from where we left off, picking up the next successor to explore. This avoids the overhead of rescanning successors, ensuring we hit the low update branch properly once the child DFS completes. That's a clean, and well-defined optimization. Now, let's shift gears to `fixups.c`. Specifically, I'm interested in how we patch instructions, and I'm looking at `__bpf_patch_insn_data`. The inclusion of `adjust_insn_aux_data` with its enum `bpf_patch_mode` suggests this function's part of the fixup strategy. I want to understand how aux data is handled during instruction patching. |
| 44/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/fixups.c",
"FirstLine": 213,
"LineCount": 60
}Results: {
"Contents": " 213:\t}\n 214:\t#endif\n 215:\t\n 216:\tenum bpf_patch_mode {\n 217:\t\tBPF_APPEND,\n 218:\t\tBPF_PREPEND,\n 219:\t};\n 220:\t\n 221:\t/* single env-\u003eprog-\u003einsni[off] instruction was replaced with the range\n 222:\t * insni[off, off + cnt). Adjust corresponding insn_aux_data by copying\n 223:\t * [0, off) and [off, end) to new locations, so the patched range stays zero\n 224:\t */\n 225:\tstatic void adjust_insn_aux_data(struct bpf_verifier_env *env,\n 226:\t\t\t\t\t struct bpf_prog *new_prog, u32 off, u32 cnt,\n 227:\t\t\t\t\t enum bpf_patch_mode mode)\n 228:\t{\n 229:\t\tstruct bpf_insn_aux_data *data = env-\u003einsn_aux_data;\n 230:\t\tstruct bpf_insn *insn = new_prog-\u003einsnsi;\n 231:\t\tu32 old_seen = data[off].seen;\n 232:\t\tu32 prog_len;\n 233:\t\tint i;\n 234:\t\n 235:\t\t/* aux info at OFF always needs adjustment, no matter fast path\n 236:\t\t * (cnt == 1) is taken or not. There is no guarantee INSN at OFF is the\n 237:\t\t * original insn at old prog.\n 238:\t\t */\n 239:\t\tdata[off].zext_dst = bpf_insn_def32(new_prog, insn + off + cnt - 1) \u003e= 0;\n 240:\t\n 241:\t\tif (cnt == 1)\n 242:\t\t\treturn;\n 243:\t\tprog_len = new_prog-\u003elen;\n 244:\t\tenv-\u003einsn_aux_data_len = prog_len;\n 245:\t\n 246:\t\tmemmove(data + off + cnt - 1, data + off,\n 247:\t\t\tsizeof(struct bpf_insn_aux_data) * (prog_len - off - cnt + 1));\n 248:\t\tmemset(data + off, 0, sizeof(struct bpf_insn_aux_data) * (cnt - 1));\n 249:\t\tfor (i = off; i \u003c off + cnt - 1; i++) {\n 250:\t\t\t/* Expand insni[off]'s seen count to the patched range. */\n 251:\t\t\tdata[i].seen = old_seen;\n 252:\t\t\tdata[i].zext_dst = bpf_insn_def32(new_prog, insn + i) \u003e= 0;\n 253:\t\t}\n 254:\t\n 255:\t\t/*\n 256:\t\t * The indirect_target flag of the original instruction was moved to the last of the\n 257:\t\t * new instructions by the above memmove and memset, but the indirect jump target is\n 258:\t\t * actually the first instruction, so move it back. This also matches with the behavior\n 259:\t\t * of bpf_insn_array_adjust(), which preserves xlated_off to point to the first new\n 260:\t\t * instruction. For BPF_PREPEND the original instruction is the last one, so the flag\n 261:\t\t * already sits where needed.\n 262:\t\t */\n 263:\t\tif (mode == BPF_APPEND \u0026\u0026 data[off + cnt - 1].indirect_target) {\n 264:\t\t\tdata[off].indirect_target = 1;\n 265:\t\t\tdata[off + cnt - 1].indirect_target = 0;\n 266:\t\t}\n 267:\t}\n 268:\t\n 269:\tstatic void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)\n 270:\t{\n 271:\t\tint i;\n 272:\t\n"
} |
| 45/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 37700 output: 37 thoughts: 806 Thoughts: **Analyzing BPF Instruction Adjustments and Liveness** Okay, so I'm digging into these BPF code changes, and a few things are immediately jumping out at me. First, let's look at `adjust_insn_aux_data`. This function is memmoving and zeroing out a patched range. That's pretty standard, but I need to understand the implications of this zeroing in relation to other data structures. Then, there's `adjust_insn_arrays`, and I see a subtle but important change with the `BPF_PREPEND` mode affecting the offset calculation. It's a single byte offset, so let's check its effects down the line. Now, `bpf_insn_array_adjust` is where things get interesting. The conditional `xlated_off < first` is likely an optimization, and I want to assess if it's correct under all circumstances. I should also think about the consequences of the added `len - 1` to `xlated_off`. Moving on to `bpf_opt_remove_nops`, the added check for `indirect_target` is crucial. The code's preventing the removal of NOPs that are indirect targets, which means we can't optimize them away due to the potential for indirect jumps. This is a good sanity check. Now, let's shift gears and look at `liveness.c`. This is all about successor analysis, and some significant changes were made. Previously, `jt = env->insn_aux_data[idx].jt` was used to determine the jump table for every abnormal return instruction (tail call, ld_abs, ld_ind). However, now it's using `bpf_find_containing_subprog(env, idx)->jt`. This suggests a move toward shared jump tables within subprograms, which is a good design choice, it reduces memory overhead, and allows for more efficient flow analysis. Instead of the jump table, for these specific cases, we are now just adding the subprogram's `exit_idx` directly into `succ->items`. I need to understand what `env->succ` is. It's allocated with `bpf_iarray_realloc(NULL, 2)`. So, at most, can `succ` only hold two items? Let's trace how many items could be in `succ` in `bpf_insn_successors`. This might be a potential bottleneck or, worse, an overflow. Let's make sure it's handled correctly to avoid potential issues down the line. I want to fully understand the impact of this change on the successor analysis and potential for incorrect control-flow decisions. |
| 46/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/liveness.c",
"FirstLine": 240,
"LineCount": 40
}Results: {
"Contents": " 240:\t\t\t_J(BPF_JSLE, {.can_jump = true, .can_fallthrough = true}),\n 241:\t\t\t_J(BPF_JCOND, {.can_jump = true, .can_fallthrough = true}),\n 242:\t\t\t_J(BPF_JSET, {.can_jump = true, .can_fallthrough = true}),\n 243:\t\t#undef _J\n 244:\t\t};\n 245:\t\tstruct bpf_prog *prog = env-\u003eprog;\n 246:\t\tstruct bpf_insn *insn = \u0026prog-\u003einsnsi[idx];\n 247:\t\tconst struct opcode_info *opcode_info;\n 248:\t\tstruct bpf_iarray *succ, *jt;\n 249:\t\tint insn_sz;\n 250:\t\n 251:\t\t/* All gotox of a subprogram share its jump table, see compute_subprog_jts() */\n 252:\t\tif (unlikely(insn_is_gotox(insn))) {\n 253:\t\t\tjt = bpf_find_containing_subprog(env, idx)-\u003ejt;\n 254:\t\t\tif (jt)\n 255:\t\t\t\treturn jt;\n 256:\t\t}\n 257:\t\n 258:\t\t/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */\n 259:\t\tsucc = env-\u003esucc;\n 260:\t\tsucc-\u003ecnt = 0;\n 261:\t\n 262:\t\topcode_info = \u0026opcode_info_tbl[BPF_CLASS(insn-\u003ecode) | BPF_OP(insn-\u003ecode)];\n 263:\t\tinsn_sz = bpf_is_ldimm64(insn) ? 2 : 1;\n 264:\t\tif (opcode_info-\u003ecan_fallthrough)\n 265:\t\t\tsucc-\u003eitems[succ-\u003ecnt++] = idx + insn_sz;\n 266:\t\n 267:\t\tif (opcode_info-\u003ecan_jump)\n 268:\t\t\tsucc-\u003eitems[succ-\u003ecnt++] = idx + bpf_jmp_offset(insn) + 1;\n 269:\t\n 270:\t\t/*\n 271:\t\t * tail_call upon success and ld_{abs,ind} upon load failure return\n 272:\t\t * from the subprogram, which is a hidden edge to its exit.\n 273:\t\t */\n 274:\t\tif (unlikely((bpf_helper_call(insn) \u0026\u0026 insn-\u003eimm == BPF_FUNC_tail_call) ||\n 275:\t\t\t (BPF_CLASS(insn-\u003ecode) == BPF_LD \u0026\u0026\n 276:\t\t\t (BPF_MODE(insn-\u003ecode) == BPF_ABS || BPF_MODE(insn-\u003ecode) == BPF_IND))))\n 277:\t\t\tsucc-\u003eitems[succ-\u003ecnt++] = bpf_find_containing_subprog(env, idx)-\u003eexit_idx;\n 278:\t\n 279:\t\treturn succ;\n"
} |
| 47/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 39919 output: 18 thoughts: 1112 Thoughts: **Analyzing BPF Verifier Control Flow and Jump Tables** Okay, let's break this down. First, I was thinking about how instructions like `tail_call` and `ld_abs/ld_ind` impact the successor count within the BPF verifier's control flow graph. For `tail_call`, which is essentially a `BPF_JMP | BPF_CALL`, the initial count `succ->cnt` will be 1 (for the next instruction). Because the `BPF_CALL` opcode's entry in `opcode_info_tbl` has `can_fallthrough = true`, the verifier correctly increments this count. Then, the `tail_call` instruction adds an `exit_idx`, increasing `succ->cnt` to 2. This seems correct. For `ld_abs/ld_ind` (using `BPF_LD`), the class is `BPF_LD` and therefore falls through. So, it should follow the same pattern - `succ->cnt` starts at 0, fallthrough increments it to 1, and the "hidden exit" further increments it to 2. This implies `succ->cnt` can never exceed 2. Since `env->succ` is allocated with a size of 2, it is safe! The verifier should avoid out-of-bounds writes because any attempt to exceed this would be caught by KASAN! Good. Next, I focused on the `check_indirect_jump` function in `verifier.c`. This part's critical for indirect jumps, specifically GOTO jumps. It validates that the jump targets are within the correct subprogram. The code extracts `subprog_start`, `subprog_end`, and the jump table `jt` for the current instruction. The jump table `jt` must contain the target of the `GOTO` instruction. The code then iterates through each target in `env->gotox_tmp_buf->items` to ensure that: 1. The target falls within the subprogram's bounds. 2. The target is present in the jump table using `jt_contains()`. The `jt_contains()` function itself uses a standard binary search, which is efficient as long as the jump table is properly sorted and free of duplicates. I verified that all items in `jt->items` are populated when created in `compute_subprog_jts()`. The `subprog_info` has `jt` field which is freed using `bpf_free_subprog_jts()` in `bpf_check` when cleaning up the environment. I want to see if `env->subprog_info` is initialized or allocated anywhere else. Let's run a grepper search for `subprog_info` to find out. |
| 48/3 | 2026/09/14 21:24 | tool | grepper |
0mArgs:{
"Expression": "subprog_info"
}Results: {
"Output": "\nFull output is too long, showing 500 out of 660 lines.\nUse more precise expression if possible.\n\ndrivers/net/ethernet/netronome/nfp/bpf/main.h=468=static inline bool is_mbpf_pseudo_call(const struct nfp_insn_meta *meta)\n--\ndrivers/net/ethernet/netronome/nfp/bpf/main.h-478-/**\ndrivers/net/ethernet/netronome/nfp/bpf/main.h:479: * struct nfp_bpf_subprog_info - nfp BPF sub-program (a.k.a. function) info\ndrivers/net/ethernet/netronome/nfp/bpf/main.h-480- * @stack_depth:\tmaximum stack depth used by this sub-program\n--\ndrivers/net/ethernet/netronome/nfp/bpf/main.h-482- */\ndrivers/net/ethernet/netronome/nfp/bpf/main.h:483:struct nfp_bpf_subprog_info {\ndrivers/net/ethernet/netronome/nfp/bpf/main.h-484-\tu16 stack_depth;\n--\ndrivers/net/ethernet/netronome/nfp/bpf/main.h=513=struct nfp_prog {\n--\ndrivers/net/ethernet/netronome/nfp/bpf/main.h-540-\tstruct nfp_bpf_neutral_map **map_records;\ndrivers/net/ethernet/netronome/nfp/bpf/main.h:541:\tstruct nfp_bpf_subprog_info *subprog;\ndrivers/net/ethernet/netronome/nfp/bpf/main.h-542-\n--\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c=763=int nfp_bpf_finalize(struct bpf_verifier_env *env)\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c-764-{\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c:765:\tstruct bpf_subprog_info *info;\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c-766-\tstruct nfp_prog *nfp_prog;\n--\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c-779-\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c:780:\tinfo = env-\u003esubprog_info;\ndrivers/net/ethernet/netronome/nfp/bpf/verifier.c-781-\tfor (i = 0; i \u003c nfp_prog-\u003esubprog_cnt; i++) {\n--\ninclude/linux/bpf_verifier.h=348=struct bpf_func_state {\n--\ninclude/linux/bpf_verifier.h-361-\tu32 diag_frame_id;\ninclude/linux/bpf_verifier.h:362:\t/* subprog number == index within subprog_info\ninclude/linux/bpf_verifier.h-363-\t * zero == main subprog\n--\ninclude/linux/bpf_verifier.h=795=enum priv_stack_mode {\n--\ninclude/linux/bpf_verifier.h-800-\ninclude/linux/bpf_verifier.h:801:struct bpf_subprog_info {\ninclude/linux/bpf_verifier.h-802-\tconst char *name; /* name extracted from BTF */\n--\ninclude/linux/bpf_verifier.h-835-\ninclude/linux/bpf_verifier.h:836:static inline u16 bpf_in_stack_arg_cnt(const struct bpf_subprog_info *sub)\ninclude/linux/bpf_verifier.h-837-{\n--\ninclude/linux/bpf_verifier.h=918=struct bpf_verifier_env {\n--\ninclude/linux/bpf_verifier.h-961-\tstruct bpf_diag *diag;\ninclude/linux/bpf_verifier.h:962:\tstruct bpf_subprog_info subprog_info[BPF_MAX_SUBPROGS + 2]; /* max + 2 for the fake and exception subprogs */\ninclude/linux/bpf_verifier.h-963-\t/* subprog indices sorted in topological order: leaves first, callers last */\n--\ninclude/linux/bpf_verifier.h-973-\t\t * vector of instruction indexes sorted in post-order, grouped by subprogram,\ninclude/linux/bpf_verifier.h:974:\t\t * see bpf_subprog_info-\u003epostorder_start.\ninclude/linux/bpf_verifier.h-975-\t\t */\n--\ninclude/linux/bpf_verifier.h=1052=static inline struct bpf_func_info_aux *subprog_aux(struct bpf_verifier_env *env, int subprog)\n--\ninclude/linux/bpf_verifier.h-1056-\ninclude/linux/bpf_verifier.h:1057:static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env, int subprog)\ninclude/linux/bpf_verifier.h-1058-{\ninclude/linux/bpf_verifier.h:1059:\treturn \u0026env-\u003esubprog_info[subprog];\ninclude/linux/bpf_verifier.h-1060-}\n--\ninclude/linux/bpf_verifier.h=1492=const char *bpf_disasm_kfunc_name(void *data, const struct bpf_insn *insn);\ninclude/linux/bpf_verifier.h-1493-\ninclude/linux/bpf_verifier.h:1494:struct bpf_subprog_info *bpf_find_containing_subprog(struct bpf_verifier_env *env, int off);\ninclude/linux/bpf_verifier.h-1495-const char *bpf_subprog_name(const struct bpf_verifier_env *env, int subprog);\n--\nkernel/bpf/btf.c=8016=int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)\n--\nkernel/bpf/btf.c-8018-\tbool is_global = subprog_aux(env, subprog)-\u003elinkage == BTF_FUNC_GLOBAL;\nkernel/bpf/btf.c:8019:\tstruct bpf_subprog_info *sub = subprog_info(env, subprog);\nkernel/bpf/btf.c-8020-\tstruct bpf_verifier_log *log = \u0026env-\u003elog;\n--\nkernel/bpf/cfg.c=54=static void mark_subprog_changes_pkt_data(struct bpf_verifier_env *env, int off)\nkernel/bpf/cfg.c-55-{\nkernel/bpf/cfg.c:56:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-57-\n--\nkernel/bpf/cfg.c=62=static void mark_subprog_might_sleep(struct bpf_verifier_env *env, int off)\nkernel/bpf/cfg.c-63-{\nkernel/bpf/cfg.c:64:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-65-\n--\nkernel/bpf/cfg.c=70=static void mark_subprog_might_throw(struct bpf_verifier_env *env, int off)\nkernel/bpf/cfg.c-71-{\nkernel/bpf/cfg.c:72:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-73-\n--\nkernel/bpf/cfg.c=84=static void merge_callee_effects(struct bpf_verifier_env *env, int t, int w)\nkernel/bpf/cfg.c-85-{\nkernel/bpf/cfg.c:86:\tstruct bpf_subprog_info *caller, *callee;\nkernel/bpf/cfg.c-87-\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-295-{\nkernel/bpf/cfg.c:296:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-297-\tstruct bpf_iarray *jt, *jt_cur;\n--\nkernel/bpf/cfg.c-342-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/cfg.c:343:\t\tjt = env-\u003esubprog_info[i].jt;\nkernel/bpf/cfg.c-344-\t\tif (jt)\n--\nkernel/bpf/cfg.c=352=void bpf_free_subprog_jts(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-355-\nkernel/bpf/cfg.c:356:\tfor (i = 0; i \u003c ARRAY_SIZE(env-\u003esubprog_info); i++) {\nkernel/bpf/cfg.c:357:\t\tkvfree(env-\u003esubprog_info[i].jt);\nkernel/bpf/cfg.c:358:\t\tenv-\u003esubprog_info[i].jt = NULL;\nkernel/bpf/cfg.c-359-\t}\n--\nkernel/bpf/cfg.c=364=subprog_jt(int t, struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-365-{\nkernel/bpf/cfg.c:366:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-367-\tint subprog_start, err;\n--\nkernel/bpf/cfg.c=580=int bpf_check_cfg(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-598-\tex_insn_beg = env-\u003eexception_callback_subprog\nkernel/bpf/cfg.c:599:\t\t ? env-\u003esubprog_info[env-\u003eexception_callback_subprog].start\nkernel/bpf/cfg.c-600-\t\t : 0;\n--\nkernel/bpf/cfg.c-666-\tret = 0; /* cfg looks good */\nkernel/bpf/cfg.c:667:\tenv-\u003eprog-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[0].changes_pkt_data;\nkernel/bpf/cfg.c:668:\tenv-\u003eprog-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[0].might_sleep;\nkernel/bpf/cfg.c-669-\n--\nkernel/bpf/cfg.c-678- * For each subprogram 'i' fill array env-\u003ecfg.insn_subprogram sub-range\nkernel/bpf/cfg.c:679: * [env-\u003esubprog_info[i].postorder_start, env-\u003esubprog_info[i+1].postorder_start)\nkernel/bpf/cfg.c-680- * with indices of 'i' instructions in postorder.\n--\nkernel/bpf/cfg.c=682=int bpf_compute_postorder(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-698-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/cfg.c:699:\t\tenv-\u003esubprog_info[i].postorder_start = cur_postorder;\nkernel/bpf/cfg.c:700:\t\tstack[0] = env-\u003esubprog_info[i].start;\nkernel/bpf/cfg.c-701-\t\tstack_sz = 1;\n--\nkernel/bpf/cfg.c-719-\t}\nkernel/bpf/cfg.c:720:\tenv-\u003esubprog_info[i].postorder_start = cur_postorder;\nkernel/bpf/cfg.c-721-\tenv-\u003ecfg.insn_postorder = postorder;\n--\nkernel/bpf/check_btf.c=10=static int check_abnormal_return(struct bpf_verifier_env *env)\n--\nkernel/bpf/check_btf.c-14-\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/check_btf.c:15:\t\tif (env-\u003esubprog_info[i].has_ld_abs) {\nkernel/bpf/check_btf.c-16-\t\t\tverbose(env, \"LD_ABS is not allowed in subprogs without BTF\\n\");\n--\nkernel/bpf/check_btf.c-18-\t\t}\nkernel/bpf/check_btf.c:19:\t\tif (env-\u003esubprog_info[i].has_tail_call) {\nkernel/bpf/check_btf.c-20-\t\t\tverbose(env, \"tail_call is not allowed in subprogs without BTF\\n\");\n--\nkernel/bpf/check_btf.c=133=static int check_btf_func(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-174-\nkernel/bpf/check_btf.c:175:\t\tif (env-\u003esubprog_info[i].start != krecord[i].insn_off) {\nkernel/bpf/check_btf.c-176-\t\t\tverbose(env, \"func_info BTF section doesn't match subprog layout in BPF program\\n\");\n--\nkernel/bpf/check_btf.c-188-\t\t\tbtf_type_is_small_int(ret_type) || btf_is_any_enum(ret_type);\nkernel/bpf/check_btf.c:189:\t\tif (i \u0026\u0026 !scalar_return \u0026\u0026 env-\u003esubprog_info[i].has_ld_abs) {\nkernel/bpf/check_btf.c-190-\t\t\tverbose(env, \"LD_ABS is only allowed in functions that return 'int'.\\n\");\n--\nkernel/bpf/check_btf.c-192-\t\t}\nkernel/bpf/check_btf.c:193:\t\tif (i \u0026\u0026 !scalar_return \u0026\u0026 env-\u003esubprog_info[i].has_tail_call) {\nkernel/bpf/check_btf.c-194-\t\t\tverbose(env, \"tail_call is only allowed in functions that return 'int'.\\n\");\n--\nkernel/bpf/check_btf.c-197-\nkernel/bpf/check_btf.c:198:\t\tenv-\u003esubprog_info[i].name = btf_name_by_offset(btf, type-\u003ename_off);\nkernel/bpf/check_btf.c-199-\t\tbpfptr_add(\u0026urecord, urec_size);\n--\nkernel/bpf/check_btf.c=213=static int check_btf_line(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-217-\tu32 i, s, nr_linfo, ncopy, expected_size, rec_size, prev_offset = 0;\nkernel/bpf/check_btf.c:218:\tstruct bpf_subprog_info *sub;\nkernel/bpf/check_btf.c-219-\tstruct bpf_line_info *linfo;\n--\nkernel/bpf/check_btf.c-248-\ts = 0;\nkernel/bpf/check_btf.c:249:\tsub = env-\u003esubprog_info;\nkernel/bpf/check_btf.c-250-\tulinfo = make_bpfptr(attr-\u003eline_info, uattr.is_kernel);\n--\nkernel/bpf/check_btf.c-279-\t\t * first sub also and the first sub must have\nkernel/bpf/check_btf.c:280:\t\t * subprog_info[0].start == 0.\nkernel/bpf/check_btf.c-281-\t\t */\n--\nkernel/bpf/const_fold.c=232=int bpf_compute_const_regs(struct bpf_verifier_env *env)\n--\nkernel/bpf/const_fold.c-250-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/const_fold.c:251:\t\tint start = env-\u003esubprog_info[i].start;\nkernel/bpf/const_fold.c-252-\n--\nkernel/bpf/core.c=1650=u16 bpf_out_stack_arg_cnt(const struct bpf_verifier_env *env, const struct bpf_prog *prog)\nkernel/bpf/core.c-1651-{\nkernel/bpf/core.c:1652:\tconst struct bpf_subprog_info *sub;\nkernel/bpf/core.c-1653-\n--\nkernel/bpf/core.c-1655-\t\treturn 0;\nkernel/bpf/core.c:1656:\tsub = \u0026env-\u003esubprog_info[prog-\u003eaux-\u003efunc_idx];\nkernel/bpf/core.c-1657-\treturn sub-\u003estack_arg_cnt - bpf_in_stack_arg_cnt(sub);\n--\nkernel/bpf/diagnostics.c=797=static void bpf_diag_source(struct bpf_verifier_env *env, u32 insn_idx, const char *label,\n--\nkernel/bpf/diagnostics.c-805-\tconst struct bpf_line_info *linfo;\nkernel/bpf/diagnostics.c:806:\tconst struct bpf_subprog_info *subprog;\nkernel/bpf/diagnostics.c-807-\tstruct btf *btf = env-\u003eprog-\u003eaux-\u003ebtf;\n--\nkernel/bpf/diagnostics.c-841-\tsubprog = bpf_find_containing_subprog(env, insn_idx);\nkernel/bpf/diagnostics.c:842:\tsubprogno = subprog ? subprog - env-\u003esubprog_info : -ENOENT;\nkernel/bpf/diagnostics.c-843-\tfunc = subprogno \u003e= 0 ? bpf_subprog_name(env, subprogno) : NULL;\n--\nkernel/bpf/diagnostics.c-860-\tlinfo_end = subprogno \u003e= 0 \u0026\u0026 subprogno + 1 \u003c env-\u003esubprog_cnt ?\nkernel/bpf/diagnostics.c:861:\t\t env-\u003esubprog_info[subprogno + 1].linfo_idx : env-\u003eprog-\u003eaux-\u003enr_linfo;\nkernel/bpf/diagnostics.c-862-\tfor (i = linfo_start; i \u003c linfo_end; i++) {\n--\nkernel/bpf/fixups.c=204=static int get_callee_stack_depth(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-211-\t\treturn -EFAULT;\nkernel/bpf/fixups.c:212:\treturn env-\u003esubprog_info[subprog].stack_depth;\nkernel/bpf/fixups.c-213-}\n--\nkernel/bpf/fixups.c=269=static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)\n--\nkernel/bpf/fixups.c-276-\tfor (i = 0; i \u003c= env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:277:\t\tif (env-\u003esubprog_info[i].start \u003c= off)\nkernel/bpf/fixups.c-278-\t\t\tcontinue;\nkernel/bpf/fixups.c:279:\t\tenv-\u003esubprog_info[i].start += len - 1;\nkernel/bpf/fixups.c-280-\t}\n--\nkernel/bpf/fixups.c=393=static int adjust_subprog_starts_after_remove(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-399-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:400:\t\tif (env-\u003esubprog_info[i].start \u003e= off)\nkernel/bpf/fixups.c-401-\t\t\tbreak;\n--\nkernel/bpf/fixups.c-403-\tfor (j = i; j \u003c env-\u003esubprog_cnt; j++)\nkernel/bpf/fixups.c:404:\t\tif (env-\u003esubprog_info[j].start \u003e= off + cnt)\nkernel/bpf/fixups.c-405-\t\t\tbreak;\n--\nkernel/bpf/fixups.c-408-\t */\nkernel/bpf/fixups.c:409:\tif (env-\u003esubprog_info[j].start != off + cnt)\nkernel/bpf/fixups.c-410-\t\tj--;\n--\nkernel/bpf/fixups.c-418-\nkernel/bpf/fixups.c:419:\t\tmemmove(env-\u003esubprog_info + i,\nkernel/bpf/fixups.c:420:\t\t\tenv-\u003esubprog_info + j,\nkernel/bpf/fixups.c:421:\t\t\tsizeof(*env-\u003esubprog_info) * move);\nkernel/bpf/fixups.c-422-\t\tenv-\u003esubprog_cnt -= j - i;\n--\nkernel/bpf/fixups.c-441-\t\t/* convert i from \"first prog to remove\" to \"first to adjust\" */\nkernel/bpf/fixups.c:442:\t\tif (env-\u003esubprog_info[i].start == off)\nkernel/bpf/fixups.c-443-\t\t\ti++;\n--\nkernel/bpf/fixups.c-447-\tfor (; i \u003c= env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:448:\t\tenv-\u003esubprog_info[i].start -= cnt;\nkernel/bpf/fixups.c-449-\n--\nkernel/bpf/fixups.c=453=static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,\n--\nkernel/bpf/fixups.c-503-\tfor (i = 0; i \u003c= env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:504:\t\tif (env-\u003esubprog_info[i].linfo_idx \u003e l_off) {\nkernel/bpf/fixups.c-505-\t\t\t/* program may have started in the removed region but\n--\nkernel/bpf/fixups.c-507-\t\t\t */\nkernel/bpf/fixups.c:508:\t\t\tif (env-\u003esubprog_info[i].linfo_idx \u003e= l_off + l_cnt)\nkernel/bpf/fixups.c:509:\t\t\t\tenv-\u003esubprog_info[i].linfo_idx -= l_cnt;\nkernel/bpf/fixups.c-510-\t\t\telse\nkernel/bpf/fixups.c:511:\t\t\t\tenv-\u003esubprog_info[i].linfo_idx = l_off;\nkernel/bpf/fixups.c-512-\t\t}\n--\nkernel/bpf/fixups.c=744=int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-745-{\nkernel/bpf/fixups.c:746:\tstruct bpf_subprog_info *subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-747-\tconst struct bpf_verifier_ops *ops = env-\u003eops;\n--\nkernel/bpf/fixups.c=1051=static u32 *bpf_dup_subprog_starts(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1057-\t\tfor (int i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:1058:\t\t\tstarts[i] = env-\u003esubprog_info[i].start;\nkernel/bpf/fixups.c-1059-\t}\n--\nkernel/bpf/fixups.c=1063=static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)\n--\nkernel/bpf/fixups.c-1065-\tfor (int i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:1066:\t\tenv-\u003esubprog_info[i].start = orig_starts[i];\nkernel/bpf/fixups.c-1067-\t/* restore the start of fake 'exit' subprog as well */\nkernel/bpf/fixups.c:1068:\tenv-\u003esubprog_info[env-\u003esubprog_cnt].start = env-\u003eprog-\u003elen;\nkernel/bpf/fixups.c-1069-}\n--\nkernel/bpf/fixups.c=1071=static int jit_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1127-\t\tsubprog_start = subprog_end;\nkernel/bpf/fixups.c:1128:\t\tsubprog_end = env-\u003esubprog_info[i + 1].start;\nkernel/bpf/fixups.c-1129-\n--\nkernel/bpf/fixups.c-1167-\t\tfunc[i]-\u003eaux-\u003ename[0] = 'F';\nkernel/bpf/fixups.c:1168:\t\tfunc[i]-\u003eaux-\u003estack_depth = env-\u003esubprog_info[i].stack_depth;\nkernel/bpf/fixups.c:1169:\t\tif (env-\u003esubprog_info[i].priv_stack_mode == PRIV_STACK_ADAPTIVE)\nkernel/bpf/fixups.c-1170-\t\t\tfunc[i]-\u003eaux-\u003ejits_use_priv_stack = true;\n--\nkernel/bpf/fixups.c-1178-\t\tfunc[i]-\u003eaux-\u003ejited_linfo = prog-\u003eaux-\u003ejited_linfo;\nkernel/bpf/fixups.c:1179:\t\tfunc[i]-\u003eaux-\u003elinfo_idx = env-\u003esubprog_info[i].linfo_idx;\nkernel/bpf/fixups.c-1180-\t\tfunc[i]-\u003eaux-\u003earena = prog-\u003eaux-\u003earena;\n--\nkernel/bpf/fixups.c-1200-\t\tfunc[i]-\u003eaux-\u003enum_exentries = num_exentries;\nkernel/bpf/fixups.c:1201:\t\tfunc[i]-\u003eaux-\u003etail_call_reachable = env-\u003esubprog_info[i].tail_call_reachable;\nkernel/bpf/fixups.c:1202:\t\tfunc[i]-\u003eaux-\u003eexception_cb = env-\u003esubprog_info[i].is_exception_cb;\nkernel/bpf/fixups.c:1203:\t\tfunc[i]-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[i].changes_pkt_data;\nkernel/bpf/fixups.c:1204:\t\tfunc[i]-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[i].might_sleep;\nkernel/bpf/fixups.c-1205-\t\tfunc[i]-\u003eaux-\u003etoken = prog-\u003eaux-\u003etoken;\n--\nkernel/bpf/fixups.c=1410=int bpf_fixup_call_args(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1419-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:1420:\t\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[i];\nkernel/bpf/fixups.c-1421-\t\tu16 outgoing = subprog-\u003estack_arg_cnt - bpf_in_stack_arg_cnt(subprog);\n--\nkernel/bpf/fixups.c-1444-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:1445:\t\tif (bpf_in_stack_arg_cnt(\u0026env-\u003esubprog_info[i])) {\nkernel/bpf/fixups.c-1446-\t\t\tverbose(env, \"stack args are not supported in non-JITed programs\\n\");\n--\nkernel/bpf/fixups.c=1484=static int add_hidden_subprog(struct bpf_verifier_env *env, struct bpf_insn *patch, int len)\nkernel/bpf/fixups.c-1485-{\nkernel/bpf/fixups.c:1486:\tstruct bpf_subprog_info *info = env-\u003esubprog_info;\nkernel/bpf/fixups.c-1487-\tint cnt = env-\u003esubprog_cnt;\n--\nkernel/bpf/fixups.c-1489-\nkernel/bpf/fixups.c:1490:\t/* We only reserve one slot for hidden subprogs in subprog_info. */\nkernel/bpf/fixups.c-1491-\tif (env-\u003ehidden_subprog_cnt) {\n--\nkernel/bpf/fixups.c=1513=int bpf_do_misc_fixups(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1526-\tint i, ret, cnt, delta = 0, cur_subprog = 0;\nkernel/bpf/fixups.c:1527:\tstruct bpf_subprog_info *subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-1528-\tu16 stack_depth = subprogs[cur_subprog].stack_depth;\n--\nkernel/bpf/fixups.c=2500=static struct bpf_prog *inline_bpf_loop(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-2566-\t/* callback start is known only after patching */\nkernel/bpf/fixups.c:2567:\tcallback_start = env-\u003esubprog_info[callback_subprogno].start;\nkernel/bpf/fixups.c-2568-\t/* Note: insn_buf[12] is an offset of BPF_CALL_REL instruction */\n--\nkernel/bpf/fixups.c=2592=int bpf_optimize_bpf_loop(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-2593-{\nkernel/bpf/fixups.c:2594:\tstruct bpf_subprog_info *subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-2595-\tint i, cur_subprog = 0, cnt, delta = 0;\n--\nkernel/bpf/fixups.c-2631-\nkernel/bpf/fixups.c:2632:\tenv-\u003eprog-\u003eaux-\u003estack_depth = env-\u003esubprog_info[0].stack_depth;\nkernel/bpf/fixups.c-2633-\n--\nkernel/bpf/fixups.c=2640=int bpf_remove_fastcall_spills_fills(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-2641-{\nkernel/bpf/fixups.c:2642:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/fixups.c-2643-\tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/liveness.c=24=struct func_instance {\n--\nkernel/bpf/liveness.c-28-\tu32 subprog;\t\t/* subprog index */\nkernel/bpf/liveness.c:29:\tu32 subprog_start;\t/* cached env-\u003esubprog_info[subprog].start */\nkernel/bpf/liveness.c-30-\tu32 insn_cnt;\t\t/* cached number of insns in the function */\n--\nkernel/bpf/liveness.c=74=static struct func_instance *call_instance(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-78-\tu32 depth = caller ? caller-\u003edepth + 1 : 0;\nkernel/bpf/liveness.c:79:\tu32 subprog_start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c-80-\tu32 lookup_key = depth \u003e 0 ? callsite : subprog_start;\n--\nkernel/bpf/liveness.c-94-\tf-\u003esubprog_start = subprog_start;\nkernel/bpf/liveness.c:95:\tf-\u003einsn_cnt = (env-\u003esubprog_info + subprog + 1)-\u003estart - subprog_start;\nkernel/bpf/liveness.c-96-\thash = instance_hash(lookup_key, depth);\n--\nkernel/bpf/liveness.c=101=static struct func_instance *lookup_instance(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-108-\nkernel/bpf/liveness.c:109:\tsubprog_start = env-\u003esubprog_info[st-\u003eframe[frameno]-\u003esubprogno].start;\nkernel/bpf/liveness.c-110-\tcallsite = frameno \u003e 0 ? st-\u003eframe[frameno]-\u003ecallsite : subprog_start;\n--\nkernel/bpf/liveness.c=318=static void update_instance(struct bpf_verifier_env *env, struct func_instance *instance)\n--\nkernel/bpf/liveness.c-321-\tint *insn_postorder = env-\u003ecfg.insn_postorder;\nkernel/bpf/liveness.c:322:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/liveness.c-323-\tbool changed;\n--\nkernel/bpf/liveness.c-325-\tinstance-\u003emust_write_initialized = true;\nkernel/bpf/liveness.c:326:\tsubprog = \u0026env-\u003esubprog_info[instance-\u003esubprog];\nkernel/bpf/liveness.c-327-\tpo_start = subprog-\u003epostorder_start;\n--\nkernel/bpf/liveness.c=419=static char *fmt_subprog(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/liveness.c-420-{\nkernel/bpf/liveness.c:421:\tconst char *name = env-\u003esubprog_info[subprog].name;\nkernel/bpf/liveness.c-422-\n--\nkernel/bpf/liveness.c=489=static void print_instance(struct bpf_verifier_env *env, struct func_instance *instance)\nkernel/bpf/liveness.c-490-{\nkernel/bpf/liveness.c:491:\tint start = env-\u003esubprog_info[instance-\u003esubprog].start;\nkernel/bpf/liveness.c-492-\tstruct bpf_insn *insns = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/liveness.c=1529=static void print_subprog_arg_access(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-1534-\tstruct bpf_insn *insns = env-\u003eprog-\u003einsnsi;\nkernel/bpf/liveness.c:1535:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c-1536-\tint len = info-\u003elen;\n--\nkernel/bpf/liveness.c=1619=static int compute_subprog_args(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-1627-\tint depth = instance-\u003edepth;\nkernel/bpf/liveness.c:1628:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c:1629:\tint po_start = env-\u003esubprog_info[subprog].postorder_start;\nkernel/bpf/liveness.c:1630:\tint end = env-\u003esubprog_info[subprog + 1].start;\nkernel/bpf/liveness.c:1631:\tint po_end = env-\u003esubprog_info[subprog + 1].postorder_start;\nkernel/bpf/liveness.c-1632-\tint len = end - start;\n--\nkernel/bpf/liveness.c=1852=static int analyze_subprog(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-1860-\tstruct bpf_insn *insns = env-\u003eprog-\u003einsnsi;\nkernel/bpf/liveness.c:1861:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c:1862:\tint po_start = env-\u003esubprog_info[subprog].postorder_start;\nkernel/bpf/liveness.c:1863:\tint po_end = env-\u003esubprog_info[subprog + 1].postorder_start;\nkernel/bpf/liveness.c-1864-\tstruct func_instance *prev_instance = NULL;\n--\nkernel/bpf/verifier.c=431=void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/verifier.c-432-{\nkernel/bpf/verifier.c:433:\tstruct bpf_subprog_info *info = subprog_info(env, subprog);\nkernel/bpf/verifier.c-434-\n--\nkernel/bpf/verifier.c=440=static bool subprog_is_exc_cb(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/verifier.c-441-{\nkernel/bpf/verifier.c:442:\treturn subprog_info(env, subprog)-\u003eis_exception_cb;\nkernel/bpf/verifier.c-443-}\n--\nkernel/bpf/verifier.c=1369=static int resize_reference_state(struct bpf_verifier_state *state, size_t n)\n--\nkernel/bpf/verifier.c-1380-/* Possibly update state-\u003eallocated_stack to be at least size bytes. Also\nkernel/bpf/verifier.c:1381: * possibly update the function's high-water mark in its bpf_subprog_info.\nkernel/bpf/verifier.c-1382- */\nkernel/bpf/verifier.c=1383=static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state *state, int size)\n--\nkernel/bpf/verifier.c-1400-\t/* update known max for given subprogram */\nkernel/bpf/verifier.c:1401:\tif (env-\u003esubprog_info[state-\u003esubprogno].stack_depth \u003c size)\nkernel/bpf/verifier.c:1402:\t\tenv-\u003esubprog_info[state-\u003esubprogno].stack_depth = size;\nkernel/bpf/verifier.c-1403-\n--\nkernel/bpf/verifier.c=2352=static int cmp_subprogs(const void *a, const void *b)\nkernel/bpf/verifier.c-2353-{\nkernel/bpf/verifier.c:2354:\treturn ((struct bpf_subprog_info *)a)-\u003estart -\nkernel/bpf/verifier.c:2355:\t ((struct bpf_subprog_info *)b)-\u003estart;\nkernel/bpf/verifier.c-2356-}\n--\nkernel/bpf/verifier.c-2358-/* Find subprogram that contains instruction at 'off' */\nkernel/bpf/verifier.c:2359:struct bpf_subprog_info *bpf_find_containing_subprog(struct bpf_verifier_env *env, int off)\nkernel/bpf/verifier.c-2360-{\nkernel/bpf/verifier.c:2361:\tstruct bpf_subprog_info *vals = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2362-\tint l, r, m;\n--\nkernel/bpf/verifier.c=2380=int bpf_find_subprog(struct bpf_verifier_env *env, int off)\nkernel/bpf/verifier.c-2381-{\nkernel/bpf/verifier.c:2382:\tstruct bpf_subprog_info *p;\nkernel/bpf/verifier.c-2383-\n--\nkernel/bpf/verifier.c-2386-\t\treturn -ENOENT;\nkernel/bpf/verifier.c:2387:\treturn p - env-\u003esubprog_info;\nkernel/bpf/verifier.c-2388-}\n--\nkernel/bpf/verifier.c=2390=static int add_subprog(struct bpf_verifier_env *env, int off)\n--\nkernel/bpf/verifier.c-2406-\t/* determine subprog starts. The end is one before the next starts */\nkernel/bpf/verifier.c:2407:\tenv-\u003esubprog_info[env-\u003esubprog_cnt++].start = off;\nkernel/bpf/verifier.c:2408:\tsort(env-\u003esubprog_info, env-\u003esubprog_cnt,\nkernel/bpf/verifier.c:2409:\t sizeof(env-\u003esubprog_info[0]), cmp_subprogs, NULL);\nkernel/bpf/verifier.c-2410-\treturn env-\u003esubprog_cnt - 1;\n--\nkernel/bpf/verifier.c=2919=static int add_subprogs(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-2920-{\nkernel/bpf/verifier.c:2921:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2922-\tint i, ret, insn_cnt = env-\u003eprog-\u003elen, ex_cb_insn;\n--\nkernel/bpf/verifier.c-2968-\t\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/verifier.c:2969:\t\t\tif (env-\u003esubprog_info[i].start != ex_cb_insn)\nkernel/bpf/verifier.c-2970-\t\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=3015=static int check_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-3017-\tint i, subprog_start, subprog_end, off, cur_subprog = 0;\nkernel/bpf/verifier.c:3018:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3019-\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=3092=static int sort_subprogs_topo(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-3093-{\nkernel/bpf/verifier.c:3094:\tstruct bpf_subprog_info *si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3095-\tint *insn_postorder = env-\u003ecfg.insn_postorder;\n--\nkernel/bpf/verifier.c=3480=static void check_fastcall_stack_contract(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-3482-{\nkernel/bpf/verifier.c:3483:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-3484-\tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/verifier.c=4170=static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_state *state,\n--\nkernel/bpf/verifier.c-4173-\tint max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;\nkernel/bpf/verifier.c:4174:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4175-\tint spi = -off / BPF_REG_SIZE - 1;\n--\nkernel/bpf/verifier.c=4213=static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_state *state,\n--\n\n"
} |
| 49/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 53316 output: 37 thoughts: 35 Thoughts: Here's my analysis of the provided snippet, as if it were my thought process: **Diving into `bpf_check` and Environment Variables** Okay, so the initial clue is that `env` gets allocated within the `bpf_check` function. That's the starting point. My next step, as always, is to dig deeper. I need to understand *how* it's allocated. What's the context? What are the calling conventions? Are we talking about a stack allocation, heap, or something more specialized within the BPF environment itself? This isn't just about where it *is*; it's about the mechanism. Gotta trace the code and see the precise instantiation, likely paying special attention to memory management within the BPF VM. The goal is to fully understand its lifecycle: allocation, usage, and deallocation (or lack thereof, which could be equally important). Let's see... let's see where this takes us. |
| 50/3 | 2026/09/14 21:24 | tool | grepper |
0mError:no matches Args: {
"Expression": "kzalloc.*sizeof\\(\\*env\\)",
"PathPrefix": "kernel/bpf/verifier.c"
}Results: {
"error": "no matches"
} |
| 51/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 53606 output: 38 thoughts: 21 Thoughts: Here's a summary of my thought process, tailored for an expert like you: **Investigating `env` Allocation in the BPF Verifier** Alright, let's dive into the guts of the BPF verifier. I need to understand how the `env` variable is being allocated and managed within `kernel/bpf/verifier.c`. This is crucial for tracing the control flow, ensuring memory safety, and potentially understanding optimization opportunities within the verifier itself. I'm focusing specifically on how `env` is *initialized* and *used*. Knowing where and how it's defined and passed to other functions is the starting point. I'll be looking for clues in the surrounding functions and data structures. I'll be searching for any potential gotchas related to how `env`'s contents might evolve during verification. Are we talking about stack allocation? Heap allocation? Something more sophisticated? I expect there to be nuances worth uncovering given the sensitive nature of BPF program verification. I'll pay close attention to potential race conditions or other concurrency-related concerns if `env` is shared or accessed across threads or contexts. This might include analyzing any locks or atomic operations related to `env`. I'm going to start with a precise grep, followed by a careful reading of the surrounding code and maybe some simple debugging, to get to the bottom of this. |
| 52/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 21245,
"LineCount": 40
}Results: {
"Contents": "21245:\t\t/* user could have requested verbose verifier output\n21246:\t\t * and supplied buffer to store the verification trace\n21247:\t\t */\n21248:\t\tret = bpf_vlog_init(\u0026env-\u003elog, attr_log-\u003elevel, attr_log-\u003eubuf, attr_log-\u003esize);\n21249:\t\tif (ret)\n21250:\t\t\tgoto err_free_env;\n21251:\t\tret = bpf_diag_init(env);\n21252:\t\tif (ret)\n21253:\t\t\tgoto err_prep;\n21254:\t\tif (env-\u003esignature) {\n21255:\t\t\tret = bpf_prog_calc_tag(env-\u003eprog);\n21256:\t\t\tif (ret \u003c 0)\n21257:\t\t\t\tgoto err_prep;\n21258:\t\t}\n21259:\t\n21260:\t\tret = process_fd_array(env, attr, uattr);\n21261:\t\tif (ret)\n21262:\t\t\tgoto err_prep;\n21263:\t\n21264:\t\tif (env-\u003esignature) {\n21265:\t\t\tret = bpf_prog_verify_signature(env, attr, uattr.is_kernel);\n21266:\t\t\tif (ret)\n21267:\t\t\t\tgoto err_prep;\n21268:\t\t}\n21269:\t\n21270:\t\tret = security_bpf_prog_load(env-\u003eprog, attr, env-\u003eprog-\u003eaux-\u003etoken,\n21271:\t\t\t\t\t uattr.is_kernel);\n21272:\t\tif (ret)\n21273:\t\t\tgoto err_prep;\n21274:\t\n21275:\t\tbpf_get_btf_vmlinux();\n21276:\t\n21277:\t\t/* Serialize verification of unprivileged programs. */\n21278:\t\tif (!is_priv)\n21279:\t\t\tmutex_lock(\u0026bpf_verifier_lock);\n21280:\t\n21281:\t\tlen = env-\u003einsn_aux_data_len = env-\u003eprog-\u003elen;\n21282:\t\tenv-\u003einsn_aux_data =\n21283:\t\t\t__vmalloc(array_size(sizeof(struct bpf_insn_aux_data), len),\n21284:\t\t\t\t GFP_KERNEL_ACCOUNT | __GFP_ZERO);\n"
} |
| 53/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 54606 output: 38 thoughts: 33 Thoughts: Here's my take on those thoughts, tailored for an expert audience: **BPF Verification: Diving into `env` Allocation** Okay, so I need to understand how the `env` variable is being handled within the `bpf_check` function. Time to drill down into the details. I'm going to jump straight to lines 21200 through 21245. This should give me a solid view of how `env` is allocated, initialized, and potentially modified within the verifier's context. I'm looking for the nitty-gritty details here – where the space comes from, any constraints on its use, and any potential for interactions with other BPF state. Understanding this is key to verifying the overall security and integrity of the BPF program I'm analyzing. Let's get to it. |
| 54/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 21210,
"LineCount": 35
}Results: {
"Contents": "21210:\t\treturn err;\n21211:\t}\n21212:\t\n21213:\tint bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n21214:\t\t struct bpf_log_attr *attr_log)\n21215:\t{\n21216:\t\tu64 start_time = ktime_get_ns();\n21217:\t\tstruct bpf_verifier_env *env;\n21218:\t\tint i, len, ret = -EINVAL, err;\n21219:\t\tbool is_priv;\n21220:\t\n21221:\t\tBTF_TYPE_EMIT(enum bpf_features);\n21222:\t\n21223:\t\t/* no program is valid */\n21224:\t\tif (ARRAY_SIZE(bpf_verifier_ops) == 0)\n21225:\t\t\treturn -EINVAL;\n21226:\t\n21227:\t\t/* 'struct bpf_verifier_env' can be global, but since it's not small,\n21228:\t\t * allocate/free it every time bpf_check() is called\n21229:\t\t */\n21230:\t\tenv = kvzalloc_obj(struct bpf_verifier_env, GFP_KERNEL_ACCOUNT);\n21231:\t\tif (!env)\n21232:\t\t\treturn -ENOMEM;\n21233:\t\n21234:\t\tenv-\u003ebt.env = env;\n21235:\t\tenv-\u003eprog = *prog;\n21236:\t\tenv-\u003eops = bpf_verifier_ops[env-\u003eprog-\u003etype];\n21237:\t\n21238:\t\tenv-\u003eallow_ptr_leaks = bpf_allow_ptr_leaks(env-\u003eprog-\u003eaux-\u003etoken);\n21239:\t\tenv-\u003eallow_uninit_stack = bpf_allow_uninit_stack(env-\u003eprog-\u003eaux-\u003etoken);\n21240:\t\tenv-\u003ebypass_spec_v1 = bpf_bypass_spec_v1(env-\u003eprog-\u003eaux-\u003etoken);\n21241:\t\tenv-\u003ebypass_spec_v4 = bpf_bypass_spec_v4(env-\u003eprog-\u003eaux-\u003etoken);\n21242:\t\tenv-\u003ebpf_capable = is_priv = bpf_token_capable(env-\u003eprog-\u003eaux-\u003etoken, CAP_BPF);\n21243:\t\tenv-\u003esignature = attr-\u003esignature;\n21244:\t\n"
} |
| 55/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 55530 output: 38 thoughts: 618 Thoughts: **My Analysis of BPF Verifier Environment Memory Management** Okay, let's break down this `bpf_verifier_env` and how its resources are managed. I see that the `env` is initialized using `kvzalloc_obj`, which is critical. This means both `env->subprog_info` and `env->cfg` are zero-initialized from the start. That's good, predictable behavior. Looking at the `bpf_verifier_env` and `bpf_subprog_info` structures, I'm specifically focused on these additions: the `jt` field within `bpf_subprog_info`, which is a `bpf_iarray`, and the `gotox_edges` and `subprog_jts_ready` fields in the `cfg` structure of `bpf_verifier_env`. Since it's zero-initialized, I know `gotox_edges` starts at 0, `subprog_jts_ready` is `false`, and all the `subprog_info[i].jt` pointers are `NULL` initially. That's a fundamental understanding. The `bpf_free_subprog_jts` function is crucial for cleaning up. It iterates through each `subprog_info[i].jt`, frees it using `kvfree`, and sets the pointer to `NULL`. It also sets `env->cfg.subprog_jts_ready` to `false`. That's the expected clean-up mechanism. Now, the important part: where is `bpf_free_subprog_jts` called? I see it's called at the end of `bpf_check`. But here is where I get concerned - it's called *after* releasing maps and btfs, *and* after freeing aux data, fd array and stack liveness info. I'm wondering if it's consistently called on *every* exit path from `bpf_check` – successful or error paths. I need to review the logic around lines 21500-21550 carefully to be certain. It is critical that this is freed on any return condition. |
| 56/3 | 2026/09/14 21:24 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 21500,
"LineCount": 45
}Results: {
"Contents": "21500:\t\tadjust_btf_func(env);\n21501:\t\n21502:\t\t/* extension progs temporarily inherit the attach_type of their targets\n21503:\t\t for verification purposes, so set it back to zero before returning\n21504:\t\t */\n21505:\t\tif (env-\u003eprog-\u003etype == BPF_PROG_TYPE_EXT)\n21506:\t\t\tenv-\u003eprog-\u003eexpected_attach_type = 0;\n21507:\t\n21508:\t\tenv-\u003eprog = __bpf_prog_select_runtime(env, env-\u003eprog, \u0026ret);\n21509:\t\n21510:\terr_release_maps:\n21511:\t\tif (ret)\n21512:\t\t\trelease_insn_arrays(env);\n21513:\t\tif (!env-\u003eprog-\u003eaux-\u003eused_maps)\n21514:\t\t\t/* if we didn't copy map pointers into bpf_prog_info, release\n21515:\t\t\t * them now. Otherwise free_used_maps() will release them.\n21516:\t\t\t */\n21517:\t\t\trelease_maps(env);\n21518:\t\tif (!env-\u003eprog-\u003eaux-\u003eused_btfs)\n21519:\t\t\trelease_btfs(env);\n21520:\t\n21521:\t\t*prog = env-\u003eprog;\n21522:\t\n21523:\t\tmodule_put(env-\u003eattach_btf_mod);\n21524:\t\tif (!is_priv)\n21525:\t\t\tmutex_unlock(\u0026bpf_verifier_lock);\n21526:\t\tgoto err_free_env;\n21527:\terr_prep:\n21528:\t\terr = bpf_log_attr_finalize(attr_log, \u0026env-\u003elog);\n21529:\t\tif (err)\n21530:\t\t\tret = err;\n21531:\t\trelease_insn_arrays(env);\n21532:\t\trelease_maps(env);\n21533:\t\trelease_btfs(env);\n21534:\terr_free_env:\n21535:\t\tbpf_free_subprog_jts(env);\n21536:\t\tvfree(env-\u003einsn_aux_data);\n21537:\t\tkvfree(env-\u003efd_array);\n21538:\t\tbpf_stack_liveness_free(env);\n21539:\t\tkvfree(env-\u003ecfg.insn_postorder);\n21540:\t\tkvfree(env-\u003escc_info);\n21541:\t\tkvfree(env-\u003esucc);\n21542:\t\tkvfree(env-\u003egotox_tmp_buf);\n21543:\t\tbpf_diag_free(env);\n21544:\t\tkvfree(env);\n"
} |
| 57/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 57417 output: 42 thoughts: 509 Thoughts: **My Analysis of Potential Memory Exposure**
Okay, so I've been reviewing this patch, specifically focusing on potential memory exposure vulnerabilities. I see that both the success and failure paths in this part of the code converge at `err_free_env:`, which calls `bpf_free_subprog_jts(env)`. After that, there's a standard cleanup, freeing the structures and calling `kvfree(env)`. This is good, a pretty standard and safe practice.
My primary concern now is to identify if this patch series introduces any new areas where memory is *copied* to userspace, or where uninitialized memory could be inadvertently exposed. I need to be sure this refactor doesn't introduce a new attack vector.
Let's see, I'll go through the code step-by-step. First, I'm checking `bpf_insn_array.c`. Alright, here's `bpf_insn_array_init`:
```c
int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
{
struct bpf_insn_array *insn_array = cast_insn_array(map);
struct bpf_insn_array_value *values = insn_array->values;
int i;
if (atomic_cmpxchg(&insn_array->used, 0, 1))
return -EBUSY;
/*
* Reset the map to its pre-verification state. The xlated and jitted
* offsets and the jitted target pointers are recomputed by the verifier
* and the JIT for this program, so any values left by a previous owner
* must be cleared here.
*/
for (i = 0; i < map->max_entries; i++) {
values[i].xlated_off = values[i].orig_off;
values[i].jitted_off = 0;
insn_array->ips[i] = 0;
}
return 0;
}
```
Okay, good. I see the loop that iterates through the `values` array. Excellent! This *explicitly* resets `jitted_off` and `ips[i]` to zero *before* any potentially unsafe operation occurs. This is exactly what I was hoping to see; this proactively clears any stale values that might have been left over from a previous program or execution. That reduces a significant attack surface.
Now, let's step back and consider the broader context of the patch. I need to quickly scan and verify that there aren't any other areas using this new code and exposing these variables. I'll search for `copy_to_user`, `put_user`, any netlink attributes, socket options, or ioctl calls that were modified by this patch.
I will also check how `bpf_map_lookup_elem` deals with a `bpf_insn_array`, checking `kernel/bpf/bpf_insn_array.c`, to be sure of how the `values` are looked up or copied to the user.
|
| 58/3 | 2026/09/14 21:24 | tool | grepper |
0mArgs:{
"Expression": "bpf_insn_array_",
"PathPrefix": "kernel/bpf/bpf_insn_array.c"
}Results: {
"Output": "kernel/bpf/bpf_insn_array.c=6=struct bpf_insn_array {\n--\nkernel/bpf/bpf_insn_array.c-9-\tlong *ips;\nkernel/bpf/bpf_insn_array.c:10:\tDECLARE_FLEX_ARRAY(struct bpf_insn_array_value, values);\nkernel/bpf/bpf_insn_array.c-11-};\n--\nkernel/bpf/bpf_insn_array.c=18=static inline u64 insn_array_alloc_size(u32 max_entries)\n--\nkernel/bpf/bpf_insn_array.c-20-\tconst u64 base_size = sizeof(struct bpf_insn_array);\nkernel/bpf/bpf_insn_array.c:21:\tconst u64 entry_size = sizeof(struct bpf_insn_array_value);\nkernel/bpf/bpf_insn_array.c-22-\n--\nkernel/bpf/bpf_insn_array.c=26=static int insn_array_alloc_check(union bpf_attr *attr)\nkernel/bpf/bpf_insn_array.c-27-{\nkernel/bpf/bpf_insn_array.c:28:\tu32 value_size = sizeof(struct bpf_insn_array_value);\nkernel/bpf/bpf_insn_array.c-29-\n--\nkernel/bpf/bpf_insn_array.c=75=static long insn_array_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags)\n--\nkernel/bpf/bpf_insn_array.c-78-\tu32 index = *(u32 *)key;\nkernel/bpf/bpf_insn_array.c:79:\tstruct bpf_insn_array_value val = {};\nkernel/bpf/bpf_insn_array.c-80-\n--\nkernel/bpf/bpf_insn_array.c=162=static inline bool valid_offsets(const struct bpf_insn_array *insn_array,\n--\nkernel/bpf/bpf_insn_array.c-182-\nkernel/bpf/bpf_insn_array.c:183:int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)\nkernel/bpf/bpf_insn_array.c-184-{\nkernel/bpf/bpf_insn_array.c-185-\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\nkernel/bpf/bpf_insn_array.c:186:\tstruct bpf_insn_array_value *values = insn_array-\u003evalues;\nkernel/bpf/bpf_insn_array.c-187-\tint i;\n--\nkernel/bpf/bpf_insn_array.c-215-\nkernel/bpf/bpf_insn_array.c:216:int bpf_insn_array_ready(struct bpf_map *map)\nkernel/bpf/bpf_insn_array.c-217-{\n--\nkernel/bpf/bpf_insn_array.c-230-\nkernel/bpf/bpf_insn_array.c:231:void bpf_insn_array_release(struct bpf_map *map)\nkernel/bpf/bpf_insn_array.c-232-{\n--\nkernel/bpf/bpf_insn_array.c-237-\nkernel/bpf/bpf_insn_array.c:238:void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)\nkernel/bpf/bpf_insn_array.c-239-{\n--\nkernel/bpf/bpf_insn_array.c-254-\nkernel/bpf/bpf_insn_array.c:255:void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len)\nkernel/bpf/bpf_insn_array.c-256-{\n"
} |
| 59/3 | 2026/09/14 21:24 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 59787 output: 41 thoughts: 28 Thoughts: Here's my take on the thought process, as if it were my own: **Diving into Instruction Array Management** Okay, so I need to get a handle on how this instruction array, specifically how the updates are handled in `insn_array_update_elem`. Let me start by tracing back through the code to understand the context. I want to see how this `insn_array` itself is constructed initially, and then how these array elements are accessed and looked up. Understanding the lifecycle from creation to modification is key. Gotta make sure I fully grasp the intricacies of this data structure before I move on to anything else. Let's see what the code reveals! |
| 60/3 | 2026/09/14 21:25 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/bpf_insn_array.c",
"FirstLine": 30,
"LineCount": 60
}Results: {
"Contents": " 30:\t\tif (attr-\u003emax_entries == 0 || attr-\u003ekey_size != 4 ||\n 31:\t\t attr-\u003evalue_size != value_size || attr-\u003emap_flags != 0)\n 32:\t\t\treturn -EINVAL;\n 33:\t\n 34:\t\treturn 0;\n 35:\t}\n 36:\t\n 37:\tstatic void insn_array_free(struct bpf_map *map)\n 38:\t{\n 39:\t\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\n 40:\t\n 41:\t\tbpf_map_area_free(insn_array);\n 42:\t}\n 43:\t\n 44:\tstatic struct bpf_map *insn_array_alloc(union bpf_attr *attr)\n 45:\t{\n 46:\t\tu64 size = insn_array_alloc_size(attr-\u003emax_entries);\n 47:\t\tstruct bpf_insn_array *insn_array;\n 48:\t\n 49:\t\tinsn_array = bpf_map_area_alloc(size, NUMA_NO_NODE);\n 50:\t\tif (!insn_array)\n 51:\t\t\treturn ERR_PTR(-ENOMEM);\n 52:\t\n 53:\t\t/* ips are allocated right after the insn_array-\u003evalues[] array */\n 54:\t\tinsn_array-\u003eips = (void *)\u0026insn_array-\u003evalues[attr-\u003emax_entries];\n 55:\t\n 56:\t\tbpf_map_init_from_attr(\u0026insn_array-\u003emap, attr);\n 57:\t\n 58:\t\t/* BPF programs aren't allowed to write to the map */\n 59:\t\tinsn_array-\u003emap.map_flags |= BPF_F_RDONLY_PROG;\n 60:\t\n 61:\t\treturn \u0026insn_array-\u003emap;\n 62:\t}\n 63:\t\n 64:\tstatic void *insn_array_lookup_elem(struct bpf_map *map, void *key)\n 65:\t{\n 66:\t\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\n 67:\t\tu32 index = *(u32 *)key;\n 68:\t\n 69:\t\tif (unlikely(index \u003e= insn_array-\u003emap.max_entries))\n 70:\t\t\treturn NULL;\n 71:\t\n 72:\t\treturn \u0026insn_array-\u003evalues[index];\n 73:\t}\n 74:\t\n 75:\tstatic long insn_array_update_elem(struct bpf_map *map, void *key, void *value, u64 map_flags)\n 76:\t{\n 77:\t\tstruct bpf_insn_array *insn_array = cast_insn_array(map);\n 78:\t\tu32 index = *(u32 *)key;\n 79:\t\tstruct bpf_insn_array_value val = {};\n 80:\t\n 81:\t\tif (unlikely(index \u003e= insn_array-\u003emap.max_entries))\n 82:\t\t\treturn -E2BIG;\n 83:\t\n 84:\t\tif (unlikely(map_flags \u0026 BPF_NOEXIST))\n 85:\t\t\treturn -EEXIST;\n 86:\t\n 87:\t\tcopy_map_value(map, \u0026val, value);\n 88:\t\tif (val.jitted_off || val.xlated_off)\n 89:\t\t\treturn -EINVAL;\n"
} |
| 61/3 | 2026/09/14 21:25 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 60932 output: 26 thoughts: 996 Thoughts: **Analyzing Jump Table Handling in BPF Verification** Okay, so I'm looking at how jump tables are handled within the BPF verifier, specifically `compute_subprog_jts`. My initial thought is to trace the memory allocation and deallocation to ensure no leaks or double-frees, and identify potential vulnerabilities. The code uses `kvmalloc` with `__GFP_ZERO` for map areas, and explicitly clears some fields, which is good. Now, focusing on `compute_subprog_jts`, I see it iterates through instruction array maps. The key is in how it manages jump tables associated with subprograms. It uses `bpf_iarray_realloc` to resize the jump table. The critical part is checking for errors during jump table reallocation. If it encounters an error like `-EINVAL` or `-ENOMEM`, the function returns early. Importantly, any jump tables (`subprog->jt`) *already* allocated before the error *remain* allocated in `env->subprog_info[i].jt`. My concern immediately is how these are handled in case of error. This leads me to review the error handling paths. It turns out that `bpf_check` calls `bpf_free_subprog_jts(env)` in both `err_prep:` and `err_free_env:`, which cleans up any allocated jump tables, iterating the subprogram info to free anything non-null. So, that should handle most cases. I then went through a hypothetical case, I was worried that `bpf_iarray_realloc` could return NULL. The code sets `subprog->jt = NULL;` before returning the error, and the previous `bpf_iarray_realloc` call already freed the old pointer, so no double free is possible. Now, a subtle bug. Let's suppose `subprog` is the *last* subprogram in `env->subprog_info` when `bpf_find_containing_subprog` is called. The check `(subprog + 1)->start` looks problematic, potentially reading uninitialized memory. I need to understand how `env->subprog_info` is initialized, what `(subprog + 1)->start` represents and how `bpf_find_containing_subprog` works. I'm going to jump into those functions, and how `add_subprogs` works to get the whole picture. |
| 62/3 | 2026/09/14 21:25 | tool | grepper |
0mArgs:{
"Expression": "subprog_info\\[env-\u003esubprog_cnt\\]"
}Results: {
"Output": "kernel/bpf/fixups.c=1063=static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)\n--\nkernel/bpf/fixups.c-1067-\t/* restore the start of fake 'exit' subprog as well */\nkernel/bpf/fixups.c:1068:\tenv-\u003esubprog_info[env-\u003esubprog_cnt].start = env-\u003eprog-\u003elen;\nkernel/bpf/fixups.c-1069-}\n"
} |
| 63/3 | 2026/09/14 21:25 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 62621 output: 37 thoughts: 15 Thoughts: Okay, let's dive into this. Here's how I'd approach summarizing the prompt from an expert's perspective: **Investigating `bpf_find_containing_subprog`** Right, let's take a look at `bpf_find_containing_subprog`. This is a core function, so I need to understand its role within the broader BPF context. My initial instinct is to consider how it's used – what kind of data structures does it operate on? What's its typical input? Clearly, it's about identifying a sub-program within a larger BPF program. I'm immediately thinking about the implications for debugging and performance analysis. If this function is part of a dynamic loading or JIT compilation process, it's *essential*. It would be how we locate the exact section of BPF code responsible for executing a specific instruction or responding to a particular event. If I were tracing a bug, this would be a likely place to start looking. I'm wondering if this is a linear search? If so, is there a caching mechanism to improve lookup speed? Or, is there an architectural structure to the program that can be leveraged to accelerate the search? Knowing this impacts the program's efficiency greatly. What algorithms are employed within the function, and how are they implemented? I'll probably want to mentally trace a few scenarios – simple programs, complex nested calls, programs with jumps and loops – to get a grasp on its behavior in different conditions. From there, I can get a better handle on the overhead involved. |
| 64/3 | 2026/09/14 21:25 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 2355,
"LineCount": 40
}Results: {
"Contents": "2355:\t\t ((struct bpf_subprog_info *)b)-\u003estart;\n2356:\t}\n2357:\t\n2358:\t/* Find subprogram that contains instruction at 'off' */\n2359:\tstruct bpf_subprog_info *bpf_find_containing_subprog(struct bpf_verifier_env *env, int off)\n2360:\t{\n2361:\t\tstruct bpf_subprog_info *vals = env-\u003esubprog_info;\n2362:\t\tint l, r, m;\n2363:\t\n2364:\t\tif (off \u003e= env-\u003eprog-\u003elen || off \u003c 0 || env-\u003esubprog_cnt == 0)\n2365:\t\t\treturn NULL;\n2366:\t\n2367:\t\tl = 0;\n2368:\t\tr = env-\u003esubprog_cnt - 1;\n2369:\t\twhile (l \u003c r) {\n2370:\t\t\tm = l + (r - l + 1) / 2;\n2371:\t\t\tif (vals[m].start \u003c= off)\n2372:\t\t\t\tl = m;\n2373:\t\t\telse\n2374:\t\t\t\tr = m - 1;\n2375:\t\t}\n2376:\t\treturn \u0026vals[l];\n2377:\t}\n2378:\t\n2379:\t/* Find subprogram that starts exactly at 'off' */\n2380:\tint bpf_find_subprog(struct bpf_verifier_env *env, int off)\n2381:\t{\n2382:\t\tstruct bpf_subprog_info *p;\n2383:\t\n2384:\t\tp = bpf_find_containing_subprog(env, off);\n2385:\t\tif (!p || p-\u003estart != off)\n2386:\t\t\treturn -ENOENT;\n2387:\t\treturn p - env-\u003esubprog_info;\n2388:\t}\n2389:\t\n2390:\tstatic int add_subprog(struct bpf_verifier_env *env, int off)\n2391:\t{\n2392:\t\tint insn_cnt = env-\u003eprog-\u003elen;\n2393:\t\tint ret;\n2394:\t\n"
} |
| 65/3 | 2026/09/14 21:25 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 63604 output: 37 thoughts: 30 Thoughts: Here's the summary, assuming I'm an expert: **Deep Dive into Subprogram Initialization** Okay, so I'm looking at how we're setting up the `env->subprog_info` array, specifically at the element indexed by `env->subprog_cnt`. The crucial question that's nagging at me is: *how* is this particular element initialized? I need to trace this back to the source. Let's see... the most likely candidate for initializing `env->subprog_info[env->subprog_cnt]` is the `add_subprogs` function. Time to crack open that function definition and see what's going on within it. I want to see how that function sets up the data and makes sure everything is as I'd expect it to be. |
| 66/3 | 2026/09/14 21:25 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 2920,
"LineCount": 55
}Results: {
"Contents": "2920:\t{\n2921:\t\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\n2922:\t\tint i, ret, insn_cnt = env-\u003eprog-\u003elen, ex_cb_insn;\n2923:\t\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n2924:\t\tconst char *operation, *suggestion;\n2925:\t\n2926:\t\t/* Add entry function. */\n2927:\t\tret = add_subprog(env, 0);\n2928:\t\tif (ret)\n2929:\t\t\treturn ret;\n2930:\t\n2931:\t\tfor (i = 0; i \u003c insn_cnt; i++, insn++) {\n2932:\t\t\tif (!bpf_pseudo_func(insn) \u0026\u0026 !bpf_pseudo_call(insn))\n2933:\t\t\t\tcontinue;\n2934:\t\n2935:\t\t\tif (!env-\u003ebpf_capable) {\n2936:\t\t\t\tif (bpf_pseudo_func(insn)) {\n2937:\t\t\t\t\toperation = \"BPF function reference\";\n2938:\t\t\t\t\tsuggestion = \"Load this program with the required capability, or avoid BPF function references in unprivileged programs.\";\n2939:\t\t\t\t} else {\n2940:\t\t\t\t\toperation = \"BPF-to-BPF function call\";\n2941:\t\t\t\t\tsuggestion = \"Load this program with the required capability, or avoid BPF-to-BPF function calls in unprivileged programs.\";\n2942:\t\t\t\t}\n2943:\t\t\t\tverbose(env, \"loading/calling other bpf or kernel functions are allowed for CAP_BPF and CAP_SYS_ADMIN\\n\");\n2944:\t\t\t\tbpf_diag_policy(\n2945:\t\t\t\t\tenv, i, operation,\n2946:\t\t\t\t\t\"loading or calling other BPF functions requires CAP_BPF or CAP_SYS_ADMIN\",\n2947:\t\t\t\t\tsuggestion);\n2948:\t\t\t\treturn -EPERM;\n2949:\t\t\t}\n2950:\t\n2951:\t\t\tret = add_subprog(env, i + insn-\u003eimm + 1);\n2952:\t\t\tif (ret \u003c 0)\n2953:\t\t\t\treturn ret;\n2954:\t\t}\n2955:\t\n2956:\t\tret = bpf_find_exception_callback_insn_off(env);\n2957:\t\tif (ret \u003c 0)\n2958:\t\t\treturn ret;\n2959:\t\tex_cb_insn = ret;\n2960:\t\n2961:\t\t/* If ex_cb_insn \u003e 0, this means that the main program has a subprog\n2962:\t\t * marked using BTF decl tag to serve as the exception callback.\n2963:\t\t */\n2964:\t\tif (ex_cb_insn) {\n2965:\t\t\tret = add_subprog(env, ex_cb_insn);\n2966:\t\t\tif (ret \u003c 0)\n2967:\t\t\t\treturn ret;\n2968:\t\t\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\n2969:\t\t\t\tif (env-\u003esubprog_info[i].start != ex_cb_insn)\n2970:\t\t\t\t\tcontinue;\n2971:\t\t\t\tenv-\u003eexception_callback_subprog = i;\n2972:\t\t\t\tbpf_mark_subprog_exc_cb(env, i);\n2973:\t\t\t\tbreak;\n2974:\t\t\t}\n"
} |
| 67/3 | 2026/09/14 21:25 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 64761 output: 37 thoughts: 14 Thoughts: Okay, here's my attempt at summarizing the thought process, as requested, in the first person and targeted towards an expert: **Delving into `add_subprogs`** Alright, let's dive into the rest of this `add_subprogs` function. I'm already familiar with the overall structure, but now I want to get a finer-grained understanding of exactly how this module is handling the integration of these subprograms. I'm particularly interested in how it interacts with the broader context, what decisions are being made based on the subprogram type, and if there are any subtle optimization techniques at play that might be worth revisiting later. I'll be looking for potential areas of concern, such as thread safety or race conditions, or anywhere where this could be a point of future vulnerability to an exploit. I'm expecting some kind of conditional logic here, probably a `match` or `if/else` block, to handle different subprogram signatures. I'm keen to see the error handling too, want to know what level of logging is being done, and how it deals with the situation when a subprogram fails to load or integrate properly. Okay, let's see. |
| 68/3 | 2026/09/14 21:25 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 2975,
"LineCount": 40
}Results: {
"Contents": "2975:\t\t}\n2976:\t\n2977:\t\t/* Add a fake 'exit' subprog which could simplify subprog iteration\n2978:\t\t * logic. 'subprog_cnt' should not be increased.\n2979:\t\t */\n2980:\t\tsubprog[env-\u003esubprog_cnt].start = insn_cnt;\n2981:\t\n2982:\t\tif (env-\u003elog.level \u0026 BPF_LOG_LEVEL2)\n2983:\t\t\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++)\n2984:\t\t\t\tverbose(env, \"func#%d @%d\\n\", i, subprog[i].start);\n2985:\t\n2986:\t\treturn 0;\n2987:\t}\n2988:\t\n2989:\tstatic int add_kfuncs(struct bpf_verifier_env *env)\n2990:\t{\n2991:\t\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n2992:\t\tint i, ret, insn_cnt = env-\u003eprog-\u003elen;\n2993:\t\n2994:\t\tfor (i = 0; i \u003c insn_cnt; i++, insn++) {\n2995:\t\t\tif (!bpf_pseudo_kfunc_call(insn))\n2996:\t\t\t\tcontinue;\n2997:\t\n2998:\t\t\tif (!env-\u003ebpf_capable) {\n2999:\t\t\t\tverbose(env, \"loading/calling other bpf or kernel functions are allowed for CAP_BPF and CAP_SYS_ADMIN\\n\");\n3000:\t\t\t\tbpf_diag_policy(\n3001:\t\t\t\t\tenv, i, \"kernel function call\",\n3002:\t\t\t\t\t\"calling kernel functions requires CAP_BPF or CAP_SYS_ADMIN\",\n3003:\t\t\t\t\t\"Load this program with the required capability, or avoid kernel function calls in unprivileged programs.\");\n3004:\t\t\t\treturn -EPERM;\n3005:\t\t\t}\n3006:\t\n3007:\t\t\tret = bpf_add_kfunc_call(env, insn-\u003eimm, insn-\u003eoff);\n3008:\t\t\tif (ret \u003c 0)\n3009:\t\t\t\treturn ret;\n3010:\t\t}\n3011:\t\n3012:\t\treturn 0;\n3013:\t}\n3014:\t\n"
} |
| 69/3 | 2026/09/14 21:25 | llm | kmsan-evaluator |
1mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 65694 output: 389 thoughts: 1632 Thoughts: **Analysis of BPF Jump Table Refactoring and KMSAN Justification** Okay, so looking at this code, my primary focus is on how it interacts with memory safety, particularly in relation to KMSAN (Kernel Memory Sanitizer). I see a critical change on line 2980: `subprog[env->subprog_cnt].start = insn_cnt;`. This initialization confirms that `(subprog + 1)->start` is always initialized, even for the final subprogram, which points to the "exit" subprogram. The array's size is also defined as `BPF_MAX_SUBPROGS + 2`, which makes sense. Next, I examined `visit_gotox_insn`. The code now uses `subprog_jt(t, env)` and crucially adds checks for overflow in `gotox_edges` and enforces the `BPF_MAX_GOTOX_EDGES` limit. This is a very important addition from a security standpoint. Now, let's address the KMSAN criteria. Firstly, there's nothing here that's introducing or modifying data that's being sent to user space in an uninitialized manner. In fact, `bpf_insn_array_init` specifically clears any stale data, which is good. Secondly, I'm confident that the logic and branching don't depend on uninitialized variables. Variables like `dfs_pos`, `subprog_info`, and others are all zero-initialized using `kvcalloc` or `kvzalloc_obj`. Thirdly, when it comes to allocation and initialization of data structures, `dfs_pos` is allocated and initialized correctly. `env` also uses `kvzalloc_obj`. The critical `subprog->jt` is allocated by `bpf_iarray_realloc`, which explicitly copies existing initialized data and updates the count. Lastly, there are strict bounds checks. Limits are placed on indirect jump edges, and it checks that `gotox` targets stay within their respective subprograms. These were all previously missed. The bugs this patch addresses are quite extensive: We have a fix for logic bugs and stale data in the `insn` array maps, jump table allocations, DFS recursion and SCC computations, and NOP instruction removal. Importantly, none of these directly relate to uninitialized memory issues. All the allocations appear fully initialized or are immediately written to. Any out-of-bounds access or use-after-free risk would be caught by KASAN (Kernel Address Sanitizer). KMSAN's purpose here is to detect uninitialized memory reads and kernel info leaks to user space. Neither is present. Now, let's confirm this against the KMSAN criteria again: - The changes primarily address array overflows, and introduce bounds checks to prevent out-of-bounds access and stale data, which are all KASAN issues. - All stack and heap structures are initialized before reading or copying, using zeroing techniques. - The patch does not introduce any risk of uninitialized memory usage or data leaks to user space. This is a clear case for `NeedsKMSAN=false`. In summary, this patch is a refactoring and tightening of BPF jump table validation. The code refactors gotox verification to be per-subprogram rather than per-instruction, enforces limits on the number of indirect jump edges, ensures indirect jump targets are within their containing subprogram, tracks DFS successor positions in SCC computation, fixes clearing of stale entries, and handles indirect jump target flags after instructions are prepended or NOP'd. Memory safety is assured by KASAN and not by KMSAN. `dfs_pos` is allocated via `kvcalloc`, `env` via `kvzalloc_obj`, and any array element populations are handled correctly by `bpf_copy_insn_array_uniq` and other safety routines. There are no uninitialized heap/stack memory reads, no uninitialized struct fields in control flow, and no user-space info leaks. So, KMSAN is not required. |
| 70/3 | 2026/09/14 21:26 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch refactors BPF indirect jump (gotox) handling and jump table verification in the BPF verifier:\n1. It consolidates jump tables to be per-subprogram (`subprog-\u003ejt`) rather than per-instruction in `insn_aux_data`, cleans up their lifecycle in `bpf_free_subprog_jts()`, and prevents gotox instructions from jumping outside their containing subprogram.\n2. In `bpf_compute_scc()`, it tracks the successor traversal resume position with `dfs_pos`, which is allocated using `kvcalloc` (zero-initialized) and freed with `kvfree`.\n3. In `bpf_insn_array.c`, `bpf_insn_array_init()` explicitly zeroes `jitted_off` and `ips[i]` to clear stale values on reuse.\n4. In `fixups.c`, it distinguishes prepend vs append instruction patching for adjusting indirect jump target flags and avoids removing NOPs if they are indirect jump targets.\n\nAll newly allocated buffers and modified structures (such as `dfs_pos`, `env-\u003ecfg`, and `env-\u003esubprog_info`) are allocated using zeroing allocators (`kvcalloc`, `kvzalloc_obj`), explicitly cleared, or initialized prior to use. Any potential bugs involving buffer bounds (e.g. `succ-\u003eitems`, `jt-\u003eitems`, or `subprog_info` indices) or pointer lifecycles would be out-of-bounds accesses or use-after-free bugs detected by standard KASAN. There is no risk of uninitialized memory reads or information disclosure to userspace, so a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch refactors BPF indirect jump (gotox) handling and jump table verification in the BPF verifier:\n1. It consolidates jump tables to be per-subprogram (`subprog-\u003ejt`) rather than per-instruction in `insn_aux_data`, cleans up their lifecycle in `bpf_free_subprog_jts()`, and prevents gotox instructions from jumping outside their containing subprogram.\n2. In `bpf_compute_scc()`, it tracks the successor traversal resume position with `dfs_pos`, which is allocated using `kvcalloc` (zero-initialized) and freed with `kvfree`.\n3. In `bpf_insn_array.c`, `bpf_insn_array_init()` explicitly zeroes `jitted_off` and `ips[i]` to clear stale values on reuse.\n4. In `fixups.c`, it distinguishes prepend vs append instruction patching for adjusting indirect jump target flags and avoids removing NOPs if they are indirect jump targets.\n\nAll newly allocated buffers and modified structures (such as `dfs_pos`, `env-\u003ecfg`, and `env-\u003esubprog_info`) are allocated using zeroing allocators (`kvcalloc`, `kvzalloc_obj`), explicitly cleared, or initialized prior to use. Any potential bugs involving buffer bounds (e.g. `succ-\u003eitems`, `jt-\u003eitems`, or `subprog_info` indices) or pointer lifecycles would be out-of-bounds accesses or use-after-free bugs detected by standard KASAN. There is no risk of uninitialized memory reads or information disclosure to userspace, so a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|