AI Trajectory

Trajectory: (Expand/Collapse All)
Seq	Timestamp	Type	Name	Duration
0/0	2026/09/09 21:17	flow	patch-triage	1m Args: null Results: { "EnableConfigs": [ "BPF_SYSCALL" ], "FocusSymbols": [ "bpf_check_cfg", "bpf_compute_scc", "check_indirect_jump", "compute_subprog_jts", "create_jt", "visit_gotox_insn", "reject_gotox_out_of_subprog", "reject_gotox_without_cfg_edge", "jt_contains" ], "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.", "NeedsKMSAN": false, "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.", "WorthFuzzing": true }
1/1	2026/09/09 21:17	action	read-patch-diff	0m Args: null Results: { "PatchDiff": "commit e16eef8d5994f14c9cd79ed40d9a24d4c4001f83\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Wed Sep 9 21:17:23 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h\nindex 36b65797877d0..baf2e17d7019b 100644\n--- a/include/linux/bpf_verifier.h\n+++ b/include/linux/bpf_verifier.h\n@@ -805,6 +805,7 @@ struct bpf_subprog_info {\n \tu32 linfo_idx; /* The idx to the main_prog-\u003eaux-\u003elinfo /\n \tu32 postorder_start; / The idx to the env-\u003ecfg.insn_postorder /\n \tu32 exit_idx; / Index of one of the BPF_EXIT instructions in this subprogram /\n+\tstruct bpf_iarray jt; /* jump table shared by all gotox of this subprogram /\n \tu16 stack_depth; / max. stack depth used by this function /\n \tu16 stack_extra;\n \tu32 insns_total;\n@@ -825,6 +826,7 @@ struct bpf_subprog_info {\n \tbool keep_fastcall_stack: 1;\n \tbool changes_pkt_data: 1;\n \tbool might_sleep: 1;\n+\tbool jt_spans_subprogs: 1;\n \tu8 arg_cnt:4;\n \n \tenum priv_stack_mode priv_stack_mode;\n@@ -977,6 +979,8 @@ struct bpf_verifier_env {\n \t\tint cur_stack;\n \t\t/ current position in the insn_postorder vector /\n \t\tint cur_postorder;\n+\t\tu32 gotox_edges;\n+\t\tbool subprog_jts_ready;\n \t} cfg;\n \tstruct backtrack_state bt;\n \tstruct bpf_jmp_history_entry cur_hist_ent;\ndiff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c\nindex 842c7d1eabccc..879587af8d086 100644\n--- a/kernel/bpf/cfg.c\n+++ b/kernel/bpf/cfg.c\n@@ -9,6 +9,8 @@\n \n #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)\n \n+#define BPF_MAX_GOTOX_EDGES\tBPF_COMPLEXITY_LIMIT_INSNS\n+\n /* non-recursive DFS pseudo code\n * 1 procedure DFS-iterative(G,v):\n * 2 label v as discovered\n@@ -284,15 +286,17 @@ static struct bpf_iarray jt_from_map(struct bpf_map map)\n }\n \n /\n- Find and collect all maps which fit in the subprog. Return the result as one\n- * combined jump table in jt-\u003eitems (allocated with kvcalloc)\n+ * Collect the jump table of every subprogram that has one, as the combined\n+ * table of all maps whose targets land inside that subprogram. All gotox\n+ * instructions of a subprogram share the same table, so this is done in a\n+ * single pass over the maps rather than once per gotox.\n /\n-static struct bpf_iarray jt_from_subprog(struct bpf_verifier_env env,\n-\t\t\t\t\t int subprog_start, int subprog_end)\n+static int compute_subprog_jts(struct bpf_verifier_env env)\n {\n-\tstruct bpf_iarray jt = NULL;\n+\tstruct bpf_subprog_info subprog;\n+\tstruct bpf_iarray jt, jt_cur;\n \tstruct bpf_map map;\n-\tstruct bpf_iarray jt_cur;\n+\tu32 old_cnt;\n \tint i;\n \n \tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++) {\n@@ -303,72 +307,97 @@ static struct bpf_iarray jt_from_subprog(struct bpf_verifier_env env,\n \t\tmap = env-\u003einsn_array_maps[i];\n \n \t\tjt_cur = jt_from_map(map);\n-\t\tif (IS_ERR(jt_cur)) {\n-\t\t\tkvfree(jt);\n-\t\t\treturn jt_cur;\n+\t\tif (IS_ERR(jt_cur))\n+\t\t\treturn PTR_ERR(jt_cur);\n+\n+\t\tsubprog = bpf_find_containing_subprog(env, jt_cur-\u003eitems[0]);\n+\t\tif (!subprog) {\n+\t\t\tkvfree(jt_cur);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (jt_cur-\u003eitems[jt_cur-\u003ecnt - 1] \u003e= (subprog + 1)-\u003estart) {\n+\t\t\tsubprog-\u003ejt_spans_subprogs = true;\n+\t\t\tkvfree(jt_cur);\n+\t\t\tcontinue;\n \t\t}\n \n-\t\t/\n-\t\t This is enough to check one element. The full table is\n-\t\t * checked to fit inside the subprog later in create_jt()\n-\t\t /\n-\t\tif (jt_cur-\u003eitems[0] \u003e= subprog_start \u0026\u0026 jt_cur-\u003eitems[0] \u003c subprog_end) {\n-\t\t\tu32 old_cnt = jt ? jt-\u003ecnt : 0;\n-\t\t\tjt = bpf_iarray_realloc(jt, old_cnt + jt_cur-\u003ecnt);\n-\t\t\tif (!jt) {\n-\t\t\t\tkvfree(jt_cur);\n-\t\t\t\treturn ERR_PTR(-ENOMEM);\n-\t\t\t}\n-\t\t\tmemcpy(jt-\u003eitems + old_cnt, jt_cur-\u003eitems, jt_cur-\u003ecnt \u003c\u003c 2);\n+\t\told_cnt = subprog-\u003ejt ? subprog-\u003ejt-\u003ecnt : 0;\n+\t\tjt = bpf_iarray_realloc(subprog-\u003ejt, old_cnt + jt_cur-\u003ecnt);\n+\t\tif (!jt) {\n+\t\t\tsubprog-\u003ejt = NULL;\n+\t\t\tkvfree(jt_cur);\n+\t\t\treturn -ENOMEM;\n \t\t}\n+\t\tmemcpy(jt-\u003eitems + old_cnt, jt_cur-\u003eitems, jt_cur-\u003ecnt \u003c\u003c 2);\n+\t\tsubprog-\u003ejt = jt;\n \n \t\tkvfree(jt_cur);\n \t}\n \n-\tif (!jt) {\n-\t\tverbose(env, \"no jump tables found for subprog starting at %u\\n\", subprog_start);\n-\t\tbpf_diag_program_structure(\n-\t\t\tenv, subprog_start, \"missing jump table\",\n-\t\t\t\"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.\",\n-\t\t\t\"No jump table was found for the subprogram that starts at instruction %u.\",\n-\t\t\tsubprog_start);\n-\t\treturn ERR_PTR(-EINVAL);\n+\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\n+\t\tjt = env-\u003esubprog_info[i].jt;\n+\t\tif (jt)\n+\t\t\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\n \t}\n \n-\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\n-\treturn jt;\n+\tenv-\u003ecfg.subprog_jts_ready = true;\n+\treturn 0;\n+}\n+\n+static void free_subprog_jts(struct bpf_verifier_env env)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(env-\u003esubprog_info); i++) {\n+\t\tkvfree(env-\u003esubprog_info[i].jt);\n+\t\tenv-\u003esubprog_info[i].jt = NULL;\n+\t\tenv-\u003esubprog_info[i].jt_spans_subprogs = false;\n+\t}\n+\tenv-\u003ecfg.subprog_jts_ready = false;\n }\n \n static struct bpf_iarray \n create_jt(int t, struct bpf_verifier_env env)\n {\n \tstruct bpf_subprog_info subprog;\n-\tint subprog_start, subprog_end;\n \tstruct bpf_iarray jt;\n-\tint i;\n+\tint subprog_start, err;\n+\n+\tif (!env-\u003ecfg.subprog_jts_ready) {\n+\t\terr = compute_subprog_jts(env);\n+\t\tif (err)\n+\t\t\treturn ERR_PTR(err);\n+\t}\n \n \tsubprog = bpf_find_containing_subprog(env, t);\n \tsubprog_start = subprog-\u003estart;\n-\tsubprog_end = (subprog + 1)-\u003estart;\n-\tjt = jt_from_subprog(env, subprog_start, subprog_end);\n-\tif (IS_ERR(jt))\n-\t\treturn jt;\n \n-\t/* Check that the every element of the jump table fits within the given subprogram /\n-\tfor (i = 0; i \u003c jt-\u003ecnt; i++) {\n-\t\tif (jt-\u003eitems[i] \u003c subprog_start \|\| jt-\u003eitems[i] \u003e= subprog_end) {\n-\t\t\tverbose(env, \"jump table for insn %d points outside of the subprog [%u,%u]\\n\",\n-\t\t\t\t\tt, subprog_start, subprog_end);\n-\t\t\tbpf_diag_program_structure(\n-\t\t\t\tenv, t, \"jump table target out of range\",\n-\t\t\t\t\"Keep every jump-table target inside the same subprogram.\",\n-\t\t\t\t\"The jump table for instruction %d points outside subprogram range [%u,%u).\",\n-\t\t\t\tt, subprog_start, subprog_end);\n-\t\t\tkvfree(jt);\n-\t\t\treturn ERR_PTR(-EINVAL);\n-\t\t}\n+\tif (subprog-\u003ejt_spans_subprogs) {\n+\t\tverbose(env, \"jump table of subprog starting at %u spans multiple subprogs\\n\",\n+\t\t\tsubprog_start);\n+\t\tbpf_diag_program_structure(\n+\t\t\tenv, subprog_start, \"jump table spans subprograms\",\n+\t\t\t\"Keep every entry of a jump table inside one subprogram.\",\n+\t\t\t\"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.\",\n+\t\t\tsubprog_start, (subprog + 1)-\u003estart);\n+\t\treturn ERR_PTR(-EINVAL);\n+\t}\n+\n+\tif (!subprog-\u003ejt) {\n+\t\tverbose(env, \"no jump tables found for subprog starting at %u\\n\", subprog_start);\n+\t\tbpf_diag_program_structure(\n+\t\t\tenv, subprog_start, \"missing jump table\",\n+\t\t\t\"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.\",\n+\t\t\t\"No jump table was found for the subprogram that starts at instruction %u.\",\n+\t\t\tsubprog_start);\n+\t\treturn ERR_PTR(-EINVAL);\n \t}\n \n+\tjt = bpf_iarray_realloc(NULL, subprog-\u003ejt-\u003ecnt);\n+\tif (!jt)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\tmemcpy(jt-\u003eitems, subprog-\u003ejt-\u003eitems, subprog-\u003ejt-\u003ecnt \u003c\u003c 2);\n+\n \treturn jt;\n }\n \n@@ -388,6 +417,19 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env env)\n \t\t\treturn PTR_ERR(jt);\n \n \t\tenv-\u003einsn_aux_data[t].jt = jt;\n+\n+\t\tif (check_add_overflow(env-\u003ecfg.gotox_edges, jt-\u003ecnt,\n+\t\t\t\t \u0026env-\u003ecfg.gotox_edges) \|\|\n+\t\t env-\u003ecfg.gotox_edges \u003e BPF_MAX_GOTOX_EDGES) {\n+\t\t\tverbose(env, \"number of indirect jump edges in the program exceeds %u\\n\",\n+\t\t\t\tBPF_MAX_GOTOX_EDGES);\n+\t\t\tbpf_diag_program_structure(\n+\t\t\t\tenv, t, \"too many indirect jump edges\",\n+\t\t\t\t\"Reduce the number of indirect jumps, or the number of distinct targets they can reach.\",\n+\t\t\t\t\"The program has more than %u indirect jump edges in total, counted over every gotox instruction.\",\n+\t\t\t\tBPF_MAX_GOTOX_EDGES);\n+\t\t\treturn -E2BIG;\n+\t\t}\n \t}\n \n \tmark_prune_point(env, t);\n@@ -678,6 +720,7 @@ int bpf_check_cfg(struct bpf_verifier_env env)\n \tenv-\u003eprog-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[0].might_sleep;\n \n err_free:\n+\tfree_subprog_jts(env);\n \tkvfree(insn_state);\n \tkvfree(insn_stack);\n \tenv-\u003ecfg.insn_state = env-\u003ecfg.insn_stack = NULL;\n@@ -749,7 +792,7 @@ int bpf_compute_scc(struct bpf_verifier_env env)\n \tstruct bpf_insn_aux_data aux = env-\u003einsn_aux_data;\n \tconst u32 insn_cnt = env-\u003eprog-\u003elen;\n \tint stack_sz, dfs_sz, err = 0;\n-\tu32 stack, pre, low, dfs;\n+\tu32 stack, pre, low, dfs, dfs_pos;\n \tu32 i, j, t, w;\n \tu32 next_preorder_num;\n \tu32 next_scc_id;\n@@ -762,13 +805,16 @@ int bpf_compute_scc(struct bpf_verifier_env env)\n \t - 'stack' accumulates vertices in DFS order, see invariant comment below;\n \t * - 'pre[t] == p' =\u003e preorder number of vertex 't' is 'p';\n \t * - 'low[t] == n' =\u003e smallest preorder number of the vertex reachable from 't' is 'n';\n-\t * - 'dfs' DFS traversal stack, used to emulate explicit recursion.\n+\t * - 'dfs' DFS traversal stack, used to emulate explicit recursion;\n+\t * - 'dfs_pos[k] == j' =\u003e the frame 'dfs[k]' resumes visiting its\n+\t * successors at index 'j'.\n \t /\n \tstack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tpre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tlow = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tdfs = kvcalloc(insn_cnt, sizeof(dfs), GFP_KERNEL_ACCOUNT);\n-\tif (!stack \|\| !pre \|\| !low \|\| !dfs) {\n+\tdfs_pos = kvcalloc(insn_cnt, sizeof(dfs_pos), GFP_KERNEL_ACCOUNT);\n+\tif (!stack \|\| !pre \|\| !low \|\| !dfs \|\| !dfs_pos) {\n \t\terr = -ENOMEM;\n \t\tgoto exit;\n \t}\n@@ -851,6 +897,7 @@ int bpf_compute_scc(struct bpf_verifier_env env)\n \t\tstack_sz = 0;\n \t\tdfs_sz = 1;\n \t\tdfs[0] = i;\n+\t\tdfs_pos[0] = 0;\n dfs_continue:\n \t\twhile (dfs_sz) {\n \t\t\tw = dfs[dfs_sz - 1];\n@@ -860,13 +907,37 @@ int bpf_compute_scc(struct bpf_verifier_env env)\n \t\t\t\tnext_preorder_num++;\n \t\t\t\tstack[stack_sz++] = w;\n \t\t\t}\n-\t\t\t/ Visit 'w' successors /\n+\t\t\t/\n+\t\t\t * Visit 'w' successors, resuming at the successor this\n+\t\t\t * frame last descended into. Restarting the scan at zero\n+\t\t\t * on every return to 'w' would examine each successor\n+\t\t\t * once per descent, i.e. quadratic in the number of\n+\t\t\t * successors, which for a gotox is the size of the jump\n+\t\t\t * table.\n+\t\t\t \n+\t\t\t Re-folding the successors before that index would be a\n+\t\t\t * no-op. Such a successor 's' has 'pre[s] != 0' by then,\n+\t\t\t * so it is never pushed onto 'dfs' again, and low[s] can\n+\t\t\t * only decrease while 's' is the top of 'dfs'. If 's' is\n+\t\t\t * still on 'dfs' it sits below 'w' and cannot become the\n+\t\t\t * top before 'w' is popped; otherwise the only remaining\n+\t\t\t * write to low[s] is the pop of its SCC, setting it to\n+\t\t\t * NOT_ON_STACK, for which the min below is a no-op.\n+\t\t\t /\n \t\t\tsucc = bpf_insn_successors(env, w);\n-\t\t\tfor (j = 0; j \u003c succ-\u003ecnt; ++j) {\n+\t\t\tfor (j = dfs_pos[dfs_sz - 1]; j \u003c succ-\u003ecnt; ++j) {\n \t\t\t\tif (pre[succ-\u003eitems[j]]) {\n \t\t\t\t\tlow[w] = min(low[w], low[succ-\u003eitems[j]]);\n \t\t\t\t} else {\n-\t\t\t\t\tdfs[dfs_sz++] = succ-\u003eitems[j];\n+\t\t\t\t\t/\n+\t\t\t\t\t * Resume at 'j', not 'j + 1': the successor\n+\t\t\t\t\t * is revisited once its DFS completes, to\n+\t\t\t\t\t * fold its low[] into low[w].\n+\t\t\t\t\t /\n+\t\t\t\t\tdfs_pos[dfs_sz - 1] = j;\n+\t\t\t\t\tdfs_pos[dfs_sz] = 0;\n+\t\t\t\t\tdfs[dfs_sz] = succ-\u003eitems[j];\n+\t\t\t\t\tdfs_sz++;\n \t\t\t\t\tgoto dfs_continue;\n \t\t\t\t}\n \t\t\t}\n@@ -916,5 +987,6 @@ int bpf_compute_scc(struct bpf_verifier_env env)\n \tkvfree(pre);\n \tkvfree(low);\n \tkvfree(dfs);\n+\tkvfree(dfs_pos);\n \treturn err;\n }\ndiff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c\nindex 72a3f5998dd27..45234e2fbee64 100644\n--- a/kernel/bpf/verifier.c\n+++ b/kernel/bpf/verifier.c\n@@ -18165,11 +18165,56 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env env,\n \treturn 0;\n }\n \n+/ 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() /\n+static bool jt_contains(const struct bpf_iarray jt, u32 target)\n+{\n+\tint l = 0, r = jt-\u003ecnt - 1, m;\n+\n+\twhile (l \u003c= r) {\n+\t\tm = l + (r - l) / 2;\n+\t\tif (jt-\u003eitems[m] == target)\n+\t\t\treturn true;\n+\t\tif (jt-\u003eitems[m] \u003c target)\n+\t\t\tl = m + 1;\n+\t\telse\n+\t\t\tr = m - 1;\n+\t}\n+\treturn false;\n+}\n+\n+static int reject_gotox_out_of_subprog(struct bpf_verifier_env env, u32 target,\n+\t\t\t\t u32 subprog_start, u32 subprog_end)\n+{\n+\tverbose(env, \"indirect jump from insn %d to %u leaves the subprog [%u,%u)\\n\",\n+\t\t env-\u003einsn_idx, target, subprog_start, subprog_end);\n+\tbpf_diag_program_structure(\n+\t\tenv, env-\u003einsn_idx, \"indirect jump leaves subprogram\",\n+\t\t\"Keep every reachable jump-table target inside the subprogram of the indirect jump.\",\n+\t\t\"Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).\",\n+\t\tenv-\u003einsn_idx, target, subprog_start, subprog_end);\n+\treturn -EINVAL;\n+}\n+\n+static int reject_gotox_without_cfg_edge(struct bpf_verifier_env env, u32 target)\n+{\n+\tverbose(env, \"indirect jump from insn %d to %u is not in the jump table of the subprog\\n\",\n+\t\t env-\u003einsn_idx, target);\n+\tbpf_diag_program_structure(\n+\t\tenv, env-\u003einsn_idx, \"indirect jump target without CFG edge\",\n+\t\t\"Resolve indirect jumps through a jump table whose entries all fall inside the subprogram of the jump.\",\n+\t\t\"Instruction %d can jump indirectly to instruction %u, which is not part of the jump table of its subprogram.\",\n+\t\tenv-\u003einsn_idx, target);\n+\treturn -EINVAL;\n+}\n+\n /* gotox dst_reg /\n static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn insn)\n {\n \tstruct bpf_verifier_state other_branch;\n+\tstruct bpf_subprog_info subprog;\n+\tu32 subprog_start, subprog_end;\n \tstruct bpf_reg_state dst_reg;\n+\tstruct bpf_iarray jt;\n \tstruct bpf_map map;\n \tu32 min_index, max_index;\n \tint err = 0;\n@@ -18212,6 +18257,26 @@ static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn in\n \t\treturn -EINVAL;\n \t}\n \n+\tsubprog = bpf_find_containing_subprog(env, env-\u003einsn_idx);\n+\tif (verifier_bug_if(!subprog, env, \"no subprog contains insn %d\", env-\u003einsn_idx))\n+\t\treturn -EFAULT;\n+\tsubprog_start = subprog-\u003estart;\n+\tsubprog_end = (subprog + 1)-\u003estart;\n+\n+\tjt = env-\u003einsn_aux_data[env-\u003einsn_idx].jt;\n+\tif (verifier_bug_if(!jt, env, \"no jump table for insn %d\", env-\u003einsn_idx))\n+\t\treturn -EFAULT;\n+\n+\tfor (i = 0; i \u003c n; i++) {\n+\t\tu32 target = env-\u003egotox_tmp_buf-\u003eitems[i];\n+\n+\t\tif (target \u003c subprog_start \|\| target \u003e= subprog_end)\n+\t\t\treturn reject_gotox_out_of_subprog(env, target, subprog_start,\n+\t\t\t\t\t\t\t subprog_end);\n+\t\tif (!jt_contains(jt, target))\n+\t\t\treturn reject_gotox_without_cfg_edge(env, target);\n+\t}\n+\n \tfor (i = 0; i \u003c n - 1; i++) {\n \t\tmark_indirect_target(env, env-\u003egotox_tmp_buf-\u003eitems[i]);\n \t\tother_branch = push_stack(env, env-\u003egotox_tmp_buf-\u003eitems[i],\ndiff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\nindex 0222a9a5d0761..d5a831a75d820 100644\n--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\n+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\n@@ -1,6 +1,7 @@\n // SPDX-License-Identifier: GPL-2.0\n \n #include \u003cbpf/bpf.h\u003e\n+#include \u003cbpf/btf.h\u003e\n #include \u003ctest_progs.h\u003e\n \n #if defined(__x86_64__) \|\| defined(__powerpc__) \|\| defined(__aarch64__)\n@@ -453,6 +454,940 @@ static void check_bpf_no_lookup(void)\n \tclose(map_fd);\n }\n \n+#define GOTOX_CNT_AT_LIMIT\t1000\n+#define GOTOX_LOG_SZ\t\t(256 1024)\n+\n+static const char gotox_limit_msg[] =\n+\t\"number of indirect jump edges in the program exceeds\";\n+\n+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)\n+{\n+\t/* the run of gotox itself, plus the exit block right after it /\n+\tconst __u32 jt_cnt = gotox_cnt + 1;\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c jt_cnt; i++) {\n+\t\tval.orig_off = first_gotox + i;\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+static int gotox_prog_load_funcs(struct bpf_insn insns, __u32 insn_cnt,\n+\t\t\t\t int fd_array, __u32 fd_array_cnt, char log,\n+\t\t\t\t int btf_fd, struct bpf_func_info fi, __u32 fi_cnt)\n+{\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tint prog_fd;\n+\n+\tlog[0] = 0;\n+\topts.fd_array = fd_array;\n+\topts.fd_array_cnt = fd_array_cnt;\n+\topts.log_buf = log;\n+\topts.log_size = GOTOX_LOG_SZ;\n+\topts.log_level = 1;\n+\tif (fi_cnt) {\n+\t\topts.prog_btf_fd = btf_fd;\n+\t\topts.func_info = fi;\n+\t\topts.func_info_cnt = fi_cnt;\n+\t\topts.func_info_rec_size = sizeof(fi);\n+\t}\n+\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", insns, insn_cnt, \u0026opts);\n+\tif (prog_fd \u003e= 0) {\n+\t\tclose(prog_fd);\n+\t\treturn 0;\n+\t}\n+\treturn prog_fd;\n+}\n+\n+static int gotox_prog_load(struct bpf_insn insns, __u32 insn_cnt,\n+\t\t\t int fd_array, __u32 fd_array_cnt, char log)\n+{\n+\treturn gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,\n+\t\t\t\t -1, NULL, 0);\n+}\n+\n+/ Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. /\n+static void gotox_run_fill(struct bpf_insn insns, __u32 gotox_cnt)\n+{\n+\t__u32 i;\n+\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);\n+\tfor (i = 1; i \u003c= gotox_cnt; i++)\n+\t\tinsns[i] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+}\n+\n+static void check_gotox_limit_hit(const char log, int err)\n+{\n+\tASSERT_EQ(err, -E2BIG, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, gotox_limit_msg, \"verifier log\");\n+}\n+\n+static bool try_load_gotox_prog(__u32 gotox_cnt, char log, int err)\n+{\n+\tconst __u32 insn_cnt = gotox_cnt + 3;\n+\tstruct bpf_insn insns;\n+\tbool attempted = false;\n+\tint map_fd;\n+\n+\tinsns = calloc(insn_cnt, sizeof(insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\treturn false;\n+\n+\tgotox_run_fill(insns, gotox_cnt);\n+\tinsns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tmap_fd = gotox_jt_create(1, gotox_cnt);\n+\tif (map_fd \u003c 0)\n+\t\tgoto free_insns;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, \u0026map_fd, 1, log);\n+\tclose(map_fd);\n+\tattempted = true;\n+free_insns:\n+\tfree(insns);\n+\treturn attempted;\n+}\n+\n+/\n+ The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt\n+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.\n+ /\n+static void check_too_many_gotox_edges(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tif (try_load_gotox_prog(gotox_cnt, log, \u0026err))\n+\t\tcheck_gotox_limit_hit(log, err);\n+\n+\tfree(log);\n+}\n+\n+/\n+ A chain of blocks, where block k loads jt[k] and jumps to it. The jump\n+ * table holds the starts of the blocks that follow plus the exit block,\n+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits\n+ * exactly at the limit and must still load.\n+ /\n+#define GOTOX_BLOCK_SZ\t\t4\n+\n+static void gotox_chain_fill(struct bpf_insn insns, __u32 gotox_cnt)\n+{\n+\tstruct bpf_insn at;\n+\t__u32 k;\n+\n+\tfor (k = 0; k \u003c gotox_cnt; k++) {\n+\t\tat = insns + k GOTOX_BLOCK_SZ;\n+\n+\t\t/* r1 = \u0026jt[0], by index 0 into fd_array /\n+\t\tat[0] = (struct bpf_insn) {\n+\t\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t\t.dst_reg = BPF_REG_1,\n+\t\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t\t.imm = 0,\n+\t\t};\n+\t\tat[1] = (struct bpf_insn) { .imm = 0 };\n+\t\tat[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k 8);\n+\t\tat[3] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+\t}\n+\n+\tinsns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();\n+}\n+\n+static int gotox_chain_jt_create(__u32 gotox_cnt)\n+{\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c gotox_cnt; i++) {\n+\t\tval.orig_off = (i + 1) * GOTOX_BLOCK_SZ;\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+static void check_gotox_edges_at_limit(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;\n+\tconst __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;\n+\tstruct bpf_insn insns;\n+\tchar log;\n+\tint map_fd, err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tinsns = calloc(insn_cnt, sizeof(insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\tgoto free_log;\n+\n+\tgotox_chain_fill(insns, gotox_cnt);\n+\n+\tmap_fd = gotox_chain_jt_create(gotox_cnt);\n+\tif (map_fd \u003c 0)\n+\t\tgoto free_insns;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, \u0026map_fd, 1, log);\n+\tclose(map_fd);\n+\n+\tif (!ASSERT_OK(err, \"program at the edge limit should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+free_insns:\n+\tfree(insns);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_edges_across_subprogs(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT 3 / 4;\n+\tconst __u32 sub_start = gotox_cnt + 3;\n+\tconst __u32 insn_cnt = 2 * (gotox_cnt + 3);\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_insn insns;\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tinsns = calloc(insn_cnt, sizeof(insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\tgoto free_log;\n+\n+\tgotox_run_fill(insns, gotox_cnt);\n+\tinsns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0,\n+\t\t\t\t\t BPF_PSEUDO_CALL, 0,\n+\t\t\t\t\t sub_start - (gotox_cnt + 1) - 1);\n+\tinsns[gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tgotox_run_fill(insns + sub_start, gotox_cnt);\n+\tinsns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tmap_fd[0] = gotox_jt_create(1, gotox_cnt);\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_insns;\n+\tmap_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);\n+\tcheck_gotox_limit_hit(log, err);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_insns:\n+\tfree(insns);\n+free_log:\n+\tfree(log);\n+}\n+\n+static int gotox_jt_create_offs(const __u32 offs, __u32 cnt)\n+{\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c cnt; i++) {\n+\t\tval.orig_off = offs[i];\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+#define GOTOX_SUB_START\t\t4\n+#define GOTOX_MAIN_TGT\t\t2\n+#define GOTOX_SUB_TGT\t\t8\n+#define GOTOX_TWO_INSN_CNT\t10\n+\n+static void gotox_two_subprogs_fill(struct bpf_insn insns, __u32 jt_idx, __u32 jt_off)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_SUB_START - 1 - 1);\n+\tinsns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_EXIT_INSN();\n+\n+\t/ r1 = \u0026jt[0], by index 'jt_idx' into fd_array /\n+\tinsns[GOTOX_SUB_START] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = jt_idx,\n+\t};\n+\tinsns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off 8);\n+\tinsns[7] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[9] = BPF_EXIT_INSN();\n+}\n+\n+/\n+ An insn_array map is not necessarily a jump table: one that tracks\n+ * instruction offsets covers the whole program and is of no subprog. Such a\n+ * map must not keep a program with a gotox elsewhere from loading.\n+ /\n+static void check_gotox_tracker_map(void)\n+{\n+\tconst __u32 jt_track[] = { 0, GOTOX_MAIN_TGT, GOTOX_SUB_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 1, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_track, ARRAY_SIZE(jt_track));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tif (!ASSERT_OK(err, \"program with a tracking map should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_target_other_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 7 to 2 leaves the subprog [4,10)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_jt_per_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 1, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, 0, \"bpf(BPF_PROG_LOAD)\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/\n+ The spanning map is of no subprog and is dropped, and the entry the gotox\n+ * register can reach is in the subprog of the gotox and in the jump table the\n+ * CFG walked, so nothing unsafe is left and the program loads.\n+ /\n+static void check_gotox_span_unreached_entry(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_MAIN_TGT, GOTOX_SUB_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 1);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tif (!ASSERT_OK(err, \"program with an unreachable spanning entry should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_FWD_GOTOX\t\t11\n+#define GOTOX_FWD_OWN_TGT\t12\n+#define GOTOX_FWD_SUB_START\t14\n+#define GOTOX_FWD_INSN_CNT\t16\n+\n+static void gotox_from_main_fill(struct bpf_insn insns)\n+{\n+\tinsns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_FWD_SUB_START - 1 - 1);\n+\tinsns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,\n+\t\t\t offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);\n+\n+\t/ r1 = \u0026jt_leaves[0], by index 1 into fd_array /\n+\tinsns[4] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 1,\n+\t};\n+\tinsns[5] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\tinsns[7] = BPF_JMP_A(3);\n+\n+\t/ r1 = \u0026jt_own[0], by index 0 into fd_array /\n+\tinsns[8] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[9] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\n+\tinsns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[13] = BPF_EXIT_INSN();\n+\tinsns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[15] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_target_subprog_from_main(void)\n+{\n+\tconst __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };\n+\tconst __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 11 to 14 leaves the subprog [0,14)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/\n+ The only map of the subprog holding the gotox reaches past that subprog, so\n+ * the subprog is left without a jump table at all.\n+ /\n+static void check_gotox_jt_spans_subprogs(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };\n+\tconst __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"jump table of subprog starting at 0 spans multiple subprogs\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/\n+ The subprog holding the gotox has a well formed jump table of its own and\n+ * also collects a map that reaches past its end. The spanning map is still\n+ * rejected, even though the subprog is not left without a table.\n+ /\n+static void check_gotox_jt_spans_with_own_table(void)\n+{\n+\tconst __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };\n+\tconst __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"jump table of subprog starting at 0 spans multiple subprogs\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_EDGE_MAIN_TGT\t2\n+#define GOTOX_EDGE_SUB_START\t4\n+#define GOTOX_EDGE_GOTOX\t9\n+#define GOTOX_EDGE_BR_TGT\t10\n+#define GOTOX_EDGE_JT_TGT\t11\n+#define GOTOX_EDGE_INSN_CNT\t12\n+\n+static void gotox_no_edge_fill(struct bpf_insn insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_EDGE_SUB_START - 1 - 1);\n+\tinsns[GOTOX_EDGE_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_EXIT_INSN();\n+\n+\tinsns[GOTOX_EDGE_SUB_START] =\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,\n+\t\t\t offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[5] = BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 4);\n+\n+\t/ r1 = \u0026jt_span[0], by index 0 into fd_array /\n+\tinsns[6] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[7] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[8] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 8);\n+\n+\tinsns[GOTOX_EDGE_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_EDGE_BR_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[GOTOX_EDGE_JT_TGT] = BPF_EXIT_INSN();\n+}\n+\n+/\n+ * The gotox resolves a target inside its own subprog, but out of a map that\n+ * spans subprogs and is therefore of no subprog. The CFG never walked that\n+ * edge, so the jump has to be rejected even though it stays in the subprog.\n+ /\n+static void check_gotox_target_without_cfg_edge(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_EDGE_MAIN_TGT, GOTOX_EDGE_BR_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_EDGE_JT_TGT };\n+\tstruct bpf_insn insns[GOTOX_EDGE_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_no_edge_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log,\n+\t\t\t \"indirect jump from insn 9 to 10 is not in the jump table of the subprog\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_SLICE_SUB_START\t6\n+#define GOTOX_SLICE_GOTOX\t14\n+#define GOTOX_SLICE_SUB_TGT\t15\n+#define GOTOX_SLICE_INSN_CNT\t17\n+\n+static void gotox_slice_fill(struct bpf_insn insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_SLICE_SUB_START - 1 - 1);\n+\tinsns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[5] = BPF_EXIT_INSN();\n+\n+\tinsns[GOTOX_SLICE_SUB_START] =\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,\n+\t\t\t offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);\n+\tinsns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);\n+\tinsns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);\n+\n+\t/ r1 = \u0026jt_main[0], by index 0 into fd_array /\n+\tinsns[10] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[11] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);\n+\tinsns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\n+\tinsns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[16] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_index_slice_other_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { 2, 3, 4 };\n+\tconst __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_SLICE_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_slice_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 14 to 3 leaves the subprog [6,17)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static int gotox_btf_create(const __u32 starts, const __u8 linkage, __u32 cnt,\n+\t\t\t struct bpf_func_info fi, struct btf pbtf)\n+{\n+\tint int_id, proto_id, id;\n+\tstruct btf btf;\n+\tchar name[24];\n+\t__u32 i;\n+\n+\tbtf = btf__new_empty();\n+\tif (!ASSERT_OK_PTR(btf, \"btf__new_empty\"))\n+\t\treturn -1;\n+\n+\tint_id = btf__add_int(btf, \"int\", 4, BTF_INT_SIGNED);\n+\tif (!ASSERT_GT(int_id, 0, \"btf__add_int\"))\n+\t\tgoto err;\n+\n+\tproto_id = btf__add_func_proto(btf, int_id);\n+\tif (!ASSERT_GT(proto_id, 0, \"btf__add_func_proto\"))\n+\t\tgoto err;\n+\n+\tfor (i = 0; i \u003c cnt; i++) {\n+\t\tsnprintf(name, sizeof(name), \"gotox_f%u\", i);\n+\t\tid = btf__add_func(btf, name, linkage[i], proto_id);\n+\t\tif (!ASSERT_GT(id, 0, \"btf__add_func\"))\n+\t\t\tgoto err;\n+\t\tfi[i].insn_off = starts[i];\n+\t\tfi[i].type_id = id;\n+\t}\n+\n+\tif (!ASSERT_OK(btf__load_into_kernel(btf), \"btf__load_into_kernel\"))\n+\t\tgoto err;\n+\n+\tpbtf = btf;\n+\treturn btf__fd(btf);\n+err:\n+\tbtf__free(btf);\n+\treturn -1;\n+}\n+\n+static void check_gotox_target_other_global_subprog(void)\n+{\n+\tconst __u32 starts[] = { 0, GOTOX_SUB_START };\n+\tconst __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_func_info fi[2];\n+\tstruct btf btf = NULL;\n+\tint btf_fd;\n+\tchar log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 0);\n+\n+\tbtf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, \u0026btf);\n+\tif (btf_fd \u003c 0)\n+\t\tgoto free_log;\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_btf;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,\n+\t\t\t\t btf_fd, fi, ARRAY_SIZE(fi));\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 7 to 2 leaves the subprog [4,10)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_btf:\n+\tbtf__free(btf);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_CB_MAIN_TGT\t6\n+#define GOTOX_CB_START\t\t8\n+#define GOTOX_CB_GOTOX\t\t11\n+#define GOTOX_CB_TGT\t\t12\n+#define GOTOX_CB_INSN_CNT\t14\n+\n+static void gotox_callback_fill(struct bpf_insn insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);\n+\t/* r2 = \u0026callback /\n+\tinsns[1] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_2,\n+\t\t.src_reg = BPF_PSEUDO_FUNC,\n+\t\t.imm = GOTOX_CB_START - 1 - 1,\n+\t};\n+\tinsns[2] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);\n+\tinsns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);\n+\tinsns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);\n+\tinsns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[7] = BPF_EXIT_INSN();\n+\n+\t/ r1 = \u0026jt_main[0], by index 0 into fd_array /\n+\tinsns[GOTOX_CB_START] = (struct bpf_insn) {\n+\t\t.code = BPF_LD \| BPF_DW \| BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[9] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\tinsns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[13] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_callback_leaves_subprog(void)\n+{\n+\tconst __u32 starts[] = { 0, GOTOX_CB_START };\n+\tconst __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };\n+\tconst __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };\n+\tconst __u32 jt_cb[] = { GOTOX_CB_TGT };\n+\tstruct bpf_insn insns[GOTOX_CB_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_func_info fi[2];\n+\tstruct btf btf = NULL;\n+\tint btf_fd;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_callback_fill(insns);\n+\n+\tbtf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, \u0026btf);\n+\tif (btf_fd \u003c 0)\n+\t\tgoto free_log;\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_btf;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,\n+\t\t\t\t btf_fd, fi, ARRAY_SIZE(fi));\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 11 to 6 leaves the subprog [8,14)\",\n+\t\t\t \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_btf:\n+\tbtf__free(btf);\n+free_log:\n+\tfree(log);\n+}\n+\n static void check_bpf_side(void)\n {\n \tcheck_bpf_no_lookup();\n@@ -490,6 +1425,48 @@ static void __test_bpf_insn_array(void)\n \n \tif (test__start_subtest(\"bpf-side-ops\"))\n \t\tcheck_bpf_side();\n+\n+\tif (test__start_subtest(\"too-many-gotox-edges\"))\n+\t\tcheck_too_many_gotox_edges();\n+\n+\tif (test__start_subtest(\"gotox-edges-at-limit\"))\n+\t\tcheck_gotox_edges_at_limit();\n+\n+\tif (test__start_subtest(\"gotox-edges-across-subprogs\"))\n+\t\tcheck_gotox_edges_across_subprogs();\n+\n+\tif (test__start_subtest(\"gotox-tracker-map\"))\n+\t\tcheck_gotox_tracker_map();\n+\n+\tif (test__start_subtest(\"gotox-jt-spans-subprogs\"))\n+\t\tcheck_gotox_jt_spans_subprogs();\n+\n+\tif (test__start_subtest(\"gotox-jt-spans-with-own-table\"))\n+\t\tcheck_gotox_jt_spans_with_own_table();\n+\n+\tif (test__start_subtest(\"gotox-target-without-cfg-edge\"))\n+\t\tcheck_gotox_target_without_cfg_edge();\n+\n+\tif (test__start_subtest(\"gotox-target-other-subprog\"))\n+\t\tcheck_gotox_target_other_subprog();\n+\n+\tif (test__start_subtest(\"gotox-jt-per-subprog\"))\n+\t\tcheck_gotox_jt_per_subprog();\n+\n+\tif (test__start_subtest(\"gotox-span-unreached-entry\"))\n+\t\tcheck_gotox_span_unreached_entry();\n+\n+\tif (test__start_subtest(\"gotox-target-subprog-from-main\"))\n+\t\tcheck_gotox_target_subprog_from_main();\n+\n+\tif (test__start_subtest(\"gotox-index-slice-other-subprog\"))\n+\t\tcheck_gotox_index_slice_other_subprog();\n+\n+\tif (test__start_subtest(\"gotox-target-other-global-subprog\"))\n+\t\tcheck_gotox_target_other_global_subprog();\n+\n+\tif (test__start_subtest(\"gotox-callback-leaves-subprog\"))\n+\t\tcheck_gotox_callback_leaves_subprog();\n }\n #else\n static void __test_bpf_insn_array(void)\ndiff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c\nindex 5b18c9a27717b..3567b29e23789 100644\n--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c\n+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c\n@@ -318,7 +318,7 @@ __used static int test_subprog(void)\n }\n \n SEC(\"socket\")\n-__failure __msg(\"jump table for insn 4 points outside of the subprog [0,10]\")\n+__failure __msg(\"jump table of subprog starting at 0 spans multiple subprogs\")\n __naked void jump_table_outside_subprog(void)\n {\n \tasm volatile (\"\t\t\t\t\t\t\\\n" }
2/1	2026/09/09 21:17	agent	patch-evaluator	0m Model: core Args: null Results: { "EnableConfigs": [ "BPF_SYSCALL" ], "FocusSymbols": [ "bpf_check_cfg", "bpf_compute_scc", "check_indirect_jump", "compute_subprog_jts", "create_jt", "visit_gotox_insn", "reject_gotox_out_of_subprog", "reject_gotox_without_cfg_edge", "jt_contains" ], "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.", "WorthFuzzing": true } Instruction: You are an expert Linux kernel maintainer and security engineer. Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on internal assumptions. You must actively use your code access tools to inspect the actual source code, callers, and surrounding context. ================================================================================ 1. CORE TRIAGE PHILOSOPHY ================================================================================ The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs, and newly triggered assertions introduced by the patch series. - REACHABILITY IS THE PRIMARY GATE: Fuzzing can only discover bugs in code that can actually execute in standard virtualized environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap). If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed, regardless of whether it adds assertions or complex logic. - DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS: Patch authors routinely label changes as "cleanups", "refactorings", or state "No functional change intended". Do NOT take these claims at face value. Code refactorings that rearrange logic, introduce helper functions, or alter state management in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs. If reachable executable code is modified or refactored, it MUST be fuzzed. - NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED: When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON, VM_WARN_ON, BUG_ON, lockdep_assert) in reachable code paths, it enforces new or stricter invariants. Even if the author believes the invariant always holds, fuzzing is essential to verify whether an unusual sequence of operations can violate it. ================================================================================ 2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA) ================================================================================ Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories: - Non-kernel and non-executable changes: * Modifications to Documentation/, comments, or spelling fixes. * User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/) that do not affect the compiled kernel image (vmlinux) or kernel modules. * Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints that do not alter control flow or data structures. * Build system or Kconfig changes that do not alter compiled C logic. - Structurally unreachable hardware: * Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU. - Unreachable execution paths: * Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during physical PCI hot-unplug or manual sysfs driver unbinding. * Code paths exclusive to architectures other than the target architecture. ================================================================================ 3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA) ================================================================================ Return WorthFuzzing=true whenever the patch touches reachable executable code, including: - Core Subsystems: * Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/), BPF, scheduler, core networking, VFS, or syscall handling. - Refactorings and Code Cleanups: * Any restructuring of reachable data structures, helper abstractions, or algorithm flows. - Runtime Assertions and Defensive Checks: * Any introduction or alteration of assertions (WARN_ON, VM_WARN_ON, BUG_ON, etc.) in reachable paths. - Reachable Drivers and Protocols: Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.). ================================================================================ 4. EXTRACTING FocusSymbols (PREVENTING DILUTION) ================================================================================ When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer: - AVOID UBIQUITOUS LIFECYCLE HOT-PATHS: Do NOT list generic, ubiquitous functions called by almost every program in the corpus (including, but not limited to: general memory allocators and deallocators, page fault and trap handlers, or core synchronization primitives; this is not an exhaustive list). Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused", which severely dilutes fuzzing effort away from the actual changes. - TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS: List functions that specifically implement the logic being added or altered, or direct API entrypoints for the subsystem feature under review. - HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h): Compiler-inlined static functions (such as static inlines in mm/.h or include/linux/.h) lack distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters. If the changes are primarily in static inline helpers, identify non-static, feature-specific caller functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers). ================================================================================ 5. IDENTIFYING EnableConfigs ================================================================================ Identify any specific CONFIG_ options required to properly compile and reach the modified code: - Inspect Kconfig files and #ifdef guards; do not make assumptions. - Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed. - Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: Target architecture: amd64 For your convenience, here is the diff of the changes: commit e16eef8d5994f14c9cd79ed40d9a24d4c4001f83 Author: syz-cluster <triage@syzkaller.com> Date: Wed Sep 9 21:17:23 2026 +0000 syz-cluster: applied patch under review diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 36b65797877d0..baf2e17d7019b 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -805,6 +805,7 @@ struct bpf_subprog_info { u32 linfo_idx; /* The idx to the main_prog->aux->linfo / u32 postorder_start; / The idx to the env->cfg.insn_postorder / u32 exit_idx; / Index of one of the BPF_EXIT instructions in this subprogram / + struct bpf_iarray jt; /* jump table shared by all gotox of this subprogram / u16 stack_depth; / max. stack depth used by this function / u16 stack_extra; u32 insns_total; @@ -825,6 +826,7 @@ struct bpf_subprog_info { bool keep_fastcall_stack: 1; bool changes_pkt_data: 1; bool might_sleep: 1; + bool jt_spans_subprogs: 1; u8 arg_cnt:4; enum priv_stack_mode priv_stack_mode; @@ -977,6 +979,8 @@ struct bpf_verifier_env { int cur_stack; / current position in the insn_postorder vector / int cur_postorder; + u32 gotox_edges; + bool subprog_jts_ready; } cfg; struct backtrack_state bt; struct bpf_jmp_history_entry cur_hist_ent; diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c index 842c7d1eabccc..879587af8d086 100644 --- a/kernel/bpf/cfg.c +++ b/kernel/bpf/cfg.c @@ -9,6 +9,8 @@ #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args) +#define BPF_MAX_GOTOX_EDGES BPF_COMPLEXITY_LIMIT_INSNS + /* non-recursive DFS pseudo code * 1 procedure DFS-iterative(G,v): * 2 label v as discovered @@ -284,15 +286,17 @@ static struct bpf_iarray jt_from_map(struct bpf_map map) } /* - * Find and collect all maps which fit in the subprog. Return the result as one - * combined jump table in jt->items (allocated with kvcalloc) + * Collect the jump table of every subprogram that has one, as the combined + * table of all maps whose targets land inside that subprogram. All gotox + * instructions of a subprogram share the same table, so this is done in a + * single pass over the maps rather than once per gotox. / -static struct bpf_iarray jt_from_subprog(struct bpf_verifier_env env, - int subprog_start, int subprog_end) +static int compute_subprog_jts(struct bpf_verifier_env env) { - struct bpf_iarray jt = NULL; + struct bpf_subprog_info subprog; + struct bpf_iarray jt, jt_cur; struct bpf_map map; - struct bpf_iarray jt_cur; + u32 old_cnt; int i; for (i = 0; i < env->insn_array_map_cnt; i++) { @@ -303,72 +307,97 @@ static struct bpf_iarray jt_from_subprog(struct bpf_verifier_env env, map = env->insn_array_maps[i]; jt_cur = jt_from_map(map); - if (IS_ERR(jt_cur)) { - kvfree(jt); - return jt_cur; + if (IS_ERR(jt_cur)) + return PTR_ERR(jt_cur); + + subprog = bpf_find_containing_subprog(env, jt_cur->items[0]); + if (!subprog) { + kvfree(jt_cur); + continue; + } + if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) { + subprog->jt_spans_subprogs = true; + kvfree(jt_cur); + continue; } - /* - * This is enough to check one element. The full table is - * checked to fit inside the subprog later in create_jt() - / - if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) { - u32 old_cnt = jt ? jt->cnt : 0; - jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt); - if (!jt) { - kvfree(jt_cur); - return ERR_PTR(-ENOMEM); - } - memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2); + old_cnt = subprog->jt ? subprog->jt->cnt : 0; + jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt); + if (!jt) { + subprog->jt = NULL; + kvfree(jt_cur); + return -ENOMEM; } + memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2); + subprog->jt = jt; kvfree(jt_cur); } - if (!jt) { - verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start); - bpf_diag_program_structure( - env, subprog_start, "missing jump table", - "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.", - "No jump table was found for the subprogram that starts at instruction %u.", - subprog_start); - return ERR_PTR(-EINVAL); + for (i = 0; i < env->subprog_cnt; i++) { + jt = env->subprog_info[i].jt; + if (jt) + jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt); } - jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt); - return jt; + env->cfg.subprog_jts_ready = true; + return 0; +} + +static void free_subprog_jts(struct bpf_verifier_env env) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) { + kvfree(env->subprog_info[i].jt); + env->subprog_info[i].jt = NULL; + env->subprog_info[i].jt_spans_subprogs = false; + } + env->cfg.subprog_jts_ready = false; } static struct bpf_iarray * create_jt(int t, struct bpf_verifier_env env) { struct bpf_subprog_info subprog; - int subprog_start, subprog_end; struct bpf_iarray jt; - int i; + int subprog_start, err; + + if (!env->cfg.subprog_jts_ready) { + err = compute_subprog_jts(env); + if (err) + return ERR_PTR(err); + } subprog = bpf_find_containing_subprog(env, t); subprog_start = subprog->start; - subprog_end = (subprog + 1)->start; - jt = jt_from_subprog(env, subprog_start, subprog_end); - if (IS_ERR(jt)) - return jt; - / Check that the every element of the jump table fits within the given subprogram / - for (i = 0; i < jt->cnt; i++) { - if (jt->items[i] < subprog_start \|\| jt->items[i] >= subprog_end) { - verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n", - t, subprog_start, subprog_end); - bpf_diag_program_structure( - env, t, "jump table target out of range", - "Keep every jump-table target inside the same subprogram.", - "The jump table for instruction %d points outside subprogram range [%u,%u).", - t, subprog_start, subprog_end); - kvfree(jt); - return ERR_PTR(-EINVAL); - } + if (subprog->jt_spans_subprogs) { + verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n", + subprog_start); + bpf_diag_program_structure( + env, subprog_start, "jump table spans subprograms", + "Keep every entry of a jump table inside one subprogram.", + "A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.", + subprog_start, (subprog + 1)->start); + return ERR_PTR(-EINVAL); + } + + if (!subprog->jt) { + verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start); + bpf_diag_program_structure( + env, subprog_start, "missing jump table", + "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.", + "No jump table was found for the subprogram that starts at instruction %u.", + subprog_start); + return ERR_PTR(-EINVAL); } + jt = bpf_iarray_realloc(NULL, subprog->jt->cnt); + if (!jt) + return ERR_PTR(-ENOMEM); + memcpy(jt->items, subprog->jt->items, subprog->jt->cnt << 2); + return jt; } @@ -388,6 +417,19 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env env) return PTR_ERR(jt); env->insn_aux_data[t].jt = jt; + + if (check_add_overflow(env->cfg.gotox_edges, jt->cnt, + &env->cfg.gotox_edges) \|\| + env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) { + verbose(env, "number of indirect jump edges in the program exceeds %u\n", + BPF_MAX_GOTOX_EDGES); + bpf_diag_program_structure( + env, t, "too many indirect jump edges", + "Reduce the number of indirect jumps, or the number of distinct targets they can reach.", + "The program has more than %u indirect jump edges in total, counted over every gotox instruction.", + BPF_MAX_GOTOX_EDGES); + return -E2BIG; + } } mark_prune_point(env, t); @@ -678,6 +720,7 @@ int bpf_check_cfg(struct bpf_verifier_env env) env->prog->aux->might_sleep = env->subprog_info[0].might_sleep; err_free: + free_subprog_jts(env); kvfree(insn_state); kvfree(insn_stack); env->cfg.insn_state = env->cfg.insn_stack = NULL; @@ -749,7 +792,7 @@ int bpf_compute_scc(struct bpf_verifier_env env) struct bpf_insn_aux_data aux = env->insn_aux_data; const u32 insn_cnt = env->prog->len; int stack_sz, dfs_sz, err = 0; - u32 stack, pre, low, dfs; + u32 stack, pre, low, dfs, dfs_pos; u32 i, j, t, w; u32 next_preorder_num; u32 next_scc_id; @@ -762,13 +805,16 @@ int bpf_compute_scc(struct bpf_verifier_env env) - 'stack' accumulates vertices in DFS order, see invariant comment below; * - 'pre[t] == p' => preorder number of vertex 't' is 'p'; * - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n'; - * - 'dfs' DFS traversal stack, used to emulate explicit recursion. + * - 'dfs' DFS traversal stack, used to emulate explicit recursion; + * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its + * successors at index 'j'. / stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT); pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT); low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT); dfs = kvcalloc(insn_cnt, sizeof(dfs), GFP_KERNEL_ACCOUNT); - if (!stack \|\| !pre \|\| !low \|\| !dfs) { + dfs_pos = kvcalloc(insn_cnt, sizeof(dfs_pos), GFP_KERNEL_ACCOUNT); + if (!stack \|\| !pre \|\| !low \|\| !dfs \|\| !dfs_pos) { err = -ENOMEM; goto exit; } @@ -851,6 +897,7 @@ int bpf_compute_scc(struct bpf_verifier_env env) stack_sz = 0; dfs_sz = 1; dfs[0] = i; + dfs_pos[0] = 0; dfs_continue: while (dfs_sz) { w = dfs[dfs_sz - 1]; @@ -860,13 +907,37 @@ int bpf_compute_scc(struct bpf_verifier_env env) next_preorder_num++; stack[stack_sz++] = w; } - / Visit 'w' successors / + / + * Visit 'w' successors, resuming at the successor this + * frame last descended into. Restarting the scan at zero + * on every return to 'w' would examine each successor + * once per descent, i.e. quadratic in the number of + * successors, which for a gotox is the size of the jump + * table. + * + * Re-folding the successors before that index would be a + * no-op. Such a successor 's' has 'pre[s] != 0' by then, + * so it is never pushed onto 'dfs' again, and low[s] can + * only decrease while 's' is the top of 'dfs'. If 's' is + * still on 'dfs' it sits below 'w' and cannot become the + * top before 'w' is popped; otherwise the only remaining + * write to low[s] is the pop of its SCC, setting it to + * NOT_ON_STACK, for which the min below is a no-op. + / succ = bpf_insn_successors(env, w); - for (j = 0; j < succ->cnt; ++j) { + for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) { if (pre[succ->items[j]]) { low[w] = min(low[w], low[succ->items[j]]); } else { - dfs[dfs_sz++] = succ->items[j]; + / + * Resume at 'j', not 'j + 1': the successor + * is revisited once its DFS completes, to + * fold its low[] into low[w]. + / + dfs_pos[dfs_sz - 1] = j; + dfs_pos[dfs_sz] = 0; + dfs[dfs_sz] = succ->items[j]; + dfs_sz++; goto dfs_continue; } } @@ -916,5 +987,6 @@ int bpf_compute_scc(struct bpf_verifier_env env) kvfree(pre); kvfree(low); kvfree(dfs); + kvfree(dfs_pos); return err; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 72a3f5998dd27..45234e2fbee64 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -18165,11 +18165,56 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env env, return 0; } +/ 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() / +static bool jt_contains(const struct bpf_iarray jt, u32 target) +{ + int l = 0, r = jt->cnt - 1, m; + + while (l <= r) { + m = l + (r - l) / 2; + if (jt->items[m] == target) + return true; + if (jt->items[m] < target) + l = m + 1; + else + r = m - 1; + } + return false; +} + +static int reject_gotox_out_of_subprog(struct bpf_verifier_env env, u32 target, + u32 subprog_start, u32 subprog_end) +{ + verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n", + env->insn_idx, target, subprog_start, subprog_end); + bpf_diag_program_structure( + env, env->insn_idx, "indirect jump leaves subprogram", + "Keep every reachable jump-table target inside the subprogram of the indirect jump.", + "Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).", + env->insn_idx, target, subprog_start, subprog_end); + return -EINVAL; +} + +static int reject_gotox_without_cfg_edge(struct bpf_verifier_env env, u32 target) +{ + verbose(env, "indirect jump from insn %d to %u is not in the jump table of the subprog\n", + env->insn_idx, target); + bpf_diag_program_structure( + env, env->insn_idx, "indirect jump target without CFG edge", + "Resolve indirect jumps through a jump table whose entries all fall inside the subprogram of the jump.", + "Instruction %d can jump indirectly to instruction %u, which is not part of the jump table of its subprogram.", + env->insn_idx, target); + return -EINVAL; +} + /* gotox dst_reg / static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn insn) { struct bpf_verifier_state other_branch; + struct bpf_subprog_info subprog; + u32 subprog_start, subprog_end; struct bpf_reg_state dst_reg; + struct bpf_iarray jt; struct bpf_map map; u32 min_index, max_index; int err = 0; @@ -18212,6 +18257,26 @@ static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn in return -EINVAL; } + subprog = bpf_find_containing_subprog(env, env->insn_idx); + if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx)) + return -EFAULT; + subprog_start = subprog->start; + subprog_end = (subprog + 1)->start; + + jt = env->insn_aux_data[env->insn_idx].jt; + if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx)) + return -EFAULT; + + for (i = 0; i < n; i++) { + u32 target = env->gotox_tmp_buf->items[i]; + + if (target < subprog_start \|\| target >= subprog_end) + return reject_gotox_out_of_subprog(env, target, subprog_start, + subprog_end); + if (!jt_contains(jt, target)) + return reject_gotox_without_cfg_edge(env, target); + } + for (i = 0; i < n - 1; i++) { mark_indirect_target(env, env->gotox_tmp_buf->items[i]); other_branch = push_stack(env, env->gotox_tmp_buf->items[i], diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c index 0222a9a5d0761..d5a831a75d820 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include <bpf/bpf.h> +#include <bpf/btf.h> #include <test_progs.h> #if defined(__x86_64__) \|\| defined(__powerpc__) \|\| defined(__aarch64__) @@ -453,6 +454,940 @@ static void check_bpf_no_lookup(void) close(map_fd); } +#define GOTOX_CNT_AT_LIMIT 1000 +#define GOTOX_LOG_SZ (256 1024) + +static const char gotox_limit_msg[] = + "number of indirect jump edges in the program exceeds"; + +static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt) +{ + /* the run of gotox itself, plus the exit block right after it / + const __u32 jt_cnt = gotox_cnt + 1; + struct bpf_insn_array_value val = {}; + int map_fd; + __u32 i; + + map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt); + if (!ASSERT_GE(map_fd, 0, "map_create")) + return map_fd; + + for (i = 0; i < jt_cnt; i++) { + val.orig_off = first_gotox + i; + if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, + "bpf_map_update_elem")) + goto err; + } + + if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze")) + goto err; + + return map_fd; +err: + close(map_fd); + return -1; +} + +static int gotox_prog_load_funcs(struct bpf_insn insns, __u32 insn_cnt, + int fd_array, __u32 fd_array_cnt, char log, + int btf_fd, struct bpf_func_info fi, __u32 fi_cnt) +{ + LIBBPF_OPTS(bpf_prog_load_opts, opts); + int prog_fd; + + log[0] = 0; + opts.fd_array = fd_array; + opts.fd_array_cnt = fd_array_cnt; + opts.log_buf = log; + opts.log_size = GOTOX_LOG_SZ; + opts.log_level = 1; + if (fi_cnt) { + opts.prog_btf_fd = btf_fd; + opts.func_info = fi; + opts.func_info_cnt = fi_cnt; + opts.func_info_rec_size = sizeof(fi); + } + + prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts); + if (prog_fd >= 0) { + close(prog_fd); + return 0; + } + return prog_fd; +} + +static int gotox_prog_load(struct bpf_insn insns, __u32 insn_cnt, + int fd_array, __u32 fd_array_cnt, char log) +{ + return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log, + -1, NULL, 0); +} + +/ Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. / +static void gotox_run_fill(struct bpf_insn insns, __u32 gotox_cnt) +{ + __u32 i; + + insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0); + for (i = 1; i <= gotox_cnt; i++) + insns[i] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); +} + +static void check_gotox_limit_hit(const char log, int err) +{ + ASSERT_EQ(err, -E2BIG, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log"); +} + +static bool try_load_gotox_prog(__u32 gotox_cnt, char log, int err) +{ + const __u32 insn_cnt = gotox_cnt + 3; + struct bpf_insn insns; + bool attempted = false; + int map_fd; + + insns = calloc(insn_cnt, sizeof(insns)); + if (!ASSERT_OK_PTR(insns, "calloc insns")) + return false; + + gotox_run_fill(insns, gotox_cnt); + insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[gotox_cnt + 2] = BPF_EXIT_INSN(); + + map_fd = gotox_jt_create(1, gotox_cnt); + if (map_fd < 0) + goto free_insns; + + err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log); + close(map_fd); + attempted = true; +free_insns: + free(insns); + return attempted; +} + +/* + * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt + * + 1) edges, hence the program is over the limit by gotox_cnt edges. + / +static void check_too_many_gotox_edges(void) +{ + const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + if (try_load_gotox_prog(gotox_cnt, log, &err)) + check_gotox_limit_hit(log, err); + + free(log); +} + +/* + * A chain of blocks, where block k loads jt[k] and jumps to it. The jump + * table holds the starts of the blocks that follow plus the exit block, + * which is gotox_cnt targets for gotox_cnt gotox, so the program sits + * exactly at the limit and must still load. + / +#define GOTOX_BLOCK_SZ 4 + +static void gotox_chain_fill(struct bpf_insn insns, __u32 gotox_cnt) +{ + struct bpf_insn at; + __u32 k; + + for (k = 0; k < gotox_cnt; k++) { + at = insns + k GOTOX_BLOCK_SZ; + + /* r1 = &jt[0], by index 0 into fd_array / + at[0] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + at[1] = (struct bpf_insn) { .imm = 0 }; + at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k 8); + at[3] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + } + + insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN(); +} + +static int gotox_chain_jt_create(__u32 gotox_cnt) +{ + struct bpf_insn_array_value val = {}; + int map_fd; + __u32 i; + + map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt); + if (!ASSERT_GE(map_fd, 0, "map_create")) + return map_fd; + + for (i = 0; i < gotox_cnt; i++) { + val.orig_off = (i + 1) * GOTOX_BLOCK_SZ; + if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, + "bpf_map_update_elem")) + goto err; + } + + if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze")) + goto err; + + return map_fd; +err: + close(map_fd); + return -1; +} + +static void check_gotox_edges_at_limit(void) +{ + const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT; + const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2; + struct bpf_insn insns; + char log; + int map_fd, err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + insns = calloc(insn_cnt, sizeof(insns)); + if (!ASSERT_OK_PTR(insns, "calloc insns")) + goto free_log; + + gotox_chain_fill(insns, gotox_cnt); + + map_fd = gotox_chain_jt_create(gotox_cnt); + if (map_fd < 0) + goto free_insns; + + err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log); + close(map_fd); + + if (!ASSERT_OK(err, "program at the edge limit should load")) + fprintf(stderr, "verifier log: %s\n", log); + +free_insns: + free(insns); +free_log: + free(log); +} + +static void check_gotox_edges_across_subprogs(void) +{ + const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT 3 / 4; + const __u32 sub_start = gotox_cnt + 3; + const __u32 insn_cnt = 2 * (gotox_cnt + 3); + int map_fd[2] = { -1, -1 }; + struct bpf_insn insns; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + insns = calloc(insn_cnt, sizeof(insns)); + if (!ASSERT_OK_PTR(insns, "calloc insns")) + goto free_log; + + gotox_run_fill(insns, gotox_cnt); + insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, + BPF_PSEUDO_CALL, 0, + sub_start - (gotox_cnt + 1) - 1); + insns[gotox_cnt + 2] = BPF_EXIT_INSN(); + + gotox_run_fill(insns + sub_start, gotox_cnt); + insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN(); + + map_fd[0] = gotox_jt_create(1, gotox_cnt); + if (map_fd[0] < 0) + goto free_insns; + map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log); + check_gotox_limit_hit(log, err); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_insns: + free(insns); +free_log: + free(log); +} + +static int gotox_jt_create_offs(const __u32 offs, __u32 cnt) +{ + struct bpf_insn_array_value val = {}; + int map_fd; + __u32 i; + + map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt); + if (!ASSERT_GE(map_fd, 0, "map_create")) + return map_fd; + + for (i = 0; i < cnt; i++) { + val.orig_off = offs[i]; + if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, + "bpf_map_update_elem")) + goto err; + } + + if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze")) + goto err; + + return map_fd; +err: + close(map_fd); + return -1; +} + +#define GOTOX_SUB_START 4 +#define GOTOX_MAIN_TGT 2 +#define GOTOX_SUB_TGT 8 +#define GOTOX_TWO_INSN_CNT 10 + +static void gotox_two_subprogs_fill(struct bpf_insn insns, __u32 jt_idx, __u32 jt_off) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_SUB_START - 1 - 1); + insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[3] = BPF_EXIT_INSN(); + + / r1 = &jt[0], by index 'jt_idx' into fd_array / + insns[GOTOX_SUB_START] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = jt_idx, + }; + insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 }; + insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off 8); + insns[7] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[9] = BPF_EXIT_INSN(); +} + +/* + * An insn_array map is not necessarily a jump table: one that tracks + * instruction offsets covers the whole program and is of no subprog. Such a + * map must not keep a program with a gotox elsewhere from loading. + / +static void check_gotox_tracker_map(void) +{ + const __u32 jt_track[] = { 0, GOTOX_MAIN_TGT, GOTOX_SUB_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 1, 0); + + map_fd[0] = gotox_jt_create_offs(jt_track, ARRAY_SIZE(jt_track)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + if (!ASSERT_OK(err, "program with a tracking map should load")) + fprintf(stderr, "verifier log: %s\n", log); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +static void check_gotox_target_other_subprog(void) +{ + const __u32 jt_main[] = { GOTOX_MAIN_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 0, 0); + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +static void check_gotox_jt_per_subprog(void) +{ + const __u32 jt_main[] = { GOTOX_MAIN_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 1, 0); + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +/* + * The spanning map is of no subprog and is dropped, and the entry the gotox + * register can reach is in the subprog of the gotox and in the jump table the + * CFG walked, so nothing unsafe is left and the program loads. + / +static void check_gotox_span_unreached_entry(void) +{ + const __u32 jt_span[] = { GOTOX_MAIN_TGT, GOTOX_SUB_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 0, 1); + + map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + if (!ASSERT_OK(err, "program with an unreachable spanning entry should load")) + fprintf(stderr, "verifier log: %s\n", log); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +#define GOTOX_FWD_GOTOX 11 +#define GOTOX_FWD_OWN_TGT 12 +#define GOTOX_FWD_SUB_START 14 +#define GOTOX_FWD_INSN_CNT 16 + +static void gotox_from_main_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_FWD_SUB_START - 1 - 1); + insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, + offsetof(struct xdp_md, ingress_ifindex)); + insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4); + + / r1 = &jt_leaves[0], by index 1 into fd_array / + insns[4] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 1, + }; + insns[5] = (struct bpf_insn) { .imm = 0 }; + insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + insns[7] = BPF_JMP_A(3); + + / r1 = &jt_own[0], by index 0 into fd_array / + insns[8] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[9] = (struct bpf_insn) { .imm = 0 }; + insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + + insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[13] = BPF_EXIT_INSN(); + insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[15] = BPF_EXIT_INSN(); +} + +static void check_gotox_target_subprog_from_main(void) +{ + const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT }; + const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START }; + struct bpf_insn insns[GOTOX_FWD_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_from_main_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +/* + * The only map of the subprog holding the gotox reaches past that subprog, so + * the subprog is left without a jump table at all. + / +static void check_gotox_jt_spans_subprogs(void) +{ + const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START }; + const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START }; + struct bpf_insn insns[GOTOX_FWD_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_from_main_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +/* + * The subprog holding the gotox has a well formed jump table of its own and + * also collects a map that reaches past its end. The spanning map is still + * rejected, even though the subprog is not left without a table. + / +static void check_gotox_jt_spans_with_own_table(void) +{ + const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT }; + const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START }; + struct bpf_insn insns[GOTOX_FWD_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_from_main_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +#define GOTOX_EDGE_MAIN_TGT 2 +#define GOTOX_EDGE_SUB_START 4 +#define GOTOX_EDGE_GOTOX 9 +#define GOTOX_EDGE_BR_TGT 10 +#define GOTOX_EDGE_JT_TGT 11 +#define GOTOX_EDGE_INSN_CNT 12 + +static void gotox_no_edge_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_EDGE_SUB_START - 1 - 1); + insns[GOTOX_EDGE_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[3] = BPF_EXIT_INSN(); + + insns[GOTOX_EDGE_SUB_START] = + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, + offsetof(struct xdp_md, ingress_ifindex)); + insns[5] = BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 4); + + / r1 = &jt_span[0], by index 0 into fd_array / + insns[6] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[7] = (struct bpf_insn) { .imm = 0 }; + insns[8] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 8); + + insns[GOTOX_EDGE_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_EDGE_BR_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[GOTOX_EDGE_JT_TGT] = BPF_EXIT_INSN(); +} + +/ + * The gotox resolves a target inside its own subprog, but out of a map that + * spans subprogs and is therefore of no subprog. The CFG never walked that + * edge, so the jump has to be rejected even though it stays in the subprog. + / +static void check_gotox_target_without_cfg_edge(void) +{ + const __u32 jt_span[] = { GOTOX_EDGE_MAIN_TGT, GOTOX_EDGE_BR_TGT }; + const __u32 jt_sub[] = { GOTOX_EDGE_JT_TGT }; + struct bpf_insn insns[GOTOX_EDGE_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_no_edge_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, + "indirect jump from insn 9 to 10 is not in the jump table of the subprog", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +#define GOTOX_SLICE_SUB_START 6 +#define GOTOX_SLICE_GOTOX 14 +#define GOTOX_SLICE_SUB_TGT 15 +#define GOTOX_SLICE_INSN_CNT 17 + +static void gotox_slice_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_SLICE_SUB_START - 1 - 1); + insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[5] = BPF_EXIT_INSN(); + + insns[GOTOX_SLICE_SUB_START] = + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, + offsetof(struct xdp_md, ingress_ifindex)); + insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1); + insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1); + insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3); + + / r1 = &jt_main[0], by index 0 into fd_array / + insns[10] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[11] = (struct bpf_insn) { .imm = 0 }; + insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2); + insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + + insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[16] = BPF_EXIT_INSN(); +} + +static void check_gotox_index_slice_other_subprog(void) +{ + const __u32 jt_main[] = { 2, 3, 4 }; + const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT }; + struct bpf_insn insns[GOTOX_SLICE_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_slice_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +static int gotox_btf_create(const __u32 starts, const __u8 linkage, __u32 cnt, + struct bpf_func_info fi, struct btf pbtf) +{ + int int_id, proto_id, id; + struct btf btf; + char name[24]; + __u32 i; + + btf = btf__new_empty(); + if (!ASSERT_OK_PTR(btf, "btf__new_empty")) + return -1; + + int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED); + if (!ASSERT_GT(int_id, 0, "btf__add_int")) + goto err; + + proto_id = btf__add_func_proto(btf, int_id); + if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto")) + goto err; + + for (i = 0; i < cnt; i++) { + snprintf(name, sizeof(name), "gotox_f%u", i); + id = btf__add_func(btf, name, linkage[i], proto_id); + if (!ASSERT_GT(id, 0, "btf__add_func")) + goto err; + fi[i].insn_off = starts[i]; + fi[i].type_id = id; + } + + if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel")) + goto err; + + pbtf = btf; + return btf__fd(btf); +err: + btf__free(btf); + return -1; +} + +static void check_gotox_target_other_global_subprog(void) +{ + const __u32 starts[] = { 0, GOTOX_SUB_START }; + const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL }; + const __u32 jt_main[] = { GOTOX_MAIN_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + struct bpf_func_info fi[2]; + struct btf btf = NULL; + int btf_fd; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 0, 0); + + btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf); + if (btf_fd < 0) + goto free_log; + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_btf; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log, + btf_fd, fi, ARRAY_SIZE(fi)); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_btf: + btf__free(btf); +free_log: + free(log); +} + +#define GOTOX_CB_MAIN_TGT 6 +#define GOTOX_CB_START 8 +#define GOTOX_CB_GOTOX 11 +#define GOTOX_CB_TGT 12 +#define GOTOX_CB_INSN_CNT 14 + +static void gotox_callback_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1); + /* r2 = &callback / + insns[1] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_2, + .src_reg = BPF_PSEUDO_FUNC, + .imm = GOTOX_CB_START - 1 - 1, + }; + insns[2] = (struct bpf_insn) { .imm = 0 }; + insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0); + insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0); + insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop); + insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[7] = BPF_EXIT_INSN(); + + / r1 = &jt_main[0], by index 0 into fd_array / + insns[GOTOX_CB_START] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[9] = (struct bpf_insn) { .imm = 0 }; + insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[13] = BPF_EXIT_INSN(); +} + +static void check_gotox_callback_leaves_subprog(void) +{ + const __u32 starts[] = { 0, GOTOX_CB_START }; + const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC }; + const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT }; + const __u32 jt_cb[] = { GOTOX_CB_TGT }; + struct bpf_insn insns[GOTOX_CB_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + struct bpf_func_info fi[2]; + struct btf btf = NULL; + int btf_fd; + char *log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_callback_fill(insns); + + btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf); + if (btf_fd < 0) + goto free_log; + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_btf; + map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log, + btf_fd, fi, ARRAY_SIZE(fi)); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_btf: + btf__free(btf); +free_log: + free(log); +} + static void check_bpf_side(void) { check_bpf_no_lookup(); @@ -490,6 +1425,48 @@ static void __test_bpf_insn_array(void) if (test__start_subtest("bpf-side-ops")) check_bpf_side(); + + if (test__start_subtest("too-many-gotox-edges")) + check_too_many_gotox_edges(); + + if (test__start_subtest("gotox-edges-at-limit")) + check_gotox_edges_at_limit(); + + if (test__start_subtest("gotox-edges-across-subprogs")) + check_gotox_edges_across_subprogs(); + + if (test__start_subtest("gotox-tracker-map")) + check_gotox_tracker_map(); + + if (test__start_subtest("gotox-jt-spans-subprogs")) + check_gotox_jt_spans_subprogs(); + + if (test__start_subtest("gotox-jt-spans-with-own-table")) + check_gotox_jt_spans_with_own_table(); + + if (test__start_subtest("gotox-target-without-cfg-edge")) + check_gotox_target_without_cfg_edge(); + + if (test__start_subtest("gotox-target-other-subprog")) + check_gotox_target_other_subprog(); + + if (test__start_subtest("gotox-jt-per-subprog")) + check_gotox_jt_per_subprog(); + + if (test__start_subtest("gotox-span-unreached-entry")) + check_gotox_span_unreached_entry(); + + if (test__start_subtest("gotox-target-subprog-from-main")) + check_gotox_target_subprog_from_main(); + + if (test__start_subtest("gotox-index-slice-other-subprog")) + check_gotox_index_slice_other_subprog(); + + if (test__start_subtest("gotox-target-other-global-subprog")) + check_gotox_target_other_global_subprog(); + + if (test__start_subtest("gotox-callback-leaves-subprog")) + check_gotox_callback_leaves_subprog(); } #else static void __test_bpf_insn_array(void) diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c index 5b18c9a27717b..3567b29e23789 100644 --- a/tools/testing/selftests/bpf/progs/verifier_gotox.c +++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c @@ -318,7 +318,7 @@ __used static int test_subprog(void) } SEC("socket") -__failure __msg("jump table for insn 4 points outside of the subprog [0,10]") +__failure __msg("jump table of subprog starting at 0 spans multiple subprogs") __naked void jump_table_outside_subprog(void) { asm volatile (" \
3/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 22781 output: 38 thoughts: 508 Thoughts: Fuzzing Target Assessment: BPF Verifier Indirect Jumps Okay, let's break this down. My initial assessment is that this patch, touching the BPF verifier, warrants fuzzing with syzkaller. First, the patch is scoped to changes in `include/linux/bpf_verifier.h`, `kernel/bpf/cfg.c`, and `kernel/bpf/verifier.c` – a clear indication of alterations within the BPF verifier and, specifically, its control-flow graph (CFG) analysis for indirect jumps using `gotox`. The fact that it also includes selftests is encouraging for reproducing any potential issues. Second, the BPF verifier itself is reachable. It's invoked through the `bpf()` syscall (`BPF_PROG_LOAD`), and amd64 is our target architecture. We can easily execute this in a standard QEMU or GCE environment. The modified code directly impacts jump table construction, CFG edge calculation, strongly connected component (SCC) computation during the DFS traversal within `bpf_compute_scc`, and crucially, verifier safety checks for indirect jumps, as seen in `check_indirect_jump`. This is all core kernel BPF code, clearly reachable. Therefore, the verdict is a definite "WorthFuzzing: true." Now, for deeper analysis. I'll take a look at the specific functions and symbols altered to guide fuzzing strategy. In `kernel/bpf/cfg.c`, we see `compute_subprog_jts`, `free_subprog_jts`, `create_jt`, and `visit_gotox_insn` (all static) and `bpf_check_cfg` (exported, called by the verifier), and `bpf_compute_scc` (non-static) are relevant. In `kernel/bpf/verifier.c`, we've got `jt_contains`, `reject_gotox_out_of_subprog`, `reject_gotox_without_cfg_edge`, and `check_indirect_jump` (all static) and `bpf_check` which in turn calls `bpf_check_cfg`, `bpf_compute_scc`, `check_indirect_jump` and `do_check`. My next step will be a deep dive with `grepper` (or codesearch) to determine where these symbols are defined and used, especially the non-static ones or the ones that seem suitable for FocusSymbols. By pinpointing the call sites for `bpf_check_cfg`, `bpf_compute_scc`, and `check_indirect_jump`, I can build a targeted fuzzer configuration for syzkaller.
4/2	2026/09/09 21:17	tool	grepper	0m Args: { "Expression": "bpf_check_cfg\|bpf_compute_scc", "PathPrefix": "kernel/bpf" } Results: { "Output": "kernel/bpf/cfg.c=495=static int visit_insn(int t, struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-631- /\nkernel/bpf/cfg.c:632:int bpf_check_cfg(struct bpf_verifier_env env)\nkernel/bpf/cfg.c-633-{\n--\nkernel/bpf/cfg.c=735=int bpf_compute_postorder(struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-787- /\nkernel/bpf/cfg.c:788:int bpf_compute_scc(struct bpf_verifier_env env)\nkernel/bpf/cfg.c-789-{\n--\nkernel/bpf/verifier.c=21197=int bpf_check(struct bpf_prog *prog, union bpf_attr attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-21342-\nkernel/bpf/verifier.c:21343:\tret = bpf_check_cfg(env);\nkernel/bpf/verifier.c-21344-\tif (ret \u003c 0)\n--\nkernel/bpf/verifier.c-21370-\nkernel/bpf/verifier.c:21371:\tret = bpf_compute_scc(env);\nkernel/bpf/verifier.c-21372-\tif (ret \u003c 0)\n" }
5/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 24309 output: 29 thoughts: 80 Thoughts: Here's my analysis of what needs to be looked at, thinking as you: Diving into the BPF Verifier and Control Flow Okay, so I need to get my hands dirty and really understand what's going on here. I'm going to start by scrutinizing the key functions involved in the BPF verification and control flow graph (CFG) construction. I'm thinking I should start with `check_indirect_jump`, `visit_gotox_insn`, `compute_subprog_jts`, `bpf_check_cfg`, and `bpf_compute_scc`. Those names alone give a decent clue about where to start. They all seem central to how the verifier handles jumps and manages the program's structure. Beyond those functions, I really need to have a clear picture of what's in the code. So, I'll be digging into `kernel/bpf/verifier.c` and `kernel/bpf/cfg.c`. I need to get a clear picture of the relevant symbols, functions and data structures. It'll be essential to understand the logic behind these functions if I want to figure out what's causing the problem. Hopefully, understanding how they are implemented will clarify the underlying issues.
6/2	2026/09/09 21:17	tool	grepper	0m Args: { "Expression": "check_indirect_jump", "PathPrefix": "kernel/bpf" } Results: { "Output": "kernel/bpf/verifier.c=18198=static int reject_gotox_without_cfg_edge(struct bpf_verifier_env env, u32 target)\n--\nkernel/bpf/verifier.c-18210-/ gotox dst_reg /\nkernel/bpf/verifier.c:18211:static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn insn)\nkernel/bpf/verifier.c-18212-{\n--\nkernel/bpf/verifier.c=18292=static int do_check_insn(struct bpf_verifier_env env, bool do_print_state)\n--\nkernel/bpf/verifier.c-18368-\t\t\tif (BPF_SRC(insn-\u003ecode) == BPF_X)\nkernel/bpf/verifier.c:18369:\t\t\t\treturn check_indirect_jump(env, insn);\nkernel/bpf/verifier.c-18370-\n" }
7/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 24957 output: 27 thoughts: 47 Thoughts: Okay, here's my thought process as I delve into the Kconfig aspects of BPF: Investigating BPF Verifier and Syscall Kconfig Dependencies Alright, time to get to the heart of the matter. I need to nail down the specific Kconfig options that govern the BPF verifier and the BPF system call. I'm thinking, "Where would I find the most definitive information?" Obviously, the `kernel/bpf/` directory is the logical starting point. I'll need to double-check both the `kernel/bpf/Makefile` for build dependencies and then the `init/Kconfig` or `kernel/bpf/Kconfig` files. That should give me a solid view of the configuration knobs and how they interrelate. I'm after the dependencies, and the options that control enabling/disabling these features. Time to dig in and see what the build system needs to be in place.
8/2	2026/09/09 21:17	tool	grepper	0m Args: { "Expression": "cfg\\.o", "PathPrefix": "kernel/bpf" } Results: { "Output": "kernel/bpf/Makefile=13=obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o\nkernel/bpf/Makefile:14:obj-$(CONFIG_BPF_SYSCALL) += fixups.o cfg.o states.o backtrack.o check_btf.o\nkernel/bpf/Makefile-15-obj-${CONFIG_BPF_LSM}\t += bpf_inode_storage.o\n" }
9/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 25358 output: 29 thoughts: 62 Thoughts: My BPF Configuration Considerations Okay, so I see that `cfg.o` and `verifier.o` are dependent on `CONFIG_BPF_SYSCALL`. That's a key dependency to keep in mind. I need to be absolutely sure this is enabled. Let's make a quick sweep of the other BPF-related configurations while I'm at it. I'll need to figure out the best way to determine if there are any other dependencies. Time to dive into the Kconfig files. I'll start with `init/Kconfig` and then check `kernel/bpf/Kconfig`. Those are typically the most relevant spots for BPF configuration options, and I want to get a complete picture before proceeding.
10/2	2026/09/09 21:17	tool	grepper	0m Error: no matches Args: { "Expression": "config BPF_SYSCALL", "PathPrefix": "init/" } Results: { "error": "no matches" }
11/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 25620 output: 31 thoughts: 13 Thoughts: Here's a summary of my thought process, as requested: Diving into the BPF Kernel Configuration Okay, time to get a handle on the current state of things. Let's start with a deep dive into the `kernel/bpf/Kconfig` file. I need to refresh my memory on the configuration options available for BPF within the kernel. I'm thinking about the various features and how they are enabled or disabled. This Kconfig file will be the central point to understand the available knobs and levers that control BPF's behaviour. I want to see what's enabled by default, what requires specific configurations, and if there are any new or changed options since I last looked. This is the bedrock of understanding how BPF is integrated and what options are available when building or configuring a kernel with BPF capabilities. Knowing this is critical for setting up my development environment and determining the best strategy for my work. So, `kernel/bpf/Kconfig`, here we go.
12/2	2026/09/09 21:17	tool	grepper	0m Args: { "Expression": "config BPF_SYSCALL", "PathPrefix": "kernel/bpf" } Results: { "Output": "kernel/bpf/Kconfig=25=menu \"BPF subsystem\"\nkernel/bpf/Kconfig-26-\nkernel/bpf/Kconfig:27:config BPF_SYSCALL\nkernel/bpf/Kconfig-28-\tbool \"Enable bpf() system call\"\n" }
13/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 25957 output: 34 thoughts: 19 Thoughts: Here's a summary of my thought process, as if I were the one thinking it: Diving into the BPF Kernel Configuration Okay, time to get my hands dirty and understand how the BPF subsystem is configured within the kernel. I'm going to navigate directly to `kernel/bpf/Kconfig`. Specifically, I want to zero in on the relevant section around line 27. Hopefully, this will provide the initial context I need to refresh my memory on the key configuration options and their dependencies. This Kconfig file will be the perfect place to start. Let's see what we can find.
14/2	2026/09/09 21:17	tool	read-file	0m Args: { "File": "kernel/bpf/Kconfig", "FirstLine": 25, "LineCount": 20 } Results: { "Contents": " 25:\tmenu \"BPF subsystem\"\n 26:\t\n 27:\tconfig BPF_SYSCALL\n 28:\t\tbool \"Enable bpf() system call\"\n 29:\t\tselect BPF\n 30:\t\tselect IRQ_WORK\n 31:\t\tselect NEED_TASKS_RCU\n 32:\t\tselect TASKS_TRACE_RCU\n 33:\t\tselect BINARY_PRINTF\n 34:\t\tselect NET_SOCK_MSG if NET\n 35:\t\tselect NET_XGRESS if NET\n 36:\t\tselect PAGE_POOL if NET\n 37:\t\tdefault n\n 38:\t\thelp\n 39:\t\t Enable the bpf() system call that allows to manipulate BPF programs\n 40:\t\t and maps via file descriptors.\n 41:\t\n 42:\tconfig BPF_JIT\n 43:\t\tbool \"Enable BPF Just In Time compiler\"\n 44:\t\tdepends on BPF\n" }
15/2	2026/09/09 21:17	llm	patch-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 26392 output: 190 thoughts: 367 Thoughts: My Analysis of the BPF Verifier Patch Okay, let's break this down. First, I need to zero in on the key functions touched by this patch. I'm focusing on: `bpf_check_cfg`, `bpf_compute_scc`, `check_indirect_jump`, and then the bunch of functions tied to jump tables: `compute_subprog_jts`, `create_jt`, `visit_gotox_insn`, `reject_gotox_out_of_subprog`, `reject_gotox_without_cfg_edge`, and `jt_contains`. These are the core elements I need to examine. I've confirmed these functions reside within `kernel/bpf/cfg.c` and `kernel/bpf/verifier.c`, which is exactly where I expect to find them. These aren't generic, widespread utilities; they are laser-focused on BPF-specific control flow graph manipulation, SCC analysis, and specifically, the handling of `gotox` indirect jumps. That’s a strong signal. The patch targets critical BPF infrastructure: CFG analysis, SCC computation, jump table handling, and the all-important verification logic for indirect jumps. The `BPF_SYSCALL` configuration enables this code path. Crucially, this means that these code paths are directly exposed to user space through the `bpf(BPF_PROG_LOAD)` system call. This is a prime target for fuzzing. Any change here has the potential to introduce vulnerabilities that can be exploited from user space. The `WorthFuzzing` flag is absolutely spot on in this case. Okay, this warrants a fuzzing campaign to thoroughly test the integrity of this code. Time to set those results.
16/2	2026/09/09 21:17	tool	set-results	0m Args: { "EnableConfigs": [ "BPF_SYSCALL" ], "FocusSymbols": [ "bpf_check_cfg", "bpf_compute_scc", "check_indirect_jump", "compute_subprog_jts", "create_jt", "visit_gotox_insn", "reject_gotox_out_of_subprog", "reject_gotox_without_cfg_edge", "jt_contains" ], "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.", "WorthFuzzing": true } Results: { "EnableConfigs": [ "BPF_SYSCALL" ], "FocusSymbols": [ "bpf_check_cfg", "bpf_compute_scc", "check_indirect_jump", "compute_subprog_jts", "create_jt", "visit_gotox_insn", "reject_gotox_out_of_subprog", "reject_gotox_without_cfg_edge", "jt_contains" ], "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.", "WorthFuzzing": true }
17/1	2026/09/09 21:17	action	If	0m Args: { "WorthFuzzing": true } Results: null
18/2	2026/09/09 21:17	agent	kmsan-evaluator	0m Model: core Args: null Results: { "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.", "NeedsKMSAN": false } Instruction: You are an expert Linux kernel security engineer specializing in kernel memory error detectors (KASAN and KMSAN). Your job is to review the provided patch series and determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer) fuzzing session in addition to standard KASAN fuzzing. CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN: - Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable a comprehensive suite of debugging tools and sanitizers, including KASAN (out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP (locking bugs and deadlocks), UB-sanitizers, and memory corruption checks. - KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap, or page allocations) and kernel-to-user memory info-leaks. Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN, LOCKDEP, OR OTHER STANDARD BUG DETECTORS. A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks that are detected ONLY by KMSAN. Look holistically at the patch series and surrounding code. Even if no direct uninitialized field accesses or new buffer allocations are added in the diff itself, a patch may alter control flow, bounds checking, or data length calculations in ways that change how the rest of the code operates on existing buffers (e.g. allowing uninitialized stack/heap memory to be read, copied to user space, or used in control flow). Do not hesitate to use your code access tools to inspect the surrounding code, called functions, and callers. Set NeedsKMSAN=true ONLY IF the patch introduces or modifies: 1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb attributes, ioctl output arguments, socket options, or BPF buffers) where fields or structure padding might not be fully initialized/zeroed. 2. Conditional logic or branching that depends on potentially uninitialized variables or struct fields. 3. Allocation or initialization of complex data structures where uninitialized fields could be read later in reachable code paths. 4. Bounds checks, lengths, or logic in a way that allows surrounding code to access uninitialized bytes of existing buffers. Set NeedsKMSAN=false IF: - The code changes primarily risk out-of-bounds access, array overflows, NULL pointer dereferences, locking deadlocks, or use-after-free bugs (these are already caught by KASAN, LOCKDEP, or standard bug detectors). - All stack/heap structures touched or introduced by the patch are fully zeroed or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied. - The patch does not introduce any risk of uninitialized memory usage or info-leaks. Use your code access tools to inspect the surrounding code if necessary, then provide detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch. Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: Target architecture: amd64 For your convenience, here is the diff of the changes: commit e16eef8d5994f14c9cd79ed40d9a24d4c4001f83 Author: syz-cluster <triage@syzkaller.com> Date: Wed Sep 9 21:17:23 2026 +0000 syz-cluster: applied patch under review diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 36b65797877d0..baf2e17d7019b 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -805,6 +805,7 @@ struct bpf_subprog_info { u32 linfo_idx; /* The idx to the main_prog->aux->linfo / u32 postorder_start; / The idx to the env->cfg.insn_postorder / u32 exit_idx; / Index of one of the BPF_EXIT instructions in this subprogram / + struct bpf_iarray jt; /* jump table shared by all gotox of this subprogram / u16 stack_depth; / max. stack depth used by this function / u16 stack_extra; u32 insns_total; @@ -825,6 +826,7 @@ struct bpf_subprog_info { bool keep_fastcall_stack: 1; bool changes_pkt_data: 1; bool might_sleep: 1; + bool jt_spans_subprogs: 1; u8 arg_cnt:4; enum priv_stack_mode priv_stack_mode; @@ -977,6 +979,8 @@ struct bpf_verifier_env { int cur_stack; / current position in the insn_postorder vector / int cur_postorder; + u32 gotox_edges; + bool subprog_jts_ready; } cfg; struct backtrack_state bt; struct bpf_jmp_history_entry cur_hist_ent; diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c index 842c7d1eabccc..879587af8d086 100644 --- a/kernel/bpf/cfg.c +++ b/kernel/bpf/cfg.c @@ -9,6 +9,8 @@ #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args) +#define BPF_MAX_GOTOX_EDGES BPF_COMPLEXITY_LIMIT_INSNS + /* non-recursive DFS pseudo code * 1 procedure DFS-iterative(G,v): * 2 label v as discovered @@ -284,15 +286,17 @@ static struct bpf_iarray jt_from_map(struct bpf_map map) } /* - * Find and collect all maps which fit in the subprog. Return the result as one - * combined jump table in jt->items (allocated with kvcalloc) + * Collect the jump table of every subprogram that has one, as the combined + * table of all maps whose targets land inside that subprogram. All gotox + * instructions of a subprogram share the same table, so this is done in a + * single pass over the maps rather than once per gotox. / -static struct bpf_iarray jt_from_subprog(struct bpf_verifier_env env, - int subprog_start, int subprog_end) +static int compute_subprog_jts(struct bpf_verifier_env env) { - struct bpf_iarray jt = NULL; + struct bpf_subprog_info subprog; + struct bpf_iarray jt, jt_cur; struct bpf_map map; - struct bpf_iarray jt_cur; + u32 old_cnt; int i; for (i = 0; i < env->insn_array_map_cnt; i++) { @@ -303,72 +307,97 @@ static struct bpf_iarray jt_from_subprog(struct bpf_verifier_env env, map = env->insn_array_maps[i]; jt_cur = jt_from_map(map); - if (IS_ERR(jt_cur)) { - kvfree(jt); - return jt_cur; + if (IS_ERR(jt_cur)) + return PTR_ERR(jt_cur); + + subprog = bpf_find_containing_subprog(env, jt_cur->items[0]); + if (!subprog) { + kvfree(jt_cur); + continue; + } + if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) { + subprog->jt_spans_subprogs = true; + kvfree(jt_cur); + continue; } - /* - * This is enough to check one element. The full table is - * checked to fit inside the subprog later in create_jt() - / - if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) { - u32 old_cnt = jt ? jt->cnt : 0; - jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt); - if (!jt) { - kvfree(jt_cur); - return ERR_PTR(-ENOMEM); - } - memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2); + old_cnt = subprog->jt ? subprog->jt->cnt : 0; + jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt); + if (!jt) { + subprog->jt = NULL; + kvfree(jt_cur); + return -ENOMEM; } + memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2); + subprog->jt = jt; kvfree(jt_cur); } - if (!jt) { - verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start); - bpf_diag_program_structure( - env, subprog_start, "missing jump table", - "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.", - "No jump table was found for the subprogram that starts at instruction %u.", - subprog_start); - return ERR_PTR(-EINVAL); + for (i = 0; i < env->subprog_cnt; i++) { + jt = env->subprog_info[i].jt; + if (jt) + jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt); } - jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt); - return jt; + env->cfg.subprog_jts_ready = true; + return 0; +} + +static void free_subprog_jts(struct bpf_verifier_env env) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) { + kvfree(env->subprog_info[i].jt); + env->subprog_info[i].jt = NULL; + env->subprog_info[i].jt_spans_subprogs = false; + } + env->cfg.subprog_jts_ready = false; } static struct bpf_iarray * create_jt(int t, struct bpf_verifier_env env) { struct bpf_subprog_info subprog; - int subprog_start, subprog_end; struct bpf_iarray jt; - int i; + int subprog_start, err; + + if (!env->cfg.subprog_jts_ready) { + err = compute_subprog_jts(env); + if (err) + return ERR_PTR(err); + } subprog = bpf_find_containing_subprog(env, t); subprog_start = subprog->start; - subprog_end = (subprog + 1)->start; - jt = jt_from_subprog(env, subprog_start, subprog_end); - if (IS_ERR(jt)) - return jt; - / Check that the every element of the jump table fits within the given subprogram / - for (i = 0; i < jt->cnt; i++) { - if (jt->items[i] < subprog_start \|\| jt->items[i] >= subprog_end) { - verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n", - t, subprog_start, subprog_end); - bpf_diag_program_structure( - env, t, "jump table target out of range", - "Keep every jump-table target inside the same subprogram.", - "The jump table for instruction %d points outside subprogram range [%u,%u).", - t, subprog_start, subprog_end); - kvfree(jt); - return ERR_PTR(-EINVAL); - } + if (subprog->jt_spans_subprogs) { + verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n", + subprog_start); + bpf_diag_program_structure( + env, subprog_start, "jump table spans subprograms", + "Keep every entry of a jump table inside one subprogram.", + "A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.", + subprog_start, (subprog + 1)->start); + return ERR_PTR(-EINVAL); + } + + if (!subprog->jt) { + verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start); + bpf_diag_program_structure( + env, subprog_start, "missing jump table", + "Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.", + "No jump table was found for the subprogram that starts at instruction %u.", + subprog_start); + return ERR_PTR(-EINVAL); } + jt = bpf_iarray_realloc(NULL, subprog->jt->cnt); + if (!jt) + return ERR_PTR(-ENOMEM); + memcpy(jt->items, subprog->jt->items, subprog->jt->cnt << 2); + return jt; } @@ -388,6 +417,19 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env env) return PTR_ERR(jt); env->insn_aux_data[t].jt = jt; + + if (check_add_overflow(env->cfg.gotox_edges, jt->cnt, + &env->cfg.gotox_edges) \|\| + env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) { + verbose(env, "number of indirect jump edges in the program exceeds %u\n", + BPF_MAX_GOTOX_EDGES); + bpf_diag_program_structure( + env, t, "too many indirect jump edges", + "Reduce the number of indirect jumps, or the number of distinct targets they can reach.", + "The program has more than %u indirect jump edges in total, counted over every gotox instruction.", + BPF_MAX_GOTOX_EDGES); + return -E2BIG; + } } mark_prune_point(env, t); @@ -678,6 +720,7 @@ int bpf_check_cfg(struct bpf_verifier_env env) env->prog->aux->might_sleep = env->subprog_info[0].might_sleep; err_free: + free_subprog_jts(env); kvfree(insn_state); kvfree(insn_stack); env->cfg.insn_state = env->cfg.insn_stack = NULL; @@ -749,7 +792,7 @@ int bpf_compute_scc(struct bpf_verifier_env env) struct bpf_insn_aux_data aux = env->insn_aux_data; const u32 insn_cnt = env->prog->len; int stack_sz, dfs_sz, err = 0; - u32 stack, pre, low, dfs; + u32 stack, pre, low, dfs, dfs_pos; u32 i, j, t, w; u32 next_preorder_num; u32 next_scc_id; @@ -762,13 +805,16 @@ int bpf_compute_scc(struct bpf_verifier_env env) - 'stack' accumulates vertices in DFS order, see invariant comment below; * - 'pre[t] == p' => preorder number of vertex 't' is 'p'; * - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n'; - * - 'dfs' DFS traversal stack, used to emulate explicit recursion. + * - 'dfs' DFS traversal stack, used to emulate explicit recursion; + * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its + * successors at index 'j'. / stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT); pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT); low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT); dfs = kvcalloc(insn_cnt, sizeof(dfs), GFP_KERNEL_ACCOUNT); - if (!stack \|\| !pre \|\| !low \|\| !dfs) { + dfs_pos = kvcalloc(insn_cnt, sizeof(dfs_pos), GFP_KERNEL_ACCOUNT); + if (!stack \|\| !pre \|\| !low \|\| !dfs \|\| !dfs_pos) { err = -ENOMEM; goto exit; } @@ -851,6 +897,7 @@ int bpf_compute_scc(struct bpf_verifier_env env) stack_sz = 0; dfs_sz = 1; dfs[0] = i; + dfs_pos[0] = 0; dfs_continue: while (dfs_sz) { w = dfs[dfs_sz - 1]; @@ -860,13 +907,37 @@ int bpf_compute_scc(struct bpf_verifier_env env) next_preorder_num++; stack[stack_sz++] = w; } - / Visit 'w' successors / + / + * Visit 'w' successors, resuming at the successor this + * frame last descended into. Restarting the scan at zero + * on every return to 'w' would examine each successor + * once per descent, i.e. quadratic in the number of + * successors, which for a gotox is the size of the jump + * table. + * + * Re-folding the successors before that index would be a + * no-op. Such a successor 's' has 'pre[s] != 0' by then, + * so it is never pushed onto 'dfs' again, and low[s] can + * only decrease while 's' is the top of 'dfs'. If 's' is + * still on 'dfs' it sits below 'w' and cannot become the + * top before 'w' is popped; otherwise the only remaining + * write to low[s] is the pop of its SCC, setting it to + * NOT_ON_STACK, for which the min below is a no-op. + / succ = bpf_insn_successors(env, w); - for (j = 0; j < succ->cnt; ++j) { + for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) { if (pre[succ->items[j]]) { low[w] = min(low[w], low[succ->items[j]]); } else { - dfs[dfs_sz++] = succ->items[j]; + / + * Resume at 'j', not 'j + 1': the successor + * is revisited once its DFS completes, to + * fold its low[] into low[w]. + / + dfs_pos[dfs_sz - 1] = j; + dfs_pos[dfs_sz] = 0; + dfs[dfs_sz] = succ->items[j]; + dfs_sz++; goto dfs_continue; } } @@ -916,5 +987,6 @@ int bpf_compute_scc(struct bpf_verifier_env env) kvfree(pre); kvfree(low); kvfree(dfs); + kvfree(dfs_pos); return err; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 72a3f5998dd27..45234e2fbee64 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -18165,11 +18165,56 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env env, return 0; } +/ 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() / +static bool jt_contains(const struct bpf_iarray jt, u32 target) +{ + int l = 0, r = jt->cnt - 1, m; + + while (l <= r) { + m = l + (r - l) / 2; + if (jt->items[m] == target) + return true; + if (jt->items[m] < target) + l = m + 1; + else + r = m - 1; + } + return false; +} + +static int reject_gotox_out_of_subprog(struct bpf_verifier_env env, u32 target, + u32 subprog_start, u32 subprog_end) +{ + verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n", + env->insn_idx, target, subprog_start, subprog_end); + bpf_diag_program_structure( + env, env->insn_idx, "indirect jump leaves subprogram", + "Keep every reachable jump-table target inside the subprogram of the indirect jump.", + "Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).", + env->insn_idx, target, subprog_start, subprog_end); + return -EINVAL; +} + +static int reject_gotox_without_cfg_edge(struct bpf_verifier_env env, u32 target) +{ + verbose(env, "indirect jump from insn %d to %u is not in the jump table of the subprog\n", + env->insn_idx, target); + bpf_diag_program_structure( + env, env->insn_idx, "indirect jump target without CFG edge", + "Resolve indirect jumps through a jump table whose entries all fall inside the subprogram of the jump.", + "Instruction %d can jump indirectly to instruction %u, which is not part of the jump table of its subprogram.", + env->insn_idx, target); + return -EINVAL; +} + /* gotox dst_reg / static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn insn) { struct bpf_verifier_state other_branch; + struct bpf_subprog_info subprog; + u32 subprog_start, subprog_end; struct bpf_reg_state dst_reg; + struct bpf_iarray jt; struct bpf_map map; u32 min_index, max_index; int err = 0; @@ -18212,6 +18257,26 @@ static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn in return -EINVAL; } + subprog = bpf_find_containing_subprog(env, env->insn_idx); + if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx)) + return -EFAULT; + subprog_start = subprog->start; + subprog_end = (subprog + 1)->start; + + jt = env->insn_aux_data[env->insn_idx].jt; + if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx)) + return -EFAULT; + + for (i = 0; i < n; i++) { + u32 target = env->gotox_tmp_buf->items[i]; + + if (target < subprog_start \|\| target >= subprog_end) + return reject_gotox_out_of_subprog(env, target, subprog_start, + subprog_end); + if (!jt_contains(jt, target)) + return reject_gotox_without_cfg_edge(env, target); + } + for (i = 0; i < n - 1; i++) { mark_indirect_target(env, env->gotox_tmp_buf->items[i]); other_branch = push_stack(env, env->gotox_tmp_buf->items[i], diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c index 0222a9a5d0761..d5a831a75d820 100644 --- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include <bpf/bpf.h> +#include <bpf/btf.h> #include <test_progs.h> #if defined(__x86_64__) \|\| defined(__powerpc__) \|\| defined(__aarch64__) @@ -453,6 +454,940 @@ static void check_bpf_no_lookup(void) close(map_fd); } +#define GOTOX_CNT_AT_LIMIT 1000 +#define GOTOX_LOG_SZ (256 1024) + +static const char gotox_limit_msg[] = + "number of indirect jump edges in the program exceeds"; + +static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt) +{ + /* the run of gotox itself, plus the exit block right after it / + const __u32 jt_cnt = gotox_cnt + 1; + struct bpf_insn_array_value val = {}; + int map_fd; + __u32 i; + + map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt); + if (!ASSERT_GE(map_fd, 0, "map_create")) + return map_fd; + + for (i = 0; i < jt_cnt; i++) { + val.orig_off = first_gotox + i; + if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, + "bpf_map_update_elem")) + goto err; + } + + if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze")) + goto err; + + return map_fd; +err: + close(map_fd); + return -1; +} + +static int gotox_prog_load_funcs(struct bpf_insn insns, __u32 insn_cnt, + int fd_array, __u32 fd_array_cnt, char log, + int btf_fd, struct bpf_func_info fi, __u32 fi_cnt) +{ + LIBBPF_OPTS(bpf_prog_load_opts, opts); + int prog_fd; + + log[0] = 0; + opts.fd_array = fd_array; + opts.fd_array_cnt = fd_array_cnt; + opts.log_buf = log; + opts.log_size = GOTOX_LOG_SZ; + opts.log_level = 1; + if (fi_cnt) { + opts.prog_btf_fd = btf_fd; + opts.func_info = fi; + opts.func_info_cnt = fi_cnt; + opts.func_info_rec_size = sizeof(fi); + } + + prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts); + if (prog_fd >= 0) { + close(prog_fd); + return 0; + } + return prog_fd; +} + +static int gotox_prog_load(struct bpf_insn insns, __u32 insn_cnt, + int fd_array, __u32 fd_array_cnt, char log) +{ + return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log, + -1, NULL, 0); +} + +/ Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. / +static void gotox_run_fill(struct bpf_insn insns, __u32 gotox_cnt) +{ + __u32 i; + + insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0); + for (i = 1; i <= gotox_cnt; i++) + insns[i] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); +} + +static void check_gotox_limit_hit(const char log, int err) +{ + ASSERT_EQ(err, -E2BIG, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log"); +} + +static bool try_load_gotox_prog(__u32 gotox_cnt, char log, int err) +{ + const __u32 insn_cnt = gotox_cnt + 3; + struct bpf_insn insns; + bool attempted = false; + int map_fd; + + insns = calloc(insn_cnt, sizeof(insns)); + if (!ASSERT_OK_PTR(insns, "calloc insns")) + return false; + + gotox_run_fill(insns, gotox_cnt); + insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[gotox_cnt + 2] = BPF_EXIT_INSN(); + + map_fd = gotox_jt_create(1, gotox_cnt); + if (map_fd < 0) + goto free_insns; + + err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log); + close(map_fd); + attempted = true; +free_insns: + free(insns); + return attempted; +} + +/* + * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt + * + 1) edges, hence the program is over the limit by gotox_cnt edges. + / +static void check_too_many_gotox_edges(void) +{ + const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + if (try_load_gotox_prog(gotox_cnt, log, &err)) + check_gotox_limit_hit(log, err); + + free(log); +} + +/* + * A chain of blocks, where block k loads jt[k] and jumps to it. The jump + * table holds the starts of the blocks that follow plus the exit block, + * which is gotox_cnt targets for gotox_cnt gotox, so the program sits + * exactly at the limit and must still load. + / +#define GOTOX_BLOCK_SZ 4 + +static void gotox_chain_fill(struct bpf_insn insns, __u32 gotox_cnt) +{ + struct bpf_insn at; + __u32 k; + + for (k = 0; k < gotox_cnt; k++) { + at = insns + k GOTOX_BLOCK_SZ; + + /* r1 = &jt[0], by index 0 into fd_array / + at[0] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + at[1] = (struct bpf_insn) { .imm = 0 }; + at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k 8); + at[3] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + } + + insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN(); +} + +static int gotox_chain_jt_create(__u32 gotox_cnt) +{ + struct bpf_insn_array_value val = {}; + int map_fd; + __u32 i; + + map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt); + if (!ASSERT_GE(map_fd, 0, "map_create")) + return map_fd; + + for (i = 0; i < gotox_cnt; i++) { + val.orig_off = (i + 1) * GOTOX_BLOCK_SZ; + if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, + "bpf_map_update_elem")) + goto err; + } + + if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze")) + goto err; + + return map_fd; +err: + close(map_fd); + return -1; +} + +static void check_gotox_edges_at_limit(void) +{ + const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT; + const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2; + struct bpf_insn insns; + char log; + int map_fd, err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + insns = calloc(insn_cnt, sizeof(insns)); + if (!ASSERT_OK_PTR(insns, "calloc insns")) + goto free_log; + + gotox_chain_fill(insns, gotox_cnt); + + map_fd = gotox_chain_jt_create(gotox_cnt); + if (map_fd < 0) + goto free_insns; + + err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log); + close(map_fd); + + if (!ASSERT_OK(err, "program at the edge limit should load")) + fprintf(stderr, "verifier log: %s\n", log); + +free_insns: + free(insns); +free_log: + free(log); +} + +static void check_gotox_edges_across_subprogs(void) +{ + const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT 3 / 4; + const __u32 sub_start = gotox_cnt + 3; + const __u32 insn_cnt = 2 * (gotox_cnt + 3); + int map_fd[2] = { -1, -1 }; + struct bpf_insn insns; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + insns = calloc(insn_cnt, sizeof(insns)); + if (!ASSERT_OK_PTR(insns, "calloc insns")) + goto free_log; + + gotox_run_fill(insns, gotox_cnt); + insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, + BPF_PSEUDO_CALL, 0, + sub_start - (gotox_cnt + 1) - 1); + insns[gotox_cnt + 2] = BPF_EXIT_INSN(); + + gotox_run_fill(insns + sub_start, gotox_cnt); + insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN(); + + map_fd[0] = gotox_jt_create(1, gotox_cnt); + if (map_fd[0] < 0) + goto free_insns; + map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log); + check_gotox_limit_hit(log, err); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_insns: + free(insns); +free_log: + free(log); +} + +static int gotox_jt_create_offs(const __u32 offs, __u32 cnt) +{ + struct bpf_insn_array_value val = {}; + int map_fd; + __u32 i; + + map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt); + if (!ASSERT_GE(map_fd, 0, "map_create")) + return map_fd; + + for (i = 0; i < cnt; i++) { + val.orig_off = offs[i]; + if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, + "bpf_map_update_elem")) + goto err; + } + + if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze")) + goto err; + + return map_fd; +err: + close(map_fd); + return -1; +} + +#define GOTOX_SUB_START 4 +#define GOTOX_MAIN_TGT 2 +#define GOTOX_SUB_TGT 8 +#define GOTOX_TWO_INSN_CNT 10 + +static void gotox_two_subprogs_fill(struct bpf_insn insns, __u32 jt_idx, __u32 jt_off) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_SUB_START - 1 - 1); + insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[3] = BPF_EXIT_INSN(); + + / r1 = &jt[0], by index 'jt_idx' into fd_array / + insns[GOTOX_SUB_START] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = jt_idx, + }; + insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 }; + insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off 8); + insns[7] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[9] = BPF_EXIT_INSN(); +} + +/* + * An insn_array map is not necessarily a jump table: one that tracks + * instruction offsets covers the whole program and is of no subprog. Such a + * map must not keep a program with a gotox elsewhere from loading. + / +static void check_gotox_tracker_map(void) +{ + const __u32 jt_track[] = { 0, GOTOX_MAIN_TGT, GOTOX_SUB_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 1, 0); + + map_fd[0] = gotox_jt_create_offs(jt_track, ARRAY_SIZE(jt_track)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + if (!ASSERT_OK(err, "program with a tracking map should load")) + fprintf(stderr, "verifier log: %s\n", log); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +static void check_gotox_target_other_subprog(void) +{ + const __u32 jt_main[] = { GOTOX_MAIN_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 0, 0); + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +static void check_gotox_jt_per_subprog(void) +{ + const __u32 jt_main[] = { GOTOX_MAIN_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 1, 0); + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +/* + * The spanning map is of no subprog and is dropped, and the entry the gotox + * register can reach is in the subprog of the gotox and in the jump table the + * CFG walked, so nothing unsafe is left and the program loads. + / +static void check_gotox_span_unreached_entry(void) +{ + const __u32 jt_span[] = { GOTOX_MAIN_TGT, GOTOX_SUB_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 0, 1); + + map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + if (!ASSERT_OK(err, "program with an unreachable spanning entry should load")) + fprintf(stderr, "verifier log: %s\n", log); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +#define GOTOX_FWD_GOTOX 11 +#define GOTOX_FWD_OWN_TGT 12 +#define GOTOX_FWD_SUB_START 14 +#define GOTOX_FWD_INSN_CNT 16 + +static void gotox_from_main_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_FWD_SUB_START - 1 - 1); + insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, + offsetof(struct xdp_md, ingress_ifindex)); + insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4); + + / r1 = &jt_leaves[0], by index 1 into fd_array / + insns[4] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 1, + }; + insns[5] = (struct bpf_insn) { .imm = 0 }; + insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + insns[7] = BPF_JMP_A(3); + + / r1 = &jt_own[0], by index 0 into fd_array / + insns[8] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[9] = (struct bpf_insn) { .imm = 0 }; + insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + + insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[13] = BPF_EXIT_INSN(); + insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[15] = BPF_EXIT_INSN(); +} + +static void check_gotox_target_subprog_from_main(void) +{ + const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT }; + const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START }; + struct bpf_insn insns[GOTOX_FWD_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_from_main_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +/* + * The only map of the subprog holding the gotox reaches past that subprog, so + * the subprog is left without a jump table at all. + / +static void check_gotox_jt_spans_subprogs(void) +{ + const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START }; + const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START }; + struct bpf_insn insns[GOTOX_FWD_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_from_main_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +/* + * The subprog holding the gotox has a well formed jump table of its own and + * also collects a map that reaches past its end. The spanning map is still + * rejected, even though the subprog is not left without a table. + / +static void check_gotox_jt_spans_with_own_table(void) +{ + const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT }; + const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START }; + struct bpf_insn insns[GOTOX_FWD_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_from_main_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +#define GOTOX_EDGE_MAIN_TGT 2 +#define GOTOX_EDGE_SUB_START 4 +#define GOTOX_EDGE_GOTOX 9 +#define GOTOX_EDGE_BR_TGT 10 +#define GOTOX_EDGE_JT_TGT 11 +#define GOTOX_EDGE_INSN_CNT 12 + +static void gotox_no_edge_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_EDGE_SUB_START - 1 - 1); + insns[GOTOX_EDGE_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[3] = BPF_EXIT_INSN(); + + insns[GOTOX_EDGE_SUB_START] = + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, + offsetof(struct xdp_md, ingress_ifindex)); + insns[5] = BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 4); + + / r1 = &jt_span[0], by index 0 into fd_array / + insns[6] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[7] = (struct bpf_insn) { .imm = 0 }; + insns[8] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 8); + + insns[GOTOX_EDGE_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_EDGE_BR_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[GOTOX_EDGE_JT_TGT] = BPF_EXIT_INSN(); +} + +/ + * The gotox resolves a target inside its own subprog, but out of a map that + * spans subprogs and is therefore of no subprog. The CFG never walked that + * edge, so the jump has to be rejected even though it stays in the subprog. + / +static void check_gotox_target_without_cfg_edge(void) +{ + const __u32 jt_span[] = { GOTOX_EDGE_MAIN_TGT, GOTOX_EDGE_BR_TGT }; + const __u32 jt_sub[] = { GOTOX_EDGE_JT_TGT }; + struct bpf_insn insns[GOTOX_EDGE_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_no_edge_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, + "indirect jump from insn 9 to 10 is not in the jump table of the subprog", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +#define GOTOX_SLICE_SUB_START 6 +#define GOTOX_SLICE_GOTOX 14 +#define GOTOX_SLICE_SUB_TGT 15 +#define GOTOX_SLICE_INSN_CNT 17 + +static void gotox_slice_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[1] = BPF_RAW_INSN(BPF_JMP \| BPF_CALL, 0, BPF_PSEUDO_CALL, 0, + GOTOX_SLICE_SUB_START - 1 - 1); + insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[5] = BPF_EXIT_INSN(); + + insns[GOTOX_SLICE_SUB_START] = + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, + offsetof(struct xdp_md, ingress_ifindex)); + insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1); + insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1); + insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3); + + / r1 = &jt_main[0], by index 0 into fd_array / + insns[10] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[11] = (struct bpf_insn) { .imm = 0 }; + insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2); + insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + + insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1); + insns[16] = BPF_EXIT_INSN(); +} + +static void check_gotox_index_slice_other_subprog(void) +{ + const __u32 jt_main[] = { 2, 3, 4 }; + const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT }; + struct bpf_insn insns[GOTOX_SLICE_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_slice_fill(insns); + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_log; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_log: + free(log); +} + +static int gotox_btf_create(const __u32 starts, const __u8 linkage, __u32 cnt, + struct bpf_func_info fi, struct btf pbtf) +{ + int int_id, proto_id, id; + struct btf btf; + char name[24]; + __u32 i; + + btf = btf__new_empty(); + if (!ASSERT_OK_PTR(btf, "btf__new_empty")) + return -1; + + int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED); + if (!ASSERT_GT(int_id, 0, "btf__add_int")) + goto err; + + proto_id = btf__add_func_proto(btf, int_id); + if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto")) + goto err; + + for (i = 0; i < cnt; i++) { + snprintf(name, sizeof(name), "gotox_f%u", i); + id = btf__add_func(btf, name, linkage[i], proto_id); + if (!ASSERT_GT(id, 0, "btf__add_func")) + goto err; + fi[i].insn_off = starts[i]; + fi[i].type_id = id; + } + + if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel")) + goto err; + + pbtf = btf; + return btf__fd(btf); +err: + btf__free(btf); + return -1; +} + +static void check_gotox_target_other_global_subprog(void) +{ + const __u32 starts[] = { 0, GOTOX_SUB_START }; + const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL }; + const __u32 jt_main[] = { GOTOX_MAIN_TGT }; + const __u32 jt_sub[] = { GOTOX_SUB_TGT }; + struct bpf_insn insns[GOTOX_TWO_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + struct bpf_func_info fi[2]; + struct btf btf = NULL; + int btf_fd; + char log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_two_subprogs_fill(insns, 0, 0); + + btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf); + if (btf_fd < 0) + goto free_log; + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_btf; + map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log, + btf_fd, fi, ARRAY_SIZE(fi)); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_btf: + btf__free(btf); +free_log: + free(log); +} + +#define GOTOX_CB_MAIN_TGT 6 +#define GOTOX_CB_START 8 +#define GOTOX_CB_GOTOX 11 +#define GOTOX_CB_TGT 12 +#define GOTOX_CB_INSN_CNT 14 + +static void gotox_callback_fill(struct bpf_insn insns) +{ + insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1); + /* r2 = &callback / + insns[1] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_2, + .src_reg = BPF_PSEUDO_FUNC, + .imm = GOTOX_CB_START - 1 - 1, + }; + insns[2] = (struct bpf_insn) { .imm = 0 }; + insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0); + insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0); + insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop); + insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[7] = BPF_EXIT_INSN(); + + / r1 = &jt_main[0], by index 0 into fd_array / + insns[GOTOX_CB_START] = (struct bpf_insn) { + .code = BPF_LD \| BPF_DW \| BPF_IMM, + .dst_reg = BPF_REG_1, + .src_reg = BPF_PSEUDO_MAP_IDX_VALUE, + .imm = 0, + }; + insns[9] = (struct bpf_insn) { .imm = 0 }; + insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0); + insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP \| BPF_JA \| BPF_X, BPF_REG_1, 0, 0, 0); + insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0); + insns[13] = BPF_EXIT_INSN(); +} + +static void check_gotox_callback_leaves_subprog(void) +{ + const __u32 starts[] = { 0, GOTOX_CB_START }; + const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC }; + const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT }; + const __u32 jt_cb[] = { GOTOX_CB_TGT }; + struct bpf_insn insns[GOTOX_CB_INSN_CNT]; + int map_fd[2] = { -1, -1 }; + struct bpf_func_info fi[2]; + struct btf btf = NULL; + int btf_fd; + char *log; + int err; + + log = calloc(1, GOTOX_LOG_SZ); + if (!ASSERT_OK_PTR(log, "calloc log")) + return; + + gotox_callback_fill(insns); + + btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf); + if (btf_fd < 0) + goto free_log; + + map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main)); + if (map_fd[0] < 0) + goto free_btf; + map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb)); + if (map_fd[1] < 0) + goto close_maps; + + err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log, + btf_fd, fi, ARRAY_SIZE(fi)); + ASSERT_EQ(err, -EINVAL, "program should have been rejected"); + ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)", + "verifier log"); + +close_maps: + close(map_fd[0]); + close(map_fd[1]); +free_btf: + btf__free(btf); +free_log: + free(log); +} + static void check_bpf_side(void) { check_bpf_no_lookup(); @@ -490,6 +1425,48 @@ static void __test_bpf_insn_array(void) if (test__start_subtest("bpf-side-ops")) check_bpf_side(); + + if (test__start_subtest("too-many-gotox-edges")) + check_too_many_gotox_edges(); + + if (test__start_subtest("gotox-edges-at-limit")) + check_gotox_edges_at_limit(); + + if (test__start_subtest("gotox-edges-across-subprogs")) + check_gotox_edges_across_subprogs(); + + if (test__start_subtest("gotox-tracker-map")) + check_gotox_tracker_map(); + + if (test__start_subtest("gotox-jt-spans-subprogs")) + check_gotox_jt_spans_subprogs(); + + if (test__start_subtest("gotox-jt-spans-with-own-table")) + check_gotox_jt_spans_with_own_table(); + + if (test__start_subtest("gotox-target-without-cfg-edge")) + check_gotox_target_without_cfg_edge(); + + if (test__start_subtest("gotox-target-other-subprog")) + check_gotox_target_other_subprog(); + + if (test__start_subtest("gotox-jt-per-subprog")) + check_gotox_jt_per_subprog(); + + if (test__start_subtest("gotox-span-unreached-entry")) + check_gotox_span_unreached_entry(); + + if (test__start_subtest("gotox-target-subprog-from-main")) + check_gotox_target_subprog_from_main(); + + if (test__start_subtest("gotox-index-slice-other-subprog")) + check_gotox_index_slice_other_subprog(); + + if (test__start_subtest("gotox-target-other-global-subprog")) + check_gotox_target_other_global_subprog(); + + if (test__start_subtest("gotox-callback-leaves-subprog")) + check_gotox_callback_leaves_subprog(); } #else static void __test_bpf_insn_array(void) diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c index 5b18c9a27717b..3567b29e23789 100644 --- a/tools/testing/selftests/bpf/progs/verifier_gotox.c +++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c @@ -318,7 +318,7 @@ __used static int test_subprog(void) } SEC("socket") -__failure __msg("jump table for insn 4 points outside of the subprog [0,10]") +__failure __msg("jump table of subprog starting at 0 spans multiple subprogs") __naked void jump_table_outside_subprog(void) { asm volatile (" \
19/3	2026/09/09 21:17	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 21993 output: 29 thoughts: 497 Thoughts: Analyzing the BPF Patch for Memory Safety Okay, let's dive into this patch and see if we can trigger any nasty uninitialized memory reads or leaks, the kind that only KMSAN might pick up. First, I'm going to take a really close look at the details. The changes are spread across these files: `include/linux/bpf_verifier.h`, `kernel/bpf/cfg.c`, `kernel/bpf/verifier.c`, and a bunch of selftests. * In `bpf_verifier.h`, we've got a couple of additions: a `struct bpf_iarray jt;` and a `bool jt_spans_subprogs: 1;` field in `struct bpf_subprog_info`. It looks like these are for handling jump tables. The other struct, `bpf_verifier_env`, got changes in its `cfg` member, with `u32 gotox_edges;` and `bool subprog_jts_ready;` being added. I'm guessing this relates to tracking control flow graph edges. `cfg.c` is where the core logic seems to be. We've replaced the old `jt_from_subprog()` with a more efficient `compute_subprog_jts()`, caching jump tables in subprog info. There's also some cleanup in `free_subprog_jts` and modifications to `create_jt`. `visit_gotox_insn` is now checking a `gotox_edges` limit. Importantly, `bpf_compute_scc` is using `kvcalloc` for a `dfs_pos` array. I need to keep an eye on how that's used. * The verifier is where the actual checks happen, with new functions like `jt_contains`, `reject_gotox_out_of_subprog`, and `reject_gotox_without_cfg_edge`. `check_indirect_jump` is making sure the targets stay within the subprogram and are in the jump table, which is good. * Finally, the selftests are there to, hopefully, cover some of the new functionality. Now, the memory management is where it gets interesting. I need to verify that all the memory is correctly allocated and initialized. Let's start with `bpf_verifier_env` and `bpf_subprog_info`. Where are those allocated and, crucially, cleared? That's the key. I'm going to use `grepper` and see how `subprog_info` is initialized. Let's track its lifecycle to make sure nothing is left hanging and uninitialized. I need to make sure the jump tables are properly handled as well. This all has the potential to introduce subtle issues, especially around control flow changes. Time to dive deeper.
20/3	2026/09/09 21:17	tool	grepper	0m Args: { "Expression": "subprog_info", "PathPrefix": "kernel/bpf/" } Results: { "Output": "\nFull output is too long, showing 500 out of 602 lines.\nUse more precise expression if possible.\n\nkernel/bpf/btf.c=8016=int btf_prepare_func_args(struct bpf_verifier_env env, int subprog)\n--\nkernel/bpf/btf.c-8018-\tbool is_global = subprog_aux(env, subprog)-\u003elinkage == BTF_FUNC_GLOBAL;\nkernel/bpf/btf.c:8019:\tstruct bpf_subprog_info sub = subprog_info(env, subprog);\nkernel/bpf/btf.c-8020-\tstruct bpf_verifier_log log = \u0026env-\u003elog;\n--\nkernel/bpf/cfg.c=54=static void mark_subprog_changes_pkt_data(struct bpf_verifier_env env, int off)\nkernel/bpf/cfg.c-55-{\nkernel/bpf/cfg.c:56:\tstruct bpf_subprog_info subprog;\nkernel/bpf/cfg.c-57-\n--\nkernel/bpf/cfg.c=62=static void mark_subprog_might_sleep(struct bpf_verifier_env env, int off)\nkernel/bpf/cfg.c-63-{\nkernel/bpf/cfg.c:64:\tstruct bpf_subprog_info subprog;\nkernel/bpf/cfg.c-65-\n--\nkernel/bpf/cfg.c=70=static void mark_subprog_might_throw(struct bpf_verifier_env env, int off)\nkernel/bpf/cfg.c-71-{\nkernel/bpf/cfg.c:72:\tstruct bpf_subprog_info subprog;\nkernel/bpf/cfg.c-73-\n--\nkernel/bpf/cfg.c=84=static void merge_callee_effects(struct bpf_verifier_env env, int t, int w)\nkernel/bpf/cfg.c-85-{\nkernel/bpf/cfg.c:86:\tstruct bpf_subprog_info caller, callee;\nkernel/bpf/cfg.c-87-\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env env)\nkernel/bpf/cfg.c-295-{\nkernel/bpf/cfg.c:296:\tstruct bpf_subprog_info subprog;\nkernel/bpf/cfg.c-297-\tstruct bpf_iarray jt, jt_cur;\n--\nkernel/bpf/cfg.c-337-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/cfg.c:338:\t\tjt = env-\u003esubprog_info[i].jt;\nkernel/bpf/cfg.c-339-\t\tif (jt)\n--\nkernel/bpf/cfg.c=347=static void free_subprog_jts(struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-350-\nkernel/bpf/cfg.c:351:\tfor (i = 0; i \u003c ARRAY_SIZE(env-\u003esubprog_info); i++) {\nkernel/bpf/cfg.c:352:\t\tkvfree(env-\u003esubprog_info[i].jt);\nkernel/bpf/cfg.c:353:\t\tenv-\u003esubprog_info[i].jt = NULL;\nkernel/bpf/cfg.c:354:\t\tenv-\u003esubprog_info[i].jt_spans_subprogs = false;\nkernel/bpf/cfg.c-355-\t}\n--\nkernel/bpf/cfg.c=360=create_jt(int t, struct bpf_verifier_env env)\nkernel/bpf/cfg.c-361-{\nkernel/bpf/cfg.c:362:\tstruct bpf_subprog_info subprog;\nkernel/bpf/cfg.c-363-\tstruct bpf_iarray jt;\n--\nkernel/bpf/cfg.c=471=static int visit_abnormal_return_insn(struct bpf_verifier_env env, int t)\nkernel/bpf/cfg.c-472-{\nkernel/bpf/cfg.c:473:\tstruct bpf_subprog_info subprog;\nkernel/bpf/cfg.c-474-\tstruct bpf_iarray jt;\n--\nkernel/bpf/cfg.c=632=int bpf_check_cfg(struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-650-\tex_insn_beg = env-\u003eexception_callback_subprog\nkernel/bpf/cfg.c:651:\t\t ? env-\u003esubprog_info[env-\u003eexception_callback_subprog].start\nkernel/bpf/cfg.c-652-\t\t : 0;\n--\nkernel/bpf/cfg.c-718-\tret = 0; /* cfg looks good /\nkernel/bpf/cfg.c:719:\tenv-\u003eprog-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[0].changes_pkt_data;\nkernel/bpf/cfg.c:720:\tenv-\u003eprog-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[0].might_sleep;\nkernel/bpf/cfg.c-721-\n--\nkernel/bpf/cfg.c-731- For each subprogram 'i' fill array env-\u003ecfg.insn_subprogram sub-range\nkernel/bpf/cfg.c:732: * [env-\u003esubprog_info[i].postorder_start, env-\u003esubprog_info[i+1].postorder_start)\nkernel/bpf/cfg.c-733- * with indices of 'i' instructions in postorder.\n--\nkernel/bpf/cfg.c=735=int bpf_compute_postorder(struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-751-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/cfg.c:752:\t\tenv-\u003esubprog_info[i].postorder_start = cur_postorder;\nkernel/bpf/cfg.c:753:\t\tstack[0] = env-\u003esubprog_info[i].start;\nkernel/bpf/cfg.c-754-\t\tstack_sz = 1;\n--\nkernel/bpf/cfg.c-772-\t}\nkernel/bpf/cfg.c:773:\tenv-\u003esubprog_info[i].postorder_start = cur_postorder;\nkernel/bpf/cfg.c-774-\tenv-\u003ecfg.insn_postorder = postorder;\n--\nkernel/bpf/check_btf.c=10=static int check_abnormal_return(struct bpf_verifier_env env)\n--\nkernel/bpf/check_btf.c-14-\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/check_btf.c:15:\t\tif (env-\u003esubprog_info[i].has_ld_abs) {\nkernel/bpf/check_btf.c-16-\t\t\tverbose(env, \"LD_ABS is not allowed in subprogs without BTF\\n\");\n--\nkernel/bpf/check_btf.c-18-\t\t}\nkernel/bpf/check_btf.c:19:\t\tif (env-\u003esubprog_info[i].has_tail_call) {\nkernel/bpf/check_btf.c-20-\t\t\tverbose(env, \"tail_call is not allowed in subprogs without BTF\\n\");\n--\nkernel/bpf/check_btf.c=133=static int check_btf_func(struct bpf_verifier_env env,\n--\nkernel/bpf/check_btf.c-174-\nkernel/bpf/check_btf.c:175:\t\tif (env-\u003esubprog_info[i].start != krecord[i].insn_off) {\nkernel/bpf/check_btf.c-176-\t\t\tverbose(env, \"func_info BTF section doesn't match subprog layout in BPF program\\n\");\n--\nkernel/bpf/check_btf.c-188-\t\t\tbtf_type_is_small_int(ret_type) \|\| btf_is_any_enum(ret_type);\nkernel/bpf/check_btf.c:189:\t\tif (i \u0026\u0026 !scalar_return \u0026\u0026 env-\u003esubprog_info[i].has_ld_abs) {\nkernel/bpf/check_btf.c-190-\t\t\tverbose(env, \"LD_ABS is only allowed in functions that return 'int'.\\n\");\n--\nkernel/bpf/check_btf.c-192-\t\t}\nkernel/bpf/check_btf.c:193:\t\tif (i \u0026\u0026 !scalar_return \u0026\u0026 env-\u003esubprog_info[i].has_tail_call) {\nkernel/bpf/check_btf.c-194-\t\t\tverbose(env, \"tail_call is only allowed in functions that return 'int'.\\n\");\n--\nkernel/bpf/check_btf.c-197-\nkernel/bpf/check_btf.c:198:\t\tenv-\u003esubprog_info[i].name = btf_name_by_offset(btf, type-\u003ename_off);\nkernel/bpf/check_btf.c-199-\t\tbpfptr_add(\u0026urecord, urec_size);\n--\nkernel/bpf/check_btf.c=213=static int check_btf_line(struct bpf_verifier_env env,\n--\nkernel/bpf/check_btf.c-217-\tu32 i, s, nr_linfo, ncopy, expected_size, rec_size, prev_offset = 0;\nkernel/bpf/check_btf.c:218:\tstruct bpf_subprog_info sub;\nkernel/bpf/check_btf.c-219-\tstruct bpf_line_info linfo;\n--\nkernel/bpf/check_btf.c-248-\ts = 0;\nkernel/bpf/check_btf.c:249:\tsub = env-\u003esubprog_info;\nkernel/bpf/check_btf.c-250-\tulinfo = make_bpfptr(attr-\u003eline_info, uattr.is_kernel);\n--\nkernel/bpf/check_btf.c-279-\t\t * first sub also and the first sub must have\nkernel/bpf/check_btf.c:280:\t\t * subprog_info[0].start == 0.\nkernel/bpf/check_btf.c-281-\t\t /\n--\nkernel/bpf/const_fold.c=232=int bpf_compute_const_regs(struct bpf_verifier_env env)\n--\nkernel/bpf/const_fold.c-250-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/const_fold.c:251:\t\tint start = env-\u003esubprog_info[i].start;\nkernel/bpf/const_fold.c-252-\n--\nkernel/bpf/core.c=1650=u16 bpf_out_stack_arg_cnt(const struct bpf_verifier_env env, const struct bpf_prog prog)\nkernel/bpf/core.c-1651-{\nkernel/bpf/core.c:1652:\tconst struct bpf_subprog_info sub;\nkernel/bpf/core.c-1653-\n--\nkernel/bpf/core.c-1655-\t\treturn 0;\nkernel/bpf/core.c:1656:\tsub = \u0026env-\u003esubprog_info[prog-\u003eaux-\u003efunc_idx];\nkernel/bpf/core.c-1657-\treturn sub-\u003estack_arg_cnt - bpf_in_stack_arg_cnt(sub);\n--\nkernel/bpf/diagnostics.c=797=static void bpf_diag_source(struct bpf_verifier_env env, u32 insn_idx, const char label,\n--\nkernel/bpf/diagnostics.c-805-\tconst struct bpf_line_info linfo;\nkernel/bpf/diagnostics.c:806:\tconst struct bpf_subprog_info subprog;\nkernel/bpf/diagnostics.c-807-\tstruct btf btf = env-\u003eprog-\u003eaux-\u003ebtf;\n--\nkernel/bpf/diagnostics.c-841-\tsubprog = bpf_find_containing_subprog(env, insn_idx);\nkernel/bpf/diagnostics.c:842:\tsubprogno = subprog ? subprog - env-\u003esubprog_info : -ENOENT;\nkernel/bpf/diagnostics.c-843-\tfunc = subprogno \u003e= 0 ? bpf_subprog_name(env, subprogno) : NULL;\n--\nkernel/bpf/diagnostics.c-860-\tlinfo_end = subprogno \u003e= 0 \u0026\u0026 subprogno + 1 \u003c env-\u003esubprog_cnt ?\nkernel/bpf/diagnostics.c:861:\t\t env-\u003esubprog_info[subprogno + 1].linfo_idx : env-\u003eprog-\u003eaux-\u003enr_linfo;\nkernel/bpf/diagnostics.c-862-\tfor (i = linfo_start; i \u003c linfo_end; i++) {\n--\nkernel/bpf/fixups.c=204=static int get_callee_stack_depth(struct bpf_verifier_env env,\n--\nkernel/bpf/fixups.c-211-\t\treturn -EFAULT;\nkernel/bpf/fixups.c:212:\treturn env-\u003esubprog_info[subprog].stack_depth;\nkernel/bpf/fixups.c-213-}\n--\nkernel/bpf/fixups.c=262=static void adjust_subprog_starts(struct bpf_verifier_env env, u32 off, u32 len)\n--\nkernel/bpf/fixups.c-269-\tfor (i = 0; i \u003c= env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:270:\t\tif (env-\u003esubprog_info[i].start \u003c= off)\nkernel/bpf/fixups.c-271-\t\t\tcontinue;\nkernel/bpf/fixups.c:272:\t\tenv-\u003esubprog_info[i].start += len - 1;\nkernel/bpf/fixups.c-273-\t}\n--\nkernel/bpf/fixups.c=379=static int adjust_subprog_starts_after_remove(struct bpf_verifier_env env,\n--\nkernel/bpf/fixups.c-385-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:386:\t\tif (env-\u003esubprog_info[i].start \u003e= off)\nkernel/bpf/fixups.c-387-\t\t\tbreak;\n--\nkernel/bpf/fixups.c-389-\tfor (j = i; j \u003c env-\u003esubprog_cnt; j++)\nkernel/bpf/fixups.c:390:\t\tif (env-\u003esubprog_info[j].start \u003e= off + cnt)\nkernel/bpf/fixups.c-391-\t\t\tbreak;\n--\nkernel/bpf/fixups.c-394-\t /\nkernel/bpf/fixups.c:395:\tif (env-\u003esubprog_info[j].start != off + cnt)\nkernel/bpf/fixups.c-396-\t\tj--;\n--\nkernel/bpf/fixups.c-404-\nkernel/bpf/fixups.c:405:\t\tmemmove(env-\u003esubprog_info + i,\nkernel/bpf/fixups.c:406:\t\t\tenv-\u003esubprog_info + j,\nkernel/bpf/fixups.c:407:\t\t\tsizeof(env-\u003esubprog_info) move);\nkernel/bpf/fixups.c-408-\t\tenv-\u003esubprog_cnt -= j - i;\n--\nkernel/bpf/fixups.c-427-\t\t/* convert i from \"first prog to remove\" to \"first to adjust\" /\nkernel/bpf/fixups.c:428:\t\tif (env-\u003esubprog_info[i].start == off)\nkernel/bpf/fixups.c-429-\t\t\ti++;\n--\nkernel/bpf/fixups.c-433-\tfor (; i \u003c= env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:434:\t\tenv-\u003esubprog_info[i].start -= cnt;\nkernel/bpf/fixups.c-435-\n--\nkernel/bpf/fixups.c=439=static int bpf_adj_linfo_after_remove(struct bpf_verifier_env env, u32 off,\n--\nkernel/bpf/fixups.c-489-\tfor (i = 0; i \u003c= env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:490:\t\tif (env-\u003esubprog_info[i].linfo_idx \u003e l_off) {\nkernel/bpf/fixups.c-491-\t\t\t/* program may have started in the removed region but\n--\nkernel/bpf/fixups.c-493-\t\t\t /\nkernel/bpf/fixups.c:494:\t\t\tif (env-\u003esubprog_info[i].linfo_idx \u003e= l_off + l_cnt)\nkernel/bpf/fixups.c:495:\t\t\t\tenv-\u003esubprog_info[i].linfo_idx -= l_cnt;\nkernel/bpf/fixups.c-496-\t\t\telse\nkernel/bpf/fixups.c:497:\t\t\t\tenv-\u003esubprog_info[i].linfo_idx = l_off;\nkernel/bpf/fixups.c-498-\t\t}\n--\nkernel/bpf/fixups.c=746=int bpf_convert_ctx_accesses(struct bpf_verifier_env env)\nkernel/bpf/fixups.c-747-{\nkernel/bpf/fixups.c:748:\tstruct bpf_subprog_info subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-749-\tconst struct bpf_verifier_ops ops = env-\u003eops;\n--\nkernel/bpf/fixups.c=1051=static u32 bpf_dup_subprog_starts(struct bpf_verifier_env env)\n--\nkernel/bpf/fixups.c-1057-\t\tfor (int i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:1058:\t\t\tstarts[i] = env-\u003esubprog_info[i].start;\nkernel/bpf/fixups.c-1059-\t}\n--\nkernel/bpf/fixups.c=1063=static void bpf_restore_subprog_starts(struct bpf_verifier_env env, u32 orig_starts)\n--\nkernel/bpf/fixups.c-1065-\tfor (int i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:1066:\t\tenv-\u003esubprog_info[i].start = orig_starts[i];\nkernel/bpf/fixups.c-1067-\t/* restore the start of fake 'exit' subprog as well /\nkernel/bpf/fixups.c:1068:\tenv-\u003esubprog_info[env-\u003esubprog_cnt].start = env-\u003eprog-\u003elen;\nkernel/bpf/fixups.c-1069-}\n--\nkernel/bpf/fixups.c=1071=static int jit_subprogs(struct bpf_verifier_env env)\n--\nkernel/bpf/fixups.c-1127-\t\tsubprog_start = subprog_end;\nkernel/bpf/fixups.c:1128:\t\tsubprog_end = env-\u003esubprog_info[i + 1].start;\nkernel/bpf/fixups.c-1129-\n--\nkernel/bpf/fixups.c-1167-\t\tfunc[i]-\u003eaux-\u003ename[0] = 'F';\nkernel/bpf/fixups.c:1168:\t\tfunc[i]-\u003eaux-\u003estack_depth = env-\u003esubprog_info[i].stack_depth;\nkernel/bpf/fixups.c:1169:\t\tif (env-\u003esubprog_info[i].priv_stack_mode == PRIV_STACK_ADAPTIVE)\nkernel/bpf/fixups.c-1170-\t\t\tfunc[i]-\u003eaux-\u003ejits_use_priv_stack = true;\n--\nkernel/bpf/fixups.c-1178-\t\tfunc[i]-\u003eaux-\u003ejited_linfo = prog-\u003eaux-\u003ejited_linfo;\nkernel/bpf/fixups.c:1179:\t\tfunc[i]-\u003eaux-\u003elinfo_idx = env-\u003esubprog_info[i].linfo_idx;\nkernel/bpf/fixups.c-1180-\t\tfunc[i]-\u003eaux-\u003earena = prog-\u003eaux-\u003earena;\n--\nkernel/bpf/fixups.c-1200-\t\tfunc[i]-\u003eaux-\u003enum_exentries = num_exentries;\nkernel/bpf/fixups.c:1201:\t\tfunc[i]-\u003eaux-\u003etail_call_reachable = env-\u003esubprog_info[i].tail_call_reachable;\nkernel/bpf/fixups.c:1202:\t\tfunc[i]-\u003eaux-\u003eexception_cb = env-\u003esubprog_info[i].is_exception_cb;\nkernel/bpf/fixups.c:1203:\t\tfunc[i]-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[i].changes_pkt_data;\nkernel/bpf/fixups.c:1204:\t\tfunc[i]-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[i].might_sleep;\nkernel/bpf/fixups.c-1205-\t\tfunc[i]-\u003eaux-\u003etoken = prog-\u003eaux-\u003etoken;\n--\nkernel/bpf/fixups.c=1410=int bpf_fixup_call_args(struct bpf_verifier_env env)\n--\nkernel/bpf/fixups.c-1419-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:1420:\t\tstruct bpf_subprog_info subprog = \u0026env-\u003esubprog_info[i];\nkernel/bpf/fixups.c-1421-\t\tu16 outgoing = subprog-\u003estack_arg_cnt - bpf_in_stack_arg_cnt(subprog);\n--\nkernel/bpf/fixups.c-1444-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:1445:\t\tif (bpf_in_stack_arg_cnt(\u0026env-\u003esubprog_info[i])) {\nkernel/bpf/fixups.c-1446-\t\t\tverbose(env, \"stack args are not supported in non-JITed programs\\n\");\n--\nkernel/bpf/fixups.c=1484=static int add_hidden_subprog(struct bpf_verifier_env env, struct bpf_insn patch, int len)\nkernel/bpf/fixups.c-1485-{\nkernel/bpf/fixups.c:1486:\tstruct bpf_subprog_info info = env-\u003esubprog_info;\nkernel/bpf/fixups.c-1487-\tint cnt = env-\u003esubprog_cnt;\n--\nkernel/bpf/fixups.c-1489-\nkernel/bpf/fixups.c:1490:\t/ We only reserve one slot for hidden subprogs in subprog_info. /\nkernel/bpf/fixups.c-1491-\tif (env-\u003ehidden_subprog_cnt) {\n--\nkernel/bpf/fixups.c=1513=int bpf_do_misc_fixups(struct bpf_verifier_env env)\n--\nkernel/bpf/fixups.c-1526-\tint i, ret, cnt, delta = 0, cur_subprog = 0;\nkernel/bpf/fixups.c:1527:\tstruct bpf_subprog_info subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-1528-\tu16 stack_depth = subprogs[cur_subprog].stack_depth;\n--\nkernel/bpf/fixups.c=2499=static struct bpf_prog inline_bpf_loop(struct bpf_verifier_env env,\n--\nkernel/bpf/fixups.c-2565-\t/ callback start is known only after patching /\nkernel/bpf/fixups.c:2566:\tcallback_start = env-\u003esubprog_info[callback_subprogno].start;\nkernel/bpf/fixups.c-2567-\t/ Note: insn_buf[12] is an offset of BPF_CALL_REL instruction /\n--\nkernel/bpf/fixups.c=2591=int bpf_optimize_bpf_loop(struct bpf_verifier_env env)\nkernel/bpf/fixups.c-2592-{\nkernel/bpf/fixups.c:2593:\tstruct bpf_subprog_info subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-2594-\tint i, cur_subprog = 0, cnt, delta = 0;\n--\nkernel/bpf/fixups.c-2630-\nkernel/bpf/fixups.c:2631:\tenv-\u003eprog-\u003eaux-\u003estack_depth = env-\u003esubprog_info[0].stack_depth;\nkernel/bpf/fixups.c-2632-\n--\nkernel/bpf/fixups.c=2639=int bpf_remove_fastcall_spills_fills(struct bpf_verifier_env env)\nkernel/bpf/fixups.c-2640-{\nkernel/bpf/fixups.c:2641:\tstruct bpf_subprog_info subprog = env-\u003esubprog_info;\nkernel/bpf/fixups.c-2642-\tstruct bpf_insn_aux_data aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/liveness.c=24=struct func_instance {\n--\nkernel/bpf/liveness.c-28-\tu32 subprog;\t\t/* subprog index /\nkernel/bpf/liveness.c:29:\tu32 subprog_start;\t/ cached env-\u003esubprog_info[subprog].start /\nkernel/bpf/liveness.c-30-\tu32 insn_cnt;\t\t/ cached number of insns in the function /\n--\nkernel/bpf/liveness.c=74=static struct func_instance call_instance(struct bpf_verifier_env env,\n--\nkernel/bpf/liveness.c-78-\tu32 depth = caller ? caller-\u003edepth + 1 : 0;\nkernel/bpf/liveness.c:79:\tu32 subprog_start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c-80-\tu32 lookup_key = depth \u003e 0 ? callsite : subprog_start;\n--\nkernel/bpf/liveness.c-94-\tf-\u003esubprog_start = subprog_start;\nkernel/bpf/liveness.c:95:\tf-\u003einsn_cnt = (env-\u003esubprog_info + subprog + 1)-\u003estart - subprog_start;\nkernel/bpf/liveness.c-96-\thash = instance_hash(lookup_key, depth);\n--\nkernel/bpf/liveness.c=101=static struct func_instance lookup_instance(struct bpf_verifier_env env,\n--\nkernel/bpf/liveness.c-108-\nkernel/bpf/liveness.c:109:\tsubprog_start = env-\u003esubprog_info[st-\u003eframe[frameno]-\u003esubprogno].start;\nkernel/bpf/liveness.c-110-\tcallsite = frameno \u003e 0 ? st-\u003eframe[frameno]-\u003ecallsite : subprog_start;\n--\nkernel/bpf/liveness.c=306=static void update_instance(struct bpf_verifier_env env, struct func_instance instance)\n--\nkernel/bpf/liveness.c-309-\tint insn_postorder = env-\u003ecfg.insn_postorder;\nkernel/bpf/liveness.c:310:\tstruct bpf_subprog_info subprog;\nkernel/bpf/liveness.c-311-\tbool changed;\n--\nkernel/bpf/liveness.c-313-\tinstance-\u003emust_write_initialized = true;\nkernel/bpf/liveness.c:314:\tsubprog = \u0026env-\u003esubprog_info[instance-\u003esubprog];\nkernel/bpf/liveness.c-315-\tpo_start = subprog-\u003epostorder_start;\n--\nkernel/bpf/liveness.c=407=static char fmt_subprog(struct bpf_verifier_env env, int subprog)\nkernel/bpf/liveness.c-408-{\nkernel/bpf/liveness.c:409:\tconst char name = env-\u003esubprog_info[subprog].name;\nkernel/bpf/liveness.c-410-\n--\nkernel/bpf/liveness.c=477=static void print_instance(struct bpf_verifier_env env, struct func_instance instance)\nkernel/bpf/liveness.c-478-{\nkernel/bpf/liveness.c:479:\tint start = env-\u003esubprog_info[instance-\u003esubprog].start;\nkernel/bpf/liveness.c-480-\tstruct bpf_insn insns = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/liveness.c=1517=static void print_subprog_arg_access(struct bpf_verifier_env env,\n--\nkernel/bpf/liveness.c-1522-\tstruct bpf_insn insns = env-\u003eprog-\u003einsnsi;\nkernel/bpf/liveness.c:1523:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c-1524-\tint len = info-\u003elen;\n--\nkernel/bpf/liveness.c=1607=static int compute_subprog_args(struct bpf_verifier_env env,\n--\nkernel/bpf/liveness.c-1615-\tint depth = instance-\u003edepth;\nkernel/bpf/liveness.c:1616:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c:1617:\tint po_start = env-\u003esubprog_info[subprog].postorder_start;\nkernel/bpf/liveness.c:1618:\tint end = env-\u003esubprog_info[subprog + 1].start;\nkernel/bpf/liveness.c:1619:\tint po_end = env-\u003esubprog_info[subprog + 1].postorder_start;\nkernel/bpf/liveness.c-1620-\tint len = end - start;\n--\nkernel/bpf/liveness.c=1840=static int analyze_subprog(struct bpf_verifier_env env,\n--\nkernel/bpf/liveness.c-1848-\tstruct bpf_insn insns = env-\u003eprog-\u003einsnsi;\nkernel/bpf/liveness.c:1849:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c:1850:\tint po_start = env-\u003esubprog_info[subprog].postorder_start;\nkernel/bpf/liveness.c:1851:\tint po_end = env-\u003esubprog_info[subprog + 1].postorder_start;\nkernel/bpf/liveness.c-1852-\tstruct func_instance prev_instance = NULL;\n--\nkernel/bpf/verifier.c=431=void bpf_mark_subprog_exc_cb(struct bpf_verifier_env env, int subprog)\nkernel/bpf/verifier.c-432-{\nkernel/bpf/verifier.c:433:\tstruct bpf_subprog_info info = subprog_info(env, subprog);\nkernel/bpf/verifier.c-434-\n--\nkernel/bpf/verifier.c=440=static bool subprog_is_exc_cb(struct bpf_verifier_env env, int subprog)\nkernel/bpf/verifier.c-441-{\nkernel/bpf/verifier.c:442:\treturn subprog_info(env, subprog)-\u003eis_exception_cb;\nkernel/bpf/verifier.c-443-}\n--\nkernel/bpf/verifier.c=1369=static int resize_reference_state(struct bpf_verifier_state state, size_t n)\n--\nkernel/bpf/verifier.c-1380-/ Possibly update state-\u003eallocated_stack to be at least size bytes. Also\nkernel/bpf/verifier.c:1381: * possibly update the function's high-water mark in its bpf_subprog_info.\nkernel/bpf/verifier.c-1382- /\nkernel/bpf/verifier.c=1383=static int grow_stack_state(struct bpf_verifier_env env, struct bpf_func_state state, int size)\n--\nkernel/bpf/verifier.c-1400-\t/ update known max for given subprogram /\nkernel/bpf/verifier.c:1401:\tif (env-\u003esubprog_info[state-\u003esubprogno].stack_depth \u003c size)\nkernel/bpf/verifier.c:1402:\t\tenv-\u003esubprog_info[state-\u003esubprogno].stack_depth = size;\nkernel/bpf/verifier.c-1403-\n--\nkernel/bpf/verifier.c=2352=static int cmp_subprogs(const void a, const void b)\nkernel/bpf/verifier.c-2353-{\nkernel/bpf/verifier.c:2354:\treturn ((struct bpf_subprog_info )a)-\u003estart -\nkernel/bpf/verifier.c:2355:\t ((struct bpf_subprog_info )b)-\u003estart;\nkernel/bpf/verifier.c-2356-}\n--\nkernel/bpf/verifier.c-2358-/ Find subprogram that contains instruction at 'off' /\nkernel/bpf/verifier.c:2359:struct bpf_subprog_info bpf_find_containing_subprog(struct bpf_verifier_env env, int off)\nkernel/bpf/verifier.c-2360-{\nkernel/bpf/verifier.c:2361:\tstruct bpf_subprog_info vals = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2362-\tint l, r, m;\n--\nkernel/bpf/verifier.c=2380=int bpf_find_subprog(struct bpf_verifier_env env, int off)\nkernel/bpf/verifier.c-2381-{\nkernel/bpf/verifier.c:2382:\tstruct bpf_subprog_info p;\nkernel/bpf/verifier.c-2383-\n--\nkernel/bpf/verifier.c-2386-\t\treturn -ENOENT;\nkernel/bpf/verifier.c:2387:\treturn p - env-\u003esubprog_info;\nkernel/bpf/verifier.c-2388-}\n--\nkernel/bpf/verifier.c=2390=static int add_subprog(struct bpf_verifier_env env, int off)\n--\nkernel/bpf/verifier.c-2406-\t/ determine subprog starts. The end is one before the next starts /\nkernel/bpf/verifier.c:2407:\tenv-\u003esubprog_info[env-\u003esubprog_cnt++].start = off;\nkernel/bpf/verifier.c:2408:\tsort(env-\u003esubprog_info, env-\u003esubprog_cnt,\nkernel/bpf/verifier.c:2409:\t sizeof(env-\u003esubprog_info[0]), cmp_subprogs, NULL);\nkernel/bpf/verifier.c-2410-\treturn env-\u003esubprog_cnt - 1;\n--\nkernel/bpf/verifier.c=2919=static int add_subprogs(struct bpf_verifier_env env)\nkernel/bpf/verifier.c-2920-{\nkernel/bpf/verifier.c:2921:\tstruct bpf_subprog_info subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2922-\tint i, ret, insn_cnt = env-\u003eprog-\u003elen, ex_cb_insn;\n--\nkernel/bpf/verifier.c-2968-\t\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/verifier.c:2969:\t\t\tif (env-\u003esubprog_info[i].start != ex_cb_insn)\nkernel/bpf/verifier.c-2970-\t\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=3015=static int check_subprogs(struct bpf_verifier_env env)\n--\nkernel/bpf/verifier.c-3017-\tint i, subprog_start, subprog_end, off, cur_subprog = 0;\nkernel/bpf/verifier.c:3018:\tstruct bpf_subprog_info subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3019-\tstruct bpf_insn insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=3089=static int sort_subprogs_topo(struct bpf_verifier_env env)\nkernel/bpf/verifier.c-3090-{\nkernel/bpf/verifier.c:3091:\tstruct bpf_subprog_info si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3092-\tint insn_postorder = env-\u003ecfg.insn_postorder;\n--\nkernel/bpf/verifier.c=3477=static void check_fastcall_stack_contract(struct bpf_verifier_env env,\n--\nkernel/bpf/verifier.c-3479-{\nkernel/bpf/verifier.c:3480:\tstruct bpf_subprog_info subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-3481-\tstruct bpf_insn_aux_data aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/verifier.c=4167=static int check_stack_arg_write(struct bpf_verifier_env env, struct bpf_func_state state,\n--\nkernel/bpf/verifier.c-4170-\tint max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;\nkernel/bpf/verifier.c:4171:\tstruct bpf_subprog_info subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4172-\tint spi = -off / BPF_REG_SIZE - 1;\n--\nkernel/bpf/verifier.c=4210=static int check_stack_arg_read(struct bpf_verifier_env env, struct bpf_func_state state,\n--\nkernel/bpf/verifier.c-4212-{\nkernel/bpf/verifier.c:4213:\tstruct bpf_subprog_info subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4214-\tstruct bpf_verifier_state vstate = env-\u003ecur_state;\n--\nkernel/bpf/verifier.c=5317=static int check_max_stack_depth_subprog(struct bpf_verifier_env env, int idx,\n--\nkernel/bpf/verifier.c-5320-{\nkernel/bpf/verifier.c:5321:\tstruct bpf_subprog_info subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-5322-\tstruct bpf_insn insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=5519=static int check_max_stack_depth(struct bpf_verifier_env env)\n--\nkernel/bpf/verifier.c-5522-\tstruct bpf_subprog_call_depth_info dinfo;\nkernel/bpf/verifier.c:5523:\tstruct bpf_subprog_info si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-5524-\tbool priv_stack_supported;\n--\nkernel/bpf/verifier.c=9668=static int btf_check_func_arg_match(struct bpf_verifier_env env, int subprog,\n--\nkernel/bpf/verifier.c-9671-{\nkernel/bpf/verifier.c:9672:\tstruct bpf_subprog_info sub = subprog_info(env, subprog);\nkernel/bpf/verifier.c-9673-\tstruct bpf_func_state caller = cur_func(env);\n--\nkernel/bpf/verifier.c=9830=static int push_callback_call(struct bpf_verifier_env env, struct bpf_insn insn,\n--\nkernel/bpf/verifier.c-9846-\t /\nkernel/bpf/verifier.c:9847:\tenv-\u003esubprog_info[subprog].is_cb = true;\nkernel/bpf/verifier.c-9848-\tif (bpf_pseudo_kfunc_call(insn) \u0026\u0026\n--\nkernel/bpf/verifier.c-9863-\t\t/ there is no real recursion here. timer and workqueue callbacks are async /\nkernel/bpf/verifier.c:9864:\t\tenv-\u003esubprog_info[subprog].is_async_cb = true;\nkernel/bpf/verifier.c:9865:\t\tasync_cb = push_async_cb(env, env-\u003esubprog_info[subprog].start,\nkernel/bpf/verifier.c-9866-\t\t\t\t\t insn_idx, subprog,\n--\nkernel/bpf/verifier.c-9883-\t /\nkernel/bpf/verifier.c:9884:\tcallback_state = push_stack(env, env-\u003esubprog_info[subprog].start, insn_idx, false);\nkernel/bpf/verifier.c-9885-\tif (IS_ERR(callback_state))\n--\nkernel/bpf/verifier.c=9902=static int check_func_call(struct bpf_verifier_env env, struct bpf_insn insn,\n--\nkernel/bpf/verifier.c-9905-\tstruct bpf_verifier_state state = env-\u003ecur_state;\nkernel/bpf/verifier.c:9906:\tstruct bpf_subprog_info caller_info;\nkernel/bpf/verifier.c-9907-\tu16 callee_incoming, stack_arg_cnt;\n--\nkernel/bpf/verifier.c-9934-\nkernel/bpf/verifier.c:9935:\t\tif (env-\u003esubprog_info[subprog].might_sleep \u0026\u0026 !in_sleepable_context(env)) {\nkernel/bpf/verifier.c-9936-\t\t\tverbose(env, \"sleepable global function %s() called in %s\\n\",\n--\nkernel/bpf/verifier.c-9953-\t\treturns_void = subprog_returns_void(env, subprog);\nkernel/bpf/verifier.c:9954:\t\tif (env-\u003esubprog_info[subprog].changes_pkt_data)\nkernel/bpf/verifier.c-9955-\t\t\tclear_all_pkt_pointers(env);\n--\nkernel/bpf/verifier.c-9970-\nkernel/bpf/verifier.c:9971:\t\tif (env-\u003esubprog_info[subprog].might_throw) {\nkernel/bpf/verifier.c-9972-\t\t\tstruct bpf_verifier_state branch;\n--\nkernel/bpf/verifier.c-9989-\t /\nkernel/bpf/verifier.c:9990:\tcaller_info = \u0026env-\u003esubprog_info[caller-\u003esubprogno];\nkernel/bpf/verifier.c:9991:\tcallee_incoming = bpf_in_stack_arg_cnt(\u0026env-\u003esubprog_info[subprog]);\n\n" }
21/3	2026/09/09 21:17	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 34765 output: 37 thoughts: 23 Thoughts: Here's a summary of my thought process, as requested: Tracing the Allocation of `struct bpf_verifier_env` Alright, let's dive in. My immediate thought is to trace the allocation of the `struct bpf_verifier_env` within the `bpf_check` function. I need to understand where this structure comes into being because its lifespan and how it's initialized directly impact the verification process. I'm going to follow the code's execution path within `bpf_check` to pinpoint the moment this structure is created, and ideally, how its initial values are determined. Identifying the allocator is key - is it on the stack, dynamically allocated with `kmalloc`, or something else entirely? The allocation method will hint at potential memory management concerns or performance implications within the BPF verifier.
22/3	2026/09/09 21:17	tool	grepper	0m Error: no matches Args: { "Expression": "kzalloc\\(sizeof\\(struct bpf_verifier_env", "PathPrefix": "kernel/bpf/" } Results: { "error": "no matches" }
23/3	2026/09/09 21:17	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 35014 output: 30 thoughts: 17 Thoughts: Here's my thought process, as if I were considering this myself: Tracing the Allocation of `struct bpf_verifier_env` Okay, so I need to figure out where the `struct bpf_verifier_env` gets allocated. This is crucial for understanding how the BPF verifier environment is managed and potentially optimized. My first instinct is to dive into the codebase and start with a search. I'll need to look for any allocation calls related to this structure. I'll be looking for `kmalloc`, `kzalloc`, `vmalloc`, or even custom memory management functions if the kernel module has its own. Then it's to check out the initializations of the bpf verifier env struct when allocated and see how it fits in. I'll also keep in mind that it might not be allocated directly in the common way. It could be embedded within a larger structure, allocated on the stack in specific contexts, or even managed by a memory pool. I'll have to be prepared to trace through several levels of function calls if the allocation is indirect. I'll also want to look at where this is deallocated. It's often related and understanding how memory is freed is just as important. Knowing when the `struct bpf_verifier_env` gets freed will also help with determining the scope and lifetime of the verifier environment, which can influence its design and impact.
24/3	2026/09/09 21:18	tool	grepper	0m Args: { "Expression": "bpf_check\\(", "PathPrefix": "kernel/bpf/" } Results: { "Output": "kernel/bpf/syscall.c=2917=static int bpf_prog_load(union bpf_attr attr, bpfptr_t uattr, struct bpf_log_attr attr_log)\n--\nkernel/bpf/syscall.c-3132-\t/* run eBPF verifier /\nkernel/bpf/syscall.c:3133:\terr = bpf_check(\u0026prog, attr, uattr, attr_log);\nkernel/bpf/syscall.c-3134-\tif (err \u003c 0)\n--\nkernel/bpf/verifier.c=58=static bool bpf_global_percpu_ma_set;\nkernel/bpf/verifier.c-59-\nkernel/bpf/verifier.c:60:/ bpf_check() is a static code analyzer that walks eBPF program\nkernel/bpf/verifier.c-61- * instruction by instruction and updates register/stack state.\n--\nkernel/bpf/verifier.c=21057=static enum bpf_sig_keyring bpf_classify_keyring(s32 keyring_id)\n--\nkernel/bpf/verifier.c-21071-/\nkernel/bpf/verifier.c:21072: Verify the PKCS#7 signature of a loaded program. Called from bpf_check()\nkernel/bpf/verifier.c-21073- * once the program's metadata maps have been resolved into used_maps, so\n--\nkernel/bpf/verifier.c=21080=static int bpf_prog_verify_signature(struct bpf_verifier_env env,\n--\nkernel/bpf/verifier.c-21196-\nkernel/bpf/verifier.c:21197:int bpf_check(struct bpf_prog prog, union bpf_attr attr, bpfptr_t uattr,\nkernel/bpf/verifier.c-21198-\t struct bpf_log_attr attr_log)\n--\nkernel/bpf/verifier.c-21211-\t/ 'struct bpf_verifier_env' can be global, but since it's not small,\nkernel/bpf/verifier.c:21212:\t * allocate/free it every time bpf_check() is called\nkernel/bpf/verifier.c-21213-\t */\n" }
25/3	2026/09/09 21:18	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 35953 output: 38 thoughts: 23 Thoughts: Okay, let's dive into this. Here's my take on those lines around 21212 of `kernel/bpf/verifier.c`, from an expert's perspective: Diving Deep into the BPF Verifier Alright, so I need to refresh my memory on what's going on around line 21212 of `kernel/bpf/verifier.c`. I know that area pretty well... it's probably smack dab in the middle of some crucial verification logic. Let me think, based on context... that's likely where the BPF verifier is doing something related to instruction analysis, state tracking, or perhaps even resource limits, type checking, or loop analysis – perhaps all of the above! It could be dealing with handling register states, memory access, jumps, or a particular instruction pattern that needs special handling. I'll need to see the surrounding code to be sure, but my gut feeling is that we are likely deep in the process of ensuring that BPF programs are safe and won't crash the kernel or create any security vulnerabilities. Given that it's the verifier, I'm expecting to see a lot of conditional checks, possibly recursion, and definitely a heavy reliance on a robust state machine. Let's see the code! I am eager to see how it tackles the edge cases.
26/3	2026/09/09 21:18	tool	read-file	0m Args: { "File": "kernel/bpf/verifier.c", "FirstLine": 21200, "LineCount": 40 } Results: { "Contents": "21200:\t\tu64 start_time = ktime_get_ns();\n21201:\t\tstruct bpf_verifier_env env;\n21202:\t\tint i, len, ret = -EINVAL, err;\n21203:\t\tbool is_priv;\n21204:\t\n21205:\t\tBTF_TYPE_EMIT(enum bpf_features);\n21206:\t\n21207:\t\t/ no program is valid /\n21208:\t\tif (ARRAY_SIZE(bpf_verifier_ops) == 0)\n21209:\t\t\treturn -EINVAL;\n21210:\t\n21211:\t\t/ 'struct bpf_verifier_env' can be global, but since it's not small,\n21212:\t\t * allocate/free it every time bpf_check() is called\n21213:\t\t /\n21214:\t\tenv = kvzalloc_obj(struct bpf_verifier_env, GFP_KERNEL_ACCOUNT);\n21215:\t\tif (!env)\n21216:\t\t\treturn -ENOMEM;\n21217:\t\n21218:\t\tenv-\u003ebt.env = env;\n21219:\t\tenv-\u003eprog = prog;\n21220:\t\tenv-\u003eops = bpf_verifier_ops[env-\u003eprog-\u003etype];\n21221:\t\n21222:\t\tenv-\u003eallow_ptr_leaks = bpf_allow_ptr_leaks(env-\u003eprog-\u003eaux-\u003etoken);\n21223:\t\tenv-\u003eallow_uninit_stack = bpf_allow_uninit_stack(env-\u003eprog-\u003eaux-\u003etoken);\n21224:\t\tenv-\u003ebypass_spec_v1 = bpf_bypass_spec_v1(env-\u003eprog-\u003eaux-\u003etoken);\n21225:\t\tenv-\u003ebypass_spec_v4 = bpf_bypass_spec_v4(env-\u003eprog-\u003eaux-\u003etoken);\n21226:\t\tenv-\u003ebpf_capable = is_priv = bpf_token_capable(env-\u003eprog-\u003eaux-\u003etoken, CAP_BPF);\n21227:\t\tenv-\u003esignature = attr-\u003esignature;\n21228:\t\n21229:\t\t/* user could have requested verbose verifier output\n21230:\t\t * and supplied buffer to store the verification trace\n21231:\t\t */\n21232:\t\tret = bpf_vlog_init(\u0026env-\u003elog, attr_log-\u003elevel, attr_log-\u003eubuf, attr_log-\u003esize);\n21233:\t\tif (ret)\n21234:\t\t\tgoto err_free_env;\n21235:\t\tret = bpf_diag_init(env);\n21236:\t\tif (ret)\n21237:\t\t\tgoto err_prep;\n21238:\t\tif (env-\u003esignature) {\n21239:\t\t\tret = bpf_prog_calc_tag(env-\u003eprog);\n" }
27/3	2026/09/09 21:18	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 37073 output: 33 thoughts: 760 Thoughts: Analyzing the BPF CFG/Jump Table Optimization Okay, so I'm looking at this patch and my initial thought is, `env` is allocated using `kvzalloc_obj`. That means everything in `env` is initialized to zero, which is good to know, especially for things like pointers and booleans. This gives me a baseline. Now, let's dive into what this patch is doing. It seems like it's addressing some optimizations and bug fixes related to the Control Flow Graph (CFG) and jump table processing, specifically around indirect jumps or `gotox` instructions within the BPF verifier. First, I see additions to the data structures. In `bpf_subprog_info`, `jt` (the jump table pointer) and the `jt_spans_subprogs` bitfield are added. This signals that jump tables are directly related to subprograms, and whether this jump table spans multiple subprograms. In `bpf_verifier_env::cfg`, `gotox_edges` (a `u32` for edge counting) and `subprog_jts_ready` (a boolean flag) are added. These suggest a way to manage GOTX edges and the readiness of jump tables. Looking at the code changes, `compute_subprog_jts(env)` is interesting. It iterates through `env->insn_array_maps` and figures out which subprogram a jump table item belongs to. It checks that the last entry in the jump table doesn't exceed the start of the next subprogram. It then reallocates the `subprog->jt` using `bpf_iarray_realloc` (that's going to be key to understand fully), copies the items, sorts and unifies them with `sort_insn_array_uniq`, and sets `env->cfg.subprog_jts_ready = true`. This seems to be the core logic to construct jump tables for each subprogram. `free_subprog_jts(env)` cleans up the jump tables. It frees the allocated memory and resets the corresponding flags and pointers. This is crucial for error handling, which makes sense because it's called in `bpf_check_cfg`'s error path (`err_free:`). Then there's `create_jt(int t, env)`. If jump tables aren't ready yet, it calls `compute_subprog_jts`. It checks if either `subprog->jt_spans_subprogs` is true or if `!subprog->jt` (the jump table doesn't exist yet). Then it allocates a new `bpf_iarray` with `bpf_iarray_realloc` (again, this is very important to fully grasp) and copies the items over. `visit_gotox_insn` is dealing with `gotox_edges`, calculating the number of edges and checking for overflows. Makes sense, we need to make sure we don't end up with an absurd number of jump targets. `bpf_compute_scc` does something with strongly connected components. It allocates and then frees `dfs_pos` (using `kvcalloc` and `kvfree`, which means it zero initializes the memory). That memory is likely used to track the successor index. Finally, in `verifier.c`, there are new functions: `jt_contains` (for binary search within the sorted jump table), and `reject_gotox_out_of_subprog`, `reject_gotox_without_cfg_edge`. There's also `check_indirect_jump`, which looks at bounds and makes sure that a jump target is contained within the jump table. I'm definitely going to need to track down what exactly `bpf_iarray_realloc` does. It seems critical for this whole thing. I'll check the definition and see how it works. I'm also interested in how this interacts with the CFG analysis overall. So far, the patch seems to be streamlining the management of jump tables.
28/3	2026/09/09 21:18	tool	grepper	0m Args: { "Expression": "bpf_iarray_realloc", "PathPrefix": "kernel/bpf/" } Results: { "Output": "kernel/bpf/cfg.c=163=static int visit_func_call_insn(int t, struct bpf_insn insns,\n--\nkernel/bpf/cfg.c-187-\nkernel/bpf/cfg.c:188:struct bpf_iarray bpf_iarray_realloc(struct bpf_iarray old, size_t n_elem)\nkernel/bpf/cfg.c-189-{\n--\nkernel/bpf/cfg.c=261=static struct bpf_iarray jt_from_map(struct bpf_map map)\n--\nkernel/bpf/cfg.c-266-\nkernel/bpf/cfg.c:267:\tjt = bpf_iarray_realloc(NULL, map-\u003emax_entries);\nkernel/bpf/cfg.c-268-\tif (!jt)\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-324-\t\told_cnt = subprog-\u003ejt ? subprog-\u003ejt-\u003ecnt : 0;\nkernel/bpf/cfg.c:325:\t\tjt = bpf_iarray_realloc(subprog-\u003ejt, old_cnt + jt_cur-\u003ecnt);\nkernel/bpf/cfg.c-326-\t\tif (!jt) {\n--\nkernel/bpf/cfg.c=360=create_jt(int t, struct bpf_verifier_env env)\n--\nkernel/bpf/cfg.c-395-\nkernel/bpf/cfg.c:396:\tjt = bpf_iarray_realloc(NULL, subprog-\u003ejt-\u003ecnt);\nkernel/bpf/cfg.c-397-\tif (!jt)\n--\nkernel/bpf/cfg.c=471=static int visit_abnormal_return_insn(struct bpf_verifier_env env, int t)\n--\nkernel/bpf/cfg.c-478-\nkernel/bpf/cfg.c:479:\tjt = bpf_iarray_realloc(NULL, 2);\nkernel/bpf/cfg.c-480-\tif (!jt)\n--\nkernel/bpf/verifier.c=18211=static int check_indirect_jump(struct bpf_verifier_env env, struct bpf_insn insn)\n--\nkernel/bpf/verifier.c-18244-\tif (!env-\u003egotox_tmp_buf \|\| env-\u003egotox_tmp_buf-\u003ecnt \u003c max_index - min_index + 1) {\nkernel/bpf/verifier.c:18245:\t\tenv-\u003egotox_tmp_buf = bpf_iarray_realloc(env-\u003egotox_tmp_buf,\nkernel/bpf/verifier.c-18246-\t\t\t\t\t\t max_index - min_index + 1);\n--\nkernel/bpf/verifier.c=21197=int bpf_check(struct bpf_prog *prog, union bpf_attr attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-21273-\t\tenv-\u003einsn_aux_data[i].orig_idx = i;\nkernel/bpf/verifier.c:21274:\tenv-\u003esucc = bpf_iarray_realloc(NULL, 2);\nkernel/bpf/verifier.c-21275-\tif (!env-\u003esucc)\n" }
29/3	2026/09/09 21:18	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 39631 output: 36 thoughts: 25 Thoughts: Here's my summary, as though I'm thinking through this process: Diving into `bpf_iarray_realloc` Okay, time to dig into the guts of the BPF verifier again. I want to see how the kernel handles reallocating the instruction array, specifically focusing on `bpf_iarray_realloc` within `kernel/bpf/cfg.c`. I know this function is crucial for program modification and optimization, especially when instructions are inserted or deleted during the verification process. I need to understand the mechanics of how it’s done. What are the reallocation strategies in play? What's the overhead? Are there any potential vulnerabilities to keep in mind? I need to analyze its implications for overall BPF program security and efficiency. Time to see how this function handles things under the hood.
30/3	2026/09/09 21:18	tool	read-file	0m Args: { "File": "kernel/bpf/cfg.c", "FirstLine": 185, "LineCount": 40 } Results: { "Contents": " 185:\t\treturn ret;\n 186:\t}\n 187:\t\n 188:\tstruct bpf_iarray bpf_iarray_realloc(struct bpf_iarray old, size_t n_elem)\n 189:\t{\n 190:\t\tsize_t new_size = sizeof(struct bpf_iarray) + n_elem * sizeof(old-\u003eitems[0]);\n 191:\t\tstruct bpf_iarray new;\n 192:\t\n 193:\t\tnew = kvrealloc(old, new_size, GFP_KERNEL_ACCOUNT);\n 194:\t\tif (!new) {\n 195:\t\t\t/ this is what callers always want, so simplify the call site /\n 196:\t\t\tkvfree(old);\n 197:\t\t\treturn NULL;\n 198:\t\t}\n 199:\t\n 200:\t\tnew-\u003ecnt = n_elem;\n 201:\t\treturn new;\n 202:\t}\n 203:\t\n 204:\tstatic int copy_insn_array(struct bpf_map map, u32 start, u32 end, u32 items)\n 205:\t{\n 206:\t\tstruct bpf_insn_array_value value;\n 207:\t\tu32 i;\n 208:\t\n 209:\t\tfor (i = start; i \u003c= end; i++) {\n 210:\t\t\tvalue = map-\u003eops-\u003emap_lookup_elem(map, \u0026i);\n 211:\t\t\t/\n 212:\t\t\t map_lookup_elem of an array map will never return an error,\n 213:\t\t\t * but not checking it makes some static analysers to worry\n 214:\t\t\t /\n 215:\t\t\tif (IS_ERR(value))\n 216:\t\t\t\treturn PTR_ERR(value);\n 217:\t\t\telse if (!value)\n 218:\t\t\t\treturn -EINVAL;\n 219:\t\t\titems[i - start] = value-\u003exlated_off;\n 220:\t\t}\n 221:\t\treturn 0;\n 222:\t}\n 223:\t\n 224:\tstatic int cmp_ptr_to_u32(const void a, const void *b)\n" }
31/3	2026/09/09 21:18	llm	kmsan-evaluator	0m Model: gemini-3.7-flash Args: null Results: null Tokens: input: 40474 output: 302 thoughts: 1267 Thoughts: KMSAN Reasoning: BPF Verifier Indirect Jump Table Validation Okay, let's step through this. I'm focusing on the potential for uninitialized memory reads or leaks here, which is what KMSAN is concerned with. Looking at `compute_subprog_jts`, I see it's all about analyzing and merging indirect jump tables. The core of this function is iterating through instruction array maps, figuring out which subprogram each jump table (`jt`) belongs to, and then merging those tables. The critical part is the reallocation and merging logic: `bpf_iarray_realloc` and `memcpy`. The key is that `bpf_iarray_realloc` always handles allocation correctly, and if it fails, handles that failure. I need to be sure that the `memcpy` is correct. We allocate the correct amount of space: `old_cnt + jt_cur->cnt`. Then, `memcpy` copies the `jt_cur->cnt` items into the newly allocated and initialized space. The previous data, `old_cnt` is correctly preserved by `kvrealloc`. This is confirmed when we look at `create_jt` where we perform a similar operation. Next, I'm checking `bpf_compute_scc`. The `dfs_pos` array is initialized with `kvcalloc`, which is good. That guarantees a zero-initialized state. All accesses are within bounds; `dfs_sz` is always within the valid range. Now, a broader check. Am I sending anything from the kernel to userspace that could be uninitialized? No. This code is purely inside the BPF verifier. The verifier itself is static analysis. The verifier logs might use formatted strings, but the parameters are things like offsets and instruction indices, which are all derived from the processed instructions themselves. Let's summarize the checks: 1. Kernel structures to userspace? None. No data is sent to userspace. This code is purely internal verifier logic. 2. Uninitialized variable use? No. Everything is explicitly initialized: `dfs_pos` with `kvcalloc`, etc. The code either does zero initialization or copies explicitly with `memcpy`. 3. Complex allocations? All structures either use zero-allocation functions or `memcpy`, which ensures proper initialization and prevents uninitialized reads. 4. Bounds/Length calculations? Yes. We prevent out-of-bounds indirect jumps and enforce edge limits, as well as checking the bounds of our data, such as for the jump tables. Therefore, because this patch only modifies the BPF verifier's CFG analysis and jump table validation, and uses fully zeroed/copied allocations (`kvcalloc`, `memcpy`, `kvzalloc_obj`), and doesn't introduce or modify any uninitialized memory reads or information leaks to userspace, `NeedsKMSAN` is `false`. Standard KASAN fuzzing, with all sanitizers and lockdep enabled, is sufficient.
32/3	2026/09/09 21:18	tool	set-results	0m Args: { "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.", "NeedsKMSAN": false } Results: { "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.", "NeedsKMSAN": false }

{
  "EnableConfigs": [
    "BPF_SYSCALL"
  ],
  "FocusSymbols": [
    "bpf_check_cfg",
    "bpf_compute_scc",
    "check_indirect_jump",
    "compute_subprog_jts",
    "create_jt",
    "visit_gotox_insn",
    "reject_gotox_out_of_subprog",
    "reject_gotox_without_cfg_edge",
    "jt_contains"
  ],
  "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.",
  "NeedsKMSAN": false,
  "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.",
  "WorthFuzzing": true
}

{
  "PatchDiff": "commit e16eef8d5994f14c9cd79ed40d9a24d4c4001f83\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate:   Wed Sep 9 21:17:23 2026 +0000\n\n    syz-cluster: applied patch under review\n\ndiff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h\nindex 36b65797877d0..baf2e17d7019b 100644\n--- a/include/linux/bpf_verifier.h\n+++ b/include/linux/bpf_verifier.h\n@@ -805,6 +805,7 @@ struct bpf_subprog_info {\n \tu32 linfo_idx; /* The idx to the main_prog-\u003eaux-\u003elinfo */\n \tu32 postorder_start; /* The idx to the env-\u003ecfg.insn_postorder */\n \tu32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */\n+\tstruct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */\n \tu16 stack_depth; /* max. stack depth used by this function */\n \tu16 stack_extra;\n \tu32 insns_total;\n@@ -825,6 +826,7 @@ struct bpf_subprog_info {\n \tbool keep_fastcall_stack: 1;\n \tbool changes_pkt_data: 1;\n \tbool might_sleep: 1;\n+\tbool jt_spans_subprogs: 1;\n \tu8 arg_cnt:4;\n \n \tenum priv_stack_mode priv_stack_mode;\n@@ -977,6 +979,8 @@ struct bpf_verifier_env {\n \t\tint cur_stack;\n \t\t/* current position in the insn_postorder vector */\n \t\tint cur_postorder;\n+\t\tu32 gotox_edges;\n+\t\tbool subprog_jts_ready;\n \t} cfg;\n \tstruct backtrack_state bt;\n \tstruct bpf_jmp_history_entry *cur_hist_ent;\ndiff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c\nindex 842c7d1eabccc..879587af8d086 100644\n--- a/kernel/bpf/cfg.c\n+++ b/kernel/bpf/cfg.c\n@@ -9,6 +9,8 @@\n \n #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)\n \n+#define BPF_MAX_GOTOX_EDGES\tBPF_COMPLEXITY_LIMIT_INSNS\n+\n /* non-recursive DFS pseudo code\n  * 1  procedure DFS-iterative(G,v):\n  * 2      label v as discovered\n@@ -284,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)\n }\n \n /*\n- * Find and collect all maps which fit in the subprog. Return the result as one\n- * combined jump table in jt-\u003eitems (allocated with kvcalloc)\n+ * Collect the jump table of every subprogram that has one, as the combined\n+ * table of all maps whose targets land inside that subprogram. All gotox\n+ * instructions of a subprogram share the same table, so this is done in a\n+ * single pass over the maps rather than once per gotox.\n  */\n-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,\n-\t\t\t\t\t  int subprog_start, int subprog_end)\n+static int compute_subprog_jts(struct bpf_verifier_env *env)\n {\n-\tstruct bpf_iarray *jt = NULL;\n+\tstruct bpf_subprog_info *subprog;\n+\tstruct bpf_iarray *jt, *jt_cur;\n \tstruct bpf_map *map;\n-\tstruct bpf_iarray *jt_cur;\n+\tu32 old_cnt;\n \tint i;\n \n \tfor (i = 0; i \u003c env-\u003einsn_array_map_cnt; i++) {\n@@ -303,72 +307,97 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,\n \t\tmap = env-\u003einsn_array_maps[i];\n \n \t\tjt_cur = jt_from_map(map);\n-\t\tif (IS_ERR(jt_cur)) {\n-\t\t\tkvfree(jt);\n-\t\t\treturn jt_cur;\n+\t\tif (IS_ERR(jt_cur))\n+\t\t\treturn PTR_ERR(jt_cur);\n+\n+\t\tsubprog = bpf_find_containing_subprog(env, jt_cur-\u003eitems[0]);\n+\t\tif (!subprog) {\n+\t\t\tkvfree(jt_cur);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (jt_cur-\u003eitems[jt_cur-\u003ecnt - 1] \u003e= (subprog + 1)-\u003estart) {\n+\t\t\tsubprog-\u003ejt_spans_subprogs = true;\n+\t\t\tkvfree(jt_cur);\n+\t\t\tcontinue;\n \t\t}\n \n-\t\t/*\n-\t\t * This is enough to check one element. The full table is\n-\t\t * checked to fit inside the subprog later in create_jt()\n-\t\t */\n-\t\tif (jt_cur-\u003eitems[0] \u003e= subprog_start \u0026\u0026 jt_cur-\u003eitems[0] \u003c subprog_end) {\n-\t\t\tu32 old_cnt = jt ? jt-\u003ecnt : 0;\n-\t\t\tjt = bpf_iarray_realloc(jt, old_cnt + jt_cur-\u003ecnt);\n-\t\t\tif (!jt) {\n-\t\t\t\tkvfree(jt_cur);\n-\t\t\t\treturn ERR_PTR(-ENOMEM);\n-\t\t\t}\n-\t\t\tmemcpy(jt-\u003eitems + old_cnt, jt_cur-\u003eitems, jt_cur-\u003ecnt \u003c\u003c 2);\n+\t\told_cnt = subprog-\u003ejt ? subprog-\u003ejt-\u003ecnt : 0;\n+\t\tjt = bpf_iarray_realloc(subprog-\u003ejt, old_cnt + jt_cur-\u003ecnt);\n+\t\tif (!jt) {\n+\t\t\tsubprog-\u003ejt = NULL;\n+\t\t\tkvfree(jt_cur);\n+\t\t\treturn -ENOMEM;\n \t\t}\n+\t\tmemcpy(jt-\u003eitems + old_cnt, jt_cur-\u003eitems, jt_cur-\u003ecnt \u003c\u003c 2);\n+\t\tsubprog-\u003ejt = jt;\n \n \t\tkvfree(jt_cur);\n \t}\n \n-\tif (!jt) {\n-\t\tverbose(env, \"no jump tables found for subprog starting at %u\\n\", subprog_start);\n-\t\tbpf_diag_program_structure(\n-\t\t\tenv, subprog_start, \"missing jump table\",\n-\t\t\t\"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.\",\n-\t\t\t\"No jump table was found for the subprogram that starts at instruction %u.\",\n-\t\t\tsubprog_start);\n-\t\treturn ERR_PTR(-EINVAL);\n+\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\n+\t\tjt = env-\u003esubprog_info[i].jt;\n+\t\tif (jt)\n+\t\t\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\n \t}\n \n-\tjt-\u003ecnt = sort_insn_array_uniq(jt-\u003eitems, jt-\u003ecnt);\n-\treturn jt;\n+\tenv-\u003ecfg.subprog_jts_ready = true;\n+\treturn 0;\n+}\n+\n+static void free_subprog_jts(struct bpf_verifier_env *env)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(env-\u003esubprog_info); i++) {\n+\t\tkvfree(env-\u003esubprog_info[i].jt);\n+\t\tenv-\u003esubprog_info[i].jt = NULL;\n+\t\tenv-\u003esubprog_info[i].jt_spans_subprogs = false;\n+\t}\n+\tenv-\u003ecfg.subprog_jts_ready = false;\n }\n \n static struct bpf_iarray *\n create_jt(int t, struct bpf_verifier_env *env)\n {\n \tstruct bpf_subprog_info *subprog;\n-\tint subprog_start, subprog_end;\n \tstruct bpf_iarray *jt;\n-\tint i;\n+\tint subprog_start, err;\n+\n+\tif (!env-\u003ecfg.subprog_jts_ready) {\n+\t\terr = compute_subprog_jts(env);\n+\t\tif (err)\n+\t\t\treturn ERR_PTR(err);\n+\t}\n \n \tsubprog = bpf_find_containing_subprog(env, t);\n \tsubprog_start = subprog-\u003estart;\n-\tsubprog_end = (subprog + 1)-\u003estart;\n-\tjt = jt_from_subprog(env, subprog_start, subprog_end);\n-\tif (IS_ERR(jt))\n-\t\treturn jt;\n \n-\t/* Check that the every element of the jump table fits within the given subprogram */\n-\tfor (i = 0; i \u003c jt-\u003ecnt; i++) {\n-\t\tif (jt-\u003eitems[i] \u003c subprog_start || jt-\u003eitems[i] \u003e= subprog_end) {\n-\t\t\tverbose(env, \"jump table for insn %d points outside of the subprog [%u,%u]\\n\",\n-\t\t\t\t\tt, subprog_start, subprog_end);\n-\t\t\tbpf_diag_program_structure(\n-\t\t\t\tenv, t, \"jump table target out of range\",\n-\t\t\t\t\"Keep every jump-table target inside the same subprogram.\",\n-\t\t\t\t\"The jump table for instruction %d points outside subprogram range [%u,%u).\",\n-\t\t\t\tt, subprog_start, subprog_end);\n-\t\t\tkvfree(jt);\n-\t\t\treturn ERR_PTR(-EINVAL);\n-\t\t}\n+\tif (subprog-\u003ejt_spans_subprogs) {\n+\t\tverbose(env, \"jump table of subprog starting at %u spans multiple subprogs\\n\",\n+\t\t\tsubprog_start);\n+\t\tbpf_diag_program_structure(\n+\t\t\tenv, subprog_start, \"jump table spans subprograms\",\n+\t\t\t\"Keep every entry of a jump table inside one subprogram.\",\n+\t\t\t\"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.\",\n+\t\t\tsubprog_start, (subprog + 1)-\u003estart);\n+\t\treturn ERR_PTR(-EINVAL);\n+\t}\n+\n+\tif (!subprog-\u003ejt) {\n+\t\tverbose(env, \"no jump tables found for subprog starting at %u\\n\", subprog_start);\n+\t\tbpf_diag_program_structure(\n+\t\t\tenv, subprog_start, \"missing jump table\",\n+\t\t\t\"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.\",\n+\t\t\t\"No jump table was found for the subprogram that starts at instruction %u.\",\n+\t\t\tsubprog_start);\n+\t\treturn ERR_PTR(-EINVAL);\n \t}\n \n+\tjt = bpf_iarray_realloc(NULL, subprog-\u003ejt-\u003ecnt);\n+\tif (!jt)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\tmemcpy(jt-\u003eitems, subprog-\u003ejt-\u003eitems, subprog-\u003ejt-\u003ecnt \u003c\u003c 2);\n+\n \treturn jt;\n }\n \n@@ -388,6 +417,19 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)\n \t\t\treturn PTR_ERR(jt);\n \n \t\tenv-\u003einsn_aux_data[t].jt = jt;\n+\n+\t\tif (check_add_overflow(env-\u003ecfg.gotox_edges, jt-\u003ecnt,\n+\t\t\t\t       \u0026env-\u003ecfg.gotox_edges) ||\n+\t\t    env-\u003ecfg.gotox_edges \u003e BPF_MAX_GOTOX_EDGES) {\n+\t\t\tverbose(env, \"number of indirect jump edges in the program exceeds %u\\n\",\n+\t\t\t\tBPF_MAX_GOTOX_EDGES);\n+\t\t\tbpf_diag_program_structure(\n+\t\t\t\tenv, t, \"too many indirect jump edges\",\n+\t\t\t\t\"Reduce the number of indirect jumps, or the number of distinct targets they can reach.\",\n+\t\t\t\t\"The program has more than %u indirect jump edges in total, counted over every gotox instruction.\",\n+\t\t\t\tBPF_MAX_GOTOX_EDGES);\n+\t\t\treturn -E2BIG;\n+\t\t}\n \t}\n \n \tmark_prune_point(env, t);\n@@ -678,6 +720,7 @@ int bpf_check_cfg(struct bpf_verifier_env *env)\n \tenv-\u003eprog-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[0].might_sleep;\n \n err_free:\n+\tfree_subprog_jts(env);\n \tkvfree(insn_state);\n \tkvfree(insn_stack);\n \tenv-\u003ecfg.insn_state = env-\u003ecfg.insn_stack = NULL;\n@@ -749,7 +792,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n \tconst u32 insn_cnt = env-\u003eprog-\u003elen;\n \tint stack_sz, dfs_sz, err = 0;\n-\tu32 *stack, *pre, *low, *dfs;\n+\tu32 *stack, *pre, *low, *dfs, *dfs_pos;\n \tu32 i, j, t, w;\n \tu32 next_preorder_num;\n \tu32 next_scc_id;\n@@ -762,13 +805,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \t * - 'stack' accumulates vertices in DFS order, see invariant comment below;\n \t * - 'pre[t] == p' =\u003e preorder number of vertex 't' is 'p';\n \t * - 'low[t] == n' =\u003e smallest preorder number of the vertex reachable from 't' is 'n';\n-\t * - 'dfs' DFS traversal stack, used to emulate explicit recursion.\n+\t * - 'dfs' DFS traversal stack, used to emulate explicit recursion;\n+\t * - 'dfs_pos[k] == j' =\u003e the frame 'dfs[k]' resumes visiting its\n+\t *   successors at index 'j'.\n \t */\n \tstack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tpre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tlow = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);\n \tdfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);\n-\tif (!stack || !pre || !low || !dfs) {\n+\tdfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);\n+\tif (!stack || !pre || !low || !dfs || !dfs_pos) {\n \t\terr = -ENOMEM;\n \t\tgoto exit;\n \t}\n@@ -851,6 +897,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \t\tstack_sz = 0;\n \t\tdfs_sz = 1;\n \t\tdfs[0] = i;\n+\t\tdfs_pos[0] = 0;\n dfs_continue:\n \t\twhile (dfs_sz) {\n \t\t\tw = dfs[dfs_sz - 1];\n@@ -860,13 +907,37 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \t\t\t\tnext_preorder_num++;\n \t\t\t\tstack[stack_sz++] = w;\n \t\t\t}\n-\t\t\t/* Visit 'w' successors */\n+\t\t\t/*\n+\t\t\t * Visit 'w' successors, resuming at the successor this\n+\t\t\t * frame last descended into. Restarting the scan at zero\n+\t\t\t * on every return to 'w' would examine each successor\n+\t\t\t * once per descent, i.e. quadratic in the number of\n+\t\t\t * successors, which for a gotox is the size of the jump\n+\t\t\t * table.\n+\t\t\t *\n+\t\t\t * Re-folding the successors before that index would be a\n+\t\t\t * no-op. Such a successor 's' has 'pre[s] != 0' by then,\n+\t\t\t * so it is never pushed onto 'dfs' again, and low[s] can\n+\t\t\t * only decrease while 's' is the top of 'dfs'. If 's' is\n+\t\t\t * still on 'dfs' it sits below 'w' and cannot become the\n+\t\t\t * top before 'w' is popped; otherwise the only remaining\n+\t\t\t * write to low[s] is the pop of its SCC, setting it to\n+\t\t\t * NOT_ON_STACK, for which the min below is a no-op.\n+\t\t\t */\n \t\t\tsucc = bpf_insn_successors(env, w);\n-\t\t\tfor (j = 0; j \u003c succ-\u003ecnt; ++j) {\n+\t\t\tfor (j = dfs_pos[dfs_sz - 1]; j \u003c succ-\u003ecnt; ++j) {\n \t\t\t\tif (pre[succ-\u003eitems[j]]) {\n \t\t\t\t\tlow[w] = min(low[w], low[succ-\u003eitems[j]]);\n \t\t\t\t} else {\n-\t\t\t\t\tdfs[dfs_sz++] = succ-\u003eitems[j];\n+\t\t\t\t\t/*\n+\t\t\t\t\t * Resume at 'j', not 'j + 1': the successor\n+\t\t\t\t\t * is revisited once its DFS completes, to\n+\t\t\t\t\t * fold its low[] into low[w].\n+\t\t\t\t\t */\n+\t\t\t\t\tdfs_pos[dfs_sz - 1] = j;\n+\t\t\t\t\tdfs_pos[dfs_sz] = 0;\n+\t\t\t\t\tdfs[dfs_sz] = succ-\u003eitems[j];\n+\t\t\t\t\tdfs_sz++;\n \t\t\t\t\tgoto dfs_continue;\n \t\t\t\t}\n \t\t\t}\n@@ -916,5 +987,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)\n \tkvfree(pre);\n \tkvfree(low);\n \tkvfree(dfs);\n+\tkvfree(dfs_pos);\n \treturn err;\n }\ndiff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c\nindex 72a3f5998dd27..45234e2fbee64 100644\n--- a/kernel/bpf/verifier.c\n+++ b/kernel/bpf/verifier.c\n@@ -18165,11 +18165,56 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,\n \treturn 0;\n }\n \n+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */\n+static bool jt_contains(const struct bpf_iarray *jt, u32 target)\n+{\n+\tint l = 0, r = jt-\u003ecnt - 1, m;\n+\n+\twhile (l \u003c= r) {\n+\t\tm = l + (r - l) / 2;\n+\t\tif (jt-\u003eitems[m] == target)\n+\t\t\treturn true;\n+\t\tif (jt-\u003eitems[m] \u003c target)\n+\t\t\tl = m + 1;\n+\t\telse\n+\t\t\tr = m - 1;\n+\t}\n+\treturn false;\n+}\n+\n+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,\n+\t\t\t\t       u32 subprog_start, u32 subprog_end)\n+{\n+\tverbose(env, \"indirect jump from insn %d to %u leaves the subprog [%u,%u)\\n\",\n+\t\t     env-\u003einsn_idx, target, subprog_start, subprog_end);\n+\tbpf_diag_program_structure(\n+\t\tenv, env-\u003einsn_idx, \"indirect jump leaves subprogram\",\n+\t\t\"Keep every reachable jump-table target inside the subprogram of the indirect jump.\",\n+\t\t\"Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).\",\n+\t\tenv-\u003einsn_idx, target, subprog_start, subprog_end);\n+\treturn -EINVAL;\n+}\n+\n+static int reject_gotox_without_cfg_edge(struct bpf_verifier_env *env, u32 target)\n+{\n+\tverbose(env, \"indirect jump from insn %d to %u is not in the jump table of the subprog\\n\",\n+\t\t     env-\u003einsn_idx, target);\n+\tbpf_diag_program_structure(\n+\t\tenv, env-\u003einsn_idx, \"indirect jump target without CFG edge\",\n+\t\t\"Resolve indirect jumps through a jump table whose entries all fall inside the subprogram of the jump.\",\n+\t\t\"Instruction %d can jump indirectly to instruction %u, which is not part of the jump table of its subprogram.\",\n+\t\tenv-\u003einsn_idx, target);\n+\treturn -EINVAL;\n+}\n+\n /* gotox *dst_reg */\n static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n {\n \tstruct bpf_verifier_state *other_branch;\n+\tstruct bpf_subprog_info *subprog;\n+\tu32 subprog_start, subprog_end;\n \tstruct bpf_reg_state *dst_reg;\n+\tstruct bpf_iarray *jt;\n \tstruct bpf_map *map;\n \tu32 min_index, max_index;\n \tint err = 0;\n@@ -18212,6 +18257,26 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in\n \t\treturn -EINVAL;\n \t}\n \n+\tsubprog = bpf_find_containing_subprog(env, env-\u003einsn_idx);\n+\tif (verifier_bug_if(!subprog, env, \"no subprog contains insn %d\", env-\u003einsn_idx))\n+\t\treturn -EFAULT;\n+\tsubprog_start = subprog-\u003estart;\n+\tsubprog_end = (subprog + 1)-\u003estart;\n+\n+\tjt = env-\u003einsn_aux_data[env-\u003einsn_idx].jt;\n+\tif (verifier_bug_if(!jt, env, \"no jump table for insn %d\", env-\u003einsn_idx))\n+\t\treturn -EFAULT;\n+\n+\tfor (i = 0; i \u003c n; i++) {\n+\t\tu32 target = env-\u003egotox_tmp_buf-\u003eitems[i];\n+\n+\t\tif (target \u003c subprog_start || target \u003e= subprog_end)\n+\t\t\treturn reject_gotox_out_of_subprog(env, target, subprog_start,\n+\t\t\t\t\t\t\t   subprog_end);\n+\t\tif (!jt_contains(jt, target))\n+\t\t\treturn reject_gotox_without_cfg_edge(env, target);\n+\t}\n+\n \tfor (i = 0; i \u003c n - 1; i++) {\n \t\tmark_indirect_target(env, env-\u003egotox_tmp_buf-\u003eitems[i]);\n \t\tother_branch = push_stack(env, env-\u003egotox_tmp_buf-\u003eitems[i],\ndiff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\nindex 0222a9a5d0761..d5a831a75d820 100644\n--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\n+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c\n@@ -1,6 +1,7 @@\n // SPDX-License-Identifier: GPL-2.0\n \n #include \u003cbpf/bpf.h\u003e\n+#include \u003cbpf/btf.h\u003e\n #include \u003ctest_progs.h\u003e\n \n #if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)\n@@ -453,6 +454,940 @@ static void check_bpf_no_lookup(void)\n \tclose(map_fd);\n }\n \n+#define GOTOX_CNT_AT_LIMIT\t1000\n+#define GOTOX_LOG_SZ\t\t(256 * 1024)\n+\n+static const char gotox_limit_msg[] =\n+\t\"number of indirect jump edges in the program exceeds\";\n+\n+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)\n+{\n+\t/* the run of gotox itself, plus the exit block right after it */\n+\tconst __u32 jt_cnt = gotox_cnt + 1;\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c jt_cnt; i++) {\n+\t\tval.orig_off = first_gotox + i;\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t       \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,\n+\t\t\t\t int *fd_array, __u32 fd_array_cnt, char *log,\n+\t\t\t\t int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)\n+{\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts);\n+\tint prog_fd;\n+\n+\tlog[0] = 0;\n+\topts.fd_array = fd_array;\n+\topts.fd_array_cnt = fd_array_cnt;\n+\topts.log_buf = log;\n+\topts.log_size = GOTOX_LOG_SZ;\n+\topts.log_level = 1;\n+\tif (fi_cnt) {\n+\t\topts.prog_btf_fd = btf_fd;\n+\t\topts.func_info = fi;\n+\t\topts.func_info_cnt = fi_cnt;\n+\t\topts.func_info_rec_size = sizeof(*fi);\n+\t}\n+\n+\tprog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, \"GPL\", insns, insn_cnt, \u0026opts);\n+\tif (prog_fd \u003e= 0) {\n+\t\tclose(prog_fd);\n+\t\treturn 0;\n+\t}\n+\treturn prog_fd;\n+}\n+\n+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,\n+\t\t\t   int *fd_array, __u32 fd_array_cnt, char *log)\n+{\n+\treturn gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,\n+\t\t\t\t     -1, NULL, 0);\n+}\n+\n+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */\n+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)\n+{\n+\t__u32 i;\n+\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);\n+\tfor (i = 1; i \u003c= gotox_cnt; i++)\n+\t\tinsns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+}\n+\n+static void check_gotox_limit_hit(const char *log, int err)\n+{\n+\tASSERT_EQ(err, -E2BIG, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, gotox_limit_msg, \"verifier log\");\n+}\n+\n+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)\n+{\n+\tconst __u32 insn_cnt = gotox_cnt + 3;\n+\tstruct bpf_insn *insns;\n+\tbool attempted = false;\n+\tint map_fd;\n+\n+\tinsns = calloc(insn_cnt, sizeof(*insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\treturn false;\n+\n+\tgotox_run_fill(insns, gotox_cnt);\n+\tinsns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tmap_fd = gotox_jt_create(1, gotox_cnt);\n+\tif (map_fd \u003c 0)\n+\t\tgoto free_insns;\n+\n+\t*err = gotox_prog_load(insns, insn_cnt, \u0026map_fd, 1, log);\n+\tclose(map_fd);\n+\tattempted = true;\n+free_insns:\n+\tfree(insns);\n+\treturn attempted;\n+}\n+\n+/*\n+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt\n+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.\n+ */\n+static void check_too_many_gotox_edges(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tif (try_load_gotox_prog(gotox_cnt, log, \u0026err))\n+\t\tcheck_gotox_limit_hit(log, err);\n+\n+\tfree(log);\n+}\n+\n+/*\n+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump\n+ * table holds the starts of the blocks that follow plus the exit block,\n+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits\n+ * exactly at the limit and must still load.\n+ */\n+#define GOTOX_BLOCK_SZ\t\t4\n+\n+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)\n+{\n+\tstruct bpf_insn *at;\n+\t__u32 k;\n+\n+\tfor (k = 0; k \u003c gotox_cnt; k++) {\n+\t\tat = insns + k * GOTOX_BLOCK_SZ;\n+\n+\t\t/* r1 = \u0026jt[0], by index 0 into fd_array */\n+\t\tat[0] = (struct bpf_insn) {\n+\t\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t\t.dst_reg = BPF_REG_1,\n+\t\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t\t.imm = 0,\n+\t\t};\n+\t\tat[1] = (struct bpf_insn) { .imm = 0 };\n+\t\tat[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);\n+\t\tat[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\t}\n+\n+\tinsns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();\n+}\n+\n+static int gotox_chain_jt_create(__u32 gotox_cnt)\n+{\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c gotox_cnt; i++) {\n+\t\tval.orig_off = (i + 1) * GOTOX_BLOCK_SZ;\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t       \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+static void check_gotox_edges_at_limit(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;\n+\tconst __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;\n+\tstruct bpf_insn *insns;\n+\tchar *log;\n+\tint map_fd, err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tinsns = calloc(insn_cnt, sizeof(*insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\tgoto free_log;\n+\n+\tgotox_chain_fill(insns, gotox_cnt);\n+\n+\tmap_fd = gotox_chain_jt_create(gotox_cnt);\n+\tif (map_fd \u003c 0)\n+\t\tgoto free_insns;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, \u0026map_fd, 1, log);\n+\tclose(map_fd);\n+\n+\tif (!ASSERT_OK(err, \"program at the edge limit should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+free_insns:\n+\tfree(insns);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_edges_across_subprogs(void)\n+{\n+\tconst __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;\n+\tconst __u32 sub_start = gotox_cnt + 3;\n+\tconst __u32 insn_cnt = 2 * (gotox_cnt + 3);\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_insn *insns;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tinsns = calloc(insn_cnt, sizeof(*insns));\n+\tif (!ASSERT_OK_PTR(insns, \"calloc insns\"))\n+\t\tgoto free_log;\n+\n+\tgotox_run_fill(insns, gotox_cnt);\n+\tinsns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,\n+\t\t\t\t\t    BPF_PSEUDO_CALL, 0,\n+\t\t\t\t\t    sub_start - (gotox_cnt + 1) - 1);\n+\tinsns[gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tgotox_run_fill(insns + sub_start, gotox_cnt);\n+\tinsns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();\n+\n+\tmap_fd[0] = gotox_jt_create(1, gotox_cnt);\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_insns;\n+\tmap_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);\n+\tcheck_gotox_limit_hit(log, err);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_insns:\n+\tfree(insns);\n+free_log:\n+\tfree(log);\n+}\n+\n+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)\n+{\n+\tstruct bpf_insn_array_value val = {};\n+\tint map_fd;\n+\t__u32 i;\n+\n+\tmap_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);\n+\tif (!ASSERT_GE(map_fd, 0, \"map_create\"))\n+\t\treturn map_fd;\n+\n+\tfor (i = 0; i \u003c cnt; i++) {\n+\t\tval.orig_off = offs[i];\n+\t\tif (!ASSERT_EQ(bpf_map_update_elem(map_fd, \u0026i, \u0026val, 0), 0,\n+\t\t\t       \"bpf_map_update_elem\"))\n+\t\t\tgoto err;\n+\t}\n+\n+\tif (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, \"bpf_map_freeze\"))\n+\t\tgoto err;\n+\n+\treturn map_fd;\n+err:\n+\tclose(map_fd);\n+\treturn -1;\n+}\n+\n+#define GOTOX_SUB_START\t\t4\n+#define GOTOX_MAIN_TGT\t\t2\n+#define GOTOX_SUB_TGT\t\t8\n+#define GOTOX_TWO_INSN_CNT\t10\n+\n+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_SUB_START - 1 - 1);\n+\tinsns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_EXIT_INSN();\n+\n+\t/* r1 = \u0026jt[0], by index 'jt_idx' into fd_array */\n+\tinsns[GOTOX_SUB_START] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = jt_idx,\n+\t};\n+\tinsns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);\n+\tinsns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[9] = BPF_EXIT_INSN();\n+}\n+\n+/*\n+ * An insn_array map is not necessarily a jump table: one that tracks\n+ * instruction offsets covers the whole program and is of no subprog. Such a\n+ * map must not keep a program with a gotox elsewhere from loading.\n+ */\n+static void check_gotox_tracker_map(void)\n+{\n+\tconst __u32 jt_track[] = { 0, GOTOX_MAIN_TGT, GOTOX_SUB_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 1, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_track, ARRAY_SIZE(jt_track));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tif (!ASSERT_OK(err, \"program with a tracking map should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_target_other_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 7 to 2 leaves the subprog [4,10)\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static void check_gotox_jt_per_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 1, 0);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, 0, \"bpf(BPF_PROG_LOAD)\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/*\n+ * The spanning map is of no subprog and is dropped, and the entry the gotox\n+ * register can reach is in the subprog of the gotox and in the jump table the\n+ * CFG walked, so nothing unsafe is left and the program loads.\n+ */\n+static void check_gotox_span_unreached_entry(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_MAIN_TGT, GOTOX_SUB_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 1);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tif (!ASSERT_OK(err, \"program with an unreachable spanning entry should load\"))\n+\t\tfprintf(stderr, \"verifier log: %s\\n\", log);\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_FWD_GOTOX\t\t11\n+#define GOTOX_FWD_OWN_TGT\t12\n+#define GOTOX_FWD_SUB_START\t14\n+#define GOTOX_FWD_INSN_CNT\t16\n+\n+static void gotox_from_main_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_FWD_SUB_START - 1 - 1);\n+\tinsns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,\n+\t\t\t       offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);\n+\n+\t/* r1 = \u0026jt_leaves[0], by index 1 into fd_array */\n+\tinsns[4] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 1,\n+\t};\n+\tinsns[5] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\tinsns[7] = BPF_JMP_A(3);\n+\n+\t/* r1 = \u0026jt_own[0], by index 0 into fd_array */\n+\tinsns[8] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[9] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\n+\tinsns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[13] = BPF_EXIT_INSN();\n+\tinsns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[15] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_target_subprog_from_main(void)\n+{\n+\tconst __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };\n+\tconst __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 11 to 14 leaves the subprog [0,14)\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/*\n+ * The only map of the subprog holding the gotox reaches past that subprog, so\n+ * the subprog is left without a jump table at all.\n+ */\n+static void check_gotox_jt_spans_subprogs(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };\n+\tconst __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"jump table of subprog starting at 0 spans multiple subprogs\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+/*\n+ * The subprog holding the gotox has a well formed jump table of its own and\n+ * also collects a map that reaches past its end. The spanning map is still\n+ * rejected, even though the subprog is not left without a table.\n+ */\n+static void check_gotox_jt_spans_with_own_table(void)\n+{\n+\tconst __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };\n+\tconst __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };\n+\tstruct bpf_insn insns[GOTOX_FWD_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_from_main_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"jump table of subprog starting at 0 spans multiple subprogs\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_EDGE_MAIN_TGT\t2\n+#define GOTOX_EDGE_SUB_START\t4\n+#define GOTOX_EDGE_GOTOX\t9\n+#define GOTOX_EDGE_BR_TGT\t10\n+#define GOTOX_EDGE_JT_TGT\t11\n+#define GOTOX_EDGE_INSN_CNT\t12\n+\n+static void gotox_no_edge_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_EDGE_SUB_START - 1 - 1);\n+\tinsns[GOTOX_EDGE_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_EXIT_INSN();\n+\n+\tinsns[GOTOX_EDGE_SUB_START] =\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,\n+\t\t\t    offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[5] = BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 4);\n+\n+\t/* r1 = \u0026jt_span[0], by index 0 into fd_array */\n+\tinsns[6] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[7] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[8] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 8);\n+\n+\tinsns[GOTOX_EDGE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_EDGE_BR_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[GOTOX_EDGE_JT_TGT] = BPF_EXIT_INSN();\n+}\n+\n+/*\n+ * The gotox resolves a target inside its own subprog, but out of a map that\n+ * spans subprogs and is therefore of no subprog. The CFG never walked that\n+ * edge, so the jump has to be rejected even though it stays in the subprog.\n+ */\n+static void check_gotox_target_without_cfg_edge(void)\n+{\n+\tconst __u32 jt_span[] = { GOTOX_EDGE_MAIN_TGT, GOTOX_EDGE_BR_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_EDGE_JT_TGT };\n+\tstruct bpf_insn insns[GOTOX_EDGE_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_no_edge_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log,\n+\t\t\t  \"indirect jump from insn 9 to 10 is not in the jump table of the subprog\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_SLICE_SUB_START\t6\n+#define GOTOX_SLICE_GOTOX\t14\n+#define GOTOX_SLICE_SUB_TGT\t15\n+#define GOTOX_SLICE_INSN_CNT\t17\n+\n+static void gotox_slice_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,\n+\t\t\t\tGOTOX_SLICE_SUB_START - 1 - 1);\n+\tinsns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[5] = BPF_EXIT_INSN();\n+\n+\tinsns[GOTOX_SLICE_SUB_START] =\n+\t\tBPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,\n+\t\t\t    offsetof(struct xdp_md, ingress_ifindex));\n+\tinsns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);\n+\tinsns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);\n+\tinsns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);\n+\n+\t/* r1 = \u0026jt_main[0], by index 0 into fd_array */\n+\tinsns[10] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[11] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);\n+\tinsns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\n+\tinsns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);\n+\tinsns[16] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_index_slice_other_subprog(void)\n+{\n+\tconst __u32 jt_main[] = { 2, 3, 4 };\n+\tconst __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_SLICE_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_slice_fill(insns);\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_log;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 14 to 3 leaves the subprog [6,17)\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_log:\n+\tfree(log);\n+}\n+\n+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,\n+\t\t\t    struct bpf_func_info *fi, struct btf **pbtf)\n+{\n+\tint int_id, proto_id, id;\n+\tstruct btf *btf;\n+\tchar name[24];\n+\t__u32 i;\n+\n+\tbtf = btf__new_empty();\n+\tif (!ASSERT_OK_PTR(btf, \"btf__new_empty\"))\n+\t\treturn -1;\n+\n+\tint_id = btf__add_int(btf, \"int\", 4, BTF_INT_SIGNED);\n+\tif (!ASSERT_GT(int_id, 0, \"btf__add_int\"))\n+\t\tgoto err;\n+\n+\tproto_id = btf__add_func_proto(btf, int_id);\n+\tif (!ASSERT_GT(proto_id, 0, \"btf__add_func_proto\"))\n+\t\tgoto err;\n+\n+\tfor (i = 0; i \u003c cnt; i++) {\n+\t\tsnprintf(name, sizeof(name), \"gotox_f%u\", i);\n+\t\tid = btf__add_func(btf, name, linkage[i], proto_id);\n+\t\tif (!ASSERT_GT(id, 0, \"btf__add_func\"))\n+\t\t\tgoto err;\n+\t\tfi[i].insn_off = starts[i];\n+\t\tfi[i].type_id = id;\n+\t}\n+\n+\tif (!ASSERT_OK(btf__load_into_kernel(btf), \"btf__load_into_kernel\"))\n+\t\tgoto err;\n+\n+\t*pbtf = btf;\n+\treturn btf__fd(btf);\n+err:\n+\tbtf__free(btf);\n+\treturn -1;\n+}\n+\n+static void check_gotox_target_other_global_subprog(void)\n+{\n+\tconst __u32 starts[] = { 0, GOTOX_SUB_START };\n+\tconst __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };\n+\tconst __u32 jt_main[] = { GOTOX_MAIN_TGT };\n+\tconst __u32 jt_sub[] = { GOTOX_SUB_TGT };\n+\tstruct bpf_insn insns[GOTOX_TWO_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_func_info fi[2];\n+\tstruct btf *btf = NULL;\n+\tint btf_fd;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_two_subprogs_fill(insns, 0, 0);\n+\n+\tbtf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, \u0026btf);\n+\tif (btf_fd \u003c 0)\n+\t\tgoto free_log;\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_btf;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,\n+\t\t\t\t    btf_fd, fi, ARRAY_SIZE(fi));\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 7 to 2 leaves the subprog [4,10)\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_btf:\n+\tbtf__free(btf);\n+free_log:\n+\tfree(log);\n+}\n+\n+#define GOTOX_CB_MAIN_TGT\t6\n+#define GOTOX_CB_START\t\t8\n+#define GOTOX_CB_GOTOX\t\t11\n+#define GOTOX_CB_TGT\t\t12\n+#define GOTOX_CB_INSN_CNT\t14\n+\n+static void gotox_callback_fill(struct bpf_insn *insns)\n+{\n+\tinsns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);\n+\t/* r2 = \u0026callback */\n+\tinsns[1] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_2,\n+\t\t.src_reg = BPF_PSEUDO_FUNC,\n+\t\t.imm = GOTOX_CB_START - 1 - 1,\n+\t};\n+\tinsns[2] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);\n+\tinsns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);\n+\tinsns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);\n+\tinsns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[7] = BPF_EXIT_INSN();\n+\n+\t/* r1 = \u0026jt_main[0], by index 0 into fd_array */\n+\tinsns[GOTOX_CB_START] = (struct bpf_insn) {\n+\t\t.code = BPF_LD | BPF_DW | BPF_IMM,\n+\t\t.dst_reg = BPF_REG_1,\n+\t\t.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,\n+\t\t.imm = 0,\n+\t};\n+\tinsns[9] = (struct bpf_insn) { .imm = 0 };\n+\tinsns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);\n+\tinsns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);\n+\tinsns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);\n+\tinsns[13] = BPF_EXIT_INSN();\n+}\n+\n+static void check_gotox_callback_leaves_subprog(void)\n+{\n+\tconst __u32 starts[] = { 0, GOTOX_CB_START };\n+\tconst __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };\n+\tconst __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };\n+\tconst __u32 jt_cb[] = { GOTOX_CB_TGT };\n+\tstruct bpf_insn insns[GOTOX_CB_INSN_CNT];\n+\tint map_fd[2] = { -1, -1 };\n+\tstruct bpf_func_info fi[2];\n+\tstruct btf *btf = NULL;\n+\tint btf_fd;\n+\tchar *log;\n+\tint err;\n+\n+\tlog = calloc(1, GOTOX_LOG_SZ);\n+\tif (!ASSERT_OK_PTR(log, \"calloc log\"))\n+\t\treturn;\n+\n+\tgotox_callback_fill(insns);\n+\n+\tbtf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, \u0026btf);\n+\tif (btf_fd \u003c 0)\n+\t\tgoto free_log;\n+\n+\tmap_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));\n+\tif (map_fd[0] \u003c 0)\n+\t\tgoto free_btf;\n+\tmap_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));\n+\tif (map_fd[1] \u003c 0)\n+\t\tgoto close_maps;\n+\n+\terr = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,\n+\t\t\t\t    btf_fd, fi, ARRAY_SIZE(fi));\n+\tASSERT_EQ(err, -EINVAL, \"program should have been rejected\");\n+\tASSERT_HAS_SUBSTR(log, \"indirect jump from insn 11 to 6 leaves the subprog [8,14)\",\n+\t\t\t  \"verifier log\");\n+\n+close_maps:\n+\tclose(map_fd[0]);\n+\tclose(map_fd[1]);\n+free_btf:\n+\tbtf__free(btf);\n+free_log:\n+\tfree(log);\n+}\n+\n static void check_bpf_side(void)\n {\n \tcheck_bpf_no_lookup();\n@@ -490,6 +1425,48 @@ static void __test_bpf_insn_array(void)\n \n \tif (test__start_subtest(\"bpf-side-ops\"))\n \t\tcheck_bpf_side();\n+\n+\tif (test__start_subtest(\"too-many-gotox-edges\"))\n+\t\tcheck_too_many_gotox_edges();\n+\n+\tif (test__start_subtest(\"gotox-edges-at-limit\"))\n+\t\tcheck_gotox_edges_at_limit();\n+\n+\tif (test__start_subtest(\"gotox-edges-across-subprogs\"))\n+\t\tcheck_gotox_edges_across_subprogs();\n+\n+\tif (test__start_subtest(\"gotox-tracker-map\"))\n+\t\tcheck_gotox_tracker_map();\n+\n+\tif (test__start_subtest(\"gotox-jt-spans-subprogs\"))\n+\t\tcheck_gotox_jt_spans_subprogs();\n+\n+\tif (test__start_subtest(\"gotox-jt-spans-with-own-table\"))\n+\t\tcheck_gotox_jt_spans_with_own_table();\n+\n+\tif (test__start_subtest(\"gotox-target-without-cfg-edge\"))\n+\t\tcheck_gotox_target_without_cfg_edge();\n+\n+\tif (test__start_subtest(\"gotox-target-other-subprog\"))\n+\t\tcheck_gotox_target_other_subprog();\n+\n+\tif (test__start_subtest(\"gotox-jt-per-subprog\"))\n+\t\tcheck_gotox_jt_per_subprog();\n+\n+\tif (test__start_subtest(\"gotox-span-unreached-entry\"))\n+\t\tcheck_gotox_span_unreached_entry();\n+\n+\tif (test__start_subtest(\"gotox-target-subprog-from-main\"))\n+\t\tcheck_gotox_target_subprog_from_main();\n+\n+\tif (test__start_subtest(\"gotox-index-slice-other-subprog\"))\n+\t\tcheck_gotox_index_slice_other_subprog();\n+\n+\tif (test__start_subtest(\"gotox-target-other-global-subprog\"))\n+\t\tcheck_gotox_target_other_global_subprog();\n+\n+\tif (test__start_subtest(\"gotox-callback-leaves-subprog\"))\n+\t\tcheck_gotox_callback_leaves_subprog();\n }\n #else\n static void __test_bpf_insn_array(void)\ndiff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c\nindex 5b18c9a27717b..3567b29e23789 100644\n--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c\n+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c\n@@ -318,7 +318,7 @@ __used static int test_subprog(void)\n }\n \n SEC(\"socket\")\n-__failure __msg(\"jump table for insn 4 points outside of the subprog [0,10]\")\n+__failure __msg(\"jump table of subprog starting at 0 spans multiple subprogs\")\n __naked void jump_table_outside_subprog(void)\n {\n \tasm volatile (\"\t\t\t\t\t\t\\\n"
}

{
  "EnableConfigs": [
    "BPF_SYSCALL"
  ],
  "FocusSymbols": [
    "bpf_check_cfg",
    "bpf_compute_scc",
    "check_indirect_jump",
    "compute_subprog_jts",
    "create_jt",
    "visit_gotox_insn",
    "reject_gotox_out_of_subprog",
    "reject_gotox_without_cfg_edge",
    "jt_contains"
  ],
  "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.",
  "WorthFuzzing": true
}

You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.

IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.

================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.

- REACHABILITY IS THE PRIMARY GATE:
  Fuzzing can only discover bugs in code that can actually execute in standard virtualized
  environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
  If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
  regardless of whether it adds assertions or complex logic.

- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
  Patch authors routinely label changes as "cleanups", "refactorings", or state
  "No functional change intended". Do NOT take these claims at face value.
  Code refactorings that rearrange logic, introduce helper functions, or alter state management
  in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
  If reachable executable code is modified or refactored, it MUST be fuzzed.

- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
  When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
  BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
  Even if the author believes the invariant always holds, fuzzing is essential to verify whether
  an unusual sequence of operations can violate it.

================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:

- Non-kernel and non-executable changes:
  * Modifications to Documentation/, comments, or spelling fixes.
  * User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
    that do not affect the compiled kernel image (vmlinux) or kernel modules.
  * Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
    that do not alter control flow or data structures.
  * Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
  * Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
    ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
  * Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
    physical PCI hot-unplug or manual sysfs driver unbinding.
  * Code paths exclusive to architectures other than the target architecture.

================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
  * Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
    BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
  * Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
  * Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
  * Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).

================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:

- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
  Do NOT list generic, ubiquitous functions called by almost every program in the corpus
  (including, but not limited to: general memory allocators and deallocators, page fault
  and trap handlers, or core synchronization primitives; this is not an exhaustive list).
  Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
  which severely dilutes fuzzing effort away from the actual changes.

- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
  List functions that specifically implement the logic being added or altered, or direct API entrypoints
  for the subsystem feature under review.

- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
  Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
  distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
  If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
  functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).

================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.


Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.

Target architecture: amd64

For your convenience, here is the diff of the changes:
commit e16eef8d5994f14c9cd79ed40d9a24d4c4001f83
Author: syz-cluster <triage@syzkaller.com>
Date:   Wed Sep 9 21:17:23 2026 +0000

    syz-cluster: applied patch under review

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 36b65797877d0..baf2e17d7019b 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -805,6 +805,7 @@ struct bpf_subprog_info {
 	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
 	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
 	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
 	u16 stack_depth; /* max. stack depth used by this function */
 	u16 stack_extra;
 	u32 insns_total;
@@ -825,6 +826,7 @@ struct bpf_subprog_info {
 	bool keep_fastcall_stack: 1;
 	bool changes_pkt_data: 1;
 	bool might_sleep: 1;
+	bool jt_spans_subprogs: 1;
 	u8 arg_cnt:4;
 
 	enum priv_stack_mode priv_stack_mode;
@@ -977,6 +979,8 @@ struct bpf_verifier_env {
 		int cur_stack;
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
+		u32 gotox_edges;
+		bool subprog_jts_ready;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 842c7d1eabccc..879587af8d086 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -9,6 +9,8 @@
 
 #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
 
+#define BPF_MAX_GOTOX_EDGES	BPF_COMPLEXITY_LIMIT_INSNS
+
 /* non-recursive DFS pseudo code
  * 1  procedure DFS-iterative(G,v):
  * 2      label v as discovered
@@ -284,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
 }
 
 /*
- * Find and collect all maps which fit in the subprog. Return the result as one
- * combined jump table in jt->items (allocated with kvcalloc)
+ * Collect the jump table of every subprogram that has one, as the combined
+ * table of all maps whose targets land inside that subprogram. All gotox
+ * instructions of a subprogram share the same table, so this is done in a
+ * single pass over the maps rather than once per gotox.
  */
-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
-					  int subprog_start, int subprog_end)
+static int compute_subprog_jts(struct bpf_verifier_env *env)
 {
-	struct bpf_iarray *jt = NULL;
+	struct bpf_subprog_info *subprog;
+	struct bpf_iarray *jt, *jt_cur;
 	struct bpf_map *map;
-	struct bpf_iarray *jt_cur;
+	u32 old_cnt;
 	int i;
 
 	for (i = 0; i < env->insn_array_map_cnt; i++) {
@@ -303,72 +307,97 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
 		map = env->insn_array_maps[i];
 
 		jt_cur = jt_from_map(map);
-		if (IS_ERR(jt_cur)) {
-			kvfree(jt);
-			return jt_cur;
+		if (IS_ERR(jt_cur))
+			return PTR_ERR(jt_cur);
+
+		subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
+		if (!subprog) {
+			kvfree(jt_cur);
+			continue;
+		}
+		if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) {
+			subprog->jt_spans_subprogs = true;
+			kvfree(jt_cur);
+			continue;
 		}
 
-		/*
-		 * This is enough to check one element. The full table is
-		 * checked to fit inside the subprog later in create_jt()
-		 */
-		if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) {
-			u32 old_cnt = jt ? jt->cnt : 0;
-			jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt);
-			if (!jt) {
-				kvfree(jt_cur);
-				return ERR_PTR(-ENOMEM);
-			}
-			memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+		old_cnt = subprog->jt ? subprog->jt->cnt : 0;
+		jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
+		if (!jt) {
+			subprog->jt = NULL;
+			kvfree(jt_cur);
+			return -ENOMEM;
 		}
+		memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+		subprog->jt = jt;
 
 		kvfree(jt_cur);
 	}
 
-	if (!jt) {
-		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
-		bpf_diag_program_structure(
-			env, subprog_start, "missing jump table",
-			"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
-			"No jump table was found for the subprogram that starts at instruction %u.",
-			subprog_start);
-		return ERR_PTR(-EINVAL);
+	for (i = 0; i < env->subprog_cnt; i++) {
+		jt = env->subprog_info[i].jt;
+		if (jt)
+			jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
 	}
 
-	jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
-	return jt;
+	env->cfg.subprog_jts_ready = true;
+	return 0;
+}
+
+static void free_subprog_jts(struct bpf_verifier_env *env)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
+		kvfree(env->subprog_info[i].jt);
+		env->subprog_info[i].jt = NULL;
+		env->subprog_info[i].jt_spans_subprogs = false;
+	}
+	env->cfg.subprog_jts_ready = false;
 }
 
 static struct bpf_iarray *
 create_jt(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_subprog_info *subprog;
-	int subprog_start, subprog_end;
 	struct bpf_iarray *jt;
-	int i;
+	int subprog_start, err;
+
+	if (!env->cfg.subprog_jts_ready) {
+		err = compute_subprog_jts(env);
+		if (err)
+			return ERR_PTR(err);
+	}
 
 	subprog = bpf_find_containing_subprog(env, t);
 	subprog_start = subprog->start;
-	subprog_end = (subprog + 1)->start;
-	jt = jt_from_subprog(env, subprog_start, subprog_end);
-	if (IS_ERR(jt))
-		return jt;
 
-	/* Check that the every element of the jump table fits within the given subprogram */
-	for (i = 0; i < jt->cnt; i++) {
-		if (jt->items[i] < subprog_start || jt->items[i] >= subprog_end) {
-			verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n",
-					t, subprog_start, subprog_end);
-			bpf_diag_program_structure(
-				env, t, "jump table target out of range",
-				"Keep every jump-table target inside the same subprogram.",
-				"The jump table for instruction %d points outside subprogram range [%u,%u).",
-				t, subprog_start, subprog_end);
-			kvfree(jt);
-			return ERR_PTR(-EINVAL);
-		}
+	if (subprog->jt_spans_subprogs) {
+		verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n",
+			subprog_start);
+		bpf_diag_program_structure(
+			env, subprog_start, "jump table spans subprograms",
+			"Keep every entry of a jump table inside one subprogram.",
+			"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.",
+			subprog_start, (subprog + 1)->start);
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (!subprog->jt) {
+		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
+		bpf_diag_program_structure(
+			env, subprog_start, "missing jump table",
+			"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
+			"No jump table was found for the subprogram that starts at instruction %u.",
+			subprog_start);
+		return ERR_PTR(-EINVAL);
 	}
 
+	jt = bpf_iarray_realloc(NULL, subprog->jt->cnt);
+	if (!jt)
+		return ERR_PTR(-ENOMEM);
+	memcpy(jt->items, subprog->jt->items, subprog->jt->cnt << 2);
+
 	return jt;
 }
 
@@ -388,6 +417,19 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 			return PTR_ERR(jt);
 
 		env->insn_aux_data[t].jt = jt;
+
+		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+				       &env->cfg.gotox_edges) ||
+		    env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+			verbose(env, "number of indirect jump edges in the program exceeds %u\n",
+				BPF_MAX_GOTOX_EDGES);
+			bpf_diag_program_structure(
+				env, t, "too many indirect jump edges",
+				"Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+				"The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+				BPF_MAX_GOTOX_EDGES);
+			return -E2BIG;
+		}
 	}
 
 	mark_prune_point(env, t);
@@ -678,6 +720,7 @@ int bpf_check_cfg(struct bpf_verifier_env *env)
 	env->prog->aux->might_sleep = env->subprog_info[0].might_sleep;
 
 err_free:
+	free_subprog_jts(env);
 	kvfree(insn_state);
 	kvfree(insn_stack);
 	env->cfg.insn_state = env->cfg.insn_stack = NULL;
@@ -749,7 +792,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	struct bpf_insn_aux_data *aux = env->insn_aux_data;
 	const u32 insn_cnt = env->prog->len;
 	int stack_sz, dfs_sz, err = 0;
-	u32 *stack, *pre, *low, *dfs;
+	u32 *stack, *pre, *low, *dfs, *dfs_pos;
 	u32 i, j, t, w;
 	u32 next_preorder_num;
 	u32 next_scc_id;
@@ -762,13 +805,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	 * - 'stack' accumulates vertices in DFS order, see invariant comment below;
 	 * - 'pre[t] == p' => preorder number of vertex 't' is 'p';
 	 * - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n';
-	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion.
+	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion;
+	 * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its
+	 *   successors at index 'j'.
 	 */
 	stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	dfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);
-	if (!stack || !pre || !low || !dfs) {
+	dfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);
+	if (!stack || !pre || !low || !dfs || !dfs_pos) {
 		err = -ENOMEM;
 		goto exit;
 	}
@@ -851,6 +897,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 		stack_sz = 0;
 		dfs_sz = 1;
 		dfs[0] = i;
+		dfs_pos[0] = 0;
 dfs_continue:
 		while (dfs_sz) {
 			w = dfs[dfs_sz - 1];
@@ -860,13 +907,37 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 				next_preorder_num++;
 				stack[stack_sz++] = w;
 			}
-			/* Visit 'w' successors */
+			/*
+			 * Visit 'w' successors, resuming at the successor this
+			 * frame last descended into. Restarting the scan at zero
+			 * on every return to 'w' would examine each successor
+			 * once per descent, i.e. quadratic in the number of
+			 * successors, which for a gotox is the size of the jump
+			 * table.
+			 *
+			 * Re-folding the successors before that index would be a
+			 * no-op. Such a successor 's' has 'pre[s] != 0' by then,
+			 * so it is never pushed onto 'dfs' again, and low[s] can
+			 * only decrease while 's' is the top of 'dfs'. If 's' is
+			 * still on 'dfs' it sits below 'w' and cannot become the
+			 * top before 'w' is popped; otherwise the only remaining
+			 * write to low[s] is the pop of its SCC, setting it to
+			 * NOT_ON_STACK, for which the min below is a no-op.
+			 */
 			succ = bpf_insn_successors(env, w);
-			for (j = 0; j < succ->cnt; ++j) {
+			for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) {
 				if (pre[succ->items[j]]) {
 					low[w] = min(low[w], low[succ->items[j]]);
 				} else {
-					dfs[dfs_sz++] = succ->items[j];
+					/*
+					 * Resume at 'j', not 'j + 1': the successor
+					 * is revisited once its DFS completes, to
+					 * fold its low[] into low[w].
+					 */
+					dfs_pos[dfs_sz - 1] = j;
+					dfs_pos[dfs_sz] = 0;
+					dfs[dfs_sz] = succ->items[j];
+					dfs_sz++;
 					goto dfs_continue;
 				}
 			}
@@ -916,5 +987,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	kvfree(pre);
 	kvfree(low);
 	kvfree(dfs);
+	kvfree(dfs_pos);
 	return err;
 }
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 72a3f5998dd27..45234e2fbee64 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18165,11 +18165,56 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,
 	return 0;
 }
 
+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */
+static bool jt_contains(const struct bpf_iarray *jt, u32 target)
+{
+	int l = 0, r = jt->cnt - 1, m;
+
+	while (l <= r) {
+		m = l + (r - l) / 2;
+		if (jt->items[m] == target)
+			return true;
+		if (jt->items[m] < target)
+			l = m + 1;
+		else
+			r = m - 1;
+	}
+	return false;
+}
+
+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,
+				       u32 subprog_start, u32 subprog_end)
+{
+	verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n",
+		     env->insn_idx, target, subprog_start, subprog_end);
+	bpf_diag_program_structure(
+		env, env->insn_idx, "indirect jump leaves subprogram",
+		"Keep every reachable jump-table target inside the subprogram of the indirect jump.",
+		"Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).",
+		env->insn_idx, target, subprog_start, subprog_end);
+	return -EINVAL;
+}
+
+static int reject_gotox_without_cfg_edge(struct bpf_verifier_env *env, u32 target)
+{
+	verbose(env, "indirect jump from insn %d to %u is not in the jump table of the subprog\n",
+		     env->insn_idx, target);
+	bpf_diag_program_structure(
+		env, env->insn_idx, "indirect jump target without CFG edge",
+		"Resolve indirect jumps through a jump table whose entries all fall inside the subprogram of the jump.",
+		"Instruction %d can jump indirectly to instruction %u, which is not part of the jump table of its subprogram.",
+		env->insn_idx, target);
+	return -EINVAL;
+}
+
 /* gotox *dst_reg */
 static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)
 {
 	struct bpf_verifier_state *other_branch;
+	struct bpf_subprog_info *subprog;
+	u32 subprog_start, subprog_end;
 	struct bpf_reg_state *dst_reg;
+	struct bpf_iarray *jt;
 	struct bpf_map *map;
 	u32 min_index, max_index;
 	int err = 0;
@@ -18212,6 +18257,26 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
 		return -EINVAL;
 	}
 
+	subprog = bpf_find_containing_subprog(env, env->insn_idx);
+	if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
+		return -EFAULT;
+	subprog_start = subprog->start;
+	subprog_end = (subprog + 1)->start;
+
+	jt = env->insn_aux_data[env->insn_idx].jt;
+	if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
+		return -EFAULT;
+
+	for (i = 0; i < n; i++) {
+		u32 target = env->gotox_tmp_buf->items[i];
+
+		if (target < subprog_start || target >= subprog_end)
+			return reject_gotox_out_of_subprog(env, target, subprog_start,
+							   subprog_end);
+		if (!jt_contains(jt, target))
+			return reject_gotox_without_cfg_edge(env, target);
+	}
+
 	for (i = 0; i < n - 1; i++) {
 		mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
 		other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 0222a9a5d0761..d5a831a75d820 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 
 #include <bpf/bpf.h>
+#include <bpf/btf.h>
 #include <test_progs.h>
 
 #if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)
@@ -453,6 +454,940 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
+#define GOTOX_CNT_AT_LIMIT	1000
+#define GOTOX_LOG_SZ		(256 * 1024)
+
+static const char gotox_limit_msg[] =
+	"number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+	/* the run of gotox itself, plus the exit block right after it */
+	const __u32 jt_cnt = gotox_cnt + 1;
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < jt_cnt; i++) {
+		val.orig_off = first_gotox + i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
+				 int *fd_array, __u32 fd_array_cnt, char *log,
+				 int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int prog_fd;
+
+	log[0] = 0;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = fd_array_cnt;
+	opts.log_buf = log;
+	opts.log_size = GOTOX_LOG_SZ;
+	opts.log_level = 1;
+	if (fi_cnt) {
+		opts.prog_btf_fd = btf_fd;
+		opts.func_info = fi;
+		opts.func_info_cnt = fi_cnt;
+		opts.func_info_rec_size = sizeof(*fi);
+	}
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+	if (prog_fd >= 0) {
+		close(prog_fd);
+		return 0;
+	}
+	return prog_fd;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,
+				     -1, NULL, 0);
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	__u32 i;
+
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+	for (i = 1; i <= gotox_cnt; i++)
+		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+	const __u32 insn_cnt = gotox_cnt + 3;
+	struct bpf_insn *insns;
+	bool attempted = false;
+	int map_fd;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		return false;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd = gotox_jt_create(1, gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+	attempted = true;
+free_insns:
+	free(insns);
+	return attempted;
+}
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	if (try_load_gotox_prog(gotox_cnt, log, &err))
+		check_gotox_limit_hit(log, err);
+
+	free(log);
+}
+
+/*
+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
+ * table holds the starts of the blocks that follow plus the exit block,
+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
+ * exactly at the limit and must still load.
+ */
+#define GOTOX_BLOCK_SZ		4
+
+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	struct bpf_insn *at;
+	__u32 k;
+
+	for (k = 0; k < gotox_cnt; k++) {
+		at = insns + k * GOTOX_BLOCK_SZ;
+
+		/* r1 = &jt[0], by index 0 into fd_array */
+		at[0] = (struct bpf_insn) {
+			.code = BPF_LD | BPF_DW | BPF_IMM,
+			.dst_reg = BPF_REG_1,
+			.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+			.imm = 0,
+		};
+		at[1] = (struct bpf_insn) { .imm = 0 };
+		at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
+		at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	}
+
+	insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
+}
+
+static int gotox_chain_jt_create(__u32 gotox_cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < gotox_cnt; i++) {
+		val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static void check_gotox_edges_at_limit(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
+	struct bpf_insn *insns;
+	char *log;
+	int map_fd, err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_chain_fill(insns, gotox_cnt);
+
+	map_fd = gotox_chain_jt_create(gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+
+	if (!ASSERT_OK(err, "program at the edge limit should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
+static void check_gotox_edges_across_subprogs(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
+	const __u32 sub_start = gotox_cnt + 3;
+	const __u32 insn_cnt = 2 * (gotox_cnt + 3);
+	int map_fd[2] = { -1, -1 };
+	struct bpf_insn *insns;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
+					    BPF_PSEUDO_CALL, 0,
+					    sub_start - (gotox_cnt + 1) - 1);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	gotox_run_fill(insns + sub_start, gotox_cnt);
+	insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd[0] = gotox_jt_create(1, gotox_cnt);
+	if (map_fd[0] < 0)
+		goto free_insns;
+	map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
+	check_gotox_limit_hit(log, err);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < cnt; i++) {
+		val.orig_off = offs[i];
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+#define GOTOX_SUB_START		4
+#define GOTOX_MAIN_TGT		2
+#define GOTOX_SUB_TGT		8
+#define GOTOX_TWO_INSN_CNT	10
+
+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SUB_START - 1 - 1);
+	insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_EXIT_INSN();
+
+	/* r1 = &jt[0], by index 'jt_idx' into fd_array */
+	insns[GOTOX_SUB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = jt_idx,
+	};
+	insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);
+	insns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[9] = BPF_EXIT_INSN();
+}
+
+/*
+ * An insn_array map is not necessarily a jump table: one that tracks
+ * instruction offsets covers the whole program and is of no subprog. Such a
+ * map must not keep a program with a gotox elsewhere from loading.
+ */
+static void check_gotox_tracker_map(void)
+{
+	const __u32 jt_track[] = { 0, GOTOX_MAIN_TGT, GOTOX_SUB_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 1, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_track, ARRAY_SIZE(jt_track));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	if (!ASSERT_OK(err, "program with a tracking map should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static void check_gotox_target_other_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static void check_gotox_jt_per_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 1, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The spanning map is of no subprog and is dropped, and the entry the gotox
+ * register can reach is in the subprog of the gotox and in the jump table the
+ * CFG walked, so nothing unsafe is left and the program loads.
+ */
+static void check_gotox_span_unreached_entry(void)
+{
+	const __u32 jt_span[] = { GOTOX_MAIN_TGT, GOTOX_SUB_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 1);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	if (!ASSERT_OK(err, "program with an unreachable spanning entry should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_FWD_GOTOX		11
+#define GOTOX_FWD_OWN_TGT	12
+#define GOTOX_FWD_SUB_START	14
+#define GOTOX_FWD_INSN_CNT	16
+
+static void gotox_from_main_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_FWD_SUB_START - 1 - 1);
+	insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,
+			       offsetof(struct xdp_md, ingress_ifindex));
+	insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);
+
+	/* r1 = &jt_leaves[0], by index 1 into fd_array */
+	insns[4] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 1,
+	};
+	insns[5] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[7] = BPF_JMP_A(3);
+
+	/* r1 = &jt_own[0], by index 0 into fd_array */
+	insns[8] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+	insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[15] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_subprog_from_main(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The only map of the subprog holding the gotox reaches past that subprog, so
+ * the subprog is left without a jump table at all.
+ */
+static void check_gotox_jt_spans_subprogs(void)
+{
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The subprog holding the gotox has a well formed jump table of its own and
+ * also collects a map that reaches past its end. The spanning map is still
+ * rejected, even though the subprog is not left without a table.
+ */
+static void check_gotox_jt_spans_with_own_table(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_EDGE_MAIN_TGT	2
+#define GOTOX_EDGE_SUB_START	4
+#define GOTOX_EDGE_GOTOX	9
+#define GOTOX_EDGE_BR_TGT	10
+#define GOTOX_EDGE_JT_TGT	11
+#define GOTOX_EDGE_INSN_CNT	12
+
+static void gotox_no_edge_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_EDGE_SUB_START - 1 - 1);
+	insns[GOTOX_EDGE_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_EXIT_INSN();
+
+	insns[GOTOX_EDGE_SUB_START] =
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+			    offsetof(struct xdp_md, ingress_ifindex));
+	insns[5] = BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 4);
+
+	/* r1 = &jt_span[0], by index 0 into fd_array */
+	insns[6] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[7] = (struct bpf_insn) { .imm = 0 };
+	insns[8] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 8);
+
+	insns[GOTOX_EDGE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_EDGE_BR_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[GOTOX_EDGE_JT_TGT] = BPF_EXIT_INSN();
+}
+
+/*
+ * The gotox resolves a target inside its own subprog, but out of a map that
+ * spans subprogs and is therefore of no subprog. The CFG never walked that
+ * edge, so the jump has to be rejected even though it stays in the subprog.
+ */
+static void check_gotox_target_without_cfg_edge(void)
+{
+	const __u32 jt_span[] = { GOTOX_EDGE_MAIN_TGT, GOTOX_EDGE_BR_TGT };
+	const __u32 jt_sub[] = { GOTOX_EDGE_JT_TGT };
+	struct bpf_insn insns[GOTOX_EDGE_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_no_edge_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log,
+			  "indirect jump from insn 9 to 10 is not in the jump table of the subprog",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_SLICE_SUB_START	6
+#define GOTOX_SLICE_GOTOX	14
+#define GOTOX_SLICE_SUB_TGT	15
+#define GOTOX_SLICE_INSN_CNT	17
+
+static void gotox_slice_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SLICE_SUB_START - 1 - 1);
+	insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[5] = BPF_EXIT_INSN();
+
+	insns[GOTOX_SLICE_SUB_START] =
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+			    offsetof(struct xdp_md, ingress_ifindex));
+	insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);
+	insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);
+	insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[10] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[11] = (struct bpf_insn) { .imm = 0 };
+	insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);
+	insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[16] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_index_slice_other_subprog(void)
+{
+	const __u32 jt_main[] = { 2, 3, 4 };
+	const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };
+	struct bpf_insn insns[GOTOX_SLICE_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_slice_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,
+			    struct bpf_func_info *fi, struct btf **pbtf)
+{
+	int int_id, proto_id, id;
+	struct btf *btf;
+	char name[24];
+	__u32 i;
+
+	btf = btf__new_empty();
+	if (!ASSERT_OK_PTR(btf, "btf__new_empty"))
+		return -1;
+
+	int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED);
+	if (!ASSERT_GT(int_id, 0, "btf__add_int"))
+		goto err;
+
+	proto_id = btf__add_func_proto(btf, int_id);
+	if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto"))
+		goto err;
+
+	for (i = 0; i < cnt; i++) {
+		snprintf(name, sizeof(name), "gotox_f%u", i);
+		id = btf__add_func(btf, name, linkage[i], proto_id);
+		if (!ASSERT_GT(id, 0, "btf__add_func"))
+			goto err;
+		fi[i].insn_off = starts[i];
+		fi[i].type_id = id;
+	}
+
+	if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel"))
+		goto err;
+
+	*pbtf = btf;
+	return btf__fd(btf);
+err:
+	btf__free(btf);
+	return -1;
+}
+
+static void check_gotox_target_other_global_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_SUB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
+#define GOTOX_CB_MAIN_TGT	6
+#define GOTOX_CB_START		8
+#define GOTOX_CB_GOTOX		11
+#define GOTOX_CB_TGT		12
+#define GOTOX_CB_INSN_CNT	14
+
+static void gotox_callback_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);
+	/* r2 = &callback */
+	insns[1] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_2,
+		.src_reg = BPF_PSEUDO_FUNC,
+		.imm = GOTOX_CB_START - 1 - 1,
+	};
+	insns[2] = (struct bpf_insn) { .imm = 0 };
+	insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);
+	insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);
+	insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[7] = BPF_EXIT_INSN();
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[GOTOX_CB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_callback_leaves_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_CB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };
+	const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };
+	const __u32 jt_cb[] = { GOTOX_CB_TGT };
+	struct bpf_insn insns[GOTOX_CB_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_callback_fill(insns);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
 static void check_bpf_side(void)
 {
 	check_bpf_no_lookup();
@@ -490,6 +1425,48 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
+
+	if (test__start_subtest("too-many-gotox-edges"))
+		check_too_many_gotox_edges();
+
+	if (test__start_subtest("gotox-edges-at-limit"))
+		check_gotox_edges_at_limit();
+
+	if (test__start_subtest("gotox-edges-across-subprogs"))
+		check_gotox_edges_across_subprogs();
+
+	if (test__start_subtest("gotox-tracker-map"))
+		check_gotox_tracker_map();
+
+	if (test__start_subtest("gotox-jt-spans-subprogs"))
+		check_gotox_jt_spans_subprogs();
+
+	if (test__start_subtest("gotox-jt-spans-with-own-table"))
+		check_gotox_jt_spans_with_own_table();
+
+	if (test__start_subtest("gotox-target-without-cfg-edge"))
+		check_gotox_target_without_cfg_edge();
+
+	if (test__start_subtest("gotox-target-other-subprog"))
+		check_gotox_target_other_subprog();
+
+	if (test__start_subtest("gotox-jt-per-subprog"))
+		check_gotox_jt_per_subprog();
+
+	if (test__start_subtest("gotox-span-unreached-entry"))
+		check_gotox_span_unreached_entry();
+
+	if (test__start_subtest("gotox-target-subprog-from-main"))
+		check_gotox_target_subprog_from_main();
+
+	if (test__start_subtest("gotox-index-slice-other-subprog"))
+		check_gotox_index_slice_other_subprog();
+
+	if (test__start_subtest("gotox-target-other-global-subprog"))
+		check_gotox_target_other_global_subprog();
+
+	if (test__start_subtest("gotox-callback-leaves-subprog"))
+		check_gotox_callback_leaves_subprog();
 }
 #else
 static void __test_bpf_insn_array(void)
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index 5b18c9a27717b..3567b29e23789 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -318,7 +318,7 @@ __used static int test_subprog(void)
 }
 
 SEC("socket")
-__failure __msg("jump table for insn 4 points outside of the subprog [0,10]")
+__failure __msg("jump table of subprog starting at 0 spans multiple subprogs")
 __naked void jump_table_outside_subprog(void)
 {
 	asm volatile ("						\

							input: 22781
							output: 38
							thoughts: 508

**Fuzzing Target Assessment: BPF Verifier Indirect Jumps**

Okay, let's break this down. My initial assessment is that this patch, touching the BPF verifier, warrants fuzzing with syzkaller.

First, the patch is scoped to changes in `include/linux/bpf_verifier.h`, `kernel/bpf/cfg.c`, and `kernel/bpf/verifier.c` – a clear indication of alterations within the BPF verifier and, specifically, its control-flow graph (CFG) analysis for indirect jumps using `gotox`. The fact that it also includes selftests is encouraging for reproducing any potential issues.

Second, the BPF verifier itself is reachable. It's invoked through the `bpf()` syscall (`BPF_PROG_LOAD`), and amd64 is our target architecture. We can easily execute this in a standard QEMU or GCE environment. The modified code directly impacts jump table construction, CFG edge calculation, strongly connected component (SCC) computation during the DFS traversal within `bpf_compute_scc`, and crucially, verifier safety checks for indirect jumps, as seen in `check_indirect_jump`. This is all core kernel BPF code, clearly reachable.

Therefore, the verdict is a definite "WorthFuzzing: true."

Now, for deeper analysis. I'll take a look at the specific functions and symbols altered to guide fuzzing strategy. In `kernel/bpf/cfg.c`, we see `compute_subprog_jts`, `free_subprog_jts`, `create_jt`, and `visit_gotox_insn` (all static) and `bpf_check_cfg` (exported, called by the verifier), and `bpf_compute_scc` (non-static) are relevant. In `kernel/bpf/verifier.c`, we've got `jt_contains`, `reject_gotox_out_of_subprog`, `reject_gotox_without_cfg_edge`, and `check_indirect_jump` (all static) and `bpf_check` which in turn calls `bpf_check_cfg`, `bpf_compute_scc`, `check_indirect_jump` and `do_check`.

My next step will be a deep dive with `grepper` (or codesearch) to determine where these symbols are defined and used, especially the non-static ones or the ones that seem suitable for FocusSymbols. By pinpointing the call sites for `bpf_check_cfg`, `bpf_compute_scc`, and `check_indirect_jump`, I can build a targeted fuzzer configuration for syzkaller.

{
  "Expression": "bpf_check_cfg|bpf_compute_scc",
  "PathPrefix": "kernel/bpf"
}

{
  "Output": "kernel/bpf/cfg.c=495=static int visit_insn(int t, struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-631- */\nkernel/bpf/cfg.c:632:int bpf_check_cfg(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-633-{\n--\nkernel/bpf/cfg.c=735=int bpf_compute_postorder(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-787- */\nkernel/bpf/cfg.c:788:int bpf_compute_scc(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-789-{\n--\nkernel/bpf/verifier.c=21197=int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-21342-\nkernel/bpf/verifier.c:21343:\tret = bpf_check_cfg(env);\nkernel/bpf/verifier.c-21344-\tif (ret \u003c 0)\n--\nkernel/bpf/verifier.c-21370-\nkernel/bpf/verifier.c:21371:\tret = bpf_compute_scc(env);\nkernel/bpf/verifier.c-21372-\tif (ret \u003c 0)\n"
}

							input: 24309
							output: 29
							thoughts: 80

Here's my analysis of what needs to be looked at, thinking as you:

**Diving into the BPF Verifier and Control Flow**

Okay, so I need to get my hands dirty and really understand what's going on here. I'm going to start by scrutinizing the key functions involved in the BPF verification and control flow graph (CFG) construction. I'm thinking I should start with `check_indirect_jump`, `visit_gotox_insn`, `compute_subprog_jts`, `bpf_check_cfg`, and `bpf_compute_scc`. Those names alone give a decent clue about where to start. They all seem central to how the verifier handles jumps and manages the program's structure.

Beyond those functions, I really need to have a clear picture of what's *in* the code. So, I'll be digging into `kernel/bpf/verifier.c` and `kernel/bpf/cfg.c`. I need to get a clear picture of the relevant symbols, functions and data structures. It'll be essential to understand the logic behind these functions if I want to figure out what's causing the problem. Hopefully, understanding how they are implemented will clarify the underlying issues.

{
  "Expression": "check_indirect_jump",
  "PathPrefix": "kernel/bpf"
}

{
  "Output": "kernel/bpf/verifier.c=18198=static int reject_gotox_without_cfg_edge(struct bpf_verifier_env *env, u32 target)\n--\nkernel/bpf/verifier.c-18210-/* gotox *dst_reg */\nkernel/bpf/verifier.c:18211:static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\nkernel/bpf/verifier.c-18212-{\n--\nkernel/bpf/verifier.c=18292=static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)\n--\nkernel/bpf/verifier.c-18368-\t\t\tif (BPF_SRC(insn-\u003ecode) == BPF_X)\nkernel/bpf/verifier.c:18369:\t\t\t\treturn check_indirect_jump(env, insn);\nkernel/bpf/verifier.c-18370-\n"
}

							input: 24957
							output: 27
							thoughts: 47

Okay, here's my thought process as I delve into the Kconfig aspects of BPF:

**Investigating BPF Verifier and Syscall Kconfig Dependencies**

Alright, time to get to the heart of the matter. I need to nail down the specific Kconfig options that govern the BPF verifier and the BPF system call. I'm thinking, "Where would I find the most definitive information?" Obviously, the `kernel/bpf/` directory is the logical starting point. I'll need to double-check both the `kernel/bpf/Makefile` for build dependencies and then the `init/Kconfig` or `kernel/bpf/Kconfig` files. That should give me a solid view of the configuration knobs and how they interrelate. I'm after the dependencies, and the options that control enabling/disabling these features. Time to dig in and see what the build system needs to be in place.

{
  "Expression": "cfg\\.o",
  "PathPrefix": "kernel/bpf"
}

{
  "Output": "kernel/bpf/Makefile=13=obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o\nkernel/bpf/Makefile:14:obj-$(CONFIG_BPF_SYSCALL) += fixups.o cfg.o states.o backtrack.o check_btf.o\nkernel/bpf/Makefile-15-obj-${CONFIG_BPF_LSM}\t  += bpf_inode_storage.o\n"
}

							input: 25358
							output: 29
							thoughts: 62

**My BPF Configuration Considerations**

Okay, so I see that `cfg.o` and `verifier.o` are dependent on `CONFIG_BPF_SYSCALL`. That's a key dependency to keep in mind. I need to be absolutely sure this is enabled. Let's make a quick sweep of the other BPF-related configurations while I'm at it. I'll need to figure out the best way to determine if there are any other dependencies. Time to dive into the Kconfig files. I'll start with `init/Kconfig` and then check `kernel/bpf/Kconfig`. Those are typically the most relevant spots for BPF configuration options, and I want to get a complete picture before proceeding.

{
  "Expression": "config BPF_SYSCALL",
  "PathPrefix": "init/"
}

{
  "error": "no matches"
}

							input: 25620
							output: 31
							thoughts: 13

Here's a summary of my thought process, as requested:

**Diving into the BPF Kernel Configuration**

Okay, time to get a handle on the current state of things. Let's start with a deep dive into the `kernel/bpf/Kconfig` file. I need to refresh my memory on the configuration options available for BPF within the kernel. I'm thinking about the various features and how they are enabled or disabled. This Kconfig file will be the central point to understand the available knobs and levers that control BPF's behaviour. I want to see what's enabled by default, what requires specific configurations, and if there are any new or changed options since I last looked. This is the bedrock of understanding how BPF is integrated and what options are available when building or configuring a kernel with BPF capabilities. Knowing this is critical for setting up my development environment and determining the best strategy for my work. So, `kernel/bpf/Kconfig`, here we go.

{
  "Expression": "config BPF_SYSCALL",
  "PathPrefix": "kernel/bpf"
}

{
  "Output": "kernel/bpf/Kconfig=25=menu \"BPF subsystem\"\nkernel/bpf/Kconfig-26-\nkernel/bpf/Kconfig:27:config BPF_SYSCALL\nkernel/bpf/Kconfig-28-\tbool \"Enable bpf() system call\"\n"
}

							input: 25957
							output: 34
							thoughts: 19

Here's a summary of my thought process, as if I were the one thinking it:

**Diving into the BPF Kernel Configuration**

Okay, time to get my hands dirty and understand how the BPF subsystem is configured within the kernel. I'm going to navigate directly to `kernel/bpf/Kconfig`. Specifically, I want to zero in on the relevant section around line 27. Hopefully, this will provide the initial context I need to refresh my memory on the key configuration options and their dependencies. This Kconfig file will be the perfect place to start. Let's see what we can find.

{
  "File": "kernel/bpf/Kconfig",
  "FirstLine": 25,
  "LineCount": 20
}

{
  "Contents": "  25:\tmenu \"BPF subsystem\"\n  26:\t\n  27:\tconfig BPF_SYSCALL\n  28:\t\tbool \"Enable bpf() system call\"\n  29:\t\tselect BPF\n  30:\t\tselect IRQ_WORK\n  31:\t\tselect NEED_TASKS_RCU\n  32:\t\tselect TASKS_TRACE_RCU\n  33:\t\tselect BINARY_PRINTF\n  34:\t\tselect NET_SOCK_MSG if NET\n  35:\t\tselect NET_XGRESS if NET\n  36:\t\tselect PAGE_POOL if NET\n  37:\t\tdefault n\n  38:\t\thelp\n  39:\t\t  Enable the bpf() system call that allows to manipulate BPF programs\n  40:\t\t  and maps via file descriptors.\n  41:\t\n  42:\tconfig BPF_JIT\n  43:\t\tbool \"Enable BPF Just In Time compiler\"\n  44:\t\tdepends on BPF\n"
}

							input: 26392
							output: 190
							thoughts: 367

**My Analysis of the BPF Verifier Patch**

Okay, let's break this down. First, I need to zero in on the key functions touched by this patch. I'm focusing on: `bpf_check_cfg`, `bpf_compute_scc`, `check_indirect_jump`, and then the bunch of functions tied to jump tables: `compute_subprog_jts`, `create_jt`, `visit_gotox_insn`, `reject_gotox_out_of_subprog`, `reject_gotox_without_cfg_edge`, and `jt_contains`. These are the core elements I need to examine.

I've confirmed these functions reside within `kernel/bpf/cfg.c` and `kernel/bpf/verifier.c`, which is exactly where I expect to find them. These aren't generic, widespread utilities; they are laser-focused on BPF-specific control flow graph manipulation, SCC analysis, and specifically, the handling of `gotox` indirect jumps. That’s a strong signal.

The patch targets critical BPF infrastructure: CFG analysis, SCC computation, jump table handling, and the all-important verification logic for indirect jumps. The `BPF_SYSCALL` configuration enables this code path. Crucially, this means that these code paths are directly exposed to user space through the `bpf(BPF_PROG_LOAD)` system call. This is a prime target for fuzzing. Any change here has the potential to introduce vulnerabilities that can be exploited from user space. The `WorthFuzzing` flag is absolutely spot on in this case.

Okay, this warrants a fuzzing campaign to thoroughly test the integrity of this code. Time to set those results.

{
  "EnableConfigs": [
    "BPF_SYSCALL"
  ],
  "FocusSymbols": [
    "bpf_check_cfg",
    "bpf_compute_scc",
    "check_indirect_jump",
    "compute_subprog_jts",
    "create_jt",
    "visit_gotox_insn",
    "reject_gotox_out_of_subprog",
    "reject_gotox_without_cfg_edge",
    "jt_contains"
  ],
  "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.",
  "WorthFuzzing": true
}

{
  "EnableConfigs": [
    "BPF_SYSCALL"
  ],
  "FocusSymbols": [
    "bpf_check_cfg",
    "bpf_compute_scc",
    "check_indirect_jump",
    "compute_subprog_jts",
    "create_jt",
    "visit_gotox_insn",
    "reject_gotox_out_of_subprog",
    "reject_gotox_without_cfg_edge",
    "jt_contains"
  ],
  "Reasoning": "The patch modifies the BPF verifier's control-flow graph (CFG) construction, SCC computation traversal, jump-table per-subprogram management, and safety verification checks for indirect jumps (gotox instructions). These paths are fully reachable via the bpf(BPF_PROG_LOAD, ...) syscall.",
  "WorthFuzzing": true
}

{
  "WorthFuzzing": true
}

{
  "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.",
  "NeedsKMSAN": false
}

You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.

CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
  a comprehensive suite of debugging tools and sanitizers, including KASAN
  (out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
  (locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
  or page allocations) and kernel-to-user memory info-leaks.

Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.

Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.

Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
   attributes, ioctl output arguments, socket options, or BPF buffers) where fields
   or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
   or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
   could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
   uninitialized bytes of existing buffers.

Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
  dereferences, locking deadlocks, or use-after-free bugs (these are already caught
  by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
  or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.

Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.


Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.

Target architecture: amd64

For your convenience, here is the diff of the changes:
commit e16eef8d5994f14c9cd79ed40d9a24d4c4001f83
Author: syz-cluster <triage@syzkaller.com>
Date:   Wed Sep 9 21:17:23 2026 +0000

    syz-cluster: applied patch under review

diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 36b65797877d0..baf2e17d7019b 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -805,6 +805,7 @@ struct bpf_subprog_info {
 	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
 	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
 	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
 	u16 stack_depth; /* max. stack depth used by this function */
 	u16 stack_extra;
 	u32 insns_total;
@@ -825,6 +826,7 @@ struct bpf_subprog_info {
 	bool keep_fastcall_stack: 1;
 	bool changes_pkt_data: 1;
 	bool might_sleep: 1;
+	bool jt_spans_subprogs: 1;
 	u8 arg_cnt:4;
 
 	enum priv_stack_mode priv_stack_mode;
@@ -977,6 +979,8 @@ struct bpf_verifier_env {
 		int cur_stack;
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
+		u32 gotox_edges;
+		bool subprog_jts_ready;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 842c7d1eabccc..879587af8d086 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -9,6 +9,8 @@
 
 #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
 
+#define BPF_MAX_GOTOX_EDGES	BPF_COMPLEXITY_LIMIT_INSNS
+
 /* non-recursive DFS pseudo code
  * 1  procedure DFS-iterative(G,v):
  * 2      label v as discovered
@@ -284,15 +286,17 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
 }
 
 /*
- * Find and collect all maps which fit in the subprog. Return the result as one
- * combined jump table in jt->items (allocated with kvcalloc)
+ * Collect the jump table of every subprogram that has one, as the combined
+ * table of all maps whose targets land inside that subprogram. All gotox
+ * instructions of a subprogram share the same table, so this is done in a
+ * single pass over the maps rather than once per gotox.
  */
-static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
-					  int subprog_start, int subprog_end)
+static int compute_subprog_jts(struct bpf_verifier_env *env)
 {
-	struct bpf_iarray *jt = NULL;
+	struct bpf_subprog_info *subprog;
+	struct bpf_iarray *jt, *jt_cur;
 	struct bpf_map *map;
-	struct bpf_iarray *jt_cur;
+	u32 old_cnt;
 	int i;
 
 	for (i = 0; i < env->insn_array_map_cnt; i++) {
@@ -303,72 +307,97 @@ static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env,
 		map = env->insn_array_maps[i];
 
 		jt_cur = jt_from_map(map);
-		if (IS_ERR(jt_cur)) {
-			kvfree(jt);
-			return jt_cur;
+		if (IS_ERR(jt_cur))
+			return PTR_ERR(jt_cur);
+
+		subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
+		if (!subprog) {
+			kvfree(jt_cur);
+			continue;
+		}
+		if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) {
+			subprog->jt_spans_subprogs = true;
+			kvfree(jt_cur);
+			continue;
 		}
 
-		/*
-		 * This is enough to check one element. The full table is
-		 * checked to fit inside the subprog later in create_jt()
-		 */
-		if (jt_cur->items[0] >= subprog_start && jt_cur->items[0] < subprog_end) {
-			u32 old_cnt = jt ? jt->cnt : 0;
-			jt = bpf_iarray_realloc(jt, old_cnt + jt_cur->cnt);
-			if (!jt) {
-				kvfree(jt_cur);
-				return ERR_PTR(-ENOMEM);
-			}
-			memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+		old_cnt = subprog->jt ? subprog->jt->cnt : 0;
+		jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
+		if (!jt) {
+			subprog->jt = NULL;
+			kvfree(jt_cur);
+			return -ENOMEM;
 		}
+		memcpy(jt->items + old_cnt, jt_cur->items, jt_cur->cnt << 2);
+		subprog->jt = jt;
 
 		kvfree(jt_cur);
 	}
 
-	if (!jt) {
-		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
-		bpf_diag_program_structure(
-			env, subprog_start, "missing jump table",
-			"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
-			"No jump table was found for the subprogram that starts at instruction %u.",
-			subprog_start);
-		return ERR_PTR(-EINVAL);
+	for (i = 0; i < env->subprog_cnt; i++) {
+		jt = env->subprog_info[i].jt;
+		if (jt)
+			jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
 	}
 
-	jt->cnt = sort_insn_array_uniq(jt->items, jt->cnt);
-	return jt;
+	env->cfg.subprog_jts_ready = true;
+	return 0;
+}
+
+static void free_subprog_jts(struct bpf_verifier_env *env)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
+		kvfree(env->subprog_info[i].jt);
+		env->subprog_info[i].jt = NULL;
+		env->subprog_info[i].jt_spans_subprogs = false;
+	}
+	env->cfg.subprog_jts_ready = false;
 }
 
 static struct bpf_iarray *
 create_jt(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_subprog_info *subprog;
-	int subprog_start, subprog_end;
 	struct bpf_iarray *jt;
-	int i;
+	int subprog_start, err;
+
+	if (!env->cfg.subprog_jts_ready) {
+		err = compute_subprog_jts(env);
+		if (err)
+			return ERR_PTR(err);
+	}
 
 	subprog = bpf_find_containing_subprog(env, t);
 	subprog_start = subprog->start;
-	subprog_end = (subprog + 1)->start;
-	jt = jt_from_subprog(env, subprog_start, subprog_end);
-	if (IS_ERR(jt))
-		return jt;
 
-	/* Check that the every element of the jump table fits within the given subprogram */
-	for (i = 0; i < jt->cnt; i++) {
-		if (jt->items[i] < subprog_start || jt->items[i] >= subprog_end) {
-			verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n",
-					t, subprog_start, subprog_end);
-			bpf_diag_program_structure(
-				env, t, "jump table target out of range",
-				"Keep every jump-table target inside the same subprogram.",
-				"The jump table for instruction %d points outside subprogram range [%u,%u).",
-				t, subprog_start, subprog_end);
-			kvfree(jt);
-			return ERR_PTR(-EINVAL);
-		}
+	if (subprog->jt_spans_subprogs) {
+		verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n",
+			subprog_start);
+		bpf_diag_program_structure(
+			env, subprog_start, "jump table spans subprograms",
+			"Keep every entry of a jump table inside one subprogram.",
+			"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.",
+			subprog_start, (subprog + 1)->start);
+		return ERR_PTR(-EINVAL);
+	}
+
+	if (!subprog->jt) {
+		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
+		bpf_diag_program_structure(
+			env, subprog_start, "missing jump table",
+			"Make sure subprograms containing gotox instructions are accompanied by jump tables referencing these subprograms.",
+			"No jump table was found for the subprogram that starts at instruction %u.",
+			subprog_start);
+		return ERR_PTR(-EINVAL);
 	}
 
+	jt = bpf_iarray_realloc(NULL, subprog->jt->cnt);
+	if (!jt)
+		return ERR_PTR(-ENOMEM);
+	memcpy(jt->items, subprog->jt->items, subprog->jt->cnt << 2);
+
 	return jt;
 }
 
@@ -388,6 +417,19 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 			return PTR_ERR(jt);
 
 		env->insn_aux_data[t].jt = jt;
+
+		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+				       &env->cfg.gotox_edges) ||
+		    env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+			verbose(env, "number of indirect jump edges in the program exceeds %u\n",
+				BPF_MAX_GOTOX_EDGES);
+			bpf_diag_program_structure(
+				env, t, "too many indirect jump edges",
+				"Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+				"The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+				BPF_MAX_GOTOX_EDGES);
+			return -E2BIG;
+		}
 	}
 
 	mark_prune_point(env, t);
@@ -678,6 +720,7 @@ int bpf_check_cfg(struct bpf_verifier_env *env)
 	env->prog->aux->might_sleep = env->subprog_info[0].might_sleep;
 
 err_free:
+	free_subprog_jts(env);
 	kvfree(insn_state);
 	kvfree(insn_stack);
 	env->cfg.insn_state = env->cfg.insn_stack = NULL;
@@ -749,7 +792,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	struct bpf_insn_aux_data *aux = env->insn_aux_data;
 	const u32 insn_cnt = env->prog->len;
 	int stack_sz, dfs_sz, err = 0;
-	u32 *stack, *pre, *low, *dfs;
+	u32 *stack, *pre, *low, *dfs, *dfs_pos;
 	u32 i, j, t, w;
 	u32 next_preorder_num;
 	u32 next_scc_id;
@@ -762,13 +805,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	 * - 'stack' accumulates vertices in DFS order, see invariant comment below;
 	 * - 'pre[t] == p' => preorder number of vertex 't' is 'p';
 	 * - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n';
-	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion.
+	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion;
+	 * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its
+	 *   successors at index 'j'.
 	 */
 	stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	dfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);
-	if (!stack || !pre || !low || !dfs) {
+	dfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);
+	if (!stack || !pre || !low || !dfs || !dfs_pos) {
 		err = -ENOMEM;
 		goto exit;
 	}
@@ -851,6 +897,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 		stack_sz = 0;
 		dfs_sz = 1;
 		dfs[0] = i;
+		dfs_pos[0] = 0;
 dfs_continue:
 		while (dfs_sz) {
 			w = dfs[dfs_sz - 1];
@@ -860,13 +907,37 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 				next_preorder_num++;
 				stack[stack_sz++] = w;
 			}
-			/* Visit 'w' successors */
+			/*
+			 * Visit 'w' successors, resuming at the successor this
+			 * frame last descended into. Restarting the scan at zero
+			 * on every return to 'w' would examine each successor
+			 * once per descent, i.e. quadratic in the number of
+			 * successors, which for a gotox is the size of the jump
+			 * table.
+			 *
+			 * Re-folding the successors before that index would be a
+			 * no-op. Such a successor 's' has 'pre[s] != 0' by then,
+			 * so it is never pushed onto 'dfs' again, and low[s] can
+			 * only decrease while 's' is the top of 'dfs'. If 's' is
+			 * still on 'dfs' it sits below 'w' and cannot become the
+			 * top before 'w' is popped; otherwise the only remaining
+			 * write to low[s] is the pop of its SCC, setting it to
+			 * NOT_ON_STACK, for which the min below is a no-op.
+			 */
 			succ = bpf_insn_successors(env, w);
-			for (j = 0; j < succ->cnt; ++j) {
+			for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) {
 				if (pre[succ->items[j]]) {
 					low[w] = min(low[w], low[succ->items[j]]);
 				} else {
-					dfs[dfs_sz++] = succ->items[j];
+					/*
+					 * Resume at 'j', not 'j + 1': the successor
+					 * is revisited once its DFS completes, to
+					 * fold its low[] into low[w].
+					 */
+					dfs_pos[dfs_sz - 1] = j;
+					dfs_pos[dfs_sz] = 0;
+					dfs[dfs_sz] = succ->items[j];
+					dfs_sz++;
 					goto dfs_continue;
 				}
 			}
@@ -916,5 +987,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	kvfree(pre);
 	kvfree(low);
 	kvfree(dfs);
+	kvfree(dfs_pos);
 	return err;
 }
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 72a3f5998dd27..45234e2fbee64 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -18165,11 +18165,56 @@ static int indirect_jump_min_max_index(struct bpf_verifier_env *env,
 	return 0;
 }
 
+/* 'jt' is sorted and free of duplicates, see sort_insn_array_uniq() */
+static bool jt_contains(const struct bpf_iarray *jt, u32 target)
+{
+	int l = 0, r = jt->cnt - 1, m;
+
+	while (l <= r) {
+		m = l + (r - l) / 2;
+		if (jt->items[m] == target)
+			return true;
+		if (jt->items[m] < target)
+			l = m + 1;
+		else
+			r = m - 1;
+	}
+	return false;
+}
+
+static int reject_gotox_out_of_subprog(struct bpf_verifier_env *env, u32 target,
+				       u32 subprog_start, u32 subprog_end)
+{
+	verbose(env, "indirect jump from insn %d to %u leaves the subprog [%u,%u)\n",
+		     env->insn_idx, target, subprog_start, subprog_end);
+	bpf_diag_program_structure(
+		env, env->insn_idx, "indirect jump leaves subprogram",
+		"Keep every reachable jump-table target inside the subprogram of the indirect jump.",
+		"Instruction %d can jump indirectly to instruction %u, which is outside its own subprogram [%u,%u).",
+		env->insn_idx, target, subprog_start, subprog_end);
+	return -EINVAL;
+}
+
+static int reject_gotox_without_cfg_edge(struct bpf_verifier_env *env, u32 target)
+{
+	verbose(env, "indirect jump from insn %d to %u is not in the jump table of the subprog\n",
+		     env->insn_idx, target);
+	bpf_diag_program_structure(
+		env, env->insn_idx, "indirect jump target without CFG edge",
+		"Resolve indirect jumps through a jump table whose entries all fall inside the subprogram of the jump.",
+		"Instruction %d can jump indirectly to instruction %u, which is not part of the jump table of its subprogram.",
+		env->insn_idx, target);
+	return -EINVAL;
+}
+
 /* gotox *dst_reg */
 static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)
 {
 	struct bpf_verifier_state *other_branch;
+	struct bpf_subprog_info *subprog;
+	u32 subprog_start, subprog_end;
 	struct bpf_reg_state *dst_reg;
+	struct bpf_iarray *jt;
 	struct bpf_map *map;
 	u32 min_index, max_index;
 	int err = 0;
@@ -18212,6 +18257,26 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
 		return -EINVAL;
 	}
 
+	subprog = bpf_find_containing_subprog(env, env->insn_idx);
+	if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
+		return -EFAULT;
+	subprog_start = subprog->start;
+	subprog_end = (subprog + 1)->start;
+
+	jt = env->insn_aux_data[env->insn_idx].jt;
+	if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
+		return -EFAULT;
+
+	for (i = 0; i < n; i++) {
+		u32 target = env->gotox_tmp_buf->items[i];
+
+		if (target < subprog_start || target >= subprog_end)
+			return reject_gotox_out_of_subprog(env, target, subprog_start,
+							   subprog_end);
+		if (!jt_contains(jt, target))
+			return reject_gotox_without_cfg_edge(env, target);
+	}
+
 	for (i = 0; i < n - 1; i++) {
 		mark_indirect_target(env, env->gotox_tmp_buf->items[i]);
 		other_branch = push_stack(env, env->gotox_tmp_buf->items[i],
diff --git a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
index 0222a9a5d0761..d5a831a75d820 100644
--- a/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
+++ b/tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 
 #include <bpf/bpf.h>
+#include <bpf/btf.h>
 #include <test_progs.h>
 
 #if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)
@@ -453,6 +454,940 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
+#define GOTOX_CNT_AT_LIMIT	1000
+#define GOTOX_LOG_SZ		(256 * 1024)
+
+static const char gotox_limit_msg[] =
+	"number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+	/* the run of gotox itself, plus the exit block right after it */
+	const __u32 jt_cnt = gotox_cnt + 1;
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < jt_cnt; i++) {
+		val.orig_off = first_gotox + i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
+				 int *fd_array, __u32 fd_array_cnt, char *log,
+				 int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int prog_fd;
+
+	log[0] = 0;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = fd_array_cnt;
+	opts.log_buf = log;
+	opts.log_size = GOTOX_LOG_SZ;
+	opts.log_level = 1;
+	if (fi_cnt) {
+		opts.prog_btf_fd = btf_fd;
+		opts.func_info = fi;
+		opts.func_info_cnt = fi_cnt;
+		opts.func_info_rec_size = sizeof(*fi);
+	}
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+	if (prog_fd >= 0) {
+		close(prog_fd);
+		return 0;
+	}
+	return prog_fd;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,
+				     -1, NULL, 0);
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	__u32 i;
+
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+	for (i = 1; i <= gotox_cnt; i++)
+		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+	const __u32 insn_cnt = gotox_cnt + 3;
+	struct bpf_insn *insns;
+	bool attempted = false;
+	int map_fd;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		return false;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd = gotox_jt_create(1, gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+	attempted = true;
+free_insns:
+	free(insns);
+	return attempted;
+}
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	if (try_load_gotox_prog(gotox_cnt, log, &err))
+		check_gotox_limit_hit(log, err);
+
+	free(log);
+}
+
+/*
+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
+ * table holds the starts of the blocks that follow plus the exit block,
+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
+ * exactly at the limit and must still load.
+ */
+#define GOTOX_BLOCK_SZ		4
+
+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	struct bpf_insn *at;
+	__u32 k;
+
+	for (k = 0; k < gotox_cnt; k++) {
+		at = insns + k * GOTOX_BLOCK_SZ;
+
+		/* r1 = &jt[0], by index 0 into fd_array */
+		at[0] = (struct bpf_insn) {
+			.code = BPF_LD | BPF_DW | BPF_IMM,
+			.dst_reg = BPF_REG_1,
+			.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+			.imm = 0,
+		};
+		at[1] = (struct bpf_insn) { .imm = 0 };
+		at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
+		at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	}
+
+	insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
+}
+
+static int gotox_chain_jt_create(__u32 gotox_cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < gotox_cnt; i++) {
+		val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static void check_gotox_edges_at_limit(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
+	struct bpf_insn *insns;
+	char *log;
+	int map_fd, err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_chain_fill(insns, gotox_cnt);
+
+	map_fd = gotox_chain_jt_create(gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+
+	if (!ASSERT_OK(err, "program at the edge limit should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
+static void check_gotox_edges_across_subprogs(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
+	const __u32 sub_start = gotox_cnt + 3;
+	const __u32 insn_cnt = 2 * (gotox_cnt + 3);
+	int map_fd[2] = { -1, -1 };
+	struct bpf_insn *insns;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
+					    BPF_PSEUDO_CALL, 0,
+					    sub_start - (gotox_cnt + 1) - 1);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	gotox_run_fill(insns + sub_start, gotox_cnt);
+	insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd[0] = gotox_jt_create(1, gotox_cnt);
+	if (map_fd[0] < 0)
+		goto free_insns;
+	map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
+	check_gotox_limit_hit(log, err);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < cnt; i++) {
+		val.orig_off = offs[i];
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+#define GOTOX_SUB_START		4
+#define GOTOX_MAIN_TGT		2
+#define GOTOX_SUB_TGT		8
+#define GOTOX_TWO_INSN_CNT	10
+
+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SUB_START - 1 - 1);
+	insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_EXIT_INSN();
+
+	/* r1 = &jt[0], by index 'jt_idx' into fd_array */
+	insns[GOTOX_SUB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = jt_idx,
+	};
+	insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);
+	insns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[9] = BPF_EXIT_INSN();
+}
+
+/*
+ * An insn_array map is not necessarily a jump table: one that tracks
+ * instruction offsets covers the whole program and is of no subprog. Such a
+ * map must not keep a program with a gotox elsewhere from loading.
+ */
+static void check_gotox_tracker_map(void)
+{
+	const __u32 jt_track[] = { 0, GOTOX_MAIN_TGT, GOTOX_SUB_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 1, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_track, ARRAY_SIZE(jt_track));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	if (!ASSERT_OK(err, "program with a tracking map should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static void check_gotox_target_other_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static void check_gotox_jt_per_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 1, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The spanning map is of no subprog and is dropped, and the entry the gotox
+ * register can reach is in the subprog of the gotox and in the jump table the
+ * CFG walked, so nothing unsafe is left and the program loads.
+ */
+static void check_gotox_span_unreached_entry(void)
+{
+	const __u32 jt_span[] = { GOTOX_MAIN_TGT, GOTOX_SUB_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 1);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	if (!ASSERT_OK(err, "program with an unreachable spanning entry should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_FWD_GOTOX		11
+#define GOTOX_FWD_OWN_TGT	12
+#define GOTOX_FWD_SUB_START	14
+#define GOTOX_FWD_INSN_CNT	16
+
+static void gotox_from_main_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_FWD_SUB_START - 1 - 1);
+	insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,
+			       offsetof(struct xdp_md, ingress_ifindex));
+	insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);
+
+	/* r1 = &jt_leaves[0], by index 1 into fd_array */
+	insns[4] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 1,
+	};
+	insns[5] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[7] = BPF_JMP_A(3);
+
+	/* r1 = &jt_own[0], by index 0 into fd_array */
+	insns[8] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+	insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[15] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_subprog_from_main(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The only map of the subprog holding the gotox reaches past that subprog, so
+ * the subprog is left without a jump table at all.
+ */
+static void check_gotox_jt_spans_subprogs(void)
+{
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The subprog holding the gotox has a well formed jump table of its own and
+ * also collects a map that reaches past its end. The spanning map is still
+ * rejected, even though the subprog is not left without a table.
+ */
+static void check_gotox_jt_spans_with_own_table(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_EDGE_MAIN_TGT	2
+#define GOTOX_EDGE_SUB_START	4
+#define GOTOX_EDGE_GOTOX	9
+#define GOTOX_EDGE_BR_TGT	10
+#define GOTOX_EDGE_JT_TGT	11
+#define GOTOX_EDGE_INSN_CNT	12
+
+static void gotox_no_edge_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_EDGE_SUB_START - 1 - 1);
+	insns[GOTOX_EDGE_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_EXIT_INSN();
+
+	insns[GOTOX_EDGE_SUB_START] =
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+			    offsetof(struct xdp_md, ingress_ifindex));
+	insns[5] = BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 4);
+
+	/* r1 = &jt_span[0], by index 0 into fd_array */
+	insns[6] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[7] = (struct bpf_insn) { .imm = 0 };
+	insns[8] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 8);
+
+	insns[GOTOX_EDGE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_EDGE_BR_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[GOTOX_EDGE_JT_TGT] = BPF_EXIT_INSN();
+}
+
+/*
+ * The gotox resolves a target inside its own subprog, but out of a map that
+ * spans subprogs and is therefore of no subprog. The CFG never walked that
+ * edge, so the jump has to be rejected even though it stays in the subprog.
+ */
+static void check_gotox_target_without_cfg_edge(void)
+{
+	const __u32 jt_span[] = { GOTOX_EDGE_MAIN_TGT, GOTOX_EDGE_BR_TGT };
+	const __u32 jt_sub[] = { GOTOX_EDGE_JT_TGT };
+	struct bpf_insn insns[GOTOX_EDGE_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_no_edge_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log,
+			  "indirect jump from insn 9 to 10 is not in the jump table of the subprog",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_SLICE_SUB_START	6
+#define GOTOX_SLICE_GOTOX	14
+#define GOTOX_SLICE_SUB_TGT	15
+#define GOTOX_SLICE_INSN_CNT	17
+
+static void gotox_slice_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SLICE_SUB_START - 1 - 1);
+	insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[5] = BPF_EXIT_INSN();
+
+	insns[GOTOX_SLICE_SUB_START] =
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+			    offsetof(struct xdp_md, ingress_ifindex));
+	insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);
+	insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);
+	insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[10] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[11] = (struct bpf_insn) { .imm = 0 };
+	insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);
+	insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[16] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_index_slice_other_subprog(void)
+{
+	const __u32 jt_main[] = { 2, 3, 4 };
+	const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };
+	struct bpf_insn insns[GOTOX_SLICE_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_slice_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,
+			    struct bpf_func_info *fi, struct btf **pbtf)
+{
+	int int_id, proto_id, id;
+	struct btf *btf;
+	char name[24];
+	__u32 i;
+
+	btf = btf__new_empty();
+	if (!ASSERT_OK_PTR(btf, "btf__new_empty"))
+		return -1;
+
+	int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED);
+	if (!ASSERT_GT(int_id, 0, "btf__add_int"))
+		goto err;
+
+	proto_id = btf__add_func_proto(btf, int_id);
+	if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto"))
+		goto err;
+
+	for (i = 0; i < cnt; i++) {
+		snprintf(name, sizeof(name), "gotox_f%u", i);
+		id = btf__add_func(btf, name, linkage[i], proto_id);
+		if (!ASSERT_GT(id, 0, "btf__add_func"))
+			goto err;
+		fi[i].insn_off = starts[i];
+		fi[i].type_id = id;
+	}
+
+	if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel"))
+		goto err;
+
+	*pbtf = btf;
+	return btf__fd(btf);
+err:
+	btf__free(btf);
+	return -1;
+}
+
+static void check_gotox_target_other_global_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_SUB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
+#define GOTOX_CB_MAIN_TGT	6
+#define GOTOX_CB_START		8
+#define GOTOX_CB_GOTOX		11
+#define GOTOX_CB_TGT		12
+#define GOTOX_CB_INSN_CNT	14
+
+static void gotox_callback_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);
+	/* r2 = &callback */
+	insns[1] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_2,
+		.src_reg = BPF_PSEUDO_FUNC,
+		.imm = GOTOX_CB_START - 1 - 1,
+	};
+	insns[2] = (struct bpf_insn) { .imm = 0 };
+	insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);
+	insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);
+	insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[7] = BPF_EXIT_INSN();
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[GOTOX_CB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_callback_leaves_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_CB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };
+	const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };
+	const __u32 jt_cb[] = { GOTOX_CB_TGT };
+	struct bpf_insn insns[GOTOX_CB_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_callback_fill(insns);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
 static void check_bpf_side(void)
 {
 	check_bpf_no_lookup();
@@ -490,6 +1425,48 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
+
+	if (test__start_subtest("too-many-gotox-edges"))
+		check_too_many_gotox_edges();
+
+	if (test__start_subtest("gotox-edges-at-limit"))
+		check_gotox_edges_at_limit();
+
+	if (test__start_subtest("gotox-edges-across-subprogs"))
+		check_gotox_edges_across_subprogs();
+
+	if (test__start_subtest("gotox-tracker-map"))
+		check_gotox_tracker_map();
+
+	if (test__start_subtest("gotox-jt-spans-subprogs"))
+		check_gotox_jt_spans_subprogs();
+
+	if (test__start_subtest("gotox-jt-spans-with-own-table"))
+		check_gotox_jt_spans_with_own_table();
+
+	if (test__start_subtest("gotox-target-without-cfg-edge"))
+		check_gotox_target_without_cfg_edge();
+
+	if (test__start_subtest("gotox-target-other-subprog"))
+		check_gotox_target_other_subprog();
+
+	if (test__start_subtest("gotox-jt-per-subprog"))
+		check_gotox_jt_per_subprog();
+
+	if (test__start_subtest("gotox-span-unreached-entry"))
+		check_gotox_span_unreached_entry();
+
+	if (test__start_subtest("gotox-target-subprog-from-main"))
+		check_gotox_target_subprog_from_main();
+
+	if (test__start_subtest("gotox-index-slice-other-subprog"))
+		check_gotox_index_slice_other_subprog();
+
+	if (test__start_subtest("gotox-target-other-global-subprog"))
+		check_gotox_target_other_global_subprog();
+
+	if (test__start_subtest("gotox-callback-leaves-subprog"))
+		check_gotox_callback_leaves_subprog();
 }
 #else
 static void __test_bpf_insn_array(void)
diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index 5b18c9a27717b..3567b29e23789 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -318,7 +318,7 @@ __used static int test_subprog(void)
 }
 
 SEC("socket")
-__failure __msg("jump table for insn 4 points outside of the subprog [0,10]")
+__failure __msg("jump table of subprog starting at 0 spans multiple subprogs")
 __naked void jump_table_outside_subprog(void)
 {
 	asm volatile ("						\

							input: 21993
							output: 29
							thoughts: 497

**Analyzing the BPF Patch for Memory Safety**

Okay, let's dive into this patch and see if we can trigger any nasty uninitialized memory reads or leaks, the kind that only KMSAN might pick up. First, I'm going to take a really close look at the details.

The changes are spread across these files: `include/linux/bpf_verifier.h`, `kernel/bpf/cfg.c`, `kernel/bpf/verifier.c`, and a bunch of selftests.

* In `bpf_verifier.h`, we've got a couple of additions: a `struct bpf_iarray *jt;` and a `bool jt_spans_subprogs: 1;` field in `struct bpf_subprog_info`. It looks like these are for handling jump tables. The other struct, `bpf_verifier_env`, got changes in its `cfg` member, with `u32 gotox_edges;` and `bool subprog_jts_ready;` being added. I'm guessing this relates to tracking control flow graph edges.

* `cfg.c` is where the core logic seems to be. We've replaced the old `jt_from_subprog()` with a more efficient `compute_subprog_jts()`, caching jump tables in subprog info. There's also some cleanup in `free_subprog_jts` and modifications to `create_jt`. `visit_gotox_insn` is now checking a `gotox_edges` limit. Importantly, `bpf_compute_scc` is using `kvcalloc` for a `dfs_pos` array. I need to keep an eye on how that's used.

* The verifier is where the actual checks happen, with new functions like `jt_contains`, `reject_gotox_out_of_subprog`, and `reject_gotox_without_cfg_edge`. `check_indirect_jump` is making sure the targets stay within the subprogram and are in the jump table, which is good.

* Finally, the selftests are there to, hopefully, cover some of the new functionality.

Now, the memory management is where it gets interesting. I need to verify that all the memory is correctly allocated and initialized. Let's start with `bpf_verifier_env` and `bpf_subprog_info`. Where are those allocated and, crucially, *cleared*? That's the key. I'm going to use `grepper` and see how `subprog_info` is initialized. Let's track its lifecycle to make sure nothing is left hanging and uninitialized. I need to make sure the jump tables are properly handled as well. This all has the potential to introduce subtle issues, especially around control flow changes. Time to dive deeper.

{
  "Expression": "subprog_info",
  "PathPrefix": "kernel/bpf/"
}

{
  "Output": "\nFull output is too long, showing 500 out of 602 lines.\nUse more precise expression if possible.\n\nkernel/bpf/btf.c=8016=int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)\n--\nkernel/bpf/btf.c-8018-\tbool is_global = subprog_aux(env, subprog)-\u003elinkage == BTF_FUNC_GLOBAL;\nkernel/bpf/btf.c:8019:\tstruct bpf_subprog_info *sub = subprog_info(env, subprog);\nkernel/bpf/btf.c-8020-\tstruct bpf_verifier_log *log = \u0026env-\u003elog;\n--\nkernel/bpf/cfg.c=54=static void mark_subprog_changes_pkt_data(struct bpf_verifier_env *env, int off)\nkernel/bpf/cfg.c-55-{\nkernel/bpf/cfg.c:56:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-57-\n--\nkernel/bpf/cfg.c=62=static void mark_subprog_might_sleep(struct bpf_verifier_env *env, int off)\nkernel/bpf/cfg.c-63-{\nkernel/bpf/cfg.c:64:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-65-\n--\nkernel/bpf/cfg.c=70=static void mark_subprog_might_throw(struct bpf_verifier_env *env, int off)\nkernel/bpf/cfg.c-71-{\nkernel/bpf/cfg.c:72:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-73-\n--\nkernel/bpf/cfg.c=84=static void merge_callee_effects(struct bpf_verifier_env *env, int t, int w)\nkernel/bpf/cfg.c-85-{\nkernel/bpf/cfg.c:86:\tstruct bpf_subprog_info *caller, *callee;\nkernel/bpf/cfg.c-87-\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-295-{\nkernel/bpf/cfg.c:296:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-297-\tstruct bpf_iarray *jt, *jt_cur;\n--\nkernel/bpf/cfg.c-337-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/cfg.c:338:\t\tjt = env-\u003esubprog_info[i].jt;\nkernel/bpf/cfg.c-339-\t\tif (jt)\n--\nkernel/bpf/cfg.c=347=static void free_subprog_jts(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-350-\nkernel/bpf/cfg.c:351:\tfor (i = 0; i \u003c ARRAY_SIZE(env-\u003esubprog_info); i++) {\nkernel/bpf/cfg.c:352:\t\tkvfree(env-\u003esubprog_info[i].jt);\nkernel/bpf/cfg.c:353:\t\tenv-\u003esubprog_info[i].jt = NULL;\nkernel/bpf/cfg.c:354:\t\tenv-\u003esubprog_info[i].jt_spans_subprogs = false;\nkernel/bpf/cfg.c-355-\t}\n--\nkernel/bpf/cfg.c=360=create_jt(int t, struct bpf_verifier_env *env)\nkernel/bpf/cfg.c-361-{\nkernel/bpf/cfg.c:362:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-363-\tstruct bpf_iarray *jt;\n--\nkernel/bpf/cfg.c=471=static int visit_abnormal_return_insn(struct bpf_verifier_env *env, int t)\nkernel/bpf/cfg.c-472-{\nkernel/bpf/cfg.c:473:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/cfg.c-474-\tstruct bpf_iarray *jt;\n--\nkernel/bpf/cfg.c=632=int bpf_check_cfg(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-650-\tex_insn_beg = env-\u003eexception_callback_subprog\nkernel/bpf/cfg.c:651:\t\t      ? env-\u003esubprog_info[env-\u003eexception_callback_subprog].start\nkernel/bpf/cfg.c-652-\t\t      : 0;\n--\nkernel/bpf/cfg.c-718-\tret = 0; /* cfg looks good */\nkernel/bpf/cfg.c:719:\tenv-\u003eprog-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[0].changes_pkt_data;\nkernel/bpf/cfg.c:720:\tenv-\u003eprog-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[0].might_sleep;\nkernel/bpf/cfg.c-721-\n--\nkernel/bpf/cfg.c-731- * For each subprogram 'i' fill array env-\u003ecfg.insn_subprogram sub-range\nkernel/bpf/cfg.c:732: * [env-\u003esubprog_info[i].postorder_start, env-\u003esubprog_info[i+1].postorder_start)\nkernel/bpf/cfg.c-733- * with indices of 'i' instructions in postorder.\n--\nkernel/bpf/cfg.c=735=int bpf_compute_postorder(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-751-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/cfg.c:752:\t\tenv-\u003esubprog_info[i].postorder_start = cur_postorder;\nkernel/bpf/cfg.c:753:\t\tstack[0] = env-\u003esubprog_info[i].start;\nkernel/bpf/cfg.c-754-\t\tstack_sz = 1;\n--\nkernel/bpf/cfg.c-772-\t}\nkernel/bpf/cfg.c:773:\tenv-\u003esubprog_info[i].postorder_start = cur_postorder;\nkernel/bpf/cfg.c-774-\tenv-\u003ecfg.insn_postorder = postorder;\n--\nkernel/bpf/check_btf.c=10=static int check_abnormal_return(struct bpf_verifier_env *env)\n--\nkernel/bpf/check_btf.c-14-\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/check_btf.c:15:\t\tif (env-\u003esubprog_info[i].has_ld_abs) {\nkernel/bpf/check_btf.c-16-\t\t\tverbose(env, \"LD_ABS is not allowed in subprogs without BTF\\n\");\n--\nkernel/bpf/check_btf.c-18-\t\t}\nkernel/bpf/check_btf.c:19:\t\tif (env-\u003esubprog_info[i].has_tail_call) {\nkernel/bpf/check_btf.c-20-\t\t\tverbose(env, \"tail_call is not allowed in subprogs without BTF\\n\");\n--\nkernel/bpf/check_btf.c=133=static int check_btf_func(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-174-\nkernel/bpf/check_btf.c:175:\t\tif (env-\u003esubprog_info[i].start != krecord[i].insn_off) {\nkernel/bpf/check_btf.c-176-\t\t\tverbose(env, \"func_info BTF section doesn't match subprog layout in BPF program\\n\");\n--\nkernel/bpf/check_btf.c-188-\t\t\tbtf_type_is_small_int(ret_type) || btf_is_any_enum(ret_type);\nkernel/bpf/check_btf.c:189:\t\tif (i \u0026\u0026 !scalar_return \u0026\u0026 env-\u003esubprog_info[i].has_ld_abs) {\nkernel/bpf/check_btf.c-190-\t\t\tverbose(env, \"LD_ABS is only allowed in functions that return 'int'.\\n\");\n--\nkernel/bpf/check_btf.c-192-\t\t}\nkernel/bpf/check_btf.c:193:\t\tif (i \u0026\u0026 !scalar_return \u0026\u0026 env-\u003esubprog_info[i].has_tail_call) {\nkernel/bpf/check_btf.c-194-\t\t\tverbose(env, \"tail_call is only allowed in functions that return 'int'.\\n\");\n--\nkernel/bpf/check_btf.c-197-\nkernel/bpf/check_btf.c:198:\t\tenv-\u003esubprog_info[i].name = btf_name_by_offset(btf, type-\u003ename_off);\nkernel/bpf/check_btf.c-199-\t\tbpfptr_add(\u0026urecord, urec_size);\n--\nkernel/bpf/check_btf.c=213=static int check_btf_line(struct bpf_verifier_env *env,\n--\nkernel/bpf/check_btf.c-217-\tu32 i, s, nr_linfo, ncopy, expected_size, rec_size, prev_offset = 0;\nkernel/bpf/check_btf.c:218:\tstruct bpf_subprog_info *sub;\nkernel/bpf/check_btf.c-219-\tstruct bpf_line_info *linfo;\n--\nkernel/bpf/check_btf.c-248-\ts = 0;\nkernel/bpf/check_btf.c:249:\tsub = env-\u003esubprog_info;\nkernel/bpf/check_btf.c-250-\tulinfo = make_bpfptr(attr-\u003eline_info, uattr.is_kernel);\n--\nkernel/bpf/check_btf.c-279-\t\t * first sub also and the first sub must have\nkernel/bpf/check_btf.c:280:\t\t * subprog_info[0].start == 0.\nkernel/bpf/check_btf.c-281-\t\t */\n--\nkernel/bpf/const_fold.c=232=int bpf_compute_const_regs(struct bpf_verifier_env *env)\n--\nkernel/bpf/const_fold.c-250-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/const_fold.c:251:\t\tint start = env-\u003esubprog_info[i].start;\nkernel/bpf/const_fold.c-252-\n--\nkernel/bpf/core.c=1650=u16 bpf_out_stack_arg_cnt(const struct bpf_verifier_env *env, const struct bpf_prog *prog)\nkernel/bpf/core.c-1651-{\nkernel/bpf/core.c:1652:\tconst struct bpf_subprog_info *sub;\nkernel/bpf/core.c-1653-\n--\nkernel/bpf/core.c-1655-\t\treturn 0;\nkernel/bpf/core.c:1656:\tsub = \u0026env-\u003esubprog_info[prog-\u003eaux-\u003efunc_idx];\nkernel/bpf/core.c-1657-\treturn sub-\u003estack_arg_cnt - bpf_in_stack_arg_cnt(sub);\n--\nkernel/bpf/diagnostics.c=797=static void bpf_diag_source(struct bpf_verifier_env *env, u32 insn_idx, const char *label,\n--\nkernel/bpf/diagnostics.c-805-\tconst struct bpf_line_info *linfo;\nkernel/bpf/diagnostics.c:806:\tconst struct bpf_subprog_info *subprog;\nkernel/bpf/diagnostics.c-807-\tstruct btf *btf = env-\u003eprog-\u003eaux-\u003ebtf;\n--\nkernel/bpf/diagnostics.c-841-\tsubprog = bpf_find_containing_subprog(env, insn_idx);\nkernel/bpf/diagnostics.c:842:\tsubprogno = subprog ? subprog - env-\u003esubprog_info : -ENOENT;\nkernel/bpf/diagnostics.c-843-\tfunc = subprogno \u003e= 0 ? bpf_subprog_name(env, subprogno) : NULL;\n--\nkernel/bpf/diagnostics.c-860-\tlinfo_end = subprogno \u003e= 0 \u0026\u0026 subprogno + 1 \u003c env-\u003esubprog_cnt ?\nkernel/bpf/diagnostics.c:861:\t\t    env-\u003esubprog_info[subprogno + 1].linfo_idx : env-\u003eprog-\u003eaux-\u003enr_linfo;\nkernel/bpf/diagnostics.c-862-\tfor (i = linfo_start; i \u003c linfo_end; i++) {\n--\nkernel/bpf/fixups.c=204=static int get_callee_stack_depth(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-211-\t\treturn -EFAULT;\nkernel/bpf/fixups.c:212:\treturn env-\u003esubprog_info[subprog].stack_depth;\nkernel/bpf/fixups.c-213-}\n--\nkernel/bpf/fixups.c=262=static void adjust_subprog_starts(struct bpf_verifier_env *env, u32 off, u32 len)\n--\nkernel/bpf/fixups.c-269-\tfor (i = 0; i \u003c= env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:270:\t\tif (env-\u003esubprog_info[i].start \u003c= off)\nkernel/bpf/fixups.c-271-\t\t\tcontinue;\nkernel/bpf/fixups.c:272:\t\tenv-\u003esubprog_info[i].start += len - 1;\nkernel/bpf/fixups.c-273-\t}\n--\nkernel/bpf/fixups.c=379=static int adjust_subprog_starts_after_remove(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-385-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:386:\t\tif (env-\u003esubprog_info[i].start \u003e= off)\nkernel/bpf/fixups.c-387-\t\t\tbreak;\n--\nkernel/bpf/fixups.c-389-\tfor (j = i; j \u003c env-\u003esubprog_cnt; j++)\nkernel/bpf/fixups.c:390:\t\tif (env-\u003esubprog_info[j].start \u003e= off + cnt)\nkernel/bpf/fixups.c-391-\t\t\tbreak;\n--\nkernel/bpf/fixups.c-394-\t */\nkernel/bpf/fixups.c:395:\tif (env-\u003esubprog_info[j].start != off + cnt)\nkernel/bpf/fixups.c-396-\t\tj--;\n--\nkernel/bpf/fixups.c-404-\nkernel/bpf/fixups.c:405:\t\tmemmove(env-\u003esubprog_info + i,\nkernel/bpf/fixups.c:406:\t\t\tenv-\u003esubprog_info + j,\nkernel/bpf/fixups.c:407:\t\t\tsizeof(*env-\u003esubprog_info) * move);\nkernel/bpf/fixups.c-408-\t\tenv-\u003esubprog_cnt -= j - i;\n--\nkernel/bpf/fixups.c-427-\t\t/* convert i from \"first prog to remove\" to \"first to adjust\" */\nkernel/bpf/fixups.c:428:\t\tif (env-\u003esubprog_info[i].start == off)\nkernel/bpf/fixups.c-429-\t\t\ti++;\n--\nkernel/bpf/fixups.c-433-\tfor (; i \u003c= env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:434:\t\tenv-\u003esubprog_info[i].start -= cnt;\nkernel/bpf/fixups.c-435-\n--\nkernel/bpf/fixups.c=439=static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,\n--\nkernel/bpf/fixups.c-489-\tfor (i = 0; i \u003c= env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:490:\t\tif (env-\u003esubprog_info[i].linfo_idx \u003e l_off) {\nkernel/bpf/fixups.c-491-\t\t\t/* program may have started in the removed region but\n--\nkernel/bpf/fixups.c-493-\t\t\t */\nkernel/bpf/fixups.c:494:\t\t\tif (env-\u003esubprog_info[i].linfo_idx \u003e= l_off + l_cnt)\nkernel/bpf/fixups.c:495:\t\t\t\tenv-\u003esubprog_info[i].linfo_idx -= l_cnt;\nkernel/bpf/fixups.c-496-\t\t\telse\nkernel/bpf/fixups.c:497:\t\t\t\tenv-\u003esubprog_info[i].linfo_idx = l_off;\nkernel/bpf/fixups.c-498-\t\t}\n--\nkernel/bpf/fixups.c=746=int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-747-{\nkernel/bpf/fixups.c:748:\tstruct bpf_subprog_info *subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-749-\tconst struct bpf_verifier_ops *ops = env-\u003eops;\n--\nkernel/bpf/fixups.c=1051=static u32 *bpf_dup_subprog_starts(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1057-\t\tfor (int i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:1058:\t\t\tstarts[i] = env-\u003esubprog_info[i].start;\nkernel/bpf/fixups.c-1059-\t}\n--\nkernel/bpf/fixups.c=1063=static void bpf_restore_subprog_starts(struct bpf_verifier_env *env, u32 *orig_starts)\n--\nkernel/bpf/fixups.c-1065-\tfor (int i = 0; i \u003c env-\u003esubprog_cnt; i++)\nkernel/bpf/fixups.c:1066:\t\tenv-\u003esubprog_info[i].start = orig_starts[i];\nkernel/bpf/fixups.c-1067-\t/* restore the start of fake 'exit' subprog as well */\nkernel/bpf/fixups.c:1068:\tenv-\u003esubprog_info[env-\u003esubprog_cnt].start = env-\u003eprog-\u003elen;\nkernel/bpf/fixups.c-1069-}\n--\nkernel/bpf/fixups.c=1071=static int jit_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1127-\t\tsubprog_start = subprog_end;\nkernel/bpf/fixups.c:1128:\t\tsubprog_end = env-\u003esubprog_info[i + 1].start;\nkernel/bpf/fixups.c-1129-\n--\nkernel/bpf/fixups.c-1167-\t\tfunc[i]-\u003eaux-\u003ename[0] = 'F';\nkernel/bpf/fixups.c:1168:\t\tfunc[i]-\u003eaux-\u003estack_depth = env-\u003esubprog_info[i].stack_depth;\nkernel/bpf/fixups.c:1169:\t\tif (env-\u003esubprog_info[i].priv_stack_mode == PRIV_STACK_ADAPTIVE)\nkernel/bpf/fixups.c-1170-\t\t\tfunc[i]-\u003eaux-\u003ejits_use_priv_stack = true;\n--\nkernel/bpf/fixups.c-1178-\t\tfunc[i]-\u003eaux-\u003ejited_linfo = prog-\u003eaux-\u003ejited_linfo;\nkernel/bpf/fixups.c:1179:\t\tfunc[i]-\u003eaux-\u003elinfo_idx = env-\u003esubprog_info[i].linfo_idx;\nkernel/bpf/fixups.c-1180-\t\tfunc[i]-\u003eaux-\u003earena = prog-\u003eaux-\u003earena;\n--\nkernel/bpf/fixups.c-1200-\t\tfunc[i]-\u003eaux-\u003enum_exentries = num_exentries;\nkernel/bpf/fixups.c:1201:\t\tfunc[i]-\u003eaux-\u003etail_call_reachable = env-\u003esubprog_info[i].tail_call_reachable;\nkernel/bpf/fixups.c:1202:\t\tfunc[i]-\u003eaux-\u003eexception_cb = env-\u003esubprog_info[i].is_exception_cb;\nkernel/bpf/fixups.c:1203:\t\tfunc[i]-\u003eaux-\u003echanges_pkt_data = env-\u003esubprog_info[i].changes_pkt_data;\nkernel/bpf/fixups.c:1204:\t\tfunc[i]-\u003eaux-\u003emight_sleep = env-\u003esubprog_info[i].might_sleep;\nkernel/bpf/fixups.c-1205-\t\tfunc[i]-\u003eaux-\u003etoken = prog-\u003eaux-\u003etoken;\n--\nkernel/bpf/fixups.c=1410=int bpf_fixup_call_args(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1419-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:1420:\t\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[i];\nkernel/bpf/fixups.c-1421-\t\tu16 outgoing = subprog-\u003estack_arg_cnt - bpf_in_stack_arg_cnt(subprog);\n--\nkernel/bpf/fixups.c-1444-\tfor (i = 0; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/fixups.c:1445:\t\tif (bpf_in_stack_arg_cnt(\u0026env-\u003esubprog_info[i])) {\nkernel/bpf/fixups.c-1446-\t\t\tverbose(env, \"stack args are not supported in non-JITed programs\\n\");\n--\nkernel/bpf/fixups.c=1484=static int add_hidden_subprog(struct bpf_verifier_env *env, struct bpf_insn *patch, int len)\nkernel/bpf/fixups.c-1485-{\nkernel/bpf/fixups.c:1486:\tstruct bpf_subprog_info *info = env-\u003esubprog_info;\nkernel/bpf/fixups.c-1487-\tint cnt = env-\u003esubprog_cnt;\n--\nkernel/bpf/fixups.c-1489-\nkernel/bpf/fixups.c:1490:\t/* We only reserve one slot for hidden subprogs in subprog_info. */\nkernel/bpf/fixups.c-1491-\tif (env-\u003ehidden_subprog_cnt) {\n--\nkernel/bpf/fixups.c=1513=int bpf_do_misc_fixups(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1526-\tint i, ret, cnt, delta = 0, cur_subprog = 0;\nkernel/bpf/fixups.c:1527:\tstruct bpf_subprog_info *subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-1528-\tu16 stack_depth = subprogs[cur_subprog].stack_depth;\n--\nkernel/bpf/fixups.c=2499=static struct bpf_prog *inline_bpf_loop(struct bpf_verifier_env *env,\n--\nkernel/bpf/fixups.c-2565-\t/* callback start is known only after patching */\nkernel/bpf/fixups.c:2566:\tcallback_start = env-\u003esubprog_info[callback_subprogno].start;\nkernel/bpf/fixups.c-2567-\t/* Note: insn_buf[12] is an offset of BPF_CALL_REL instruction */\n--\nkernel/bpf/fixups.c=2591=int bpf_optimize_bpf_loop(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-2592-{\nkernel/bpf/fixups.c:2593:\tstruct bpf_subprog_info *subprogs = env-\u003esubprog_info;\nkernel/bpf/fixups.c-2594-\tint i, cur_subprog = 0, cnt, delta = 0;\n--\nkernel/bpf/fixups.c-2630-\nkernel/bpf/fixups.c:2631:\tenv-\u003eprog-\u003eaux-\u003estack_depth = env-\u003esubprog_info[0].stack_depth;\nkernel/bpf/fixups.c-2632-\n--\nkernel/bpf/fixups.c=2639=int bpf_remove_fastcall_spills_fills(struct bpf_verifier_env *env)\nkernel/bpf/fixups.c-2640-{\nkernel/bpf/fixups.c:2641:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/fixups.c-2642-\tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/liveness.c=24=struct func_instance {\n--\nkernel/bpf/liveness.c-28-\tu32 subprog;\t\t/* subprog index */\nkernel/bpf/liveness.c:29:\tu32 subprog_start;\t/* cached env-\u003esubprog_info[subprog].start */\nkernel/bpf/liveness.c-30-\tu32 insn_cnt;\t\t/* cached number of insns in the function */\n--\nkernel/bpf/liveness.c=74=static struct func_instance *call_instance(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-78-\tu32 depth = caller ? caller-\u003edepth + 1 : 0;\nkernel/bpf/liveness.c:79:\tu32 subprog_start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c-80-\tu32 lookup_key = depth \u003e 0 ? callsite : subprog_start;\n--\nkernel/bpf/liveness.c-94-\tf-\u003esubprog_start = subprog_start;\nkernel/bpf/liveness.c:95:\tf-\u003einsn_cnt = (env-\u003esubprog_info + subprog + 1)-\u003estart - subprog_start;\nkernel/bpf/liveness.c-96-\thash = instance_hash(lookup_key, depth);\n--\nkernel/bpf/liveness.c=101=static struct func_instance *lookup_instance(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-108-\nkernel/bpf/liveness.c:109:\tsubprog_start = env-\u003esubprog_info[st-\u003eframe[frameno]-\u003esubprogno].start;\nkernel/bpf/liveness.c-110-\tcallsite = frameno \u003e 0 ? st-\u003eframe[frameno]-\u003ecallsite : subprog_start;\n--\nkernel/bpf/liveness.c=306=static void update_instance(struct bpf_verifier_env *env, struct func_instance *instance)\n--\nkernel/bpf/liveness.c-309-\tint *insn_postorder = env-\u003ecfg.insn_postorder;\nkernel/bpf/liveness.c:310:\tstruct bpf_subprog_info *subprog;\nkernel/bpf/liveness.c-311-\tbool changed;\n--\nkernel/bpf/liveness.c-313-\tinstance-\u003emust_write_initialized = true;\nkernel/bpf/liveness.c:314:\tsubprog = \u0026env-\u003esubprog_info[instance-\u003esubprog];\nkernel/bpf/liveness.c-315-\tpo_start = subprog-\u003epostorder_start;\n--\nkernel/bpf/liveness.c=407=static char *fmt_subprog(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/liveness.c-408-{\nkernel/bpf/liveness.c:409:\tconst char *name = env-\u003esubprog_info[subprog].name;\nkernel/bpf/liveness.c-410-\n--\nkernel/bpf/liveness.c=477=static void print_instance(struct bpf_verifier_env *env, struct func_instance *instance)\nkernel/bpf/liveness.c-478-{\nkernel/bpf/liveness.c:479:\tint start = env-\u003esubprog_info[instance-\u003esubprog].start;\nkernel/bpf/liveness.c-480-\tstruct bpf_insn *insns = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/liveness.c=1517=static void print_subprog_arg_access(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-1522-\tstruct bpf_insn *insns = env-\u003eprog-\u003einsnsi;\nkernel/bpf/liveness.c:1523:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c-1524-\tint len = info-\u003elen;\n--\nkernel/bpf/liveness.c=1607=static int compute_subprog_args(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-1615-\tint depth = instance-\u003edepth;\nkernel/bpf/liveness.c:1616:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c:1617:\tint po_start = env-\u003esubprog_info[subprog].postorder_start;\nkernel/bpf/liveness.c:1618:\tint end = env-\u003esubprog_info[subprog + 1].start;\nkernel/bpf/liveness.c:1619:\tint po_end = env-\u003esubprog_info[subprog + 1].postorder_start;\nkernel/bpf/liveness.c-1620-\tint len = end - start;\n--\nkernel/bpf/liveness.c=1840=static int analyze_subprog(struct bpf_verifier_env *env,\n--\nkernel/bpf/liveness.c-1848-\tstruct bpf_insn *insns = env-\u003eprog-\u003einsnsi;\nkernel/bpf/liveness.c:1849:\tint start = env-\u003esubprog_info[subprog].start;\nkernel/bpf/liveness.c:1850:\tint po_start = env-\u003esubprog_info[subprog].postorder_start;\nkernel/bpf/liveness.c:1851:\tint po_end = env-\u003esubprog_info[subprog + 1].postorder_start;\nkernel/bpf/liveness.c-1852-\tstruct func_instance *prev_instance = NULL;\n--\nkernel/bpf/verifier.c=431=void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/verifier.c-432-{\nkernel/bpf/verifier.c:433:\tstruct bpf_subprog_info *info = subprog_info(env, subprog);\nkernel/bpf/verifier.c-434-\n--\nkernel/bpf/verifier.c=440=static bool subprog_is_exc_cb(struct bpf_verifier_env *env, int subprog)\nkernel/bpf/verifier.c-441-{\nkernel/bpf/verifier.c:442:\treturn subprog_info(env, subprog)-\u003eis_exception_cb;\nkernel/bpf/verifier.c-443-}\n--\nkernel/bpf/verifier.c=1369=static int resize_reference_state(struct bpf_verifier_state *state, size_t n)\n--\nkernel/bpf/verifier.c-1380-/* Possibly update state-\u003eallocated_stack to be at least size bytes. Also\nkernel/bpf/verifier.c:1381: * possibly update the function's high-water mark in its bpf_subprog_info.\nkernel/bpf/verifier.c-1382- */\nkernel/bpf/verifier.c=1383=static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state *state, int size)\n--\nkernel/bpf/verifier.c-1400-\t/* update known max for given subprogram */\nkernel/bpf/verifier.c:1401:\tif (env-\u003esubprog_info[state-\u003esubprogno].stack_depth \u003c size)\nkernel/bpf/verifier.c:1402:\t\tenv-\u003esubprog_info[state-\u003esubprogno].stack_depth = size;\nkernel/bpf/verifier.c-1403-\n--\nkernel/bpf/verifier.c=2352=static int cmp_subprogs(const void *a, const void *b)\nkernel/bpf/verifier.c-2353-{\nkernel/bpf/verifier.c:2354:\treturn ((struct bpf_subprog_info *)a)-\u003estart -\nkernel/bpf/verifier.c:2355:\t       ((struct bpf_subprog_info *)b)-\u003estart;\nkernel/bpf/verifier.c-2356-}\n--\nkernel/bpf/verifier.c-2358-/* Find subprogram that contains instruction at 'off' */\nkernel/bpf/verifier.c:2359:struct bpf_subprog_info *bpf_find_containing_subprog(struct bpf_verifier_env *env, int off)\nkernel/bpf/verifier.c-2360-{\nkernel/bpf/verifier.c:2361:\tstruct bpf_subprog_info *vals = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2362-\tint l, r, m;\n--\nkernel/bpf/verifier.c=2380=int bpf_find_subprog(struct bpf_verifier_env *env, int off)\nkernel/bpf/verifier.c-2381-{\nkernel/bpf/verifier.c:2382:\tstruct bpf_subprog_info *p;\nkernel/bpf/verifier.c-2383-\n--\nkernel/bpf/verifier.c-2386-\t\treturn -ENOENT;\nkernel/bpf/verifier.c:2387:\treturn p - env-\u003esubprog_info;\nkernel/bpf/verifier.c-2388-}\n--\nkernel/bpf/verifier.c=2390=static int add_subprog(struct bpf_verifier_env *env, int off)\n--\nkernel/bpf/verifier.c-2406-\t/* determine subprog starts. The end is one before the next starts */\nkernel/bpf/verifier.c:2407:\tenv-\u003esubprog_info[env-\u003esubprog_cnt++].start = off;\nkernel/bpf/verifier.c:2408:\tsort(env-\u003esubprog_info, env-\u003esubprog_cnt,\nkernel/bpf/verifier.c:2409:\t     sizeof(env-\u003esubprog_info[0]), cmp_subprogs, NULL);\nkernel/bpf/verifier.c-2410-\treturn env-\u003esubprog_cnt - 1;\n--\nkernel/bpf/verifier.c=2919=static int add_subprogs(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-2920-{\nkernel/bpf/verifier.c:2921:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-2922-\tint i, ret, insn_cnt = env-\u003eprog-\u003elen, ex_cb_insn;\n--\nkernel/bpf/verifier.c-2968-\t\tfor (i = 1; i \u003c env-\u003esubprog_cnt; i++) {\nkernel/bpf/verifier.c:2969:\t\t\tif (env-\u003esubprog_info[i].start != ex_cb_insn)\nkernel/bpf/verifier.c-2970-\t\t\t\tcontinue;\n--\nkernel/bpf/verifier.c=3015=static int check_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-3017-\tint i, subprog_start, subprog_end, off, cur_subprog = 0;\nkernel/bpf/verifier.c:3018:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3019-\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=3089=static int sort_subprogs_topo(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-3090-{\nkernel/bpf/verifier.c:3091:\tstruct bpf_subprog_info *si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-3092-\tint *insn_postorder = env-\u003ecfg.insn_postorder;\n--\nkernel/bpf/verifier.c=3477=static void check_fastcall_stack_contract(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-3479-{\nkernel/bpf/verifier.c:3480:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-3481-\tstruct bpf_insn_aux_data *aux = env-\u003einsn_aux_data;\n--\nkernel/bpf/verifier.c=4167=static int check_stack_arg_write(struct bpf_verifier_env *env, struct bpf_func_state *state,\n--\nkernel/bpf/verifier.c-4170-\tint max_stack_arg_regs = MAX_BPF_FUNC_ARGS - MAX_BPF_FUNC_REG_ARGS;\nkernel/bpf/verifier.c:4171:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4172-\tint spi = -off / BPF_REG_SIZE - 1;\n--\nkernel/bpf/verifier.c=4210=static int check_stack_arg_read(struct bpf_verifier_env *env, struct bpf_func_state *state,\n--\nkernel/bpf/verifier.c-4212-{\nkernel/bpf/verifier.c:4213:\tstruct bpf_subprog_info *subprog = \u0026env-\u003esubprog_info[state-\u003esubprogno];\nkernel/bpf/verifier.c-4214-\tstruct bpf_verifier_state *vstate = env-\u003ecur_state;\n--\nkernel/bpf/verifier.c=5317=static int check_max_stack_depth_subprog(struct bpf_verifier_env *env, int idx,\n--\nkernel/bpf/verifier.c-5320-{\nkernel/bpf/verifier.c:5321:\tstruct bpf_subprog_info *subprog = env-\u003esubprog_info;\nkernel/bpf/verifier.c-5322-\tstruct bpf_insn *insn = env-\u003eprog-\u003einsnsi;\n--\nkernel/bpf/verifier.c=5519=static int check_max_stack_depth(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-5522-\tstruct bpf_subprog_call_depth_info *dinfo;\nkernel/bpf/verifier.c:5523:\tstruct bpf_subprog_info *si = env-\u003esubprog_info;\nkernel/bpf/verifier.c-5524-\tbool priv_stack_supported;\n--\nkernel/bpf/verifier.c=9668=static int btf_check_func_arg_match(struct bpf_verifier_env *env, int subprog,\n--\nkernel/bpf/verifier.c-9671-{\nkernel/bpf/verifier.c:9672:\tstruct bpf_subprog_info *sub = subprog_info(env, subprog);\nkernel/bpf/verifier.c-9673-\tstruct bpf_func_state *caller = cur_func(env);\n--\nkernel/bpf/verifier.c=9830=static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-9846-\t */\nkernel/bpf/verifier.c:9847:\tenv-\u003esubprog_info[subprog].is_cb = true;\nkernel/bpf/verifier.c-9848-\tif (bpf_pseudo_kfunc_call(insn) \u0026\u0026\n--\nkernel/bpf/verifier.c-9863-\t\t/* there is no real recursion here. timer and workqueue callbacks are async */\nkernel/bpf/verifier.c:9864:\t\tenv-\u003esubprog_info[subprog].is_async_cb = true;\nkernel/bpf/verifier.c:9865:\t\tasync_cb = push_async_cb(env, env-\u003esubprog_info[subprog].start,\nkernel/bpf/verifier.c-9866-\t\t\t\t\t insn_idx, subprog,\n--\nkernel/bpf/verifier.c-9883-\t */\nkernel/bpf/verifier.c:9884:\tcallback_state = push_stack(env, env-\u003esubprog_info[subprog].start, insn_idx, false);\nkernel/bpf/verifier.c-9885-\tif (IS_ERR(callback_state))\n--\nkernel/bpf/verifier.c=9902=static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-9905-\tstruct bpf_verifier_state *state = env-\u003ecur_state;\nkernel/bpf/verifier.c:9906:\tstruct bpf_subprog_info *caller_info;\nkernel/bpf/verifier.c-9907-\tu16 callee_incoming, stack_arg_cnt;\n--\nkernel/bpf/verifier.c-9934-\nkernel/bpf/verifier.c:9935:\t\tif (env-\u003esubprog_info[subprog].might_sleep \u0026\u0026 !in_sleepable_context(env)) {\nkernel/bpf/verifier.c-9936-\t\t\tverbose(env, \"sleepable global function %s() called in %s\\n\",\n--\nkernel/bpf/verifier.c-9953-\t\treturns_void = subprog_returns_void(env, subprog);\nkernel/bpf/verifier.c:9954:\t\tif (env-\u003esubprog_info[subprog].changes_pkt_data)\nkernel/bpf/verifier.c-9955-\t\t\tclear_all_pkt_pointers(env);\n--\nkernel/bpf/verifier.c-9970-\nkernel/bpf/verifier.c:9971:\t\tif (env-\u003esubprog_info[subprog].might_throw) {\nkernel/bpf/verifier.c-9972-\t\t\tstruct bpf_verifier_state *branch;\n--\nkernel/bpf/verifier.c-9989-\t */\nkernel/bpf/verifier.c:9990:\tcaller_info = \u0026env-\u003esubprog_info[caller-\u003esubprogno];\nkernel/bpf/verifier.c:9991:\tcallee_incoming = bpf_in_stack_arg_cnt(\u0026env-\u003esubprog_info[subprog]);\n\n"
}

							input: 34765
							output: 37
							thoughts: 23

Here's a summary of my thought process, as requested:

**Tracing the Allocation of `struct bpf_verifier_env`**

Alright, let's dive in. My immediate thought is to trace the allocation of the `struct bpf_verifier_env` within the `bpf_check` function. I need to understand where this structure comes into being because its lifespan and how it's initialized directly impact the verification process. I'm going to follow the code's execution path within `bpf_check` to pinpoint the moment this structure is created, and ideally, how its initial values are determined. Identifying the allocator is key - is it on the stack, dynamically allocated with `kmalloc`, or something else entirely? The allocation method will hint at potential memory management concerns or performance implications within the BPF verifier.

{
  "Expression": "kzalloc\\(sizeof\\(struct bpf_verifier_env",
  "PathPrefix": "kernel/bpf/"
}

{
  "error": "no matches"
}

							input: 35014
							output: 30
							thoughts: 17

Here's my thought process, as if I were considering this myself:

**Tracing the Allocation of `struct bpf_verifier_env`**

Okay, so I need to figure out where the `struct bpf_verifier_env` gets allocated. This is crucial for understanding how the BPF verifier environment is managed and potentially optimized. My first instinct is to dive into the codebase and start with a search. I'll need to look for any allocation calls related to this structure. I'll be looking for `kmalloc`, `kzalloc`, `vmalloc`, or even custom memory management functions if the kernel module has its own. Then it's to check out the initializations of the bpf verifier env struct when allocated and see how it fits in.

I'll also keep in mind that it might not be allocated directly in the common way. It could be embedded within a larger structure, allocated on the stack in specific contexts, or even managed by a memory pool. I'll have to be prepared to trace through several levels of function calls if the allocation is indirect.

I'll also want to look at where this is deallocated. It's often related and understanding how memory is freed is just as important. Knowing when the `struct bpf_verifier_env` gets freed will also help with determining the scope and lifetime of the verifier environment, which can influence its design and impact.

{
  "Expression": "bpf_check\\(",
  "PathPrefix": "kernel/bpf/"
}

{
  "Output": "kernel/bpf/syscall.c=2917=static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_attr *attr_log)\n--\nkernel/bpf/syscall.c-3132-\t/* run eBPF verifier */\nkernel/bpf/syscall.c:3133:\terr = bpf_check(\u0026prog, attr, uattr, attr_log);\nkernel/bpf/syscall.c-3134-\tif (err \u003c 0)\n--\nkernel/bpf/verifier.c=58=static bool bpf_global_percpu_ma_set;\nkernel/bpf/verifier.c-59-\nkernel/bpf/verifier.c:60:/* bpf_check() is a static code analyzer that walks eBPF program\nkernel/bpf/verifier.c-61- * instruction by instruction and updates register/stack state.\n--\nkernel/bpf/verifier.c=21057=static enum bpf_sig_keyring bpf_classify_keyring(s32 keyring_id)\n--\nkernel/bpf/verifier.c-21071-/*\nkernel/bpf/verifier.c:21072: * Verify the PKCS#7 signature of a loaded program. Called from bpf_check()\nkernel/bpf/verifier.c-21073- * once the program's metadata maps have been resolved into used_maps, so\n--\nkernel/bpf/verifier.c=21080=static int bpf_prog_verify_signature(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-21196-\nkernel/bpf/verifier.c:21197:int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\nkernel/bpf/verifier.c-21198-\t      struct bpf_log_attr *attr_log)\n--\nkernel/bpf/verifier.c-21211-\t/* 'struct bpf_verifier_env' can be global, but since it's not small,\nkernel/bpf/verifier.c:21212:\t * allocate/free it every time bpf_check() is called\nkernel/bpf/verifier.c-21213-\t */\n"
}

							input: 35953
							output: 38
							thoughts: 23

Okay, let's dive into this. Here's my take on those lines around 21212 of `kernel/bpf/verifier.c`, from an expert's perspective:

**Diving Deep into the BPF Verifier**

Alright, so I need to refresh my memory on what's going on around line 21212 of `kernel/bpf/verifier.c`.  I know that area pretty well... it's probably smack dab in the middle of some crucial verification logic. Let me think, based on context... that's likely where the BPF verifier is doing something related to instruction analysis, state tracking, or perhaps even resource limits, type checking, or loop analysis – perhaps all of the above! It could be dealing with handling register states, memory access, jumps, or a particular instruction pattern that needs special handling. I'll need to see the surrounding code to be sure, but my gut feeling is that we are likely deep in the process of ensuring that BPF programs are safe and won't crash the kernel or create any security vulnerabilities.  Given that it's the verifier, I'm expecting to see a lot of conditional checks, possibly recursion, and definitely a heavy reliance on a robust state machine.  Let's see the code!  I am eager to see how it tackles the edge cases.

{
  "File": "kernel/bpf/verifier.c",
  "FirstLine": 21200,
  "LineCount": 40
}

{
  "Contents": "21200:\t\tu64 start_time = ktime_get_ns();\n21201:\t\tstruct bpf_verifier_env *env;\n21202:\t\tint i, len, ret = -EINVAL, err;\n21203:\t\tbool is_priv;\n21204:\t\n21205:\t\tBTF_TYPE_EMIT(enum bpf_features);\n21206:\t\n21207:\t\t/* no program is valid */\n21208:\t\tif (ARRAY_SIZE(bpf_verifier_ops) == 0)\n21209:\t\t\treturn -EINVAL;\n21210:\t\n21211:\t\t/* 'struct bpf_verifier_env' can be global, but since it's not small,\n21212:\t\t * allocate/free it every time bpf_check() is called\n21213:\t\t */\n21214:\t\tenv = kvzalloc_obj(struct bpf_verifier_env, GFP_KERNEL_ACCOUNT);\n21215:\t\tif (!env)\n21216:\t\t\treturn -ENOMEM;\n21217:\t\n21218:\t\tenv-\u003ebt.env = env;\n21219:\t\tenv-\u003eprog = *prog;\n21220:\t\tenv-\u003eops = bpf_verifier_ops[env-\u003eprog-\u003etype];\n21221:\t\n21222:\t\tenv-\u003eallow_ptr_leaks = bpf_allow_ptr_leaks(env-\u003eprog-\u003eaux-\u003etoken);\n21223:\t\tenv-\u003eallow_uninit_stack = bpf_allow_uninit_stack(env-\u003eprog-\u003eaux-\u003etoken);\n21224:\t\tenv-\u003ebypass_spec_v1 = bpf_bypass_spec_v1(env-\u003eprog-\u003eaux-\u003etoken);\n21225:\t\tenv-\u003ebypass_spec_v4 = bpf_bypass_spec_v4(env-\u003eprog-\u003eaux-\u003etoken);\n21226:\t\tenv-\u003ebpf_capable = is_priv = bpf_token_capable(env-\u003eprog-\u003eaux-\u003etoken, CAP_BPF);\n21227:\t\tenv-\u003esignature = attr-\u003esignature;\n21228:\t\n21229:\t\t/* user could have requested verbose verifier output\n21230:\t\t * and supplied buffer to store the verification trace\n21231:\t\t */\n21232:\t\tret = bpf_vlog_init(\u0026env-\u003elog, attr_log-\u003elevel, attr_log-\u003eubuf, attr_log-\u003esize);\n21233:\t\tif (ret)\n21234:\t\t\tgoto err_free_env;\n21235:\t\tret = bpf_diag_init(env);\n21236:\t\tif (ret)\n21237:\t\t\tgoto err_prep;\n21238:\t\tif (env-\u003esignature) {\n21239:\t\t\tret = bpf_prog_calc_tag(env-\u003eprog);\n"
}

							input: 37073
							output: 33
							thoughts: 760

**Analyzing the BPF CFG/Jump Table Optimization**

Okay, so I'm looking at this patch and my initial thought is, `env` is allocated using `kvzalloc_obj`. That means everything in `env` is initialized to zero, which is good to know, especially for things like pointers and booleans. This gives me a baseline.

Now, let's dive into what this patch is doing. It seems like it's addressing some optimizations and bug fixes related to the Control Flow Graph (CFG) and jump table processing, specifically around indirect jumps or `gotox` instructions within the BPF verifier.

First, I see additions to the data structures. In `bpf_subprog_info`, `jt` (the jump table pointer) and the `jt_spans_subprogs` bitfield are added. This signals that jump tables are directly related to subprograms, and whether this jump table spans multiple subprograms. In `bpf_verifier_env::cfg`, `gotox_edges` (a `u32` for edge counting) and `subprog_jts_ready` (a boolean flag) are added. These suggest a way to manage GOTX edges and the readiness of jump tables.

Looking at the code changes, `compute_subprog_jts(env)` is interesting. It iterates through `env->insn_array_maps` and figures out which subprogram a jump table item belongs to. It checks that the last entry in the jump table doesn't exceed the start of the *next* subprogram. It then reallocates the `subprog->jt` using `bpf_iarray_realloc` (that's going to be key to understand fully), copies the items, sorts and unifies them with `sort_insn_array_uniq`, and sets `env->cfg.subprog_jts_ready = true`. This seems to be the core logic to construct jump tables for each subprogram.

`free_subprog_jts(env)` cleans up the jump tables. It frees the allocated memory and resets the corresponding flags and pointers. This is crucial for error handling, which makes sense because it's called in `bpf_check_cfg`'s error path (`err_free:`).

Then there's `create_jt(int t, env)`. If jump tables aren't ready yet, it calls `compute_subprog_jts`. It checks if either `subprog->jt_spans_subprogs` is true or if `!subprog->jt` (the jump table doesn't exist yet). Then it allocates a *new* `bpf_iarray` with `bpf_iarray_realloc` (again, this is very important to fully grasp) and copies the items over.

`visit_gotox_insn` is dealing with `gotox_edges`, calculating the number of edges and checking for overflows. Makes sense, we need to make sure we don't end up with an absurd number of jump targets.

`bpf_compute_scc` does something with strongly connected components. It allocates and then frees `dfs_pos` (using `kvcalloc` and `kvfree`, which means it zero initializes the memory). That memory is likely used to track the successor index.

Finally, in `verifier.c`, there are new functions: `jt_contains` (for binary search within the sorted jump table), and `reject_gotox_out_of_subprog`, `reject_gotox_without_cfg_edge`. There's also `check_indirect_jump`, which looks at bounds and makes sure that a jump target is contained within the jump table.

I'm definitely going to need to track down what exactly `bpf_iarray_realloc` does. It seems critical for this whole thing. I'll check the definition and see how it works. I'm also interested in how this interacts with the CFG analysis overall. So far, the patch seems to be streamlining the management of jump tables.

{
  "Expression": "bpf_iarray_realloc",
  "PathPrefix": "kernel/bpf/"
}

{
  "Output": "kernel/bpf/cfg.c=163=static int visit_func_call_insn(int t, struct bpf_insn *insns,\n--\nkernel/bpf/cfg.c-187-\nkernel/bpf/cfg.c:188:struct bpf_iarray *bpf_iarray_realloc(struct bpf_iarray *old, size_t n_elem)\nkernel/bpf/cfg.c-189-{\n--\nkernel/bpf/cfg.c=261=static struct bpf_iarray *jt_from_map(struct bpf_map *map)\n--\nkernel/bpf/cfg.c-266-\nkernel/bpf/cfg.c:267:\tjt = bpf_iarray_realloc(NULL, map-\u003emax_entries);\nkernel/bpf/cfg.c-268-\tif (!jt)\n--\nkernel/bpf/cfg.c=294=static int compute_subprog_jts(struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-324-\t\told_cnt = subprog-\u003ejt ? subprog-\u003ejt-\u003ecnt : 0;\nkernel/bpf/cfg.c:325:\t\tjt = bpf_iarray_realloc(subprog-\u003ejt, old_cnt + jt_cur-\u003ecnt);\nkernel/bpf/cfg.c-326-\t\tif (!jt) {\n--\nkernel/bpf/cfg.c=360=create_jt(int t, struct bpf_verifier_env *env)\n--\nkernel/bpf/cfg.c-395-\nkernel/bpf/cfg.c:396:\tjt = bpf_iarray_realloc(NULL, subprog-\u003ejt-\u003ecnt);\nkernel/bpf/cfg.c-397-\tif (!jt)\n--\nkernel/bpf/cfg.c=471=static int visit_abnormal_return_insn(struct bpf_verifier_env *env, int t)\n--\nkernel/bpf/cfg.c-478-\nkernel/bpf/cfg.c:479:\tjt = bpf_iarray_realloc(NULL, 2);\nkernel/bpf/cfg.c-480-\tif (!jt)\n--\nkernel/bpf/verifier.c=18211=static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *insn)\n--\nkernel/bpf/verifier.c-18244-\tif (!env-\u003egotox_tmp_buf || env-\u003egotox_tmp_buf-\u003ecnt \u003c max_index - min_index + 1) {\nkernel/bpf/verifier.c:18245:\t\tenv-\u003egotox_tmp_buf = bpf_iarray_realloc(env-\u003egotox_tmp_buf,\nkernel/bpf/verifier.c-18246-\t\t\t\t\t\t        max_index - min_index + 1);\n--\nkernel/bpf/verifier.c=21197=int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-21273-\t\tenv-\u003einsn_aux_data[i].orig_idx = i;\nkernel/bpf/verifier.c:21274:\tenv-\u003esucc = bpf_iarray_realloc(NULL, 2);\nkernel/bpf/verifier.c-21275-\tif (!env-\u003esucc)\n"
}

							input: 39631
							output: 36
							thoughts: 25

Here's my summary, as though I'm thinking through this process:

**Diving into `bpf_iarray_realloc`**

Okay, time to dig into the guts of the BPF verifier again. I want to see how the kernel handles reallocating the instruction array, specifically focusing on `bpf_iarray_realloc` within `kernel/bpf/cfg.c`. I know this function is crucial for program modification and optimization, especially when instructions are inserted or deleted during the verification process. I need to understand the mechanics of how it’s done. What are the reallocation strategies in play? What's the overhead? Are there any potential vulnerabilities to keep in mind? I need to analyze its implications for overall BPF program security and efficiency. Time to see how this function handles things under the hood.

{
  "File": "kernel/bpf/cfg.c",
  "FirstLine": 185,
  "LineCount": 40
}

{
  "Contents": " 185:\t\treturn ret;\n 186:\t}\n 187:\t\n 188:\tstruct bpf_iarray *bpf_iarray_realloc(struct bpf_iarray *old, size_t n_elem)\n 189:\t{\n 190:\t\tsize_t new_size = sizeof(struct bpf_iarray) + n_elem * sizeof(old-\u003eitems[0]);\n 191:\t\tstruct bpf_iarray *new;\n 192:\t\n 193:\t\tnew = kvrealloc(old, new_size, GFP_KERNEL_ACCOUNT);\n 194:\t\tif (!new) {\n 195:\t\t\t/* this is what callers always want, so simplify the call site */\n 196:\t\t\tkvfree(old);\n 197:\t\t\treturn NULL;\n 198:\t\t}\n 199:\t\n 200:\t\tnew-\u003ecnt = n_elem;\n 201:\t\treturn new;\n 202:\t}\n 203:\t\n 204:\tstatic int copy_insn_array(struct bpf_map *map, u32 start, u32 end, u32 *items)\n 205:\t{\n 206:\t\tstruct bpf_insn_array_value *value;\n 207:\t\tu32 i;\n 208:\t\n 209:\t\tfor (i = start; i \u003c= end; i++) {\n 210:\t\t\tvalue = map-\u003eops-\u003emap_lookup_elem(map, \u0026i);\n 211:\t\t\t/*\n 212:\t\t\t * map_lookup_elem of an array map will never return an error,\n 213:\t\t\t * but not checking it makes some static analysers to worry\n 214:\t\t\t */\n 215:\t\t\tif (IS_ERR(value))\n 216:\t\t\t\treturn PTR_ERR(value);\n 217:\t\t\telse if (!value)\n 218:\t\t\t\treturn -EINVAL;\n 219:\t\t\titems[i - start] = value-\u003exlated_off;\n 220:\t\t}\n 221:\t\treturn 0;\n 222:\t}\n 223:\t\n 224:\tstatic int cmp_ptr_to_u32(const void *a, const void *b)\n"
}

							input: 40474
							output: 302
							thoughts: 1267

**KMSAN Reasoning: BPF Verifier Indirect Jump Table Validation**

Okay, let's step through this. I'm focusing on the potential for uninitialized memory reads or leaks here, which is what KMSAN is concerned with. Looking at `compute_subprog_jts`, I see it's all about analyzing and merging indirect jump tables. The core of this function is iterating through instruction array maps, figuring out which subprogram each jump table (`jt`) belongs to, and then merging those tables.

The critical part is the reallocation and merging logic: `bpf_iarray_realloc` and `memcpy`. The key is that `bpf_iarray_realloc` *always* handles allocation correctly, and if it fails, handles that failure. I need to be sure that the `memcpy` is *correct*. We allocate the correct amount of space: `old_cnt + jt_cur->cnt`. Then, `memcpy` copies the `jt_cur->cnt` items into the newly allocated and initialized space. The previous data, `old_cnt` is correctly preserved by `kvrealloc`. This is confirmed when we look at `create_jt` where we perform a similar operation.

Next, I'm checking `bpf_compute_scc`. The `dfs_pos` array is initialized with `kvcalloc`, which is good. That guarantees a zero-initialized state. All accesses are within bounds; `dfs_sz` is always within the valid range.

Now, a broader check. Am I sending anything from the kernel to userspace that could be uninitialized? No. This code is purely inside the BPF verifier. The verifier itself is static analysis. The verifier logs might use formatted strings, but the parameters are things like offsets and instruction indices, which are all derived from the processed instructions themselves.

Let's summarize the checks:

1. **Kernel structures to userspace?** None. No data is sent to userspace. This code is purely internal verifier logic.
2. **Uninitialized variable use?** No. Everything is explicitly initialized: `dfs_pos` with `kvcalloc`, etc. The code either does zero initialization or copies explicitly with `memcpy`.
3. **Complex allocations?** All structures either use zero-allocation functions or `memcpy`, which ensures proper initialization and prevents uninitialized reads.
4. **Bounds/Length calculations?** Yes. We prevent out-of-bounds indirect jumps and enforce edge limits, as well as checking the bounds of our data, such as for the jump tables.

Therefore, because this patch *only* modifies the BPF verifier's CFG analysis and jump table validation, and uses fully zeroed/copied allocations (`kvcalloc`, `memcpy`, `kvzalloc_obj`), and doesn't introduce or modify any uninitialized memory reads or information leaks to userspace, `NeedsKMSAN` is `false`. Standard KASAN fuzzing, with all sanitizers and lockdep enabled, is sufficient.

{
  "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.",
  "NeedsKMSAN": false
}

{
  "KMSANReasoning": "The patch modifies the eBPF verifier's control flow graph (CFG) analysis and jump-table validation for indirect jumps (`gotox`). It caches per-subprogram jump tables, bounds-checks the number of indirect jump edges, prevents gotox jumps from leaving their containing subprogram, and optimizes Tarjan's strongly connected components (SCC) algorithm.\n\nAll newly allocated buffers and struct members are fully initialized:\n- `struct bpf_verifier_env` (and its nested `subprog_info` and `cfg` structs) is zero-initialized via `kvzalloc_obj` at verifier entry.\n- `dfs_pos` in `bpf_compute_scc()` is allocated with `kvcalloc()`, zero-initializing all elements.\n- Jump table buffers allocated with `bpf_iarray_realloc()` are immediately filled with `memcpy()` based on the exact element count.\n- There are no uninitialized struct fields, uninitialized stack/heap variables, or kernel-to-user info leaks introduced by the patch.\n\nAny potential issues arising from this patch would be logic errors, out-of-bounds array access, or memory leaks, which are handled by standard KASAN/sanitizer configurations. Therefore, a dedicated KMSAN fuzzing session is not required.",
  "NeedsKMSAN": false
}

LLM Calls Summary:
Total Calls	Total Tokens	Avg Tokens	Total Duration (Seconds)	Avg Duration (Seconds)

Tool Calls Summary:
Total Calls	Total Duration (Seconds)	Avg Duration (Seconds)