From: Alexei Starovoitov ld_imm64 BPF_PSEUDO_FUNC may take the address of a function that nothing calls: the helper call that takes it as a callback is dead code, or the prog only stores or compares the pointer. The function is not seen by the main pass and bpf_opt_remove_dead_code() removes it, while ld_imm64 stays and points to whatever follows. If the function was the last one jit_subprogs() hits: verifier bug: No program to jit at insn 8 (subprog < 0) Otherwise the register gets the address of the next function. Keep the first insn of such function and make it a trap like sanitize_dead_code() does, so the address stays valid and unique. Signed-off-by: Alexei Starovoitov --- kernel/bpf/fixups.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c index 2add8001c3ec..75936842d177 100644 --- a/kernel/bpf/fixups.c +++ b/kernel/bpf/fixups.c @@ -619,12 +619,41 @@ void bpf_opt_hard_wire_dead_code_branches(struct bpf_verifier_env *env) } } +/* + * The address of a function might be taken by the code that is not dead, + * while the function itself is dead code, since nothing calls it. Keep the first + * instruction of such function, so that the address remains valid and differs + * from the addresses of other functions. Make it a trap like + * sanitize_dead_code() does. + */ +static void keep_funcs_with_addr_taken(struct bpf_verifier_env *env) +{ + struct bpf_insn_aux_data *aux_data = env->insn_aux_data; + struct bpf_insn trap = BPF_JMP_IMM(BPF_JA, 0, 0, -1); + struct bpf_insn *insn = env->prog->insnsi; + int insn_cnt = env->prog->len; + int i, t; + + for (i = 0; i < insn_cnt; i++) { + if (!aux_data[i].seen || !bpf_pseudo_func(insn + i)) + continue; + t = i + insn[i].imm + 1; + if (aux_data[t].seen) + continue; + memcpy(insn + t, &trap, sizeof(trap)); + aux_data[t].zext_dst = false; + aux_data[t].seen = env->pass_cnt; + } +} + int bpf_opt_remove_dead_code(struct bpf_verifier_env *env) { struct bpf_insn_aux_data *aux_data = env->insn_aux_data; int insn_cnt = env->prog->len; int i, err; + keep_funcs_with_addr_taken(env); + for (i = 0; i < insn_cnt; i++) { int j; -- 2.55.0