| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/11/12 15:30 | flow | patch-triage |
6mArgs:null Results: {
"EnableConfigs": [
"BPF_SYSCALL",
"DEBUG_INFO_BTF",
"DEBUG_INFO_BTF_MODULES",
"BPF_JIT",
"LLVM_PLUGINS",
"CLANG_PLUGIN_BPF_SCAN_IR",
"FUNCTION_ERROR_INJECTION"
],
"FocusSymbols": [
"bpf_check_attach_target",
"check_attach_btf_id",
"fmodret_retval_init",
"check_fmodret_retval",
"btf_register_fmodret_tags",
"btf_kfunc_is_modify_return",
"btf_kfunc_check_flag"
],
"KMSANReasoning": "The patch series introduces build-time and verification mechanisms for BPF fmod_ret targets. Specifically, it adds an LLVM pass plugin to scan IR for clean error-returning functions, records metadata in ELF sections, annotates BTF types with declaration tags in resolve_btfids, and updates the kernel BPF verifier and BTF subsystem to restrict fmod_ret return values based on these tags.\n\nFrom a memory sanitization perspective:\n1. No uninitialized memory usage:\n - In `kernel/bpf/btf.c`, `btf_register_fmodret_tags()` allocates `struct btf_id_set8` using `kmalloc()` and explicitly initializes `cnt`, `flags`, and all populated `pairs[]` elements (`id` and `flags`).\n - In `kernel/bpf/verifier.c`, `fmodret_retval_init()` parses tag values and allocates `retvals` via `kmalloc_array()`, writing every element sequentially up to `cnt`.\n - `prog-\u003eaux` is allocated using `kzalloc_obj()`, ensuring that `fmodret_retvals` and `fmodret_retval_cnt` are zero-initialized by default.\n - Stack buffers (e.g. `char buf[24]` in `fmodret_retval_next()`) are fully initialized (`memcpy` followed by explicit null termination) before being passed to `kstrtos64()`.\n2. No kernel-to-user info-leaks:\n - The newly introduced structures (`struct btf_id_set8`, `aux-\u003efmodret_retvals`) are kernel-internal tracking structures and are never copied to userspace via `copy_to_user`, `put_user`, socket options, or BPF syscall inspection interfaces.\n3. KASAN vs KMSAN applicability:\n - Potential risks in this series pertain to bounds checking, string parsing/tokenization, or dynamic array management. Any out-of-bounds access or use-after-free bugs in these paths are already covered by KASAN.\n - There are no uninitialized memory reads, uninitialized padding leaks, or uninitialized branch conditions that would uniquely require KMSAN.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false,
"Reasoning": "The patch introduces support for discovering and tagging BPF modify-return (fmod_ret) target functions and enforcing allowed return values via the BPF verifier. It adds new verifier logic (check_fmodret_retval, fmodret_retval_init, and restrictions on tail calls and freplace extensions) as well as BTF registration for fmod_ret tags (btf_register_fmodret_tags). These changes modify core BPF subsystem code reachable through standard bpf() system call operations (program loading and link attachment) in standard virtualized environments.",
"WorthFuzzing": true
} |
| 1/1 | 2026/11/12 15:30 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit b2dd76da41c0d4300e9a8c4bb643fc7c656fa2c8\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Thu Nov 12 15:30:31 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/MAINTAINERS b/MAINTAINERS\nindex 93abbb315065de..29940e8d7205b7 100644\n--- a/MAINTAINERS\n+++ b/MAINTAINERS\n@@ -5064,6 +5064,14 @@ S:\tSupported\n F:\tarch/x86/net/\n X:\tarch/x86/net/bpf_jit_comp32.c\n \n+BPF [COMPILER PLUGINS]\n+M:\tAnton Protopopov \u003ca.s.protopopov@gmail.com\u003e\n+L:\tbpf@vger.kernel.org\n+S:\tMaintained\n+F:\tscripts/bpf/\n+F:\tscripts/llvm-plugins/bpf_*\n+F:\ttools/bpf/resolve_btfids/fmodret.*\n+\n BPF [CORE]\n M:\tAlexei Starovoitov \u003cast@kernel.org\u003e\n M:\tDaniel Borkmann \u003cdaniel@iogearbox.net\u003e\n@@ -6562,7 +6570,9 @@ C:\tirc://irc.libera.chat/clangbuiltlinux\n F:\tDocumentation/kbuild/llvm.rst\n F:\tinclude/linux/compiler-clang.h\n F:\tscripts/Makefile.clang\n+F:\tscripts/Makefile.llvm-plugins\n F:\tscripts/clang-tools/\n+F:\tscripts/llvm-plugins/\n K:\t\\b(?i:clang|llvm)\\b\n \n CLK API\ndiff --git a/Makefile b/Makefile\nindex f895752d4b05fa..2fcad810957218 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -457,10 +457,13 @@ endif\n \n HOSTCC\t= $(LLVM_PREFIX)clang$(LLVM_SUFFIX)\n HOSTCXX\t= $(LLVM_PREFIX)clang++$(LLVM_SUFFIX)\n+LLVM_CONFIG ?= $(LLVM_PREFIX)llvm-config$(LLVM_SUFFIX)\n else\n HOSTCC\t= gcc\n HOSTCXX\t= g++\n+LLVM_CONFIG ?= llvm-config\n endif\n+export LLVM_CONFIG\n HOSTRUSTC = rustc\n HOSTPKG_CONFIG\t= pkg-config\n \n@@ -1211,10 +1214,11 @@ include-$(CONFIG_AUTOFDO_CLANG)\t+= scripts/Makefile.autofdo\n include-$(CONFIG_PROPELLER_CLANG)\t+= scripts/Makefile.propeller\n include-$(CONFIG_WARN_CONTEXT_ANALYSIS) += scripts/Makefile.context-analysis\n include-$(CONFIG_GCC_PLUGINS)\t+= scripts/Makefile.gcc-plugins\n+include-$(CONFIG_LLVM_PLUGINS)\t+= scripts/Makefile.llvm-plugins\n \n include $(addprefix $(srctree)/, $(include-y))\n \n-# scripts/Makefile.gcc-plugins is intentionally included last.\n+# The compiler plugin Makefiles are intentionally included last.\n # Do not add $(call cc-option,...) below this line. When you build the kernel\n # from the clean source tree, the GCC plugins do not exist at this point.\n \ndiff --git a/arch/Kconfig b/arch/Kconfig\nindex 1163736de1f996..4f1f36d3e82d8e 100644\n--- a/arch/Kconfig\n+++ b/arch/Kconfig\n@@ -1772,6 +1772,7 @@ config HAVE_GENERIC_TIF_BITS\n source \"kernel/gcov/Kconfig\"\n \n source \"scripts/gcc-plugins/Kconfig\"\n+source \"scripts/llvm-plugins/Kconfig\"\n \n config FUNCTION_ALIGNMENT_4B\n \tbool\ndiff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile\nindex a273ab5af805c6..5813bb75c99251 100644\n--- a/arch/arm64/kernel/pi/Makefile\n+++ b/arch/arm64/kernel/pi/Makefile\n@@ -10,6 +10,7 @@ KBUILD_CFLAGS\t:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \\\n \t\t -D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \\\n \t\t -fno-asynchronous-unwind-tables -fno-unwind-tables \\\n \t\t $(CONFIG_CC_OPT_NO_ADDRSIG)\n+KBUILD_CFLAGS\t:= $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))\n \n # this code may run with the MMU off so disable unaligned accesses\n CFLAGS_map_range.o += -mstrict-align\ndiff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile\nindex 5e499cfb29b5c0..7036e939097e5d 100644\n--- a/arch/x86/boot/startup/Makefile\n+++ b/arch/x86/boot/startup/Makefile\n@@ -12,6 +12,8 @@ KBUILD_CFLAGS\t\t+= -D__DISABLE_EXPORTS -mcmodel=small -fPIC \\\n # disable ftrace hooks and LTO\n KBUILD_CFLAGS\t:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS))\n KBUILD_CFLAGS\t:= $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS))\n+# LLVM plugins may add data with absolute addresses\n+KBUILD_CFLAGS\t:= $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))\n KASAN_SANITIZE\t:= n\n KCSAN_SANITIZE\t:= n\n KMSAN_SANITIZE\t:= n\ndiff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile\nindex 7664cbedbe30fb..0e32c9ae2b5afc 100644\n--- a/arch/x86/um/vdso/Makefile\n+++ b/arch/x86/um/vdso/Makefile\n@@ -37,7 +37,7 @@ CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \\\n $(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \\\n -fno-omit-frame-pointer -foptimize-sibling-calls\n \n-$(vobjs): KBUILD_CFLAGS += $(CFL)\n+$(vobjs): KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)\n \n #\n # vDSO code runs in userspace and -pg doesn't help with profiling anyway.\ndiff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile\nindex bbab12b8eb4b23..6f69d5d6773141 100644\n--- a/drivers/firmware/efi/libstub/Makefile\n+++ b/drivers/firmware/efi/libstub/Makefile\n@@ -6,7 +6,7 @@\n # enabled, even if doing so doesn't break the build.\n #\n \n-cflags-y\t\t\t:= $(KBUILD_CFLAGS)\n+cflags-y\t\t\t:= $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))\n \n # x86 links the stub into the decompressor rather than the kernel proper, so\n # drop the kernel's code model, calling convention, and stack alignment, and\ndiff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h\nindex 3eb732942d699c..77e86ff2f9a44d 100644\n--- a/include/asm-generic/vmlinux.lds.h\n+++ b/include/asm-generic/vmlinux.lds.h\n@@ -198,7 +198,9 @@\n #define MCOUNT_REC()\t. = ALIGN(8);\t\t\t\t\\\n \t\t\t__start_mcount_loc = .;\t\t\t\\\n \t\t\tKEEP(*(__mcount_loc))\t\t\t\\\n+\t\t\t__start_patchable_function_entries = .;\t\\\n \t\t\tKEEP_PATCHABLE\t\t\t\t\\\n+\t\t\t__stop_patchable_function_entries = .;\t\\\n \t\t\t__stop_mcount_loc = .;\t\t\t\\\n \t\t\tFTRACE_STUB_HACK\t\t\t\\\n \t\t\tftrace_ops_list_func = arch_ftrace_ops_list_func;\n@@ -714,7 +716,7 @@\n \tBTF_INLINE_LINK\t\t\t\t\t\t\\\n \t. = ALIGN(PAGE_SIZE);\t\t\t\t\t\t\\\n \t.BTF_ids : AT(ADDR(.BTF_ids) - LOAD_OFFSET) {\t\t\t\\\n-\t\t*(.BTF_ids)\t\t\t\t\t\t\\\n+\t\tKEEP(*(.BTF_ids))\t\t\t\t\t\\\n \t}\n #else\n #define BTF\n@@ -886,7 +888,8 @@\n \t\tKLP_SYMID\n \n #define MODINFO\t\t\t\t\t\t\t\t\\\n-\t\t.modinfo (INFO) : { KEEP(*(.modinfo)) }\n+\t\t.modinfo (INFO) : { KEEP(*(.modinfo)) }\t\\\n+\t\t.BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }\n \n #ifdef CONFIG_GENERIC_BUG\n #define BUG_TABLE\t\t\t\t\t\t\t\\\ndiff --git a/include/linux/bpf.h b/include/linux/bpf.h\nindex 520fd72e21fd07..56a5e4b9f138a6 100644\n--- a/include/linux/bpf.h\n+++ b/include/linux/bpf.h\n@@ -1969,6 +1969,12 @@ struct bpf_prog_aux {\n \tconst struct btf_type *attach_func_proto;\n \t/* function name for valid attach_btf_id */\n \tconst char *attach_func_name;\n+\t/*\n+\t * The values which an fmod_ret program may return besides 0, from the\n+\t * \"bpf_fmodret_retval:\" tag of its target, or NULL\n+\t */\n+\ts64 *fmodret_retvals;\n+\tu32 fmodret_retval_cnt;\n \tstruct bpf_prog **func;\n \tstruct bpf_prog_aux *main_prog_aux;\n \t/* main prog: there when the prog has timed may_goto */\ndiff --git a/include/linux/btf.h b/include/linux/btf.h\nindex 87fad4076f0e9d..5522c7fe13735a 100644\n--- a/include/linux/btf.h\n+++ b/include/linux/btf.h\n@@ -558,11 +558,17 @@ static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id)\n \treturn bsearch(\u0026id, set-\u003eids, set-\u003ecnt, sizeof(u32), btf_id_cmp_func) != NULL;\n }\n \n-static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)\n+static inline struct btf_id_set8_pair *\n+btf_id_set8_entry(const struct btf_id_set8 *set, u32 id)\n {\n \treturn bsearch(\u0026id, set-\u003epairs, set-\u003ecnt, sizeof(set-\u003epairs[0]), btf_id_cmp_func);\n }\n \n+static inline bool btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)\n+{\n+\treturn btf_id_set8_entry(set, id) != NULL;\n+}\n+\n bool btf_param_match_suffix(const struct btf *btf,\n \t\t\t const struct btf_param *arg,\n \t\t\t const char *suffix);\n@@ -626,7 +632,6 @@ u32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,\n \t\t\t\tconst struct bpf_prog *prog);\n int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,\n \t\t\t const struct btf_kfunc_id_set *s);\n-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset);\n s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id);\n int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt,\n \t\t\t\tstruct module *owner);\ndiff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h\nindex 8b5a9ee92513d4..f5985acf661b1c 100644\n--- a/include/linux/btf_ids.h\n+++ b/include/linux/btf_ids.h\n@@ -13,13 +13,21 @@ struct btf_id_set {\n /* This flag implies BTF_SET8 holds kfunc(s) */\n #define BTF_SET8_KFUNCS\t\t(1 \u003c\u003c 0)\n \n+/*\n+ * This flag implies BTF_SET8 holds functions which BPF fmod_ret, fentry and\n+ * fexit programs may attach to. Per-function flags: KF_SLEEPABLE only.\n+ */\n+#define BTF_SET8_FMODRET\t(1 \u003c\u003c 1)\n+\n+struct btf_id_set8_pair {\n+\tu32 id;\n+\tu32 flags;\n+};\n+\n struct btf_id_set8 {\n \tu32 cnt;\n \tu32 flags;\n-\tstruct {\n-\t\tu32 id;\n-\t\tu32 flags;\n-\t} pairs[];\n+\tstruct btf_id_set8_pair pairs[];\n };\n \n #ifdef CONFIG_DEBUG_INFO_BTF\n@@ -215,6 +223,12 @@ __BTF_SET8_START(name, local, BTF_SET8_KFUNCS)\n #define BTF_KFUNCS_END(name)\t\t\t\t\\\n BTF_SET8_END(name)\n \n+#define BTF_FMODRET_START(name)\t\t\t\t\\\n+__BTF_SET8_START(name, local, BTF_SET8_FMODRET)\n+\n+#define BTF_FMODRET_END(name)\t\t\t\t\\\n+BTF_SET8_END(name)\n+\n #else\n \n #define BTF_ID_LIST(name) static u32 __maybe_unused name[128];\n@@ -231,6 +245,8 @@ BTF_SET8_END(name)\n #define BTF_SET8_END(name)\n #define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };\n #define BTF_KFUNCS_END(name)\n+#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };\n+#define BTF_FMODRET_END(name)\n \n #endif /* CONFIG_DEBUG_INFO_BTF */\n \ndiff --git a/include/linux/compiler-version.h b/include/linux/compiler-version.h\nindex ac1665a98a15db..e6a5a9f3fbcd33 100644\n--- a/include/linux/compiler-version.h\n+++ b/include/linux/compiler-version.h\n@@ -24,6 +24,11 @@\n #include \u003cgenerated/gcc-plugins.h\u003e\n #endif\n \n+/* Rebuild objects when an LLVM plugin changes. */\n+#ifdef LLVM_PLUGINS\n+#include \u003cgenerated/llvm-plugins.h\u003e\n+#endif\n+\n /*\n * If the randstruct seed itself changes (whether for GCC plugins or\n * Clang), the entire tree needs to be rebuilt since the randomization of\ndiff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c\nindex 228b93d4f8c940..f81a8ae9b92c32 100644\n--- a/kernel/bpf/btf.c\n+++ b/kernel/bpf/btf.c\n@@ -9307,6 +9307,8 @@ enum {\n };\n \n #ifdef CONFIG_DEBUG_INFO_BTF_MODULES\n+static int btf_register_fmodret_tags(struct btf *btf);\n+\n struct btf_module {\n \tstruct list_head list;\n \tstruct module *module;\n@@ -9476,6 +9478,15 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,\n \t\t\t}\n \t\t\tgoto out;\n \t\t}\n+\t\t/*\n+\t\t * Do this before the BTF is visible to anyone else. A failure\n+\t\t * is not fatal: the module loads, but without fmod_ret targets.\n+\t\t */\n+\t\terr = btf_register_fmodret_tags(btf);\n+\t\tif (err)\n+\t\t\tpr_warn(\"failed to register fmod_ret targets of module [%s]: %d\\n\",\n+\t\t\t\tmod-\u003ename, err);\n+\n \t\terr = btf_alloc_id(btf);\n \t\tif (err) {\n \t\t\tbtf_free(btf);\n@@ -9973,8 +9984,8 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,\n \t\t\t\t enum btf_kfunc_hook hook,\n \t\t\t\t u32 kfunc_btf_id)\n {\n+\tstruct btf_id_set8_pair *pair;\n \tstruct btf_id_set8 *set;\n-\tu32 *id;\n \n \tif (hook \u003e= BTF_KFUNC_HOOK_MAX)\n \t\treturn NULL;\n@@ -9983,11 +9994,10 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,\n \tset = btf-\u003ekfunc_set_tab-\u003esets[hook];\n \tif (!set)\n \t\treturn NULL;\n-\tid = btf_id_set8_contains(set, kfunc_btf_id);\n-\tif (!id)\n+\tpair = btf_id_set8_entry(set, kfunc_btf_id);\n+\tif (!pair)\n \t\treturn NULL;\n-\t/* The flags for BTF ID are located next to it */\n-\treturn id + 1;\n+\treturn \u0026pair-\u003eflags;\n }\n \n static bool __btf_kfunc_is_allowed(const struct btf *btf,\n@@ -10101,7 +10111,9 @@ u32 *btf_kfunc_flags(const struct btf *btf, u32 kfunc_btf_id, const struct bpf_p\n }\n \n /*\n- * Check a single KF_* @flag on a kfunc across all of its hook sets.\n+ * Check a single KF_* @flag on a kfunc across all of its hook sets. The set of\n+ * fmod_ret targets is not one of them: its flags only say how a target may be\n+ * attached, and a kfunc which is also a target keeps its own flags.\n * Returns:\n * * 1 if @flag is set\n * * 0 if @flag is not set\n@@ -10116,6 +10128,8 @@ int btf_kfunc_check_flag(const struct btf *btf, u32 kfunc_btf_id, u32 flag)\n \tu32 *flags;\n \n \tfor (hook = 0; hook \u003c BTF_KFUNC_HOOK_MAX; hook++) {\n+\t\tif (hook == BTF_KFUNC_HOOK_FMODRET)\n+\t\t\tcontinue;\n \t\tflags = btf_kfunc_id_set_contains(btf, hook, kfunc_btf_id);\n \t\tif (!flags)\n \t\t\tcontinue;\n@@ -10183,12 +10197,102 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,\n }\n EXPORT_SYMBOL_GPL(register_btf_kfunc_id_set);\n \n-/* This function must be invoked only from initcalls/module init functions */\n-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset)\n+/*\n+ * Targets of fmod_ret programs are marked in BTF by resolve_btfids, see\n+ * BTF_SET8_FMODRET. A \"bpf_fmodret\" decl tag on a function makes it a target.\n+ * A target which sleepable programs may attach to also has a\n+ * \"bpf_fmodret_sleepable\" tag, which never comes without \"bpf_fmodret\".\n+ */\n+#define BTF_FMODRET_TAG\t\t\t\"bpf_fmodret\"\n+#define BTF_FMODRET_SLEEPABLE_TAG\t\"bpf_fmodret_sleepable\"\n+\n+/* Return the function which T tags with decl tag NAME, or 0 */\n+static u32 btf_fmodret_tag_func(const struct btf *btf, const struct btf_type *t,\n+\t\t\t\tconst char *name)\n {\n-\treturn __register_btf_kfunc_id_set(BTF_KFUNC_HOOK_FMODRET, kset);\n+\tconst struct btf_type *func;\n+\n+\tif (!btf_type_is_decl_tag(t) || btf_type_decl_tag(t)-\u003ecomponent_idx != -1)\n+\t\treturn 0;\n+\tif (strcmp(btf_name_by_offset(btf, t-\u003ename_off), name))\n+\t\treturn 0;\n+\tfunc = btf_type_by_id(btf, t-\u003etype);\n+\tif (!func || !btf_type_is_func(func))\n+\t\treturn 0;\n+\treturn t-\u003etype;\n+}\n+\n+/* Build the set of fmod_ret targets of BTF from its decl tags */\n+static int btf_register_fmodret_tags(struct btf *btf)\n+{\n+\tu32 id, func_id, nr_types = btf_nr_types(btf), cnt = 0;\n+\tstruct btf_kfunc_set_tab *tab;\n+\tstruct btf_id_set8 *set;\n+\n+\tfor (id = btf_start_id(btf); id \u003c nr_types; id++)\n+\t\tcnt += !!btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);\n+\tif (!cnt)\n+\t\treturn 0;\n+\n+\tset = kmalloc(struct_size(set, pairs, cnt), GFP_KERNEL | __GFP_NOWARN);\n+\tif (!set)\n+\t\treturn -ENOMEM;\n+\tset-\u003eflags = 0;\n+\tset-\u003ecnt = 0;\n+\tfor (id = btf_start_id(btf); id \u003c nr_types; id++) {\n+\t\tfunc_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);\n+\t\tif (!func_id)\n+\t\t\tcontinue;\n+\t\tset-\u003epairs[set-\u003ecnt].id = func_id;\n+\t\tset-\u003epairs[set-\u003ecnt].flags = 0;\n+\t\tset-\u003ecnt++;\n+\t}\n+\tsort(set-\u003epairs, set-\u003ecnt, sizeof(set-\u003epairs[0]), btf_id_cmp_func, NULL);\n+\n+\t/* Each sleepable tag marks a function which is already in the set */\n+\tfor (id = btf_start_id(btf); id \u003c nr_types; id++) {\n+\t\tstruct btf_id_set8_pair *pair;\n+\n+\t\tfunc_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id),\n+\t\t\t\t\t BTF_FMODRET_SLEEPABLE_TAG);\n+\t\tif (!func_id)\n+\t\t\tcontinue;\n+\t\tpair = btf_id_set8_entry(set, func_id);\n+\t\tif (pair)\n+\t\t\tpair-\u003eflags |= KF_SLEEPABLE;\n+\t}\n+\n+\ttab = btf-\u003ekfunc_set_tab;\n+\tif (!tab) {\n+\t\ttab = kzalloc_obj(*tab, GFP_KERNEL | __GFP_NOWARN);\n+\t\tif (!tab) {\n+\t\t\tkfree(set);\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\t\tbtf-\u003ekfunc_set_tab = tab;\n+\t}\n+\tif (tab-\u003esets[BTF_KFUNC_HOOK_FMODRET]) {\n+\t\tkfree(set);\n+\t\treturn -EEXIST;\n+\t}\n+\ttab-\u003esets[BTF_KFUNC_HOOK_FMODRET] = set;\n+\treturn 0;\n+}\n+\n+static int __init btf_fmodret_tags_init(void)\n+{\n+\tstruct btf *btf;\n+\tint err;\n+\n+\tbtf = btf_get_module_btf(NULL);\n+\tif (IS_ERR_OR_NULL(btf))\n+\t\treturn PTR_ERR_OR_ZERO(btf);\n+\n+\terr = btf_register_fmodret_tags(btf);\n+\tbtf_put(btf);\n+\treturn err;\n }\n-EXPORT_SYMBOL_GPL(register_btf_fmodret_id_set);\n+fs_initcall(btf_fmodret_tags_init);\n \n s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id)\n {\ndiff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c\nindex 9004f2673ccf96..0a78f66a1622c2 100644\n--- a/kernel/bpf/syscall.c\n+++ b/kernel/bpf/syscall.c\n@@ -2490,6 +2490,7 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)\n \tmodule_put(prog-\u003eaux-\u003emod);\n \tkfree(prog-\u003eaux-\u003ekfunc_tab);\n \tkfree(prog-\u003eaux-\u003ectx_arg_info);\n+\tkfree(prog-\u003eaux-\u003efmodret_retvals);\n \tif (prog-\u003eaux-\u003eattach_btf)\n \t\tbtf_put(prog-\u003eaux-\u003eattach_btf);\n \ndiff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c\nindex 20b1b9ff574c7d..5045d9fa68b928 100644\n--- a/kernel/bpf/verifier.c\n+++ b/kernel/bpf/verifier.c\n@@ -13561,6 +13561,14 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn\n \t}\n \n \tif (func_id == BPF_FUNC_tail_call) {\n+\t\t/*\n+\t\t * The program which a tail call jumps to returns for this one,\n+\t\t * and a program array does not check the values it may return.\n+\t\t */\n+\t\tif (env-\u003eprog-\u003eaux-\u003efmodret_retvals) {\n+\t\t\tverbose(env, \"tail calls are not allowed in fmod_ret programs with restricted return values\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n \t\tif (env-\u003ecur_state-\u003ecurframe) {\n \t\t\tstruct bpf_verifier_state *branch;\n \n@@ -20260,6 +20268,67 @@ static bool program_returns_void(struct bpf_verifier_env *env)\n \treturn false;\n }\n \n+/*\n+ * A function which the build found by analysing its code has a\n+ * \"bpf_fmodret_retval:\" decl tag with the values it returns on paths without\n+ * side effects, for example \"bpf_fmodret_retval:-22,-12\". See the comment at\n+ * the top of tools/bpf/resolve_btfids/main.c. An fmod_ret program attached to\n+ * it may return only these values, or 0 to call the function.\n+ */\n+#define BTF_FMODRET_RETVAL_TAG \"bpf_fmodret_retval:\"\n+\n+/* Parse the next value of the list at *LIST and move *LIST past it */\n+static int fmodret_retval_next(const char **list, s64 *value)\n+{\n+\tconst char *end = strchrnul(*list, ',');\n+\tsize_t len = end - *list;\n+\tchar buf[24];\n+\n+\tif (!len || len \u003e= sizeof(buf) || (*end \u0026\u0026 !end[1]))\n+\t\treturn -EINVAL;\n+\tmemcpy(buf, *list, len);\n+\tbuf[len] = 0;\n+\t*list = *end ? end + 1 : end;\n+\treturn kstrtos64(buf, 10, value);\n+}\n+\n+static bool fmodret_retval_allowed(const struct bpf_prog_aux *aux, u64 value)\n+{\n+\tconst struct btf_type *t;\n+\tu32 i;\n+\n+\t/* \"w0 = -22\" zero-extends, the caller of an int function sees -22 */\n+\tt = btf_type_skip_modifiers(aux-\u003eattach_btf, aux-\u003eattach_func_proto-\u003etype, NULL);\n+\tif (t-\u003esize == 4 \u0026\u0026 !upper_32_bits(value))\n+\t\tvalue = (s32)value;\n+\tif (!value)\n+\t\treturn true;\n+\tfor (i = 0; i \u003c aux-\u003efmodret_retval_cnt; i++) {\n+\t\tif (aux-\u003efmodret_retvals[i] == (s64)value)\n+\t\t\treturn true;\n+\t}\n+\treturn false;\n+}\n+\n+/* The program must return a constant: 0 or one of the values of the tag */\n+static int check_fmodret_retval(struct bpf_verifier_env *env, int regno,\n+\t\t\t\tconst char *exit_ctx, const char *reg_name)\n+{\n+\tconst struct bpf_prog_aux *aux = env-\u003eprog-\u003eaux;\n+\tstruct bpf_reg_state *reg = reg_state(env, regno);\n+\tu32 i;\n+\n+\tif (reg-\u003etype == SCALAR_VALUE \u0026\u0026 tnum_is_const(reg-\u003evar_off) \u0026\u0026\n+\t fmodret_retval_allowed(aux, reg-\u003evar_off.value))\n+\t\treturn 0;\n+\n+\tverbose(env, \"%s the register %s must be a constant, it should be 0 or one of\", exit_ctx, reg_name);\n+\tfor (i = 0; i \u003c aux-\u003efmodret_retval_cnt; i++)\n+\t\tverbose(env, \"%s%lld\", i ? \",\" : \" \", aux-\u003efmodret_retvals[i]);\n+\tverbose(env, \"\\n\");\n+\treturn -EINVAL;\n+}\n+\n static int check_return_code(struct bpf_verifier_env *env, int regno, const char *reg_name)\n {\n \tconst char *exit_ctx = \"At program exit\";\n@@ -20316,6 +20385,9 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char\n \tif (prog_type == BPF_PROG_TYPE_CGROUP_SKB \u0026\u0026 (env-\u003eprog-\u003eexpected_attach_type == BPF_CGROUP_INET_EGRESS))\n \t\tenforce_attach_type_range = tnum_range(2, 3);\n \n+\tif (env-\u003eprog-\u003eaux-\u003efmodret_retvals)\n+\t\treturn check_fmodret_retval(env, regno, exit_ctx, reg_name);\n+\n \tif (!return_retval_range(env, \u0026range))\n \t\treturn 0;\n \n@@ -23759,6 +23831,61 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)\n \n #endif /* CONFIG_FUNCTION_ERROR_INJECTION */\n \n+/*\n+ * Programs attached to a function found by the analysis may return only the\n+ * values of its \"bpf_fmodret_retval:\" tag. A function which the static rules\n+ * allow stays unrestricted.\n+ */\n+static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,\n+\t\t\t unsigned long addr, const char *tname)\n+{\n+\tstruct bpf_prog_aux *aux = env-\u003eprog-\u003eaux;\n+\tconst struct btf *btf = aux-\u003eattach_btf;\n+\tconst struct btf_type *t = btf_type_by_id(btf, btf_id);\n+\ts64 *retvals = NULL;\n+\tconst char *values, *list;\n+\tu32 cnt, size = 0;\n+\ts64 v;\n+\n+\tif (!check_attach_modify_return(addr, tname))\n+\t\treturn 0;\n+\tvalues = btf_find_decl_tag_value(btf, t, -1, BTF_FMODRET_RETVAL_TAG);\n+\tif (IS_ERR(values))\n+\t\treturn PTR_ERR(values) == -ENOENT ? 0 : PTR_ERR(values);\n+\n+\tt = btf_type_by_id(btf, t-\u003etype);\n+\tt = t ? btf_type_skip_modifiers(btf, t-\u003etype, NULL) : NULL;\n+\tif (t \u0026\u0026 (btf_type_is_int(t) || btf_is_any_enum(t)))\n+\t\tsize = t-\u003esize;\n+\tif (size != 4 \u0026\u0026 size != 8)\n+\t\tgoto invalid;\n+\n+\tfor (list = values, cnt = 1; *list; list++)\n+\t\tcnt += *list == ',';\n+\tretvals = kmalloc_array(cnt, sizeof(*retvals), GFP_KERNEL_ACCOUNT);\n+\tif (!retvals)\n+\t\treturn -ENOMEM;\n+\tfor (list = values, cnt = 0; *list;) {\n+\t\tif (fmodret_retval_next(\u0026list, \u0026v) || v \u003e= 0 || (size == 4 \u0026\u0026 v \u003c S32_MIN))\n+\t\t\tgoto invalid;\n+\t\tretvals[cnt++] = v;\n+\t}\n+\tif (!cnt)\n+\t\tgoto invalid;\n+\n+\taux-\u003efmodret_retvals = retvals;\n+\taux-\u003efmodret_retval_cnt = cnt;\n+\t/* The interpreter returns only 32 bits */\n+\tif (size == 8)\n+\t\tenv-\u003eprog-\u003ejit_required = 1;\n+\treturn 0;\n+\n+invalid:\n+\tkfree(retvals);\n+\tverbose(env, \"invalid %s%s tag of %s()\\n\", BTF_FMODRET_RETVAL_TAG, values, tname);\n+\treturn -EINVAL;\n+}\n+\n static bool is_tracing_multi_id(const struct bpf_prog *prog, u32 btf_id)\n {\n \treturn is_tracing_multi(prog-\u003eexpected_attach_type) \u0026\u0026 bpf_multi_func_btf_id[0] == btf_id;\n@@ -24173,6 +24300,15 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,\n \t\t\tbpf_log(log, \"Cannot extend fentry/fexit/fsession\\n\");\n \t\t\treturn -EINVAL;\n \t\t}\n+\t\t/*\n+\t\t * An extension could return any value for an fmod_ret program\n+\t\t * whose return values are restricted. This is also checked when\n+\t\t * an extension is attached to another program than at load.\n+\t\t */\n+\t\tif (prog_extension \u0026\u0026 tgt_prog-\u003eaux-\u003efmodret_retvals) {\n+\t\t\tbpf_log(log, \"Cannot extend an fmod_ret program with restricted return values\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n \t} else {\n \t\tif (prog_extension) {\n \t\t\tbpf_log(log, \"Cannot replace kernel functions\\n\");\n@@ -24552,6 +24688,13 @@ static int check_attach_btf_id(struct bpf_verifier_env *env)\n \t\treturn -EINVAL;\n \t}\n \n+\t/* Only a kernel function has a \"bpf_fmodret_retval:\" tag */\n+\tif (prog-\u003eexpected_attach_type == BPF_MODIFY_RETURN \u0026\u0026 !tgt_prog) {\n+\t\tret = fmodret_retval_init(env, btf_id, tgt_info.tgt_addr, tgt_info.tgt_name);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\n \t/*\n \t * We don't get trampoline for tracing_multi programs at this point,\n \t * it's done when tracing_multi link is created.\ndiff --git a/lib/vdso/Makefile.include b/lib/vdso/Makefile.include\nindex cedbf15f80874d..848d58296ba231 100644\n--- a/lib/vdso/Makefile.include\n+++ b/lib/vdso/Makefile.include\n@@ -1,5 +1,8 @@\n # SPDX-License-Identifier: GPL-2.0\n \n+# vDSO code executes outside the kernel.\n+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))\n+\n GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))\n GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))\n \ndiff --git a/net/bpf/test_run.c b/net/bpf/test_run.c\nindex d6c9dc62182ee4..d63c3b6975527e 100644\n--- a/net/bpf/test_run.c\n+++ b/net/bpf/test_run.c\n@@ -643,17 +643,12 @@ CFI_NOSEAL(bpf_kfunc_call_memb_release_dtor);\n \n __bpf_kfunc_end_defs();\n \n-BTF_KFUNCS_START(bpf_test_modify_return_ids)\n+BTF_FMODRET_START(bpf_test_modify_return_ids)\n BTF_ID_FLAGS(func, bpf_modify_return_test)\n BTF_ID_FLAGS(func, bpf_modify_return_test2)\n BTF_ID_FLAGS(func, bpf_modify_return_test_tp)\n BTF_ID_FLAGS(func, bpf_fentry_test1, KF_SLEEPABLE)\n-BTF_KFUNCS_END(bpf_test_modify_return_ids)\n-\n-static const struct btf_kfunc_id_set bpf_test_modify_return_set = {\n-\t.owner = THIS_MODULE,\n-\t.set = \u0026bpf_test_modify_return_ids,\n-};\n+BTF_FMODRET_END(bpf_test_modify_return_ids)\n \n BTF_KFUNCS_START(test_sk_check_kfunc_ids)\n BTF_ID_FLAGS(func, bpf_kfunc_call_test_release, KF_RELEASE)\n@@ -1905,8 +1900,7 @@ static int __init bpf_prog_test_run_init(void)\n \t};\n \tint ret;\n \n-\tret = register_btf_fmodret_id_set(\u0026bpf_test_modify_return_set);\n-\tret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, \u0026bpf_prog_test_kfunc_set);\n+\tret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, \u0026bpf_prog_test_kfunc_set);\n \tret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, \u0026bpf_prog_test_kfunc_set);\n \tret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, \u0026bpf_prog_test_kfunc_set);\n \treturn ret ?: register_btf_id_dtor_kfuncs(bpf_prog_test_dtor_kfunc,\ndiff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c\nindex 4cc16cbeb32817..d0d489624689a4 100644\n--- a/net/mptcp/bpf.c\n+++ b/net/mptcp/bpf.c\n@@ -20,17 +20,6 @@ struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)\n \treturn NULL;\n }\n \n-BTF_SET8_START(bpf_mptcp_fmodret_ids)\n+BTF_FMODRET_START(bpf_mptcp_fmodret_ids)\n BTF_ID_FLAGS(func, update_socket_protocol)\n-BTF_SET8_END(bpf_mptcp_fmodret_ids)\n-\n-static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = {\n-\t.owner = THIS_MODULE,\n-\t.set = \u0026bpf_mptcp_fmodret_ids,\n-};\n-\n-static int __init bpf_mptcp_kfunc_init(void)\n-{\n-\treturn register_btf_fmodret_id_set(\u0026bpf_mptcp_fmodret_set);\n-}\n-late_initcall(bpf_mptcp_kfunc_init);\n+BTF_FMODRET_END(bpf_mptcp_fmodret_ids)\ndiff --git a/scripts/Makefile b/scripts/Makefile\nindex d7fbb7ca098b18..bc5dacc29207b2 100644\n--- a/scripts/Makefile\n+++ b/scripts/Makefile\n@@ -66,6 +66,7 @@ hostprogs += unifdef gen_packed_field_checks\n targets += module.lds\n \n subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins\n+subdir-$(CONFIG_LLVM_PLUGINS) += llvm-plugins\n subdir-$(CONFIG_GENKSYMS) += genksyms\n subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms\n subdir-$(CONFIG_SECURITY_SELINUX) += selinux\ndiff --git a/scripts/Makefile.btf b/scripts/Makefile.btf\nindex 98012344a81ec3..3aa3d084cc6c08 100644\n--- a/scripts/Makefile.btf\n+++ b/scripts/Makefile.btf\n@@ -35,5 +35,6 @@ resolve-btfids-flags-y :=\n resolve-btfids-flags-$(CONFIG_WERROR) += --fatal_warnings\n resolve-btfids-flags-$(if $(KBUILD_EXTMOD),y) += --distill_base\n resolve-btfids-flags-$(if $(KBUILD_VERBOSE),y) += --verbose\n+resolve-btfids-flags-$(CONFIG_FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY) += --patchable_ftrace\n \n export RESOLVE_BTFIDS_FLAGS := $(resolve-btfids-flags-y)\ndiff --git a/scripts/Makefile.llvm-plugins b/scripts/Makefile.llvm-plugins\nnew file mode 100644\nindex 00000000000000..9ff364f4db5957\n--- /dev/null\n+++ b/scripts/Makefile.llvm-plugins\n@@ -0,0 +1,10 @@\n+# SPDX-License-Identifier: GPL-2.0-only\n+\n+llvm-plugin-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_scan_ir.so\n+\n+LLVM_PLUGIN := $(llvm-plugin-y)\n+LLVM_PLUGINS_CFLAGS := $(strip $(addprefix -fpass-plugin=$(objtree)/scripts/llvm-plugins/, \\\n+\t$(LLVM_PLUGIN)) -DLLVM_PLUGINS)\n+export LLVM_PLUGIN LLVM_PLUGINS_CFLAGS\n+\n+KBUILD_CFLAGS += $(LLVM_PLUGINS_CFLAGS)\ndiff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal\nindex 01a37ec872b905..042c31e3bdfb78 100644\n--- a/scripts/Makefile.modfinal\n+++ b/scripts/Makefile.modfinal\n@@ -42,6 +42,7 @@ quiet_cmd_btf_ko = BTF [M] $@\n cmd_btf_ko = \t\t\t\t\t\t\t\\\n \tif [ ! -f $(objtree)/vmlinux ]; then\t\t\t\t\\\n \t\tprintf \"Skipping BTF generation for %s due to unavailability of vmlinux\\n\" $@ 1\u003e\u00262; \\\n+\t\t$(OBJCOPY) --remove-section=.BTF_fmodret_candidates $@;\t\\\n \telse\t\\\n \t\t$(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \\\n \tfi;\ndiff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux\nindex 95b523fe4f9eeb..c8651454c3d366 100644\n--- a/scripts/Makefile.vmlinux\n+++ b/scripts/Makefile.vmlinux\n@@ -73,6 +73,9 @@ vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD\n ifdef CONFIG_DEBUG_INFO_BTF\n vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh\n endif\n+ifdef CONFIG_CLANG_PLUGIN_BPF_SCAN_IR\n+vmlinux.unstripped: $(srctree)/scripts/bpf/fmodret.lds\n+endif\n \n ifdef CONFIG_BUILDTIME_TABLE_SORT\n vmlinux.unstripped: scripts/sorttable\ndiff --git a/scripts/bpf/fmodret.lds b/scripts/bpf/fmodret.lds\nnew file mode 100644\nindex 00000000000000..6bdfde9e767979\n--- /dev/null\n+++ b/scripts/bpf/fmodret.lds\n@@ -0,0 +1,6 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+SECTIONS {\n+\t/DISCARD/ : {\n+\t\t*(.BTF_fmodret_candidates)\n+\t}\n+}\ndiff --git a/scripts/bpf/fmodret_metadata.h b/scripts/bpf/fmodret_metadata.h\nnew file mode 100644\nindex 00000000000000..209f266f89a375\n--- /dev/null\n+++ b/scripts/bpf/fmodret_metadata.h\n@@ -0,0 +1,29 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+#ifndef FMODRET_METADATA_H\n+#define FMODRET_METADATA_H\n+\n+/*\n+ * The records which the BPF compiler plugins write to .BTF_fmodret_candidates\n+ * for resolve_btfids, one for each function with values and for each weak\n+ * definition. If a function has two weak definitions, the record of the one\n+ * which loses the link would describe the body of the other, so both need a\n+ * record to make the function ambiguous. A record is packed, target-endian and\n+ * aligned to the target pointer size:\n+ *\n+ *\tu32 magic, version, size, flags;\n+ *\tptr function;\n+ *\tu32 nr_values, reserved;\n+ *\ts64 values[nr_values];\n+ *\n+ * The values are the errors which clean paths of the function return, sorted\n+ * and unique.\n+ */\n+#define FMODRET_META_MAGIC 0x46524d32U\n+#define FMODRET_META_VERSION 1U\n+#define FMODRET_META_HEADER_SIZE(ptr_size) (24 + (ptr_size))\n+#define FMODRET_META_VALUE_SIZE 8\n+\n+#define FMODRET_FN_WEAK (1U \u003c\u003c 0)\n+#define FMODRET_FN_RET64 (1U \u003c\u003c 1)\n+\n+#endif\ndiff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh\nindex b17f1189298a55..5bca864631e0b2 100755\n--- a/scripts/gen-btf.sh\n+++ b/scripts/gen-btf.sh\n@@ -120,7 +120,9 @@ gen_btf_o()\n \n embed_btf_data()\n {\n-\t${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF ${ELF_FILE}\n+\t# resolve_btfids has consumed the compiler plugin records of the module\n+\t${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \\\n+\t\t--remove-section=.BTF_fmodret_candidates ${ELF_FILE}\n \n \t# a module might not have a .BTF_ids or .BTF.base section\n \tbtf_base=\"${ELF_FILE}.BTF.base\"\ndiff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh\nindex d5710e900bd408..f39e2d1f1400a2 100755\n--- a/scripts/link-vmlinux.sh\n+++ b/scripts/link-vmlinux.sh\n@@ -89,6 +89,13 @@ vmlinux_link()\n \t\tldlibs=\n \tfi\n \n+\tif is_enabled CONFIG_CLANG_PLUGIN_BPF_SCAN_IR; then\n+\t\tif [ \"${output}\" = \".tmp_vmlinux1\" ]; then\n+\t\t\tldflags=\"${ldflags} ${wl}--emit-relocs\"\n+\t\telse\n+\t\t\tldflags=\"${ldflags} ${wl}--script=${srctree}/scripts/bpf/fmodret.lds\"\n+\t\tfi\n+\tfi\n \tldflags=\"${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}\"\n \n \t# The kallsyms linking does not need debug symbols included.\ndiff --git a/scripts/llvm-plugins/Kconfig b/scripts/llvm-plugins/Kconfig\nnew file mode 100644\nindex 00000000000000..c4d62ae5f65d25\n--- /dev/null\n+++ b/scripts/llvm-plugins/Kconfig\n@@ -0,0 +1,33 @@\n+# SPDX-License-Identifier: GPL-2.0-only\n+\n+menuconfig LLVM_PLUGINS\n+\tbool \"LLVM plugins\"\n+\tdepends on CC_IS_CLANG\n+\tdepends on $(success,$(LLVM_CONFIG) --version)\n+\t# The plugins are built against the headers of llvm-config and loaded\n+\t# by clang, so both must come from the same LLVM release\n+\tdepends on $(success,test \"$(shell,$(LLVM_CONFIG) --version 2\u003e/dev/null | cut -d. -f1)\" = \"$(shell,$(CC) -dumpversion | cut -d. -f1)\")\n+\t# The host C++ compiler must accept the LLVM headers and flags\n+\tdepends on $(success,echo '#include \u003cllvm/Config/llvm-config.h\u003e' | $(HOSTCXX) $(shell,$(LLVM_CONFIG) --cxxflags 2\u003e/dev/null) -x c++ -fsyntax-only -)\n+\tdefault y\n+\thelp\n+\t LLVM plugins add compiler passes to kernel and module builds.\n+\t Building them requires LLVM development headers and llvm-config\n+\t of the same major version as clang. Each plugin has its own option\n+\t below and is built only if that option is enabled.\n+\n+if LLVM_PLUGINS\n+\n+config CLANG_PLUGIN_BPF_SCAN_IR\n+\tbool \"Scan optimized LLVM IR for BPF\"\n+\tdepends on X86_64 || ARM64\n+\tdepends on DEBUG_INFO_BTF\n+\tdepends on !LTO_CLANG\n+\tdepends on CLANG_VERSION \u003e= 190000\n+\thelp\n+\t Find the functions which return errors on paths without side\n+\t effects and describe them in .BTF_fmodret_candidates for\n+\t resolve_btfids. The section is discarded from the final kernel\n+\t and modules.\n+\n+endif\ndiff --git a/scripts/llvm-plugins/Makefile b/scripts/llvm-plugins/Makefile\nnew file mode 100644\nindex 00000000000000..a1f8503c8bcd12\n--- /dev/null\n+++ b/scripts/llvm-plugins/Makefile\n@@ -0,0 +1,21 @@\n+# SPDX-License-Identifier: GPL-2.0-only\n+\n+always-y += $(LLVM_PLUGIN)\n+targets += $(LLVM_PLUGIN)\n+clean-files += *.so\n+\n+# Run llvm-config only to build a plugin, not when this Makefile is parsed\n+llvm_cxxflags = $(shell $(LLVM_CONFIG) --cxxflags)\n+llvm_ldflags = $(shell $(LLVM_CONFIG) --ldflags)\n+\n+quiet_cmd_llvm_plugin = HOSTCXX $@\n+ cmd_llvm_plugin = $(HOSTCXX) -Wp,-MD,$(depfile) $(KBUILD_HOSTCXXFLAGS) \\\n+\t\t$(llvm_cxxflags) -fPIC -shared $(KBUILD_HOSTLDFLAGS) -o $@ $\u003c \\\n+\t\t$(llvm_ldflags)\n+\n+$(obj)/%.so: $(src)/%.cpp FORCE\n+\t$(call if_changed_dep,llvm_plugin)\n+\n+$(obj)/../../include/generated/llvm-plugins.h: $(addprefix $(obj)/, $(LLVM_PLUGIN)) FORCE\n+\t$(call if_changed,touch)\n+always-y += ../../include/generated/llvm-plugins.h\ndiff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp\nnew file mode 100644\nindex 00000000000000..a3c6336662fc1e\n--- /dev/null\n+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp\n@@ -0,0 +1,374 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/* bpf_scan_ir: optimizer-last LLVM IR analysis for BPF. */\n+\n+#include \"../bpf/fmodret_metadata.h\"\n+\n+#include \u003calgorithm\u003e\n+#include \"llvm/ADT/DenseMap.h\"\n+#include \"llvm/ADT/SmallPtrSet.h\"\n+#include \"llvm/ADT/SmallVector.h\"\n+#include \"llvm/Analysis/CaptureTracking.h\"\n+#include \"llvm/Analysis/LazyValueInfo.h\"\n+#include \"llvm/Analysis/ValueTracking.h\"\n+#include \"llvm/Config/llvm-config.h\"\n+#include \"llvm/IR/Analysis.h\"\n+#include \"llvm/IR/BasicBlock.h\"\n+#include \"llvm/IR/CFG.h\"\n+#include \"llvm/IR/ConstantRange.h\"\n+#include \"llvm/IR/Constants.h\"\n+#include \"llvm/IR/Dominators.h\"\n+#include \"llvm/IR/GlobalVariable.h\"\n+#include \"llvm/IR/Instructions.h\"\n+#include \"llvm/IR/IntrinsicInst.h\"\n+#include \"llvm/IR/Module.h\"\n+#include \"llvm/IR/PassManager.h\"\n+#include \"llvm/Passes/PassBuilder.h\"\n+#if __has_include(\"llvm/Plugins/PassPlugin.h\")\n+#include \"llvm/Plugins/PassPlugin.h\"\n+#else\n+#include \"llvm/Passes/PassPlugin.h\"\n+#endif\n+#include \"llvm/Transforms/Utils/ModuleUtils.h\"\n+\n+using namespace llvm;\n+\n+namespace {\n+\n+static constexpr int MaxErrno = 4095;\n+\n+// is Range in [-4095, 0) ?\n+static bool isErrnoRange(const ConstantRange \u0026Range)\n+{\n+\tunsigned Width = Range.getBitWidth();\n+\n+\tif (Width != 32 \u0026\u0026 Width != 64)\n+\t\treturn false;\n+\n+\tauto L = APInt(Width, 0) - APInt(Width, MaxErrno);\n+\tauto R = APInt(Width, 0);\n+\treturn !Range.isEmptySet() \u0026\u0026 ConstantRange(L, R).contains(Range);\n+}\n+\n+/* The errors which clean paths of a function return */\n+using ReturnValues = SmallVector\u003cint64_t, 8\u003e;\n+\n+/* Return true and set N if RANGE holds a single errno */\n+static bool singleErrno(const ConstantRange \u0026Range, int64_t \u0026N)\n+{\n+\tconst APInt *C = Range.getSingleElement();\n+\n+\tif (!C || !isErrnoRange(Range))\n+\t\treturn false;\n+\tN = C-\u003egetSExtValue();\n+\treturn true;\n+}\n+\n+/* Return true and set N if the value at use U is a single errno */\n+static bool singleErrnoAtUse(const Use \u0026U, LazyValueInfo \u0026LVI, int64_t \u0026N)\n+{\n+\treturn U.get()-\u003egetType()-\u003eisIntegerTy() \u0026\u0026\n+\t singleErrno(LVI.getConstantRangeAtUse(U, false), N);\n+}\n+\n+/* Return true and set N if V is an errno constant */\n+static bool errnoConstant(const Value *V, int64_t \u0026N)\n+{\n+\tauto *C = dyn_cast\u003cConstantInt\u003e(V);\n+\n+\tif (!C || (C-\u003egetBitWidth() != 32 \u0026\u0026 C-\u003egetBitWidth() != 64))\n+\t\treturn false;\n+\n+\tN = C-\u003egetSExtValue();\n+\treturn N \u003e= -MaxErrno \u0026\u0026 N \u003c 0;\n+}\n+\n+/* A sign extension or a truncation between integers of at least 32 bits */\n+static bool preservesErrnoCast(const CastInst *Cast)\n+{\n+\tType *Src = Cast-\u003egetSrcTy(), *Dst = Cast-\u003egetDestTy();\n+\n+\tif (Src-\u003eisIntegerTy() \u0026\u0026 Dst-\u003eisIntegerTy())\n+\t\treturn (Cast-\u003egetOpcode() == Instruction::SExt ||\n+\t\t\tCast-\u003egetOpcode() == Instruction::Trunc) \u0026\u0026\n+\t\t Src-\u003egetIntegerBitWidth() \u003e= 32 \u0026\u0026 Dst-\u003egetIntegerBitWidth() \u003e= 32;\n+\treturn false;\n+}\n+\n+/*\n+ * Add the errors which V, the value returned in RET, is known to be on the\n+ * edges to RET from CLEANBBS to VALUES. Follow the casts in RET which keep the\n+ * error to a PHI in RET. A PHI in an earlier block is not followed: a later\n+ * branch may split its values between clean and dirty paths.\n+ */\n+static void collectErrnos(Value *V, BasicBlock *Ret, LazyValueInfo \u0026LVI,\n+\t\t\t const SmallPtrSetImpl\u003cBasicBlock *\u003e \u0026CleanBBs, ReturnValues \u0026Values)\n+{\n+\tint64_t N;\n+\n+\twhile (auto *Cast = dyn_cast\u003cCastInst\u003e(V)) {\n+\t\tif (Cast-\u003egetParent() != Ret || !preservesErrnoCast(Cast))\n+\t\t\treturn;\n+\t\tif (singleErrnoAtUse(Cast-\u003egetOperandUse(0), LVI, N)) {\n+\t\t\tValues.push_back(N);\n+\t\t\treturn;\n+\t\t}\n+\t\tV = Cast-\u003egetOperand(0);\n+\t}\n+\n+\tauto *Phi = dyn_cast\u003cPHINode\u003e(V);\n+\n+\tif (!Phi || Phi-\u003egetParent() != Ret)\n+\t\treturn;\n+\tfor (unsigned i = 0; i \u003c Phi-\u003egetNumIncomingValues(); i++) {\n+\t\tValue *Incoming = Phi-\u003egetIncomingValue(i);\n+\t\tBasicBlock *Pred = Phi-\u003egetIncomingBlock(i);\n+\n+\t\tif (!CleanBBs.contains(Pred))\n+\t\t\tcontinue;\n+\t\tif (errnoConstant(Incoming, N)) {\n+\t\t\tValues.push_back(N);\n+\t\t\tcontinue;\n+\t\t}\n+\t\t/*\n+\t\t * LazyValueInfo knows the ranges of integers only. On an edge it\n+\t\t * also uses the branch condition, as in \"if (err == -EINVAL) goto\n+\t\t * out\", where err is the error.\n+\t\t */\n+\t\tif (Incoming-\u003egetType()-\u003eisIntegerTy() \u0026\u0026\n+\t\t singleErrno(LVI.getConstantRangeOnEdge(Incoming, Pred, Ret), N))\n+\t\t\tValues.push_back(N);\n+\t}\n+}\n+\n+class CleanInstructions {\n+\tDominatorTree \u0026DT;\n+\tDenseMap\u003cconst Instruction *, bool\u003e PrivateWrites;\n+\n+\tbool privateDestination(const Value *Pointer, const Instruction \u0026I)\n+\t{\n+\t\tauto It = PrivateWrites.find(\u0026I);\n+\t\tif (It != PrivateWrites.end())\n+\t\t\treturn It-\u003esecond;\n+\t\tauto *Allocation = dyn_cast\u003cAllocaInst\u003e(getUnderlyingObject(Pointer));\n+\t\tbool Private = false;\n+\n+\t\tif (Allocation \u0026\u0026 Allocation-\u003egetFunction() == I.getFunction()) {\n+\t\t\t// A later call may receive the initialized object. Only captures\n+\t\t\t// that can precede this write make the write externally visible.\n+#if LLVM_VERSION_MAJOR \u003e= 21\n+\t\t\tPrivate = !PointerMayBeCapturedBefore(Allocation, true, \u0026I, \u0026DT, true, 256);\n+#else\n+\t\t\tPrivate = !PointerMayBeCapturedBefore(Allocation, true, true, \u0026I, \u0026DT, true, 256);\n+#endif\n+\t\t}\n+\t\tPrivateWrites[\u0026I] = Private;\n+\t\treturn Private;\n+\t}\n+\n+public:\n+\texplicit CleanInstructions(DominatorTree \u0026DT) : DT(DT) {}\n+\n+\tbool isClean(const Instruction \u0026I)\n+\t{\n+\t\tif (auto *Store = dyn_cast\u003cStoreInst\u003e(\u0026I))\n+\t\t\treturn !Store-\u003eisVolatile() \u0026\u0026 !Store-\u003eisAtomic() \u0026\u0026\n+\t\t\t privateDestination(Store-\u003egetPointerOperand(), I);\n+\t\tif (auto *Memory = dyn_cast\u003cMemIntrinsic\u003e(\u0026I))\n+\t\t\treturn !Memory-\u003eisVolatile() \u0026\u0026\n+\t\t\t privateDestination(Memory-\u003egetRawDest(), I);\n+\t\tif (auto *Intrinsic = dyn_cast\u003cIntrinsicInst\u003e(\u0026I)) {\n+\t\t\tswitch (Intrinsic-\u003egetIntrinsicID()) {\n+\t\t\tcase Intrinsic::assume:\n+\t\t\tcase Intrinsic::lifetime_start:\n+\t\t\tcase Intrinsic::lifetime_end:\n+\t\t\tcase Intrinsic::invariant_start:\n+\t\t\tcase Intrinsic::invariant_end:\n+\t\t\t\treturn true;\n+\t\t\tdefault:\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t\treturn !I.mayHaveSideEffects();\n+\t}\n+};\n+\n+static bool cleanBB(BasicBlock *BB, CleanInstructions \u0026Clean)\n+{\n+\tfor (Instruction \u0026I : *BB) {\n+\t\tif (!Clean.isClean(I))\n+\t\t\treturn false;\n+\t}\n+\treturn true;\n+}\n+\n+/* Add the errors which clean paths of F are known to return to VALUES */\n+static void cleanReturnValues(Function \u0026F, LazyValueInfo \u0026LVI, CleanInstructions \u0026Clean,\n+\t\t\t ReturnValues \u0026Values)\n+{\n+\tSmallPtrSet\u003cBasicBlock *, 32\u003e CleanBBs;\n+\tSmallVector\u003cBasicBlock *, 32\u003e Worklist;\n+\tbool Changed;\n+\n+\t/* The clean blocks which the entry reaches through clean blocks */\n+\tif (cleanBB(\u0026F.getEntryBlock(), Clean)) {\n+\t\tCleanBBs.insert(\u0026F.getEntryBlock());\n+\t\tWorklist.push_back(\u0026F.getEntryBlock());\n+\t}\n+\twhile (!Worklist.empty()) {\n+\t\tBasicBlock *BB = Worklist.pop_back_val();\n+\n+\t\tfor (BasicBlock *Successor : successors(BB)) {\n+\t\t\tif (!CleanBBs.contains(Successor) \u0026\u0026 cleanBB(Successor, Clean)) {\n+\t\t\t\tCleanBBs.insert(Successor);\n+\t\t\t\tWorklist.push_back(Successor);\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * Keep the blocks which only clean paths reach: after a dirty path joins\n+\t * a clean one, a branch may depend on what the dirty path did\n+\t */\n+\tdo {\n+\t\tChanged = false;\n+\t\tfor (BasicBlock \u0026BB : F) {\n+\t\t\tif (!CleanBBs.contains(\u0026BB))\n+\t\t\t\tcontinue;\n+\t\t\tfor (BasicBlock *Pred : predecessors(\u0026BB)) {\n+\t\t\t\tif (!CleanBBs.contains(Pred)) {\n+\t\t\t\t\tCleanBBs.erase(\u0026BB);\n+\t\t\t\t\tChanged = true;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t}\n+\t} while (Changed);\n+\n+\t/*\n+\t * The returns of the clean blocks which only clean paths reach or which\n+\t * one of them branches to\n+\t */\n+\tfor (BasicBlock \u0026BB : F) {\n+\t\tauto *Return = dyn_cast\u003cReturnInst\u003e(BB.getTerminator());\n+\t\tbool Entered = CleanBBs.contains(\u0026BB);\n+\t\tint64_t N;\n+\n+\t\tif (!Return || !Return-\u003egetReturnValue() || !cleanBB(\u0026BB, Clean))\n+\t\t\tcontinue;\n+\t\tfor (BasicBlock *Pred : predecessors(\u0026BB))\n+\t\t\tEntered |= CleanBBs.contains(Pred);\n+\t\tif (!Entered)\n+\t\t\tcontinue;\n+\t\tif (singleErrnoAtUse(Return-\u003egetOperandUse(0), LVI, N)) {\n+\t\t\tValues.push_back(N);\n+\t\t\tcontinue;\n+\t\t}\n+\t\tcollectErrnos(Return-\u003egetReturnValue(), \u0026BB, LVI, CleanBBs, Values);\n+\t}\n+\tllvm::sort(Values);\n+\tValues.erase(std::unique(Values.begin(), Values.end()), Values.end());\n+}\n+\n+struct FunctionDescription {\n+\tFunction *Fn;\n+\tunsigned Flags;\n+\tReturnValues Values;\n+};\n+\n+static void emitDescriptions(Module \u0026M, ArrayRef\u003cFunctionDescription\u003e Functions)\n+{\n+\tauto *I32 = Type::getInt32Ty(M.getContext());\n+\tauto *I64 = Type::getInt64Ty(M.getContext());\n+\tunsigned PtrSize = M.getDataLayout().getPointerSize();\n+\tauto N = [\u0026](unsigned V) -\u003e Constant * { return ConstantInt::get(I32, V); };\n+\n+\tfor (const auto \u0026F : Functions) {\n+\t\tunsigned Size = FMODRET_META_HEADER_SIZE(PtrSize) +\n+\t\t\t\tF.Values.size() * FMODRET_META_VALUE_SIZE;\n+\t\tSmallVector\u003cConstant *, 16\u003e Fields = {\n+\t\t\tN(FMODRET_META_MAGIC), N(FMODRET_META_VERSION), N(Size), N(F.Flags),\n+\t\t\tF.Fn, N(F.Values.size()), N(0),\n+\t\t};\n+\n+\t\tfor (int64_t V : F.Values)\n+\t\t\tFields.push_back(ConstantInt::getSigned(I64, V));\n+\t\tConstant *Record = ConstantStruct::getAnon(M.getContext(), Fields, true);\n+\t\tauto *GV = new GlobalVariable(M, Record-\u003egetType(), true, GlobalValue::PrivateLinkage,\n+\t\t\t\t\t Record, \"fmodret.function.\" + F.Fn-\u003egetName());\n+\t\tGV-\u003esetSection(\".BTF_fmodret_candidates\");\n+\t\tGV-\u003esetAlignment(M.getDataLayout().getPointerABIAlignment(0));\n+\t\tappendToUsed(M, {GV});\n+\t}\n+}\n+\n+#if LLVM_VERSION_MAJOR \u003e= 24\n+class BpfScanIRPass : public RequiredPassInfoMixin\u003cBpfScanIRPass\u003e\n+#else\n+class BpfScanIRPass : public PassInfoMixin\u003cBpfScanIRPass\u003e\n+#endif\n+{\n+public:\n+\tPreservedAnalyses run(Module \u0026M, ModuleAnalysisManager \u0026MAM)\n+\t{\n+\t\t// Avoid emitting duplicate descriptions if bpf_scan_ir runs again\n+\t\t// on the same module.\n+\t\tif (M.getNamedMetadata(\"bpf_scan_ir.done\"))\n+\t\t\treturn PreservedAnalyses::all();\n+\n+\t\tSmallVector\u003cFunctionDescription, 32\u003e Descriptions;\n+\t\tauto \u0026FAM = MAM.getResult\u003cFunctionAnalysisManagerModuleProxy\u003e(M).getManager();\n+\n+\t\tfor (Function \u0026F : M) {\n+\t\t\tif (F.isIntrinsic() || F.isDeclaration() || F.hasAvailableExternallyLinkage())\n+\t\t\t\tcontinue;\n+\t\t\tType *Ret = F.getReturnType();\n+\t\t\tunsigned Flags = 0;\n+\t\t\tReturnValues Values;\n+\n+\t\t\tif (!Ret-\u003eisIntegerTy(32) \u0026\u0026 !Ret-\u003eisIntegerTy(64))\n+\t\t\t\tcontinue;\n+\t\t\tif (F.isWeakForLinker())\n+\t\t\t\tFlags |= FMODRET_FN_WEAK;\n+\t\t\tif (Ret-\u003eisIntegerTy(64))\n+\t\t\t\tFlags |= FMODRET_FN_RET64;\n+\t\t\tauto \u0026LVI = FAM.getResult\u003cLazyValueAnalysis\u003e(F);\n+\t\t\tauto \u0026DT = FAM.getResult\u003cDominatorTreeAnalysis\u003e(F);\n+\t\t\tCleanInstructions Clean(DT);\n+\n+\t\t\tcleanReturnValues(F, LVI, Clean, Values);\n+\t\t\t/* A weak definition always has a record, see fmodret_metadata.h */\n+\t\t\tif (!Values.empty() || (Flags \u0026 FMODRET_FN_WEAK))\n+\t\t\t\tDescriptions.push_back({\u0026F, Flags, std::move(Values)});\n+\t\t}\n+\t\tif (Descriptions.empty())\n+\t\t\treturn PreservedAnalyses::all();\n+\t\temitDescriptions(M, Descriptions);\n+\t\tM.getOrInsertNamedMetadata(\"bpf_scan_ir.done\");\n+\n+\t\tPreservedAnalyses PA;\n+\t\tPA.preserve\u003cFunctionAnalysisManagerModuleProxy\u003e();\n+\t\tPA.preserveSet\u003cCFGAnalyses\u003e();\n+\t\tPA.preserve\u003cLazyValueAnalysis\u003e();\n+\t\treturn PA;\n+\t}\n+\n+\tstatic bool isRequired() { return true; }\n+};\n+\n+} // namespace\n+\n+extern \"C\" LLVM_ATTRIBUTE_WEAK PassPluginLibraryInfo llvmGetPassPluginInfo()\n+{\n+\treturn {\n+\t\tLLVM_PLUGIN_API_VERSION,\n+\t\t\"bpf_scan_ir\",\n+\t\t\"1.0\",\n+\t\t[](PassBuilder \u0026PB) {\n+\t\t\tPB.registerOptimizerLastEPCallback(\n+\t\t\t\t[](ModulePassManager \u0026MPM, OptimizationLevel, auto...) {\n+\t\t\t\t\tMPM.addPass(BpfScanIRPass());\n+\t\t\t\t }\n+\t\t\t);\n+\t\t}\n+\t};\n+}\ndiff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c\nindex 75374c64b8cc78..3bae985492288f 100644\n--- a/scripts/mod/modpost.c\n+++ b/scripts/mod/modpost.c\n@@ -772,6 +772,7 @@ static const char *const section_white_list[] =\n \t\".llvm.call-graph-profile\",\t/* call graph */\n \t\"__llvm_covfun\",\n \t\"__llvm_covmap\",\n+\t\".BTF_fmodret_candidates\",\n \t\".klp.symid\",\t\t\t/* objtool --klp-symids */\n \tNULL\n };\ndiff --git a/scripts/module.lds.S b/scripts/module.lds.S\nindex d0f20042895763..e6ba499dc87661 100644\n--- a/scripts/module.lds.S\n+++ b/scripts/module.lds.S\n@@ -27,6 +27,11 @@ SECTIONS {\n \t__ex_table\t\t0 : ALIGN(4) { KEEP(*(__ex_table)) }\n \n \t__patchable_function_entries 0 : { *(__patchable_function_entries) }\n+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES\n+\t.BTF_fmodret_candidates\t0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }\n+#else\n+\t/DISCARD/ : { *(.BTF_fmodret_candidates) }\n+#endif\n \n \t.init.klp_funcs\t\t0 : ALIGN(8) { KEEP(*(.init.klp_funcs)) }\n \t.init.klp_objects\t0 : ALIGN(8) { KEEP(*(.init.klp_objects)) }\ndiff --git a/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst\nnew file mode 100644\nindex 00000000000000..1580fd614010ba\n--- /dev/null\n+++ b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst\n@@ -0,0 +1,111 @@\n+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)\n+\n+===============\n+bpftool-fmodret\n+===============\n+-------------------------------------------------------------------------------\n+tool to inspect BPF fmod_ret targets\n+-------------------------------------------------------------------------------\n+\n+:Manual section: 8\n+\n+.. include:: substitutions.rst\n+\n+SYNOPSIS\n+========\n+\n+**bpftool** [*OPTIONS*] **fmodret** *COMMAND*\n+\n+*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } *FILE* }\n+\n+*COMMANDS* := { **list** | **show** | **help** }\n+\n+FMODRET COMMANDS\n+================\n+\n+| **bpftool** **fmodret list** [*OBJECT*]\n+| **bpftool** **fmodret show** *FUNC* [*OBJECT*]\n+| **bpftool** **fmodret help**\n+|\n+| *FUNC* := { **name** *NAME* | **id** *BTF_ID* }\n+| *OBJECT* := { **file** *FILE* | **vmlinux** | **module** *NAME* }\n+\n+DESCRIPTION\n+===========\n+\n+bpftool fmodret list [*OBJECT*]\n+ List the fmod_ret targets, the functions which the kernel build tags with\n+ the **bpf_fmodret** BTF decl tag, in vmlinux and in loaded modules. Each\n+ entry contains the BTF ID of the function, its name and the name of the\n+ object it belongs to. Targets which sleepable programs may attach to are\n+ marked **sleepable**.\n+\n+ With **file**, read the BTF from *FILE* (for example a vmlinux image or a\n+ **BTF** file) instead of the running kernel. With **vmlinux** or\n+ **module**, list only the targets of vmlinux or of the module *NAME*.\n+\n+ With **--json**, each entry contains **btf_id**, **name**, **owner** and\n+ **sleepable** fields.\n+\n+ The verifier also accepts functions by its static rules, such as the error\n+ injection list or **security_** hooks. Such functions are listed only if\n+ the kernel build tagged them. Even if they are listed, the restrictions of\n+ their **bpf_fmodret_retval:** tags do not apply to them: programs may\n+ return any value, and sleepable programs may attach to some of them without\n+ the **sleepable** mark. This may change in the future by either not listing\n+ them, or listing them without restrictions. Normal verifier and tracing\n+ checks apply to all of them.\n+\n+bpftool fmodret show *FUNC* [*OBJECT*]\n+ Show the function named *NAME* or with BTF ID *BTF_ID* in detail. It does\n+ not have to be a target. A name may belong to several functions, for\n+ example to functions of different modules. A BTF ID is looked up in\n+ vmlinux, unless **module** or **file** is given. *OBJECT* is as for\n+ **list**.\n+\n+ Besides the BTF ID, the name and the object, show the prototype of the\n+ function and if it is a target: **found by the kernel build**, **declared**\n+ or **not tagged**. For a target show if sleepable programs may attach to it\n+ and the values which programs may return: 0 and the values of its\n+ **bpf_fmodret_retval:** tag with the names of the errors on the host, or\n+ **any** for a declared target. A tag which the verifier would reject,\n+ malformed, not the only one or with values which the return type cannot\n+ hold, is shown as invalid. The values do not apply to a function which the\n+ static rules allow, see **list**. For the running kernel, also show the\n+ type of the entry of the function in the error injection list, if the list\n+ is available in debugfs.\n+\n+ With **--json**, each function contains **btf_id**, **name**, **owner**,\n+ **prototype**, **target** (**found**, **declared** or **none**) and\n+ **sleepable** fields, a **retval** array for a target found by the build\n+ with the values programs may return, 0 included, or a **retval_error**\n+ string if its tag is invalid, and an **error_injection** field if the\n+ function is in the error injection list. A BTF ID gives one function, a\n+ name an array of them.\n+\n+bpftool fmodret help\n+ Print short help message.\n+\n+OPTIONS\n+========\n+\n+.. include:: common_options.rst\n+\n+-B, --base-btf *FILE*\n+ Pass the base BTF of a split BTF *FILE*, usually the BTF of vmlinux. Raw\n+ BTF of a module needs it. A module image may carry the base it was built\n+ against in its **.BTF.base** section instead, but its BTF IDs then differ\n+ from those of the running kernel.\n+\n+EXAMPLES\n+========\n+\n+**# bpftool fmodret show name bpf_testmod_retval_join module bpf_test_retval**\n+\n+::\n+\n+ 123456 bpf_testmod_retval_join (bpf_test_retval)\n+ int bpf_testmod_retval_join(int *result, int value)\n+ target: found by the kernel build\n+ sleepable: no\n+ retval: 0, -34 (ERANGE), -22 (EINVAL)\ndiff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst\nindex f38ae5c4043917..2c64c22a50269a 100644\n--- a/tools/bpf/bpftool/Documentation/bpftool.rst\n+++ b/tools/bpf/bpftool/Documentation/bpftool.rst\n@@ -21,7 +21,7 @@ SYNOPSIS\n **bpftool** **version**\n \n *OBJECT* := { **map** | **prog** | **link** | **cgroup** | **perf** | **net** | **feature** |\n-**btf** | **gen** | **struct_ops** | **iter** }\n+**btf** | **gen** | **struct_ops** | **iter** | **fmodret** }\n \n *OPTIONS* := { { **-V** | **--version** } | |COMMON_OPTIONS| }\n \ndiff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool\nindex a28af31937ad4b..83288c22b3d34e 100644\n--- a/tools/bpf/bpftool/bash-completion/bpftool\n+++ b/tools/bpf/bpftool/bash-completion/bpftool\n@@ -675,6 +675,37 @@ _bpftool()\n ;;\n esac\n ;;\n+ fmodret)\n+ case $command in\n+ list|show)\n+ case $prev in\n+ file)\n+ _filedir\n+ ;;\n+ module)\n+ COMPREPLY=( $( compgen -W \"$( command ls \\\n+ /sys/kernel/btf 2\u003e/dev/null | \\\n+ command grep -vx -e vmlinux -e '.*\\.inline' )\" \\\n+ -- \"$cur\" ) )\n+ ;;\n+ name|id)\n+ ;;\n+ *)\n+ if [[ $prev == show ]]; then\n+ COMPREPLY=( $( compgen -W 'name id' -- \"$cur\" ) )\n+ else\n+ _bpftool_one_of_list 'file vmlinux module'\n+ fi\n+ ;;\n+ esac\n+ return 0\n+ ;;\n+ *)\n+ [[ $prev == $object ]] \u0026\u0026 \\\n+ COMPREPLY=( $( compgen -W 'help list show' -- \"$cur\" ) )\n+ ;;\n+ esac\n+ ;;\n map)\n case $command in\n show|list|dump|peek|pop|dequeue|freeze)\ndiff --git a/tools/bpf/bpftool/fmodret.c b/tools/bpf/bpftool/fmodret.c\nnew file mode 100644\nindex 00000000000000..941166da53cc23\n--- /dev/null\n+++ b/tools/bpf/bpftool/fmodret.c\n@@ -0,0 +1,896 @@\n+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)\n+\n+#ifndef _GNU_SOURCE\n+#define _GNU_SOURCE\n+#endif\n+#include \u003cdirent.h\u003e\n+#include \u003cerrno.h\u003e\n+#include \u003climits.h\u003e\n+#include \u003cstdarg.h\u003e\n+#include \u003cstdbool.h\u003e\n+#include \u003cstdio.h\u003e\n+#include \u003cstdlib.h\u003e\n+#include \u003cstring.h\u003e\n+\n+#include \u003cbpf/btf.h\u003e\n+#include \u003cbpf/libbpf.h\u003e\n+\n+#include \"main.h\"\n+\n+/*\n+ * Decl tags added by resolve_btfids to functions in BTF_FMODRET sets. A target\n+ * which sleepable programs may attach to has both tags. A target which the\n+ * build found by analysing its code has a \"bpf_fmodret_retval:\" tag with the\n+ * values programs may return, such as \"bpf_fmodret_retval:-22,-12\".\n+ */\n+#define TAG_FMODRET\t\t\"bpf_fmodret\"\n+#define TAG_FMODRET_SLEEPABLE\t\"bpf_fmodret_sleepable\"\n+#define TAG_FMODRET_RETVAL\t\"bpf_fmodret_retval:\"\n+\n+#define SYS_BTF_DIR\t\t\"/sys/kernel/btf\"\n+#define ERROR_INJECTION_LIST\t\"/sys/kernel/debug/error_injection/list\"\n+\n+struct fmodret_entry {\n+\t__u32 btf_id;\n+\tbool target;\n+\tbool sleepable;\n+\tconst char *retval;\n+\t/* More than one \"bpf_fmodret_retval:\" tag, which the verifier rejects */\n+\tbool retval_dup;\n+};\n+\n+struct fmodret_filter {\n+\t/* Only vmlinux, or only the module with this name */\n+\tbool vmlinux;\n+\tconst char *module;\n+\t/* The function to show, by name or by BTF ID */\n+\tconst char *name;\n+\t__u32 btf_id;\n+\tunsigned int found;\n+\t/* The BTF is the one of the running kernel */\n+\tbool live;\n+};\n+\n+typedef int (*btf_cb)(const char *owner, const struct btf *btf,\n+\t\t struct fmodret_filter *filter);\n+\n+#define ERRNO_NAME(e) { e, #e }\n+\n+/* The errors of the uapi and the kernel internal ones, which libc lacks */\n+static const struct {\n+\tint value;\n+\tconst char *name;\n+} errno_names[] = {\n+\tERRNO_NAME(EPERM),\n+\tERRNO_NAME(ENOENT),\n+\tERRNO_NAME(ESRCH),\n+\tERRNO_NAME(EINTR),\n+\tERRNO_NAME(EIO),\n+\tERRNO_NAME(ENXIO),\n+\tERRNO_NAME(E2BIG),\n+\tERRNO_NAME(ENOEXEC),\n+\tERRNO_NAME(EBADF),\n+\tERRNO_NAME(ECHILD),\n+\tERRNO_NAME(EAGAIN),\n+\tERRNO_NAME(ENOMEM),\n+\tERRNO_NAME(EACCES),\n+\tERRNO_NAME(EFAULT),\n+\tERRNO_NAME(ENOTBLK),\n+\tERRNO_NAME(EBUSY),\n+\tERRNO_NAME(EEXIST),\n+\tERRNO_NAME(EXDEV),\n+\tERRNO_NAME(ENODEV),\n+\tERRNO_NAME(ENOTDIR),\n+\tERRNO_NAME(EISDIR),\n+\tERRNO_NAME(EINVAL),\n+\tERRNO_NAME(ENFILE),\n+\tERRNO_NAME(EMFILE),\n+\tERRNO_NAME(ENOTTY),\n+\tERRNO_NAME(ETXTBSY),\n+\tERRNO_NAME(EFBIG),\n+\tERRNO_NAME(ENOSPC),\n+\tERRNO_NAME(ESPIPE),\n+\tERRNO_NAME(EROFS),\n+\tERRNO_NAME(EMLINK),\n+\tERRNO_NAME(EPIPE),\n+\tERRNO_NAME(EDOM),\n+\tERRNO_NAME(ERANGE),\n+\tERRNO_NAME(EDEADLK),\n+#if defined(EDEADLOCK) \u0026\u0026 EDEADLOCK != EDEADLK\n+\tERRNO_NAME(EDEADLOCK),\n+#endif\n+\tERRNO_NAME(ENAMETOOLONG),\n+\tERRNO_NAME(ENOLCK),\n+\tERRNO_NAME(ENOSYS),\n+\tERRNO_NAME(ENOTEMPTY),\n+\tERRNO_NAME(ELOOP),\n+\tERRNO_NAME(ENOMSG),\n+\tERRNO_NAME(EIDRM),\n+\tERRNO_NAME(ECHRNG),\n+\tERRNO_NAME(EL2NSYNC),\n+\tERRNO_NAME(EL3HLT),\n+\tERRNO_NAME(EL3RST),\n+\tERRNO_NAME(ELNRNG),\n+\tERRNO_NAME(EUNATCH),\n+\tERRNO_NAME(ENOCSI),\n+\tERRNO_NAME(EL2HLT),\n+\tERRNO_NAME(EBADE),\n+\tERRNO_NAME(EBADR),\n+\tERRNO_NAME(EXFULL),\n+\tERRNO_NAME(ENOANO),\n+\tERRNO_NAME(EBADRQC),\n+\tERRNO_NAME(EBADSLT),\n+\tERRNO_NAME(EBFONT),\n+\tERRNO_NAME(ENOSTR),\n+\tERRNO_NAME(ENODATA),\n+\tERRNO_NAME(ETIME),\n+\tERRNO_NAME(ENOSR),\n+\tERRNO_NAME(ENONET),\n+\tERRNO_NAME(ENOPKG),\n+\tERRNO_NAME(EREMOTE),\n+\tERRNO_NAME(ENOLINK),\n+\tERRNO_NAME(EADV),\n+\tERRNO_NAME(ESRMNT),\n+\tERRNO_NAME(ECOMM),\n+\tERRNO_NAME(EPROTO),\n+\tERRNO_NAME(EMULTIHOP),\n+\tERRNO_NAME(EDOTDOT),\n+\tERRNO_NAME(EBADMSG),\n+\tERRNO_NAME(EOVERFLOW),\n+\tERRNO_NAME(ENOTUNIQ),\n+\tERRNO_NAME(EBADFD),\n+\tERRNO_NAME(EREMCHG),\n+\tERRNO_NAME(ELIBACC),\n+\tERRNO_NAME(ELIBBAD),\n+\tERRNO_NAME(ELIBSCN),\n+\tERRNO_NAME(ELIBMAX),\n+\tERRNO_NAME(ELIBEXEC),\n+\tERRNO_NAME(EILSEQ),\n+\tERRNO_NAME(ERESTART),\n+\tERRNO_NAME(ESTRPIPE),\n+\tERRNO_NAME(EUSERS),\n+\tERRNO_NAME(ENOTSOCK),\n+\tERRNO_NAME(EDESTADDRREQ),\n+\tERRNO_NAME(EMSGSIZE),\n+\tERRNO_NAME(EPROTOTYPE),\n+\tERRNO_NAME(ENOPROTOOPT),\n+\tERRNO_NAME(EPROTONOSUPPORT),\n+\tERRNO_NAME(ESOCKTNOSUPPORT),\n+\tERRNO_NAME(EOPNOTSUPP),\n+\tERRNO_NAME(EPFNOSUPPORT),\n+\tERRNO_NAME(EAFNOSUPPORT),\n+\tERRNO_NAME(EADDRINUSE),\n+\tERRNO_NAME(EADDRNOTAVAIL),\n+\tERRNO_NAME(ENETDOWN),\n+\tERRNO_NAME(ENETUNREACH),\n+\tERRNO_NAME(ENETRESET),\n+\tERRNO_NAME(ECONNABORTED),\n+\tERRNO_NAME(ECONNRESET),\n+\tERRNO_NAME(ENOBUFS),\n+\tERRNO_NAME(EISCONN),\n+\tERRNO_NAME(ENOTCONN),\n+\tERRNO_NAME(ESHUTDOWN),\n+\tERRNO_NAME(ETOOMANYREFS),\n+\tERRNO_NAME(ETIMEDOUT),\n+\tERRNO_NAME(ECONNREFUSED),\n+\tERRNO_NAME(EHOSTDOWN),\n+\tERRNO_NAME(EHOSTUNREACH),\n+\tERRNO_NAME(EALREADY),\n+\tERRNO_NAME(EINPROGRESS),\n+\tERRNO_NAME(ESTALE),\n+\tERRNO_NAME(EUCLEAN),\n+\tERRNO_NAME(ENOTNAM),\n+\tERRNO_NAME(ENAVAIL),\n+\tERRNO_NAME(EISNAM),\n+\tERRNO_NAME(EREMOTEIO),\n+\tERRNO_NAME(EDQUOT),\n+\tERRNO_NAME(ENOMEDIUM),\n+\tERRNO_NAME(EMEDIUMTYPE),\n+\tERRNO_NAME(ECANCELED),\n+\tERRNO_NAME(ENOKEY),\n+\tERRNO_NAME(EKEYEXPIRED),\n+\tERRNO_NAME(EKEYREVOKED),\n+\tERRNO_NAME(EKEYREJECTED),\n+\tERRNO_NAME(EOWNERDEAD),\n+\tERRNO_NAME(ENOTRECOVERABLE),\n+\tERRNO_NAME(ERFKILL),\n+\tERRNO_NAME(EHWPOISON),\n+#ifdef EFTYPE\n+\tERRNO_NAME(EFTYPE),\n+#endif\n+\t{ 512, \"ERESTARTSYS\" },\n+\t{ 513, \"ERESTARTNOINTR\" },\n+\t{ 514, \"ERESTARTNOHAND\" },\n+\t{ 515, \"ENOIOCTLCMD\" },\n+\t{ 516, \"ERESTART_RESTARTBLOCK\" },\n+\t{ 517, \"EPROBE_DEFER\" },\n+\t{ 518, \"EOPENSTALE\" },\n+\t{ 519, \"ENOPARAM\" },\n+\t{ 521, \"EBADHANDLE\" },\n+\t{ 522, \"ENOTSYNC\" },\n+\t{ 523, \"EBADCOOKIE\" },\n+\t{ 524, \"ENOTSUPP\" },\n+\t{ 525, \"ETOOSMALL\" },\n+\t{ 526, \"ESERVERFAULT\" },\n+\t{ 527, \"EBADTYPE\" },\n+\t{ 528, \"EJUKEBOX\" },\n+\t{ 529, \"EIOCBQUEUED\" },\n+\t{ 530, \"ERECALLCONFLICT\" },\n+\t{ 531, \"ENOGRACE\" },\n+};\n+\n+static const char *errno_name(long long value)\n+{\n+\tsize_t i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(errno_names); i++)\n+\t\tif (-(long long)errno_names[i].value == value)\n+\t\t\treturn errno_names[i].name;\n+\treturn NULL;\n+}\n+\n+static int cmp_entry(const void *a, const void *b)\n+{\n+\tconst struct fmodret_entry *x = a, *y = b;\n+\n+\treturn (x-\u003ebtf_id \u003e y-\u003ebtf_id) - (x-\u003ebtf_id \u003c y-\u003ebtf_id);\n+}\n+\n+/* The first type ID which belongs to BTF and not to its base */\n+static __u32 first_own_type(const struct btf *btf)\n+{\n+\tconst struct btf *base = btf__base_btf(btf);\n+\n+\treturn base ? btf__type_cnt(base) : 1;\n+}\n+\n+/*\n+ * Return the function which T tags with decl tag NAME, or 0. A NAME which ends\n+ * with ':' matches the tags which start with it.\n+ */\n+static __u32 tag_func(const struct btf *btf, const struct btf_type *t, const char *name)\n+{\n+\tsize_t len = strlen(name);\n+\tconst char *tag;\n+\n+\tif (!btf_is_decl_tag(t) || btf_decl_tag(t)-\u003ecomponent_idx != -1)\n+\t\treturn 0;\n+\ttag = btf__name_by_offset(btf, t-\u003ename_off);\n+\tif (strncmp(tag, name, len) || (name[len - 1] != ':' \u0026\u0026 tag[len]))\n+\t\treturn 0;\n+\tif (!btf_is_func(btf__type_by_id(btf, t-\u003etype)))\n+\t\treturn 0;\n+\treturn t-\u003etype;\n+}\n+\n+/* Return the fmod_ret targets of BTF, sorted by BTF ID */\n+static int collect_targets(const struct btf *btf, struct fmodret_entry **targets,\n+\t\t\t size_t *target_cnt)\n+{\n+\t__u32 id, func_id, start_id = first_own_type(btf);\n+\t__u32 nr_types = btf__type_cnt(btf);\n+\tstruct fmodret_entry *entries = NULL, *entry, *tmp;\n+\tsize_t cnt = 0, capacity = 0;\n+\n+\tfor (id = start_id; id \u003c nr_types; id++) {\n+\t\tfunc_id = tag_func(btf, btf__type_by_id(btf, id), TAG_FMODRET);\n+\t\tif (!func_id)\n+\t\t\tcontinue;\n+\n+\t\tif (cnt == capacity) {\n+\t\t\tcapacity = capacity ? 2 * capacity : 64;\n+\t\t\ttmp = realloc(entries, capacity * sizeof(*entries));\n+\t\t\tif (!tmp) {\n+\t\t\t\tfree(entries);\n+\t\t\t\treturn -ENOMEM;\n+\t\t\t}\n+\t\t\tentries = tmp;\n+\t\t}\n+\t\tentries[cnt].btf_id = func_id;\n+\t\tentries[cnt].sleepable = false;\n+\t\tcnt++;\n+\t}\n+\tif (cnt)\n+\t\tqsort(entries, cnt, sizeof(*entries), cmp_entry);\n+\n+\t/* The other tags mark functions which are already targets */\n+\tfor (id = start_id; cnt \u0026\u0026 id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\t\tstruct fmodret_entry key = {};\n+\n+\t\tkey.btf_id = tag_func(btf, t, TAG_FMODRET_SLEEPABLE);\n+\t\tif (key.btf_id) {\n+\t\t\tentry = bsearch(\u0026key, entries, cnt, sizeof(*entries), cmp_entry);\n+\t\t\tif (entry)\n+\t\t\t\tentry-\u003esleepable = true;\n+\t\t}\n+\t}\n+\n+\t*targets = entries;\n+\t*target_cnt = cnt;\n+\treturn 0;\n+}\n+\n+/* Fill ENTRY from the decl tags of function FUNC_ID */\n+static void func_entry(const struct btf *btf, __u32 func_id, struct fmodret_entry *entry)\n+{\n+\t__u32 id, nr_types = btf__type_cnt(btf);\n+\n+\tmemset(entry, 0, sizeof(*entry));\n+\tentry-\u003ebtf_id = func_id;\n+\tfor (id = first_own_type(btf); id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\n+\t\tif (tag_func(btf, t, TAG_FMODRET) == func_id)\n+\t\t\tentry-\u003etarget = true;\n+\t\telse if (tag_func(btf, t, TAG_FMODRET_SLEEPABLE) == func_id)\n+\t\t\tentry-\u003esleepable = true;\n+\t\telse if (tag_func(btf, t, TAG_FMODRET_RETVAL) == func_id) {\n+\t\t\tentry-\u003eretval_dup = entry-\u003eretval;\n+\t\t\tentry-\u003eretval = btf__name_by_offset(btf, t-\u003ename_off) +\n+\t\t\t\t\tstrlen(TAG_FMODRET_RETVAL);\n+\t\t}\n+\t}\n+}\n+\n+static void list_target(const char *owner, const struct btf *btf,\n+\t\t\tconst struct fmodret_entry *entry)\n+{\n+\tconst struct btf_type *t = btf__type_by_id(btf, entry-\u003ebtf_id);\n+\tconst char *name = btf__name_by_offset(btf, t-\u003ename_off);\n+\n+\tif (json_output) {\n+\t\tjsonw_start_object(json_wtr);\n+\t\tjsonw_uint_field(json_wtr, \"btf_id\", entry-\u003ebtf_id);\n+\t\tjsonw_string_field(json_wtr, \"name\", name);\n+\t\tjsonw_string_field(json_wtr, \"owner\", owner);\n+\t\tjsonw_bool_field(json_wtr, \"sleepable\", entry-\u003esleepable);\n+\t\tjsonw_end_object(json_wtr);\n+\t} else {\n+\t\tprintf(\"%u %s (%s)%s\\n\", entry-\u003ebtf_id, name, owner,\n+\t\t entry-\u003esleepable ? \" sleepable\" : \"\");\n+\t}\n+}\n+\n+static int list_btf(const char *owner, const struct btf *btf,\n+\t\t struct fmodret_filter *filter)\n+{\n+\tstruct fmodret_entry *targets = NULL;\n+\tsize_t cnt = 0, i;\n+\tint err;\n+\n+\terr = collect_targets(btf, \u0026targets, \u0026cnt);\n+\tif (err) {\n+\t\tp_err(\"failed to collect fmod_ret targets of %s: %s\", owner,\n+\t\t strerror(-err));\n+\t\treturn err;\n+\t}\n+\tfor (i = 0; i \u003c cnt; i++)\n+\t\tlist_target(owner, btf, \u0026targets[i]);\n+\tfree(targets);\n+\treturn 0;\n+}\n+\n+struct error_injection {\n+\tchar *name;\n+\tchar *module;\n+\tchar *type;\n+};\n+\n+static struct error_injection *error_injections;\n+static size_t error_injection_cnt;\n+\n+/*\n+ * Read the error injection list of the running kernel. Its lines are\n+ * \"FUNC\\tTYPE\" and \"FUNC [MODULE]\\tTYPE\". Without debugfs the list is empty.\n+ */\n+static void read_error_injection_list(void)\n+{\n+\tstruct error_injection *tmp;\n+\tchar *line = NULL;\n+\tsize_t len = 0;\n+\tFILE *f;\n+\n+\tf = fopen(ERROR_INJECTION_LIST, \"r\");\n+\tif (!f)\n+\t\treturn;\n+\twhile (getline(\u0026line, \u0026len, f) \u003e 0) {\n+\t\tchar *name, *module = NULL, *type, *save;\n+\n+\t\t/* \"NAME [MODULE]\\tTYPE\" or \"NAME\\tTYPE\" */\n+\t\tname = strtok_r(line, \" \\t\\n\", \u0026save);\n+\t\ttype = strtok_r(NULL, \" \\t\\n\", \u0026save);\n+\t\tif (type \u0026\u0026 type[0] == '[') {\n+\t\t\tmodule = type + 1;\n+\t\t\tmodule[strcspn(module, \"]\")] = '\\0';\n+\t\t\ttype = strtok_r(NULL, \" \\t\\n\", \u0026save);\n+\t\t}\n+\t\tif (!name || !type)\n+\t\t\tcontinue;\n+\t\ttmp = realloc(error_injections,\n+\t\t\t (error_injection_cnt + 1) * sizeof(*error_injections));\n+\t\tif (!tmp)\n+\t\t\tbreak;\n+\t\terror_injections = tmp;\n+\t\ttmp = \u0026error_injections[error_injection_cnt];\n+\t\ttmp-\u003ename = strdup(name);\n+\t\ttmp-\u003emodule = strdup(module ?: \"\");\n+\t\ttmp-\u003etype = strdup(type);\n+\t\tif (!tmp-\u003ename || !tmp-\u003emodule || !tmp-\u003etype) {\n+\t\t\tfree(tmp-\u003ename);\n+\t\t\tfree(tmp-\u003emodule);\n+\t\t\tfree(tmp-\u003etype);\n+\t\t\tbreak;\n+\t\t}\n+\t\terror_injection_cnt++;\n+\t}\n+\tfree(line);\n+\tfclose(f);\n+}\n+\n+static void free_error_injection_list(void)\n+{\n+\tsize_t i;\n+\n+\tfor (i = 0; i \u003c error_injection_cnt; i++) {\n+\t\tfree(error_injections[i].name);\n+\t\tfree(error_injections[i].module);\n+\t\tfree(error_injections[i].type);\n+\t}\n+\tfree(error_injections);\n+\terror_injections = NULL;\n+\terror_injection_cnt = 0;\n+}\n+\n+/* Return the error injection type of function NAME of OWNER, or NULL */\n+static const char *error_injection_type(const char *name, const char *owner)\n+{\n+\tconst char *module = strcmp(owner, \"vmlinux\") ? owner : \"\";\n+\tsize_t i;\n+\n+\tfor (i = 0; i \u003c error_injection_cnt; i++)\n+\t\tif (!strcmp(error_injections[i].name, name) \u0026\u0026\n+\t\t !strcmp(error_injections[i].module, module))\n+\t\t\treturn error_injections[i].type;\n+\treturn NULL;\n+}\n+\n+static void __printf(2, 0) prototype_printf(void *ctx, const char *fmt, va_list args)\n+{\n+\tvfprintf(ctx, fmt, args);\n+}\n+\n+/* Return the declaration of function FUNC_ID, or NULL */\n+static char *func_prototype(const struct btf *btf, __u32 func_id)\n+{\n+\tconst struct btf_type *t = btf__type_by_id(btf, func_id);\n+\tLIBBPF_OPTS(btf_dump_emit_type_decl_opts, opts,\n+\t\t.field_name = btf__name_by_offset(btf, t-\u003ename_off),\n+\t);\n+\tstruct btf_dump *d;\n+\tchar *buf = NULL;\n+\tsize_t size = 0;\n+\tFILE *f;\n+\tint err;\n+\n+\tf = open_memstream(\u0026buf, \u0026size);\n+\tif (!f)\n+\t\treturn NULL;\n+\td = btf_dump__new(btf, prototype_printf, f, NULL);\n+\terr = libbpf_get_error(d);\n+\tif (!err) {\n+\t\terr = btf_dump__emit_type_decl(d, t-\u003etype, \u0026opts);\n+\t\tbtf_dump__free(d);\n+\t}\n+\tfclose(f);\n+\tif (err) {\n+\t\tfree(buf);\n+\t\treturn NULL;\n+\t}\n+\treturn buf;\n+}\n+\n+/* Show 0 and the values of a \"bpf_fmodret_retval:\" tag as a JSON array */\n+static void show_retval_json(const char *retval)\n+{\n+\tconst char *p = retval;\n+\tchar *end;\n+\n+\tjsonw_name(json_wtr, \"retval\");\n+\tjsonw_start_array(json_wtr);\n+\tjsonw_int(json_wtr, 0);\n+\twhile (*p) {\n+\t\tlong long value;\n+\n+\t\terrno = 0;\n+\t\tvalue = strtoll(p, \u0026end, 10);\n+\t\tif (end == p || errno)\n+\t\t\tbreak;\n+\t\tjsonw_int(json_wtr, value);\n+\t\tp = *end == ',' ? end + 1 : end;\n+\t}\n+\tjsonw_end_array(json_wtr);\n+}\n+\n+/*\n+ * Check the \"bpf_fmodret_retval:\" tag of FUNC_ID as the verifier does: negative\n+ * decimal numbers separated by commas, which the integer or enum of 4 or 8 bytes\n+ * that the function returns can hold. Return why it is invalid, or NULL.\n+ */\n+static const char *retval_invalid(const struct btf *btf, __u32 func_id,\n+\t\t\t\t const char *retval)\n+{\n+\tconst struct btf_type *t = btf__type_by_id(btf, func_id);\n+\tconst char *p = retval;\n+\tint ret_id;\n+\tchar *end;\n+\n+\tt = btf__type_by_id(btf, t-\u003etype);\n+\tret_id = t ? btf__resolve_type(btf, t-\u003etype) : -EINVAL;\n+\tt = ret_id \u003e 0 ? btf__type_by_id(btf, ret_id) : NULL;\n+\tif (!t || !(btf_is_int(t) || btf_is_any_enum(t)) || (t-\u003esize != 4 \u0026\u0026 t-\u003esize != 8))\n+\t\treturn \"the function does not return an integer of 4 or 8 bytes\";\n+\tfor (;;) {\n+\t\tlong long value;\n+\n+\t\tif (*p != '-')\n+\t\t\treturn \"malformed tag\";\n+\t\terrno = 0;\n+\t\tvalue = strtoll(p, \u0026end, 10);\n+\t\tif (end == p || errno || value \u003e= 0)\n+\t\t\treturn \"malformed tag\";\n+\t\tif (t-\u003esize == 4 \u0026\u0026 value \u003c INT_MIN)\n+\t\t\treturn \"a value does not fit the return type\";\n+\t\tif (!*end)\n+\t\t\treturn NULL;\n+\t\tif (*end != ',')\n+\t\t\treturn \"malformed tag\";\n+\t\tp = end + 1;\n+\t}\n+}\n+\n+/* Show the values of a \"bpf_fmodret_retval:\" tag with the names of errors */\n+static void show_retval_plain(const char *retval)\n+{\n+\tconst char *p = retval, *name;\n+\tchar *end;\n+\n+\tprintf(\"\\tretval: 0\");\n+\twhile (*p) {\n+\t\tlong long value;\n+\n+\t\terrno = 0;\n+\t\tvalue = strtoll(p, \u0026end, 10);\n+\t\tif (end == p || errno)\n+\t\t\tbreak;\n+\t\tprintf(\", %lld\", value);\n+\t\tname = errno_name(value);\n+\t\tif (name)\n+\t\t\tprintf(\" (%s)\", name);\n+\t\tp = *end == ',' ? end + 1 : end;\n+\t}\n+\tprintf(\"\\n\");\n+}\n+\n+static int show_function(const char *owner, const struct btf *btf,\n+\t\t\t const struct fmodret_entry *entry,\n+\t\t\t struct fmodret_filter *filter)\n+{\n+\tconst struct btf_type *t = btf__type_by_id(btf, entry-\u003ebtf_id);\n+\tconst char *name = btf__name_by_offset(btf, t-\u003ename_off);\n+\tconst char *injection = NULL, *invalid = NULL;\n+\tchar *prototype;\n+\n+\t/* The verifier rejects programs on a function with an invalid tag */\n+\tif (entry-\u003etarget \u0026\u0026 entry-\u003eretval_dup)\n+\t\tinvalid = \"more than one tag\";\n+\telse if (entry-\u003etarget \u0026\u0026 entry-\u003eretval)\n+\t\tinvalid = retval_invalid(btf, entry-\u003ebtf_id, entry-\u003eretval);\n+\tprototype = func_prototype(btf, entry-\u003ebtf_id);\n+\tif (!prototype) {\n+\t\tp_err(\"failed to print the prototype of %s\", name);\n+\t\treturn -EINVAL;\n+\t}\n+\tif (filter-\u003elive)\n+\t\tinjection = error_injection_type(name, owner);\n+\n+\tif (json_output) {\n+\t\tjsonw_start_object(json_wtr);\n+\t\tjsonw_uint_field(json_wtr, \"btf_id\", entry-\u003ebtf_id);\n+\t\tjsonw_string_field(json_wtr, \"name\", name);\n+\t\tjsonw_string_field(json_wtr, \"owner\", owner);\n+\t\tjsonw_string_field(json_wtr, \"prototype\", prototype);\n+\t\tjsonw_string_field(json_wtr, \"target\", !entry-\u003etarget ? \"none\" :\n+\t\t\t\t entry-\u003eretval ? \"found\" : \"declared\");\n+\t\tjsonw_bool_field(json_wtr, \"sleepable\", entry-\u003etarget \u0026\u0026 entry-\u003esleepable);\n+\t\tif (invalid)\n+\t\t\tjsonw_string_field(json_wtr, \"retval_error\", invalid);\n+\t\telse if (entry-\u003etarget \u0026\u0026 entry-\u003eretval)\n+\t\t\tshow_retval_json(entry-\u003eretval);\n+\t\tif (injection)\n+\t\t\tjsonw_string_field(json_wtr, \"error_injection\", injection);\n+\t\tjsonw_end_object(json_wtr);\n+\t} else {\n+\t\tprintf(\"%u %s (%s)\\n\", entry-\u003ebtf_id, name, owner);\n+\t\tprintf(\"\\t%s\\n\", prototype);\n+\t\tif (entry-\u003etarget) {\n+\t\t\tprintf(\"\\ttarget: %s\\n\", entry-\u003eretval ? \"found by the kernel build\" :\n+\t\t\t \"declared\");\n+\t\t\tprintf(\"\\tsleepable: %s\\n\", entry-\u003esleepable ? \"yes\" : \"no\");\n+\t\t\tif (invalid)\n+\t\t\t\tprintf(\"\\tretval: invalid, %s\\n\", invalid);\n+\t\t\telse if (entry-\u003eretval)\n+\t\t\t\tshow_retval_plain(entry-\u003eretval);\n+\t\t\telse\n+\t\t\t\tprintf(\"\\tretval: any\\n\");\n+\t\t} else {\n+\t\t\tprintf(\"\\ttarget: not tagged, the verifier may accept it by other rules\\n\");\n+\t\t}\n+\t\tif (injection)\n+\t\t\tprintf(\"\\terror injection: %s\\n\", injection);\n+\t}\n+\tfree(prototype);\n+\tfilter-\u003efound++;\n+\treturn 0;\n+}\n+\n+static int show_btf(const char *owner, const struct btf *btf,\n+\t\t struct fmodret_filter *filter)\n+{\n+\t__u32 id, nr_types = btf__type_cnt(btf);\n+\tstruct fmodret_entry entry;\n+\tint err;\n+\n+\tif (filter-\u003ebtf_id) {\n+\t\tif (filter-\u003ebtf_id \u003c first_own_type(btf) || filter-\u003ebtf_id \u003e= nr_types ||\n+\t\t !btf_is_func(btf__type_by_id(btf, filter-\u003ebtf_id)))\n+\t\t\treturn 0;\n+\t\tfunc_entry(btf, filter-\u003ebtf_id, \u0026entry);\n+\t\treturn show_function(owner, btf, \u0026entry, filter);\n+\t}\n+\n+\tfor (id = first_own_type(btf); id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\n+\t\tif (!btf_is_func(t) ||\n+\t\t strcmp(btf__name_by_offset(btf, t-\u003ename_off), filter-\u003ename))\n+\t\t\tcontinue;\n+\t\tfunc_entry(btf, id, \u0026entry);\n+\t\terr = show_function(owner, btf, \u0026entry, filter);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\treturn 0;\n+}\n+\n+static int filter_module(const struct dirent *entry)\n+{\n+\tsize_t len = strlen(entry-\u003ed_name);\n+\n+\t/* Skip \".\", \"..\", vmlinux and the \".inline\" companions of modules */\n+\treturn entry-\u003ed_name[0] != '.' \u0026\u0026 strcmp(entry-\u003ed_name, \"vmlinux\") \u0026\u0026\n+\t !(len \u003e 7 \u0026\u0026 !strcmp(entry-\u003ed_name + len - 7, \".inline\"));\n+}\n+\n+static int walk_modules(struct btf *vmlinux, struct fmodret_filter *filter, btf_cb cb)\n+{\n+\tstruct dirent **modules = NULL;\n+\tint i, nr_modules, err = 0;\n+\n+\tif (filter-\u003emodule) {\n+\t\tstruct btf *btf;\n+\n+\t\tbtf = btf__load_module_btf(filter-\u003emodule, vmlinux);\n+\t\terr = libbpf_get_error(btf);\n+\t\tif (err) {\n+\t\t\tp_err(\"failed to load BTF of module %s: %s\", filter-\u003emodule,\n+\t\t\t strerror(-err));\n+\t\t\treturn err;\n+\t\t}\n+\t\terr = cb(filter-\u003emodule, btf, filter);\n+\t\tbtf__free(btf);\n+\t\treturn err;\n+\t}\n+\n+\tnr_modules = scandir(SYS_BTF_DIR, \u0026modules, filter_module, alphasort);\n+\tif (nr_modules \u003c 0) {\n+\t\t/* There is no module directory without module BTF, which is fine */\n+\t\tif (errno == ENOENT)\n+\t\t\treturn 0;\n+\t\terr = -errno;\n+\t\tp_err(\"failed to read %s: %s\", SYS_BTF_DIR, strerror(errno));\n+\t\treturn err;\n+\t}\n+\tfor (i = 0; i \u003c nr_modules; i++) {\n+\t\tconst char *name = modules[i]-\u003ed_name;\n+\t\tstruct btf *btf;\n+\n+\t\tbtf = btf__load_module_btf(name, vmlinux);\n+\t\terr = libbpf_get_error(btf);\n+\t\tif (err == -ENOENT) {\n+\t\t\t/* The module was unloaded meanwhile */\n+\t\t\terr = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (err) {\n+\t\t\tp_err(\"failed to load BTF of module %s: %s\", name, strerror(-err));\n+\t\t\tbreak;\n+\t\t}\n+\t\terr = cb(name, btf, filter);\n+\t\tbtf__free(btf);\n+\t\tif (err)\n+\t\t\tbreak;\n+\t}\n+\tfor (i = 0; i \u003c nr_modules; i++)\n+\t\tfree(modules[i]);\n+\tfree(modules);\n+\treturn err;\n+}\n+\n+static int walk_live_btf(struct fmodret_filter *filter, btf_cb cb)\n+{\n+\tstruct btf *vmlinux;\n+\tint err = 0;\n+\n+\tvmlinux = btf__load_vmlinux_btf();\n+\terr = libbpf_get_error(vmlinux);\n+\tif (err) {\n+\t\tp_err(\"failed to load vmlinux BTF: %s\", strerror(-err));\n+\t\treturn err;\n+\t}\n+\tif (!filter-\u003emodule)\n+\t\terr = cb(\"vmlinux\", vmlinux, filter);\n+\tif (!err \u0026\u0026 !filter-\u003evmlinux)\n+\t\terr = walk_modules(vmlinux, filter, cb);\n+\tbtf__free(vmlinux);\n+\treturn err;\n+}\n+\n+static int walk_file_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)\n+{\n+\tstruct btf *btf;\n+\tint err;\n+\n+\tbtf = base_btf ? btf__parse_split(path, base_btf) : btf__parse(path, NULL);\n+\terr = libbpf_get_error(btf);\n+\tif (err) {\n+\t\tp_err(\"failed to parse BTF from %s: %s\", path, strerror(-err));\n+\t\treturn err;\n+\t}\n+\terr = cb(path, btf, filter);\n+\tbtf__free(btf);\n+\treturn err;\n+}\n+\n+/* Parse the optional OBJECT := { file FILE | vmlinux | module NAME } */\n+static int parse_object(int argc, char **argv, const char **path,\n+\t\t\tstruct fmodret_filter *filter)\n+{\n+\tif (!argc)\n+\t\treturn 0;\n+\tif (is_prefix(*argv, \"vmlinux\")) {\n+\t\tfilter-\u003evmlinux = true;\n+\t\tNEXT_ARG();\n+\t} else if (is_prefix(*argv, \"file\")) {\n+\t\tNEXT_ARG();\n+\t\tif (!REQ_ARGS(1))\n+\t\t\treturn -1;\n+\t\t*path = GET_ARG();\n+\t} else if (is_prefix(*argv, \"module\")) {\n+\t\tNEXT_ARG();\n+\t\tif (!REQ_ARGS(1))\n+\t\t\treturn -1;\n+\t\tfilter-\u003emodule = GET_ARG();\n+\t}\n+\tif (argc)\n+\t\treturn BAD_ARG();\n+\treturn 0;\n+}\n+\n+static int walk_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)\n+{\n+\treturn path ? walk_file_btf(path, filter, cb) : walk_live_btf(filter, cb);\n+}\n+\n+static int do_list(int argc, char **argv)\n+{\n+\tstruct fmodret_filter filter = {};\n+\tconst char *path = NULL;\n+\tint err;\n+\n+\tif (parse_object(argc, argv, \u0026path, \u0026filter))\n+\t\treturn -1;\n+\n+\tif (json_output)\n+\t\tjsonw_start_array(json_wtr);\n+\terr = walk_btf(path, \u0026filter, list_btf);\n+\tif (json_output)\n+\t\tjsonw_end_array(json_wtr);\n+\treturn err;\n+}\n+\n+static int do_show(int argc, char **argv)\n+{\n+\tstruct fmodret_filter filter = {};\n+\tconst char *path = NULL;\n+\tunsigned long id;\n+\tchar *endptr;\n+\tint err;\n+\n+\tif (!REQ_ARGS(2))\n+\t\treturn -1;\n+\tif (is_prefix(*argv, \"name\")) {\n+\t\tNEXT_ARG();\n+\t\tfilter.name = GET_ARG();\n+\t} else if (is_prefix(*argv, \"id\")) {\n+\t\tNEXT_ARG();\n+\t\terrno = 0;\n+\t\tid = strtoul(*argv, \u0026endptr, 0);\n+\t\tif (errno || *endptr || !id || id \u003e UINT_MAX || **argv == '-') {\n+\t\t\tp_err(\"can't parse %s as BTF ID\", *argv);\n+\t\t\treturn -1;\n+\t\t}\n+\t\tfilter.btf_id = id;\n+\t\tNEXT_ARG();\n+\t} else {\n+\t\tp_err(\"expected 'name' or 'id', got: '%s'\", *argv);\n+\t\treturn -1;\n+\t}\n+\tif (parse_object(argc, argv, \u0026path, \u0026filter))\n+\t\treturn -1;\n+\t/* The BTF IDs of different modules overlap */\n+\tif (filter.btf_id \u0026\u0026 !path \u0026\u0026 !filter.module)\n+\t\tfilter.vmlinux = true;\n+\n+\tfilter.live = !path;\n+\tif (filter.live)\n+\t\tread_error_injection_list();\n+\t/* A BTF ID gives one function, a name an array of them */\n+\tif (json_output \u0026\u0026 !filter.btf_id)\n+\t\tjsonw_start_array(json_wtr);\n+\terr = walk_btf(path, \u0026filter, show_btf);\n+\tif (!err \u0026\u0026 !filter.found) {\n+\t\tif (filter.name)\n+\t\t\tp_err(\"no function named %s\", filter.name);\n+\t\telse\n+\t\t\tp_err(\"no function with BTF ID %u\", filter.btf_id);\n+\t\terr = -ENOENT;\n+\t}\n+\tif (json_output \u0026\u0026 !filter.btf_id)\n+\t\tjsonw_end_array(json_wtr);\n+\tfree_error_injection_list();\n+\treturn err;\n+}\n+\n+static int do_help(int argc, char **argv)\n+{\n+\tif (json_output) {\n+\t\tjsonw_null(json_wtr);\n+\t\treturn 0;\n+\t}\n+\n+\tfprintf(stderr,\n+\t\t\"Usage: %1$s %2$s list [OBJECT]\\n\"\n+\t\t\" %1$s %2$s show FUNC [OBJECT]\\n\"\n+\t\t\" %1$s %2$s help\\n\"\n+\t\t\"\\n\"\n+\t\t\" FUNC := { name NAME | id BTF_ID }\\n\"\n+\t\t\" OBJECT := { file FILE | vmlinux | module NAME }\\n\"\n+\t\t\" \" HELP_SPEC_OPTIONS \" |\\n\"\n+\t\t\" {-B|--base-btf} FILE }\\n\"\n+\t\t\"\\n\",\n+\t\tbin_name, \"fmodret\");\n+\treturn 0;\n+}\n+\n+static const struct cmd cmds[] = {\n+\t{ \"list\",\tdo_list },\n+\t{ \"show\",\tdo_show },\n+\t{ \"help\",\tdo_help },\n+\t{ 0 }\n+};\n+\n+int do_fmodret(int argc, char **argv)\n+{\n+\treturn cmd_select(cmds, argc, argv, do_help);\n+}\ndiff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c\nindex d15fec71b64eec..e2a7571b097c7e 100644\n--- a/tools/bpf/bpftool/main.c\n+++ b/tools/bpf/bpftool/main.c\n@@ -65,7 +65,7 @@ static int do_help(int argc, char **argv)\n \t\t\" %s batch file FILE\\n\"\n \t\t\" %s version\\n\"\n \t\t\"\\n\"\n-\t\t\" OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | token }\\n\"\n+\t\t\" OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | fmodret | token }\\n\"\n \t\t\" \" HELP_SPEC_OPTIONS \" |\\n\"\n \t\t\" {-V|--version} }\\n\"\n \t\t\"\",\n@@ -91,6 +91,7 @@ static const struct cmd commands[] = {\n \t{ \"gen\",\tdo_gen },\n \t{ \"struct_ops\",\tdo_struct_ops },\n \t{ \"iter\",\tdo_iter },\n+\t{ \"fmodret\",\tdo_fmodret },\n \t{ \"token\",\tdo_token },\n \t{ \"version\",\tdo_version },\n \t{ 0 }\ndiff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h\nindex b1eb4b5d991b8a..4d877b803b4765 100644\n--- a/tools/bpf/bpftool/main.h\n+++ b/tools/bpf/bpftool/main.h\n@@ -178,6 +178,7 @@ int do_tracelog(int argc, char **arg) __weak;\n int do_feature(int argc, char **argv) __weak;\n int do_struct_ops(int argc, char **argv) __weak;\n int do_iter(int argc, char **argv) __weak;\n+int do_fmodret(int argc, char **argv) __weak;\n int do_token(int argc, char **argv) __weak;\n \n int parse_u32_arg(int *argc, char ***argv, __u32 *val, const char *what);\ndiff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build\nindex 602e1508bf0922..af777afa451b27 100644\n--- a/tools/bpf/resolve_btfids/Build\n+++ b/tools/bpf/resolve_btfids/Build\n@@ -1,6 +1,7 @@\n hostprogs := resolve_btfids\n \n resolve_btfids-y += main.o\n+resolve_btfids-y += fmodret.o\n resolve_btfids-y += btf_colors.o\n resolve_btfids-y += rbtree.o\n resolve_btfids-y += zalloc.o\ndiff --git a/tools/bpf/resolve_btfids/fmodret.c b/tools/bpf/resolve_btfids/fmodret.c\nnew file mode 100644\nindex 00000000000000..df44b93eea734c\n--- /dev/null\n+++ b/tools/bpf/resolve_btfids/fmodret.c\n@@ -0,0 +1,1205 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/*\n+ * Read the records which the bpf_scan_ir compiler plugin writes to\n+ * .BTF_fmodret_candidates, see scripts/llvm-plugins/bpf_scan_ir.cpp, and find\n+ * the functions which can become fmod_ret targets. A record lists the errors\n+ * which clean paths of a function return, that is paths without side effects.\n+ *\n+ * A function is chosen if\n+ * - exactly one record describes its body, or one record of a strong\n+ * definition next to records of the weak ones it overrides,\n+ * - the record is of a strong definition or the symbol is still weak,\n+ * - the record lists at least one error,\n+ * - it has an ftrace call site and is not init, exit or noinstr code,\n+ * - its name belongs to exactly one ELF function and one BTF FUNC,\n+ * - it returns an integer of the size the record describes.\n+ *\n+ * The plugin records no errors for functions which return pointers yet.\n+ */\n+\n+#include \u003cerrno.h\u003e\n+#include \u003cgelf.h\u003e\n+#include \u003clibelf.h\u003e\n+#include \u003cstdbool.h\u003e\n+#include \u003cstdint.h\u003e\n+#include \u003cstdio.h\u003e\n+#include \u003cstdlib.h\u003e\n+#include \u003cstring.h\u003e\n+\n+#include \u003cbpf/btf.h\u003e\n+#include \u003clinux/kernel.h\u003e\n+\n+#include \"fmodret.h\"\n+#include \"../../../scripts/bpf/fmodret_metadata.h\"\n+\n+#define FMODRET_CANDIDATES_SECTION \".BTF_fmodret_candidates\"\n+#define NO_FUNCTION SIZE_MAX\n+\n+struct elf_function {\n+\tconst char *name;\n+\tGElf_Addr value;\n+\tGElf_Xword size;\n+\tElf64_Section shndx;\n+\tbool traceable;\n+\t/* The symbol is weak: no strong definition replaced it at the link */\n+\tbool weak_symbol;\n+\t/* The records which describe the body at this address */\n+\tunsigned int nr_strong, nr_weak;\n+\tsize_t strong, weak;\n+};\n+\n+struct elf_function_list {\n+\tstruct elf_function *functions;\n+\tsize_t nr;\n+\tsize_t capacity;\n+};\n+\n+struct fmodret_stats {\n+\tsize_t records, unmapped, with_values;\n+\tsize_t ambiguous_body, not_traceable, ambiguous_name;\n+\tsize_t overridden, no_btf, return_mismatch, added;\n+};\n+\n+static int find_section(Elf *elf, size_t shstrndx, const char *wanted,\n+\t\t\tElf_Scn **result)\n+{\n+\tElf_Scn *scn = NULL;\n+\n+\twhile ((scn = elf_nextscn(elf, scn))) {\n+\t\tGElf_Shdr shdr;\n+\t\tconst char *name;\n+\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tname = elf_strptr(elf, shstrndx, shdr.sh_name);\n+\t\tif (!name)\n+\t\t\treturn -1;\n+\t\tif (!strcmp(name, wanted)) {\n+\t\t\t*result = scn;\n+\t\t\treturn 0;\n+\t\t}\n+\t}\n+\n+\t*result = NULL;\n+\treturn 0;\n+}\n+\n+static int find_symbol(Elf *elf, const char *wanted, GElf_Sym *result,\n+\t\t bool *found)\n+{\n+\tElf_Scn *scn = NULL;\n+\n+\t*found = false;\n+\twhile ((scn = elf_nextscn(elf, scn))) {\n+\t\tGElf_Shdr shdr;\n+\t\tElf_Data *data;\n+\t\tsize_t nr_symbols;\n+\t\tsize_t i;\n+\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tif (shdr.sh_type != SHT_SYMTAB)\n+\t\t\tcontinue;\n+\t\tif (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)\n+\t\t\treturn -EINVAL;\n+\t\tdata = elf_getdata(scn, NULL);\n+\t\tif (!data || elf_getdata(scn, data))\n+\t\t\treturn -1;\n+\t\tnr_symbols = shdr.sh_size / shdr.sh_entsize;\n+\t\tfor (i = 0; i \u003c nr_symbols; i++) {\n+\t\t\tconst char *name;\n+\t\t\tGElf_Sym sym;\n+\n+\t\t\tif (!gelf_getsym(data, i, \u0026sym))\n+\t\t\t\treturn -1;\n+\t\t\tif (sym.st_shndx == SHN_UNDEF || !sym.st_name)\n+\t\t\t\tcontinue;\n+\t\t\tname = elf_strptr(elf, shdr.sh_link, sym.st_name);\n+\t\t\tif (!name)\n+\t\t\t\treturn -1;\n+\t\t\tif (strcmp(name, wanted))\n+\t\t\t\tcontinue;\n+\t\t\t*result = sym;\n+\t\t\t*found = true;\n+\t\t\treturn 0;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int section_for_address(Elf *elf, GElf_Addr address,\n+\t\t\t Elf64_Section *result)\n+{\n+\tElf_Scn *scn = NULL;\n+\n+\twhile ((scn = elf_nextscn(elf, scn))) {\n+\t\tGElf_Shdr shdr;\n+\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tif (!(shdr.sh_flags \u0026 SHF_EXECINSTR) || address \u003c shdr.sh_addr ||\n+\t\t address - shdr.sh_addr \u003e= shdr.sh_size)\n+\t\t\tcontinue;\n+\t\t*result = elf_ndxscn(scn);\n+\t\treturn 0;\n+\t}\n+\t*result = SHN_UNDEF;\n+\treturn 0;\n+}\n+\n+static bool relocation_offset(const GElf_Shdr *target, GElf_Addr offset,\n+\t\t\t size_t width, size_t *result)\n+{\n+\tif (target-\u003esh_size \u003c width)\n+\t\treturn false;\n+\tif (offset \u003e= target-\u003esh_addr \u0026\u0026\n+\t offset - target-\u003esh_addr \u003c= target-\u003esh_size - width) {\n+\t\t*result = offset - target-\u003esh_addr;\n+\t\treturn true;\n+\t}\n+\t/* ET_REL relocation offsets are relative to their target section. */\n+\tif (offset \u003c= target-\u003esh_size - width) {\n+\t\t*result = offset;\n+\t\treturn true;\n+\t}\n+\treturn false;\n+}\n+\n+static uint64_t read_elf_value(const unsigned char *p, size_t size,\n+\t\t\t unsigned int encoding)\n+{\n+\tuint64_t value = 0;\n+\tsize_t i;\n+\n+\tif (encoding == ELFDATA2LSB) {\n+\t\tfor (i = 0; i \u003c size; i++)\n+\t\t\tvalue |= (uint64_t)p[i] \u003c\u003c (8 * i);\n+\t} else {\n+\t\tfor (i = 0; i \u003c size; i++)\n+\t\t\tvalue = (value \u003c\u003c 8) | p[i];\n+\t}\n+\treturn value;\n+}\n+\n+static int copy_section_data(Elf_Scn *scn, const GElf_Shdr *shdr,\n+\t\t\t unsigned char **result)\n+{\n+\tElf_Data *data = NULL;\n+\tunsigned char *buf;\n+\n+\tbuf = calloc(1, shdr-\u003esh_size ?: 1);\n+\tif (!buf)\n+\t\treturn -ENOMEM;\n+\twhile ((data = elf_getdata(scn, data))) {\n+\t\tif (data-\u003ed_off \u003e shdr-\u003esh_size ||\n+\t\t data-\u003ed_size \u003e shdr-\u003esh_size - data-\u003ed_off) {\n+\t\t\tfree(buf);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tmemcpy(buf + data-\u003ed_off, data-\u003ed_buf, data-\u003ed_size);\n+\t}\n+\t*result = buf;\n+\treturn 0;\n+}\n+\n+static int add_elf_function(struct elf_function_list *list, const char *name,\n+\t\t\t const GElf_Sym *sym)\n+{\n+\tstruct elf_function *functions;\n+\tsize_t capacity;\n+\n+\tif (list-\u003enr == list-\u003ecapacity) {\n+\t\tcapacity = list-\u003ecapacity ? 2 * list-\u003ecapacity : 1024;\n+\t\tif (capacity \u003e SIZE_MAX / sizeof(*functions))\n+\t\t\treturn -E2BIG;\n+\t\tfunctions = realloc(list-\u003efunctions, capacity * sizeof(*functions));\n+\t\tif (!functions)\n+\t\t\treturn -ENOMEM;\n+\t\tlist-\u003efunctions = functions;\n+\t\tlist-\u003ecapacity = capacity;\n+\t}\n+\n+\tlist-\u003efunctions[list-\u003enr++] = (struct elf_function) {\n+\t\t.name = name,\n+\t\t.value = sym-\u003est_value,\n+\t\t.size = sym-\u003est_size,\n+\t\t.shndx = sym-\u003est_shndx,\n+\t\t.weak_symbol = GELF_ST_BIND(sym-\u003est_info) == STB_WEAK,\n+\t\t.strong = NO_FUNCTION,\n+\t\t.weak = NO_FUNCTION,\n+\t};\n+\treturn 0;\n+}\n+\n+static int collect_elf_functions(Elf *elf, struct elf_function_list *list)\n+{\n+\tElf_Scn *scn = NULL;\n+\n+\twhile ((scn = elf_nextscn(elf, scn))) {\n+\t\tGElf_Shdr shdr;\n+\t\tElf_Data *data;\n+\t\tsize_t nr_symbols;\n+\t\tsize_t i;\n+\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tif (shdr.sh_type != SHT_SYMTAB)\n+\t\t\tcontinue;\n+\t\tif (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)\n+\t\t\treturn -EINVAL;\n+\t\tdata = elf_getdata(scn, NULL);\n+\t\tif (!data || elf_getdata(scn, data))\n+\t\t\treturn -1;\n+\n+\t\tnr_symbols = shdr.sh_size / shdr.sh_entsize;\n+\t\tfor (i = 0; i \u003c nr_symbols; i++) {\n+\t\t\tconst char *name;\n+\t\t\tGElf_Sym sym;\n+\t\t\tint err;\n+\n+\t\t\tif (!gelf_getsym(data, i, \u0026sym))\n+\t\t\t\treturn -1;\n+\t\t\tif (GELF_ST_TYPE(sym.st_info) != STT_FUNC ||\n+\t\t\t sym.st_shndx == SHN_UNDEF || !sym.st_name)\n+\t\t\t\tcontinue;\n+\t\t\tname = elf_strptr(elf, shdr.sh_link, sym.st_name);\n+\t\t\tif (!name)\n+\t\t\t\treturn -1;\n+\t\t\terr = add_elf_function(list, name, \u0026sym);\n+\t\t\tif (err)\n+\t\t\t\treturn err;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int cmp_elf_function_location(const void *va, const void *vb)\n+{\n+\tconst struct elf_function *a = va;\n+\tconst struct elf_function *b = vb;\n+\n+\tif (a-\u003eshndx != b-\u003eshndx)\n+\t\treturn a-\u003eshndx \u003e b-\u003eshndx ? 1 : -1;\n+\tif (a-\u003evalue != b-\u003evalue)\n+\t\treturn a-\u003evalue \u003e b-\u003evalue ? 1 : -1;\n+\treturn 0;\n+}\n+\n+static int cmp_elf_function_name(const void *va, const void *vb)\n+{\n+\tconst struct elf_function *a = va;\n+\tconst struct elf_function *b = vb;\n+\n+\treturn strcmp(a-\u003ename, b-\u003ename) ?: cmp_elf_function_location(va, vb);\n+}\n+\n+/* Return the first function at VALUE in SHNDX. Functions are sorted by location. */\n+static size_t function_at(const struct elf_function_list *list,\n+\t\t\t Elf64_Section shndx, GElf_Addr value)\n+{\n+\tsize_t lo = 0, hi = list-\u003enr;\n+\n+\twhile (lo \u003c hi) {\n+\t\tsize_t mid = lo + (hi - lo) / 2;\n+\t\tconst struct elf_function *f = \u0026list-\u003efunctions[mid];\n+\n+\t\tif (f-\u003eshndx \u003c shndx || (f-\u003eshndx == shndx \u0026\u0026 f-\u003evalue \u003c value))\n+\t\t\tlo = mid + 1;\n+\t\telse\n+\t\t\thi = mid;\n+\t}\n+\tif (lo == list-\u003enr || list-\u003efunctions[lo].shndx != shndx ||\n+\t list-\u003efunctions[lo].value != value)\n+\t\treturn NO_FUNCTION;\n+\treturn lo;\n+}\n+\n+static bool function_contains(const struct elf_function *function,\n+\t\t\t GElf_Addr address)\n+{\n+\tif (address \u003c function-\u003evalue)\n+\t\treturn false;\n+\tif (!function-\u003esize)\n+\t\treturn address == function-\u003evalue;\n+\treturn address - function-\u003evalue \u003c function-\u003esize;\n+}\n+\n+enum ftrace_site_kind {\n+\tFTRACE_SITE_MCOUNT,\n+\tFTRACE_SITE_PATCHABLE,\n+\tFTRACE_SITE_UNKNOWN,\n+};\n+\n+/*\n+ * The NOPs which arm64 puts before a function with CALL_OPS. From the entry of\n+ * an overridden weak function, whose body has no symbol left, a longer distance\n+ * could reach the next function. Other arches may need more.\n+ */\n+#define MAX_PATCHABLE_FUNCTION_ENTRY_SIZE 8\n+\n+/* Return true if function I or another function at its address contains ADDRESS */\n+static bool alias_contains(const struct elf_function_list *list, size_t i,\n+\t\t\t GElf_Addr address)\n+{\n+\tconst struct elf_function *f = \u0026list-\u003efunctions[i];\n+\n+\tfor (;;) {\n+\t\tif (function_contains(\u0026list-\u003efunctions[i], address))\n+\t\t\treturn true;\n+\t\tif (!i || list-\u003efunctions[i - 1].shndx != f-\u003eshndx ||\n+\t\t list-\u003efunctions[i - 1].value != f-\u003evalue)\n+\t\t\treturn false;\n+\t\ti--;\n+\t}\n+}\n+\n+/*\n+ * Mark the functions at (SHNDX, VALUE) traceable, or, with CHECK, only those\n+ * which contain ADDRESS.\n+ */\n+static void mark_aliases(struct elf_function_list *list, Elf64_Section shndx,\n+\t\t\t GElf_Addr value, GElf_Addr address, bool check)\n+{\n+\tsize_t first = function_at(list, shndx, value), i;\n+\n+\tif (first == NO_FUNCTION)\n+\t\treturn;\n+\tfor (i = first; i \u003c list-\u003enr \u0026\u0026 list-\u003efunctions[i].shndx == shndx \u0026\u0026\n+\t list-\u003efunctions[i].value == value; i++) {\n+\t\tif (check \u0026\u0026 !function_contains(\u0026list-\u003efunctions[i], address))\n+\t\t\tcontinue;\n+\t\tlist-\u003efunctions[i].traceable = true;\n+\t}\n+}\n+\n+/*\n+ * Mark the function containing an ftrace call site. Functions are sorted by\n+ * section and address. Mark all aliases at the same address: at run time BTF\n+ * attachment is name based and may resolve to any of them.\n+ */\n+static void mark_traceable_function(struct elf_function_list *list,\n+\t\t\t\t Elf64_Section shndx, GElf_Addr address,\n+\t\t\t\t enum ftrace_site_kind kind)\n+{\n+\tsize_t left = 0;\n+\tsize_t right = list-\u003enr;\n+\tsize_t first;\n+\n+\t/* Find the first function after ADDRESS in SHNDX. */\n+\twhile (left \u003c right) {\n+\t\tsize_t middle = left + (right - left) / 2;\n+\t\tstruct elf_function *function = \u0026list-\u003efunctions[middle];\n+\n+\t\tif (function-\u003eshndx \u003c shndx ||\n+\t\t (function-\u003eshndx == shndx \u0026\u0026 function-\u003evalue \u003c= address))\n+\t\t\tleft = middle + 1;\n+\t\telse\n+\t\t\tright = middle;\n+\t}\n+\t/*\n+\t * A patchable entry is at the start of its function, a few bytes into it\n+\t * (after an arm64 BTI) or in the NOPs before it\n+\t */\n+\tfirst = SIZE_MAX;\n+\tif (kind != FTRACE_SITE_MCOUNT) {\n+\t\tif (left \u0026\u0026 list-\u003efunctions[left - 1].shndx == shndx \u0026\u0026\n+\t\t address - list-\u003efunctions[left - 1].value \u003c\n+\t\t\t\t\tMAX_PATCHABLE_FUNCTION_ENTRY_SIZE \u0026\u0026\n+\t\t alias_contains(list, left - 1, address))\n+\t\t\tfirst = left - 1;\n+\t\telse if (left \u003c list-\u003enr \u0026\u0026 list-\u003efunctions[left].shndx == shndx \u0026\u0026\n+\t\t\t list-\u003efunctions[left].value - address \u003c=\n+\t\t\t\t\tMAX_PATCHABLE_FUNCTION_ENTRY_SIZE)\n+\t\t\tfirst = left;\n+\t}\n+\tif (first != SIZE_MAX) {\n+\t\tmark_aliases(list, shndx, list-\u003efunctions[first].value, address, false);\n+\t\treturn;\n+\t}\n+\tif (kind == FTRACE_SITE_PATCHABLE || !left)\n+\t\treturn;\n+\n+\t/* The nearest preceding function owns the call site. */\n+\tfirst = left - 1;\n+\tif (list-\u003efunctions[first].shndx != shndx)\n+\t\treturn;\n+\tmark_aliases(list, shndx, list-\u003efunctions[first].value, address, true);\n+}\n+\n+static int mark_relocated_ftrace_sites(Elf *elf, size_t target_ndx,\n+\t\t\t\t const unsigned char *target_data,\n+\t\t\t\t const GElf_Shdr *target_shdr,\n+\t\t\t\t size_t begin, size_t end,\n+\t\t\t\t size_t ptr_size, unsigned int encoding,\n+\t\t\t\t struct elf_function_list *functions,\n+\t\t\t\t enum ftrace_site_kind kind,\n+\t\t\t\t bool *relocated)\n+{\n+\tElf_Scn *scn = NULL;\n+\n+\t*relocated = false;\n+\twhile ((scn = elf_nextscn(elf, scn))) {\n+\t\tGElf_Shdr shdr;\n+\t\tElf_Data *data;\n+\t\tsize_t nr_relocations;\n+\t\tsize_t i;\n+\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tif ((shdr.sh_type != SHT_RELA \u0026\u0026 shdr.sh_type != SHT_REL) ||\n+\t\t (shdr.sh_info \u0026\u0026 shdr.sh_info != target_ndx))\n+\t\t\tcontinue;\n+\t\tif (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)\n+\t\t\treturn -EINVAL;\n+\t\tdata = elf_getdata(scn, NULL);\n+\t\tif (!data || elf_getdata(scn, data))\n+\t\t\treturn -1;\n+\n+\t\tnr_relocations = shdr.sh_size / shdr.sh_entsize;\n+\t\tfor (i = 0; i \u003c nr_relocations; i++) {\n+\t\t\tElf_Scn *symtab_scn;\n+\t\t\tElf_Data *symtab_data;\n+\t\t\tGElf_Shdr symtab_shdr;\n+\t\t\tGElf_Sxword addend;\n+\t\t\tGElf_Addr raw_offset;\n+\t\t\tGElf_Addr address;\n+\t\t\tGElf_Sym sym;\n+\t\t\tsize_t sym_ndx;\n+\t\t\tsize_t offset;\n+\n+\t\t\tif (shdr.sh_type == SHT_RELA) {\n+\t\t\t\tGElf_Rela rela;\n+\n+\t\t\t\tif (!gelf_getrela(data, i, \u0026rela))\n+\t\t\t\t\treturn -1;\n+\t\t\t\tsym_ndx = GELF_R_SYM(rela.r_info);\n+\t\t\t\traw_offset = rela.r_offset;\n+\t\t\t\taddend = rela.r_addend;\n+\t\t\t} else {\n+\t\t\t\tGElf_Rel rel;\n+\n+\t\t\t\tif (!gelf_getrel(data, i, \u0026rel))\n+\t\t\t\t\treturn -1;\n+\t\t\t\tsym_ndx = GELF_R_SYM(rel.r_info);\n+\t\t\t\traw_offset = rel.r_offset;\n+\t\t\t\tif (!relocation_offset(target_shdr, raw_offset,\n+\t\t\t\t\t\t ptr_size, \u0026offset))\n+\t\t\t\t\tcontinue;\n+\t\t\t\taddend = read_elf_value(target_data + offset,\n+\t\t\t\t\t\t\tptr_size, encoding);\n+\t\t\t}\n+\t\t\tif (!relocation_offset(target_shdr, raw_offset, ptr_size,\n+\t\t\t\t\t \u0026offset))\n+\t\t\t\tcontinue;\n+\t\t\tif (offset \u003c begin || offset \u003e= end)\n+\t\t\t\tcontinue;\n+\t\t\t*relocated = true;\n+\n+\t\t\t/*\n+\t\t\t * PIE vmlinux uses symbol-less RELATIVE relocations. Its\n+\t\t\t * addend is the link-time call-site address, represented as\n+\t\t\t * a signed ELF addend.\n+\t\t\t */\n+\t\t\tif (!sym_ndx) {\n+\t\t\t\tElf64_Section target_shndx;\n+\n+\t\t\t\taddress = (GElf_Addr)addend;\n+\t\t\t\tif (section_for_address(elf, address,\n+\t\t\t\t\t\t\t\u0026target_shndx))\n+\t\t\t\t\treturn -1;\n+\t\t\t\tif (target_shndx != SHN_UNDEF)\n+\t\t\t\t\tmark_traceable_function(functions,\n+\t\t\t\t\t\t\t\ttarget_shndx,\n+\t\t\t\t\t\t\t\taddress, kind);\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\n+\t\t\tsymtab_scn = elf_getscn(elf, shdr.sh_link);\n+\t\t\tif (!symtab_scn ||\n+\t\t\t !gelf_getshdr(symtab_scn, \u0026symtab_shdr))\n+\t\t\t\treturn -1;\n+\t\t\tsymtab_data = elf_getdata(symtab_scn, NULL);\n+\t\t\tif (!symtab_data || elf_getdata(symtab_scn, symtab_data) ||\n+\t\t\t !gelf_getsym(symtab_data, sym_ndx, \u0026sym))\n+\t\t\t\treturn -1;\n+\t\t\tif (sym.st_shndx == SHN_UNDEF)\n+\t\t\t\tcontinue;\n+\t\t\tif (sym.st_shndx \u003e= SHN_LORESERVE)\n+\t\t\t\tcontinue;\n+\t\t\tif (addend \u003c 0) {\n+\t\t\t\tGElf_Addr delta = -(addend + 1);\n+\n+\t\t\t\tdelta++;\n+\t\t\t\tif (delta \u003e sym.st_value)\n+\t\t\t\t\tcontinue;\n+\t\t\t\taddress = sym.st_value - delta;\n+\t\t\t} else {\n+\t\t\t\tif ((GElf_Addr)addend \u003e UINT64_MAX - sym.st_value)\n+\t\t\t\t\tcontinue;\n+\t\t\t\taddress = sym.st_value + addend;\n+\t\t\t}\n+\t\t\tmark_traceable_function(functions, sym.st_shndx, address,\n+\t\t\t\t\t\tkind);\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int read_ftrace_site_range(Elf *elf, Elf_Scn *scn, size_t begin,\n+\t\t\t\t size_t end, size_t ptr_size,\n+\t\t\t\t unsigned int encoding,\n+\t\t\t\t struct elf_function_list *functions,\n+\t\t\t\t enum ftrace_site_kind kind)\n+{\n+\tunsigned char *buf = NULL;\n+\tGElf_Shdr shdr;\n+\tbool relocated;\n+\tsize_t offset;\n+\tint err;\n+\n+\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\treturn -1;\n+\tif (begin \u003e end || end \u003e shdr.sh_size ||\n+\t (end - begin) % ptr_size)\n+\t\treturn -EINVAL;\n+\tif (begin == end)\n+\t\treturn 0;\n+\terr = copy_section_data(scn, \u0026shdr, \u0026buf);\n+\tif (err)\n+\t\treturn err;\n+\terr = mark_relocated_ftrace_sites(elf, elf_ndxscn(scn), buf, \u0026shdr,\n+\t\t\t\t\t begin, end, ptr_size, encoding,\n+\t\t\t\t\t functions, kind, \u0026relocated);\n+\tif (err || relocated)\n+\t\tgoto out;\n+\n+\tfor (offset = begin; offset \u003c end; offset += ptr_size) {\n+\t\tGElf_Addr address;\n+\t\tElf64_Section shndx;\n+\n+\t\taddress = read_elf_value(buf + offset, ptr_size, encoding);\n+\t\terr = section_for_address(elf, address, \u0026shndx);\n+\t\tif (err)\n+\t\t\tgoto out;\n+\t\tif (shndx != SHN_UNDEF)\n+\t\t\tmark_traceable_function(functions, shndx, address, kind);\n+\t}\n+out:\n+\tfree(buf);\n+\treturn err;\n+}\n+\n+static bool section_prefix(const char *name, const char *prefix)\n+{\n+\tsize_t len = strlen(prefix);\n+\n+\treturn !strncmp(name, prefix, len) \u0026\u0026\n+\t (!name[len] || name[len] == '.');\n+}\n+\n+static bool section_cannot_be_attached(const char *name)\n+{\n+\treturn section_prefix(name, \".init.text\") ||\n+\t section_prefix(name, \".exit.text\") ||\n+\t section_prefix(name, \".noinstr.text\");\n+}\n+\n+static int remove_unattachable_functions(Elf *elf,\n+\t\t\t\t\t struct elf_function_list *functions,\n+\t\t\t\t\t size_t shstrndx)\n+{\n+\tGElf_Sym noinstr_start;\n+\tGElf_Sym noinstr_end;\n+\tbool found_start;\n+\tbool found_end;\n+\tsize_t i;\n+\tint err;\n+\n+\terr = find_symbol(elf, \"__noinstr_text_start\", \u0026noinstr_start,\n+\t\t\t \u0026found_start);\n+\tif (err)\n+\t\treturn err;\n+\terr = find_symbol(elf, \"__noinstr_text_end\", \u0026noinstr_end, \u0026found_end);\n+\tif (err)\n+\t\treturn err;\n+\n+\tfor (i = 0; i \u003c functions-\u003enr; i++) {\n+\t\tstruct elf_function *function = \u0026functions-\u003efunctions[i];\n+\t\tElf_Scn *scn;\n+\t\tGElf_Shdr shdr;\n+\t\tconst char *name;\n+\n+\t\tif (!function-\u003etraceable)\n+\t\t\tcontinue;\n+\t\tscn = elf_getscn(elf, function-\u003eshndx);\n+\t\tif (!scn || !gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tname = elf_strptr(elf, shstrndx, shdr.sh_name);\n+\t\tif (!name)\n+\t\t\treturn -1;\n+\t\tif (section_cannot_be_attached(name))\n+\t\t\tfunction-\u003etraceable = false;\n+\t\telse if (found_start \u0026\u0026 found_end \u0026\u0026\n+\t\t\t function-\u003eshndx == noinstr_start.st_shndx \u0026\u0026\n+\t\t\t noinstr_start.st_shndx == noinstr_end.st_shndx \u0026\u0026\n+\t\t\t function-\u003evalue \u003e= noinstr_start.st_value \u0026\u0026\n+\t\t\t function-\u003evalue \u003c noinstr_end.st_value)\n+\t\t\tfunction-\u003etraceable = false;\n+\t}\n+\treturn 0;\n+}\n+\n+static int read_ftrace_symbol_range(Elf *elf, const GElf_Sym *start,\n+\t\t\t\t const GElf_Sym *stop, size_t ptr_size,\n+\t\t\t\t unsigned int encoding,\n+\t\t\t\t struct elf_function_list *functions,\n+\t\t\t\t enum ftrace_site_kind kind)\n+{\n+\tElf_Scn *scn;\n+\tGElf_Shdr shdr;\n+\tsize_t begin;\n+\tsize_t end;\n+\n+\tif (start-\u003est_shndx != stop-\u003est_shndx)\n+\t\treturn -EINVAL;\n+\tscn = elf_getscn(elf, start-\u003est_shndx);\n+\tif (!scn || !gelf_getshdr(scn, \u0026shdr))\n+\t\treturn -1;\n+\tif (start-\u003est_value \u003c shdr.sh_addr || stop-\u003est_value \u003c start-\u003est_value ||\n+\t stop-\u003est_value - shdr.sh_addr \u003e shdr.sh_size)\n+\t\treturn -EINVAL;\n+\tbegin = start-\u003est_value - shdr.sh_addr;\n+\tend = stop-\u003est_value - shdr.sh_addr;\n+\treturn read_ftrace_site_range(elf, scn, begin, end, ptr_size, encoding,\n+\t\t\t\t functions, kind);\n+}\n+\n+/*\n+ * Mark the functions with an ftrace call site. A module keeps the sites in the\n+ * section FTRACE_CALLSITE_SECTION: __patchable_function_entries if PATCHABLE,\n+ * else __mcount_loc. x86 also has call padding entries, which are not call\n+ * sites, in __patchable_function_entries. vmlinux merges the sites into init\n+ * data between the __start_mcount_loc and __stop_mcount_loc symbols, see\n+ * MCOUNT_REC().\n+ */\n+static int read_ftrace_sites(Elf *elf, size_t shstrndx,\n+\t\t\t size_t ptr_size, unsigned int encoding,\n+\t\t\t struct elf_function_list *functions, bool patchable)\n+{\n+\tconst char *section = patchable ? \"__patchable_function_entries\" : \"__mcount_loc\";\n+\tElf_Scn *scn = NULL;\n+\tGElf_Sym start;\n+\tGElf_Sym stop;\n+\tGElf_Sym patchable_start;\n+\tGElf_Sym patchable_stop;\n+\tbool found_start;\n+\tbool found_stop;\n+\tbool found_patchable_start;\n+\tbool found_patchable_stop;\n+\tGElf_Shdr shdr;\n+\tint err;\n+\n+\terr = find_section(elf, shstrndx, section, \u0026scn);\n+\tif (err)\n+\t\treturn err;\n+\tif (scn) {\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\terr = read_ftrace_site_range(elf, scn, 0, shdr.sh_size,\n+\t\t\t\t\t ptr_size, encoding, functions,\n+\t\t\t\t\t patchable ? FTRACE_SITE_PATCHABLE :\n+\t\t\t\t\t\t\t FTRACE_SITE_MCOUNT);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\n+\terr = find_symbol(elf, \"__start_mcount_loc\", \u0026start, \u0026found_start);\n+\tif (err)\n+\t\treturn err;\n+\terr = find_symbol(elf, \"__stop_mcount_loc\", \u0026stop, \u0026found_stop);\n+\tif (err)\n+\t\treturn err;\n+\terr = find_symbol(elf, \"__start_patchable_function_entries\",\n+\t\t\t \u0026patchable_start, \u0026found_patchable_start);\n+\tif (err)\n+\t\treturn err;\n+\terr = find_symbol(elf, \"__stop_patchable_function_entries\",\n+\t\t\t \u0026patchable_stop, \u0026found_patchable_stop);\n+\tif (err)\n+\t\treturn err;\n+\tif (found_start \u0026\u0026 found_stop) {\n+\t\tif (found_patchable_start \u0026\u0026 found_patchable_stop) {\n+\t\t\terr = read_ftrace_symbol_range(elf, \u0026start,\n+\t\t\t\t\t\t \u0026patchable_start,\n+\t\t\t\t\t\t ptr_size, encoding,\n+\t\t\t\t\t\t functions,\n+\t\t\t\t\t\t FTRACE_SITE_MCOUNT);\n+\t\t\tif (err)\n+\t\t\t\treturn err;\n+\t\t\terr = read_ftrace_symbol_range(elf, \u0026patchable_start,\n+\t\t\t\t\t\t \u0026patchable_stop,\n+\t\t\t\t\t\t ptr_size, encoding,\n+\t\t\t\t\t\t functions,\n+\t\t\t\t\t\t FTRACE_SITE_PATCHABLE);\n+\t\t} else {\n+\t\t\terr = read_ftrace_symbol_range(elf, \u0026start, \u0026stop,\n+\t\t\t\t\t\t ptr_size, encoding,\n+\t\t\t\t\t\t functions,\n+\t\t\t\t\t\t FTRACE_SITE_UNKNOWN);\n+\t\t}\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\n+\treturn remove_unattachable_functions(elf, functions, shstrndx);\n+}\n+\n+/*\n+ * Find the function each record points to. In a module the pointer is a\n+ * relocation, in a linked image it is the address. REFS maps each pointer\n+ * sized slot of the section to a function.\n+ */\n+static int resolve_record_relocations(Elf *elf, const struct elf_function_list *functions,\n+\t\t\t\t size_t metadata_ndx, const GElf_Shdr *metadata_shdr,\n+\t\t\t\t const unsigned char *metadata, size_t *refs,\n+\t\t\t\t size_t ptr_size, unsigned int encoding)\n+{\n+\tElf_Scn *scn = NULL;\n+\n+\twhile ((scn = elf_nextscn(elf, scn))) {\n+\t\tElf_Data *data, *symbols;\n+\t\tElf_Scn *symtab;\n+\t\tGElf_Shdr shdr;\n+\t\tsize_t i;\n+\n+\t\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\t\treturn -1;\n+\t\tif ((shdr.sh_type != SHT_RELA \u0026\u0026 shdr.sh_type != SHT_REL) ||\n+\t\t shdr.sh_info != metadata_ndx)\n+\t\t\tcontinue;\n+\t\tif (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)\n+\t\t\treturn -EINVAL;\n+\t\tdata = elf_getdata(scn, NULL);\n+\t\tsymtab = elf_getscn(elf, shdr.sh_link);\n+\t\tsymbols = symtab ? elf_getdata(symtab, NULL) : NULL;\n+\t\tif (!data || !symbols || elf_getdata(scn, data))\n+\t\t\treturn -1;\n+\t\tfor (i = 0; i \u003c shdr.sh_size / shdr.sh_entsize; i++) {\n+\t\t\tGElf_Rela rela;\n+\t\t\tGElf_Sym sym;\n+\t\t\tsize_t offset;\n+\n+\t\t\tif (shdr.sh_type == SHT_RELA) {\n+\t\t\t\tif (!gelf_getrela(data, i, \u0026rela))\n+\t\t\t\t\treturn -1;\n+\t\t\t} else {\n+\t\t\t\tGElf_Rel rel;\n+\n+\t\t\t\tif (!gelf_getrel(data, i, \u0026rel))\n+\t\t\t\t\treturn -1;\n+\t\t\t\trela = (GElf_Rela){ .r_offset = rel.r_offset,\n+\t\t\t\t\t\t .r_info = rel.r_info };\n+\t\t\t\t/* REL addends are stored in the pointer itself. */\n+\t\t\t\tif (!relocation_offset(metadata_shdr, rel.r_offset, ptr_size,\n+\t\t\t\t\t\t \u0026offset))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\trela.r_addend = read_elf_value(metadata + offset, ptr_size,\n+\t\t\t\t\t\t\t encoding);\n+\t\t\t}\n+\t\t\tif (!relocation_offset(metadata_shdr, rela.r_offset, ptr_size, \u0026offset) ||\n+\t\t\t offset % ptr_size)\n+\t\t\t\treturn -EINVAL;\n+\t\t\tif (!gelf_getsym(symbols, GELF_R_SYM(rela.r_info), \u0026sym))\n+\t\t\t\treturn -1;\n+\t\t\tif (sym.st_shndx == SHN_UNDEF || sym.st_shndx \u003e= SHN_LORESERVE)\n+\t\t\t\tcontinue;\n+\t\t\trefs[offset / ptr_size] = function_at(functions, sym.st_shndx,\n+\t\t\t\t\t\t\t sym.st_value + rela.r_addend);\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static size_t record_function(Elf *elf, const struct elf_function_list *functions,\n+\t\t\t const unsigned char *buf, const size_t *refs,\n+\t\t\t size_t offset, size_t ptr_size, unsigned int encoding)\n+{\n+\tElf64_Section section;\n+\tGElf_Ehdr ehdr;\n+\tGElf_Addr value;\n+\n+\tif (refs[offset / ptr_size] != NO_FUNCTION)\n+\t\treturn refs[offset / ptr_size];\n+\tif (!gelf_getehdr(elf, \u0026ehdr) || ehdr.e_type == ET_REL)\n+\t\treturn NO_FUNCTION;\n+\tvalue = read_elf_value(buf + offset, ptr_size, encoding);\n+\tif (!value || section_for_address(elf, value, \u0026section))\n+\t\treturn NO_FUNCTION;\n+\treturn function_at(functions, section, value);\n+}\n+\n+/* Remember the records of a body for all the functions at its address */\n+static void add_record(struct elf_function_list *functions, size_t first,\n+\t\t size_t offset, bool weak)\n+{\n+\tstruct elf_function *f = \u0026functions-\u003efunctions[first];\n+\tsize_t i;\n+\n+\tfor (i = first; i \u003c functions-\u003enr \u0026\u0026\n+\t !cmp_elf_function_location(f, \u0026functions-\u003efunctions[i]); i++) {\n+\t\tif (weak) {\n+\t\t\tfunctions-\u003efunctions[i].nr_weak++;\n+\t\t\tfunctions-\u003efunctions[i].weak = offset;\n+\t\t} else {\n+\t\t\tfunctions-\u003efunctions[i].nr_strong++;\n+\t\t\tfunctions-\u003efunctions[i].strong = offset;\n+\t\t}\n+\t}\n+}\n+\n+static int parse_records(Elf *elf, struct elf_function_list *functions,\n+\t\t\t const unsigned char *buf, size_t size, const size_t *refs,\n+\t\t\t size_t ptr_size, unsigned int encoding,\n+\t\t\t struct fmodret_stats *stats)\n+{\n+\tsize_t offset = 0;\n+\n+\twhile (offset \u003c size) {\n+\t\tunsigned int flags, length, nr_values;\n+\t\tsize_t function;\n+\n+\t\tif (size - offset \u003c FMODRET_META_HEADER_SIZE(ptr_size) ||\n+\t\t read_elf_value(buf + offset, 4, encoding) != FMODRET_META_MAGIC ||\n+\t\t read_elf_value(buf + offset + 4, 4, encoding) != FMODRET_META_VERSION)\n+\t\t\tgoto invalid;\n+\t\tlength = read_elf_value(buf + offset + 8, 4, encoding);\n+\t\tflags = read_elf_value(buf + offset + 12, 4, encoding);\n+\t\tnr_values = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);\n+\t\tif (length \u003c FMODRET_META_HEADER_SIZE(ptr_size) || length \u003e size - offset ||\n+\t\t length % ptr_size ||\n+\t\t nr_values != (length - FMODRET_META_HEADER_SIZE(ptr_size)) /\n+\t\t\t\t FMODRET_META_VALUE_SIZE ||\n+\t\t (length - FMODRET_META_HEADER_SIZE(ptr_size)) % FMODRET_META_VALUE_SIZE)\n+\t\t\tgoto invalid;\n+\t\tstats-\u003erecords++;\n+\t\tfunction = record_function(elf, functions, buf, refs, offset + 16,\n+\t\t\t\t\t ptr_size, encoding);\n+\t\t/* A function in a discarded section has no ELF function */\n+\t\tif (function == NO_FUNCTION)\n+\t\t\tstats-\u003eunmapped++;\n+\t\telse\n+\t\t\tadd_record(functions, function, offset, flags \u0026 FMODRET_FN_WEAK);\n+\t\toffset += length;\n+\t}\n+\treturn 0;\n+invalid:\n+\tfprintf(stderr,\n+\t\t\"invalid or old fmodret metadata at offset %#zx; rebuild all plugin objects\\n\",\n+\t\toffset);\n+\treturn -EINVAL;\n+}\n+\n+/* Return the record which describes the body of F, or NO_FUNCTION */\n+static size_t chosen_record(const struct elf_function *f)\n+{\n+\tif (f-\u003enr_strong == 1)\n+\t\treturn f-\u003estrong;\n+\tif (!f-\u003enr_strong \u0026\u0026 f-\u003enr_weak == 1)\n+\t\treturn f-\u003eweak;\n+\treturn NO_FUNCTION;\n+}\n+\n+struct btf_func {\n+\tconst char *name;\n+\t__u32 id;\n+};\n+\n+static int cmp_btf_func(const void *a, const void *b)\n+{\n+\tconst struct btf_func *x = a, *y = b;\n+\n+\treturn strcmp(x-\u003ename, y-\u003ename);\n+}\n+\n+/*\n+ * Return the FUNCs of BTF, without those of its base, sorted by name. The BTF\n+ * is sorted only after the tags are added, and searching all of it for each\n+ * candidate is slow for vmlinux.\n+ */\n+static struct btf_func *collect_btf_funcs(const struct btf *btf, size_t *cnt)\n+{\n+\tconst struct btf *base = btf__base_btf(btf);\n+\t__u32 id, nr_types = btf__type_cnt(btf);\n+\tstruct btf_func *funcs;\n+\tsize_t n = 0;\n+\n+\tfuncs = malloc(nr_types * sizeof(*funcs));\n+\tif (!funcs)\n+\t\treturn NULL;\n+\tfor (id = base ? btf__type_cnt(base) : 1; id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\n+\t\tif (!btf_is_func(t))\n+\t\t\tcontinue;\n+\t\tfuncs[n].name = btf__name_by_offset(btf, t-\u003ename_off);\n+\t\tfuncs[n++].id = id;\n+\t}\n+\tqsort(funcs, n, sizeof(*funcs), cmp_btf_func);\n+\t*cnt = n;\n+\treturn funcs;\n+}\n+\n+/* Return the BTF ID of the only FUNC named NAME, or 0 */\n+static int find_btf_func(const struct btf_func *funcs, size_t cnt, const char *name)\n+{\n+\tconst struct btf_func key = { .name = name }, *f;\n+\n+\tf = bsearch(\u0026key, funcs, cnt, sizeof(*funcs), cmp_btf_func);\n+\tif (!f)\n+\t\treturn 0;\n+\tif ((f \u003e funcs \u0026\u0026 !strcmp(f[-1].name, name)) ||\n+\t (f + 1 \u003c funcs + cnt \u0026\u0026 !strcmp(f[1].name, name)))\n+\t\treturn 0;\n+\treturn f-\u003eid;\n+}\n+\n+/* Return the size of the integer which FUNC returns, or 0 */\n+static __u32 btf_func_int_return_size(const struct btf *btf, int func)\n+{\n+\tconst struct btf_type *t = btf__type_by_id(btf, func);\n+\tint ret;\n+\n+\tt = btf__type_by_id(btf, t-\u003etype);\n+\tif (!t || !btf_is_func_proto(t) || !t-\u003etype)\n+\t\treturn 0;\n+\tret = btf__resolve_type(btf, t-\u003etype);\n+\tif (ret \u003c 0)\n+\t\treturn 0;\n+\tt = btf__type_by_id(btf, ret);\n+\tif (!btf_is_int(t) \u0026\u0026 !btf_is_enum(t) \u0026\u0026 !btf_is_enum64(t))\n+\t\treturn 0;\n+\treturn t-\u003esize;\n+}\n+\n+/* Format the values of the record at OFFSET as \"-22,-12\" */\n+static char *format_values(const unsigned char *buf, size_t offset, size_t ptr_size,\n+\t\t\t unsigned int encoding)\n+{\n+\tunsigned int nr = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);\n+\tconst unsigned char *p = buf + offset + FMODRET_META_HEADER_SIZE(ptr_size);\n+\tsize_t size = (size_t)nr * 21 + 1, len = 0;\n+\tchar *s = malloc(size);\n+\tunsigned int i;\n+\n+\tif (!s)\n+\t\treturn NULL;\n+\tfor (i = 0; i \u003c nr; i++) {\n+\t\tint64_t value = read_elf_value(p + i * FMODRET_META_VALUE_SIZE,\n+\t\t\t\t\t FMODRET_META_VALUE_SIZE, encoding);\n+\n+\t\tlen += snprintf(s + len, size - len, \"%s%lld\", i ? \",\" : \"\",\n+\t\t\t\t(long long)value);\n+\t}\n+\ts[len] = 0;\n+\treturn s;\n+}\n+\n+static int cmp_retval(const void *va, const void *vb)\n+{\n+\tconst struct fmodret_retval *a = va;\n+\tconst struct fmodret_retval *b = vb;\n+\n+\treturn a-\u003ebtf_id \u003e b-\u003ebtf_id ? 1 : a-\u003ebtf_id \u003c b-\u003ebtf_id ? -1 : 0;\n+}\n+\n+/* Choose the targets among the functions and describe them in RESULT */\n+static int choose_functions(struct elf_function_list *functions, const struct btf *btf,\n+\t\t\t const struct btf_func *btf_funcs, size_t nr_btf_funcs,\n+\t\t\t const unsigned char *buf, size_t ptr_size,\n+\t\t\t unsigned int encoding, struct fmodret_retvals *result,\n+\t\t\t struct fmodret_stats *stats, int verbose)\n+{\n+\tsize_t i = 0, j;\n+\n+\tqsort(functions-\u003efunctions, functions-\u003enr, sizeof(*functions-\u003efunctions),\n+\t cmp_elf_function_name);\n+\tfor (i = 0; i \u003c functions-\u003enr; i = j) {\n+\t\tconst struct elf_function *f = \u0026functions-\u003efunctions[i];\n+\t\tsize_t record = chosen_record(f);\n+\t\tconst char *reason = NULL;\n+\t\tunsigned int flags = 0;\n+\t\tstruct fmodret_retval *r;\n+\t\t__u32 size;\n+\t\tint id = 0;\n+\n+\t\tfor (j = i + 1; j \u003c functions-\u003enr; j++)\n+\t\t\tif (strcmp(f-\u003ename, functions-\u003efunctions[j].name))\n+\t\t\t\tbreak;\n+\n+\t\tif (!f-\u003enr_strong \u0026\u0026 !f-\u003enr_weak)\n+\t\t\tcontinue;\n+\t\tif (record == NO_FUNCTION) {\n+\t\t\tstats-\u003eambiguous_body++;\n+\t\t\treason = \"ambiguous-body\";\n+\t\t\tgoto rejected;\n+\t\t}\n+\t\t/*\n+\t\t * A strong definition without values has no record. If it\n+\t\t * replaced a weak one, only the record of the weak one points\n+\t\t * to its body.\n+\t\t */\n+\t\tif (!f-\u003enr_strong \u0026\u0026 !f-\u003eweak_symbol) {\n+\t\t\tstats-\u003eoverridden++;\n+\t\t\treason = \"overridden-weak\";\n+\t\t\tgoto rejected;\n+\t\t}\n+\t\tif (!read_elf_value(buf + record + 16 + ptr_size, 4, encoding))\n+\t\t\tcontinue;\n+\t\tstats-\u003ewith_values++;\n+\t\tflags = read_elf_value(buf + record + 12, 4, encoding);\n+\t\tif (j - i != 1) {\n+\t\t\tstats-\u003eambiguous_name++;\n+\t\t\treason = \"ambiguous-name\";\n+\t\t\tgoto rejected;\n+\t\t}\n+\t\tif (!f-\u003etraceable) {\n+\t\t\tstats-\u003enot_traceable++;\n+\t\t\treason = \"not-traceable\";\n+\t\t\tgoto rejected;\n+\t\t}\n+\t\tid = find_btf_func(btf_funcs, nr_btf_funcs, f-\u003ename);\n+\t\tif (id \u003c= 0) {\n+\t\t\tstats-\u003eno_btf++;\n+\t\t\treason = \"no-unique-BTF-FUNC\";\n+\t\t\tgoto rejected;\n+\t\t}\n+\t\tsize = btf_func_int_return_size(btf, id);\n+\t\tif (size != (flags \u0026 FMODRET_FN_RET64 ? 8 : 4)) {\n+\t\t\tstats-\u003ereturn_mismatch++;\n+\t\t\treason = \"BTF-return-type\";\n+\t\t}\n+rejected:\n+\t\tif (reason) {\n+\t\t\tif (verbose \u003e 1)\n+\t\t\t\tfprintf(stderr, \"fmodret: rejected %s: %s\\n\", f-\u003ename, reason);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tif (result-\u003ecnt == result-\u003ecap) {\n+\t\t\tsize_t cap = result-\u003ecap ? 2 * result-\u003ecap : 256;\n+\n+\t\t\tr = realloc(result-\u003eretvals, cap * sizeof(*r));\n+\t\t\tif (!r)\n+\t\t\t\treturn -ENOMEM;\n+\t\t\tresult-\u003eretvals = r;\n+\t\t\tresult-\u003ecap = cap;\n+\t\t}\n+\t\tr = \u0026result-\u003eretvals[result-\u003ecnt];\n+\t\tr-\u003ebtf_id = id;\n+\t\tr-\u003evalues = format_values(buf, record, ptr_size, encoding);\n+\t\tif (!r-\u003evalues)\n+\t\t\treturn -ENOMEM;\n+\t\tresult-\u003ecnt++;\n+\t\tstats-\u003eadded++;\n+\t\tif (verbose \u003e 1)\n+\t\t\tfprintf(stderr, \"fmodret: %s: BTF ID %d, retval %s\\n\",\n+\t\t\t\tf-\u003ename, id, r-\u003evalues);\n+\t}\n+\tif (result-\u003ecnt)\n+\t\tqsort(result-\u003eretvals, result-\u003ecnt, sizeof(*result-\u003eretvals), cmp_retval);\n+\treturn 0;\n+}\n+\n+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,\n+\t\t bool patchable_ftrace, int verbose)\n+{\n+\tstruct elf_function_list functions = {};\n+\tstruct btf_func *btf_funcs = NULL;\n+\tstruct fmodret_stats stats = {};\n+\tunsigned char *buf = NULL;\n+\tsize_t nr_btf_funcs;\n+\tsize_t *refs = NULL;\n+\tsize_t shstrndx, ptr_size, i;\n+\tunsigned int encoding;\n+\tElf_Scn *scn;\n+\tGElf_Shdr shdr;\n+\tint err;\n+\n+\tif (elf_getshdrstrndx(elf, \u0026shstrndx))\n+\t\treturn -EINVAL;\n+\terr = find_section(elf, shstrndx, FMODRET_CANDIDATES_SECTION, \u0026scn);\n+\tif (err || !scn)\n+\t\treturn err;\n+\tif (!gelf_getshdr(scn, \u0026shdr))\n+\t\treturn -EINVAL;\n+\tptr_size = gelf_getclass(elf) == ELFCLASS64 ? 8 : 4;\n+\tencoding = elf_getident(elf, NULL)[EI_DATA];\n+\tif (shdr.sh_size % ptr_size)\n+\t\treturn -EINVAL;\n+\n+\terr = collect_elf_functions(elf, \u0026functions);\n+\tif (err)\n+\t\tgoto out;\n+\tqsort(functions.functions, functions.nr, sizeof(*functions.functions),\n+\t cmp_elf_function_location);\n+\n+\terr = copy_section_data(scn, \u0026shdr, \u0026buf);\n+\tif (err)\n+\t\tgoto out;\n+\trefs = malloc((shdr.sh_size / ptr_size ?: 1) * sizeof(*refs));\n+\tif (!refs) {\n+\t\terr = -ENOMEM;\n+\t\tgoto out;\n+\t}\n+\tfor (i = 0; i \u003c shdr.sh_size / ptr_size; i++)\n+\t\trefs[i] = NO_FUNCTION;\n+\terr = resolve_record_relocations(elf, \u0026functions, elf_ndxscn(scn), \u0026shdr, buf,\n+\t\t\t\t\t refs, ptr_size, encoding);\n+\tif (err)\n+\t\tgoto out;\n+\terr = parse_records(elf, \u0026functions, buf, shdr.sh_size, refs, ptr_size,\n+\t\t\t encoding, \u0026stats);\n+\tif (err)\n+\t\tgoto out;\n+\terr = read_ftrace_sites(elf, shstrndx, ptr_size, encoding, \u0026functions,\n+\t\t\t\tpatchable_ftrace);\n+\tif (err)\n+\t\tgoto out;\n+\tbtf_funcs = collect_btf_funcs(btf, \u0026nr_btf_funcs);\n+\tif (!btf_funcs) {\n+\t\terr = -ENOMEM;\n+\t\tgoto out;\n+\t}\n+\terr = choose_functions(\u0026functions, btf, btf_funcs, nr_btf_funcs, buf, ptr_size,\n+\t\t\t encoding, result, \u0026stats, verbose);\n+\tif (err)\n+\t\tgoto out;\n+\n+\tif (verbose)\n+\t\tfprintf(stderr,\n+\t\t\t\"fmodret: %zu records (%zu without a function), %zu functions with values, %zu added; rejected: %zu ambiguous body, %zu ambiguous name, %zu not traceable, %zu overridden weak, %zu without unique BTF FUNC, %zu BTF return type\\n\",\n+\t\t\tstats.records, stats.unmapped, stats.with_values, stats.added,\n+\t\t\tstats.ambiguous_body,\n+\t\t\tstats.ambiguous_name, stats.not_traceable, stats.overridden,\n+\t\t\tstats.no_btf, stats.return_mismatch);\n+out:\n+\tif (err == -1)\n+\t\tfprintf(stderr, \"failed to read %s: %s\\n\", FMODRET_CANDIDATES_SECTION,\n+\t\t\telf_errmsg(-1));\n+\tfree(btf_funcs);\n+\tfree(refs);\n+\tfree(buf);\n+\tfree(functions.functions);\n+\treturn err;\n+}\n+\n+void fmodret_free(struct fmodret_retvals *retvals)\n+{\n+\tsize_t i;\n+\n+\tfor (i = 0; i \u003c retvals-\u003ecnt; i++)\n+\t\tfree(retvals-\u003eretvals[i].values);\n+\tfree(retvals-\u003eretvals);\n+\tretvals-\u003eretvals = NULL;\n+\tretvals-\u003ecnt = retvals-\u003ecap = 0;\n+}\ndiff --git a/tools/bpf/resolve_btfids/fmodret.h b/tools/bpf/resolve_btfids/fmodret.h\nnew file mode 100644\nindex 00000000000000..c1d5db64970b62\n--- /dev/null\n+++ b/tools/bpf/resolve_btfids/fmodret.h\n@@ -0,0 +1,28 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+#ifndef __RESOLVE_BTFIDS_FMODRET_H\n+#define __RESOLVE_BTFIDS_FMODRET_H\n+\n+#include \u003cstdbool.h\u003e\n+#include \u003cstddef.h\u003e\n+#include \u003clibelf.h\u003e\n+#include \u003clinux/types.h\u003e\n+\n+struct btf;\n+\n+/* A function found by the analysis, and the values it may return */\n+struct fmodret_retval {\n+\t__u32 btf_id;\n+\tchar *values;\n+};\n+\n+/* Sorted by BTF ID */\n+struct fmodret_retvals {\n+\tstruct fmodret_retval *retvals;\n+\tsize_t cnt, cap;\n+};\n+\n+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,\n+\t\t bool patchable_ftrace, int verbose);\n+void fmodret_free(struct fmodret_retvals *retvals);\n+\n+#endif\ndiff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c\nindex c16dd93fa6d4c2..124c4f01c8878f 100644\n--- a/tools/bpf/resolve_btfids/main.c\n+++ b/tools/bpf/resolve_btfids/main.c\n@@ -69,6 +69,18 @@\n * - rewrites the prototype of KF_IMPLICIT_ARGS kfuncs.\n *\n * These kfunc annotations were historically produced by pahole.\n+ *\n+ * Functions found in BTF_SET8_FMODRET sets are the targets of BPF fmod_ret\n+ * programs. For each of them resolve_btfids emits a \"bpf_fmodret\" decl tag,\n+ * and also \"bpf_fmodret_sleepable\" when KF_SLEEPABLE is set, so the sleepable\n+ * tag never comes alone. The kernel builds the set of fmod_ret targets from\n+ * these tags when it loads BTF.\n+ *\n+ * Functions which the analysis of the compiler plugin records, see fmodret.c,\n+ * get \"bpf_fmodret\" too, and \"bpf_fmodret_retval:\" with the values they\n+ * return on paths without side effects, such as \"bpf_fmodret_retval:-22,-12\".\n+ * The verifier lets programs attached to them return only these values or 0.\n+ * A function in a BTF_FMODRET set keeps only its own tags.\n */\n \n #define _GNU_SOURCE\n@@ -90,6 +102,7 @@\n #include \u003cbpf/btf.h\u003e\n #include \u003cbpf/libbpf.h\u003e\n #include \"btf_colors.h\"\n+#include \"fmodret.h\"\n #include \u003csubcmd/parse-options.h\u003e\n \n #define BTF_IDS_SECTION\t\".BTF_ids\"\n@@ -148,6 +161,11 @@ struct object {\n \tstruct btf *orig_btf;\n \tbool distill_base;\n \tbool extract_inline;\n+\t/* The ftrace call sites are in __patchable_function_entries */\n+\tbool patchable_ftrace;\n+\n+\t/* Functions found by the analysis of the compiler plugin, see fmodret.c */\n+\tstruct fmodret_retvals fmodret_retvals;\n \n \tstruct {\n \t\tint\t\t fd;\n@@ -197,7 +215,11 @@ struct btf_links {\n \n #define DECL_TAG_FASTCALL \"bpf_fastcall\"\n #define DECL_TAG_KFUNC \"bpf_kfunc\"\n+#define DECL_TAG_FMODRET \"bpf_fmodret\"\n+#define DECL_TAG_FMODRET_SLEEPABLE \"bpf_fmodret_sleepable\"\n+#define DECL_TAG_FMODRET_RETVAL \"bpf_fmodret_retval:\"\n \n+#define KF_SLEEPABLE\t(1 \u003c\u003c 5)\n #define KF_FASTCALL\t(1 \u003c\u003c 12)\n #define KF_ARENA_RET\t(1 \u003c\u003c 13)\n #define KF_ARENA_ARG1\t(1 \u003c\u003c 14)\n@@ -208,6 +230,7 @@ struct btf_links {\n #define PARAM_SUFFIX_ARENA \"__arena\"\n #define PARAM_SUFFIX_ARENA_NULLABLE \"__arena__nullable\"\n \n+/* A function found in a BTF ID set: a kfunc or an fmod_ret target */\n struct kfunc {\n \tstruct rb_node rb_node;\n \tconst char *name;\n@@ -221,6 +244,7 @@ struct btf2btf_context {\n \tu32 nr_decl_tags;\n \tu32 max_decl_tags;\n \tstruct rb_root kfuncs;\n+\tstruct rb_root fmodrets;\n };\n \n static int verbose;\n@@ -1256,16 +1280,16 @@ static int push_decl_tag_id(struct btf2btf_context *ctx, u32 decl_tag_id)\n \treturn 0;\n }\n \n-static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)\n+static int push_set_func(struct rb_root *root, const char *kind, struct kfunc *kfunc)\n {\n-\tstruct rb_node **p = \u0026ctx-\u003ekfuncs.rb_node;\n+\tstruct rb_node **p = \u0026root-\u003erb_node;\n \tstruct rb_node *parent = NULL;\n \tstruct kfunc *k;\n \n \t/*\n-\t * Dedup by BTF ID: collecting the same kfunc twice is a no-op,\n-\t * UNLESS the kfunc flags are inconsistent, in which case we\n-\t * fail hard because it indicates a bug in a kfunc set declaration.\n+\t * Dedup by BTF ID: collecting the same function twice is a no-op,\n+\t * UNLESS its flags are inconsistent, in which case we fail hard\n+\t * because it indicates a bug in a BTF ID set declaration.\n \t */\n \twhile (*p) {\n \t\tparent = *p;\n@@ -1278,8 +1302,8 @@ static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)\n \t\t} else if (k-\u003eflags == kfunc-\u003eflags) {\n \t\t\treturn 0;\n \t\t} else {\n-\t\t\tpr_err(\"ERROR: resolve_btfids: kfunc %s has inconsistent flags across BTF ID sets: 0x%x != 0x%x\\n\",\n-\t\t\t kfunc-\u003ename, k-\u003eflags, kfunc-\u003eflags);\n+\t\t\tpr_err(\"ERROR: resolve_btfids: %s %s has inconsistent flags across BTF ID sets: 0x%x != 0x%x\\n\",\n+\t\t\t kind, kfunc-\u003ename, k-\u003eflags, kfunc-\u003eflags);\n \t\t\treturn -EINVAL;\n \t\t}\n \t}\n@@ -1290,7 +1314,7 @@ static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)\n \n \t*k = *kfunc;\n \trb_link_node(\u0026k-\u003erb_node, parent, p);\n-\trb_insert_color(\u0026k-\u003erb_node, \u0026ctx-\u003ekfuncs);\n+\trb_insert_color(\u0026k-\u003erb_node, root);\n \n \treturn 0;\n }\n@@ -1344,7 +1368,16 @@ static bool is_arena_param(const struct btf *btf, const struct btf_param *param)\n \t param_name_has_suffix(name, PARAM_SUFFIX_ARENA_NULLABLE);\n }\n \n-static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)\n+typedef int (*set_func_fn)(struct btf2btf_context *ctx, const char *name,\n+\t\t\t u32 btf_id, u32 flags);\n+\n+/*\n+ * Call FN for each function in the BTF ID sets which have SET_FLAG set. Reject\n+ * an entry with flags other than ALLOWED_FLAGS, also if its function is missing.\n+ */\n+static int collect_set_funcs(struct object *obj, struct btf2btf_context *ctx,\n+\t\t\t u32 set_flag, u32 allowed_flags, const char *kind,\n+\t\t\t set_func_fn fn)\n {\n \tElf_Data *idlist = obj-\u003eefile.idlist;\n \tstruct btf *btf = ctx-\u003ebtf;\n@@ -1356,24 +1389,39 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)\n \tfor (next = rb_first(\u0026obj-\u003esets); next; next = rb_next(next)) {\n \t\tstruct btf_id_set8 *set8;\n \t\tstruct btf_id *set_id;\n-\t\tu64 set_addr;\n+\t\tu64 set_addr, off;\n \n \t\tset_id = rb_entry(next, struct btf_id, rb_node);\n \t\tif (set_id-\u003ekind != BTF_ID_KIND_SET8 || set_id-\u003eaddr_cnt != 1)\n \t\t\tcontinue;\n \n \t\tset_addr = set_id-\u003eaddr[0];\n-\t\tset8 = idlist-\u003ed_buf + (set_addr - obj-\u003eefile.idlist_addr);\n-\t\tif (!(set8-\u003eflags \u0026 BTF_SET8_KFUNCS))\n+\t\toff = set_addr - obj-\u003eefile.idlist_addr;\n+\t\t/* A malformed set must not reach out of the section */\n+\t\tif (set_addr \u003c obj-\u003eefile.idlist_addr || off \u003e idlist-\u003ed_size ||\n+\t\t idlist-\u003ed_size - off \u003c sizeof(*set8) || set_id-\u003ecnt \u003c 0 ||\n+\t\t (u64)set_id-\u003ecnt \u003e (idlist-\u003ed_size - off - sizeof(*set8)) /\n+\t\t\t\t sizeof(set8-\u003epairs[0])) {\n+\t\t\tpr_err(\"ERROR: resolve_btfids: set %s is out of %s\\n\",\n+\t\t\t set_id-\u003ename, BTF_IDS_SECTION);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tset8 = idlist-\u003ed_buf + off;\n+\t\tif (!(set8-\u003eflags \u0026 set_flag))\n \t\t\tcontinue;\n \n \t\tfor (u32 i = 0; i \u003c set_id-\u003ecnt; i++) {\n \t\t\tsize_t off = (char *)\u0026set8-\u003epairs[i] - (char *)set8;\n \t\t\tconst char *name = find_name_by_addr(obj, set_addr + off);\n-\t\t\tstruct kfunc kfunc;\n+\t\t\tu32 flags = set8-\u003epairs[i].flags;\n \t\t\ts32 func_id;\n \t\t\tint err;\n \n+\t\t\tif (flags \u0026 ~allowed_flags) {\n+\t\t\t\tpr_err(\"ERROR: resolve_btfids: %s %s in %s has unsupported flags 0x%x\\n\",\n+\t\t\t\t kind, name ?: \"?\", set_id-\u003ename, flags \u0026 ~allowed_flags);\n+\t\t\t\treturn -EINVAL;\n+\t\t\t}\n \t\t\tif (!name) {\n \t\t\t\tpr_err(\"WARN: resolve_btfids: no BTF ID symbol for %s entry %u\\n\",\n \t\t\t\t set_id-\u003ename, i);\n@@ -1383,18 +1431,15 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)\n \n \t\t\tfunc_id = btf__find_by_name_kind_own(btf, name, BTF_KIND_FUNC);\n \t\t\tif (func_id \u003c 0) {\n-\t\t\t\tpr_err(\"WARN: resolve_btfids: no BTF func for kfunc %s in %s\\n\",\n-\t\t\t\t name, set_id-\u003ename);\n+\t\t\t\tpr_err(\"WARN: resolve_btfids: no BTF func for %s %s in %s\\n\",\n+\t\t\t\t kind, name, set_id-\u003ename);\n \t\t\t\twarnings++;\n \t\t\t\tcontinue;\n \t\t\t}\n \n-\t\t\tpr_debug(\"found kfunc %s in %s\\n\", name, set_id-\u003ename);\n+\t\t\tpr_debug(\"found %s %s in %s\\n\", kind, name, set_id-\u003ename);\n \n-\t\t\tkfunc.name = name;\n-\t\t\tkfunc.btf_id = func_id;\n-\t\t\tkfunc.flags = set8-\u003epairs[i].flags;\n-\t\t\terr = push_kfunc(ctx, \u0026kfunc);\n+\t\t\terr = fn(ctx, name, func_id, flags);\n \t\t\tif (err)\n \t\t\t\treturn err;\n \t\t}\n@@ -1403,6 +1448,33 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)\n \treturn 0;\n }\n \n+static int push_kfunc(struct btf2btf_context *ctx, const char *name,\n+\t\t u32 btf_id, u32 flags)\n+{\n+\tstruct kfunc kfunc = { .name = name, .btf_id = btf_id, .flags = flags };\n+\n+\treturn push_set_func(\u0026ctx-\u003ekfuncs, \"kfunc\", \u0026kfunc);\n+}\n+\n+static int push_fmodret(struct btf2btf_context *ctx, const char *name,\n+\t\t\tu32 btf_id, u32 flags)\n+{\n+\tstruct kfunc fmodret = { .name = name, .btf_id = btf_id, .flags = flags };\n+\n+\treturn push_set_func(\u0026ctx-\u003efmodrets, \"fmod_ret target\", \u0026fmodret);\n+}\n+\n+static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)\n+{\n+\treturn collect_set_funcs(obj, ctx, BTF_SET8_KFUNCS, ~0U, \"kfunc\", push_kfunc);\n+}\n+\n+static int collect_fmodrets(struct object *obj, struct btf2btf_context *ctx)\n+{\n+\treturn collect_set_funcs(obj, ctx, BTF_SET8_FMODRET, KF_SLEEPABLE,\n+\t\t\t\t \"fmod_ret target\", push_fmodret);\n+}\n+\n static int build_btf2btf_context(struct object *obj, struct btf2btf_context *ctx)\n {\n \tint err;\n@@ -1421,6 +1493,12 @@ static int build_btf2btf_context(struct object *obj, struct btf2btf_context *ctx\n \t\treturn err;\n \t}\n \n+\terr = collect_fmodrets(obj, ctx);\n+\tif (err) {\n+\t\tpr_err(\"ERROR: resolve_btfids: failed to collect fmod_ret targets from BTF\\n\");\n+\t\treturn err;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -1727,6 +1805,40 @@ static int add_decl_tag(struct btf2btf_context *ctx, const char *tag_name,\n \treturn push_decl_tag_id(ctx, new_id);\n }\n \n+static bool has_set_func(const struct rb_root *root, u32 btf_id)\n+{\n+\tconst struct rb_node *node = root-\u003erb_node;\n+\n+\twhile (node) {\n+\t\tconst struct kfunc *k = rb_entry(node, struct kfunc, rb_node);\n+\n+\t\tif (btf_id \u003c k-\u003ebtf_id)\n+\t\t\tnode = node-\u003erb_left;\n+\t\telse if (btf_id \u003e k-\u003ebtf_id)\n+\t\t\tnode = node-\u003erb_right;\n+\t\telse\n+\t\t\treturn true;\n+\t}\n+\treturn false;\n+}\n+\n+static int add_fmodret_retval_tags(struct btf2btf_context *ctx,\n+\t\t\t\t const struct fmodret_retval *retval)\n+{\n+\tchar *tag;\n+\tint err;\n+\n+\terr = add_decl_tag(ctx, DECL_TAG_FMODRET, retval-\u003ebtf_id, -1);\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (asprintf(\u0026tag, \"%s%s\", DECL_TAG_FMODRET_RETVAL, retval-\u003evalues) \u003c 0)\n+\t\treturn -ENOMEM;\n+\terr = add_decl_tag(ctx, tag, retval-\u003ebtf_id, -1);\n+\tfree(tag);\n+\treturn err;\n+}\n+\n static int btf2btf(struct object *obj)\n {\n \tstruct btf2btf_context ctx = {};\n@@ -1761,10 +1873,36 @@ static int btf2btf(struct object *obj)\n \t\t\tgoto out;\n \t}\n \n+\tfor (next = rb_first(\u0026ctx.fmodrets); next; next = rb_next(next)) {\n+\t\tstruct kfunc *fmodret = rb_entry(next, struct kfunc, rb_node);\n+\n+\t\terr = add_decl_tag(\u0026ctx, DECL_TAG_FMODRET, fmodret-\u003ebtf_id, -1);\n+\t\tif (err)\n+\t\t\tgoto out;\n+\n+\t\tif (fmodret-\u003eflags \u0026 KF_SLEEPABLE) {\n+\t\t\terr = add_decl_tag(\u0026ctx, DECL_TAG_FMODRET_SLEEPABLE,\n+\t\t\t\t\t fmodret-\u003ebtf_id, -1);\n+\t\t\tif (err)\n+\t\t\t\tgoto out;\n+\t\t}\n+\t}\n+\n+\tfor (size_t i = 0; i \u003c obj-\u003efmodret_retvals.cnt; i++) {\n+\t\tconst struct fmodret_retval *retval = \u0026obj-\u003efmodret_retvals.retvals[i];\n+\n+\t\tif (has_set_func(\u0026ctx.fmodrets, retval-\u003ebtf_id))\n+\t\t\tcontinue;\n+\t\terr = add_fmodret_retval_tags(\u0026ctx, retval);\n+\t\tif (err)\n+\t\t\tgoto out;\n+\t}\n+\n \terr = 0;\n out:\n \tfree(ctx.decl_tags);\n \tfree_kfuncs(\u0026ctx.kfuncs);\n+\tfree_kfuncs(\u0026ctx.fmodrets);\n \n \treturn err;\n }\n@@ -2113,6 +2251,8 @@ int main(int argc, const char **argv)\n \t\t\t \"distill --btf_base and emit .BTF.base section data\"),\n \t\tOPT_BOOLEAN(0, \"inline\", \u0026obj.extract_inline,\n \t\t\t \"extract location BTF into a .BTF.inline file\"),\n+\t\tOPT_BOOLEAN(0, \"patchable_ftrace\", \u0026obj.patchable_ftrace,\n+\t\t\t \"ftrace call sites are in __patchable_function_entries\"),\n \t\tOPT_CALLBACK(0, \"btf_link\", \u0026btf_links, \"section:module:btf-file\",\n \t\t\t \"patch a BTF link with --patch_btfids\", parse_btf_link),\n \t\tOPT_STRING(0, \"patch_btfids\", \u0026btfids_path, \"file\",\n@@ -2171,6 +2311,10 @@ int main(int argc, const char **argv)\n \tif (load_btf(\u0026obj))\n \t\tgoto out;\n \n+\tif (fmodret_collect(obj.efile.elf, obj.btf, \u0026obj.fmodret_retvals,\n+\t\t\t obj.patchable_ftrace, verbose))\n+\t\tgoto out;\n+\n \tif (btf2btf(\u0026obj))\n \t\tgoto out;\n \n@@ -2228,6 +2372,7 @@ int main(int argc, const char **argv)\n \tbtf_id__free_all(\u0026obj.funcs);\n \tbtf_id__free_all(\u0026obj.sets);\n \tfree(obj.addr_syms);\n+\tfmodret_free(\u0026obj.fmodret_retvals);\n \tfree(obj.efile.func_symbols);\n \tif (obj.efile.elf) {\n \t\telf_end(obj.efile.elf);\ndiff --git a/tools/include/linux/btf_ids.h b/tools/include/linux/btf_ids.h\nindex 4fe5c5f1558ca9..3b5b2856c6285f 100644\n--- a/tools/include/linux/btf_ids.h\n+++ b/tools/include/linux/btf_ids.h\n@@ -13,13 +13,21 @@ struct btf_id_set {\n /* This flag implies BTF_SET8 holds kfunc(s) */\n #define BTF_SET8_KFUNCS\t\t(1 \u003c\u003c 0)\n \n+/*\n+ * This flag implies BTF_SET8 holds functions which BPF fmod_ret, fentry and\n+ * fexit programs may attach to. Per-function flags: KF_SLEEPABLE only.\n+ */\n+#define BTF_SET8_FMODRET\t(1 \u003c\u003c 1)\n+\n+struct btf_id_set8_pair {\n+\tu32 id;\n+\tu32 flags;\n+};\n+\n struct btf_id_set8 {\n \tu32 cnt;\n \tu32 flags;\n-\tstruct {\n-\t\tu32 id;\n-\t\tu32 flags;\n-\t} pairs[];\n+\tstruct btf_id_set8_pair pairs[];\n };\n \n #ifdef CONFIG_DEBUG_INFO_BTF\n@@ -214,6 +222,12 @@ __BTF_SET8_START(name, local, BTF_SET8_KFUNCS)\n #define BTF_KFUNCS_END(name)\t\t\t\t\\\n BTF_SET8_END(name)\n \n+#define BTF_FMODRET_START(name)\t\t\t\t\\\n+__BTF_SET8_START(name, local, BTF_SET8_FMODRET)\n+\n+#define BTF_FMODRET_END(name)\t\t\t\t\\\n+BTF_SET8_END(name)\n+\n #else\n \n #define BTF_ID_LIST(name) static u32 __maybe_unused name[128];\n@@ -230,6 +244,8 @@ BTF_SET8_END(name)\n #define BTF_SET8_END(name)\n #define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };\n #define BTF_KFUNCS_END(name)\n+#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };\n+#define BTF_FMODRET_END(name)\n \n #endif /* CONFIG_DEBUG_INFO_BTF */\n \ndiff --git a/tools/objtool/check.c b/tools/objtool/check.c\nindex 931c5c412f15d8..e6120bee45052a 100644\n--- a/tools/objtool/check.c\n+++ b/tools/objtool/check.c\n@@ -4657,6 +4657,7 @@ static int validate_ibt(struct objtool_file *file)\n \t\t !strcmp(sec-\u003ename, \"__jump_table\")\t\t\t||\n \t\t !strcmp(sec-\u003ename, \".init.klp_funcs\")\t\t||\n \t\t !strcmp(sec-\u003ename, \"__mcount_loc\")\t\t\t||\n+\t\t !strcmp(sec-\u003ename, \".BTF_fmodret_candidates\")\t||\n \t\t !strcmp(sec-\u003ename, \".llvm.call-graph-profile\")\t||\n \t\t !strcmp(sec-\u003ename, \".llvm_bb_addr_map\")\t\t||\n \t\t !strcmp(sec-\u003ename, \"__tracepoints\")\t\t\t||\ndiff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile\nindex 77682bf28c6ece..84554a2adfc1bc 100644\n--- a/tools/testing/selftests/bpf/Makefile\n+++ b/tools/testing/selftests/bpf/Makefile\n@@ -47,7 +47,7 @@ TEST_PROGS_EXTENDED := \\\n \tima_setup.sh verify_sig_setup.sh\n \n TEST_KMODS := bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \\\n-\tbpf_test_modorder_y.ko bpf_test_rqspinlock.ko\n+\tbpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_retval.ko\n TEST_KMOD_TARGETS = $(addprefix $(OUTPUT)/,$(TEST_KMODS))\n \n # Compile but not part of 'make run_tests'\ndiff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config\nindex 49dfe939467cc8..5461c32cd3cd2c 100644\n--- a/tools/testing/selftests/bpf/config\n+++ b/tools/testing/selftests/bpf/config\n@@ -11,6 +11,7 @@ CONFIG_BPF_STREAM_PARSER=y\n CONFIG_BPF_SYSCALL=y\n # CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set\n CONFIG_CGROUP_BPF=y\n+CONFIG_CLANG_PLUGIN_BPF_SCAN_IR=y\n CONFIG_CRYPTO_HMAC=y\n CONFIG_CRYPTO_MLDSA=y\n CONFIG_CRYPTO_SHA256=y\n@@ -56,6 +57,7 @@ CONFIG_IPV6_TUNNEL=y\n CONFIG_KEYS=y\n CONFIG_LIRC=y\n CONFIG_LIVEPATCH=y\n+CONFIG_LLVM_PLUGINS=y\n CONFIG_LWTUNNEL=y\n CONFIG_LWTUNNEL_BPF=y\n CONFIG_MEMCG=y\ndiff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c\nnew file mode 100644\nindex 00000000000000..7b374fe6813027\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c\n@@ -0,0 +1,211 @@\n+// SPDX-License-Identifier: GPL-2.0\n+\n+#include \u003ctest_progs.h\u003e\n+#include \u003cbpf/btf.h\u003e\n+#include \"fmodret_retval.skel.h\"\n+#include \"fmodret_retval_ext.skel.h\"\n+\n+#define TAG_FMODRET\t\t\"bpf_fmodret\"\n+#define TAG_RETVAL\t\t\"bpf_fmodret_retval:\"\n+#define TAG_TEST_RETVAL\t\t\"test_retval:\"\n+\n+#define RETVAL_MODULE\t\t\"bpf_test_retval\"\n+\n+/* The decl tags of a function */\n+struct func_tags {\n+\tconst char *test_retval;\n+\tconst char *retval;\n+\tbool target;\n+};\n+\n+static bool tag_value(const char *name, const char *key, const char **value)\n+{\n+\tsize_t len = strlen(key);\n+\n+\tif (strncmp(name, key, len))\n+\t\treturn false;\n+\t*value = name + len;\n+\treturn true;\n+}\n+\n+/*\n+ * Return the decl tags of the functions of module BTF, indexed by BTF ID. Only\n+ * the types of the module itself are read, not those of its vmlinux base.\n+ */\n+static struct func_tags *collect_tags(const struct btf *btf, int start_id)\n+{\n+\tint id, nr_types = btf__type_cnt(btf);\n+\tstruct func_tags *tags;\n+\n+\ttags = calloc(nr_types, sizeof(*tags));\n+\tif (!tags)\n+\t\treturn NULL;\n+\tfor (id = start_id; id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\t\tstruct func_tags *f;\n+\t\tconst char *name;\n+\n+\t\tif (!btf_is_decl_tag(t) || btf_decl_tag(t)-\u003ecomponent_idx != -1 ||\n+\t\t t-\u003etype \u003e= nr_types)\n+\t\t\tcontinue;\n+\t\tf = \u0026tags[t-\u003etype];\n+\t\tname = btf__name_by_offset(btf, t-\u003ename_off);\n+\t\tif (!tag_value(name, TAG_TEST_RETVAL, \u0026f-\u003etest_retval) \u0026\u0026\n+\t\t !tag_value(name, TAG_RETVAL, \u0026f-\u003eretval) \u0026\u0026 !strcmp(name, TAG_FMODRET))\n+\t\t\tf-\u003etarget = true;\n+\t}\n+\n+\treturn tags;\n+}\n+\n+/* Compare the tags of every fixture with the values it expects */\n+static void test_tags(const struct btf *btf, const struct func_tags *tags, int start_id)\n+{\n+\tint id, nr_types = btf__type_cnt(btf), nr_fixtures = 0;\n+\n+\tfor (id = start_id; id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\t\tconst char *name;\n+\n+\t\tif (!btf_is_func(t) || !tags[id].test_retval)\n+\t\t\tcontinue;\n+\t\tname = btf__name_by_offset(btf, t-\u003ename_off);\n+\t\tASSERT_STREQ(tags[id].retval ?: \"\", tags[id].test_retval, name);\n+\t\tif (*tags[id].test_retval)\n+\t\t\tASSERT_TRUE(tags[id].target, name);\n+\t\telse if (strcmp(name, \"bpf_testmod_retval_declared\"))\n+\t\t\tASSERT_FALSE(tags[id].target, name);\n+\t\tnr_fixtures++;\n+\t}\n+\tASSERT_GT(nr_fixtures, 0, \"fixtures\");\n+\n+\tid = btf__find_by_name_kind(btf, \"bpf_testmod_retval_declared\", BTF_KIND_FUNC);\n+\tif (ASSERT_GE(id, start_id, \"declared\"))\n+\t\tASSERT_TRUE(tags[id].target, \"declared_target\");\n+}\n+\n+/* Open the programs again with only the extension, to replace FUNC of TARGET */\n+static struct fmodret_retval_ext *open_extension(struct bpf_program *target,\n+\t\t\t\t\t\t const char *func, char *log, size_t log_size)\n+{\n+\tstruct fmodret_retval_ext *skel;\n+\n+\tskel = fmodret_retval_ext__open();\n+\tif (!ASSERT_OK_PTR(skel, \"fmodret_retval_ext__open\"))\n+\t\treturn NULL;\n+\tbpf_program__set_autoload(skel-\u003eprogs.restricted, false);\n+\tbpf_program__set_autoload(skel-\u003eprogs.unrestricted, false);\n+\tif (!ASSERT_OK(bpf_program__set_attach_target(skel-\u003eprogs.extension,\n+\t\t\t\t\t\t bpf_program__fd(target), func),\n+\t\t \"set_attach_target\")) {\n+\t\tfmodret_retval_ext__destroy(skel);\n+\t\treturn NULL;\n+\t}\n+\tbpf_program__set_log_buf(skel-\u003eprogs.extension, log, log_size);\n+\treturn skel;\n+}\n+\n+/*\n+ * An extension could return any value for an fmod_ret program, so it must not\n+ * replace one whose return values are restricted, neither at load nor when it\n+ * is attached later to another program.\n+ */\n+static void test_extension(void)\n+{\n+\tstruct fmodret_retval_ext *targets, *skel = NULL;\n+\tstruct bpf_link *link;\n+\tchar log[4096] = {};\n+\n+\tif (!is_jit_enabled()) {\n+\t\tprintf(\"%s:SKIP:extensions need the JIT\\n\", __func__);\n+\t\ttest__skip();\n+\t\treturn;\n+\t}\n+\n+\ttargets = fmodret_retval_ext__open();\n+\tif (!ASSERT_OK_PTR(targets, \"fmodret_retval_ext__open\"))\n+\t\treturn;\n+\tbpf_program__set_autoload(targets-\u003eprogs.extension, false);\n+\tif (!ASSERT_OK(fmodret_retval_ext__load(targets), \"load_targets\"))\n+\t\tgoto out;\n+\n+\tskel = open_extension(targets-\u003eprogs.restricted, \"restricted_value\", log, sizeof(log));\n+\tif (!skel)\n+\t\tgoto out;\n+\tASSERT_ERR(fmodret_retval_ext__load(skel), \"load_restricted\");\n+\tASSERT_HAS_SUBSTR(log, \"Cannot extend an fmod_ret program with restricted return values\",\n+\t\t\t \"log_restricted\");\n+\tfmodret_retval_ext__destroy(skel);\n+\n+\tskel = open_extension(targets-\u003eprogs.unrestricted, \"unrestricted_value\", log, sizeof(log));\n+\tif (!skel)\n+\t\tgoto out;\n+\tif (!ASSERT_OK(fmodret_retval_ext__load(skel), \"load_unrestricted\"))\n+\t\tgoto out;\n+\tlink = bpf_program__attach_freplace(skel-\u003eprogs.extension,\n+\t\t\t\t\t bpf_program__fd(targets-\u003eprogs.restricted),\n+\t\t\t\t\t \"restricted_value\");\n+\tif (!ASSERT_ERR_PTR(link, \"attach_restricted\"))\n+\t\tbpf_link__destroy(link);\n+\tlink = bpf_program__attach_freplace(skel-\u003eprogs.extension,\n+\t\t\t\t\t bpf_program__fd(targets-\u003eprogs.unrestricted),\n+\t\t\t\t\t \"unrestricted_value\");\n+\tif (ASSERT_OK_PTR(link, \"attach_unrestricted\"))\n+\t\tbpf_link__destroy(link);\n+out:\n+\tfmodret_retval_ext__destroy(skel);\n+\tfmodret_retval_ext__destroy(targets);\n+}\n+\n+void test_fmodret_retval(void)\n+{\n+\tstruct btf *vmlinux_btf, *btf = NULL;\n+\tstruct func_tags *tags = NULL;\n+\tint id, start_id, nr_types;\n+\tbool has_fixtures = false;\n+\n+\tif (!env.has_testmod) {\n+\t\ttest__skip();\n+\t\treturn;\n+\t}\n+\n+\tif (!ASSERT_OK(load_module(RETVAL_MODULE \".ko\", env_verbosity \u003e VERBOSE_NONE),\n+\t\t \"load_module\"))\n+\t\treturn;\n+\tvmlinux_btf = btf__load_vmlinux_btf();\n+\tif (!ASSERT_OK_PTR(vmlinux_btf, \"btf__load_vmlinux_btf\"))\n+\t\tgoto out;\n+\tbtf = btf__load_module_btf(RETVAL_MODULE, vmlinux_btf);\n+\tif (!btf \u0026\u0026 errno == ENOENT) {\n+\t\tprintf(\"%s:SKIP:%s has no BTF\\n\", __func__, RETVAL_MODULE);\n+\t\ttest__skip();\n+\t\tgoto out;\n+\t}\n+\tif (!ASSERT_OK_PTR(btf, \"btf__load_module_btf\"))\n+\t\tgoto out;\n+\tstart_id = btf__type_cnt(vmlinux_btf);\n+\tnr_types = btf__type_cnt(btf);\n+\ttags = collect_tags(btf, start_id);\n+\tif (!ASSERT_OK_PTR(tags, \"collect_tags\"))\n+\t\tgoto out;\n+\n+\t/* The fixtures are built only with the compiler plugin */\n+\tfor (id = start_id; id \u003c nr_types \u0026\u0026 !has_fixtures; id++)\n+\t\thas_fixtures = tags[id].test_retval;\n+\tif (!has_fixtures) {\n+\t\tprintf(\"%s:SKIP:%s has no %s tags\\n\", __func__, RETVAL_MODULE, TAG_TEST_RETVAL);\n+\t\ttest__skip();\n+\t\tgoto out;\n+\t}\n+\n+\tif (test__start_subtest(\"tags\"))\n+\t\ttest_tags(btf, tags, start_id);\n+\tif (test__start_subtest(\"extension\"))\n+\t\ttest_extension();\n+\tRUN_TESTS(fmodret_retval);\n+out:\n+\tfree(tags);\n+\tbtf__free(btf);\n+\tbtf__free(vmlinux_btf);\n+\tunload_module(RETVAL_MODULE, env_verbosity \u003e VERBOSE_NONE);\n+}\ndiff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c\nnew file mode 100644\nindex 00000000000000..62efe11af826fb\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c\n@@ -0,0 +1,155 @@\n+// SPDX-License-Identifier: GPL-2.0\n+\n+#include \u003ctest_progs.h\u003e\n+#include \u003clinux/filter.h\u003e\n+#include \u003cbpf/btf.h\u003e\n+\n+#define TAG_FMODRET\t\t\"bpf_fmodret\"\n+#define TAG_FMODRET_SLEEPABLE\t\"bpf_fmodret_sleepable\"\n+\n+static bool func_has_tag(const struct btf *btf, const char *func, const char *tag)\n+{\n+\tint func_id, id, nr_types;\n+\n+\tfunc_id = btf__find_by_name_kind(btf, func, BTF_KIND_FUNC);\n+\tif (func_id \u003c 0)\n+\t\treturn false;\n+\n+\tnr_types = btf__type_cnt(btf);\n+\tfor (id = 1; id \u003c nr_types; id++) {\n+\t\tconst struct btf_type *t = btf__type_by_id(btf, id);\n+\n+\t\tif (!btf_is_decl_tag(t) || t-\u003etype != func_id ||\n+\t\t btf_decl_tag(t)-\u003ecomponent_idx != -1)\n+\t\t\tcontinue;\n+\t\tif (!strcmp(btf__name_by_offset(btf, t-\u003ename_off), tag))\n+\t\t\treturn true;\n+\t}\n+\n+\treturn false;\n+}\n+\n+static void test_vmlinux_tags(void)\n+{\n+\tstruct btf *btf;\n+\n+\tbtf = btf__load_vmlinux_btf();\n+\tif (!ASSERT_OK_PTR(btf, \"btf__load_vmlinux_btf\"))\n+\t\treturn;\n+\n+\tASSERT_TRUE(func_has_tag(btf, \"bpf_modify_return_test\", TAG_FMODRET), \"modify_return_test\");\n+\tASSERT_TRUE(func_has_tag(btf, \"bpf_modify_return_test2\", TAG_FMODRET),\n+\t\t \"modify_return_test2\");\n+\tASSERT_TRUE(func_has_tag(btf, \"bpf_modify_return_test_tp\", TAG_FMODRET),\n+\t\t \"modify_return_test_tp\");\n+\tASSERT_TRUE(func_has_tag(btf, \"bpf_fentry_test1\", TAG_FMODRET), \"fentry_test1\");\n+\tASSERT_TRUE(func_has_tag(btf, \"update_socket_protocol\", TAG_FMODRET),\n+\t\t \"update_socket_protocol\");\n+\n+\t/* Only bpf_fentry_test1 is a sleepable target */\n+\tASSERT_TRUE(func_has_tag(btf, \"bpf_fentry_test1\", TAG_FMODRET_SLEEPABLE),\n+\t\t \"fentry_test1_sleepable\");\n+\tASSERT_FALSE(func_has_tag(btf, \"bpf_modify_return_test\", TAG_FMODRET_SLEEPABLE),\n+\t\t \"modify_return_test_sleepable\");\n+\tASSERT_FALSE(func_has_tag(btf, \"bpf_modify_return_test2\", TAG_FMODRET_SLEEPABLE),\n+\t\t \"modify_return_test2_sleepable\");\n+\tASSERT_FALSE(func_has_tag(btf, \"bpf_modify_return_test_tp\", TAG_FMODRET_SLEEPABLE),\n+\t\t \"modify_return_test_tp_sleepable\");\n+\tASSERT_FALSE(func_has_tag(btf, \"update_socket_protocol\", TAG_FMODRET_SLEEPABLE),\n+\t\t \"update_socket_protocol_sleepable\");\n+\n+\t/* A function which is not a target has no tag */\n+\tASSERT_GT(btf__find_by_name_kind(btf, \"bpf_fentry_test2\", BTF_KIND_FUNC), 0,\n+\t\t \"fentry_test2_exists\");\n+\tASSERT_FALSE(func_has_tag(btf, \"bpf_fentry_test2\", TAG_FMODRET), \"fentry_test2\");\n+\tASSERT_FALSE(func_has_tag(btf, \"bpf_fentry_test2\", TAG_FMODRET_SLEEPABLE),\n+\t\t \"fentry_test2_sleepable\");\n+\n+\tbtf__free(btf);\n+}\n+\n+static void test_module_tags(void)\n+{\n+\tstruct btf *vmlinux_btf, *module_btf;\n+\n+\tif (!env.has_testmod) {\n+\t\ttest__skip();\n+\t\treturn;\n+\t}\n+\n+\tvmlinux_btf = btf__load_vmlinux_btf();\n+\tif (!ASSERT_OK_PTR(vmlinux_btf, \"btf__load_vmlinux_btf\"))\n+\t\treturn;\n+\tmodule_btf = btf__load_module_btf(\"bpf_testmod\", vmlinux_btf);\n+\tif (!module_btf \u0026\u0026 errno == ENOENT) {\n+\t\tprintf(\"%s:SKIP:bpf_testmod has no BTF\\n\", __func__);\n+\t\ttest__skip();\n+\t\tgoto out;\n+\t}\n+\tif (!ASSERT_OK_PTR(module_btf, \"btf__load_module_btf\"))\n+\t\tgoto out;\n+\n+\tASSERT_TRUE(func_has_tag(module_btf, \"bpf_testmod_trampoline_count_test\", TAG_FMODRET),\n+\t\t \"trampoline_count_test\");\n+\tASSERT_FALSE(func_has_tag(module_btf, \"bpf_testmod_trampoline_count_test\",\n+\t\t\t\t TAG_FMODRET_SLEEPABLE), \"trampoline_count_test_sleepable\");\n+\tASSERT_FALSE(func_has_tag(module_btf, \"bpf_testmod_fentry_test1\", TAG_FMODRET),\n+\t\t \"testmod_fentry_test1\");\n+\n+\tbtf__free(module_btf);\n+out:\n+\tbtf__free(vmlinux_btf);\n+}\n+\n+/* Load a minimal fmod_ret program which attaches to function BTF_ID */\n+static int load_fmod_ret(int btf_id, __u32 prog_flags)\n+{\n+\tstruct bpf_insn insns[] = {\n+\t\tBPF_MOV64_IMM(BPF_REG_0, 0),\n+\t\tBPF_EXIT_INSN(),\n+\t};\n+\tLIBBPF_OPTS(bpf_prog_load_opts, opts,\n+\t\t.expected_attach_type = BPF_MODIFY_RETURN,\n+\t\t.prog_flags = prog_flags,\n+\t\t.attach_btf_id = btf_id,\n+\t);\n+\n+\treturn bpf_prog_load(BPF_PROG_TYPE_TRACING, NULL, \"GPL\", insns, ARRAY_SIZE(insns), \u0026opts);\n+}\n+\n+static void check_load(const char *func, __u32 prog_flags, bool allowed, const char *name)\n+{\n+\tint btf_id, fd;\n+\n+\tbtf_id = libbpf_find_vmlinux_btf_id(func, BPF_MODIFY_RETURN);\n+\tif (!ASSERT_GT(btf_id, 0, func))\n+\t\treturn;\n+\tfd = load_fmod_ret(btf_id, prog_flags);\n+\tif (allowed)\n+\t\tASSERT_GE(fd, 0, name);\n+\telse\n+\t\tASSERT_LT(fd, 0, name);\n+\tif (fd \u003e= 0)\n+\t\tclose(fd);\n+}\n+\n+static void test_attach_permission(void)\n+{\n+\tcheck_load(\"bpf_modify_return_test\", 0, true, \"target\");\n+\tcheck_load(\"bpf_fentry_test1\", 0, true, \"sleepable_target\");\n+\tcheck_load(\"bpf_fentry_test2\", 0, false, \"not_a_target\");\n+\n+\tcheck_load(\"bpf_fentry_test1\", BPF_F_SLEEPABLE, true, \"sleepable_prog_sleepable_target\");\n+\tcheck_load(\"bpf_modify_return_test\", BPF_F_SLEEPABLE, false,\n+\t\t \"sleepable_prog_nonsleepable_target\");\n+}\n+\n+void test_fmodret_tags(void)\n+{\n+\tif (test__start_subtest(\"vmlinux\"))\n+\t\ttest_vmlinux_tags();\n+\tif (test__start_subtest(\"module\"))\n+\t\ttest_module_tags();\n+\tif (test__start_subtest(\"attach_permission\"))\n+\t\ttest_attach_permission();\n+}\ndiff --git a/tools/testing/selftests/bpf/progs/fmodret_retval.c b/tools/testing/selftests/bpf/progs/fmodret_retval.c\nnew file mode 100644\nindex 00000000000000..bf5c6f53a99936\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/fmodret_retval.c\n@@ -0,0 +1,196 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * fmod_ret programs on the targets which the LLVM IR scanner finds in\n+ * bpf_test_retval, see test_kmods/bpf_testmod_retval.c. They may return only\n+ * 0 or the values of the \"bpf_fmodret_retval:\" tag of their target.\n+ */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \u003cbpf/bpf_tracing.h\u003e\n+#include \"bpf_misc.h\"\n+\n+char _license[] SEC(\"license\") = \"GPL\";\n+\n+/* bpf_testmod_retval_constant() returns -EINVAL */\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__success\n+int BPF_PROG(constant_einval, int ret)\n+{\n+\treturn -22;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__success\n+int BPF_PROG(constant_zero, int ret)\n+{\n+\treturn 0;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__failure __msg(\"it should be 0 or one of -22\")\n+int BPF_PROG(constant_other, int ret)\n+{\n+\treturn -1;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__failure __msg(\"it should be 0 or one of -22\")\n+__naked void constant_unknown(void)\n+{\n+\tasm volatile (\n+\t\"call %[bpf_get_prandom_u32];\"\n+\t\"exit;\"\n+\t:\n+\t: __imm(bpf_get_prandom_u32)\n+\t: __clobber_all);\n+}\n+\n+/* The function returns int, so only the low 32 bits count */\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__success\n+__naked void constant_32bit(void)\n+{\n+\tasm volatile (\n+\t\"w0 = -22;\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+/* Not 0, so the function is not called, but the caller would see 0 */\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__failure __msg(\"it should be 0 or one of -22\")\n+__naked void constant_high_bits(void)\n+{\n+\tasm volatile (\n+\t\"r0 = 1;\"\n+\t\"r0 \u003c\u003c= 32;\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+/* bpf_testmod_retval_join() returns -EINVAL or -ERANGE */\n+SEC(\"fmod_ret/bpf_testmod_retval_join\")\n+__success\n+int BPF_PROG(join_einval, int *result, int value, int ret)\n+{\n+\treturn -22;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_join\")\n+__success\n+int BPF_PROG(join_erange, int *result, int value, int ret)\n+{\n+\treturn -34;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_join\")\n+__failure __msg(\"it should be 0 or one of -34,-22\")\n+int BPF_PROG(join_other, int *result, int value, int ret)\n+{\n+\treturn -1;\n+}\n+\n+/* Each path returns an allowed value */\n+SEC(\"fmod_ret/bpf_testmod_retval_join\")\n+__success\n+__naked void join_both_paths(void)\n+{\n+\tasm volatile (\n+\t\"r2 = *(u64 *)(r1 + 8);\"\n+\t\"if r2 s\u003c 0 goto 1f;\"\n+\t\"r0 = -34;\"\n+\t\"exit;\"\n+\"1:\"\n+\t\"r0 = -22;\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+/*\n+ * One of two paths to the same exit returns a forbidden value. The verifier\n+ * checks the allowed one first and must not prune the other.\n+ */\n+SEC(\"fmod_ret/bpf_testmod_retval_join\")\n+__failure __msg(\"it should be 0 or one of -34,-22\")\n+__naked void join_one_forbidden(void)\n+{\n+\tasm volatile (\n+\t\"r2 = *(u64 *)(r1 + 8);\"\n+\t\"if r2 s\u003c 0 goto 1f;\"\n+\t\"r0 = -34;\"\n+\t\"goto 2f;\"\n+\"1:\"\n+\t\"r0 = -1;\"\n+\"2:\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+/* bpf_testmod_retval_wide() returns s64, so all 64 bits count */\n+SEC(\"fmod_ret/bpf_testmod_retval_wide\")\n+__success\n+__load_if_JITed()\n+__naked void wide_64bit(void)\n+{\n+\tasm volatile (\n+\t\"r0 = -14;\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_wide\")\n+__failure __msg(\"it should be 0 or one of -14\")\n+__naked void wide_32bit(void)\n+{\n+\tasm volatile (\n+\t\"w0 = -14;\"\n+\t\"exit;\"\n+\t::: __clobber_all);\n+}\n+\n+/* Declared targets are not restricted, in module and vmlinux BTF */\n+SEC(\"fmod_ret/bpf_testmod_retval_declared\")\n+__success\n+int BPF_PROG(declared_module_any, int ret)\n+{\n+\treturn -1;\n+}\n+\n+SEC(\"fmod_ret/bpf_modify_return_test\")\n+__success\n+int BPF_PROG(declared_vmlinux_any, int a, int *b, int ret)\n+{\n+\treturn -1;\n+}\n+\n+/* The error injection list allows any value although the target has a tag */\n+SEC(\"fmod_ret/bpf_testmod_retval_injectable\")\n+__success\n+int BPF_PROG(static_rule_tagged, int value, int ret)\n+{\n+\treturn -1;\n+}\n+\n+struct {\n+\t__uint(type, BPF_MAP_TYPE_PROG_ARRAY);\n+\t__uint(max_entries, 1);\n+\t__uint(key_size, sizeof(__u32));\n+\t__uint(value_size, sizeof(__u32));\n+} jmp_table SEC(\".maps\");\n+\n+/* A program in a program array may return any value */\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+__failure __msg(\"tail calls are not allowed in fmod_ret programs with restricted return values\")\n+int BPF_PROG(constant_tail_call, int ret)\n+{\n+\tbpf_tail_call(ctx, \u0026jmp_table, 0);\n+\treturn 0;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_declared\")\n+__success\n+int BPF_PROG(declared_tail_call, int ret)\n+{\n+\tbpf_tail_call(ctx, \u0026jmp_table, 0);\n+\treturn 0;\n+}\ndiff --git a/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c\nnew file mode 100644\nindex 00000000000000..dd105eea05ee46\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c\n@@ -0,0 +1,53 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * An extension of fmod_ret programs, see prog_tests/fmodret_retval.c. The\n+ * return values of the program on bpf_testmod_retval_constant() are restricted\n+ * by its \"bpf_fmodret_retval:\" tag, those of the program on the declared target\n+ * are not. An extension cannot replace the main function of a tracing program,\n+ * so each of them calls a global function which the extension replaces.\n+ */\n+#include \u003cvmlinux.h\u003e\n+#include \u003cbpf/bpf_helpers.h\u003e\n+#include \u003cbpf/bpf_tracing.h\u003e\n+\n+char _license[] SEC(\"license\") = \"GPL\";\n+\n+/* Keeps clang from folding the calls */\n+int value_bias;\n+\n+__noinline int restricted_value(int value)\n+{\n+\treturn value + value_bias;\n+}\n+\n+__noinline int unrestricted_value(int value)\n+{\n+\treturn value + value_bias;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_constant\")\n+int BPF_PROG(restricted, int ret)\n+{\n+\tint zero = 0;\n+\n+\tif (restricted_value(ret))\n+\t\treturn -22;\n+\t/*\n+\t * fmod_ret BPF progs must return constants (0 or -22 in this case).\n+\t * The barrier here forces GCC to return a constant\n+\t */\n+\tbarrier_var(zero);\n+\treturn zero;\n+}\n+\n+SEC(\"fmod_ret/bpf_testmod_retval_declared\")\n+int BPF_PROG(unrestricted, int ret)\n+{\n+\treturn unrestricted_value(ret);\n+}\n+\n+SEC(\"freplace\")\n+int extension(int value)\n+{\n+\treturn -1;\n+}\ndiff --git a/tools/testing/selftests/bpf/progs/fsession_test.c b/tools/testing/selftests/bpf/progs/fsession_test.c\nindex 86e8a2fe467e20..4295f6a78dd128 100644\n--- a/tools/testing/selftests/bpf/progs/fsession_test.c\n+++ b/tools/testing/selftests/bpf/progs/fsession_test.c\n@@ -4,6 +4,8 @@\n #include \u003cbpf/bpf_helpers.h\u003e\n #include \u003cbpf/bpf_tracing.h\u003e\n \n+extern int bpf_fentry_test1(int a) __ksym;\n+\n char _license[] SEC(\"license\") = \"GPL\";\n \n __u64 test1_entry_result = 0;\ndiff --git a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c\nindex bbeea0d512e338..51a7f605f9c5d7 100644\n--- a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c\n+++ b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c\n@@ -3,6 +3,8 @@\n #include \u003cbpf/bpf_helpers.h\u003e\n #include \u003cbpf/bpf_tracing.h\u003e\n \n+extern int bpf_fentry_test1(int a) __ksym;\n+\n char _license[] SEC(\"license\") = \"GPL\";\n \n __u64 test1_entry_result = 0;\ndiff --git a/tools/testing/selftests/bpf/progs/tracing_multi_check.c b/tools/testing/selftests/bpf/progs/tracing_multi_check.c\nindex b2959ba711790e..9b008ecd65651e 100644\n--- a/tools/testing/selftests/bpf/progs/tracing_multi_check.c\n+++ b/tools/testing/selftests/bpf/progs/tracing_multi_check.c\n@@ -8,7 +8,7 @@ char _license[] SEC(\"license\") = \"GPL\";\n int pid = 0;\n bool test_cookies = false;\n \n-/* bpf_fentry_test1 is exported as kfunc via vmlinux.h */\n+extern int bpf_fentry_test1(int a) __ksym;\n extern const void bpf_fentry_test2 __ksym;\n extern const void bpf_fentry_test3 __ksym;\n extern const void bpf_fentry_test4 __ksym;\ndiff --git a/tools/testing/selftests/bpf/test_kmods/Makefile b/tools/testing/selftests/bpf/test_kmods/Makefile\nindex 031c7454ce65f6..f4af3c3ab54c2d 100644\n--- a/tools/testing/selftests/bpf/test_kmods/Makefile\n+++ b/tools/testing/selftests/bpf/test_kmods/Makefile\n@@ -19,10 +19,23 @@ Q = @\n endif\n \n MODULES = bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \\\n-\tbpf_test_modorder_y.ko bpf_test_rqspinlock.ko\n+\tbpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_retval.ko\n \n $(foreach m,$(MODULES),$(eval obj-m += $(m:.ko=.o)))\n \n+bpf_test_retval-y := bpf_test_retval_main.o\n+# The functions for bpf_scan_ir use clang extensions and need the plugin\n+bpf_test_retval-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_testmod_retval.o \\\n+\t\t\t\t\t\t bpf_testmod_retval_strong.o\n+# Instrumentation and branch profiling make every path of the fixtures dirty\n+GCOV_PROFILE_bpf_testmod_retval.o := n\n+KCOV_INSTRUMENT_bpf_testmod_retval.o := n\n+KMSAN_SANITIZE_bpf_testmod_retval.o := n\n+KASAN_SANITIZE_bpf_testmod_retval.o := n\n+KCSAN_SANITIZE_bpf_testmod_retval.o := n\n+UBSAN_SANITIZE_bpf_testmod_retval.o := n\n+CFLAGS_bpf_testmod_retval.o += -DDISABLE_BRANCH_PROFILING\n+\n CFLAGS_bpf_testmod.o = -I$(src)\n \n # When BPF_STRICT_BUILD != 0, a missing KDIR is fatal (the default).\ndiff --git a/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c b/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c\nnew file mode 100644\nindex 00000000000000..c3a9093fa3310c\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c\n@@ -0,0 +1,12 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * A module for the functions of bpf_testmod_retval.c and\n+ * bpf_testmod_retval_strong.c, see prog_tests/fmodret_retval.c. They are not\n+ * in bpf_testmod: pahole drops the per-CPU variables of a module which GNU ld\n+ * links from several objects.\n+ */\n+#include \u003clinux/module.h\u003e\n+\n+MODULE_AUTHOR(\"Anton Protopopov\");\n+MODULE_DESCRIPTION(\"BPF selftest module for fmod_ret targets found by bpf_scan_ir\");\n+MODULE_LICENSE(\"GPL\");\ndiff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c\nindex 41ff36f0ca77e9..91cb311a2dc8b1 100644\n--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c\n+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c\n@@ -2607,15 +2607,9 @@ struct bpf_struct_ops testmod_multi_st_ops = {\n \n extern int bpf_fentry_test1(int a);\n \n-BTF_KFUNCS_START(bpf_testmod_trampoline_count_ids)\n+BTF_FMODRET_START(bpf_testmod_trampoline_count_ids)\n BTF_ID_FLAGS(func, bpf_testmod_trampoline_count_test)\n-BTF_KFUNCS_END(bpf_testmod_trampoline_count_ids)\n-\n-static const struct\n-btf_kfunc_id_set bpf_testmod_trampoline_count_fmodret_set = {\n-\t.owner = THIS_MODULE,\n-\t.set = \u0026bpf_testmod_trampoline_count_ids,\n-};\n+BTF_FMODRET_END(bpf_testmod_trampoline_count_ids)\n \n static int bpf_testmod_init(void)\n {\n@@ -2633,7 +2627,6 @@ static int bpf_testmod_init(void)\n \tret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, \u0026bpf_testmod_kfunc_set);\n \tret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, \u0026bpf_testmod_kfunc_set);\n \tret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, \u0026bpf_testmod_kfunc_set);\n-\tret = ret ?: register_btf_fmodret_id_set(\u0026bpf_testmod_trampoline_count_fmodret_set);\n \tret = ret ?: register_bpf_struct_ops(\u0026bpf_bpf_testmod_ops, bpf_testmod_ops);\n \tret = ret ?: register_bpf_struct_ops(\u0026bpf_testmod_ops2, bpf_testmod_ops2);\n \tret = ret ?: register_bpf_struct_ops(\u0026bpf_testmod_ops3, bpf_testmod_ops3);\ndiff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c\nnew file mode 100644\nindex 00000000000000..fd63c6317cb944\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c\n@@ -0,0 +1,311 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Functions for the fmod_ret targets which the LLVM IR scanner finds, see\n+ * prog_tests/fmodret_retval.c. The \"test_retval:\" decl tag of each lists the\n+ * values the scanner must find on paths without side effects. An empty list\n+ * means that the function must not become such a target.\n+ */\n+#include \u003clinux/atomic.h\u003e\n+#include \u003clinux/btf_ids.h\u003e\n+#include \u003clinux/compiler.h\u003e\n+#include \u003clinux/err.h\u003e\n+#include \u003clinux/errno.h\u003e\n+#include \u003clinux/error-injection.h\u003e\n+#include \u003clinux/spinlock.h\u003e\n+#include \u003clinux/types.h\u003e\n+\n+#define RETVAL(values)\t__attribute__((btf_decl_tag(\"test_retval:\" values)))\n+\n+static __used int bpf_testmod_retval_state;\n+static __used int *bpf_testmod_retval_captured;\n+\n+static noinline void bpf_testmod_retval_effect(void)\n+{\n+\tWRITE_ONCE(bpf_testmod_retval_state, 1);\n+}\n+\n+static noinline void bpf_testmod_retval_capture(int *ptr)\n+{\n+\tWRITE_ONCE(bpf_testmod_retval_captured, ptr);\n+}\n+\n+static noinline int bpf_testmod_retval_store(int value)\n+{\n+\tWRITE_ONCE(bpf_testmod_retval_state, value);\n+\treturn value;\n+}\n+\n+static noinline int bpf_testmod_retval_clear(int value)\n+{\n+\tWRITE_ONCE(bpf_testmod_retval_state, 0);\n+\treturn value - 1;\n+}\n+\n+/* Constant errors and both return widths */\n+static __used noinline RETVAL(\"-22\") int bpf_testmod_retval_constant(void)\n+{\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"-14\") s64 bpf_testmod_retval_wide(void)\n+{\n+\treturn -EFAULT;\n+}\n+\n+static __used noinline RETVAL(\"-1\") int bpf_testmod_retval_first_errno(void)\n+{\n+\treturn -1;\n+}\n+\n+static __used noinline RETVAL(\"-4095\") int bpf_testmod_retval_last_errno(void)\n+{\n+\treturn -MAX_ERRNO;\n+}\n+\n+/*\n+ * PTR_ERR() of the ERR_PTR() of an inlined helper. Not recognized yet: the\n+ * error reaches the return through a PHI in an earlier block.\n+ */\n+static __always_inline int *bpf_testmod_retval_slot(int *buf, int idx)\n+{\n+\tif (idx \u003c 0)\n+\t\treturn ERR_PTR(-EINVAL);\n+\tif (idx \u003e 8)\n+\t\treturn ERR_PTR(-EFAULT);\n+\treturn \u0026buf[idx];\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_err_ptr(int *buf, int idx)\n+{\n+\tint *slot = bpf_testmod_retval_slot(buf, idx);\n+\n+\tif (IS_ERR(slot))\n+\t\treturn PTR_ERR(slot);\n+\t*slot = 1;\n+\treturn 0;\n+}\n+\n+/* A clean error path may join a dirty success path */\n+static __used noinline RETVAL(\"-22\") int bpf_testmod_retval_before_store(int *result, int value)\n+{\n+\tif (value \u003c 0)\n+\t\treturn -EINVAL;\n+\t*result = value;\n+\treturn 0;\n+}\n+\n+static __used noinline RETVAL(\"-34,-22\") int bpf_testmod_retval_join(int *result, int value)\n+{\n+\tint ret;\n+\n+\tif (value \u003c 0)\n+\t\tret = -EINVAL;\n+\telse if (value \u003e 32)\n+\t\tret = -ERANGE;\n+\telse {\n+\t\t*result = value;\n+\t\tret = 0;\n+\t}\n+\treturn ret;\n+}\n+\n+/*\n+ * An int error which a long function returns: the PHI of the two calls keeps\n+ * the sign extension in the block of the return\n+ */\n+static __used noinline RETVAL(\"-22\") long bpf_testmod_retval_sign_extend(int value)\n+{\n+\tint ret;\n+\n+\tif (value \u003c 0)\n+\t\tret = -EINVAL;\n+\telse if (value \u003e 32)\n+\t\tret = bpf_testmod_retval_store(value);\n+\telse\n+\t\tret = bpf_testmod_retval_clear(value);\n+\treturn ret;\n+}\n+\n+/* An assumption does not make a path dirty */\n+static __used noinline RETVAL(\"-22\") int bpf_testmod_retval_assume(int value)\n+{\n+\t__builtin_assume(value \u003e 0);\n+\treturn -EINVAL;\n+}\n+\n+/* Writes to private stack objects are clean, optnone keeps them in the IR */\n+static __used noinline RETVAL(\"-22\") __attribute__((optnone))\n+int bpf_testmod_retval_private_store(int value)\n+{\n+\tint scratch[2];\n+\n+\tscratch[0] = value;\n+\tscratch[1] = 0;\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"-22\") __attribute__((optnone))\n+int bpf_testmod_retval_private_memset(void)\n+{\n+\tchar scratch[32];\n+\n+\t__builtin_memset(scratch, 0, sizeof(scratch));\n+\treturn -EINVAL;\n+}\n+\n+/* An error which is not known exactly is not recorded */\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_masked(unsigned int value)\n+{\n+\treturn -1 - (value \u0026 31);\n+}\n+\n+static __used noinline RETVAL(\"\") long bpf_testmod_retval_ptr_err(void *ptr, int *state)\n+{\n+\tif (IS_ERR(ptr))\n+\t\treturn PTR_ERR(ptr);\n+\t*state = 1;\n+\treturn 0;\n+}\n+\n+/* Pointer functions are not targets yet */\n+static __used noinline RETVAL(\"\") void *bpf_testmod_retval_error_pointer(void)\n+{\n+\treturn ERR_PTR(-ENOMEM);\n+}\n+\n+/* Every error path has an observable effect */\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_argument_store(int *state)\n+{\n+\t*state = 1;\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_global_store(void)\n+{\n+\tbpf_testmod_retval_state = 1;\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_volatile_read(int *state)\n+{\n+\treturn READ_ONCE(*state) ? -EINVAL : -ERANGE;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_atomic(atomic_t *state)\n+{\n+\tatomic_inc(state);\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_call(void)\n+{\n+\tbpf_testmod_retval_effect();\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_barrier(void)\n+{\n+\tbarrier();\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\")\n+int bpf_testmod_retval_only_error_is_dirty(int *state, bool reject)\n+{\n+\tif (reject) {\n+\t\t*state = 1;\n+\t\treturn -EINVAL;\n+\t}\n+\treturn 0;\n+}\n+\n+static __used noinline RETVAL(\"\")\n+int bpf_testmod_retval_lock_held(spinlock_t *lock) __acquires(lock)\n+{\n+\tspin_lock(lock);\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_stack_escape(void)\n+{\n+\tint scratch;\n+\n+\tbpf_testmod_retval_capture(\u0026scratch);\n+\tscratch = 1;\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_external_memset(void *dest)\n+{\n+\t__builtin_memset(dest, 0, 32);\n+\treturn -EINVAL;\n+}\n+\n+/* Not recognized yet, although they have no side effects */\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_balanced_spinlock(spinlock_t *lock)\n+{\n+\tspin_lock(lock);\n+\tspin_unlock(lock);\n+\treturn -EINVAL;\n+}\n+\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_input_errno(int error)\n+{\n+\tif (error \u003e= -MAX_ERRNO \u0026\u0026 error \u003c 0)\n+\t\treturn error;\n+\treturn 0;\n+}\n+\n+/* No ftrace call site, but call padding in __patchable_function_entries on x86 */\n+static __used noinline RETVAL(\"\") notrace int bpf_testmod_retval_notrace(void)\n+{\n+\treturn -EINVAL;\n+}\n+\n+/*\n+ * A weak definition with an error, replaced by a strong one without errors in\n+ * bpf_testmod_retval_strong.c\n+ */\n+int bpf_testmod_retval_overridden(void);\n+\n+__weak noinline __attribute__((btf_decl_tag(\"test_retval:\")))\n+int bpf_testmod_retval_overridden(void)\n+{\n+\treturn -EINVAL;\n+}\n+\n+/*\n+ * Two weak definitions: this one with a side effect is linked first and wins\n+ * over the one with an error in bpf_testmod_retval_strong.c\n+ */\n+int bpf_testmod_retval_two_weak(void);\n+\n+__weak noinline __attribute__((btf_decl_tag(\"test_retval:\")))\n+int bpf_testmod_retval_two_weak(void)\n+{\n+\tbpf_testmod_retval_effect();\n+\treturn 0;\n+}\n+\n+/*\n+ * A function on the error injection list gets its tag, but the static rules let\n+ * programs return any value\n+ */\n+static __used noinline RETVAL(\"-22\") int bpf_testmod_retval_injectable(int value)\n+{\n+\tif (value \u003c 0)\n+\t\treturn -EINVAL;\n+\tbpf_testmod_retval_effect();\n+\treturn 0;\n+}\n+ALLOW_ERROR_INJECTION(bpf_testmod_retval_injectable, ERRNO);\n+\n+/* A declared target keeps only its own tags */\n+static __used noinline RETVAL(\"\") int bpf_testmod_retval_declared(void)\n+{\n+\treturn -EINVAL;\n+}\n+\n+BTF_FMODRET_START(bpf_testmod_retval_declared_ids)\n+BTF_ID_FLAGS(func, bpf_testmod_retval_declared)\n+BTF_FMODRET_END(bpf_testmod_retval_declared_ids)\ndiff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c\nnew file mode 100644\nindex 00000000000000..a246d7f619bc51\n--- /dev/null\n+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c\n@@ -0,0 +1,28 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Replaces the weak bpf_testmod_retval_overridden() of bpf_testmod_retval.c.\n+ * This definition returns no errors, so the function must not become an\n+ * fmod_ret target, see prog_tests/fmodret_retval.c.\n+ *\n+ * The weak bpf_testmod_retval_two_weak() here loses the link to the one of\n+ * bpf_testmod_retval.c, so its error must not make the function a target\n+ * either.\n+ */\n+#include \u003clinux/compiler.h\u003e\n+#include \u003clinux/errno.h\u003e\n+\n+int bpf_testmod_retval_overridden(void);\n+\n+noinline __attribute__((btf_decl_tag(\"test_retval:\")))\n+int bpf_testmod_retval_overridden(void)\n+{\n+\treturn 0;\n+}\n+\n+int bpf_testmod_retval_two_weak(void);\n+\n+__weak noinline __attribute__((btf_decl_tag(\"test_retval:\")))\n+int bpf_testmod_retval_two_weak(void)\n+{\n+\treturn -EINVAL;\n+}\n"
} |
| 2/1 | 2026/11/12 15:30 | agent | patch-evaluator |
3mModel:core Args: null Results: {
"EnableConfigs": [
"BPF_SYSCALL",
"DEBUG_INFO_BTF",
"DEBUG_INFO_BTF_MODULES",
"BPF_JIT",
"LLVM_PLUGINS",
"CLANG_PLUGIN_BPF_SCAN_IR",
"FUNCTION_ERROR_INJECTION"
],
"FocusSymbols": [
"bpf_check_attach_target",
"check_attach_btf_id",
"fmodret_retval_init",
"check_fmodret_retval",
"btf_register_fmodret_tags",
"btf_kfunc_is_modify_return",
"btf_kfunc_check_flag"
],
"Reasoning": "The patch introduces support for discovering and tagging BPF modify-return (fmod_ret) target functions and enforcing allowed return values via the BPF verifier. It adds new verifier logic (check_fmodret_retval, fmodret_retval_init, and restrictions on tail calls and freplace extensions) as well as BTF registration for fmod_ret tags (btf_register_fmodret_tags). These changes modify core BPF subsystem code reachable through standard bpf() system call operations (program loading and link attachment) in standard virtualized environments.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit b2dd76da41c0d4300e9a8c4bb643fc7c656fa2c8
Author: syz-cluster <triage@syzkaller.com>
Date: Thu Nov 12 15:30:31 2026 +0000
syz-cluster: applied patch under review
diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065de..29940e8d7205b7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5064,6 +5064,14 @@ S: Supported
F: arch/x86/net/
X: arch/x86/net/bpf_jit_comp32.c
+BPF [COMPILER PLUGINS]
+M: Anton Protopopov <a.s.protopopov@gmail.com>
+L: bpf@vger.kernel.org
+S: Maintained
+F: scripts/bpf/
+F: scripts/llvm-plugins/bpf_*
+F: tools/bpf/resolve_btfids/fmodret.*
+
BPF [CORE]
M: Alexei Starovoitov <ast@kernel.org>
M: Daniel Borkmann <daniel@iogearbox.net>
@@ -6562,7 +6570,9 @@ C: irc://irc.libera.chat/clangbuiltlinux
F: Documentation/kbuild/llvm.rst
F: include/linux/compiler-clang.h
F: scripts/Makefile.clang
+F: scripts/Makefile.llvm-plugins
F: scripts/clang-tools/
+F: scripts/llvm-plugins/
K: \b(?i:clang|llvm)\b
CLK API
diff --git a/Makefile b/Makefile
index f895752d4b05fa..2fcad810957218 100644
--- a/Makefile
+++ b/Makefile
@@ -457,10 +457,13 @@ endif
HOSTCC = $(LLVM_PREFIX)clang$(LLVM_SUFFIX)
HOSTCXX = $(LLVM_PREFIX)clang++$(LLVM_SUFFIX)
+LLVM_CONFIG ?= $(LLVM_PREFIX)llvm-config$(LLVM_SUFFIX)
else
HOSTCC = gcc
HOSTCXX = g++
+LLVM_CONFIG ?= llvm-config
endif
+export LLVM_CONFIG
HOSTRUSTC = rustc
HOSTPKG_CONFIG = pkg-config
@@ -1211,10 +1214,11 @@ include-$(CONFIG_AUTOFDO_CLANG) += scripts/Makefile.autofdo
include-$(CONFIG_PROPELLER_CLANG) += scripts/Makefile.propeller
include-$(CONFIG_WARN_CONTEXT_ANALYSIS) += scripts/Makefile.context-analysis
include-$(CONFIG_GCC_PLUGINS) += scripts/Makefile.gcc-plugins
+include-$(CONFIG_LLVM_PLUGINS) += scripts/Makefile.llvm-plugins
include $(addprefix $(srctree)/, $(include-y))
-# scripts/Makefile.gcc-plugins is intentionally included last.
+# The compiler plugin Makefiles are intentionally included last.
# Do not add $(call cc-option,...) below this line. When you build the kernel
# from the clean source tree, the GCC plugins do not exist at this point.
diff --git a/arch/Kconfig b/arch/Kconfig
index 1163736de1f996..4f1f36d3e82d8e 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1772,6 +1772,7 @@ config HAVE_GENERIC_TIF_BITS
source "kernel/gcov/Kconfig"
source "scripts/gcc-plugins/Kconfig"
+source "scripts/llvm-plugins/Kconfig"
config FUNCTION_ALIGNMENT_4B
bool
diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile
index a273ab5af805c6..5813bb75c99251 100644
--- a/arch/arm64/kernel/pi/Makefile
+++ b/arch/arm64/kernel/pi/Makefile
@@ -10,6 +10,7 @@ KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
-D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \
-fno-asynchronous-unwind-tables -fno-unwind-tables \
$(CONFIG_CC_OPT_NO_ADDRSIG)
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# this code may run with the MMU off so disable unaligned accesses
CFLAGS_map_range.o += -mstrict-align
diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile
index 5e499cfb29b5c0..7036e939097e5d 100644
--- a/arch/x86/boot/startup/Makefile
+++ b/arch/x86/boot/startup/Makefile
@@ -12,6 +12,8 @@ KBUILD_CFLAGS += -D__DISABLE_EXPORTS -mcmodel=small -fPIC \
# disable ftrace hooks and LTO
KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS))
KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS))
+# LLVM plugins may add data with absolute addresses
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
KMSAN_SANITIZE := n
diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
index 7664cbedbe30fb..0e32c9ae2b5afc 100644
--- a/arch/x86/um/vdso/Makefile
+++ b/arch/x86/um/vdso/Makefile
@@ -37,7 +37,7 @@ CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \
$(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \
-fno-omit-frame-pointer -foptimize-sibling-calls
-$(vobjs): KBUILD_CFLAGS += $(CFL)
+$(vobjs): KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)
#
# vDSO code runs in userspace and -pg doesn't help with profiling anyway.
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index bbab12b8eb4b23..6f69d5d6773141 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -6,7 +6,7 @@
# enabled, even if doing so doesn't break the build.
#
-cflags-y := $(KBUILD_CFLAGS)
+cflags-y := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# x86 links the stub into the decompressor rather than the kernel proper, so
# drop the kernel's code model, calling convention, and stack alignment, and
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 3eb732942d699c..77e86ff2f9a44d 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -198,7 +198,9 @@
#define MCOUNT_REC() . = ALIGN(8); \
__start_mcount_loc = .; \
KEEP(*(__mcount_loc)) \
+ __start_patchable_function_entries = .; \
KEEP_PATCHABLE \
+ __stop_patchable_function_entries = .; \
__stop_mcount_loc = .; \
FTRACE_STUB_HACK \
ftrace_ops_list_func = arch_ftrace_ops_list_func;
@@ -714,7 +716,7 @@
BTF_INLINE_LINK \
. = ALIGN(PAGE_SIZE); \
.BTF_ids : AT(ADDR(.BTF_ids) - LOAD_OFFSET) { \
- *(.BTF_ids) \
+ KEEP(*(.BTF_ids)) \
}
#else
#define BTF
@@ -886,7 +888,8 @@
KLP_SYMID
#define MODINFO \
- .modinfo (INFO) : { KEEP(*(.modinfo)) }
+ .modinfo (INFO) : { KEEP(*(.modinfo)) } \
+ .BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }
#ifdef CONFIG_GENERIC_BUG
#define BUG_TABLE \
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 520fd72e21fd07..56a5e4b9f138a6 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1969,6 +1969,12 @@ struct bpf_prog_aux {
const struct btf_type *attach_func_proto;
/* function name for valid attach_btf_id */
const char *attach_func_name;
+ /*
+ * The values which an fmod_ret program may return besides 0, from the
+ * "bpf_fmodret_retval:" tag of its target, or NULL
+ */
+ s64 *fmodret_retvals;
+ u32 fmodret_retval_cnt;
struct bpf_prog **func;
struct bpf_prog_aux *main_prog_aux;
/* main prog: there when the prog has timed may_goto */
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 87fad4076f0e9d..5522c7fe13735a 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -558,11 +558,17 @@ static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id)
return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL;
}
-static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
+static inline struct btf_id_set8_pair *
+btf_id_set8_entry(const struct btf_id_set8 *set, u32 id)
{
return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func);
}
+static inline bool btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
+{
+ return btf_id_set8_entry(set, id) != NULL;
+}
+
bool btf_param_match_suffix(const struct btf *btf,
const struct btf_param *arg,
const char *suffix);
@@ -626,7 +632,6 @@ u32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,
const struct bpf_prog *prog);
int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
const struct btf_kfunc_id_set *s);
-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset);
s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id);
int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt,
struct module *owner);
diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
index 8b5a9ee92513d4..f5985acf661b1c 100644
--- a/include/linux/btf_ids.h
+++ b/include/linux/btf_ids.h
@@ -13,13 +13,21 @@ struct btf_id_set {
/* This flag implies BTF_SET8 holds kfunc(s) */
#define BTF_SET8_KFUNCS (1 << 0)
+/*
+ * This flag implies BTF_SET8 holds functions which BPF fmod_ret, fentry and
+ * fexit programs may attach to. Per-function flags: KF_SLEEPABLE only.
+ */
+#define BTF_SET8_FMODRET (1 << 1)
+
+struct btf_id_set8_pair {
+ u32 id;
+ u32 flags;
+};
+
struct btf_id_set8 {
u32 cnt;
u32 flags;
- struct {
- u32 id;
- u32 flags;
- } pairs[];
+ struct btf_id_set8_pair pairs[];
};
#ifdef CONFIG_DEBUG_INFO_BTF
@@ -215,6 +223,12 @@ __BTF_SET8_START(name, local, BTF_SET8_KFUNCS)
#define BTF_KFUNCS_END(name) \
BTF_SET8_END(name)
+#define BTF_FMODRET_START(name) \
+__BTF_SET8_START(name, local, BTF_SET8_FMODRET)
+
+#define BTF_FMODRET_END(name) \
+BTF_SET8_END(name)
+
#else
#define BTF_ID_LIST(name) static u32 __maybe_unused name[128];
@@ -231,6 +245,8 @@ BTF_SET8_END(name)
#define BTF_SET8_END(name)
#define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };
#define BTF_KFUNCS_END(name)
+#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };
+#define BTF_FMODRET_END(name)
#endif /* CONFIG_DEBUG_INFO_BTF */
diff --git a/include/linux/compiler-version.h b/include/linux/compiler-version.h
index ac1665a98a15db..e6a5a9f3fbcd33 100644
--- a/include/linux/compiler-version.h
+++ b/include/linux/compiler-version.h
@@ -24,6 +24,11 @@
#include <generated/gcc-plugins.h>
#endif
+/* Rebuild objects when an LLVM plugin changes. */
+#ifdef LLVM_PLUGINS
+#include <generated/llvm-plugins.h>
+#endif
+
/*
* If the randstruct seed itself changes (whether for GCC plugins or
* Clang), the entire tree needs to be rebuilt since the randomization of
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 228b93d4f8c940..f81a8ae9b92c32 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -9307,6 +9307,8 @@ enum {
};
#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+static int btf_register_fmodret_tags(struct btf *btf);
+
struct btf_module {
struct list_head list;
struct module *module;
@@ -9476,6 +9478,15 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
}
goto out;
}
+ /*
+ * Do this before the BTF is visible to anyone else. A failure
+ * is not fatal: the module loads, but without fmod_ret targets.
+ */
+ err = btf_register_fmodret_tags(btf);
+ if (err)
+ pr_warn("failed to register fmod_ret targets of module [%s]: %d\n",
+ mod->name, err);
+
err = btf_alloc_id(btf);
if (err) {
btf_free(btf);
@@ -9973,8 +9984,8 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,
enum btf_kfunc_hook hook,
u32 kfunc_btf_id)
{
+ struct btf_id_set8_pair *pair;
struct btf_id_set8 *set;
- u32 *id;
if (hook >= BTF_KFUNC_HOOK_MAX)
return NULL;
@@ -9983,11 +9994,10 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,
set = btf->kfunc_set_tab->sets[hook];
if (!set)
return NULL;
- id = btf_id_set8_contains(set, kfunc_btf_id);
- if (!id)
+ pair = btf_id_set8_entry(set, kfunc_btf_id);
+ if (!pair)
return NULL;
- /* The flags for BTF ID are located next to it */
- return id + 1;
+ return &pair->flags;
}
static bool __btf_kfunc_is_allowed(const struct btf *btf,
@@ -10101,7 +10111,9 @@ u32 *btf_kfunc_flags(const struct btf *btf, u32 kfunc_btf_id, const struct bpf_p
}
/*
- * Check a single KF_* @flag on a kfunc across all of its hook sets.
+ * Check a single KF_* @flag on a kfunc across all of its hook sets. The set of
+ * fmod_ret targets is not one of them: its flags only say how a target may be
+ * attached, and a kfunc which is also a target keeps its own flags.
* Returns:
* * 1 if @flag is set
* * 0 if @flag is not set
@@ -10116,6 +10128,8 @@ int btf_kfunc_check_flag(const struct btf *btf, u32 kfunc_btf_id, u32 flag)
u32 *flags;
for (hook = 0; hook < BTF_KFUNC_HOOK_MAX; hook++) {
+ if (hook == BTF_KFUNC_HOOK_FMODRET)
+ continue;
flags = btf_kfunc_id_set_contains(btf, hook, kfunc_btf_id);
if (!flags)
continue;
@@ -10183,12 +10197,102 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
}
EXPORT_SYMBOL_GPL(register_btf_kfunc_id_set);
-/* This function must be invoked only from initcalls/module init functions */
-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset)
+/*
+ * Targets of fmod_ret programs are marked in BTF by resolve_btfids, see
+ * BTF_SET8_FMODRET. A "bpf_fmodret" decl tag on a function makes it a target.
+ * A target which sleepable programs may attach to also has a
+ * "bpf_fmodret_sleepable" tag, which never comes without "bpf_fmodret".
+ */
+#define BTF_FMODRET_TAG "bpf_fmodret"
+#define BTF_FMODRET_SLEEPABLE_TAG "bpf_fmodret_sleepable"
+
+/* Return the function which T tags with decl tag NAME, or 0 */
+static u32 btf_fmodret_tag_func(const struct btf *btf, const struct btf_type *t,
+ const char *name)
{
- return __register_btf_kfunc_id_set(BTF_KFUNC_HOOK_FMODRET, kset);
+ const struct btf_type *func;
+
+ if (!btf_type_is_decl_tag(t) || btf_type_decl_tag(t)->component_idx != -1)
+ return 0;
+ if (strcmp(btf_name_by_offset(btf, t->name_off), name))
+ return 0;
+ func = btf_type_by_id(btf, t->type);
+ if (!func || !btf_type_is_func(func))
+ return 0;
+ return t->type;
+}
+
+/* Build the set of fmod_ret targets of BTF from its decl tags */
+static int btf_register_fmodret_tags(struct btf *btf)
+{
+ u32 id, func_id, nr_types = btf_nr_types(btf), cnt = 0;
+ struct btf_kfunc_set_tab *tab;
+ struct btf_id_set8 *set;
+
+ for (id = btf_start_id(btf); id < nr_types; id++)
+ cnt += !!btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);
+ if (!cnt)
+ return 0;
+
+ set = kmalloc(struct_size(set, pairs, cnt), GFP_KERNEL | __GFP_NOWARN);
+ if (!set)
+ return -ENOMEM;
+ set->flags = 0;
+ set->cnt = 0;
+ for (id = btf_start_id(btf); id < nr_types; id++) {
+ func_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);
+ if (!func_id)
+ continue;
+ set->pairs[set->cnt].id = func_id;
+ set->pairs[set->cnt].flags = 0;
+ set->cnt++;
+ }
+ sort(set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func, NULL);
+
+ /* Each sleepable tag marks a function which is already in the set */
+ for (id = btf_start_id(btf); id < nr_types; id++) {
+ struct btf_id_set8_pair *pair;
+
+ func_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id),
+ BTF_FMODRET_SLEEPABLE_TAG);
+ if (!func_id)
+ continue;
+ pair = btf_id_set8_entry(set, func_id);
+ if (pair)
+ pair->flags |= KF_SLEEPABLE;
+ }
+
+ tab = btf->kfunc_set_tab;
+ if (!tab) {
+ tab = kzalloc_obj(*tab, GFP_KERNEL | __GFP_NOWARN);
+ if (!tab) {
+ kfree(set);
+ return -ENOMEM;
+ }
+ btf->kfunc_set_tab = tab;
+ }
+ if (tab->sets[BTF_KFUNC_HOOK_FMODRET]) {
+ kfree(set);
+ return -EEXIST;
+ }
+ tab->sets[BTF_KFUNC_HOOK_FMODRET] = set;
+ return 0;
+}
+
+static int __init btf_fmodret_tags_init(void)
+{
+ struct btf *btf;
+ int err;
+
+ btf = btf_get_module_btf(NULL);
+ if (IS_ERR_OR_NULL(btf))
+ return PTR_ERR_OR_ZERO(btf);
+
+ err = btf_register_fmodret_tags(btf);
+ btf_put(btf);
+ return err;
}
-EXPORT_SYMBOL_GPL(register_btf_fmodret_id_set);
+fs_initcall(btf_fmodret_tags_init);
s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id)
{
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 9004f2673ccf96..0a78f66a1622c2 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2490,6 +2490,7 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)
module_put(prog->aux->mod);
kfree(prog->aux->kfunc_tab);
kfree(prog->aux->ctx_arg_info);
+ kfree(prog->aux->fmodret_retvals);
if (prog->aux->attach_btf)
btf_put(prog->aux->attach_btf);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 20b1b9ff574c7d..5045d9fa68b928 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -13561,6 +13561,14 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
}
if (func_id == BPF_FUNC_tail_call) {
+ /*
+ * The program which a tail call jumps to returns for this one,
+ * and a program array does not check the values it may return.
+ */
+ if (env->prog->aux->fmodret_retvals) {
+ verbose(env, "tail calls are not allowed in fmod_ret programs with restricted return values\n");
+ return -EINVAL;
+ }
if (env->cur_state->curframe) {
struct bpf_verifier_state *branch;
@@ -20260,6 +20268,67 @@ static bool program_returns_void(struct bpf_verifier_env *env)
return false;
}
+/*
+ * A function which the build found by analysing its code has a
+ * "bpf_fmodret_retval:" decl tag with the values it returns on paths without
+ * side effects, for example "bpf_fmodret_retval:-22,-12". See the comment at
+ * the top of tools/bpf/resolve_btfids/main.c. An fmod_ret program attached to
+ * it may return only these values, or 0 to call the function.
+ */
+#define BTF_FMODRET_RETVAL_TAG "bpf_fmodret_retval:"
+
+/* Parse the next value of the list at *LIST and move *LIST past it */
+static int fmodret_retval_next(const char **list, s64 *value)
+{
+ const char *end = strchrnul(*list, ',');
+ size_t len = end - *list;
+ char buf[24];
+
+ if (!len || len >= sizeof(buf) || (*end && !end[1]))
+ return -EINVAL;
+ memcpy(buf, *list, len);
+ buf[len] = 0;
+ *list = *end ? end + 1 : end;
+ return kstrtos64(buf, 10, value);
+}
+
+static bool fmodret_retval_allowed(const struct bpf_prog_aux *aux, u64 value)
+{
+ const struct btf_type *t;
+ u32 i;
+
+ /* "w0 = -22" zero-extends, the caller of an int function sees -22 */
+ t = btf_type_skip_modifiers(aux->attach_btf, aux->attach_func_proto->type, NULL);
+ if (t->size == 4 && !upper_32_bits(value))
+ value = (s32)value;
+ if (!value)
+ return true;
+ for (i = 0; i < aux->fmodret_retval_cnt; i++) {
+ if (aux->fmodret_retvals[i] == (s64)value)
+ return true;
+ }
+ return false;
+}
+
+/* The program must return a constant: 0 or one of the values of the tag */
+static int check_fmodret_retval(struct bpf_verifier_env *env, int regno,
+ const char *exit_ctx, const char *reg_name)
+{
+ const struct bpf_prog_aux *aux = env->prog->aux;
+ struct bpf_reg_state *reg = reg_state(env, regno);
+ u32 i;
+
+ if (reg->type == SCALAR_VALUE && tnum_is_const(reg->var_off) &&
+ fmodret_retval_allowed(aux, reg->var_off.value))
+ return 0;
+
+ verbose(env, "%s the register %s must be a constant, it should be 0 or one of", exit_ctx, reg_name);
+ for (i = 0; i < aux->fmodret_retval_cnt; i++)
+ verbose(env, "%s%lld", i ? "," : " ", aux->fmodret_retvals[i]);
+ verbose(env, "\n");
+ return -EINVAL;
+}
+
static int check_return_code(struct bpf_verifier_env *env, int regno, const char *reg_name)
{
const char *exit_ctx = "At program exit";
@@ -20316,6 +20385,9 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
if (prog_type == BPF_PROG_TYPE_CGROUP_SKB && (env->prog->expected_attach_type == BPF_CGROUP_INET_EGRESS))
enforce_attach_type_range = tnum_range(2, 3);
+ if (env->prog->aux->fmodret_retvals)
+ return check_fmodret_retval(env, regno, exit_ctx, reg_name);
+
if (!return_retval_range(env, &range))
return 0;
@@ -23759,6 +23831,61 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
#endif /* CONFIG_FUNCTION_ERROR_INJECTION */
+/*
+ * Programs attached to a function found by the analysis may return only the
+ * values of its "bpf_fmodret_retval:" tag. A function which the static rules
+ * allow stays unrestricted.
+ */
+static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,
+ unsigned long addr, const char *tname)
+{
+ struct bpf_prog_aux *aux = env->prog->aux;
+ const struct btf *btf = aux->attach_btf;
+ const struct btf_type *t = btf_type_by_id(btf, btf_id);
+ s64 *retvals = NULL;
+ const char *values, *list;
+ u32 cnt, size = 0;
+ s64 v;
+
+ if (!check_attach_modify_return(addr, tname))
+ return 0;
+ values = btf_find_decl_tag_value(btf, t, -1, BTF_FMODRET_RETVAL_TAG);
+ if (IS_ERR(values))
+ return PTR_ERR(values) == -ENOENT ? 0 : PTR_ERR(values);
+
+ t = btf_type_by_id(btf, t->type);
+ t = t ? btf_type_skip_modifiers(btf, t->type, NULL) : NULL;
+ if (t && (btf_type_is_int(t) || btf_is_any_enum(t)))
+ size = t->size;
+ if (size != 4 && size != 8)
+ goto invalid;
+
+ for (list = values, cnt = 1; *list; list++)
+ cnt += *list == ',';
+ retvals = kmalloc_array(cnt, sizeof(*retvals), GFP_KERNEL_ACCOUNT);
+ if (!retvals)
+ return -ENOMEM;
+ for (list = values, cnt = 0; *list;) {
+ if (fmodret_retval_next(&list, &v) || v >= 0 || (size == 4 && v < S32_MIN))
+ goto invalid;
+ retvals[cnt++] = v;
+ }
+ if (!cnt)
+ goto invalid;
+
+ aux->fmodret_retvals = retvals;
+ aux->fmodret_retval_cnt = cnt;
+ /* The interpreter returns only 32 bits */
+ if (size == 8)
+ env->prog->jit_required = 1;
+ return 0;
+
+invalid:
+ kfree(retvals);
+ verbose(env, "invalid %s%s tag of %s()\n", BTF_FMODRET_RETVAL_TAG, values, tname);
+ return -EINVAL;
+}
+
static bool is_tracing_multi_id(const struct bpf_prog *prog, u32 btf_id)
{
return is_tracing_multi(prog->expected_attach_type) && bpf_multi_func_btf_id[0] == btf_id;
@@ -24173,6 +24300,15 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
bpf_log(log, "Cannot extend fentry/fexit/fsession\n");
return -EINVAL;
}
+ /*
+ * An extension could return any value for an fmod_ret program
+ * whose return values are restricted. This is also checked when
+ * an extension is attached to another program than at load.
+ */
+ if (prog_extension && tgt_prog->aux->fmodret_retvals) {
+ bpf_log(log, "Cannot extend an fmod_ret program with restricted return values\n");
+ return -EINVAL;
+ }
} else {
if (prog_extension) {
bpf_log(log, "Cannot replace kernel functions\n");
@@ -24552,6 +24688,13 @@ static int check_attach_btf_id(struct bpf_verifier_env *env)
return -EINVAL;
}
+ /* Only a kernel function has a "bpf_fmodret_retval:" tag */
+ if (prog->expected_attach_type == BPF_MODIFY_RETURN && !tgt_prog) {
+ ret = fmodret_retval_init(env, btf_id, tgt_info.tgt_addr, tgt_info.tgt_name);
+ if (ret)
+ return ret;
+ }
+
/*
* We don't get trampoline for tracing_multi programs at this point,
* it's done when tracing_multi link is created.
diff --git a/lib/vdso/Makefile.include b/lib/vdso/Makefile.include
index cedbf15f80874d..848d58296ba231 100644
--- a/lib/vdso/Makefile.include
+++ b/lib/vdso/Makefile.include
@@ -1,5 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
+# vDSO code executes outside the kernel.
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
+
GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index d6c9dc62182ee4..d63c3b6975527e 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -643,17 +643,12 @@ CFI_NOSEAL(bpf_kfunc_call_memb_release_dtor);
__bpf_kfunc_end_defs();
-BTF_KFUNCS_START(bpf_test_modify_return_ids)
+BTF_FMODRET_START(bpf_test_modify_return_ids)
BTF_ID_FLAGS(func, bpf_modify_return_test)
BTF_ID_FLAGS(func, bpf_modify_return_test2)
BTF_ID_FLAGS(func, bpf_modify_return_test_tp)
BTF_ID_FLAGS(func, bpf_fentry_test1, KF_SLEEPABLE)
-BTF_KFUNCS_END(bpf_test_modify_return_ids)
-
-static const struct btf_kfunc_id_set bpf_test_modify_return_set = {
- .owner = THIS_MODULE,
- .set = &bpf_test_modify_return_ids,
-};
+BTF_FMODRET_END(bpf_test_modify_return_ids)
BTF_KFUNCS_START(test_sk_check_kfunc_ids)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_release, KF_RELEASE)
@@ -1905,8 +1900,7 @@ static int __init bpf_prog_test_run_init(void)
};
int ret;
- ret = register_btf_fmodret_id_set(&bpf_test_modify_return_set);
- ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set);
+ ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_prog_test_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_prog_test_kfunc_set);
return ret ?: register_btf_id_dtor_kfuncs(bpf_prog_test_dtor_kfunc,
diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index 4cc16cbeb32817..d0d489624689a4 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -20,17 +20,6 @@ struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)
return NULL;
}
-BTF_SET8_START(bpf_mptcp_fmodret_ids)
+BTF_FMODRET_START(bpf_mptcp_fmodret_ids)
BTF_ID_FLAGS(func, update_socket_protocol)
-BTF_SET8_END(bpf_mptcp_fmodret_ids)
-
-static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = {
- .owner = THIS_MODULE,
- .set = &bpf_mptcp_fmodret_ids,
-};
-
-static int __init bpf_mptcp_kfunc_init(void)
-{
- return register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set);
-}
-late_initcall(bpf_mptcp_kfunc_init);
+BTF_FMODRET_END(bpf_mptcp_fmodret_ids)
diff --git a/scripts/Makefile b/scripts/Makefile
index d7fbb7ca098b18..bc5dacc29207b2 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -66,6 +66,7 @@ hostprogs += unifdef gen_packed_field_checks
targets += module.lds
subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins
+subdir-$(CONFIG_LLVM_PLUGINS) += llvm-plugins
subdir-$(CONFIG_GENKSYMS) += genksyms
subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms
subdir-$(CONFIG_SECURITY_SELINUX) += selinux
diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
index 98012344a81ec3..3aa3d084cc6c08 100644
--- a/scripts/Makefile.btf
+++ b/scripts/Makefile.btf
@@ -35,5 +35,6 @@ resolve-btfids-flags-y :=
resolve-btfids-flags-$(CONFIG_WERROR) += --fatal_warnings
resolve-btfids-flags-$(if $(KBUILD_EXTMOD),y) += --distill_base
resolve-btfids-flags-$(if $(KBUILD_VERBOSE),y) += --verbose
+resolve-btfids-flags-$(CONFIG_FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY) += --patchable_ftrace
export RESOLVE_BTFIDS_FLAGS := $(resolve-btfids-flags-y)
diff --git a/scripts/Makefile.llvm-plugins b/scripts/Makefile.llvm-plugins
new file mode 100644
index 00000000000000..9ff364f4db5957
--- /dev/null
+++ b/scripts/Makefile.llvm-plugins
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+llvm-plugin-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_scan_ir.so
+
+LLVM_PLUGIN := $(llvm-plugin-y)
+LLVM_PLUGINS_CFLAGS := $(strip $(addprefix -fpass-plugin=$(objtree)/scripts/llvm-plugins/, \
+ $(LLVM_PLUGIN)) -DLLVM_PLUGINS)
+export LLVM_PLUGIN LLVM_PLUGINS_CFLAGS
+
+KBUILD_CFLAGS += $(LLVM_PLUGINS_CFLAGS)
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 01a37ec872b905..042c31e3bdfb78 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -42,6 +42,7 @@ quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ ! -f $(objtree)/vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
+ $(OBJCOPY) --remove-section=.BTF_fmodret_candidates $@; \
else \
$(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
fi;
diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
index 95b523fe4f9eeb..c8651454c3d366 100644
--- a/scripts/Makefile.vmlinux
+++ b/scripts/Makefile.vmlinux
@@ -73,6 +73,9 @@ vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD
ifdef CONFIG_DEBUG_INFO_BTF
vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh
endif
+ifdef CONFIG_CLANG_PLUGIN_BPF_SCAN_IR
+vmlinux.unstripped: $(srctree)/scripts/bpf/fmodret.lds
+endif
ifdef CONFIG_BUILDTIME_TABLE_SORT
vmlinux.unstripped: scripts/sorttable
diff --git a/scripts/bpf/fmodret.lds b/scripts/bpf/fmodret.lds
new file mode 100644
index 00000000000000..6bdfde9e767979
--- /dev/null
+++ b/scripts/bpf/fmodret.lds
@@ -0,0 +1,6 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+SECTIONS {
+ /DISCARD/ : {
+ *(.BTF_fmodret_candidates)
+ }
+}
diff --git a/scripts/bpf/fmodret_metadata.h b/scripts/bpf/fmodret_metadata.h
new file mode 100644
index 00000000000000..209f266f89a375
--- /dev/null
+++ b/scripts/bpf/fmodret_metadata.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef FMODRET_METADATA_H
+#define FMODRET_METADATA_H
+
+/*
+ * The records which the BPF compiler plugins write to .BTF_fmodret_candidates
+ * for resolve_btfids, one for each function with values and for each weak
+ * definition. If a function has two weak definitions, the record of the one
+ * which loses the link would describe the body of the other, so both need a
+ * record to make the function ambiguous. A record is packed, target-endian and
+ * aligned to the target pointer size:
+ *
+ * u32 magic, version, size, flags;
+ * ptr function;
+ * u32 nr_values, reserved;
+ * s64 values[nr_values];
+ *
+ * The values are the errors which clean paths of the function return, sorted
+ * and unique.
+ */
+#define FMODRET_META_MAGIC 0x46524d32U
+#define FMODRET_META_VERSION 1U
+#define FMODRET_META_HEADER_SIZE(ptr_size) (24 + (ptr_size))
+#define FMODRET_META_VALUE_SIZE 8
+
+#define FMODRET_FN_WEAK (1U << 0)
+#define FMODRET_FN_RET64 (1U << 1)
+
+#endif
diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
index b17f1189298a55..5bca864631e0b2 100755
--- a/scripts/gen-btf.sh
+++ b/scripts/gen-btf.sh
@@ -120,7 +120,9 @@ gen_btf_o()
embed_btf_data()
{
- ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF ${ELF_FILE}
+ # resolve_btfids has consumed the compiler plugin records of the module
+ ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \
+ --remove-section=.BTF_fmodret_candidates ${ELF_FILE}
# a module might not have a .BTF_ids or .BTF.base section
btf_base="${ELF_FILE}.BTF.base"
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d5710e900bd408..f39e2d1f1400a2 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -89,6 +89,13 @@ vmlinux_link()
ldlibs=
fi
+ if is_enabled CONFIG_CLANG_PLUGIN_BPF_SCAN_IR; then
+ if [ "${output}" = ".tmp_vmlinux1" ]; then
+ ldflags="${ldflags} ${wl}--emit-relocs"
+ else
+ ldflags="${ldflags} ${wl}--script=${srctree}/scripts/bpf/fmodret.lds"
+ fi
+ fi
ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}"
# The kallsyms linking does not need debug symbols included.
diff --git a/scripts/llvm-plugins/Kconfig b/scripts/llvm-plugins/Kconfig
new file mode 100644
index 00000000000000..c4d62ae5f65d25
--- /dev/null
+++ b/scripts/llvm-plugins/Kconfig
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+menuconfig LLVM_PLUGINS
+ bool "LLVM plugins"
+ depends on CC_IS_CLANG
+ depends on $(success,$(LLVM_CONFIG) --version)
+ # The plugins are built against the headers of llvm-config and loaded
+ # by clang, so both must come from the same LLVM release
+ depends on $(success,test "$(shell,$(LLVM_CONFIG) --version 2>/dev/null | cut -d. -f1)" = "$(shell,$(CC) -dumpversion | cut -d. -f1)")
+ # The host C++ compiler must accept the LLVM headers and flags
+ depends on $(success,echo '#include <llvm/Config/llvm-config.h>' | $(HOSTCXX) $(shell,$(LLVM_CONFIG) --cxxflags 2>/dev/null) -x c++ -fsyntax-only -)
+ default y
+ help
+ LLVM plugins add compiler passes to kernel and module builds.
+ Building them requires LLVM development headers and llvm-config
+ of the same major version as clang. Each plugin has its own option
+ below and is built only if that option is enabled.
+
+if LLVM_PLUGINS
+
+config CLANG_PLUGIN_BPF_SCAN_IR
+ bool "Scan optimized LLVM IR for BPF"
+ depends on X86_64 || ARM64
+ depends on DEBUG_INFO_BTF
+ depends on !LTO_CLANG
+ depends on CLANG_VERSION >= 190000
+ help
+ Find the functions which return errors on paths without side
+ effects and describe them in .BTF_fmodret_candidates for
+ resolve_btfids. The section is discarded from the final kernel
+ and modules.
+
+endif
diff --git a/scripts/llvm-plugins/Makefile b/scripts/llvm-plugins/Makefile
new file mode 100644
index 00000000000000..a1f8503c8bcd12
--- /dev/null
+++ b/scripts/llvm-plugins/Makefile
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+always-y += $(LLVM_PLUGIN)
+targets += $(LLVM_PLUGIN)
+clean-files += *.so
+
+# Run llvm-config only to build a plugin, not when this Makefile is parsed
+llvm_cxxflags = $(shell $(LLVM_CONFIG) --cxxflags)
+llvm_ldflags = $(shell $(LLVM_CONFIG) --ldflags)
+
+quiet_cmd_llvm_plugin = HOSTCXX $@
+ cmd_llvm_plugin = $(HOSTCXX) -Wp,-MD,$(depfile) $(KBUILD_HOSTCXXFLAGS) \
+ $(llvm_cxxflags) -fPIC -shared $(KBUILD_HOSTLDFLAGS) -o $@ $< \
+ $(llvm_ldflags)
+
+$(obj)/%.so: $(src)/%.cpp FORCE
+ $(call if_changed_dep,llvm_plugin)
+
+$(obj)/../../include/generated/llvm-plugins.h: $(addprefix $(obj)/, $(LLVM_PLUGIN)) FORCE
+ $(call if_changed,touch)
+always-y += ../../include/generated/llvm-plugins.h
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
new file mode 100644
index 00000000000000..a3c6336662fc1e
--- /dev/null
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -0,0 +1,374 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* bpf_scan_ir: optimizer-last LLVM IR analysis for BPF. */
+
+#include "../bpf/fmodret_metadata.h"
+
+#include <algorithm>
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Analysis/CaptureTracking.h"
+#include "llvm/Analysis/LazyValueInfo.h"
+#include "llvm/Analysis/ValueTracking.h"
+#include "llvm/Config/llvm-config.h"
+#include "llvm/IR/Analysis.h"
+#include "llvm/IR/BasicBlock.h"
+#include "llvm/IR/CFG.h"
+#include "llvm/IR/ConstantRange.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/Dominators.h"
+#include "llvm/IR/GlobalVariable.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/IntrinsicInst.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/Passes/PassBuilder.h"
+#if __has_include("llvm/Plugins/PassPlugin.h")
+#include "llvm/Plugins/PassPlugin.h"
+#else
+#include "llvm/Passes/PassPlugin.h"
+#endif
+#include "llvm/Transforms/Utils/ModuleUtils.h"
+
+using namespace llvm;
+
+namespace {
+
+static constexpr int MaxErrno = 4095;
+
+// is Range in [-4095, 0) ?
+static bool isErrnoRange(const ConstantRange &Range)
+{
+ unsigned Width = Range.getBitWidth();
+
+ if (Width != 32 && Width != 64)
+ return false;
+
+ auto L = APInt(Width, 0) - APInt(Width, MaxErrno);
+ auto R = APInt(Width, 0);
+ return !Range.isEmptySet() && ConstantRange(L, R).contains(Range);
+}
+
+/* The errors which clean paths of a function return */
+using ReturnValues = SmallVector<int64_t, 8>;
+
+/* Return true and set N if RANGE holds a single errno */
+static bool singleErrno(const ConstantRange &Range, int64_t &N)
+{
+ const APInt *C = Range.getSingleElement();
+
+ if (!C || !isErrnoRange(Range))
+ return false;
+ N = C->getSExtValue();
+ return true;
+}
+
+/* Return true and set N if the value at use U is a single errno */
+static bool singleErrnoAtUse(const Use &U, LazyValueInfo &LVI, int64_t &N)
+{
+ return U.get()->getType()->isIntegerTy() &&
+ singleErrno(LVI.getConstantRangeAtUse(U, false), N);
+}
+
+/* Return true and set N if V is an errno constant */
+static bool errnoConstant(const Value *V, int64_t &N)
+{
+ auto *C = dyn_cast<ConstantInt>(V);
+
+ if (!C || (C->getBitWidth() != 32 && C->getBitWidth() != 64))
+ return false;
+
+ N = C->getSExtValue();
+ return N >= -MaxErrno && N < 0;
+}
+
+/* A sign extension or a truncation between integers of at least 32 bits */
+static bool preservesErrnoCast(const CastInst *Cast)
+{
+ Type *Src = Cast->getSrcTy(), *Dst = Cast->getDestTy();
+
+ if (Src->isIntegerTy() && Dst->isIntegerTy())
+ return (Cast->getOpcode() == Instruction::SExt ||
+ Cast->getOpcode() == Instruction::Trunc) &&
+ Src->getIntegerBitWidth() >= 32 && Dst->getIntegerBitWidth() >= 32;
+ return false;
+}
+
+/*
+ * Add the errors which V, the value returned in RET, is known to be on the
+ * edges to RET from CLEANBBS to VALUES. Follow the casts in RET which keep the
+ * error to a PHI in RET. A PHI in an earlier block is not followed: a later
+ * branch may split its values between clean and dirty paths.
+ */
+static void collectErrnos(Value *V, BasicBlock *Ret, LazyValueInfo &LVI,
+ const SmallPtrSetImpl<BasicBlock *> &CleanBBs, ReturnValues &Values)
+{
+ int64_t N;
+
+ while (auto *Cast = dyn_cast<CastInst>(V)) {
+ if (Cast->getParent() != Ret || !preservesErrnoCast(Cast))
+ return;
+ if (singleErrnoAtUse(Cast->getOperandUse(0), LVI, N)) {
+ Values.push_back(N);
+ return;
+ }
+ V = Cast->getOperand(0);
+ }
+
+ auto *Phi = dyn_cast<PHINode>(V);
+
+ if (!Phi || Phi->getParent() != Ret)
+ return;
+ for (unsigned i = 0; i < Phi->getNumIncomingValues(); i++) {
+ Value *Incoming = Phi->getIncomingValue(i);
+ BasicBlock *Pred = Phi->getIncomingBlock(i);
+
+ if (!CleanBBs.contains(Pred))
+ continue;
+ if (errnoConstant(Incoming, N)) {
+ Values.push_back(N);
+ continue;
+ }
+ /*
+ * LazyValueInfo knows the ranges of integers only. On an edge it
+ * also uses the branch condition, as in "if (err == -EINVAL) goto
+ * out", where err is the error.
+ */
+ if (Incoming->getType()->isIntegerTy() &&
+ singleErrno(LVI.getConstantRangeOnEdge(Incoming, Pred, Ret), N))
+ Values.push_back(N);
+ }
+}
+
+class CleanInstructions {
+ DominatorTree &DT;
+ DenseMap<const Instruction *, bool> PrivateWrites;
+
+ bool privateDestination(const Value *Pointer, const Instruction &I)
+ {
+ auto It = PrivateWrites.find(&I);
+ if (It != PrivateWrites.end())
+ return It->second;
+ auto *Allocation = dyn_cast<AllocaInst>(getUnderlyingObject(Pointer));
+ bool Private = false;
+
+ if (Allocation && Allocation->getFunction() == I.getFunction()) {
+ // A later call may receive the initialized object. Only captures
+ // that can precede this write make the write externally visible.
+#if LLVM_VERSION_MAJOR >= 21
+ Private = !PointerMayBeCapturedBefore(Allocation, true, &I, &DT, true, 256);
+#else
+ Private = !PointerMayBeCapturedBefore(Allocation, true, true, &I, &DT, true, 256);
+#endif
+ }
+ PrivateWrites[&I] = Private;
+ return Private;
+ }
+
+public:
+ explicit CleanInstructions(DominatorTree &DT) : DT(DT) {}
+
+ bool isClean(const Instruction &I)
+ {
+ if (auto *Store = dyn_cast<StoreInst>(&I))
+ return !Store->isVolatile() && !Store->isAtomic() &&
+ privateDestination(Store->getPointerOperand(), I);
+ if (auto *Memory = dyn_cast<MemIntrinsic>(&I))
+ return !Memory->isVolatile() &&
+ privateDestination(Memory->getRawDest(), I);
+ if (auto *Intrinsic = dyn_cast<IntrinsicInst>(&I)) {
+ switch (Intrinsic->getIntrinsicID()) {
+ case Intrinsic::assume:
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end:
+ case Intrinsic::invariant_start:
+ case Intrinsic::invariant_end:
+ return true;
+ default:
+ break;
+ }
+ }
+ return !I.mayHaveSideEffects();
+ }
+};
+
+static bool cleanBB(BasicBlock *BB, CleanInstructions &Clean)
+{
+ for (Instruction &I : *BB) {
+ if (!Clean.isClean(I))
+ return false;
+ }
+ return true;
+}
+
+/* Add the errors which clean paths of F are known to return to VALUES */
+static void cleanReturnValues(Function &F, LazyValueInfo &LVI, CleanInstructions &Clean,
+ ReturnValues &Values)
+{
+ SmallPtrSet<BasicBlock *, 32> CleanBBs;
+ SmallVector<BasicBlock *, 32> Worklist;
+ bool Changed;
+
+ /* The clean blocks which the entry reaches through clean blocks */
+ if (cleanBB(&F.getEntryBlock(), Clean)) {
+ CleanBBs.insert(&F.getEntryBlock());
+ Worklist.push_back(&F.getEntryBlock());
+ }
+ while (!Worklist.empty()) {
+ BasicBlock *BB = Worklist.pop_back_val();
+
+ for (BasicBlock *Successor : successors(BB)) {
+ if (!CleanBBs.contains(Successor) && cleanBB(Successor, Clean)) {
+ CleanBBs.insert(Successor);
+ Worklist.push_back(Successor);
+ }
+ }
+ }
+
+ /*
+ * Keep the blocks which only clean paths reach: after a dirty path joins
+ * a clean one, a branch may depend on what the dirty path did
+ */
+ do {
+ Changed = false;
+ for (BasicBlock &BB : F) {
+ if (!CleanBBs.contains(&BB))
+ continue;
+ for (BasicBlock *Pred : predecessors(&BB)) {
+ if (!CleanBBs.contains(Pred)) {
+ CleanBBs.erase(&BB);
+ Changed = true;
+ break;
+ }
+ }
+ }
+ } while (Changed);
+
+ /*
+ * The returns of the clean blocks which only clean paths reach or which
+ * one of them branches to
+ */
+ for (BasicBlock &BB : F) {
+ auto *Return = dyn_cast<ReturnInst>(BB.getTerminator());
+ bool Entered = CleanBBs.contains(&BB);
+ int64_t N;
+
+ if (!Return || !Return->getReturnValue() || !cleanBB(&BB, Clean))
+ continue;
+ for (BasicBlock *Pred : predecessors(&BB))
+ Entered |= CleanBBs.contains(Pred);
+ if (!Entered)
+ continue;
+ if (singleErrnoAtUse(Return->getOperandUse(0), LVI, N)) {
+ Values.push_back(N);
+ continue;
+ }
+ collectErrnos(Return->getReturnValue(), &BB, LVI, CleanBBs, Values);
+ }
+ llvm::sort(Values);
+ Values.erase(std::unique(Values.begin(), Values.end()), Values.end());
+}
+
+struct FunctionDescription {
+ Function *Fn;
+ unsigned Flags;
+ ReturnValues Values;
+};
+
+static void emitDescriptions(Module &M, ArrayRef<FunctionDescription> Functions)
+{
+ auto *I32 = Type::getInt32Ty(M.getContext());
+ auto *I64 = Type::getInt64Ty(M.getContext());
+ unsigned PtrSize = M.getDataLayout().getPointerSize();
+ auto N = [&](unsigned V) -> Constant * { return ConstantInt::get(I32, V); };
+
+ for (const auto &F : Functions) {
+ unsigned Size = FMODRET_META_HEADER_SIZE(PtrSize) +
+ F.Values.size() * FMODRET_META_VALUE_SIZE;
+ SmallVector<Constant *, 16> Fields = {
+ N(FMODRET_META_MAGIC), N(FMODRET_META_VERSION), N(Size), N(F.Flags),
+ F.Fn, N(F.Values.size()), N(0),
+ };
+
+ for (int64_t V : F.Values)
+ Fields.push_back(ConstantInt::getSigned(I64, V));
+ Constant *Record = ConstantStruct::getAnon(M.getContext(), Fields, true);
+ auto *GV = new GlobalVariable(M, Record->getType(), true, GlobalValue::PrivateLinkage,
+ Record, "fmodret.function." + F.Fn->getName());
+ GV->setSection(".BTF_fmodret_candidates");
+ GV->setAlignment(M.getDataLayout().getPointerABIAlignment(0));
+ appendToUsed(M, {GV});
+ }
+}
+
+#if LLVM_VERSION_MAJOR >= 24
+class BpfScanIRPass : public RequiredPassInfoMixin<BpfScanIRPass>
+#else
+class BpfScanIRPass : public PassInfoMixin<BpfScanIRPass>
+#endif
+{
+public:
+ PreservedAnalyses run(Module &M, ModuleAnalysisManager &MAM)
+ {
+ // Avoid emitting duplicate descriptions if bpf_scan_ir runs again
+ // on the same module.
+ if (M.getNamedMetadata("bpf_scan_ir.done"))
+ return PreservedAnalyses::all();
+
+ SmallVector<FunctionDescription, 32> Descriptions;
+ auto &FAM = MAM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
+
+ for (Function &F : M) {
+ if (F.isIntrinsic() || F.isDeclaration() || F.hasAvailableExternallyLinkage())
+ continue;
+ Type *Ret = F.getReturnType();
+ unsigned Flags = 0;
+ ReturnValues Values;
+
+ if (!Ret->isIntegerTy(32) && !Ret->isIntegerTy(64))
+ continue;
+ if (F.isWeakForLinker())
+ Flags |= FMODRET_FN_WEAK;
+ if (Ret->isIntegerTy(64))
+ Flags |= FMODRET_FN_RET64;
+ auto &LVI = FAM.getResult<LazyValueAnalysis>(F);
+ auto &DT = FAM.getResult<DominatorTreeAnalysis>(F);
+ CleanInstructions Clean(DT);
+
+ cleanReturnValues(F, LVI, Clean, Values);
+ /* A weak definition always has a record, see fmodret_metadata.h */
+ if (!Values.empty() || (Flags & FMODRET_FN_WEAK))
+ Descriptions.push_back({&F, Flags, std::move(Values)});
+ }
+ if (Descriptions.empty())
+ return PreservedAnalyses::all();
+ emitDescriptions(M, Descriptions);
+ M.getOrInsertNamedMetadata("bpf_scan_ir.done");
+
+ PreservedAnalyses PA;
+ PA.preserve<FunctionAnalysisManagerModuleProxy>();
+ PA.preserveSet<CFGAnalyses>();
+ PA.preserve<LazyValueAnalysis>();
+ return PA;
+ }
+
+ static bool isRequired() { return true; }
+};
+
+} // namespace
+
+extern "C" LLVM_ATTRIBUTE_WEAK PassPluginLibraryInfo llvmGetPassPluginInfo()
+{
+ return {
+ LLVM_PLUGIN_API_VERSION,
+ "bpf_scan_ir",
+ "1.0",
+ [](PassBuilder &PB) {
+ PB.registerOptimizerLastEPCallback(
+ [](ModulePassManager &MPM, OptimizationLevel, auto...) {
+ MPM.addPass(BpfScanIRPass());
+ }
+ );
+ }
+ };
+}
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 75374c64b8cc78..3bae985492288f 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -772,6 +772,7 @@ static const char *const section_white_list[] =
".llvm.call-graph-profile", /* call graph */
"__llvm_covfun",
"__llvm_covmap",
+ ".BTF_fmodret_candidates",
".klp.symid", /* objtool --klp-symids */
NULL
};
diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index d0f20042895763..e6ba499dc87661 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -27,6 +27,11 @@ SECTIONS {
__ex_table 0 : ALIGN(4) { KEEP(*(__ex_table)) }
__patchable_function_entries 0 : { *(__patchable_function_entries) }
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+ .BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }
+#else
+ /DISCARD/ : { *(.BTF_fmodret_candidates) }
+#endif
.init.klp_funcs 0 : ALIGN(8) { KEEP(*(.init.klp_funcs)) }
.init.klp_objects 0 : ALIGN(8) { KEEP(*(.init.klp_objects)) }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
new file mode 100644
index 00000000000000..1580fd614010ba
--- /dev/null
+++ b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
@@ -0,0 +1,111 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+===============
+bpftool-fmodret
+===============
+-------------------------------------------------------------------------------
+tool to inspect BPF fmod_ret targets
+-------------------------------------------------------------------------------
+
+:Manual section: 8
+
+.. include:: substitutions.rst
+
+SYNOPSIS
+========
+
+**bpftool** [*OPTIONS*] **fmodret** *COMMAND*
+
+*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } *FILE* }
+
+*COMMANDS* := { **list** | **show** | **help** }
+
+FMODRET COMMANDS
+================
+
+| **bpftool** **fmodret list** [*OBJECT*]
+| **bpftool** **fmodret show** *FUNC* [*OBJECT*]
+| **bpftool** **fmodret help**
+|
+| *FUNC* := { **name** *NAME* | **id** *BTF_ID* }
+| *OBJECT* := { **file** *FILE* | **vmlinux** | **module** *NAME* }
+
+DESCRIPTION
+===========
+
+bpftool fmodret list [*OBJECT*]
+ List the fmod_ret targets, the functions which the kernel build tags with
+ the **bpf_fmodret** BTF decl tag, in vmlinux and in loaded modules. Each
+ entry contains the BTF ID of the function, its name and the name of the
+ object it belongs to. Targets which sleepable programs may attach to are
+ marked **sleepable**.
+
+ With **file**, read the BTF from *FILE* (for example a vmlinux image or a
+ **BTF** file) instead of the running kernel. With **vmlinux** or
+ **module**, list only the targets of vmlinux or of the module *NAME*.
+
+ With **--json**, each entry contains **btf_id**, **name**, **owner** and
+ **sleepable** fields.
+
+ The verifier also accepts functions by its static rules, such as the error
+ injection list or **security_** hooks. Such functions are listed only if
+ the kernel build tagged them. Even if they are listed, the restrictions of
+ their **bpf_fmodret_retval:** tags do not apply to them: programs may
+ return any value, and sleepable programs may attach to some of them without
+ the **sleepable** mark. This may change in the future by either not listing
+ them, or listing them without restrictions. Normal verifier and tracing
+ checks apply to all of them.
+
+bpftool fmodret show *FUNC* [*OBJECT*]
+ Show the function named *NAME* or with BTF ID *BTF_ID* in detail. It does
+ not have to be a target. A name may belong to several functions, for
+ example to functions of different modules. A BTF ID is looked up in
+ vmlinux, unless **module** or **file** is given. *OBJECT* is as for
+ **list**.
+
+ Besides the BTF ID, the name and the object, show the prototype of the
+ function and if it is a target: **found by the kernel build**, **declared**
+ or **not tagged**. For a target show if sleepable programs may attach to it
+ and the values which programs may return: 0 and the values of its
+ **bpf_fmodret_retval:** tag with the names of the errors on the host, or
+ **any** for a declared target. A tag which the verifier would reject,
+ malformed, not the only one or with values which the return type cannot
+ hold, is shown as invalid. The values do not apply to a function which the
+ static rules allow, see **list**. For the running kernel, also show the
+ type of the entry of the function in the error injection list, if the list
+ is available in debugfs.
+
+ With **--json**, each function contains **btf_id**, **name**, **owner**,
+ **prototype**, **target** (**found**, **declared** or **none**) and
+ **sleepable** fields, a **retval** array for a target found by the build
+ with the values programs may return, 0 included, or a **retval_error**
+ string if its tag is invalid, and an **error_injection** field if the
+ function is in the error injection list. A BTF ID gives one function, a
+ name an array of them.
+
+bpftool fmodret help
+ Print short help message.
+
+OPTIONS
+========
+
+.. include:: common_options.rst
+
+-B, --base-btf *FILE*
+ Pass the base BTF of a split BTF *FILE*, usually the BTF of vmlinux. Raw
+ BTF of a module needs it. A module image may carry the base it was built
+ against in its **.BTF.base** section instead, but its BTF IDs then differ
+ from those of the running kernel.
+
+EXAMPLES
+========
+
+**# bpftool fmodret show name bpf_testmod_retval_join module bpf_test_retval**
+
+::
+
+ 123456 bpf_testmod_retval_join (bpf_test_retval)
+ int bpf_testmod_retval_join(int *result, int value)
+ target: found by the kernel build
+ sleepable: no
+ retval: 0, -34 (ERANGE), -22 (EINVAL)
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index f38ae5c4043917..2c64c22a50269a 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -21,7 +21,7 @@ SYNOPSIS
**bpftool** **version**
*OBJECT* := { **map** | **prog** | **link** | **cgroup** | **perf** | **net** | **feature** |
-**btf** | **gen** | **struct_ops** | **iter** }
+**btf** | **gen** | **struct_ops** | **iter** | **fmodret** }
*OPTIONS* := { { **-V** | **--version** } | |COMMON_OPTIONS| }
diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index a28af31937ad4b..83288c22b3d34e 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -675,6 +675,37 @@ _bpftool()
;;
esac
;;
+ fmodret)
+ case $command in
+ list|show)
+ case $prev in
+ file)
+ _filedir
+ ;;
+ module)
+ COMPREPLY=( $( compgen -W "$( command ls \
+ /sys/kernel/btf 2>/dev/null | \
+ command grep -vx -e vmlinux -e '.*\.inline' )" \
+ -- "$cur" ) )
+ ;;
+ name|id)
+ ;;
+ *)
+ if [[ $prev == show ]]; then
+ COMPREPLY=( $( compgen -W 'name id' -- "$cur" ) )
+ else
+ _bpftool_one_of_list 'file vmlinux module'
+ fi
+ ;;
+ esac
+ return 0
+ ;;
+ *)
+ [[ $prev == $object ]] && \
+ COMPREPLY=( $( compgen -W 'help list show' -- "$cur" ) )
+ ;;
+ esac
+ ;;
map)
case $command in
show|list|dump|peek|pop|dequeue|freeze)
diff --git a/tools/bpf/bpftool/fmodret.c b/tools/bpf/bpftool/fmodret.c
new file mode 100644
index 00000000000000..941166da53cc23
--- /dev/null
+++ b/tools/bpf/bpftool/fmodret.c
@@ -0,0 +1,896 @@
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include <dirent.h>
+#include <errno.h>
+#include <limits.h>
+#include <stdarg.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+
+#include "main.h"
+
+/*
+ * Decl tags added by resolve_btfids to functions in BTF_FMODRET sets. A target
+ * which sleepable programs may attach to has both tags. A target which the
+ * build found by analysing its code has a "bpf_fmodret_retval:" tag with the
+ * values programs may return, such as "bpf_fmodret_retval:-22,-12".
+ */
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+#define TAG_FMODRET_RETVAL "bpf_fmodret_retval:"
+
+#define SYS_BTF_DIR "/sys/kernel/btf"
+#define ERROR_INJECTION_LIST "/sys/kernel/debug/error_injection/list"
+
+struct fmodret_entry {
+ __u32 btf_id;
+ bool target;
+ bool sleepable;
+ const char *retval;
+ /* More than one "bpf_fmodret_retval:" tag, which the verifier rejects */
+ bool retval_dup;
+};
+
+struct fmodret_filter {
+ /* Only vmlinux, or only the module with this name */
+ bool vmlinux;
+ const char *module;
+ /* The function to show, by name or by BTF ID */
+ const char *name;
+ __u32 btf_id;
+ unsigned int found;
+ /* The BTF is the one of the running kernel */
+ bool live;
+};
+
+typedef int (*btf_cb)(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter);
+
+#define ERRNO_NAME(e) { e, #e }
+
+/* The errors of the uapi and the kernel internal ones, which libc lacks */
+static const struct {
+ int value;
+ const char *name;
+} errno_names[] = {
+ ERRNO_NAME(EPERM),
+ ERRNO_NAME(ENOENT),
+ ERRNO_NAME(ESRCH),
+ ERRNO_NAME(EINTR),
+ ERRNO_NAME(EIO),
+ ERRNO_NAME(ENXIO),
+ ERRNO_NAME(E2BIG),
+ ERRNO_NAME(ENOEXEC),
+ ERRNO_NAME(EBADF),
+ ERRNO_NAME(ECHILD),
+ ERRNO_NAME(EAGAIN),
+ ERRNO_NAME(ENOMEM),
+ ERRNO_NAME(EACCES),
+ ERRNO_NAME(EFAULT),
+ ERRNO_NAME(ENOTBLK),
+ ERRNO_NAME(EBUSY),
+ ERRNO_NAME(EEXIST),
+ ERRNO_NAME(EXDEV),
+ ERRNO_NAME(ENODEV),
+ ERRNO_NAME(ENOTDIR),
+ ERRNO_NAME(EISDIR),
+ ERRNO_NAME(EINVAL),
+ ERRNO_NAME(ENFILE),
+ ERRNO_NAME(EMFILE),
+ ERRNO_NAME(ENOTTY),
+ ERRNO_NAME(ETXTBSY),
+ ERRNO_NAME(EFBIG),
+ ERRNO_NAME(ENOSPC),
+ ERRNO_NAME(ESPIPE),
+ ERRNO_NAME(EROFS),
+ ERRNO_NAME(EMLINK),
+ ERRNO_NAME(EPIPE),
+ ERRNO_NAME(EDOM),
+ ERRNO_NAME(ERANGE),
+ ERRNO_NAME(EDEADLK),
+#if defined(EDEADLOCK) && EDEADLOCK != EDEADLK
+ ERRNO_NAME(EDEADLOCK),
+#endif
+ ERRNO_NAME(ENAMETOOLONG),
+ ERRNO_NAME(ENOLCK),
+ ERRNO_NAME(ENOSYS),
+ ERRNO_NAME(ENOTEMPTY),
+ ERRNO_NAME(ELOOP),
+ ERRNO_NAME(ENOMSG),
+ ERRNO_NAME(EIDRM),
+ ERRNO_NAME(ECHRNG),
+ ERRNO_NAME(EL2NSYNC),
+ ERRNO_NAME(EL3HLT),
+ ERRNO_NAME(EL3RST),
+ ERRNO_NAME(ELNRNG),
+ ERRNO_NAME(EUNATCH),
+ ERRNO_NAME(ENOCSI),
+ ERRNO_NAME(EL2HLT),
+ ERRNO_NAME(EBADE),
+ ERRNO_NAME(EBADR),
+ ERRNO_NAME(EXFULL),
+ ERRNO_NAME(ENOANO),
+ ERRNO_NAME(EBADRQC),
+ ERRNO_NAME(EBADSLT),
+ ERRNO_NAME(EBFONT),
+ ERRNO_NAME(ENOSTR),
+ ERRNO_NAME(ENODATA),
+ ERRNO_NAME(ETIME),
+ ERRNO_NAME(ENOSR),
+ ERRNO_NAME(ENONET),
+ ERRNO_NAME(ENOPKG),
+ ERRNO_NAME(EREMOTE),
+ ERRNO_NAME(ENOLINK),
+ ERRNO_NAME(EADV),
+ ERRNO_NAME(ESRMNT),
+ ERRNO_NAME(ECOMM),
+ ERRNO_NAME(EPROTO),
+ ERRNO_NAME(EMULTIHOP),
+ ERRNO_NAME(EDOTDOT),
+ ERRNO_NAME(EBADMSG),
+ ERRNO_NAME(EOVERFLOW),
+ ERRNO_NAME(ENOTUNIQ),
+ ERRNO_NAME(EBADFD),
+ ERRNO_NAME(EREMCHG),
+ ERRNO_NAME(ELIBACC),
+ ERRNO_NAME(ELIBBAD),
+ ERRNO_NAME(ELIBSCN),
+ ERRNO_NAME(ELIBMAX),
+ ERRNO_NAME(ELIBEXEC),
+ ERRNO_NAME(EILSEQ),
+ ERRNO_NAME(ERESTART),
+ ERRNO_NAME(ESTRPIPE),
+ ERRNO_NAME(EUSERS),
+ ERRNO_NAME(ENOTSOCK),
+ ERRNO_NAME(EDESTADDRREQ),
+ ERRNO_NAME(EMSGSIZE),
+ ERRNO_NAME(EPROTOTYPE),
+ ERRNO_NAME(ENOPROTOOPT),
+ ERRNO_NAME(EPROTONOSUPPORT),
+ ERRNO_NAME(ESOCKTNOSUPPORT),
+ ERRNO_NAME(EOPNOTSUPP),
+ ERRNO_NAME(EPFNOSUPPORT),
+ ERRNO_NAME(EAFNOSUPPORT),
+ ERRNO_NAME(EADDRINUSE),
+ ERRNO_NAME(EADDRNOTAVAIL),
+ ERRNO_NAME(ENETDOWN),
+ ERRNO_NAME(ENETUNREACH),
+ ERRNO_NAME(ENETRESET),
+ ERRNO_NAME(ECONNABORTED),
+ ERRNO_NAME(ECONNRESET),
+ ERRNO_NAME(ENOBUFS),
+ ERRNO_NAME(EISCONN),
+ ERRNO_NAME(ENOTCONN),
+ ERRNO_NAME(ESHUTDOWN),
+ ERRNO_NAME(ETOOMANYREFS),
+ ERRNO_NAME(ETIMEDOUT),
+ ERRNO_NAME(ECONNREFUSED),
+ ERRNO_NAME(EHOSTDOWN),
+ ERRNO_NAME(EHOSTUNREACH),
+ ERRNO_NAME(EALREADY),
+ ERRNO_NAME(EINPROGRESS),
+ ERRNO_NAME(ESTALE),
+ ERRNO_NAME(EUCLEAN),
+ ERRNO_NAME(ENOTNAM),
+ ERRNO_NAME(ENAVAIL),
+ ERRNO_NAME(EISNAM),
+ ERRNO_NAME(EREMOTEIO),
+ ERRNO_NAME(EDQUOT),
+ ERRNO_NAME(ENOMEDIUM),
+ ERRNO_NAME(EMEDIUMTYPE),
+ ERRNO_NAME(ECANCELED),
+ ERRNO_NAME(ENOKEY),
+ ERRNO_NAME(EKEYEXPIRED),
+ ERRNO_NAME(EKEYREVOKED),
+ ERRNO_NAME(EKEYREJECTED),
+ ERRNO_NAME(EOWNERDEAD),
+ ERRNO_NAME(ENOTRECOVERABLE),
+ ERRNO_NAME(ERFKILL),
+ ERRNO_NAME(EHWPOISON),
+#ifdef EFTYPE
+ ERRNO_NAME(EFTYPE),
+#endif
+ { 512, "ERESTARTSYS" },
+ { 513, "ERESTARTNOINTR" },
+ { 514, "ERESTARTNOHAND" },
+ { 515, "ENOIOCTLCMD" },
+ { 516, "ERESTART_RESTARTBLOCK" },
+ { 517, "EPROBE_DEFER" },
+ { 518, "EOPENSTALE" },
+ { 519, "ENOPARAM" },
+ { 521, "EBADHANDLE" },
+ { 522, "ENOTSYNC" },
+ { 523, "EBADCOOKIE" },
+ { 524, "ENOTSUPP" },
+ { 525, "ETOOSMALL" },
+ { 526, "ESERVERFAULT" },
+ { 527, "EBADTYPE" },
+ { 528, "EJUKEBOX" },
+ { 529, "EIOCBQUEUED" },
+ { 530, "ERECALLCONFLICT" },
+ { 531, "ENOGRACE" },
+};
+
+static const char *errno_name(long long value)
+{
+ size_t i;
+
+ for (i = 0; i < ARRAY_SIZE(errno_names); i++)
+ if (-(long long)errno_names[i].value == value)
+ return errno_names[i].name;
+ return NULL;
+}
+
+static int cmp_entry(const void *a, const void *b)
+{
+ const struct fmodret_entry *x = a, *y = b;
+
+ return (x->btf_id > y->btf_id) - (x->btf_id < y->btf_id);
+}
+
+/* The first type ID which belongs to BTF and not to its base */
+static __u32 first_own_type(const struct btf *btf)
+{
+ const struct btf *base = btf__base_btf(btf);
+
+ return base ? btf__type_cnt(base) : 1;
+}
+
+/*
+ * Return the function which T tags with decl tag NAME, or 0. A NAME which ends
+ * with ':' matches the tags which start with it.
+ */
+static __u32 tag_func(const struct btf *btf, const struct btf_type *t, const char *name)
+{
+ size_t len = strlen(name);
+ const char *tag;
+
+ if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1)
+ return 0;
+ tag = btf__name_by_offset(btf, t->name_off);
+ if (strncmp(tag, name, len) || (name[len - 1] != ':' && tag[len]))
+ return 0;
+ if (!btf_is_func(btf__type_by_id(btf, t->type)))
+ return 0;
+ return t->type;
+}
+
+/* Return the fmod_ret targets of BTF, sorted by BTF ID */
+static int collect_targets(const struct btf *btf, struct fmodret_entry **targets,
+ size_t *target_cnt)
+{
+ __u32 id, func_id, start_id = first_own_type(btf);
+ __u32 nr_types = btf__type_cnt(btf);
+ struct fmodret_entry *entries = NULL, *entry, *tmp;
+ size_t cnt = 0, capacity = 0;
+
+ for (id = start_id; id < nr_types; id++) {
+ func_id = tag_func(btf, btf__type_by_id(btf, id), TAG_FMODRET);
+ if (!func_id)
+ continue;
+
+ if (cnt == capacity) {
+ capacity = capacity ? 2 * capacity : 64;
+ tmp = realloc(entries, capacity * sizeof(*entries));
+ if (!tmp) {
+ free(entries);
+ return -ENOMEM;
+ }
+ entries = tmp;
+ }
+ entries[cnt].btf_id = func_id;
+ entries[cnt].sleepable = false;
+ cnt++;
+ }
+ if (cnt)
+ qsort(entries, cnt, sizeof(*entries), cmp_entry);
+
+ /* The other tags mark functions which are already targets */
+ for (id = start_id; cnt && id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ struct fmodret_entry key = {};
+
+ key.btf_id = tag_func(btf, t, TAG_FMODRET_SLEEPABLE);
+ if (key.btf_id) {
+ entry = bsearch(&key, entries, cnt, sizeof(*entries), cmp_entry);
+ if (entry)
+ entry->sleepable = true;
+ }
+ }
+
+ *targets = entries;
+ *target_cnt = cnt;
+ return 0;
+}
+
+/* Fill ENTRY from the decl tags of function FUNC_ID */
+static void func_entry(const struct btf *btf, __u32 func_id, struct fmodret_entry *entry)
+{
+ __u32 id, nr_types = btf__type_cnt(btf);
+
+ memset(entry, 0, sizeof(*entry));
+ entry->btf_id = func_id;
+ for (id = first_own_type(btf); id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (tag_func(btf, t, TAG_FMODRET) == func_id)
+ entry->target = true;
+ else if (tag_func(btf, t, TAG_FMODRET_SLEEPABLE) == func_id)
+ entry->sleepable = true;
+ else if (tag_func(btf, t, TAG_FMODRET_RETVAL) == func_id) {
+ entry->retval_dup = entry->retval;
+ entry->retval = btf__name_by_offset(btf, t->name_off) +
+ strlen(TAG_FMODRET_RETVAL);
+ }
+ }
+}
+
+static void list_target(const char *owner, const struct btf *btf,
+ const struct fmodret_entry *entry)
+{
+ const struct btf_type *t = btf__type_by_id(btf, entry->btf_id);
+ const char *name = btf__name_by_offset(btf, t->name_off);
+
+ if (json_output) {
+ jsonw_start_object(json_wtr);
+ jsonw_uint_field(json_wtr, "btf_id", entry->btf_id);
+ jsonw_string_field(json_wtr, "name", name);
+ jsonw_string_field(json_wtr, "owner", owner);
+ jsonw_bool_field(json_wtr, "sleepable", entry->sleepable);
+ jsonw_end_object(json_wtr);
+ } else {
+ printf("%u %s (%s)%s\n", entry->btf_id, name, owner,
+ entry->sleepable ? " sleepable" : "");
+ }
+}
+
+static int list_btf(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter)
+{
+ struct fmodret_entry *targets = NULL;
+ size_t cnt = 0, i;
+ int err;
+
+ err = collect_targets(btf, &targets, &cnt);
+ if (err) {
+ p_err("failed to collect fmod_ret targets of %s: %s", owner,
+ strerror(-err));
+ return err;
+ }
+ for (i = 0; i < cnt; i++)
+ list_target(owner, btf, &targets[i]);
+ free(targets);
+ return 0;
+}
+
+struct error_injection {
+ char *name;
+ char *module;
+ char *type;
+};
+
+static struct error_injection *error_injections;
+static size_t error_injection_cnt;
+
+/*
+ * Read the error injection list of the running kernel. Its lines are
+ * "FUNC\tTYPE" and "FUNC [MODULE]\tTYPE". Without debugfs the list is empty.
+ */
+static void read_error_injection_list(void)
+{
+ struct error_injection *tmp;
+ char *line = NULL;
+ size_t len = 0;
+ FILE *f;
+
+ f = fopen(ERROR_INJECTION_LIST, "r");
+ if (!f)
+ return;
+ while (getline(&line, &len, f) > 0) {
+ char *name, *module = NULL, *type, *save;
+
+ /* "NAME [MODULE]\tTYPE" or "NAME\tTYPE" */
+ name = strtok_r(line, " \t\n", &save);
+ type = strtok_r(NULL, " \t\n", &save);
+ if (type && type[0] == '[') {
+ module = type + 1;
+ module[strcspn(module, "]")] = '\0';
+ type = strtok_r(NULL, " \t\n", &save);
+ }
+ if (!name || !type)
+ continue;
+ tmp = realloc(error_injections,
+ (error_injection_cnt + 1) * sizeof(*error_injections));
+ if (!tmp)
+ break;
+ error_injections = tmp;
+ tmp = &error_injections[error_injection_cnt];
+ tmp->name = strdup(name);
+ tmp->module = strdup(module ?: "");
+ tmp->type = strdup(type);
+ if (!tmp->name || !tmp->module || !tmp->type) {
+ free(tmp->name);
+ free(tmp->module);
+ free(tmp->type);
+ break;
+ }
+ error_injection_cnt++;
+ }
+ free(line);
+ fclose(f);
+}
+
+static void free_error_injection_list(void)
+{
+ size_t i;
+
+ for (i = 0; i < error_injection_cnt; i++) {
+ free(error_injections[i].name);
+ free(error_injections[i].module);
+ free(error_injections[i].type);
+ }
+ free(error_injections);
+ error_injections = NULL;
+ error_injection_cnt = 0;
+}
+
+/* Return the error injection type of function NAME of OWNER, or NULL */
+static const char *error_injection_type(const char *name, const char *owner)
+{
+ const char *module = strcmp(owner, "vmlinux") ? owner : "";
+ size_t i;
+
+ for (i = 0; i < error_injection_cnt; i++)
+ if (!strcmp(error_injections[i].name, name) &&
+ !strcmp(error_injections[i].module, module))
+ return error_injections[i].type;
+ return NULL;
+}
+
+static void __printf(2, 0) prototype_printf(void *ctx, const char *fmt, va_list args)
+{
+ vfprintf(ctx, fmt, args);
+}
+
+/* Return the declaration of function FUNC_ID, or NULL */
+static char *func_prototype(const struct btf *btf, __u32 func_id)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func_id);
+ LIBBPF_OPTS(btf_dump_emit_type_decl_opts, opts,
+ .field_name = btf__name_by_offset(btf, t->name_off),
+ );
+ struct btf_dump *d;
+ char *buf = NULL;
+ size_t size = 0;
+ FILE *f;
+ int err;
+
+ f = open_memstream(&buf, &size);
+ if (!f)
+ return NULL;
+ d = btf_dump__new(btf, prototype_printf, f, NULL);
+ err = libbpf_get_error(d);
+ if (!err) {
+ err = btf_dump__emit_type_decl(d, t->type, &opts);
+ btf_dump__free(d);
+ }
+ fclose(f);
+ if (err) {
+ free(buf);
+ return NULL;
+ }
+ return buf;
+}
+
+/* Show 0 and the values of a "bpf_fmodret_retval:" tag as a JSON array */
+static void show_retval_json(const char *retval)
+{
+ const char *p = retval;
+ char *end;
+
+ jsonw_name(json_wtr, "retval");
+ jsonw_start_array(json_wtr);
+ jsonw_int(json_wtr, 0);
+ while (*p) {
+ long long value;
+
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno)
+ break;
+ jsonw_int(json_wtr, value);
+ p = *end == ',' ? end + 1 : end;
+ }
+ jsonw_end_array(json_wtr);
+}
+
+/*
+ * Check the "bpf_fmodret_retval:" tag of FUNC_ID as the verifier does: negative
+ * decimal numbers separated by commas, which the integer or enum of 4 or 8 bytes
+ * that the function returns can hold. Return why it is invalid, or NULL.
+ */
+static const char *retval_invalid(const struct btf *btf, __u32 func_id,
+ const char *retval)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func_id);
+ const char *p = retval;
+ int ret_id;
+ char *end;
+
+ t = btf__type_by_id(btf, t->type);
+ ret_id = t ? btf__resolve_type(btf, t->type) : -EINVAL;
+ t = ret_id > 0 ? btf__type_by_id(btf, ret_id) : NULL;
+ if (!t || !(btf_is_int(t) || btf_is_any_enum(t)) || (t->size != 4 && t->size != 8))
+ return "the function does not return an integer of 4 or 8 bytes";
+ for (;;) {
+ long long value;
+
+ if (*p != '-')
+ return "malformed tag";
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno || value >= 0)
+ return "malformed tag";
+ if (t->size == 4 && value < INT_MIN)
+ return "a value does not fit the return type";
+ if (!*end)
+ return NULL;
+ if (*end != ',')
+ return "malformed tag";
+ p = end + 1;
+ }
+}
+
+/* Show the values of a "bpf_fmodret_retval:" tag with the names of errors */
+static void show_retval_plain(const char *retval)
+{
+ const char *p = retval, *name;
+ char *end;
+
+ printf("\tretval: 0");
+ while (*p) {
+ long long value;
+
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno)
+ break;
+ printf(", %lld", value);
+ name = errno_name(value);
+ if (name)
+ printf(" (%s)", name);
+ p = *end == ',' ? end + 1 : end;
+ }
+ printf("\n");
+}
+
+static int show_function(const char *owner, const struct btf *btf,
+ const struct fmodret_entry *entry,
+ struct fmodret_filter *filter)
+{
+ const struct btf_type *t = btf__type_by_id(btf, entry->btf_id);
+ const char *name = btf__name_by_offset(btf, t->name_off);
+ const char *injection = NULL, *invalid = NULL;
+ char *prototype;
+
+ /* The verifier rejects programs on a function with an invalid tag */
+ if (entry->target && entry->retval_dup)
+ invalid = "more than one tag";
+ else if (entry->target && entry->retval)
+ invalid = retval_invalid(btf, entry->btf_id, entry->retval);
+ prototype = func_prototype(btf, entry->btf_id);
+ if (!prototype) {
+ p_err("failed to print the prototype of %s", name);
+ return -EINVAL;
+ }
+ if (filter->live)
+ injection = error_injection_type(name, owner);
+
+ if (json_output) {
+ jsonw_start_object(json_wtr);
+ jsonw_uint_field(json_wtr, "btf_id", entry->btf_id);
+ jsonw_string_field(json_wtr, "name", name);
+ jsonw_string_field(json_wtr, "owner", owner);
+ jsonw_string_field(json_wtr, "prototype", prototype);
+ jsonw_string_field(json_wtr, "target", !entry->target ? "none" :
+ entry->retval ? "found" : "declared");
+ jsonw_bool_field(json_wtr, "sleepable", entry->target && entry->sleepable);
+ if (invalid)
+ jsonw_string_field(json_wtr, "retval_error", invalid);
+ else if (entry->target && entry->retval)
+ show_retval_json(entry->retval);
+ if (injection)
+ jsonw_string_field(json_wtr, "error_injection", injection);
+ jsonw_end_object(json_wtr);
+ } else {
+ printf("%u %s (%s)\n", entry->btf_id, name, owner);
+ printf("\t%s\n", prototype);
+ if (entry->target) {
+ printf("\ttarget: %s\n", entry->retval ? "found by the kernel build" :
+ "declared");
+ printf("\tsleepable: %s\n", entry->sleepable ? "yes" : "no");
+ if (invalid)
+ printf("\tretval: invalid, %s\n", invalid);
+ else if (entry->retval)
+ show_retval_plain(entry->retval);
+ else
+ printf("\tretval: any\n");
+ } else {
+ printf("\ttarget: not tagged, the verifier may accept it by other rules\n");
+ }
+ if (injection)
+ printf("\terror injection: %s\n", injection);
+ }
+ free(prototype);
+ filter->found++;
+ return 0;
+}
+
+static int show_btf(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter)
+{
+ __u32 id, nr_types = btf__type_cnt(btf);
+ struct fmodret_entry entry;
+ int err;
+
+ if (filter->btf_id) {
+ if (filter->btf_id < first_own_type(btf) || filter->btf_id >= nr_types ||
+ !btf_is_func(btf__type_by_id(btf, filter->btf_id)))
+ return 0;
+ func_entry(btf, filter->btf_id, &entry);
+ return show_function(owner, btf, &entry, filter);
+ }
+
+ for (id = first_own_type(btf); id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_func(t) ||
+ strcmp(btf__name_by_offset(btf, t->name_off), filter->name))
+ continue;
+ func_entry(btf, id, &entry);
+ err = show_function(owner, btf, &entry, filter);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+static int filter_module(const struct dirent *entry)
+{
+ size_t len = strlen(entry->d_name);
+
+ /* Skip ".", "..", vmlinux and the ".inline" companions of modules */
+ return entry->d_name[0] != '.' && strcmp(entry->d_name, "vmlinux") &&
+ !(len > 7 && !strcmp(entry->d_name + len - 7, ".inline"));
+}
+
+static int walk_modules(struct btf *vmlinux, struct fmodret_filter *filter, btf_cb cb)
+{
+ struct dirent **modules = NULL;
+ int i, nr_modules, err = 0;
+
+ if (filter->module) {
+ struct btf *btf;
+
+ btf = btf__load_module_btf(filter->module, vmlinux);
+ err = libbpf_get_error(btf);
+ if (err) {
+ p_err("failed to load BTF of module %s: %s", filter->module,
+ strerror(-err));
+ return err;
+ }
+ err = cb(filter->module, btf, filter);
+ btf__free(btf);
+ return err;
+ }
+
+ nr_modules = scandir(SYS_BTF_DIR, &modules, filter_module, alphasort);
+ if (nr_modules < 0) {
+ /* There is no module directory without module BTF, which is fine */
+ if (errno == ENOENT)
+ return 0;
+ err = -errno;
+ p_err("failed to read %s: %s", SYS_BTF_DIR, strerror(errno));
+ return err;
+ }
+ for (i = 0; i < nr_modules; i++) {
+ const char *name = modules[i]->d_name;
+ struct btf *btf;
+
+ btf = btf__load_module_btf(name, vmlinux);
+ err = libbpf_get_error(btf);
+ if (err == -ENOENT) {
+ /* The module was unloaded meanwhile */
+ err = 0;
+ continue;
+ }
+ if (err) {
+ p_err("failed to load BTF of module %s: %s", name, strerror(-err));
+ break;
+ }
+ err = cb(name, btf, filter);
+ btf__free(btf);
+ if (err)
+ break;
+ }
+ for (i = 0; i < nr_modules; i++)
+ free(modules[i]);
+ free(modules);
+ return err;
+}
+
+static int walk_live_btf(struct fmodret_filter *filter, btf_cb cb)
+{
+ struct btf *vmlinux;
+ int err = 0;
+
+ vmlinux = btf__load_vmlinux_btf();
+ err = libbpf_get_error(vmlinux);
+ if (err) {
+ p_err("failed to load vmlinux BTF: %s", strerror(-err));
+ return err;
+ }
+ if (!filter->module)
+ err = cb("vmlinux", vmlinux, filter);
+ if (!err && !filter->vmlinux)
+ err = walk_modules(vmlinux, filter, cb);
+ btf__free(vmlinux);
+ return err;
+}
+
+static int walk_file_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)
+{
+ struct btf *btf;
+ int err;
+
+ btf = base_btf ? btf__parse_split(path, base_btf) : btf__parse(path, NULL);
+ err = libbpf_get_error(btf);
+ if (err) {
+ p_err("failed to parse BTF from %s: %s", path, strerror(-err));
+ return err;
+ }
+ err = cb(path, btf, filter);
+ btf__free(btf);
+ return err;
+}
+
+/* Parse the optional OBJECT := { file FILE | vmlinux | module NAME } */
+static int parse_object(int argc, char **argv, const char **path,
+ struct fmodret_filter *filter)
+{
+ if (!argc)
+ return 0;
+ if (is_prefix(*argv, "vmlinux")) {
+ filter->vmlinux = true;
+ NEXT_ARG();
+ } else if (is_prefix(*argv, "file")) {
+ NEXT_ARG();
+ if (!REQ_ARGS(1))
+ return -1;
+ *path = GET_ARG();
+ } else if (is_prefix(*argv, "module")) {
+ NEXT_ARG();
+ if (!REQ_ARGS(1))
+ return -1;
+ filter->module = GET_ARG();
+ }
+ if (argc)
+ return BAD_ARG();
+ return 0;
+}
+
+static int walk_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)
+{
+ return path ? walk_file_btf(path, filter, cb) : walk_live_btf(filter, cb);
+}
+
+static int do_list(int argc, char **argv)
+{
+ struct fmodret_filter filter = {};
+ const char *path = NULL;
+ int err;
+
+ if (parse_object(argc, argv, &path, &filter))
+ return -1;
+
+ if (json_output)
+ jsonw_start_array(json_wtr);
+ err = walk_btf(path, &filter, list_btf);
+ if (json_output)
+ jsonw_end_array(json_wtr);
+ return err;
+}
+
+static int do_show(int argc, char **argv)
+{
+ struct fmodret_filter filter = {};
+ const char *path = NULL;
+ unsigned long id;
+ char *endptr;
+ int err;
+
+ if (!REQ_ARGS(2))
+ return -1;
+ if (is_prefix(*argv, "name")) {
+ NEXT_ARG();
+ filter.name = GET_ARG();
+ } else if (is_prefix(*argv, "id")) {
+ NEXT_ARG();
+ errno = 0;
+ id = strtoul(*argv, &endptr, 0);
+ if (errno || *endptr || !id || id > UINT_MAX || **argv == '-') {
+ p_err("can't parse %s as BTF ID", *argv);
+ return -1;
+ }
+ filter.btf_id = id;
+ NEXT_ARG();
+ } else {
+ p_err("expected 'name' or 'id', got: '%s'", *argv);
+ return -1;
+ }
+ if (parse_object(argc, argv, &path, &filter))
+ return -1;
+ /* The BTF IDs of different modules overlap */
+ if (filter.btf_id && !path && !filter.module)
+ filter.vmlinux = true;
+
+ filter.live = !path;
+ if (filter.live)
+ read_error_injection_list();
+ /* A BTF ID gives one function, a name an array of them */
+ if (json_output && !filter.btf_id)
+ jsonw_start_array(json_wtr);
+ err = walk_btf(path, &filter, show_btf);
+ if (!err && !filter.found) {
+ if (filter.name)
+ p_err("no function named %s", filter.name);
+ else
+ p_err("no function with BTF ID %u", filter.btf_id);
+ err = -ENOENT;
+ }
+ if (json_output && !filter.btf_id)
+ jsonw_end_array(json_wtr);
+ free_error_injection_list();
+ return err;
+}
+
+static int do_help(int argc, char **argv)
+{
+ if (json_output) {
+ jsonw_null(json_wtr);
+ return 0;
+ }
+
+ fprintf(stderr,
+ "Usage: %1$s %2$s list [OBJECT]\n"
+ " %1$s %2$s show FUNC [OBJECT]\n"
+ " %1$s %2$s help\n"
+ "\n"
+ " FUNC := { name NAME | id BTF_ID }\n"
+ " OBJECT := { file FILE | vmlinux | module NAME }\n"
+ " " HELP_SPEC_OPTIONS " |\n"
+ " {-B|--base-btf} FILE }\n"
+ "\n",
+ bin_name, "fmodret");
+ return 0;
+}
+
+static const struct cmd cmds[] = {
+ { "list", do_list },
+ { "show", do_show },
+ { "help", do_help },
+ { 0 }
+};
+
+int do_fmodret(int argc, char **argv)
+{
+ return cmd_select(cmds, argc, argv, do_help);
+}
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index d15fec71b64eec..e2a7571b097c7e 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -65,7 +65,7 @@ static int do_help(int argc, char **argv)
" %s batch file FILE\n"
" %s version\n"
"\n"
- " OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | token }\n"
+ " OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | fmodret | token }\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-V|--version} }\n"
"",
@@ -91,6 +91,7 @@ static const struct cmd commands[] = {
{ "gen", do_gen },
{ "struct_ops", do_struct_ops },
{ "iter", do_iter },
+ { "fmodret", do_fmodret },
{ "token", do_token },
{ "version", do_version },
{ 0 }
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index b1eb4b5d991b8a..4d877b803b4765 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -178,6 +178,7 @@ int do_tracelog(int argc, char **arg) __weak;
int do_feature(int argc, char **argv) __weak;
int do_struct_ops(int argc, char **argv) __weak;
int do_iter(int argc, char **argv) __weak;
+int do_fmodret(int argc, char **argv) __weak;
int do_token(int argc, char **argv) __weak;
int parse_u32_arg(int *argc, char ***argv, __u32 *val, const char *what);
diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
index 602e1508bf0922..af777afa451b27 100644
--- a/tools/bpf/resolve_btfids/Build
+++ b/tools/bpf/resolve_btfids/Build
@@ -1,6 +1,7 @@
hostprogs := resolve_btfids
resolve_btfids-y += main.o
+resolve_btfids-y += fmodret.o
resolve_btfids-y += btf_colors.o
resolve_btfids-y += rbtree.o
resolve_btfids-y += zalloc.o
diff --git a/tools/bpf/resolve_btfids/fmodret.c b/tools/bpf/resolve_btfids/fmodret.c
new file mode 100644
index 00000000000000..df44b93eea734c
--- /dev/null
+++ b/tools/bpf/resolve_btfids/fmodret.c
@@ -0,0 +1,1205 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Read the records which the bpf_scan_ir compiler plugin writes to
+ * .BTF_fmodret_candidates, see scripts/llvm-plugins/bpf_scan_ir.cpp, and find
+ * the functions which can become fmod_ret targets. A record lists the errors
+ * which clean paths of a function return, that is paths without side effects.
+ *
+ * A function is chosen if
+ * - exactly one record describes its body, or one record of a strong
+ * definition next to records of the weak ones it overrides,
+ * - the record is of a strong definition or the symbol is still weak,
+ * - the record lists at least one error,
+ * - it has an ftrace call site and is not init, exit or noinstr code,
+ * - its name belongs to exactly one ELF function and one BTF FUNC,
+ * - it returns an integer of the size the record describes.
+ *
+ * The plugin records no errors for functions which return pointers yet.
+ */
+
+#include <errno.h>
+#include <gelf.h>
+#include <libelf.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <bpf/btf.h>
+#include <linux/kernel.h>
+
+#include "fmodret.h"
+#include "../../../scripts/bpf/fmodret_metadata.h"
+
+#define FMODRET_CANDIDATES_SECTION ".BTF_fmodret_candidates"
+#define NO_FUNCTION SIZE_MAX
+
+struct elf_function {
+ const char *name;
+ GElf_Addr value;
+ GElf_Xword size;
+ Elf64_Section shndx;
+ bool traceable;
+ /* The symbol is weak: no strong definition replaced it at the link */
+ bool weak_symbol;
+ /* The records which describe the body at this address */
+ unsigned int nr_strong, nr_weak;
+ size_t strong, weak;
+};
+
+struct elf_function_list {
+ struct elf_function *functions;
+ size_t nr;
+ size_t capacity;
+};
+
+struct fmodret_stats {
+ size_t records, unmapped, with_values;
+ size_t ambiguous_body, not_traceable, ambiguous_name;
+ size_t overridden, no_btf, return_mismatch, added;
+};
+
+static int find_section(Elf *elf, size_t shstrndx, const char *wanted,
+ Elf_Scn **result)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ const char *name;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ return -1;
+ if (!strcmp(name, wanted)) {
+ *result = scn;
+ return 0;
+ }
+ }
+
+ *result = NULL;
+ return 0;
+}
+
+static int find_symbol(Elf *elf, const char *wanted, GElf_Sym *result,
+ bool *found)
+{
+ Elf_Scn *scn = NULL;
+
+ *found = false;
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_symbols;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+ nr_symbols = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_symbols; i++) {
+ const char *name;
+ GElf_Sym sym;
+
+ if (!gelf_getsym(data, i, &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF || !sym.st_name)
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (!name)
+ return -1;
+ if (strcmp(name, wanted))
+ continue;
+ *result = sym;
+ *found = true;
+ return 0;
+ }
+ }
+ return 0;
+}
+
+static int section_for_address(Elf *elf, GElf_Addr address,
+ Elf64_Section *result)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (!(shdr.sh_flags & SHF_EXECINSTR) || address < shdr.sh_addr ||
+ address - shdr.sh_addr >= shdr.sh_size)
+ continue;
+ *result = elf_ndxscn(scn);
+ return 0;
+ }
+ *result = SHN_UNDEF;
+ return 0;
+}
+
+static bool relocation_offset(const GElf_Shdr *target, GElf_Addr offset,
+ size_t width, size_t *result)
+{
+ if (target->sh_size < width)
+ return false;
+ if (offset >= target->sh_addr &&
+ offset - target->sh_addr <= target->sh_size - width) {
+ *result = offset - target->sh_addr;
+ return true;
+ }
+ /* ET_REL relocation offsets are relative to their target section. */
+ if (offset <= target->sh_size - width) {
+ *result = offset;
+ return true;
+ }
+ return false;
+}
+
+static uint64_t read_elf_value(const unsigned char *p, size_t size,
+ unsigned int encoding)
+{
+ uint64_t value = 0;
+ size_t i;
+
+ if (encoding == ELFDATA2LSB) {
+ for (i = 0; i < size; i++)
+ value |= (uint64_t)p[i] << (8 * i);
+ } else {
+ for (i = 0; i < size; i++)
+ value = (value << 8) | p[i];
+ }
+ return value;
+}
+
+static int copy_section_data(Elf_Scn *scn, const GElf_Shdr *shdr,
+ unsigned char **result)
+{
+ Elf_Data *data = NULL;
+ unsigned char *buf;
+
+ buf = calloc(1, shdr->sh_size ?: 1);
+ if (!buf)
+ return -ENOMEM;
+ while ((data = elf_getdata(scn, data))) {
+ if (data->d_off > shdr->sh_size ||
+ data->d_size > shdr->sh_size - data->d_off) {
+ free(buf);
+ return -EINVAL;
+ }
+ memcpy(buf + data->d_off, data->d_buf, data->d_size);
+ }
+ *result = buf;
+ return 0;
+}
+
+static int add_elf_function(struct elf_function_list *list, const char *name,
+ const GElf_Sym *sym)
+{
+ struct elf_function *functions;
+ size_t capacity;
+
+ if (list->nr == list->capacity) {
+ capacity = list->capacity ? 2 * list->capacity : 1024;
+ if (capacity > SIZE_MAX / sizeof(*functions))
+ return -E2BIG;
+ functions = realloc(list->functions, capacity * sizeof(*functions));
+ if (!functions)
+ return -ENOMEM;
+ list->functions = functions;
+ list->capacity = capacity;
+ }
+
+ list->functions[list->nr++] = (struct elf_function) {
+ .name = name,
+ .value = sym->st_value,
+ .size = sym->st_size,
+ .shndx = sym->st_shndx,
+ .weak_symbol = GELF_ST_BIND(sym->st_info) == STB_WEAK,
+ .strong = NO_FUNCTION,
+ .weak = NO_FUNCTION,
+ };
+ return 0;
+}
+
+static int collect_elf_functions(Elf *elf, struct elf_function_list *list)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_symbols;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+
+ nr_symbols = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_symbols; i++) {
+ const char *name;
+ GElf_Sym sym;
+ int err;
+
+ if (!gelf_getsym(data, i, &sym))
+ return -1;
+ if (GELF_ST_TYPE(sym.st_info) != STT_FUNC ||
+ sym.st_shndx == SHN_UNDEF || !sym.st_name)
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (!name)
+ return -1;
+ err = add_elf_function(list, name, &sym);
+ if (err)
+ return err;
+ }
+ }
+ return 0;
+}
+
+static int cmp_elf_function_location(const void *va, const void *vb)
+{
+ const struct elf_function *a = va;
+ const struct elf_function *b = vb;
+
+ if (a->shndx != b->shndx)
+ return a->shndx > b->shndx ? 1 : -1;
+ if (a->value != b->value)
+ return a->value > b->value ? 1 : -1;
+ return 0;
+}
+
+static int cmp_elf_function_name(const void *va, const void *vb)
+{
+ const struct elf_function *a = va;
+ const struct elf_function *b = vb;
+
+ return strcmp(a->name, b->name) ?: cmp_elf_function_location(va, vb);
+}
+
+/* Return the first function at VALUE in SHNDX. Functions are sorted by location. */
+static size_t function_at(const struct elf_function_list *list,
+ Elf64_Section shndx, GElf_Addr value)
+{
+ size_t lo = 0, hi = list->nr;
+
+ while (lo < hi) {
+ size_t mid = lo + (hi - lo) / 2;
+ const struct elf_function *f = &list->functions[mid];
+
+ if (f->shndx < shndx || (f->shndx == shndx && f->value < value))
+ lo = mid + 1;
+ else
+ hi = mid;
+ }
+ if (lo == list->nr || list->functions[lo].shndx != shndx ||
+ list->functions[lo].value != value)
+ return NO_FUNCTION;
+ return lo;
+}
+
+static bool function_contains(const struct elf_function *function,
+ GElf_Addr address)
+{
+ if (address < function->value)
+ return false;
+ if (!function->size)
+ return address == function->value;
+ return address - function->value < function->size;
+}
+
+enum ftrace_site_kind {
+ FTRACE_SITE_MCOUNT,
+ FTRACE_SITE_PATCHABLE,
+ FTRACE_SITE_UNKNOWN,
+};
+
+/*
+ * The NOPs which arm64 puts before a function with CALL_OPS. From the entry of
+ * an overridden weak function, whose body has no symbol left, a longer distance
+ * could reach the next function. Other arches may need more.
+ */
+#define MAX_PATCHABLE_FUNCTION_ENTRY_SIZE 8
+
+/* Return true if function I or another function at its address contains ADDRESS */
+static bool alias_contains(const struct elf_function_list *list, size_t i,
+ GElf_Addr address)
+{
+ const struct elf_function *f = &list->functions[i];
+
+ for (;;) {
+ if (function_contains(&list->functions[i], address))
+ return true;
+ if (!i || list->functions[i - 1].shndx != f->shndx ||
+ list->functions[i - 1].value != f->value)
+ return false;
+ i--;
+ }
+}
+
+/*
+ * Mark the functions at (SHNDX, VALUE) traceable, or, with CHECK, only those
+ * which contain ADDRESS.
+ */
+static void mark_aliases(struct elf_function_list *list, Elf64_Section shndx,
+ GElf_Addr value, GElf_Addr address, bool check)
+{
+ size_t first = function_at(list, shndx, value), i;
+
+ if (first == NO_FUNCTION)
+ return;
+ for (i = first; i < list->nr && list->functions[i].shndx == shndx &&
+ list->functions[i].value == value; i++) {
+ if (check && !function_contains(&list->functions[i], address))
+ continue;
+ list->functions[i].traceable = true;
+ }
+}
+
+/*
+ * Mark the function containing an ftrace call site. Functions are sorted by
+ * section and address. Mark all aliases at the same address: at run time BTF
+ * attachment is name based and may resolve to any of them.
+ */
+static void mark_traceable_function(struct elf_function_list *list,
+ Elf64_Section shndx, GElf_Addr address,
+ enum ftrace_site_kind kind)
+{
+ size_t left = 0;
+ size_t right = list->nr;
+ size_t first;
+
+ /* Find the first function after ADDRESS in SHNDX. */
+ while (left < right) {
+ size_t middle = left + (right - left) / 2;
+ struct elf_function *function = &list->functions[middle];
+
+ if (function->shndx < shndx ||
+ (function->shndx == shndx && function->value <= address))
+ left = middle + 1;
+ else
+ right = middle;
+ }
+ /*
+ * A patchable entry is at the start of its function, a few bytes into it
+ * (after an arm64 BTI) or in the NOPs before it
+ */
+ first = SIZE_MAX;
+ if (kind != FTRACE_SITE_MCOUNT) {
+ if (left && list->functions[left - 1].shndx == shndx &&
+ address - list->functions[left - 1].value <
+ MAX_PATCHABLE_FUNCTION_ENTRY_SIZE &&
+ alias_contains(list, left - 1, address))
+ first = left - 1;
+ else if (left < list->nr && list->functions[left].shndx == shndx &&
+ list->functions[left].value - address <=
+ MAX_PATCHABLE_FUNCTION_ENTRY_SIZE)
+ first = left;
+ }
+ if (first != SIZE_MAX) {
+ mark_aliases(list, shndx, list->functions[first].value, address, false);
+ return;
+ }
+ if (kind == FTRACE_SITE_PATCHABLE || !left)
+ return;
+
+ /* The nearest preceding function owns the call site. */
+ first = left - 1;
+ if (list->functions[first].shndx != shndx)
+ return;
+ mark_aliases(list, shndx, list->functions[first].value, address, true);
+}
+
+static int mark_relocated_ftrace_sites(Elf *elf, size_t target_ndx,
+ const unsigned char *target_data,
+ const GElf_Shdr *target_shdr,
+ size_t begin, size_t end,
+ size_t ptr_size, unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind,
+ bool *relocated)
+{
+ Elf_Scn *scn = NULL;
+
+ *relocated = false;
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_relocations;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if ((shdr.sh_type != SHT_RELA && shdr.sh_type != SHT_REL) ||
+ (shdr.sh_info && shdr.sh_info != target_ndx))
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+
+ nr_relocations = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_relocations; i++) {
+ Elf_Scn *symtab_scn;
+ Elf_Data *symtab_data;
+ GElf_Shdr symtab_shdr;
+ GElf_Sxword addend;
+ GElf_Addr raw_offset;
+ GElf_Addr address;
+ GElf_Sym sym;
+ size_t sym_ndx;
+ size_t offset;
+
+ if (shdr.sh_type == SHT_RELA) {
+ GElf_Rela rela;
+
+ if (!gelf_getrela(data, i, &rela))
+ return -1;
+ sym_ndx = GELF_R_SYM(rela.r_info);
+ raw_offset = rela.r_offset;
+ addend = rela.r_addend;
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ return -1;
+ sym_ndx = GELF_R_SYM(rel.r_info);
+ raw_offset = rel.r_offset;
+ if (!relocation_offset(target_shdr, raw_offset,
+ ptr_size, &offset))
+ continue;
+ addend = read_elf_value(target_data + offset,
+ ptr_size, encoding);
+ }
+ if (!relocation_offset(target_shdr, raw_offset, ptr_size,
+ &offset))
+ continue;
+ if (offset < begin || offset >= end)
+ continue;
+ *relocated = true;
+
+ /*
+ * PIE vmlinux uses symbol-less RELATIVE relocations. Its
+ * addend is the link-time call-site address, represented as
+ * a signed ELF addend.
+ */
+ if (!sym_ndx) {
+ Elf64_Section target_shndx;
+
+ address = (GElf_Addr)addend;
+ if (section_for_address(elf, address,
+ &target_shndx))
+ return -1;
+ if (target_shndx != SHN_UNDEF)
+ mark_traceable_function(functions,
+ target_shndx,
+ address, kind);
+ continue;
+ }
+
+ symtab_scn = elf_getscn(elf, shdr.sh_link);
+ if (!symtab_scn ||
+ !gelf_getshdr(symtab_scn, &symtab_shdr))
+ return -1;
+ symtab_data = elf_getdata(symtab_scn, NULL);
+ if (!symtab_data || elf_getdata(symtab_scn, symtab_data) ||
+ !gelf_getsym(symtab_data, sym_ndx, &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF)
+ continue;
+ if (sym.st_shndx >= SHN_LORESERVE)
+ continue;
+ if (addend < 0) {
+ GElf_Addr delta = -(addend + 1);
+
+ delta++;
+ if (delta > sym.st_value)
+ continue;
+ address = sym.st_value - delta;
+ } else {
+ if ((GElf_Addr)addend > UINT64_MAX - sym.st_value)
+ continue;
+ address = sym.st_value + addend;
+ }
+ mark_traceable_function(functions, sym.st_shndx, address,
+ kind);
+ }
+ }
+ return 0;
+}
+
+static int read_ftrace_site_range(Elf *elf, Elf_Scn *scn, size_t begin,
+ size_t end, size_t ptr_size,
+ unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind)
+{
+ unsigned char *buf = NULL;
+ GElf_Shdr shdr;
+ bool relocated;
+ size_t offset;
+ int err;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (begin > end || end > shdr.sh_size ||
+ (end - begin) % ptr_size)
+ return -EINVAL;
+ if (begin == end)
+ return 0;
+ err = copy_section_data(scn, &shdr, &buf);
+ if (err)
+ return err;
+ err = mark_relocated_ftrace_sites(elf, elf_ndxscn(scn), buf, &shdr,
+ begin, end, ptr_size, encoding,
+ functions, kind, &relocated);
+ if (err || relocated)
+ goto out;
+
+ for (offset = begin; offset < end; offset += ptr_size) {
+ GElf_Addr address;
+ Elf64_Section shndx;
+
+ address = read_elf_value(buf + offset, ptr_size, encoding);
+ err = section_for_address(elf, address, &shndx);
+ if (err)
+ goto out;
+ if (shndx != SHN_UNDEF)
+ mark_traceable_function(functions, shndx, address, kind);
+ }
+out:
+ free(buf);
+ return err;
+}
+
+static bool section_prefix(const char *name, const char *prefix)
+{
+ size_t len = strlen(prefix);
+
+ return !strncmp(name, prefix, len) &&
+ (!name[len] || name[len] == '.');
+}
+
+static bool section_cannot_be_attached(const char *name)
+{
+ return section_prefix(name, ".init.text") ||
+ section_prefix(name, ".exit.text") ||
+ section_prefix(name, ".noinstr.text");
+}
+
+static int remove_unattachable_functions(Elf *elf,
+ struct elf_function_list *functions,
+ size_t shstrndx)
+{
+ GElf_Sym noinstr_start;
+ GElf_Sym noinstr_end;
+ bool found_start;
+ bool found_end;
+ size_t i;
+ int err;
+
+ err = find_symbol(elf, "__noinstr_text_start", &noinstr_start,
+ &found_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__noinstr_text_end", &noinstr_end, &found_end);
+ if (err)
+ return err;
+
+ for (i = 0; i < functions->nr; i++) {
+ struct elf_function *function = &functions->functions[i];
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ const char *name;
+
+ if (!function->traceable)
+ continue;
+ scn = elf_getscn(elf, function->shndx);
+ if (!scn || !gelf_getshdr(scn, &shdr))
+ return -1;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ return -1;
+ if (section_cannot_be_attached(name))
+ function->traceable = false;
+ else if (found_start && found_end &&
+ function->shndx == noinstr_start.st_shndx &&
+ noinstr_start.st_shndx == noinstr_end.st_shndx &&
+ function->value >= noinstr_start.st_value &&
+ function->value < noinstr_end.st_value)
+ function->traceable = false;
+ }
+ return 0;
+}
+
+static int read_ftrace_symbol_range(Elf *elf, const GElf_Sym *start,
+ const GElf_Sym *stop, size_t ptr_size,
+ unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind)
+{
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ size_t begin;
+ size_t end;
+
+ if (start->st_shndx != stop->st_shndx)
+ return -EINVAL;
+ scn = elf_getscn(elf, start->st_shndx);
+ if (!scn || !gelf_getshdr(scn, &shdr))
+ return -1;
+ if (start->st_value < shdr.sh_addr || stop->st_value < start->st_value ||
+ stop->st_value - shdr.sh_addr > shdr.sh_size)
+ return -EINVAL;
+ begin = start->st_value - shdr.sh_addr;
+ end = stop->st_value - shdr.sh_addr;
+ return read_ftrace_site_range(elf, scn, begin, end, ptr_size, encoding,
+ functions, kind);
+}
+
+/*
+ * Mark the functions with an ftrace call site. A module keeps the sites in the
+ * section FTRACE_CALLSITE_SECTION: __patchable_function_entries if PATCHABLE,
+ * else __mcount_loc. x86 also has call padding entries, which are not call
+ * sites, in __patchable_function_entries. vmlinux merges the sites into init
+ * data between the __start_mcount_loc and __stop_mcount_loc symbols, see
+ * MCOUNT_REC().
+ */
+static int read_ftrace_sites(Elf *elf, size_t shstrndx,
+ size_t ptr_size, unsigned int encoding,
+ struct elf_function_list *functions, bool patchable)
+{
+ const char *section = patchable ? "__patchable_function_entries" : "__mcount_loc";
+ Elf_Scn *scn = NULL;
+ GElf_Sym start;
+ GElf_Sym stop;
+ GElf_Sym patchable_start;
+ GElf_Sym patchable_stop;
+ bool found_start;
+ bool found_stop;
+ bool found_patchable_start;
+ bool found_patchable_stop;
+ GElf_Shdr shdr;
+ int err;
+
+ err = find_section(elf, shstrndx, section, &scn);
+ if (err)
+ return err;
+ if (scn) {
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ err = read_ftrace_site_range(elf, scn, 0, shdr.sh_size,
+ ptr_size, encoding, functions,
+ patchable ? FTRACE_SITE_PATCHABLE :
+ FTRACE_SITE_MCOUNT);
+ if (err)
+ return err;
+ }
+
+ err = find_symbol(elf, "__start_mcount_loc", &start, &found_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__stop_mcount_loc", &stop, &found_stop);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__start_patchable_function_entries",
+ &patchable_start, &found_patchable_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__stop_patchable_function_entries",
+ &patchable_stop, &found_patchable_stop);
+ if (err)
+ return err;
+ if (found_start && found_stop) {
+ if (found_patchable_start && found_patchable_stop) {
+ err = read_ftrace_symbol_range(elf, &start,
+ &patchable_start,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_MCOUNT);
+ if (err)
+ return err;
+ err = read_ftrace_symbol_range(elf, &patchable_start,
+ &patchable_stop,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_PATCHABLE);
+ } else {
+ err = read_ftrace_symbol_range(elf, &start, &stop,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_UNKNOWN);
+ }
+ if (err)
+ return err;
+ }
+
+ return remove_unattachable_functions(elf, functions, shstrndx);
+}
+
+/*
+ * Find the function each record points to. In a module the pointer is a
+ * relocation, in a linked image it is the address. REFS maps each pointer
+ * sized slot of the section to a function.
+ */
+static int resolve_record_relocations(Elf *elf, const struct elf_function_list *functions,
+ size_t metadata_ndx, const GElf_Shdr *metadata_shdr,
+ const unsigned char *metadata, size_t *refs,
+ size_t ptr_size, unsigned int encoding)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ Elf_Data *data, *symbols;
+ Elf_Scn *symtab;
+ GElf_Shdr shdr;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if ((shdr.sh_type != SHT_RELA && shdr.sh_type != SHT_REL) ||
+ shdr.sh_info != metadata_ndx)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ symtab = elf_getscn(elf, shdr.sh_link);
+ symbols = symtab ? elf_getdata(symtab, NULL) : NULL;
+ if (!data || !symbols || elf_getdata(scn, data))
+ return -1;
+ for (i = 0; i < shdr.sh_size / shdr.sh_entsize; i++) {
+ GElf_Rela rela;
+ GElf_Sym sym;
+ size_t offset;
+
+ if (shdr.sh_type == SHT_RELA) {
+ if (!gelf_getrela(data, i, &rela))
+ return -1;
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ return -1;
+ rela = (GElf_Rela){ .r_offset = rel.r_offset,
+ .r_info = rel.r_info };
+ /* REL addends are stored in the pointer itself. */
+ if (!relocation_offset(metadata_shdr, rel.r_offset, ptr_size,
+ &offset))
+ return -EINVAL;
+ rela.r_addend = read_elf_value(metadata + offset, ptr_size,
+ encoding);
+ }
+ if (!relocation_offset(metadata_shdr, rela.r_offset, ptr_size, &offset) ||
+ offset % ptr_size)
+ return -EINVAL;
+ if (!gelf_getsym(symbols, GELF_R_SYM(rela.r_info), &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF || sym.st_shndx >= SHN_LORESERVE)
+ continue;
+ refs[offset / ptr_size] = function_at(functions, sym.st_shndx,
+ sym.st_value + rela.r_addend);
+ }
+ }
+ return 0;
+}
+
+static size_t record_function(Elf *elf, const struct elf_function_list *functions,
+ const unsigned char *buf, const size_t *refs,
+ size_t offset, size_t ptr_size, unsigned int encoding)
+{
+ Elf64_Section section;
+ GElf_Ehdr ehdr;
+ GElf_Addr value;
+
+ if (refs[offset / ptr_size] != NO_FUNCTION)
+ return refs[offset / ptr_size];
+ if (!gelf_getehdr(elf, &ehdr) || ehdr.e_type == ET_REL)
+ return NO_FUNCTION;
+ value = read_elf_value(buf + offset, ptr_size, encoding);
+ if (!value || section_for_address(elf, value, §ion))
+ return NO_FUNCTION;
+ return function_at(functions, section, value);
+}
+
+/* Remember the records of a body for all the functions at its address */
+static void add_record(struct elf_function_list *functions, size_t first,
+ size_t offset, bool weak)
+{
+ struct elf_function *f = &functions->functions[first];
+ size_t i;
+
+ for (i = first; i < functions->nr &&
+ !cmp_elf_function_location(f, &functions->functions[i]); i++) {
+ if (weak) {
+ functions->functions[i].nr_weak++;
+ functions->functions[i].weak = offset;
+ } else {
+ functions->functions[i].nr_strong++;
+ functions->functions[i].strong = offset;
+ }
+ }
+}
+
+static int parse_records(Elf *elf, struct elf_function_list *functions,
+ const unsigned char *buf, size_t size, const size_t *refs,
+ size_t ptr_size, unsigned int encoding,
+ struct fmodret_stats *stats)
+{
+ size_t offset = 0;
+
+ while (offset < size) {
+ unsigned int flags, length, nr_values;
+ size_t function;
+
+ if (size - offset < FMODRET_META_HEADER_SIZE(ptr_size) ||
+ read_elf_value(buf + offset, 4, encoding) != FMODRET_META_MAGIC ||
+ read_elf_value(buf + offset + 4, 4, encoding) != FMODRET_META_VERSION)
+ goto invalid;
+ length = read_elf_value(buf + offset + 8, 4, encoding);
+ flags = read_elf_value(buf + offset + 12, 4, encoding);
+ nr_values = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);
+ if (length < FMODRET_META_HEADER_SIZE(ptr_size) || length > size - offset ||
+ length % ptr_size ||
+ nr_values != (length - FMODRET_META_HEADER_SIZE(ptr_size)) /
+ FMODRET_META_VALUE_SIZE ||
+ (length - FMODRET_META_HEADER_SIZE(ptr_size)) % FMODRET_META_VALUE_SIZE)
+ goto invalid;
+ stats->records++;
+ function = record_function(elf, functions, buf, refs, offset + 16,
+ ptr_size, encoding);
+ /* A function in a discarded section has no ELF function */
+ if (function == NO_FUNCTION)
+ stats->unmapped++;
+ else
+ add_record(functions, function, offset, flags & FMODRET_FN_WEAK);
+ offset += length;
+ }
+ return 0;
+invalid:
+ fprintf(stderr,
+ "invalid or old fmodret metadata at offset %#zx; rebuild all plugin objects\n",
+ offset);
+ return -EINVAL;
+}
+
+/* Return the record which describes the body of F, or NO_FUNCTION */
+static size_t chosen_record(const struct elf_function *f)
+{
+ if (f->nr_strong == 1)
+ return f->strong;
+ if (!f->nr_strong && f->nr_weak == 1)
+ return f->weak;
+ return NO_FUNCTION;
+}
+
+struct btf_func {
+ const char *name;
+ __u32 id;
+};
+
+static int cmp_btf_func(const void *a, const void *b)
+{
+ const struct btf_func *x = a, *y = b;
+
+ return strcmp(x->name, y->name);
+}
+
+/*
+ * Return the FUNCs of BTF, without those of its base, sorted by name. The BTF
+ * is sorted only after the tags are added, and searching all of it for each
+ * candidate is slow for vmlinux.
+ */
+static struct btf_func *collect_btf_funcs(const struct btf *btf, size_t *cnt)
+{
+ const struct btf *base = btf__base_btf(btf);
+ __u32 id, nr_types = btf__type_cnt(btf);
+ struct btf_func *funcs;
+ size_t n = 0;
+
+ funcs = malloc(nr_types * sizeof(*funcs));
+ if (!funcs)
+ return NULL;
+ for (id = base ? btf__type_cnt(base) : 1; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_func(t))
+ continue;
+ funcs[n].name = btf__name_by_offset(btf, t->name_off);
+ funcs[n++].id = id;
+ }
+ qsort(funcs, n, sizeof(*funcs), cmp_btf_func);
+ *cnt = n;
+ return funcs;
+}
+
+/* Return the BTF ID of the only FUNC named NAME, or 0 */
+static int find_btf_func(const struct btf_func *funcs, size_t cnt, const char *name)
+{
+ const struct btf_func key = { .name = name }, *f;
+
+ f = bsearch(&key, funcs, cnt, sizeof(*funcs), cmp_btf_func);
+ if (!f)
+ return 0;
+ if ((f > funcs && !strcmp(f[-1].name, name)) ||
+ (f + 1 < funcs + cnt && !strcmp(f[1].name, name)))
+ return 0;
+ return f->id;
+}
+
+/* Return the size of the integer which FUNC returns, or 0 */
+static __u32 btf_func_int_return_size(const struct btf *btf, int func)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func);
+ int ret;
+
+ t = btf__type_by_id(btf, t->type);
+ if (!t || !btf_is_func_proto(t) || !t->type)
+ return 0;
+ ret = btf__resolve_type(btf, t->type);
+ if (ret < 0)
+ return 0;
+ t = btf__type_by_id(btf, ret);
+ if (!btf_is_int(t) && !btf_is_enum(t) && !btf_is_enum64(t))
+ return 0;
+ return t->size;
+}
+
+/* Format the values of the record at OFFSET as "-22,-12" */
+static char *format_values(const unsigned char *buf, size_t offset, size_t ptr_size,
+ unsigned int encoding)
+{
+ unsigned int nr = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);
+ const unsigned char *p = buf + offset + FMODRET_META_HEADER_SIZE(ptr_size);
+ size_t size = (size_t)nr * 21 + 1, len = 0;
+ char *s = malloc(size);
+ unsigned int i;
+
+ if (!s)
+ return NULL;
+ for (i = 0; i < nr; i++) {
+ int64_t value = read_elf_value(p + i * FMODRET_META_VALUE_SIZE,
+ FMODRET_META_VALUE_SIZE, encoding);
+
+ len += snprintf(s + len, size - len, "%s%lld", i ? "," : "",
+ (long long)value);
+ }
+ s[len] = 0;
+ return s;
+}
+
+static int cmp_retval(const void *va, const void *vb)
+{
+ const struct fmodret_retval *a = va;
+ const struct fmodret_retval *b = vb;
+
+ return a->btf_id > b->btf_id ? 1 : a->btf_id < b->btf_id ? -1 : 0;
+}
+
+/* Choose the targets among the functions and describe them in RESULT */
+static int choose_functions(struct elf_function_list *functions, const struct btf *btf,
+ const struct btf_func *btf_funcs, size_t nr_btf_funcs,
+ const unsigned char *buf, size_t ptr_size,
+ unsigned int encoding, struct fmodret_retvals *result,
+ struct fmodret_stats *stats, int verbose)
+{
+ size_t i = 0, j;
+
+ qsort(functions->functions, functions->nr, sizeof(*functions->functions),
+ cmp_elf_function_name);
+ for (i = 0; i < functions->nr; i = j) {
+ const struct elf_function *f = &functions->functions[i];
+ size_t record = chosen_record(f);
+ const char *reason = NULL;
+ unsigned int flags = 0;
+ struct fmodret_retval *r;
+ __u32 size;
+ int id = 0;
+
+ for (j = i + 1; j < functions->nr; j++)
+ if (strcmp(f->name, functions->functions[j].name))
+ break;
+
+ if (!f->nr_strong && !f->nr_weak)
+ continue;
+ if (record == NO_FUNCTION) {
+ stats->ambiguous_body++;
+ reason = "ambiguous-body";
+ goto rejected;
+ }
+ /*
+ * A strong definition without values has no record. If it
+ * replaced a weak one, only the record of the weak one points
+ * to its body.
+ */
+ if (!f->nr_strong && !f->weak_symbol) {
+ stats->overridden++;
+ reason = "overridden-weak";
+ goto rejected;
+ }
+ if (!read_elf_value(buf + record + 16 + ptr_size, 4, encoding))
+ continue;
+ stats->with_values++;
+ flags = read_elf_value(buf + record + 12, 4, encoding);
+ if (j - i != 1) {
+ stats->ambiguous_name++;
+ reason = "ambiguous-name";
+ goto rejected;
+ }
+ if (!f->traceable) {
+ stats->not_traceable++;
+ reason = "not-traceable";
+ goto rejected;
+ }
+ id = find_btf_func(btf_funcs, nr_btf_funcs, f->name);
+ if (id <= 0) {
+ stats->no_btf++;
+ reason = "no-unique-BTF-FUNC";
+ goto rejected;
+ }
+ size = btf_func_int_return_size(btf, id);
+ if (size != (flags & FMODRET_FN_RET64 ? 8 : 4)) {
+ stats->return_mismatch++;
+ reason = "BTF-return-type";
+ }
+rejected:
+ if (reason) {
+ if (verbose > 1)
+ fprintf(stderr, "fmodret: rejected %s: %s\n", f->name, reason);
+ continue;
+ }
+
+ if (result->cnt == result->cap) {
+ size_t cap = result->cap ? 2 * result->cap : 256;
+
+ r = realloc(result->retvals, cap * sizeof(*r));
+ if (!r)
+ return -ENOMEM;
+ result->retvals = r;
+ result->cap = cap;
+ }
+ r = &result->retvals[result->cnt];
+ r->btf_id = id;
+ r->values = format_values(buf, record, ptr_size, encoding);
+ if (!r->values)
+ return -ENOMEM;
+ result->cnt++;
+ stats->added++;
+ if (verbose > 1)
+ fprintf(stderr, "fmodret: %s: BTF ID %d, retval %s\n",
+ f->name, id, r->values);
+ }
+ if (result->cnt)
+ qsort(result->retvals, result->cnt, sizeof(*result->retvals), cmp_retval);
+ return 0;
+}
+
+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,
+ bool patchable_ftrace, int verbose)
+{
+ struct elf_function_list functions = {};
+ struct btf_func *btf_funcs = NULL;
+ struct fmodret_stats stats = {};
+ unsigned char *buf = NULL;
+ size_t nr_btf_funcs;
+ size_t *refs = NULL;
+ size_t shstrndx, ptr_size, i;
+ unsigned int encoding;
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ int err;
+
+ if (elf_getshdrstrndx(elf, &shstrndx))
+ return -EINVAL;
+ err = find_section(elf, shstrndx, FMODRET_CANDIDATES_SECTION, &scn);
+ if (err || !scn)
+ return err;
+ if (!gelf_getshdr(scn, &shdr))
+ return -EINVAL;
+ ptr_size = gelf_getclass(elf) == ELFCLASS64 ? 8 : 4;
+ encoding = elf_getident(elf, NULL)[EI_DATA];
+ if (shdr.sh_size % ptr_size)
+ return -EINVAL;
+
+ err = collect_elf_functions(elf, &functions);
+ if (err)
+ goto out;
+ qsort(functions.functions, functions.nr, sizeof(*functions.functions),
+ cmp_elf_function_location);
+
+ err = copy_section_data(scn, &shdr, &buf);
+ if (err)
+ goto out;
+ refs = malloc((shdr.sh_size / ptr_size ?: 1) * sizeof(*refs));
+ if (!refs) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < shdr.sh_size / ptr_size; i++)
+ refs[i] = NO_FUNCTION;
+ err = resolve_record_relocations(elf, &functions, elf_ndxscn(scn), &shdr, buf,
+ refs, ptr_size, encoding);
+ if (err)
+ goto out;
+ err = parse_records(elf, &functions, buf, shdr.sh_size, refs, ptr_size,
+ encoding, &stats);
+ if (err)
+ goto out;
+ err = read_ftrace_sites(elf, shstrndx, ptr_size, encoding, &functions,
+ patchable_ftrace);
+ if (err)
+ goto out;
+ btf_funcs = collect_btf_funcs(btf, &nr_btf_funcs);
+ if (!btf_funcs) {
+ err = -ENOMEM;
+ goto out;
+ }
+ err = choose_functions(&functions, btf, btf_funcs, nr_btf_funcs, buf, ptr_size,
+ encoding, result, &stats, verbose);
+ if (err)
+ goto out;
+
+ if (verbose)
+ fprintf(stderr,
+ "fmodret: %zu records (%zu without a function), %zu functions with values, %zu added; rejected: %zu ambiguous body, %zu ambiguous name, %zu not traceable, %zu overridden weak, %zu without unique BTF FUNC, %zu BTF return type\n",
+ stats.records, stats.unmapped, stats.with_values, stats.added,
+ stats.ambiguous_body,
+ stats.ambiguous_name, stats.not_traceable, stats.overridden,
+ stats.no_btf, stats.return_mismatch);
+out:
+ if (err == -1)
+ fprintf(stderr, "failed to read %s: %s\n", FMODRET_CANDIDATES_SECTION,
+ elf_errmsg(-1));
+ free(btf_funcs);
+ free(refs);
+ free(buf);
+ free(functions.functions);
+ return err;
+}
+
+void fmodret_free(struct fmodret_retvals *retvals)
+{
+ size_t i;
+
+ for (i = 0; i < retvals->cnt; i++)
+ free(retvals->retvals[i].values);
+ free(retvals->retvals);
+ retvals->retvals = NULL;
+ retvals->cnt = retvals->cap = 0;
+}
diff --git a/tools/bpf/resolve_btfids/fmodret.h b/tools/bpf/resolve_btfids/fmodret.h
new file mode 100644
index 00000000000000..c1d5db64970b62
--- /dev/null
+++ b/tools/bpf/resolve_btfids/fmodret.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __RESOLVE_BTFIDS_FMODRET_H
+#define __RESOLVE_BTFIDS_FMODRET_H
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <libelf.h>
+#include <linux/types.h>
+
+struct btf;
+
+/* A function found by the analysis, and the values it may return */
+struct fmodret_retval {
+ __u32 btf_id;
+ char *values;
+};
+
+/* Sorted by BTF ID */
+struct fmodret_retvals {
+ struct fmodret_retval *retvals;
+ size_t cnt, cap;
+};
+
+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,
+ bool patchable_ftrace, int verbose);
+void fmodret_free(struct fmodret_retvals *retvals);
+
+#endif
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index c16dd93fa6d4c2..124c4f01c8878f 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -69,6 +69,18 @@
* - rewrites the prototype of KF_IMPLICIT_ARGS kfuncs.
*
* These kfunc annotations were historically produced by pahole.
+ *
+ * Functions found in BTF_SET8_FMODRET sets are the targets of BPF fmod_ret
+ * programs. For each of them resolve_btfids emits a "bpf_fmodret" decl tag,
+ * and also "bpf_fmodret_sleepable" when KF_SLEEPABLE is set, so the sleepable
+ * tag never comes alone. The kernel builds the set of fmod_ret targets from
+ * these tags when it loads BTF.
+ *
+ * Functions which the analysis of the compiler plugin records, see fmodret.c,
+ * get "bpf_fmodret" too, and "bpf_fmodret_retval:" with the values they
+ * return on paths without side effects, such as "bpf_fmodret_retval:-22,-12".
+ * The verifier lets programs attached to them return only these values or 0.
+ * A function in a BTF_FMODRET set keeps only its own tags.
*/
#define _GNU_SOURCE
@@ -90,6 +102,7 @@
#include <bpf/btf.h>
#include <bpf/libbpf.h>
#include "btf_colors.h"
+#include "fmodret.h"
#include <subcmd/parse-options.h>
#define BTF_IDS_SECTION ".BTF_ids"
@@ -148,6 +161,11 @@ struct object {
struct btf *orig_btf;
bool distill_base;
bool extract_inline;
+ /* The ftrace call sites are in __patchable_function_entries */
+ bool patchable_ftrace;
+
+ /* Functions found by the analysis of the compiler plugin, see fmodret.c */
+ struct fmodret_retvals fmodret_retvals;
struct {
int fd;
@@ -197,7 +215,11 @@ struct btf_links {
#define DECL_TAG_FASTCALL "bpf_fastcall"
#define DECL_TAG_KFUNC "bpf_kfunc"
+#define DECL_TAG_FMODRET "bpf_fmodret"
+#define DECL_TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+#define DECL_TAG_FMODRET_RETVAL "bpf_fmodret_retval:"
+#define KF_SLEEPABLE (1 << 5)
#define KF_FASTCALL (1 << 12)
#define KF_ARENA_RET (1 << 13)
#define KF_ARENA_ARG1 (1 << 14)
@@ -208,6 +230,7 @@ struct btf_links {
#define PARAM_SUFFIX_ARENA "__arena"
#define PARAM_SUFFIX_ARENA_NULLABLE "__arena__nullable"
+/* A function found in a BTF ID set: a kfunc or an fmod_ret target */
struct kfunc {
struct rb_node rb_node;
const char *name;
@@ -221,6 +244,7 @@ struct btf2btf_context {
u32 nr_decl_tags;
u32 max_decl_tags;
struct rb_root kfuncs;
+ struct rb_root fmodrets;
};
static int verbose;
@@ -1256,16 +1280,16 @@ static int push_decl_tag_id(struct btf2btf_context *ctx, u32 decl_tag_id)
return 0;
}
-static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)
+static int push_set_func(struct rb_root *root, const char *kind, struct kfunc *kfunc)
{
- struct rb_node **p = &ctx->kfuncs.rb_node;
+ struct rb_node **p = &root->rb_node;
struct rb_node *parent = NULL;
struct kfunc *k;
/*
- * Dedup by BTF ID: collecting the same kfunc twice is a no-op,
- * UNLESS the kfunc flags are inconsistent, in which case we
- * fail hard because it indicates a bug in a kfunc set declaration.
+ * Dedup by BTF ID: collecting the same function twice is a no-op,
+ * UNLESS its flags are inconsistent, in which case we fail hard
+ * because it indicates a bug in a BTF ID set declaration.
*/
while (*p) {
parent = *p;
@@ -1278,8 +1302,8 @@ static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)
} else if (k->flags == kfunc->flags) {
return 0;
} else {
- pr_err("ERROR: resolve_btfids: kfunc %s has inconsistent flags across BTF ID sets: 0x%x != 0x%x\n",
- kfunc->name, k->flags, kfunc->flags);
+ pr_err("ERROR: resolve_btfids: %s %s has inconsistent flags across BTF ID sets: 0x%x != 0x%x\n",
+ kind, kfunc->name, k->flags, kfunc->flags);
return -EINVAL;
}
}
@@ -1290,7 +1314,7 @@ static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)
*k = *kfunc;
rb_link_node(&k->rb_node, parent, p);
- rb_insert_color(&k->rb_node, &ctx->kfuncs);
+ rb_insert_color(&k->rb_node, root);
return 0;
}
@@ -1344,7 +1368,16 @@ static bool is_arena_param(const struct btf *btf, const struct btf_param *param)
param_name_has_suffix(name, PARAM_SUFFIX_ARENA_NULLABLE);
}
-static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
+typedef int (*set_func_fn)(struct btf2btf_context *ctx, const char *name,
+ u32 btf_id, u32 flags);
+
+/*
+ * Call FN for each function in the BTF ID sets which have SET_FLAG set. Reject
+ * an entry with flags other than ALLOWED_FLAGS, also if its function is missing.
+ */
+static int collect_set_funcs(struct object *obj, struct btf2btf_context *ctx,
+ u32 set_flag, u32 allowed_flags, const char *kind,
+ set_func_fn fn)
{
Elf_Data *idlist = obj->efile.idlist;
struct btf *btf = ctx->btf;
@@ -1356,24 +1389,39 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
for (next = rb_first(&obj->sets); next; next = rb_next(next)) {
struct btf_id_set8 *set8;
struct btf_id *set_id;
- u64 set_addr;
+ u64 set_addr, off;
set_id = rb_entry(next, struct btf_id, rb_node);
if (set_id->kind != BTF_ID_KIND_SET8 || set_id->addr_cnt != 1)
continue;
set_addr = set_id->addr[0];
- set8 = idlist->d_buf + (set_addr - obj->efile.idlist_addr);
- if (!(set8->flags & BTF_SET8_KFUNCS))
+ off = set_addr - obj->efile.idlist_addr;
+ /* A malformed set must not reach out of the section */
+ if (set_addr < obj->efile.idlist_addr || off > idlist->d_size ||
+ idlist->d_size - off < sizeof(*set8) || set_id->cnt < 0 ||
+ (u64)set_id->cnt > (idlist->d_size - off - sizeof(*set8)) /
+ sizeof(set8->pairs[0])) {
+ pr_err("ERROR: resolve_btfids: set %s is out of %s\n",
+ set_id->name, BTF_IDS_SECTION);
+ return -EINVAL;
+ }
+ set8 = idlist->d_buf + off;
+ if (!(set8->flags & set_flag))
continue;
for (u32 i = 0; i < set_id->cnt; i++) {
size_t off = (char *)&set8->pairs[i] - (char *)set8;
const char *name = find_name_by_addr(obj, set_addr + off);
- struct kfunc kfunc;
+ u32 flags = set8->pairs[i].flags;
s32 func_id;
int err;
+ if (flags & ~allowed_flags) {
+ pr_err("ERROR: resolve_btfids: %s %s in %s has unsupported flags 0x%x\n",
+ kind, name ?: "?", set_id->name, flags & ~allowed_flags);
+ return -EINVAL;
+ }
if (!name) {
pr_err("WARN: resolve_btfids: no BTF ID symbol for %s entry %u\n",
set_id->name, i);
@@ -1383,18 +1431,15 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
func_id = btf__find_by_name_kind_own(btf, name, BTF_KIND_FUNC);
if (func_id < 0) {
- pr_err("WARN: resolve_btfids: no BTF func for kfunc %s in %s\n",
- name, set_id->name);
+ pr_err("WARN: resolve_btfids: no BTF func for %s %s in %s\n",
+ kind, name, set_id->name);
warnings++;
continue;
}
- pr_debug("found kfunc %s in %s\n", name, set_id->name);
+ pr_debug("found %s %s in %s\n", kind, name, set_id->name);
- kfunc.name = name;
- kfunc.btf_id = func_id;
- kfunc.flags = set8->pairs[i].flags;
- err = push_kfunc(ctx, &kfunc);
+ err = fn(ctx, name, func_id, flags);
if (err)
return err;
}
@@ -1403,6 +1448,33 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
return 0;
}
+static int push_kfunc(struct btf2btf_context *ctx, const char *name,
+ u32 btf_id, u32 flags)
+{
+ struct kfunc kfunc = { .name = name, .btf_id = btf_id, .flags = flags };
+
+ return push_set_func(&ctx->kfuncs, "kfunc", &kfunc);
+}
+
+static int push_fmodret(struct btf2btf_context *ctx, const char *name,
+ u32 btf_id, u32 flags)
+{
+ struct kfunc fmodret = { .name = name, .btf_id = btf_id, .flags = flags };
+
+ return push_set_func(&ctx->fmodrets, "fmod_ret target", &fmodret);
+}
+
+static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
+{
+ return collect_set_funcs(obj, ctx, BTF_SET8_KFUNCS, ~0U, "kfunc", push_kfunc);
+}
+
+static int collect_fmodrets(struct object *obj, struct btf2btf_context *ctx)
+{
+ return collect_set_funcs(obj, ctx, BTF_SET8_FMODRET, KF_SLEEPABLE,
+ "fmod_ret target", push_fmodret);
+}
+
static int build_btf2btf_context(struct object *obj, struct btf2btf_context *ctx)
{
int err;
@@ -1421,6 +1493,12 @@ static int build_btf2btf_context(struct object *obj, struct btf2btf_context *ctx
return err;
}
+ err = collect_fmodrets(obj, ctx);
+ if (err) {
+ pr_err("ERROR: resolve_btfids: failed to collect fmod_ret targets from BTF\n");
+ return err;
+ }
+
return 0;
}
@@ -1727,6 +1805,40 @@ static int add_decl_tag(struct btf2btf_context *ctx, const char *tag_name,
return push_decl_tag_id(ctx, new_id);
}
+static bool has_set_func(const struct rb_root *root, u32 btf_id)
+{
+ const struct rb_node *node = root->rb_node;
+
+ while (node) {
+ const struct kfunc *k = rb_entry(node, struct kfunc, rb_node);
+
+ if (btf_id < k->btf_id)
+ node = node->rb_left;
+ else if (btf_id > k->btf_id)
+ node = node->rb_right;
+ else
+ return true;
+ }
+ return false;
+}
+
+static int add_fmodret_retval_tags(struct btf2btf_context *ctx,
+ const struct fmodret_retval *retval)
+{
+ char *tag;
+ int err;
+
+ err = add_decl_tag(ctx, DECL_TAG_FMODRET, retval->btf_id, -1);
+ if (err)
+ return err;
+
+ if (asprintf(&tag, "%s%s", DECL_TAG_FMODRET_RETVAL, retval->values) < 0)
+ return -ENOMEM;
+ err = add_decl_tag(ctx, tag, retval->btf_id, -1);
+ free(tag);
+ return err;
+}
+
static int btf2btf(struct object *obj)
{
struct btf2btf_context ctx = {};
@@ -1761,10 +1873,36 @@ static int btf2btf(struct object *obj)
goto out;
}
+ for (next = rb_first(&ctx.fmodrets); next; next = rb_next(next)) {
+ struct kfunc *fmodret = rb_entry(next, struct kfunc, rb_node);
+
+ err = add_decl_tag(&ctx, DECL_TAG_FMODRET, fmodret->btf_id, -1);
+ if (err)
+ goto out;
+
+ if (fmodret->flags & KF_SLEEPABLE) {
+ err = add_decl_tag(&ctx, DECL_TAG_FMODRET_SLEEPABLE,
+ fmodret->btf_id, -1);
+ if (err)
+ goto out;
+ }
+ }
+
+ for (size_t i = 0; i < obj->fmodret_retvals.cnt; i++) {
+ const struct fmodret_retval *retval = &obj->fmodret_retvals.retvals[i];
+
+ if (has_set_func(&ctx.fmodrets, retval->btf_id))
+ continue;
+ err = add_fmodret_retval_tags(&ctx, retval);
+ if (err)
+ goto out;
+ }
+
err = 0;
out:
free(ctx.decl_tags);
free_kfuncs(&ctx.kfuncs);
+ free_kfuncs(&ctx.fmodrets);
return err;
}
@@ -2113,6 +2251,8 @@ int main(int argc, const char **argv)
"distill --btf_base and emit .BTF.base section data"),
OPT_BOOLEAN(0, "inline", &obj.extract_inline,
"extract location BTF into a .BTF.inline file"),
+ OPT_BOOLEAN(0, "patchable_ftrace", &obj.patchable_ftrace,
+ "ftrace call sites are in __patchable_function_entries"),
OPT_CALLBACK(0, "btf_link", &btf_links, "section:module:btf-file",
"patch a BTF link with --patch_btfids", parse_btf_link),
OPT_STRING(0, "patch_btfids", &btfids_path, "file",
@@ -2171,6 +2311,10 @@ int main(int argc, const char **argv)
if (load_btf(&obj))
goto out;
+ if (fmodret_collect(obj.efile.elf, obj.btf, &obj.fmodret_retvals,
+ obj.patchable_ftrace, verbose))
+ goto out;
+
if (btf2btf(&obj))
goto out;
@@ -2228,6 +2372,7 @@ int main(int argc, const char **argv)
btf_id__free_all(&obj.funcs);
btf_id__free_all(&obj.sets);
free(obj.addr_syms);
+ fmodret_free(&obj.fmodret_retvals);
free(obj.efile.func_symbols);
if (obj.efile.elf) {
elf_end(obj.efile.elf);
diff --git a/tools/include/linux/btf_ids.h b/tools/include/linux/btf_ids.h
index 4fe5c5f1558ca9..3b5b2856c6285f 100644
--- a/tools/include/linux/btf_ids.h
+++ b/tools/include/linux/btf_ids.h
@@ -13,13 +13,21 @@ struct btf_id_set {
/* This flag implies BTF_SET8 holds kfunc(s) */
#define BTF_SET8_KFUNCS (1 << 0)
+/*
+ * This flag implies BTF_SET8 holds functions which BPF fmod_ret, fentry and
+ * fexit programs may attach to. Per-function flags: KF_SLEEPABLE only.
+ */
+#define BTF_SET8_FMODRET (1 << 1)
+
+struct btf_id_set8_pair {
+ u32 id;
+ u32 flags;
+};
+
struct btf_id_set8 {
u32 cnt;
u32 flags;
- struct {
- u32 id;
- u32 flags;
- } pairs[];
+ struct btf_id_set8_pair pairs[];
};
#ifdef CONFIG_DEBUG_INFO_BTF
@@ -214,6 +222,12 @@ __BTF_SET8_START(name, local, BTF_SET8_KFUNCS)
#define BTF_KFUNCS_END(name) \
BTF_SET8_END(name)
+#define BTF_FMODRET_START(name) \
+__BTF_SET8_START(name, local, BTF_SET8_FMODRET)
+
+#define BTF_FMODRET_END(name) \
+BTF_SET8_END(name)
+
#else
#define BTF_ID_LIST(name) static u32 __maybe_unused name[128];
@@ -230,6 +244,8 @@ BTF_SET8_END(name)
#define BTF_SET8_END(name)
#define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };
#define BTF_KFUNCS_END(name)
+#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };
+#define BTF_FMODRET_END(name)
#endif /* CONFIG_DEBUG_INFO_BTF */
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 931c5c412f15d8..e6120bee45052a 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -4657,6 +4657,7 @@ static int validate_ibt(struct objtool_file *file)
!strcmp(sec->name, "__jump_table") ||
!strcmp(sec->name, ".init.klp_funcs") ||
!strcmp(sec->name, "__mcount_loc") ||
+ !strcmp(sec->name, ".BTF_fmodret_candidates") ||
!strcmp(sec->name, ".llvm.call-graph-profile") ||
!strcmp(sec->name, ".llvm_bb_addr_map") ||
!strcmp(sec->name, "__tracepoints") ||
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 77682bf28c6ece..84554a2adfc1bc 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -47,7 +47,7 @@ TEST_PROGS_EXTENDED := \
ima_setup.sh verify_sig_setup.sh
TEST_KMODS := bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
- bpf_test_modorder_y.ko bpf_test_rqspinlock.ko
+ bpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_retval.ko
TEST_KMOD_TARGETS = $(addprefix $(OUTPUT)/,$(TEST_KMODS))
# Compile but not part of 'make run_tests'
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index 49dfe939467cc8..5461c32cd3cd2c 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -11,6 +11,7 @@ CONFIG_BPF_STREAM_PARSER=y
CONFIG_BPF_SYSCALL=y
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
CONFIG_CGROUP_BPF=y
+CONFIG_CLANG_PLUGIN_BPF_SCAN_IR=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_MLDSA=y
CONFIG_CRYPTO_SHA256=y
@@ -56,6 +57,7 @@ CONFIG_IPV6_TUNNEL=y
CONFIG_KEYS=y
CONFIG_LIRC=y
CONFIG_LIVEPATCH=y
+CONFIG_LLVM_PLUGINS=y
CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_MEMCG=y
diff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
new file mode 100644
index 00000000000000..7b374fe6813027
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include "fmodret_retval.skel.h"
+#include "fmodret_retval_ext.skel.h"
+
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_RETVAL "bpf_fmodret_retval:"
+#define TAG_TEST_RETVAL "test_retval:"
+
+#define RETVAL_MODULE "bpf_test_retval"
+
+/* The decl tags of a function */
+struct func_tags {
+ const char *test_retval;
+ const char *retval;
+ bool target;
+};
+
+static bool tag_value(const char *name, const char *key, const char **value)
+{
+ size_t len = strlen(key);
+
+ if (strncmp(name, key, len))
+ return false;
+ *value = name + len;
+ return true;
+}
+
+/*
+ * Return the decl tags of the functions of module BTF, indexed by BTF ID. Only
+ * the types of the module itself are read, not those of its vmlinux base.
+ */
+static struct func_tags *collect_tags(const struct btf *btf, int start_id)
+{
+ int id, nr_types = btf__type_cnt(btf);
+ struct func_tags *tags;
+
+ tags = calloc(nr_types, sizeof(*tags));
+ if (!tags)
+ return NULL;
+ for (id = start_id; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ struct func_tags *f;
+ const char *name;
+
+ if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1 ||
+ t->type >= nr_types)
+ continue;
+ f = &tags[t->type];
+ name = btf__name_by_offset(btf, t->name_off);
+ if (!tag_value(name, TAG_TEST_RETVAL, &f->test_retval) &&
+ !tag_value(name, TAG_RETVAL, &f->retval) && !strcmp(name, TAG_FMODRET))
+ f->target = true;
+ }
+
+ return tags;
+}
+
+/* Compare the tags of every fixture with the values it expects */
+static void test_tags(const struct btf *btf, const struct func_tags *tags, int start_id)
+{
+ int id, nr_types = btf__type_cnt(btf), nr_fixtures = 0;
+
+ for (id = start_id; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ const char *name;
+
+ if (!btf_is_func(t) || !tags[id].test_retval)
+ continue;
+ name = btf__name_by_offset(btf, t->name_off);
+ ASSERT_STREQ(tags[id].retval ?: "", tags[id].test_retval, name);
+ if (*tags[id].test_retval)
+ ASSERT_TRUE(tags[id].target, name);
+ else if (strcmp(name, "bpf_testmod_retval_declared"))
+ ASSERT_FALSE(tags[id].target, name);
+ nr_fixtures++;
+ }
+ ASSERT_GT(nr_fixtures, 0, "fixtures");
+
+ id = btf__find_by_name_kind(btf, "bpf_testmod_retval_declared", BTF_KIND_FUNC);
+ if (ASSERT_GE(id, start_id, "declared"))
+ ASSERT_TRUE(tags[id].target, "declared_target");
+}
+
+/* Open the programs again with only the extension, to replace FUNC of TARGET */
+static struct fmodret_retval_ext *open_extension(struct bpf_program *target,
+ const char *func, char *log, size_t log_size)
+{
+ struct fmodret_retval_ext *skel;
+
+ skel = fmodret_retval_ext__open();
+ if (!ASSERT_OK_PTR(skel, "fmodret_retval_ext__open"))
+ return NULL;
+ bpf_program__set_autoload(skel->progs.restricted, false);
+ bpf_program__set_autoload(skel->progs.unrestricted, false);
+ if (!ASSERT_OK(bpf_program__set_attach_target(skel->progs.extension,
+ bpf_program__fd(target), func),
+ "set_attach_target")) {
+ fmodret_retval_ext__destroy(skel);
+ return NULL;
+ }
+ bpf_program__set_log_buf(skel->progs.extension, log, log_size);
+ return skel;
+}
+
+/*
+ * An extension could return any value for an fmod_ret program, so it must not
+ * replace one whose return values are restricted, neither at load nor when it
+ * is attached later to another program.
+ */
+static void test_extension(void)
+{
+ struct fmodret_retval_ext *targets, *skel = NULL;
+ struct bpf_link *link;
+ char log[4096] = {};
+
+ if (!is_jit_enabled()) {
+ printf("%s:SKIP:extensions need the JIT\n", __func__);
+ test__skip();
+ return;
+ }
+
+ targets = fmodret_retval_ext__open();
+ if (!ASSERT_OK_PTR(targets, "fmodret_retval_ext__open"))
+ return;
+ bpf_program__set_autoload(targets->progs.extension, false);
+ if (!ASSERT_OK(fmodret_retval_ext__load(targets), "load_targets"))
+ goto out;
+
+ skel = open_extension(targets->progs.restricted, "restricted_value", log, sizeof(log));
+ if (!skel)
+ goto out;
+ ASSERT_ERR(fmodret_retval_ext__load(skel), "load_restricted");
+ ASSERT_HAS_SUBSTR(log, "Cannot extend an fmod_ret program with restricted return values",
+ "log_restricted");
+ fmodret_retval_ext__destroy(skel);
+
+ skel = open_extension(targets->progs.unrestricted, "unrestricted_value", log, sizeof(log));
+ if (!skel)
+ goto out;
+ if (!ASSERT_OK(fmodret_retval_ext__load(skel), "load_unrestricted"))
+ goto out;
+ link = bpf_program__attach_freplace(skel->progs.extension,
+ bpf_program__fd(targets->progs.restricted),
+ "restricted_value");
+ if (!ASSERT_ERR_PTR(link, "attach_restricted"))
+ bpf_link__destroy(link);
+ link = bpf_program__attach_freplace(skel->progs.extension,
+ bpf_program__fd(targets->progs.unrestricted),
+ "unrestricted_value");
+ if (ASSERT_OK_PTR(link, "attach_unrestricted"))
+ bpf_link__destroy(link);
+out:
+ fmodret_retval_ext__destroy(skel);
+ fmodret_retval_ext__destroy(targets);
+}
+
+void test_fmodret_retval(void)
+{
+ struct btf *vmlinux_btf, *btf = NULL;
+ struct func_tags *tags = NULL;
+ int id, start_id, nr_types;
+ bool has_fixtures = false;
+
+ if (!env.has_testmod) {
+ test__skip();
+ return;
+ }
+
+ if (!ASSERT_OK(load_module(RETVAL_MODULE ".ko", env_verbosity > VERBOSE_NONE),
+ "load_module"))
+ return;
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "btf__load_vmlinux_btf"))
+ goto out;
+ btf = btf__load_module_btf(RETVAL_MODULE, vmlinux_btf);
+ if (!btf && errno == ENOENT) {
+ printf("%s:SKIP:%s has no BTF\n", __func__, RETVAL_MODULE);
+ test__skip();
+ goto out;
+ }
+ if (!ASSERT_OK_PTR(btf, "btf__load_module_btf"))
+ goto out;
+ start_id = btf__type_cnt(vmlinux_btf);
+ nr_types = btf__type_cnt(btf);
+ tags = collect_tags(btf, start_id);
+ if (!ASSERT_OK_PTR(tags, "collect_tags"))
+ goto out;
+
+ /* The fixtures are built only with the compiler plugin */
+ for (id = start_id; id < nr_types && !has_fixtures; id++)
+ has_fixtures = tags[id].test_retval;
+ if (!has_fixtures) {
+ printf("%s:SKIP:%s has no %s tags\n", __func__, RETVAL_MODULE, TAG_TEST_RETVAL);
+ test__skip();
+ goto out;
+ }
+
+ if (test__start_subtest("tags"))
+ test_tags(btf, tags, start_id);
+ if (test__start_subtest("extension"))
+ test_extension();
+ RUN_TESTS(fmodret_retval);
+out:
+ free(tags);
+ btf__free(btf);
+ btf__free(vmlinux_btf);
+ unload_module(RETVAL_MODULE, env_verbosity > VERBOSE_NONE);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
new file mode 100644
index 00000000000000..62efe11af826fb
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <linux/filter.h>
+#include <bpf/btf.h>
+
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+
+static bool func_has_tag(const struct btf *btf, const char *func, const char *tag)
+{
+ int func_id, id, nr_types;
+
+ func_id = btf__find_by_name_kind(btf, func, BTF_KIND_FUNC);
+ if (func_id < 0)
+ return false;
+
+ nr_types = btf__type_cnt(btf);
+ for (id = 1; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_decl_tag(t) || t->type != func_id ||
+ btf_decl_tag(t)->component_idx != -1)
+ continue;
+ if (!strcmp(btf__name_by_offset(btf, t->name_off), tag))
+ return true;
+ }
+
+ return false;
+}
+
+static void test_vmlinux_tags(void)
+{
+ struct btf *btf;
+
+ btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(btf, "btf__load_vmlinux_btf"))
+ return;
+
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test", TAG_FMODRET), "modify_return_test");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test2", TAG_FMODRET),
+ "modify_return_test2");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test_tp", TAG_FMODRET),
+ "modify_return_test_tp");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_fentry_test1", TAG_FMODRET), "fentry_test1");
+ ASSERT_TRUE(func_has_tag(btf, "update_socket_protocol", TAG_FMODRET),
+ "update_socket_protocol");
+
+ /* Only bpf_fentry_test1 is a sleepable target */
+ ASSERT_TRUE(func_has_tag(btf, "bpf_fentry_test1", TAG_FMODRET_SLEEPABLE),
+ "fentry_test1_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test2", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test2_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test_tp", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test_tp_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "update_socket_protocol", TAG_FMODRET_SLEEPABLE),
+ "update_socket_protocol_sleepable");
+
+ /* A function which is not a target has no tag */
+ ASSERT_GT(btf__find_by_name_kind(btf, "bpf_fentry_test2", BTF_KIND_FUNC), 0,
+ "fentry_test2_exists");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_fentry_test2", TAG_FMODRET), "fentry_test2");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_fentry_test2", TAG_FMODRET_SLEEPABLE),
+ "fentry_test2_sleepable");
+
+ btf__free(btf);
+}
+
+static void test_module_tags(void)
+{
+ struct btf *vmlinux_btf, *module_btf;
+
+ if (!env.has_testmod) {
+ test__skip();
+ return;
+ }
+
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "btf__load_vmlinux_btf"))
+ return;
+ module_btf = btf__load_module_btf("bpf_testmod", vmlinux_btf);
+ if (!module_btf && errno == ENOENT) {
+ printf("%s:SKIP:bpf_testmod has no BTF\n", __func__);
+ test__skip();
+ goto out;
+ }
+ if (!ASSERT_OK_PTR(module_btf, "btf__load_module_btf"))
+ goto out;
+
+ ASSERT_TRUE(func_has_tag(module_btf, "bpf_testmod_trampoline_count_test", TAG_FMODRET),
+ "trampoline_count_test");
+ ASSERT_FALSE(func_has_tag(module_btf, "bpf_testmod_trampoline_count_test",
+ TAG_FMODRET_SLEEPABLE), "trampoline_count_test_sleepable");
+ ASSERT_FALSE(func_has_tag(module_btf, "bpf_testmod_fentry_test1", TAG_FMODRET),
+ "testmod_fentry_test1");
+
+ btf__free(module_btf);
+out:
+ btf__free(vmlinux_btf);
+}
+
+/* Load a minimal fmod_ret program which attaches to function BTF_ID */
+static int load_fmod_ret(int btf_id, __u32 prog_flags)
+{
+ struct bpf_insn insns[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .expected_attach_type = BPF_MODIFY_RETURN,
+ .prog_flags = prog_flags,
+ .attach_btf_id = btf_id,
+ );
+
+ return bpf_prog_load(BPF_PROG_TYPE_TRACING, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+}
+
+static void check_load(const char *func, __u32 prog_flags, bool allowed, const char *name)
+{
+ int btf_id, fd;
+
+ btf_id = libbpf_find_vmlinux_btf_id(func, BPF_MODIFY_RETURN);
+ if (!ASSERT_GT(btf_id, 0, func))
+ return;
+ fd = load_fmod_ret(btf_id, prog_flags);
+ if (allowed)
+ ASSERT_GE(fd, 0, name);
+ else
+ ASSERT_LT(fd, 0, name);
+ if (fd >= 0)
+ close(fd);
+}
+
+static void test_attach_permission(void)
+{
+ check_load("bpf_modify_return_test", 0, true, "target");
+ check_load("bpf_fentry_test1", 0, true, "sleepable_target");
+ check_load("bpf_fentry_test2", 0, false, "not_a_target");
+
+ check_load("bpf_fentry_test1", BPF_F_SLEEPABLE, true, "sleepable_prog_sleepable_target");
+ check_load("bpf_modify_return_test", BPF_F_SLEEPABLE, false,
+ "sleepable_prog_nonsleepable_target");
+}
+
+void test_fmodret_tags(void)
+{
+ if (test__start_subtest("vmlinux"))
+ test_vmlinux_tags();
+ if (test__start_subtest("module"))
+ test_module_tags();
+ if (test__start_subtest("attach_permission"))
+ test_attach_permission();
+}
diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval.c b/tools/testing/selftests/bpf/progs/fmodret_retval.c
new file mode 100644
index 00000000000000..bf5c6f53a99936
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/fmodret_retval.c
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * fmod_ret programs on the targets which the LLVM IR scanner finds in
+ * bpf_test_retval, see test_kmods/bpf_testmod_retval.c. They may return only
+ * 0 or the values of the "bpf_fmodret_retval:" tag of their target.
+ */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+/* bpf_testmod_retval_constant() returns -EINVAL */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+int BPF_PROG(constant_einval, int ret)
+{
+ return -22;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+int BPF_PROG(constant_zero, int ret)
+{
+ return 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+int BPF_PROG(constant_other, int ret)
+{
+ return -1;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+__naked void constant_unknown(void)
+{
+ asm volatile (
+ "call %[bpf_get_prandom_u32];"
+ "exit;"
+ :
+ : __imm(bpf_get_prandom_u32)
+ : __clobber_all);
+}
+
+/* The function returns int, so only the low 32 bits count */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+__naked void constant_32bit(void)
+{
+ asm volatile (
+ "w0 = -22;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* Not 0, so the function is not called, but the caller would see 0 */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+__naked void constant_high_bits(void)
+{
+ asm volatile (
+ "r0 = 1;"
+ "r0 <<= 32;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* bpf_testmod_retval_join() returns -EINVAL or -ERANGE */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+int BPF_PROG(join_einval, int *result, int value, int ret)
+{
+ return -22;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+int BPF_PROG(join_erange, int *result, int value, int ret)
+{
+ return -34;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_join")
+__failure __msg("it should be 0 or one of -34,-22")
+int BPF_PROG(join_other, int *result, int value, int ret)
+{
+ return -1;
+}
+
+/* Each path returns an allowed value */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+__naked void join_both_paths(void)
+{
+ asm volatile (
+ "r2 = *(u64 *)(r1 + 8);"
+ "if r2 s< 0 goto 1f;"
+ "r0 = -34;"
+ "exit;"
+"1:"
+ "r0 = -22;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/*
+ * One of two paths to the same exit returns a forbidden value. The verifier
+ * checks the allowed one first and must not prune the other.
+ */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__failure __msg("it should be 0 or one of -34,-22")
+__naked void join_one_forbidden(void)
+{
+ asm volatile (
+ "r2 = *(u64 *)(r1 + 8);"
+ "if r2 s< 0 goto 1f;"
+ "r0 = -34;"
+ "goto 2f;"
+"1:"
+ "r0 = -1;"
+"2:"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* bpf_testmod_retval_wide() returns s64, so all 64 bits count */
+SEC("fmod_ret/bpf_testmod_retval_wide")
+__success
+__load_if_JITed()
+__naked void wide_64bit(void)
+{
+ asm volatile (
+ "r0 = -14;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+SEC("fmod_ret/bpf_testmod_retval_wide")
+__failure __msg("it should be 0 or one of -14")
+__naked void wide_32bit(void)
+{
+ asm volatile (
+ "w0 = -14;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* Declared targets are not restricted, in module and vmlinux BTF */
+SEC("fmod_ret/bpf_testmod_retval_declared")
+__success
+int BPF_PROG(declared_module_any, int ret)
+{
+ return -1;
+}
+
+SEC("fmod_ret/bpf_modify_return_test")
+__success
+int BPF_PROG(declared_vmlinux_any, int a, int *b, int ret)
+{
+ return -1;
+}
+
+/* The error injection list allows any value although the target has a tag */
+SEC("fmod_ret/bpf_testmod_retval_injectable")
+__success
+int BPF_PROG(static_rule_tagged, int value, int ret)
+{
+ return -1;
+}
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __uint(max_entries, 1);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(__u32));
+} jmp_table SEC(".maps");
+
+/* A program in a program array may return any value */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("tail calls are not allowed in fmod_ret programs with restricted return values")
+int BPF_PROG(constant_tail_call, int ret)
+{
+ bpf_tail_call(ctx, &jmp_table, 0);
+ return 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_declared")
+__success
+int BPF_PROG(declared_tail_call, int ret)
+{
+ bpf_tail_call(ctx, &jmp_table, 0);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
new file mode 100644
index 00000000000000..dd105eea05ee46
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * An extension of fmod_ret programs, see prog_tests/fmodret_retval.c. The
+ * return values of the program on bpf_testmod_retval_constant() are restricted
+ * by its "bpf_fmodret_retval:" tag, those of the program on the declared target
+ * are not. An extension cannot replace the main function of a tracing program,
+ * so each of them calls a global function which the extension replaces.
+ */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+/* Keeps clang from folding the calls */
+int value_bias;
+
+__noinline int restricted_value(int value)
+{
+ return value + value_bias;
+}
+
+__noinline int unrestricted_value(int value)
+{
+ return value + value_bias;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+int BPF_PROG(restricted, int ret)
+{
+ int zero = 0;
+
+ if (restricted_value(ret))
+ return -22;
+ /*
+ * fmod_ret BPF progs must return constants (0 or -22 in this case).
+ * The barrier here forces GCC to return a constant
+ */
+ barrier_var(zero);
+ return zero;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_declared")
+int BPF_PROG(unrestricted, int ret)
+{
+ return unrestricted_value(ret);
+}
+
+SEC("freplace")
+int extension(int value)
+{
+ return -1;
+}
diff --git a/tools/testing/selftests/bpf/progs/fsession_test.c b/tools/testing/selftests/bpf/progs/fsession_test.c
index 86e8a2fe467e20..4295f6a78dd128 100644
--- a/tools/testing/selftests/bpf/progs/fsession_test.c
+++ b/tools/testing/selftests/bpf/progs/fsession_test.c
@@ -4,6 +4,8 @@
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
+extern int bpf_fentry_test1(int a) __ksym;
+
char _license[] SEC("license") = "GPL";
__u64 test1_entry_result = 0;
diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
index bbeea0d512e338..51a7f605f9c5d7 100644
--- a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
+++ b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
@@ -3,6 +3,8 @@
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
+extern int bpf_fentry_test1(int a) __ksym;
+
char _license[] SEC("license") = "GPL";
__u64 test1_entry_result = 0;
diff --git a/tools/testing/selftests/bpf/progs/tracing_multi_check.c b/tools/testing/selftests/bpf/progs/tracing_multi_check.c
index b2959ba711790e..9b008ecd65651e 100644
--- a/tools/testing/selftests/bpf/progs/tracing_multi_check.c
+++ b/tools/testing/selftests/bpf/progs/tracing_multi_check.c
@@ -8,7 +8,7 @@ char _license[] SEC("license") = "GPL";
int pid = 0;
bool test_cookies = false;
-/* bpf_fentry_test1 is exported as kfunc via vmlinux.h */
+extern int bpf_fentry_test1(int a) __ksym;
extern const void bpf_fentry_test2 __ksym;
extern const void bpf_fentry_test3 __ksym;
extern const void bpf_fentry_test4 __ksym;
diff --git a/tools/testing/selftests/bpf/test_kmods/Makefile b/tools/testing/selftests/bpf/test_kmods/Makefile
index 031c7454ce65f6..f4af3c3ab54c2d 100644
--- a/tools/testing/selftests/bpf/test_kmods/Makefile
+++ b/tools/testing/selftests/bpf/test_kmods/Makefile
@@ -19,10 +19,23 @@ Q = @
endif
MODULES = bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
- bpf_test_modorder_y.ko bpf_test_rqspinlock.ko
+ bpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_retval.ko
$(foreach m,$(MODULES),$(eval obj-m += $(m:.ko=.o)))
+bpf_test_retval-y := bpf_test_retval_main.o
+# The functions for bpf_scan_ir use clang extensions and need the plugin
+bpf_test_retval-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_testmod_retval.o \
+ bpf_testmod_retval_strong.o
+# Instrumentation and branch profiling make every path of the fixtures dirty
+GCOV_PROFILE_bpf_testmod_retval.o := n
+KCOV_INSTRUMENT_bpf_testmod_retval.o := n
+KMSAN_SANITIZE_bpf_testmod_retval.o := n
+KASAN_SANITIZE_bpf_testmod_retval.o := n
+KCSAN_SANITIZE_bpf_testmod_retval.o := n
+UBSAN_SANITIZE_bpf_testmod_retval.o := n
+CFLAGS_bpf_testmod_retval.o += -DDISABLE_BRANCH_PROFILING
+
CFLAGS_bpf_testmod.o = -I$(src)
# When BPF_STRICT_BUILD != 0, a missing KDIR is fatal (the default).
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c b/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c
new file mode 100644
index 00000000000000..c3a9093fa3310c
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A module for the functions of bpf_testmod_retval.c and
+ * bpf_testmod_retval_strong.c, see prog_tests/fmodret_retval.c. They are not
+ * in bpf_testmod: pahole drops the per-CPU variables of a module which GNU ld
+ * links from several objects.
+ */
+#include <linux/module.h>
+
+MODULE_AUTHOR("Anton Protopopov");
+MODULE_DESCRIPTION("BPF selftest module for fmod_ret targets found by bpf_scan_ir");
+MODULE_LICENSE("GPL");
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index 41ff36f0ca77e9..91cb311a2dc8b1 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -2607,15 +2607,9 @@ struct bpf_struct_ops testmod_multi_st_ops = {
extern int bpf_fentry_test1(int a);
-BTF_KFUNCS_START(bpf_testmod_trampoline_count_ids)
+BTF_FMODRET_START(bpf_testmod_trampoline_count_ids)
BTF_ID_FLAGS(func, bpf_testmod_trampoline_count_test)
-BTF_KFUNCS_END(bpf_testmod_trampoline_count_ids)
-
-static const struct
-btf_kfunc_id_set bpf_testmod_trampoline_count_fmodret_set = {
- .owner = THIS_MODULE,
- .set = &bpf_testmod_trampoline_count_ids,
-};
+BTF_FMODRET_END(bpf_testmod_trampoline_count_ids)
static int bpf_testmod_init(void)
{
@@ -2633,7 +2627,6 @@ static int bpf_testmod_init(void)
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_testmod_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_testmod_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_testmod_kfunc_set);
- ret = ret ?: register_btf_fmodret_id_set(&bpf_testmod_trampoline_count_fmodret_set);
ret = ret ?: register_bpf_struct_ops(&bpf_bpf_testmod_ops, bpf_testmod_ops);
ret = ret ?: register_bpf_struct_ops(&bpf_testmod_ops2, bpf_testmod_ops2);
ret = ret ?: register_bpf_struct_ops(&bpf_testmod_ops3, bpf_testmod_ops3);
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
new file mode 100644
index 00000000000000..fd63c6317cb944
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Functions for the fmod_ret targets which the LLVM IR scanner finds, see
+ * prog_tests/fmodret_retval.c. The "test_retval:" decl tag of each lists the
+ * values the scanner must find on paths without side effects. An empty list
+ * means that the function must not become such a target.
+ */
+#include <linux/atomic.h>
+#include <linux/btf_ids.h>
+#include <linux/compiler.h>
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/error-injection.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#define RETVAL(values) __attribute__((btf_decl_tag("test_retval:" values)))
+
+static __used int bpf_testmod_retval_state;
+static __used int *bpf_testmod_retval_captured;
+
+static noinline void bpf_testmod_retval_effect(void)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, 1);
+}
+
+static noinline void bpf_testmod_retval_capture(int *ptr)
+{
+ WRITE_ONCE(bpf_testmod_retval_captured, ptr);
+}
+
+static noinline int bpf_testmod_retval_store(int value)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, value);
+ return value;
+}
+
+static noinline int bpf_testmod_retval_clear(int value)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, 0);
+ return value - 1;
+}
+
+/* Constant errors and both return widths */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_constant(void)
+{
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("-14") s64 bpf_testmod_retval_wide(void)
+{
+ return -EFAULT;
+}
+
+static __used noinline RETVAL("-1") int bpf_testmod_retval_first_errno(void)
+{
+ return -1;
+}
+
+static __used noinline RETVAL("-4095") int bpf_testmod_retval_last_errno(void)
+{
+ return -MAX_ERRNO;
+}
+
+/*
+ * PTR_ERR() of the ERR_PTR() of an inlined helper. Not recognized yet: the
+ * error reaches the return through a PHI in an earlier block.
+ */
+static __always_inline int *bpf_testmod_retval_slot(int *buf, int idx)
+{
+ if (idx < 0)
+ return ERR_PTR(-EINVAL);
+ if (idx > 8)
+ return ERR_PTR(-EFAULT);
+ return &buf[idx];
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_err_ptr(int *buf, int idx)
+{
+ int *slot = bpf_testmod_retval_slot(buf, idx);
+
+ if (IS_ERR(slot))
+ return PTR_ERR(slot);
+ *slot = 1;
+ return 0;
+}
+
+/* A clean error path may join a dirty success path */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_before_store(int *result, int value)
+{
+ if (value < 0)
+ return -EINVAL;
+ *result = value;
+ return 0;
+}
+
+static __used noinline RETVAL("-34,-22") int bpf_testmod_retval_join(int *result, int value)
+{
+ int ret;
+
+ if (value < 0)
+ ret = -EINVAL;
+ else if (value > 32)
+ ret = -ERANGE;
+ else {
+ *result = value;
+ ret = 0;
+ }
+ return ret;
+}
+
+/*
+ * An int error which a long function returns: the PHI of the two calls keeps
+ * the sign extension in the block of the return
+ */
+static __used noinline RETVAL("-22") long bpf_testmod_retval_sign_extend(int value)
+{
+ int ret;
+
+ if (value < 0)
+ ret = -EINVAL;
+ else if (value > 32)
+ ret = bpf_testmod_retval_store(value);
+ else
+ ret = bpf_testmod_retval_clear(value);
+ return ret;
+}
+
+/* An assumption does not make a path dirty */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_assume(int value)
+{
+ __builtin_assume(value > 0);
+ return -EINVAL;
+}
+
+/* Writes to private stack objects are clean, optnone keeps them in the IR */
+static __used noinline RETVAL("-22") __attribute__((optnone))
+int bpf_testmod_retval_private_store(int value)
+{
+ int scratch[2];
+
+ scratch[0] = value;
+ scratch[1] = 0;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("-22") __attribute__((optnone))
+int bpf_testmod_retval_private_memset(void)
+{
+ char scratch[32];
+
+ __builtin_memset(scratch, 0, sizeof(scratch));
+ return -EINVAL;
+}
+
+/* An error which is not known exactly is not recorded */
+static __used noinline RETVAL("") int bpf_testmod_retval_masked(unsigned int value)
+{
+ return -1 - (value & 31);
+}
+
+static __used noinline RETVAL("") long bpf_testmod_retval_ptr_err(void *ptr, int *state)
+{
+ if (IS_ERR(ptr))
+ return PTR_ERR(ptr);
+ *state = 1;
+ return 0;
+}
+
+/* Pointer functions are not targets yet */
+static __used noinline RETVAL("") void *bpf_testmod_retval_error_pointer(void)
+{
+ return ERR_PTR(-ENOMEM);
+}
+
+/* Every error path has an observable effect */
+static __used noinline RETVAL("") int bpf_testmod_retval_argument_store(int *state)
+{
+ *state = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_global_store(void)
+{
+ bpf_testmod_retval_state = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_volatile_read(int *state)
+{
+ return READ_ONCE(*state) ? -EINVAL : -ERANGE;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_atomic(atomic_t *state)
+{
+ atomic_inc(state);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_call(void)
+{
+ bpf_testmod_retval_effect();
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_barrier(void)
+{
+ barrier();
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("")
+int bpf_testmod_retval_only_error_is_dirty(int *state, bool reject)
+{
+ if (reject) {
+ *state = 1;
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static __used noinline RETVAL("")
+int bpf_testmod_retval_lock_held(spinlock_t *lock) __acquires(lock)
+{
+ spin_lock(lock);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_stack_escape(void)
+{
+ int scratch;
+
+ bpf_testmod_retval_capture(&scratch);
+ scratch = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_external_memset(void *dest)
+{
+ __builtin_memset(dest, 0, 32);
+ return -EINVAL;
+}
+
+/* Not recognized yet, although they have no side effects */
+static __used noinline RETVAL("") int bpf_testmod_retval_balanced_spinlock(spinlock_t *lock)
+{
+ spin_lock(lock);
+ spin_unlock(lock);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_input_errno(int error)
+{
+ if (error >= -MAX_ERRNO && error < 0)
+ return error;
+ return 0;
+}
+
+/* No ftrace call site, but call padding in __patchable_function_entries on x86 */
+static __used noinline RETVAL("") notrace int bpf_testmod_retval_notrace(void)
+{
+ return -EINVAL;
+}
+
+/*
+ * A weak definition with an error, replaced by a strong one without errors in
+ * bpf_testmod_retval_strong.c
+ */
+int bpf_testmod_retval_overridden(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_overridden(void)
+{
+ return -EINVAL;
+}
+
+/*
+ * Two weak definitions: this one with a side effect is linked first and wins
+ * over the one with an error in bpf_testmod_retval_strong.c
+ */
+int bpf_testmod_retval_two_weak(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_two_weak(void)
+{
+ bpf_testmod_retval_effect();
+ return 0;
+}
+
+/*
+ * A function on the error injection list gets its tag, but the static rules let
+ * programs return any value
+ */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_injectable(int value)
+{
+ if (value < 0)
+ return -EINVAL;
+ bpf_testmod_retval_effect();
+ return 0;
+}
+ALLOW_ERROR_INJECTION(bpf_testmod_retval_injectable, ERRNO);
+
+/* A declared target keeps only its own tags */
+static __used noinline RETVAL("") int bpf_testmod_retval_declared(void)
+{
+ return -EINVAL;
+}
+
+BTF_FMODRET_START(bpf_testmod_retval_declared_ids)
+BTF_ID_FLAGS(func, bpf_testmod_retval_declared)
+BTF_FMODRET_END(bpf_testmod_retval_declared_ids)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
new file mode 100644
index 00000000000000..a246d7f619bc51
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Replaces the weak bpf_testmod_retval_overridden() of bpf_testmod_retval.c.
+ * This definition returns no errors, so the function must not become an
+ * fmod_ret target, see prog_tests/fmodret_retval.c.
+ *
+ * The weak bpf_testmod_retval_two_weak() here loses the link to the one of
+ * bpf_testmod_retval.c, so its error must not make the function a target
+ * either.
+ */
+#include <linux/compiler.h>
+#include <linux/errno.h>
+
+int bpf_testmod_retval_overridden(void);
+
+noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_overridden(void)
+{
+ return 0;
+}
+
+int bpf_testmod_retval_two_weak(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_two_weak(void)
+{
+ return -EINVAL;
+}
|
| 3/2 | 2026/11/12 15:30 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 69031 output: 66 thoughts: 1210 Thoughts: **BPF fmodret and Syzkaller Fuzzing Analysis** Okay, let's dive into this patch. My immediate thought is, does this warrant syzkaller fuzzing? And if so, what's the best approach – configs, focus symbols, the works? First, I need to understand what this patch *does*. Initial scan shows a lot of BPF-related changes: `MAINTAINERS`, `Makefile` updates, `Kconfig` modifications, and significant changes within `kernel/bpf/`. Specifically, I see alterations to `btf.c`, `syscall.c`, and `verifier.c`. This is promising – BPF is a known attack surface. Peering deeper into the `kernel/bpf/` changes, I notice `btf_register_fmodret_tags`, modifications to `btf_kfunc_id_set_contains`, verifier additions for `BTF_FMODRET_RETVAL_TAG`, and logic related to `fmodret_retval_init` and `check_fmodret_retval`. Hmm, it's modifying the BPF verifier and BTF handling, specifically for programs with `BPF_MODIFY_RETURN`! That’s interesting. It looks like the verifier is now enforcing return value checks. Now the question is, *can* this be reached through standard, syzkaller-friendly virtualized environments? YES! BPF syscalls are routinely fuzzed by syzkaller. Loading a BPF tracing program, a modify_return program, that attaches to functions triggers this new verifier logic. And `btf_register_fmodret_tags` runs at boot, and module load, too. This is core kernel code; it's right in the BPF subsystem, which covers the runtime checks, the verifier – this is a prime target for fuzzing. Let's do a quick Section 1, 2, 3 check: This ticks all the WorthFuzzing criteria boxes: it's touching reachable executable code in a core subsystem (BPF). We're modifying code and data structure (new return value checking), and adding runtime assertions. WorthFuzzing is a definite yes. Now for configs. I'm going to determine what's needed to compile and *reach* this code. I need to figure out the minimal set to activate this logic. I'll need to check the Kconfig, and the diff itself, but it looks like I need a standard `BPF` build, `BTF` support, etc. Looks like standard syzkaller config. Okay, let's get those configs sorted out and start crafting a fuzzing strategy. |
| 4/2 | 2026/11/12 15:31 | tool | grepper |
0mArgs:{
"Expression": "config BPF_SYSCALL|config DEBUG_INFO_BTF|config CLANG_PLUGIN_BPF_SCAN_IR",
"PathPrefixes": [
"init/Kconfig",
"kernel/bpf/",
"scripts/"
]
}Results: {
"Output": "kernel/bpf/Kconfig=34=menu \"BPF subsystem\"\nkernel/bpf/Kconfig-35-\nkernel/bpf/Kconfig:36:config BPF_SYSCALL\nkernel/bpf/Kconfig-37-\tbool \"Enable bpf() system call\"\n--\nscripts/llvm-plugins/Kconfig=19=if LLVM_PLUGINS\nscripts/llvm-plugins/Kconfig-20-\nscripts/llvm-plugins/Kconfig:21:config CLANG_PLUGIN_BPF_SCAN_IR\nscripts/llvm-plugins/Kconfig-22-\tbool \"Scan optimized LLVM IR for BPF\"\n"
} |
| 5/2 | 2026/11/12 15:31 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 71049 output: 36 thoughts: 39 Thoughts: Okay, let's dive into this. Here's how I'd approach this task, as if these were my internal thoughts: **Investigating DEBUG_INFO_BTF Configuration in LLVM Plugins** Alright, I need to understand the relationship between `scripts/llvm-plugins/Kconfig` and `DEBUG_INFO_BTF`. It seems like I need to see how the latter is defined or used within the former, specifically in the context of the LLVM plugins. My initial instinct is to use `grep`. I'll use it to search the `Kconfig` file for all instances of `DEBUG_INFO_BTF`. This should give me a good handle on how this configuration option is being handled and where it plays a role in the build process of these plugins. That's the best first step. Let's see what `grep` turns up. |
| 6/2 | 2026/11/12 15:31 | tool | grepper |
0mArgs:{
"Expression": "config DEBUG_INFO_BTF\\b",
"PathPrefixes": [
"lib/Kconfig.debug"
]
}Results: {
"Output": "lib/Kconfig.debug=378=config DEBUG_INFO_SPLIT\n--\nlib/Kconfig.debug-397-\nlib/Kconfig.debug:398:config DEBUG_INFO_BTF\nlib/Kconfig.debug-399-\tbool \"Generate BTF type information\"\n"
} |
| 7/2 | 2026/11/12 15:31 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 71393 output: 40 thoughts: 162 Thoughts: **Investigating Potential Conditional Compilation** Okay, so I need to make sure I understand how this new code integrates, specifically any conditional compilation that might affect its behavior. My first thought is to check `kernel/bpf/verifier.c` and `kernel/bpf/btf.c` for any `#ifdef` directives that might surround the relevant additions. Right, so digging into `kernel/bpf/btf.c`, I see `btf_register_fmodret_tags` is called in two places: within `btf_fmodret_tags_init`, which is set up as a `fs_initcall`, and also inside `btf_module_notify` under the `CONFIG_DEBUG_INFO_BTF_MODULES` flag. This is interesting. Now I'm wondering if `btf_register_fmodret_tags` itself is conditionally compiled. I need to confirm that. Let me scan around lines 9307 and 10183 in `kernel/bpf/btf.c` to be certain. |
| 8/2 | 2026/11/12 15:31 | tool | grepper |
0mArgs:{
"Expression": "btf_register_fmodret_tags",
"PathPrefixes": [
"kernel/bpf/btf.c"
]
}Results: {
"Output": "kernel/bpf/btf.c=9305=enum {\n--\nkernel/bpf/btf.c-9309-#ifdef CONFIG_DEBUG_INFO_BTF_MODULES\nkernel/bpf/btf.c:9310:static int btf_register_fmodret_tags(struct btf *btf);\nkernel/bpf/btf.c-9311-\n--\nkernel/bpf/btf.c=9431=static int btf_module_notify(struct notifier_block *nb, unsigned long op,\n--\nkernel/bpf/btf.c-9484-\t\t */\nkernel/bpf/btf.c:9485:\t\terr = btf_register_fmodret_tags(btf);\nkernel/bpf/btf.c-9486-\t\tif (err)\n--\nkernel/bpf/btf.c=10210=static u32 btf_fmodret_tag_func(const struct btf *btf, const struct btf_type *t,\n--\nkernel/bpf/btf.c-10225-/* Build the set of fmod_ret targets of BTF from its decl tags */\nkernel/bpf/btf.c:10226:static int btf_register_fmodret_tags(struct btf *btf)\nkernel/bpf/btf.c-10227-{\n--\nkernel/bpf/btf.c=10282=static int __init btf_fmodret_tags_init(void)\n--\nkernel/bpf/btf.c-10290-\nkernel/bpf/btf.c:10291:\terr = btf_register_fmodret_tags(btf);\nkernel/bpf/btf.c-10292-\tbtf_put(btf);\n"
} |
| 9/2 | 2026/11/12 15:31 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 72308 output: 41 thoughts: 668 Thoughts: **Investigating the BPF fmodret Implementation** Okay, let's dive into this `fmodret` stuff. I'm starting by looking at `scripts/llvm-plugins/Kconfig` to understand the dependencies and configuration options. It looks like `CONFIG_LLVM_PLUGINS` is the parent option, and then `CONFIG_CLANG_PLUGIN_BPF_SCAN_IR` depends on that, along with some architecture and clang version checks. Now, is `CONFIG_CLANG_PLUGIN_BPF_SCAN_IR` truly essential, or are `CONFIG_BPF_SYSCALL` and `CONFIG_DEBUG_INFO_BTF` the real prerequisites? The documentation suggests that if `CONFIG_CLANG_PLUGIN_BPF_SCAN_IR` is enabled, a compiler plugin scans the optimized LLVM IR for `fmodret` candidates, creating a `.BTF_fmodret_candidates` section. `resolve_btfids` then parses this and adds `bpf_fmodret` tags to the BTF data. But hold on... what happens if the plugin isn't enabled? I need to know. I'm seeing references to `BTF_FMODRET_START` macros in `net/bpf/test_run.c` and `net/mptcp/bpf.c`. Even for statically declared targets with those macros, `resolve_btfids` still emits `bpf_fmodret` and `bpf_fmodret_sleepable` decl tags. Then, at runtime, `btf_fmodret_tags_init` and `btf_register_fmodret_tags` build the set of `fmod_ret` targets based on these decl tags. Now, let's look at `kernel/bpf/verifier.c`. Functions like `fmodret_retval_init`, `check_fmodret_retval`, `fmodret_retval_allowed`, and `fmodret_retval_next` are called during verification. *Wait a minute!* `fmodret_retval_init` specifically looks for `BTF_FMODRET_RETVAL_TAG`, which comes from the *output* of the `CLANG_PLUGIN_BPF_SCAN_IR` plugin in `resolve_btfids`! So, the key question is whether or not `CONFIG_CLANG_PLUGIN_BPF_SCAN_IR` or even just `CONFIG_LLVM_PLUGINS` is needed to generate those tags for compiler-scanned targets. I need to figure out exactly what configs are necessary to compile the verifier code, and the BTF code, and for the fmodret functionality to be active. I'll need to check the `kernel/bpf/Makefile` next. |
| 10/2 | 2026/11/12 15:31 | tool | grepper |
0mArgs:{
"Expression": "obj-.*verifier\\.o|obj-.*btf\\.o",
"PathPrefixes": [
"kernel/bpf/Makefile"
]
}Results: {
"Output": "kernel/bpf/Makefile=7=CFLAGS_core.o += -Wno-override-init $(cflags-nogcse-yy)\nkernel/bpf/Makefile-8-\nkernel/bpf/Makefile:9:obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o cnum.o log.o token.o liveness.o const_fold.o diagnostics.o loops.o scev.o\nkernel/bpf/Makefile-10-obj-$(CONFIG_BPF_SYSCALL) += bpf_iter.o map_iter.o task_iter.o prog_iter.o link_iter.o\n--\nkernel/bpf/Makefile=13=obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o\nkernel/bpf/Makefile:14:obj-$(CONFIG_BPF_SYSCALL) += fixups.o cfg.o states.o check_btf.o flow.o join.o\nkernel/bpf/Makefile-15-obj-${CONFIG_BPF_LSM}\t += bpf_inode_storage.o\n--\nkernel/bpf/Makefile=17=obj-$(CONFIG_BPF_JIT) += trampoline.o\nkernel/bpf/Makefile:18:obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o rqspinlock.o stream.o\nkernel/bpf/Makefile-19-ifeq ($(CONFIG_MMU)$(CONFIG_64BIT),yy)\n--\nkernel/bpf/Makefile=43=ifeq ($(CONFIG_SYSFS),y)\nkernel/bpf/Makefile:44:obj-$(CONFIG_DEBUG_INFO_BTF) += sysfs_btf.o\nkernel/bpf/Makefile-45-obj-$(CONFIG_DEBUG_INFO_BTF_INLINE) += btf_vmlinux_inline.o\n"
} |
| 11/2 | 2026/11/12 15:31 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 74105 output: 32 thoughts: 257 Thoughts: **My BPF Compilation Thoughts** Okay, so I'm thinking about how `BPF_SYSCALL` handles the compilation process. It pulls together `verifier.o`, `syscall.o`, and `btf.o`. Now, I need to keep in mind the role of `DEBUG_INFO_BTF`. Let me quickly look at `include/linux/btf_ids.h`. Ah, right, `#ifdef CONFIG_DEBUG_INFO_BTF` is the key. That's where things like `BTF_FMODRET_START` and `BTF_SET8_START` are defined. Without that configuration option, those are just empty static structs, `btf_id_set8`. And the `resolve_btfids` function only gets called if `CONFIG_DEBUG_INFO_BTF` is actually set. Plus, there is also `DEBUG_INFO_BTF_MODULES` which handles BTF for modules. And I can't forget about `CONFIG_BPF_JIT`. The `fmod_ret` programs use a trampoline for attachment, and trampolines need `BPF_JIT`. Now, to be really thorough, let me double-check what `BPF_MODIFY_RETURN` actually needs, in `verifier.c`. Time for a quick grep for `BPF_MODIFY_RETURN` within the `kernel/bpf/` directory. |
| 12/2 | 2026/11/12 15:31 | tool | grepper |
0mArgs:{
"Expression": "BPF_MODIFY_RETURN",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "kernel/bpf/btf.c=6697=static int btf_validate_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf,\n--\nkernel/bpf/btf.c-6778-\t\tcase BPF_TRACE_FEXIT:\nkernel/bpf/btf.c:6779:\t\tcase BPF_MODIFY_RETURN:\nkernel/bpf/btf.c-6780-\t\tcase BPF_TRACE_FSESSION:\n--\nkernel/bpf/btf.c=7452=bool btf_ctx_access(int off, int size, enum bpf_access_type type,\n--\nkernel/bpf/btf.c-7506-\t\t\t *\nkernel/bpf/btf.c:7507:\t\t\t * While the LSM programs are BPF_MODIFY_RETURN-like\nkernel/bpf/btf.c-7508-\t\t\t * the check:\n--\nkernel/bpf/btf.c-7519-\t\t\tbreak;\nkernel/bpf/btf.c:7520:\t\tcase BPF_MODIFY_RETURN:\nkernel/bpf/btf.c:7521:\t\t\t/* For now the BPF_MODIFY_RETURN can only be attached to\nkernel/bpf/btf.c-7522-\t\t\t * functions that return an int.\n--\nkernel/bpf/fixups.c=2593=int bpf_do_misc_fixups(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-3287-\t\t\t eatype == BPF_TRACE_FSESSION_MULTI ||\nkernel/bpf/fixups.c:3288:\t\t\t eatype == BPF_MODIFY_RETURN) {\nkernel/bpf/fixups.c-3289-\t\t\t\t/* Load nr_args from ctx - 8 */\n--\nkernel/bpf/syscall.c=3674=static int bpf_tracing_prog_attach(struct bpf_prog *prog,\n--\nkernel/bpf/syscall.c-3691-\t\t prog-\u003eexpected_attach_type != BPF_TRACE_FSESSION \u0026\u0026\nkernel/bpf/syscall.c:3692:\t\t prog-\u003eexpected_attach_type != BPF_MODIFY_RETURN) {\nkernel/bpf/syscall.c-3693-\t\t\terr = -EINVAL;\n--\nkernel/bpf/syscall.c=4454=attach_type_to_prog_type(enum bpf_attach_type attach_type)\n--\nkernel/bpf/syscall.c-4509-\tcase BPF_TRACE_FEXIT_MULTI:\nkernel/bpf/syscall.c:4510:\tcase BPF_MODIFY_RETURN:\nkernel/bpf/syscall.c-4511-\t\treturn BPF_PROG_TYPE_TRACING;\n--\nkernel/bpf/trampoline.c=177=bool bpf_prog_has_trampoline(const struct bpf_prog *prog)\n--\nkernel/bpf/trampoline.c-184-\t\tif (eatype == BPF_TRACE_FENTRY || eatype == BPF_TRACE_FEXIT ||\nkernel/bpf/trampoline.c:185:\t\t eatype == BPF_MODIFY_RETURN || eatype == BPF_TRACE_FSESSION ||\nkernel/bpf/trampoline.c-186-\t\t eatype == BPF_TRACE_FENTRY_MULTI || eatype == BPF_TRACE_FEXIT_MULTI ||\n--\nkernel/bpf/trampoline.c=828=static enum bpf_tramp_prog_type bpf_attach_type_to_tramp(struct bpf_prog *prog)\n--\nkernel/bpf/trampoline.c-833-\t\treturn BPF_TRAMP_FENTRY;\nkernel/bpf/trampoline.c:834:\tcase BPF_MODIFY_RETURN:\nkernel/bpf/trampoline.c-835-\t\treturn BPF_TRAMP_MODIFY_RETURN;\n--\nkernel/bpf/verifier.c=20128=static bool return_retval_range(struct bpf_verifier_env *env, struct bpf_retval_range *range)\n--\nkernel/bpf/verifier.c-20182-\t\tcase BPF_TRACE_RAW_TP:\nkernel/bpf/verifier.c:20183:\t\tcase BPF_MODIFY_RETURN:\nkernel/bpf/verifier.c-20184-\t\t\treturn false;\n--\nkernel/bpf/verifier.c=23987=static bool attach_uses_trampoline_retval(enum bpf_attach_type type)\n--\nkernel/bpf/verifier.c-23989-\tswitch (type) {\nkernel/bpf/verifier.c:23990:\tcase BPF_MODIFY_RETURN:\nkernel/bpf/verifier.c-23991-\tcase BPF_TRACE_FEXIT:\n--\nkernel/bpf/verifier.c=24108=int bpf_check_attach_target(struct bpf_verifier_log *log,\n--\nkernel/bpf/verifier.c-24393-\t\tfallthrough;\nkernel/bpf/verifier.c:24394:\tcase BPF_MODIFY_RETURN:\nkernel/bpf/verifier.c-24395-\tcase BPF_LSM_MAC:\n--\nkernel/bpf/verifier.c-24502-\t\t\t}\nkernel/bpf/verifier.c:24503:\t\t} else if (prog-\u003eexpected_attach_type == BPF_MODIFY_RETURN) {\nkernel/bpf/verifier.c-24504-\t\t\tif (tgt_prog) {\n--\nkernel/bpf/verifier.c=24575=static bool can_be_sleepable(struct bpf_prog *prog)\n--\nkernel/bpf/verifier.c-24580-\t\tcase BPF_TRACE_FEXIT:\nkernel/bpf/verifier.c:24581:\t\tcase BPF_MODIFY_RETURN:\nkernel/bpf/verifier.c-24582-\t\tcase BPF_TRACE_ITER:\n--\nkernel/bpf/verifier.c=24602=static int check_attach_btf_id(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-24683-\t\t prog-\u003eexpected_attach_type == BPF_TRACE_FSESSION_MULTI ||\nkernel/bpf/verifier.c:24684:\t\t prog-\u003eexpected_attach_type == BPF_MODIFY_RETURN) \u0026\u0026\nkernel/bpf/verifier.c-24685-\t\t btf_id_set_contains(\u0026noreturn_deny, btf_id)) {\n--\nkernel/bpf/verifier.c-24691-\t/* Only a kernel function has a \"bpf_fmodret_retval:\" tag */\nkernel/bpf/verifier.c:24692:\tif (prog-\u003eexpected_attach_type == BPF_MODIFY_RETURN \u0026\u0026 !tgt_prog) {\nkernel/bpf/verifier.c-24693-\t\tret = fmodret_retval_init(env, btf_id, tgt_info.tgt_addr, tgt_info.tgt_name);\n"
} |
| 13/2 | 2026/11/12 15:31 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 76529 output: 36 thoughts: 60 Thoughts: Okay, let's dive into this. **Navigating the Kernel's BPF Verifier** Alright, I'm going to zero in on `kernel/bpf/verifier.c`, specifically around line 23759. I'm looking at something here... wait a minute, I see a `#ifdef CONFIG_FUNCTION_ERROR_INJECTION` block. That's a good place to start, but what's really grabbing my attention is `check_attach_modify_return`. Where *is* that defined? I need to check where that function actually lives to understand its context. Let's trace that definition down, and see what the flow looks like. |
| 14/2 | 2026/11/12 15:31 | tool | grepper |
0mArgs:{
"Expression": "check_attach_modify_return",
"PathPrefixes": [
"kernel/bpf/verifier.c"
]
}Results: {
"Output": "kernel/bpf/verifier.c=23765=static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)\n--\nkernel/bpf/verifier.c-23776-\nkernel/bpf/verifier.c:23777:static int check_attach_modify_return(unsigned long addr, const char *func_name)\nkernel/bpf/verifier.c-23778-{\n--\nkernel/bpf/verifier.c=23815=static int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)\n--\nkernel/bpf/verifier.c-23822-\nkernel/bpf/verifier.c:23823:static int check_attach_modify_return(unsigned long addr, const char *func_name)\nkernel/bpf/verifier.c-23824-{\n--\nkernel/bpf/verifier.c=23839=static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,\n--\nkernel/bpf/verifier.c-23849-\nkernel/bpf/verifier.c:23850:\tif (!check_attach_modify_return(addr, tname))\nkernel/bpf/verifier.c-23851-\t\treturn 0;\n--\nkernel/bpf/verifier.c=24108=int bpf_check_attach_target(struct bpf_verifier_log *log,\n--\nkernel/bpf/verifier.c-24510-\t\t\tif (btf_kfunc_is_modify_return(btf, btf_id, prog) ||\nkernel/bpf/verifier.c:24511:\t\t\t !check_attach_modify_return(addr, tname))\nkernel/bpf/verifier.c-24512-\t\t\t\tret = 0;\n"
} |
| 15/2 | 2026/11/12 15:31 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 77268 output: 38 thoughts: 29 Thoughts: Okay, let's dive into this. Here's my take on those lines from `kernel/bpf/verifier.c`: **Analyzing BPF Verifier Logic: Section 23770-23840** Alright, time to get my hands dirty with some BPF verifier code. Let's see, the file is `kernel/bpf/verifier.c`, and I'm interested in lines 23770 to 23840. That typically means we're dealing with a specific aspect of the verification process, likely a relatively granular check. Given the location within the verifier, I'd expect it to relate to state management, instruction analysis, potential safety checks, or perhaps even error reporting during verification. I'll need to look at the immediate surrounding code to quickly understand the context. This likely involves handling registers, stack frames, program boundaries or perhaps dealing with indirect accesses. Let's start the analysis and see what it reveals. This part of the code likely determines if certain BPF programs are safe to execute, or flags potential vulnerabilities which could be exploited. |
| 16/2 | 2026/11/12 15:32 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/verifier.c",
"FirstLine": 23765,
"LineCount": 70
}Results: {
"Contents": "23765:\tstatic int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)\n23766:\t{\n23767:\t\t/* fentry/fexit/fmod_ret progs can be sleepable if they are\n23768:\t\t * attached to ALLOW_ERROR_INJECTION and are not in denylist.\n23769:\t\t */\n23770:\t\tif (!check_non_sleepable_error_inject(btf_id) \u0026\u0026\n23771:\t\t within_error_injection_list(addr))\n23772:\t\t\treturn 0;\n23773:\t\n23774:\t\treturn -EINVAL;\n23775:\t}\n23776:\t\n23777:\tstatic int check_attach_modify_return(unsigned long addr, const char *func_name)\n23778:\t{\n23779:\t\tif (within_error_injection_list(addr) ||\n23780:\t\t !strncmp(SECURITY_PREFIX, func_name, sizeof(SECURITY_PREFIX) - 1))\n23781:\t\t\treturn 0;\n23782:\t\n23783:\t\treturn -EINVAL;\n23784:\t}\n23785:\t\n23786:\t#else\n23787:\t\n23788:\t/* Unfortunately, the arch-specific prefixes are hard-coded in arch syscall code\n23789:\t * so we need to hard-code them, too. Ftrace has arch_syscall_match_sym_name()\n23790:\t * but that just compares two concrete function names.\n23791:\t */\n23792:\tstatic bool has_arch_syscall_prefix(const char *func_name)\n23793:\t{\n23794:\t#if defined(__x86_64__)\n23795:\t\treturn !strncmp(func_name, \"__x64_\", 6);\n23796:\t#elif defined(__i386__)\n23797:\t\treturn !strncmp(func_name, \"__ia32_\", 7);\n23798:\t#elif defined(__s390x__)\n23799:\t\treturn !strncmp(func_name, \"__s390x_\", 8);\n23800:\t#elif defined(__aarch64__)\n23801:\t\treturn !strncmp(func_name, \"__arm64_\", 8);\n23802:\t#elif defined(__riscv)\n23803:\t\treturn !strncmp(func_name, \"__riscv_\", 8);\n23804:\t#elif defined(__powerpc__) || defined(__powerpc64__)\n23805:\t\treturn !strncmp(func_name, \"sys_\", 4);\n23806:\t#elif defined(__loongarch__)\n23807:\t\treturn !strncmp(func_name, \"sys_\", 4);\n23808:\t#else\n23809:\t\treturn false;\n23810:\t#endif\n23811:\t}\n23812:\t\n23813:\t/* Without error injection, allow sleepable and fmod_ret progs on syscalls. */\n23814:\t\n23815:\tstatic int check_attach_sleepable(u32 btf_id, unsigned long addr, const char *func_name)\n23816:\t{\n23817:\t\tif (has_arch_syscall_prefix(func_name))\n23818:\t\t\treturn 0;\n23819:\t\n23820:\t\treturn -EINVAL;\n23821:\t}\n23822:\t\n23823:\tstatic int check_attach_modify_return(unsigned long addr, const char *func_name)\n23824:\t{\n23825:\t\tif (has_arch_syscall_prefix(func_name) ||\n23826:\t\t !strncmp(SECURITY_PREFIX, func_name, sizeof(SECURITY_PREFIX) - 1))\n23827:\t\t\treturn 0;\n23828:\t\n23829:\t\treturn -EINVAL;\n23830:\t}\n23831:\t\n23832:\t#endif /* CONFIG_FUNCTION_ERROR_INJECTION */\n23833:\t\n23834:\t/*\n"
} |
| 17/2 | 2026/11/12 15:32 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 78740 output: 37 thoughts: 573 Thoughts: **Decoding the fmodret Return Value Restriction Logic** Okay, let's break this down. My initial thought is to examine the `check_attach_modify_return` function. It seems to be the linchpin here, determining whether a function return value can be modified by a BPF program. Specifically, it looks like this function checks if an address, `addr`, falls under static rules for error injection or security prefixes. If `check_attach_modify_return` returns non-zero, it means the address *is* covered by a static rule – an error injection or security prefix. In this case, `fmodret_retval_init` proceeds to evaluate the `bpf_fmodret_retval:` decl tag associated with the BTF ID. However, if `check_attach_modify_return` returns zero (or its negation, `!check_attach_modify_return` is true), the function *isn't* subject to the static restrictions. Thus, the program would move on to handle BTF tags. This implies the function could be a legitimate fmodret target allowed via BTF tags. Essentially, if `check_attach_modify_return` *fails* (returns non-zero), we continue the checks! If it is `0`, then static checks *succeed*, and we can bypass the BTF tag check for restriction! The important commentary highlights this: programs attached to a function identified through analysis are *only* permitted to return values specified in its `"bpf_fmodret_retval:"` tag, unless static rules allow unrestricted modification. So how was the attachment to the function permitted in the first place? Line 24510 reveals the answer: `btf_kfunc_is_modify_return` or, the negation of `check_attach_modify_return` allowed it to attach! It's likely that a prior check, `btf_kfunc_is_modify_return`, determines if a function is *eligible* for return value modification. I should probably grep `kernel/bpf/` for the implementation of `btf_kfunc_is_modify_return` to get a better understanding. |
| 18/2 | 2026/11/12 15:32 | tool | grepper |
0mArgs:{
"Expression": "btf_kfunc_is_modify_return",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "kernel/bpf/btf.c=10123=int btf_kfunc_check_flag(const struct btf *btf, u32 kfunc_btf_id, u32 flag)\n--\nkernel/bpf/btf.c-10145-\nkernel/bpf/btf.c:10146:u32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,\nkernel/bpf/btf.c-10147-\t\t\t\tconst struct bpf_prog *prog)\n--\nkernel/bpf/verifier.c=23894=static int btf_id_allow_sleepable(u32 btf_id, unsigned long addr, const struct bpf_prog *prog,\n--\nkernel/bpf/verifier.c-23925-\t\telse {\nkernel/bpf/verifier.c:23926:\t\t\tu32 *flags = btf_kfunc_is_modify_return(btf, btf_id, prog);\nkernel/bpf/verifier.c-23927-\n--\nkernel/bpf/verifier.c=24108=int bpf_check_attach_target(struct bpf_verifier_log *log,\n--\nkernel/bpf/verifier.c-24509-\t\t\tret = -EINVAL;\nkernel/bpf/verifier.c:24510:\t\t\tif (btf_kfunc_is_modify_return(btf, btf_id, prog) ||\nkernel/bpf/verifier.c-24511-\t\t\t !check_attach_modify_return(addr, tname))\n"
} |
| 19/2 | 2026/11/12 15:32 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 80248 output: 39 thoughts: 29 Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person: **Diving into BTF Function Verification** Alright, time to dig into the `btf_kfunc_is_modify_return` function. I'm heading over to `kernel/bpf/btf.c`. This is where the core logic resides for verifying if a KFunc, within the context of BTF, is capable of modifying its return value. I need to understand exactly what criteria are being checked here. Specifically, I'm interested in the mechanisms and conditions used to determine whether the KFunc’s return is passed via a pointer in one of its arguments, potentially allowing for modification outside of the normal return register usage. The function likely delves deep into the BTF type information, analyzing arguments, return types, and their relationships. I expect to see checks for pointers, reference types, and perhaps even some more sophisticated analysis involving type casts or complex data structures. Getting this right is crucial for ensuring the safety and correctness of any BPF programs that rely on these KFuncs. |
| 20/2 | 2026/11/12 15:32 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/btf.c",
"FirstLine": 10145,
"LineCount": 20
}Results: {
"Contents": "10145:\t\n10146:\tu32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,\n10147:\t\t\t\t\tconst struct bpf_prog *prog)\n10148:\t{\n10149:\t\tif (!__btf_kfunc_is_allowed(btf, BTF_KFUNC_HOOK_FMODRET, kfunc_btf_id, prog))\n10150:\t\t\treturn NULL;\n10151:\t\n10152:\t\treturn btf_kfunc_id_set_contains(btf, BTF_KFUNC_HOOK_FMODRET, kfunc_btf_id);\n10153:\t}\n10154:\t\n10155:\tstatic int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook,\n10156:\t\t\t\t\t const struct btf_kfunc_id_set *kset)\n10157:\t{\n10158:\t\tstruct btf *btf;\n10159:\t\tint ret, i;\n10160:\t\n10161:\t\tbtf = btf_get_module_btf(kset-\u003eowner);\n10162:\t\tif (!btf)\n10163:\t\t\treturn check_btf_kconfigs(kset-\u003eowner, \"kfunc\");\n10164:\t\tif (IS_ERR(btf))\n"
} |
| 21/2 | 2026/11/12 15:32 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 80949 output: 34 thoughts: 1746 Thoughts: **Fmodret and BPF: A Deep Dive**
Okay, here's what's running through my head about this fmodret feature. `btf_kfunc_id_set_contains` is checking if a specific kfunc ID is in the `BTF_KFUNC_HOOK_FMODRET` set. The patch introduces `btf_register_fmodret_tags`, which dynamically populates this set by scanning BTF decl tags for `BTF_FMODRET_TAG` ("bpf_fmodret") and `BTF_FMODRET_SLEEPABLE_TAG`. This used to be done during initcalls, but now it's handled dynamically during BTF initialization and module loading. This is a pretty significant shift.
The really interesting bit is the clang plugin, enabled by `CONFIG_CLANG_PLUGIN_BPF_SCAN_IR`. This plugin automatically finds functions with clean error returns in the IR, marks them, and creates `bpf_fmodret_retval:` tags. This is what allows BPF programs to attach to these functions and restrict the return values. The verifier then gets in on the action: it parses the `bpf_fmodret_retval:` tag, verifies that the returned value matches an allowed value or is zero, and enforces restrictions on helper calls and freplace operations.
This is a pretty exciting set of changes to `kernel/bpf/btf.c`, `kernel/bpf/verifier.c`, and `kernel/bpf/syscall.c`.
Good news: this code is definitely reachable in QEMU. Loading a BPF program with `BPF_MODIFY_RETURN` and a specific `attach_btf_id` via the `BPF_PROG_LOAD` syscall exercises the verifier logic. Module loading also hits `btf_register_fmodret_tags`, and testing extension with freplace uses `bpf_check_attach_target`. WorthFuzzing is TRUE.
Now, let's nail down those config dependencies. We need `BPF_SYSCALL`, `DEBUG_INFO_BTF`, `BPF_JIT`, and most importantly, `CLANG_PLUGIN_BPF_SCAN_IR` *and* `LLVM_PLUGINS`. Wait... `FUNCTION_ERROR_INJECTION` is also relevant because `check_attach_modify_return` is under that flag. And `DEBUG_INFO_BTF_MODULES` is needed for `btf_module_notify`.
But wait, is `CLANG_PLUGIN_BPF_SCAN_IR` *strictly* required for `btf_register_fmodret_tags`? No! Looking at `net/bpf/test_run.c` and `tools/bpf/resolve_btfids/main.c`, `BTF_FMODRET_START` creates `BTF_SET8_FMODRET`, which `resolve_btfids` converts into `bpf_fmodret` and `bpf_fmodret_sleepable` tags. Those are parsed by `btf_register_fmodret_tags`. However, the `bpf_fmodret_retval:` tags – those that trigger the return value restrictions – *are* only generated with `CLANG_PLUGIN_BPF_SCAN_IR`. So, enabling `CLANG_PLUGIN_BPF_SCAN_IR` *and* `LLVM_PLUGINS` is key to getting the full functionality and testing those return value restrictions.
Let's do a quick scan of other configs: `BPF_EVENTS`, `KPROBES`, `FTRACE`... and double check what `BPF_MODIFY_RETURN` needs in `bpf_check_attach_target`.
|
| 22/2 | 2026/11/12 15:32 | tool | grepper |
0mArgs:{
"Expression": "bpf_check_attach_target",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "kernel/bpf/btf.c=7452=bool btf_ctx_access(int off, int size, enum bpf_access_type type,\n--\nkernel/bpf/btf.c-7636-\t\t\treturn false;\nkernel/bpf/btf.c:7637:\t\t/* Checked by bpf_check_attach_target */\nkernel/bpf/btf.c-7638-\t\ttname += sizeof(\"btf_trace_\") - 1;\n--\nkernel/bpf/syscall.c=3674=static int bpf_tracing_prog_attach(struct bpf_prog *prog,\n--\nkernel/bpf/syscall.c-3806-\nkernel/bpf/syscall.c:3807:\t\terr = bpf_check_attach_target(NULL, prog, tgt_prog, btf_id,\nkernel/bpf/syscall.c-3808-\t\t\t\t\t \u0026tgt_info);\n--\nkernel/bpf/trampoline.c=1176=int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog,\n--\nkernel/bpf/trampoline.c-1186-\nkernel/bpf/trampoline.c:1187:\terr = bpf_check_attach_target(NULL, prog, NULL,\nkernel/bpf/trampoline.c-1188-\t\t\t\t prog-\u003eaux-\u003eattach_btf_id,\n--\nkernel/bpf/verifier.c=15375=static int process_kf_arg_ptr_to_rbtree_node(struct bpf_verifier_env *env,\n--\nkernel/bpf/verifier.c-15386- * LSM hooks and iters (both sleepable and non-sleepable) are safe.\nkernel/bpf/verifier.c:15387: * Any sleepable progs are also safe since bpf_check_attach_target() enforce\nkernel/bpf/verifier.c-15388- * them can only be attached to some specific hook points.\n--\nkernel/bpf/verifier.c=24066=static bool func_has_rcu_read_lock(const struct bpf_prog *prog, int subprog)\n--\nkernel/bpf/verifier.c-24081- *\nkernel/bpf/verifier.c:24082: * While an extension is loaded, bpf_check_attach_target() runs before its\nkernel/bpf/verifier.c-24083- * arguments are known. check_attach_btf_id() does the check when they are.\n--\nkernel/bpf/verifier.c=24085=static int check_ext_args(struct bpf_verifier_log *log, const struct bpf_prog *prog,\n--\nkernel/bpf/verifier.c-24107-\nkernel/bpf/verifier.c:24108:int bpf_check_attach_target(struct bpf_verifier_log *log,\nkernel/bpf/verifier.c-24109-\t\t\t const struct bpf_prog *prog,\n--\nkernel/bpf/verifier.c=24602=static int check_attach_btf_id(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-24632-\nkernel/bpf/verifier.c:24633:\tret = bpf_check_attach_target(\u0026env-\u003elog, prog, tgt_prog, btf_id, \u0026tgt_info);\nkernel/bpf/verifier.c-24634-\tif (ret)\n"
} |
| 23/2 | 2026/11/12 15:32 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 84455 output: 36 thoughts: 561 Thoughts: **Analyzing BPF Selftest Configurations and Relevant Symbols** Okay, so I'm looking into these BPF selftests, and the first thing I'm doing is checking the diff to see what configs they've added. Specifically, it seems we've got `CONFIG_CLANG_PLUGIN_BPF_SCAN_IR=y` and `CONFIG_LLVM_PLUGINS=y` in `tools/testing/selftests/bpf/config`, and those are linked to `LLVM_PLUGINS` and `CLANG_PLUGIN_BPF_SCAN_IR` in `scripts/llvm-plugins/Kconfig`. Noted. Now, let's get down to identifying the key symbols. I need to make sure I'm following the guidelines, particularly Section 4. I remember that we want to *avoid* ubiquitous functions, the usual suspects like allocators or synchronization primitives, because they'll just muddy the waters. The goal is to focus on feature-specific logic and entrypoints. If there are inline functions, then I have to think about finding non-static callers in `.c` files to exercise them. So, digging into the kernel symbols, specifically in `kernel/bpf/btf.c`: I see a bunch of symbols, some static, some not. There's `btf_register_fmodret_tags`, `btf_fmodret_tags_init`, `btf_fmodret_tag_func`, and `btf_kfunc_id_set_contains` – all static. Then there's `btf_kfunc_check_flag` and `btf_kfunc_is_modify_return`, which are non-static. Ah, wait, hold on. `btf_module_notify` calls `btf_register_fmodret_tags` when a module is loaded. This is critical. That suggests a chain of interactions! Let's zero in on those non-static functions in `btf.c`. Yep, `btf_kfunc_is_modify_return` is non-static and linked into the chain from `btf_module_notify`. I need to figure out if `btf_module_notify` itself is static or not because that gives me another entrypoint. |
| 24/2 | 2026/11/12 15:33 | tool | grepper |
0mArgs:{
"Expression": "btf_module_notify",
"PathPrefixes": [
"kernel/bpf/btf.c"
]
}Results: {
"Output": "kernel/bpf/btf.c=9425=static void btf_vmlinux_inline_lazy_init(void)\n--\nkernel/bpf/btf.c-9430-\nkernel/bpf/btf.c:9431:static int btf_module_notify(struct notifier_block *nb, unsigned long op,\nkernel/bpf/btf.c-9432-\t\t\t void *module)\n--\nkernel/bpf/btf.c=9574=static struct notifier_block btf_module_nb = {\nkernel/bpf/btf.c:9575:\t.notifier_call = btf_module_notify,\nkernel/bpf/btf.c-9576-};\n"
} |
| 25/2 | 2026/11/12 15:33 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 85745 output: 37 thoughts: 207 Thoughts: **Analyzing Recent Changes in the BPF Verifier** Okay, let's dive into this. My initial focus is on `kernel/bpf/verifier.c` to understand the recent additions or modifications. I need to get a handle on what functions have been introduced or altered. I see: * `check_fmodret_retval` (static), `fmodret_retval_allowed` (static), `fmodret_retval_next` (static), `fmodret_retval_init` (static): These seem related, probably around floating-point modulo return values. Interesting. * `check_return_code` (static, called during `do_check`): Okay, part of the core verification process. Makes sense. * `check_helper_call` (static, within `verifier.c`): A function for checking calls to helper functions, a key part of BPF security. * `bpf_check_attach_target` (non-static): This one sticks out because it's *not* static, so it's likely exported or used externally. Definitely worth noting. * `check_attach_btf_id` (static): A function used to check BTF IDs. This appears to be static. I am particularly interested in that last function, `check_attach_btf_id`, that is static. I want to know who is calling `check_attach_btf_id`. Time to check the callers of `check_attach_btf_id` within the file `kernel/bpf/verifier.c`. |
| 26/2 | 2026/11/12 15:33 | tool | grepper |
0mArgs:{
"Expression": "check_attach_btf_id",
"PathPrefixes": [
"kernel/bpf/verifier.c"
]
}Results: {
"Output": "kernel/bpf/verifier.c=24066=static bool func_has_rcu_read_lock(const struct bpf_prog *prog, int subprog)\n--\nkernel/bpf/verifier.c-24082- * While an extension is loaded, bpf_check_attach_target() runs before its\nkernel/bpf/verifier.c:24083: * arguments are known. check_attach_btf_id() does the check when they are.\nkernel/bpf/verifier.c-24084- */\n--\nkernel/bpf/verifier.c=24575=static bool can_be_sleepable(struct bpf_prog *prog)\n--\nkernel/bpf/verifier.c-24601-\nkernel/bpf/verifier.c:24602:static int check_attach_btf_id(struct bpf_verifier_env *env)\nkernel/bpf/verifier.c-24603-{\n--\nkernel/bpf/verifier.c-24717-\nkernel/bpf/verifier.c:24718:int bpf_check_attach_btf_id_multi(struct btf *btf, struct bpf_prog *prog, u32 btf_id,\nkernel/bpf/verifier.c-24719-\t\t\t\t struct bpf_attach_target_info *tgt_info)\n--\nkernel/bpf/verifier.c=25452=int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,\n--\nkernel/bpf/verifier.c-25618-\nkernel/bpf/verifier.c:25619:\tret = check_attach_btf_id(env);\nkernel/bpf/verifier.c-25620-\tif (ret)\n"
} |
| 27/2 | 2026/11/12 15:33 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 86806 output: 34 thoughts: 739 Thoughts: **Analyzing BPF Verifier Entry Points and Relevant Functions** Okay, so I'm thinking about how the BPF verifier works, specifically the entry points. My initial thought was that `bpf_check` is the main one, but is that *universally* true? Let me think... Ah, right! `bpf_check_attach_target` is a non-static symbol, which is specifically used for the attach target verification, and it's directly modified, making it a key focus. Good catch! It's defined as: `int bpf_check_attach_target(struct bpf_verifier_log *log, const struct bpf_prog *prog, const struct bpf_prog *tgt_prog, u32 btf_id, struct bpf_attach_target_info *tgt_info)`. And what about those `fmodret` functions? I recall `fmodret_retval_init` and `check_fmodret_retval` being involved. Now, are those static functions in `verifier.c` actually symbols in vmlinux? Wait a minute... often, static functions *in .c files*, if not inlined, *do* have symbols in kallsyms and vmlinux! So, are those `fmodret` functions inlined? Let's take a look. `fmodret_retval_init` is reasonably complex, it's about 50 lines long, contains loops, calls `kmalloc_array`, `kstrtos64` and so on, so it's probably not inlined. `check_fmodret_retval` is called from `check_return_code`, so that's helpful. Okay, can static functions in .c files be listed as FocusSymbols if they are specific? Let me go back to the rule in Section 4. It says: "- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS: List functions that specifically implement the logic being added or altered, or direct API entrypoints for the subsystem feature under review. - HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h): Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack distinct symbol addresses in vmlinux..." Aha! It highlights that static functions in *headers* are a problem, but *not* static functions in .c files! Static functions in .c files frequently exist in the symbol table, unless the compiler has inlined them for optimization. However, the best practice is to include the direct entrypoints/non-static callers AND the functions implementing the altered logic. So, let me list the functions implementing the altered logic: * `bpf_check_attach_target` * `check_attach_btf_id` * `fmodret_retval_init` * `check_fmodret_retval` * `fmodret_retval_allowed` * `btf_register_fmodret_tags` * `btf_kfunc_is_modify_return` * `btf_kfunc_check_flag` Now, what about `bpf_tracing_prog_attach` in `kernel/bpf/syscall.c`? That seems relevant, and I should double-check whether that is static or not. |
| 28/2 | 2026/11/12 15:33 | tool | grepper |
0mArgs:{
"Expression": "bpf_tracing_prog_attach",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "kernel/bpf/syscall.c=3667=static const struct bpf_link_ops bpf_tracing_link_lops = {\n--\nkernel/bpf/syscall.c-3673-\nkernel/bpf/syscall.c:3674:static int bpf_tracing_prog_attach(struct bpf_prog *prog,\nkernel/bpf/syscall.c-3675-\t\t\t\t int tgt_prog_fd,\n--\nkernel/bpf/syscall.c=4353=static int bpf_raw_tp_link_attach(struct bpf_prog *prog,\n--\nkernel/bpf/syscall.c-4377-\t\t}\nkernel/bpf/syscall.c:4378:\t\treturn bpf_tracing_prog_attach(prog, 0, 0, 0, attach_type);\nkernel/bpf/syscall.c-4379-\tcase BPF_PROG_TYPE_RAW_TRACEPOINT:\n--\nkernel/bpf/syscall.c=5862=static int link_create(union bpf_attr *attr, bpfptr_t uattr, struct bpf_common_attr *attr_common,\n--\nkernel/bpf/syscall.c-5906-\tcase BPF_PROG_TYPE_EXT:\nkernel/bpf/syscall.c:5907:\t\tret = bpf_tracing_prog_attach(prog,\nkernel/bpf/syscall.c-5908-\t\t\t\t\t attr-\u003elink_create.target_fd,\n--\nkernel/bpf/syscall.c-5928-\t\telse\nkernel/bpf/syscall.c:5929:\t\t\tret = bpf_tracing_prog_attach(prog,\nkernel/bpf/syscall.c-5930-\t\t\t\t\t\t attr-\u003elink_create.target_fd,\n"
} |
| 29/2 | 2026/11/12 15:33 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 88760 output: 300 thoughts: 1543 Thoughts: **Analysis of fmod_ret Implementation for Fuzzing Focus** Okay, let's break this down. My goal is to determine the most effective focus symbols and configuration options for fuzzing the fmod_ret functionality in the kernel. First, I examined the call chain. `bpf_tracing_prog_attach` leads to `bpf_check_attach_target`, which then invokes `check_attach_btf_id`. Within the verifier (`kernel/bpf/verifier.c`), `check_attach_btf_id` calls `fmodret_retval_init`, and `check_return_code` calls `check_fmodret_retval`. Importantly, `bpf_check_attach_target` itself checks `tgt_prog->aux->fmodret_retvals`. This indicates the key touchpoints for this new functionality. Next, I identified the new and modified functions across `kernel/bpf/verifier.c` and `kernel/bpf/btf.c`: These included functions to handle attach targets, BTF ID checks, return value initialization, return code checking, BTF registration, and function modification checks (e.g., `btf_kfunc_is_modify_return`). I also noted the static inline `btf_id_set8_entry` in `include/linux/btf.h`. Based on this, I need to choose the right focus symbols. I carefully considered these candidates: `bpf_check_attach_target`, `check_attach_btf_id`, `fmodret_retval_init`, `check_fmodret_retval`, `btf_register_fmodret_tags`, `btf_kfunc_is_modify_return`, and `btf_kfunc_check_flag`. These are all directly related to fmod_ret and not generic kernel functions, making them appropriate for fuzzing, and not ubiquitous code paths that would provide little value. I moved on to examine required configurations. I meticulously reviewed each config, including dependencies. We need `BPF_SYSCALL` for the core BPF subsystem, including the verifier and BTF. `DEBUG_INFO_BTF` is essential for BTF generation, which is required by `bpf_check_attach_target` and `BTF_FMODRET_START`. The BPF trampoline requires `BPF_JIT`. `CLANG_PLUGIN_BPF_SCAN_IR` enables the crucial IR scanning for fmod_ret candidates and tag generation, and `LLVM_PLUGINS` is its parent. `DEBUG_INFO_BTF_MODULES` handles BTF for modules, and `FUNCTION_ERROR_INJECTION` is used to determine whether static rules or tag-based rules are applied, based on the implementation of `check_attach_modify_return`. I double-checked `CLANG_PLUGIN_BPF_SCAN_IR`'s dependencies in `scripts/llvm-plugins/Kconfig` to ensure accuracy. Finally, I confirmed that my selection aligns with the core triage philosophy: is the modified code reachable? Yes, through BPF system calls, BTF loading, and module loading, all making it reachable in QEMU. Given these factors, my selected symbols and configurations are appropriate to fuzz this specific new feature. This checks all of the core criteria. With this analysis complete, and after verifying the requirements, I am confident in setting the results. |
| 30/2 | 2026/11/12 15:34 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"BPF_SYSCALL",
"DEBUG_INFO_BTF",
"DEBUG_INFO_BTF_MODULES",
"BPF_JIT",
"LLVM_PLUGINS",
"CLANG_PLUGIN_BPF_SCAN_IR",
"FUNCTION_ERROR_INJECTION"
],
"FocusSymbols": [
"bpf_check_attach_target",
"check_attach_btf_id",
"fmodret_retval_init",
"check_fmodret_retval",
"btf_register_fmodret_tags",
"btf_kfunc_is_modify_return",
"btf_kfunc_check_flag"
],
"Reasoning": "The patch introduces support for discovering and tagging BPF modify-return (fmod_ret) target functions and enforcing allowed return values via the BPF verifier. It adds new verifier logic (check_fmodret_retval, fmodret_retval_init, and restrictions on tail calls and freplace extensions) as well as BTF registration for fmod_ret tags (btf_register_fmodret_tags). These changes modify core BPF subsystem code reachable through standard bpf() system call operations (program loading and link attachment) in standard virtualized environments.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"BPF_SYSCALL",
"DEBUG_INFO_BTF",
"DEBUG_INFO_BTF_MODULES",
"BPF_JIT",
"LLVM_PLUGINS",
"CLANG_PLUGIN_BPF_SCAN_IR",
"FUNCTION_ERROR_INJECTION"
],
"FocusSymbols": [
"bpf_check_attach_target",
"check_attach_btf_id",
"fmodret_retval_init",
"check_fmodret_retval",
"btf_register_fmodret_tags",
"btf_kfunc_is_modify_return",
"btf_kfunc_check_flag"
],
"Reasoning": "The patch introduces support for discovering and tagging BPF modify-return (fmod_ret) target functions and enforcing allowed return values via the BPF verifier. It adds new verifier logic (check_fmodret_retval, fmodret_retval_init, and restrictions on tail calls and freplace extensions) as well as BTF registration for fmod_ret tags (btf_register_fmodret_tags). These changes modify core BPF subsystem code reachable through standard bpf() system call operations (program loading and link attachment) in standard virtualized environments.",
"WorthFuzzing": true
} |
| 31/1 | 2026/11/12 15:34 | action | If |
2mArgs:{
"WorthFuzzing": true
}Results: null |
| 32/2 | 2026/11/12 15:34 | agent | kmsan-evaluator |
2mModel:core Args: null Results: {
"KMSANReasoning": "The patch series introduces build-time and verification mechanisms for BPF fmod_ret targets. Specifically, it adds an LLVM pass plugin to scan IR for clean error-returning functions, records metadata in ELF sections, annotates BTF types with declaration tags in resolve_btfids, and updates the kernel BPF verifier and BTF subsystem to restrict fmod_ret return values based on these tags.\n\nFrom a memory sanitization perspective:\n1. No uninitialized memory usage:\n - In `kernel/bpf/btf.c`, `btf_register_fmodret_tags()` allocates `struct btf_id_set8` using `kmalloc()` and explicitly initializes `cnt`, `flags`, and all populated `pairs[]` elements (`id` and `flags`).\n - In `kernel/bpf/verifier.c`, `fmodret_retval_init()` parses tag values and allocates `retvals` via `kmalloc_array()`, writing every element sequentially up to `cnt`.\n - `prog-\u003eaux` is allocated using `kzalloc_obj()`, ensuring that `fmodret_retvals` and `fmodret_retval_cnt` are zero-initialized by default.\n - Stack buffers (e.g. `char buf[24]` in `fmodret_retval_next()`) are fully initialized (`memcpy` followed by explicit null termination) before being passed to `kstrtos64()`.\n2. No kernel-to-user info-leaks:\n - The newly introduced structures (`struct btf_id_set8`, `aux-\u003efmodret_retvals`) are kernel-internal tracking structures and are never copied to userspace via `copy_to_user`, `put_user`, socket options, or BPF syscall inspection interfaces.\n3. KASAN vs KMSAN applicability:\n - Potential risks in this series pertain to bounds checking, string parsing/tokenization, or dynamic array management. Any out-of-bounds access or use-after-free bugs in these paths are already covered by KASAN.\n - There are no uninitialized memory reads, uninitialized padding leaks, or uninitialized branch conditions that would uniquely require KMSAN.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit b2dd76da41c0d4300e9a8c4bb643fc7c656fa2c8
Author: syz-cluster <triage@syzkaller.com>
Date: Thu Nov 12 15:30:31 2026 +0000
syz-cluster: applied patch under review
diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065de..29940e8d7205b7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5064,6 +5064,14 @@ S: Supported
F: arch/x86/net/
X: arch/x86/net/bpf_jit_comp32.c
+BPF [COMPILER PLUGINS]
+M: Anton Protopopov <a.s.protopopov@gmail.com>
+L: bpf@vger.kernel.org
+S: Maintained
+F: scripts/bpf/
+F: scripts/llvm-plugins/bpf_*
+F: tools/bpf/resolve_btfids/fmodret.*
+
BPF [CORE]
M: Alexei Starovoitov <ast@kernel.org>
M: Daniel Borkmann <daniel@iogearbox.net>
@@ -6562,7 +6570,9 @@ C: irc://irc.libera.chat/clangbuiltlinux
F: Documentation/kbuild/llvm.rst
F: include/linux/compiler-clang.h
F: scripts/Makefile.clang
+F: scripts/Makefile.llvm-plugins
F: scripts/clang-tools/
+F: scripts/llvm-plugins/
K: \b(?i:clang|llvm)\b
CLK API
diff --git a/Makefile b/Makefile
index f895752d4b05fa..2fcad810957218 100644
--- a/Makefile
+++ b/Makefile
@@ -457,10 +457,13 @@ endif
HOSTCC = $(LLVM_PREFIX)clang$(LLVM_SUFFIX)
HOSTCXX = $(LLVM_PREFIX)clang++$(LLVM_SUFFIX)
+LLVM_CONFIG ?= $(LLVM_PREFIX)llvm-config$(LLVM_SUFFIX)
else
HOSTCC = gcc
HOSTCXX = g++
+LLVM_CONFIG ?= llvm-config
endif
+export LLVM_CONFIG
HOSTRUSTC = rustc
HOSTPKG_CONFIG = pkg-config
@@ -1211,10 +1214,11 @@ include-$(CONFIG_AUTOFDO_CLANG) += scripts/Makefile.autofdo
include-$(CONFIG_PROPELLER_CLANG) += scripts/Makefile.propeller
include-$(CONFIG_WARN_CONTEXT_ANALYSIS) += scripts/Makefile.context-analysis
include-$(CONFIG_GCC_PLUGINS) += scripts/Makefile.gcc-plugins
+include-$(CONFIG_LLVM_PLUGINS) += scripts/Makefile.llvm-plugins
include $(addprefix $(srctree)/, $(include-y))
-# scripts/Makefile.gcc-plugins is intentionally included last.
+# The compiler plugin Makefiles are intentionally included last.
# Do not add $(call cc-option,...) below this line. When you build the kernel
# from the clean source tree, the GCC plugins do not exist at this point.
diff --git a/arch/Kconfig b/arch/Kconfig
index 1163736de1f996..4f1f36d3e82d8e 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1772,6 +1772,7 @@ config HAVE_GENERIC_TIF_BITS
source "kernel/gcov/Kconfig"
source "scripts/gcc-plugins/Kconfig"
+source "scripts/llvm-plugins/Kconfig"
config FUNCTION_ALIGNMENT_4B
bool
diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile
index a273ab5af805c6..5813bb75c99251 100644
--- a/arch/arm64/kernel/pi/Makefile
+++ b/arch/arm64/kernel/pi/Makefile
@@ -10,6 +10,7 @@ KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
-D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \
-fno-asynchronous-unwind-tables -fno-unwind-tables \
$(CONFIG_CC_OPT_NO_ADDRSIG)
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# this code may run with the MMU off so disable unaligned accesses
CFLAGS_map_range.o += -mstrict-align
diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile
index 5e499cfb29b5c0..7036e939097e5d 100644
--- a/arch/x86/boot/startup/Makefile
+++ b/arch/x86/boot/startup/Makefile
@@ -12,6 +12,8 @@ KBUILD_CFLAGS += -D__DISABLE_EXPORTS -mcmodel=small -fPIC \
# disable ftrace hooks and LTO
KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS))
KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS))
+# LLVM plugins may add data with absolute addresses
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
KMSAN_SANITIZE := n
diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
index 7664cbedbe30fb..0e32c9ae2b5afc 100644
--- a/arch/x86/um/vdso/Makefile
+++ b/arch/x86/um/vdso/Makefile
@@ -37,7 +37,7 @@ CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \
$(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \
-fno-omit-frame-pointer -foptimize-sibling-calls
-$(vobjs): KBUILD_CFLAGS += $(CFL)
+$(vobjs): KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)
#
# vDSO code runs in userspace and -pg doesn't help with profiling anyway.
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index bbab12b8eb4b23..6f69d5d6773141 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -6,7 +6,7 @@
# enabled, even if doing so doesn't break the build.
#
-cflags-y := $(KBUILD_CFLAGS)
+cflags-y := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# x86 links the stub into the decompressor rather than the kernel proper, so
# drop the kernel's code model, calling convention, and stack alignment, and
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 3eb732942d699c..77e86ff2f9a44d 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -198,7 +198,9 @@
#define MCOUNT_REC() . = ALIGN(8); \
__start_mcount_loc = .; \
KEEP(*(__mcount_loc)) \
+ __start_patchable_function_entries = .; \
KEEP_PATCHABLE \
+ __stop_patchable_function_entries = .; \
__stop_mcount_loc = .; \
FTRACE_STUB_HACK \
ftrace_ops_list_func = arch_ftrace_ops_list_func;
@@ -714,7 +716,7 @@
BTF_INLINE_LINK \
. = ALIGN(PAGE_SIZE); \
.BTF_ids : AT(ADDR(.BTF_ids) - LOAD_OFFSET) { \
- *(.BTF_ids) \
+ KEEP(*(.BTF_ids)) \
}
#else
#define BTF
@@ -886,7 +888,8 @@
KLP_SYMID
#define MODINFO \
- .modinfo (INFO) : { KEEP(*(.modinfo)) }
+ .modinfo (INFO) : { KEEP(*(.modinfo)) } \
+ .BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }
#ifdef CONFIG_GENERIC_BUG
#define BUG_TABLE \
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 520fd72e21fd07..56a5e4b9f138a6 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1969,6 +1969,12 @@ struct bpf_prog_aux {
const struct btf_type *attach_func_proto;
/* function name for valid attach_btf_id */
const char *attach_func_name;
+ /*
+ * The values which an fmod_ret program may return besides 0, from the
+ * "bpf_fmodret_retval:" tag of its target, or NULL
+ */
+ s64 *fmodret_retvals;
+ u32 fmodret_retval_cnt;
struct bpf_prog **func;
struct bpf_prog_aux *main_prog_aux;
/* main prog: there when the prog has timed may_goto */
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 87fad4076f0e9d..5522c7fe13735a 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -558,11 +558,17 @@ static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id)
return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL;
}
-static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
+static inline struct btf_id_set8_pair *
+btf_id_set8_entry(const struct btf_id_set8 *set, u32 id)
{
return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func);
}
+static inline bool btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
+{
+ return btf_id_set8_entry(set, id) != NULL;
+}
+
bool btf_param_match_suffix(const struct btf *btf,
const struct btf_param *arg,
const char *suffix);
@@ -626,7 +632,6 @@ u32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,
const struct bpf_prog *prog);
int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
const struct btf_kfunc_id_set *s);
-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset);
s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id);
int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt,
struct module *owner);
diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
index 8b5a9ee92513d4..f5985acf661b1c 100644
--- a/include/linux/btf_ids.h
+++ b/include/linux/btf_ids.h
@@ -13,13 +13,21 @@ struct btf_id_set {
/* This flag implies BTF_SET8 holds kfunc(s) */
#define BTF_SET8_KFUNCS (1 << 0)
+/*
+ * This flag implies BTF_SET8 holds functions which BPF fmod_ret, fentry and
+ * fexit programs may attach to. Per-function flags: KF_SLEEPABLE only.
+ */
+#define BTF_SET8_FMODRET (1 << 1)
+
+struct btf_id_set8_pair {
+ u32 id;
+ u32 flags;
+};
+
struct btf_id_set8 {
u32 cnt;
u32 flags;
- struct {
- u32 id;
- u32 flags;
- } pairs[];
+ struct btf_id_set8_pair pairs[];
};
#ifdef CONFIG_DEBUG_INFO_BTF
@@ -215,6 +223,12 @@ __BTF_SET8_START(name, local, BTF_SET8_KFUNCS)
#define BTF_KFUNCS_END(name) \
BTF_SET8_END(name)
+#define BTF_FMODRET_START(name) \
+__BTF_SET8_START(name, local, BTF_SET8_FMODRET)
+
+#define BTF_FMODRET_END(name) \
+BTF_SET8_END(name)
+
#else
#define BTF_ID_LIST(name) static u32 __maybe_unused name[128];
@@ -231,6 +245,8 @@ BTF_SET8_END(name)
#define BTF_SET8_END(name)
#define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };
#define BTF_KFUNCS_END(name)
+#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };
+#define BTF_FMODRET_END(name)
#endif /* CONFIG_DEBUG_INFO_BTF */
diff --git a/include/linux/compiler-version.h b/include/linux/compiler-version.h
index ac1665a98a15db..e6a5a9f3fbcd33 100644
--- a/include/linux/compiler-version.h
+++ b/include/linux/compiler-version.h
@@ -24,6 +24,11 @@
#include <generated/gcc-plugins.h>
#endif
+/* Rebuild objects when an LLVM plugin changes. */
+#ifdef LLVM_PLUGINS
+#include <generated/llvm-plugins.h>
+#endif
+
/*
* If the randstruct seed itself changes (whether for GCC plugins or
* Clang), the entire tree needs to be rebuilt since the randomization of
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 228b93d4f8c940..f81a8ae9b92c32 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -9307,6 +9307,8 @@ enum {
};
#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+static int btf_register_fmodret_tags(struct btf *btf);
+
struct btf_module {
struct list_head list;
struct module *module;
@@ -9476,6 +9478,15 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
}
goto out;
}
+ /*
+ * Do this before the BTF is visible to anyone else. A failure
+ * is not fatal: the module loads, but without fmod_ret targets.
+ */
+ err = btf_register_fmodret_tags(btf);
+ if (err)
+ pr_warn("failed to register fmod_ret targets of module [%s]: %d\n",
+ mod->name, err);
+
err = btf_alloc_id(btf);
if (err) {
btf_free(btf);
@@ -9973,8 +9984,8 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,
enum btf_kfunc_hook hook,
u32 kfunc_btf_id)
{
+ struct btf_id_set8_pair *pair;
struct btf_id_set8 *set;
- u32 *id;
if (hook >= BTF_KFUNC_HOOK_MAX)
return NULL;
@@ -9983,11 +9994,10 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,
set = btf->kfunc_set_tab->sets[hook];
if (!set)
return NULL;
- id = btf_id_set8_contains(set, kfunc_btf_id);
- if (!id)
+ pair = btf_id_set8_entry(set, kfunc_btf_id);
+ if (!pair)
return NULL;
- /* The flags for BTF ID are located next to it */
- return id + 1;
+ return &pair->flags;
}
static bool __btf_kfunc_is_allowed(const struct btf *btf,
@@ -10101,7 +10111,9 @@ u32 *btf_kfunc_flags(const struct btf *btf, u32 kfunc_btf_id, const struct bpf_p
}
/*
- * Check a single KF_* @flag on a kfunc across all of its hook sets.
+ * Check a single KF_* @flag on a kfunc across all of its hook sets. The set of
+ * fmod_ret targets is not one of them: its flags only say how a target may be
+ * attached, and a kfunc which is also a target keeps its own flags.
* Returns:
* * 1 if @flag is set
* * 0 if @flag is not set
@@ -10116,6 +10128,8 @@ int btf_kfunc_check_flag(const struct btf *btf, u32 kfunc_btf_id, u32 flag)
u32 *flags;
for (hook = 0; hook < BTF_KFUNC_HOOK_MAX; hook++) {
+ if (hook == BTF_KFUNC_HOOK_FMODRET)
+ continue;
flags = btf_kfunc_id_set_contains(btf, hook, kfunc_btf_id);
if (!flags)
continue;
@@ -10183,12 +10197,102 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
}
EXPORT_SYMBOL_GPL(register_btf_kfunc_id_set);
-/* This function must be invoked only from initcalls/module init functions */
-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset)
+/*
+ * Targets of fmod_ret programs are marked in BTF by resolve_btfids, see
+ * BTF_SET8_FMODRET. A "bpf_fmodret" decl tag on a function makes it a target.
+ * A target which sleepable programs may attach to also has a
+ * "bpf_fmodret_sleepable" tag, which never comes without "bpf_fmodret".
+ */
+#define BTF_FMODRET_TAG "bpf_fmodret"
+#define BTF_FMODRET_SLEEPABLE_TAG "bpf_fmodret_sleepable"
+
+/* Return the function which T tags with decl tag NAME, or 0 */
+static u32 btf_fmodret_tag_func(const struct btf *btf, const struct btf_type *t,
+ const char *name)
{
- return __register_btf_kfunc_id_set(BTF_KFUNC_HOOK_FMODRET, kset);
+ const struct btf_type *func;
+
+ if (!btf_type_is_decl_tag(t) || btf_type_decl_tag(t)->component_idx != -1)
+ return 0;
+ if (strcmp(btf_name_by_offset(btf, t->name_off), name))
+ return 0;
+ func = btf_type_by_id(btf, t->type);
+ if (!func || !btf_type_is_func(func))
+ return 0;
+ return t->type;
+}
+
+/* Build the set of fmod_ret targets of BTF from its decl tags */
+static int btf_register_fmodret_tags(struct btf *btf)
+{
+ u32 id, func_id, nr_types = btf_nr_types(btf), cnt = 0;
+ struct btf_kfunc_set_tab *tab;
+ struct btf_id_set8 *set;
+
+ for (id = btf_start_id(btf); id < nr_types; id++)
+ cnt += !!btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);
+ if (!cnt)
+ return 0;
+
+ set = kmalloc(struct_size(set, pairs, cnt), GFP_KERNEL | __GFP_NOWARN);
+ if (!set)
+ return -ENOMEM;
+ set->flags = 0;
+ set->cnt = 0;
+ for (id = btf_start_id(btf); id < nr_types; id++) {
+ func_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);
+ if (!func_id)
+ continue;
+ set->pairs[set->cnt].id = func_id;
+ set->pairs[set->cnt].flags = 0;
+ set->cnt++;
+ }
+ sort(set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func, NULL);
+
+ /* Each sleepable tag marks a function which is already in the set */
+ for (id = btf_start_id(btf); id < nr_types; id++) {
+ struct btf_id_set8_pair *pair;
+
+ func_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id),
+ BTF_FMODRET_SLEEPABLE_TAG);
+ if (!func_id)
+ continue;
+ pair = btf_id_set8_entry(set, func_id);
+ if (pair)
+ pair->flags |= KF_SLEEPABLE;
+ }
+
+ tab = btf->kfunc_set_tab;
+ if (!tab) {
+ tab = kzalloc_obj(*tab, GFP_KERNEL | __GFP_NOWARN);
+ if (!tab) {
+ kfree(set);
+ return -ENOMEM;
+ }
+ btf->kfunc_set_tab = tab;
+ }
+ if (tab->sets[BTF_KFUNC_HOOK_FMODRET]) {
+ kfree(set);
+ return -EEXIST;
+ }
+ tab->sets[BTF_KFUNC_HOOK_FMODRET] = set;
+ return 0;
+}
+
+static int __init btf_fmodret_tags_init(void)
+{
+ struct btf *btf;
+ int err;
+
+ btf = btf_get_module_btf(NULL);
+ if (IS_ERR_OR_NULL(btf))
+ return PTR_ERR_OR_ZERO(btf);
+
+ err = btf_register_fmodret_tags(btf);
+ btf_put(btf);
+ return err;
}
-EXPORT_SYMBOL_GPL(register_btf_fmodret_id_set);
+fs_initcall(btf_fmodret_tags_init);
s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id)
{
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 9004f2673ccf96..0a78f66a1622c2 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2490,6 +2490,7 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)
module_put(prog->aux->mod);
kfree(prog->aux->kfunc_tab);
kfree(prog->aux->ctx_arg_info);
+ kfree(prog->aux->fmodret_retvals);
if (prog->aux->attach_btf)
btf_put(prog->aux->attach_btf);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 20b1b9ff574c7d..5045d9fa68b928 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -13561,6 +13561,14 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
}
if (func_id == BPF_FUNC_tail_call) {
+ /*
+ * The program which a tail call jumps to returns for this one,
+ * and a program array does not check the values it may return.
+ */
+ if (env->prog->aux->fmodret_retvals) {
+ verbose(env, "tail calls are not allowed in fmod_ret programs with restricted return values\n");
+ return -EINVAL;
+ }
if (env->cur_state->curframe) {
struct bpf_verifier_state *branch;
@@ -20260,6 +20268,67 @@ static bool program_returns_void(struct bpf_verifier_env *env)
return false;
}
+/*
+ * A function which the build found by analysing its code has a
+ * "bpf_fmodret_retval:" decl tag with the values it returns on paths without
+ * side effects, for example "bpf_fmodret_retval:-22,-12". See the comment at
+ * the top of tools/bpf/resolve_btfids/main.c. An fmod_ret program attached to
+ * it may return only these values, or 0 to call the function.
+ */
+#define BTF_FMODRET_RETVAL_TAG "bpf_fmodret_retval:"
+
+/* Parse the next value of the list at *LIST and move *LIST past it */
+static int fmodret_retval_next(const char **list, s64 *value)
+{
+ const char *end = strchrnul(*list, ',');
+ size_t len = end - *list;
+ char buf[24];
+
+ if (!len || len >= sizeof(buf) || (*end && !end[1]))
+ return -EINVAL;
+ memcpy(buf, *list, len);
+ buf[len] = 0;
+ *list = *end ? end + 1 : end;
+ return kstrtos64(buf, 10, value);
+}
+
+static bool fmodret_retval_allowed(const struct bpf_prog_aux *aux, u64 value)
+{
+ const struct btf_type *t;
+ u32 i;
+
+ /* "w0 = -22" zero-extends, the caller of an int function sees -22 */
+ t = btf_type_skip_modifiers(aux->attach_btf, aux->attach_func_proto->type, NULL);
+ if (t->size == 4 && !upper_32_bits(value))
+ value = (s32)value;
+ if (!value)
+ return true;
+ for (i = 0; i < aux->fmodret_retval_cnt; i++) {
+ if (aux->fmodret_retvals[i] == (s64)value)
+ return true;
+ }
+ return false;
+}
+
+/* The program must return a constant: 0 or one of the values of the tag */
+static int check_fmodret_retval(struct bpf_verifier_env *env, int regno,
+ const char *exit_ctx, const char *reg_name)
+{
+ const struct bpf_prog_aux *aux = env->prog->aux;
+ struct bpf_reg_state *reg = reg_state(env, regno);
+ u32 i;
+
+ if (reg->type == SCALAR_VALUE && tnum_is_const(reg->var_off) &&
+ fmodret_retval_allowed(aux, reg->var_off.value))
+ return 0;
+
+ verbose(env, "%s the register %s must be a constant, it should be 0 or one of", exit_ctx, reg_name);
+ for (i = 0; i < aux->fmodret_retval_cnt; i++)
+ verbose(env, "%s%lld", i ? "," : " ", aux->fmodret_retvals[i]);
+ verbose(env, "\n");
+ return -EINVAL;
+}
+
static int check_return_code(struct bpf_verifier_env *env, int regno, const char *reg_name)
{
const char *exit_ctx = "At program exit";
@@ -20316,6 +20385,9 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
if (prog_type == BPF_PROG_TYPE_CGROUP_SKB && (env->prog->expected_attach_type == BPF_CGROUP_INET_EGRESS))
enforce_attach_type_range = tnum_range(2, 3);
+ if (env->prog->aux->fmodret_retvals)
+ return check_fmodret_retval(env, regno, exit_ctx, reg_name);
+
if (!return_retval_range(env, &range))
return 0;
@@ -23759,6 +23831,61 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
#endif /* CONFIG_FUNCTION_ERROR_INJECTION */
+/*
+ * Programs attached to a function found by the analysis may return only the
+ * values of its "bpf_fmodret_retval:" tag. A function which the static rules
+ * allow stays unrestricted.
+ */
+static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,
+ unsigned long addr, const char *tname)
+{
+ struct bpf_prog_aux *aux = env->prog->aux;
+ const struct btf *btf = aux->attach_btf;
+ const struct btf_type *t = btf_type_by_id(btf, btf_id);
+ s64 *retvals = NULL;
+ const char *values, *list;
+ u32 cnt, size = 0;
+ s64 v;
+
+ if (!check_attach_modify_return(addr, tname))
+ return 0;
+ values = btf_find_decl_tag_value(btf, t, -1, BTF_FMODRET_RETVAL_TAG);
+ if (IS_ERR(values))
+ return PTR_ERR(values) == -ENOENT ? 0 : PTR_ERR(values);
+
+ t = btf_type_by_id(btf, t->type);
+ t = t ? btf_type_skip_modifiers(btf, t->type, NULL) : NULL;
+ if (t && (btf_type_is_int(t) || btf_is_any_enum(t)))
+ size = t->size;
+ if (size != 4 && size != 8)
+ goto invalid;
+
+ for (list = values, cnt = 1; *list; list++)
+ cnt += *list == ',';
+ retvals = kmalloc_array(cnt, sizeof(*retvals), GFP_KERNEL_ACCOUNT);
+ if (!retvals)
+ return -ENOMEM;
+ for (list = values, cnt = 0; *list;) {
+ if (fmodret_retval_next(&list, &v) || v >= 0 || (size == 4 && v < S32_MIN))
+ goto invalid;
+ retvals[cnt++] = v;
+ }
+ if (!cnt)
+ goto invalid;
+
+ aux->fmodret_retvals = retvals;
+ aux->fmodret_retval_cnt = cnt;
+ /* The interpreter returns only 32 bits */
+ if (size == 8)
+ env->prog->jit_required = 1;
+ return 0;
+
+invalid:
+ kfree(retvals);
+ verbose(env, "invalid %s%s tag of %s()\n", BTF_FMODRET_RETVAL_TAG, values, tname);
+ return -EINVAL;
+}
+
static bool is_tracing_multi_id(const struct bpf_prog *prog, u32 btf_id)
{
return is_tracing_multi(prog->expected_attach_type) && bpf_multi_func_btf_id[0] == btf_id;
@@ -24173,6 +24300,15 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
bpf_log(log, "Cannot extend fentry/fexit/fsession\n");
return -EINVAL;
}
+ /*
+ * An extension could return any value for an fmod_ret program
+ * whose return values are restricted. This is also checked when
+ * an extension is attached to another program than at load.
+ */
+ if (prog_extension && tgt_prog->aux->fmodret_retvals) {
+ bpf_log(log, "Cannot extend an fmod_ret program with restricted return values\n");
+ return -EINVAL;
+ }
} else {
if (prog_extension) {
bpf_log(log, "Cannot replace kernel functions\n");
@@ -24552,6 +24688,13 @@ static int check_attach_btf_id(struct bpf_verifier_env *env)
return -EINVAL;
}
+ /* Only a kernel function has a "bpf_fmodret_retval:" tag */
+ if (prog->expected_attach_type == BPF_MODIFY_RETURN && !tgt_prog) {
+ ret = fmodret_retval_init(env, btf_id, tgt_info.tgt_addr, tgt_info.tgt_name);
+ if (ret)
+ return ret;
+ }
+
/*
* We don't get trampoline for tracing_multi programs at this point,
* it's done when tracing_multi link is created.
diff --git a/lib/vdso/Makefile.include b/lib/vdso/Makefile.include
index cedbf15f80874d..848d58296ba231 100644
--- a/lib/vdso/Makefile.include
+++ b/lib/vdso/Makefile.include
@@ -1,5 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
+# vDSO code executes outside the kernel.
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
+
GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index d6c9dc62182ee4..d63c3b6975527e 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -643,17 +643,12 @@ CFI_NOSEAL(bpf_kfunc_call_memb_release_dtor);
__bpf_kfunc_end_defs();
-BTF_KFUNCS_START(bpf_test_modify_return_ids)
+BTF_FMODRET_START(bpf_test_modify_return_ids)
BTF_ID_FLAGS(func, bpf_modify_return_test)
BTF_ID_FLAGS(func, bpf_modify_return_test2)
BTF_ID_FLAGS(func, bpf_modify_return_test_tp)
BTF_ID_FLAGS(func, bpf_fentry_test1, KF_SLEEPABLE)
-BTF_KFUNCS_END(bpf_test_modify_return_ids)
-
-static const struct btf_kfunc_id_set bpf_test_modify_return_set = {
- .owner = THIS_MODULE,
- .set = &bpf_test_modify_return_ids,
-};
+BTF_FMODRET_END(bpf_test_modify_return_ids)
BTF_KFUNCS_START(test_sk_check_kfunc_ids)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_release, KF_RELEASE)
@@ -1905,8 +1900,7 @@ static int __init bpf_prog_test_run_init(void)
};
int ret;
- ret = register_btf_fmodret_id_set(&bpf_test_modify_return_set);
- ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set);
+ ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_prog_test_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_prog_test_kfunc_set);
return ret ?: register_btf_id_dtor_kfuncs(bpf_prog_test_dtor_kfunc,
diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index 4cc16cbeb32817..d0d489624689a4 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -20,17 +20,6 @@ struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)
return NULL;
}
-BTF_SET8_START(bpf_mptcp_fmodret_ids)
+BTF_FMODRET_START(bpf_mptcp_fmodret_ids)
BTF_ID_FLAGS(func, update_socket_protocol)
-BTF_SET8_END(bpf_mptcp_fmodret_ids)
-
-static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = {
- .owner = THIS_MODULE,
- .set = &bpf_mptcp_fmodret_ids,
-};
-
-static int __init bpf_mptcp_kfunc_init(void)
-{
- return register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set);
-}
-late_initcall(bpf_mptcp_kfunc_init);
+BTF_FMODRET_END(bpf_mptcp_fmodret_ids)
diff --git a/scripts/Makefile b/scripts/Makefile
index d7fbb7ca098b18..bc5dacc29207b2 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -66,6 +66,7 @@ hostprogs += unifdef gen_packed_field_checks
targets += module.lds
subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins
+subdir-$(CONFIG_LLVM_PLUGINS) += llvm-plugins
subdir-$(CONFIG_GENKSYMS) += genksyms
subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms
subdir-$(CONFIG_SECURITY_SELINUX) += selinux
diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
index 98012344a81ec3..3aa3d084cc6c08 100644
--- a/scripts/Makefile.btf
+++ b/scripts/Makefile.btf
@@ -35,5 +35,6 @@ resolve-btfids-flags-y :=
resolve-btfids-flags-$(CONFIG_WERROR) += --fatal_warnings
resolve-btfids-flags-$(if $(KBUILD_EXTMOD),y) += --distill_base
resolve-btfids-flags-$(if $(KBUILD_VERBOSE),y) += --verbose
+resolve-btfids-flags-$(CONFIG_FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY) += --patchable_ftrace
export RESOLVE_BTFIDS_FLAGS := $(resolve-btfids-flags-y)
diff --git a/scripts/Makefile.llvm-plugins b/scripts/Makefile.llvm-plugins
new file mode 100644
index 00000000000000..9ff364f4db5957
--- /dev/null
+++ b/scripts/Makefile.llvm-plugins
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+llvm-plugin-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_scan_ir.so
+
+LLVM_PLUGIN := $(llvm-plugin-y)
+LLVM_PLUGINS_CFLAGS := $(strip $(addprefix -fpass-plugin=$(objtree)/scripts/llvm-plugins/, \
+ $(LLVM_PLUGIN)) -DLLVM_PLUGINS)
+export LLVM_PLUGIN LLVM_PLUGINS_CFLAGS
+
+KBUILD_CFLAGS += $(LLVM_PLUGINS_CFLAGS)
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 01a37ec872b905..042c31e3bdfb78 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -42,6 +42,7 @@ quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ ! -f $(objtree)/vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
+ $(OBJCOPY) --remove-section=.BTF_fmodret_candidates $@; \
else \
$(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
fi;
diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
index 95b523fe4f9eeb..c8651454c3d366 100644
--- a/scripts/Makefile.vmlinux
+++ b/scripts/Makefile.vmlinux
@@ -73,6 +73,9 @@ vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD
ifdef CONFIG_DEBUG_INFO_BTF
vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh
endif
+ifdef CONFIG_CLANG_PLUGIN_BPF_SCAN_IR
+vmlinux.unstripped: $(srctree)/scripts/bpf/fmodret.lds
+endif
ifdef CONFIG_BUILDTIME_TABLE_SORT
vmlinux.unstripped: scripts/sorttable
diff --git a/scripts/bpf/fmodret.lds b/scripts/bpf/fmodret.lds
new file mode 100644
index 00000000000000..6bdfde9e767979
--- /dev/null
+++ b/scripts/bpf/fmodret.lds
@@ -0,0 +1,6 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+SECTIONS {
+ /DISCARD/ : {
+ *(.BTF_fmodret_candidates)
+ }
+}
diff --git a/scripts/bpf/fmodret_metadata.h b/scripts/bpf/fmodret_metadata.h
new file mode 100644
index 00000000000000..209f266f89a375
--- /dev/null
+++ b/scripts/bpf/fmodret_metadata.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef FMODRET_METADATA_H
+#define FMODRET_METADATA_H
+
+/*
+ * The records which the BPF compiler plugins write to .BTF_fmodret_candidates
+ * for resolve_btfids, one for each function with values and for each weak
+ * definition. If a function has two weak definitions, the record of the one
+ * which loses the link would describe the body of the other, so both need a
+ * record to make the function ambiguous. A record is packed, target-endian and
+ * aligned to the target pointer size:
+ *
+ * u32 magic, version, size, flags;
+ * ptr function;
+ * u32 nr_values, reserved;
+ * s64 values[nr_values];
+ *
+ * The values are the errors which clean paths of the function return, sorted
+ * and unique.
+ */
+#define FMODRET_META_MAGIC 0x46524d32U
+#define FMODRET_META_VERSION 1U
+#define FMODRET_META_HEADER_SIZE(ptr_size) (24 + (ptr_size))
+#define FMODRET_META_VALUE_SIZE 8
+
+#define FMODRET_FN_WEAK (1U << 0)
+#define FMODRET_FN_RET64 (1U << 1)
+
+#endif
diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
index b17f1189298a55..5bca864631e0b2 100755
--- a/scripts/gen-btf.sh
+++ b/scripts/gen-btf.sh
@@ -120,7 +120,9 @@ gen_btf_o()
embed_btf_data()
{
- ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF ${ELF_FILE}
+ # resolve_btfids has consumed the compiler plugin records of the module
+ ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \
+ --remove-section=.BTF_fmodret_candidates ${ELF_FILE}
# a module might not have a .BTF_ids or .BTF.base section
btf_base="${ELF_FILE}.BTF.base"
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d5710e900bd408..f39e2d1f1400a2 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -89,6 +89,13 @@ vmlinux_link()
ldlibs=
fi
+ if is_enabled CONFIG_CLANG_PLUGIN_BPF_SCAN_IR; then
+ if [ "${output}" = ".tmp_vmlinux1" ]; then
+ ldflags="${ldflags} ${wl}--emit-relocs"
+ else
+ ldflags="${ldflags} ${wl}--script=${srctree}/scripts/bpf/fmodret.lds"
+ fi
+ fi
ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}"
# The kallsyms linking does not need debug symbols included.
diff --git a/scripts/llvm-plugins/Kconfig b/scripts/llvm-plugins/Kconfig
new file mode 100644
index 00000000000000..c4d62ae5f65d25
--- /dev/null
+++ b/scripts/llvm-plugins/Kconfig
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+menuconfig LLVM_PLUGINS
+ bool "LLVM plugins"
+ depends on CC_IS_CLANG
+ depends on $(success,$(LLVM_CONFIG) --version)
+ # The plugins are built against the headers of llvm-config and loaded
+ # by clang, so both must come from the same LLVM release
+ depends on $(success,test "$(shell,$(LLVM_CONFIG) --version 2>/dev/null | cut -d. -f1)" = "$(shell,$(CC) -dumpversion | cut -d. -f1)")
+ # The host C++ compiler must accept the LLVM headers and flags
+ depends on $(success,echo '#include <llvm/Config/llvm-config.h>' | $(HOSTCXX) $(shell,$(LLVM_CONFIG) --cxxflags 2>/dev/null) -x c++ -fsyntax-only -)
+ default y
+ help
+ LLVM plugins add compiler passes to kernel and module builds.
+ Building them requires LLVM development headers and llvm-config
+ of the same major version as clang. Each plugin has its own option
+ below and is built only if that option is enabled.
+
+if LLVM_PLUGINS
+
+config CLANG_PLUGIN_BPF_SCAN_IR
+ bool "Scan optimized LLVM IR for BPF"
+ depends on X86_64 || ARM64
+ depends on DEBUG_INFO_BTF
+ depends on !LTO_CLANG
+ depends on CLANG_VERSION >= 190000
+ help
+ Find the functions which return errors on paths without side
+ effects and describe them in .BTF_fmodret_candidates for
+ resolve_btfids. The section is discarded from the final kernel
+ and modules.
+
+endif
diff --git a/scripts/llvm-plugins/Makefile b/scripts/llvm-plugins/Makefile
new file mode 100644
index 00000000000000..a1f8503c8bcd12
--- /dev/null
+++ b/scripts/llvm-plugins/Makefile
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+always-y += $(LLVM_PLUGIN)
+targets += $(LLVM_PLUGIN)
+clean-files += *.so
+
+# Run llvm-config only to build a plugin, not when this Makefile is parsed
+llvm_cxxflags = $(shell $(LLVM_CONFIG) --cxxflags)
+llvm_ldflags = $(shell $(LLVM_CONFIG) --ldflags)
+
+quiet_cmd_llvm_plugin = HOSTCXX $@
+ cmd_llvm_plugin = $(HOSTCXX) -Wp,-MD,$(depfile) $(KBUILD_HOSTCXXFLAGS) \
+ $(llvm_cxxflags) -fPIC -shared $(KBUILD_HOSTLDFLAGS) -o $@ $< \
+ $(llvm_ldflags)
+
+$(obj)/%.so: $(src)/%.cpp FORCE
+ $(call if_changed_dep,llvm_plugin)
+
+$(obj)/../../include/generated/llvm-plugins.h: $(addprefix $(obj)/, $(LLVM_PLUGIN)) FORCE
+ $(call if_changed,touch)
+always-y += ../../include/generated/llvm-plugins.h
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
new file mode 100644
index 00000000000000..a3c6336662fc1e
--- /dev/null
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -0,0 +1,374 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* bpf_scan_ir: optimizer-last LLVM IR analysis for BPF. */
+
+#include "../bpf/fmodret_metadata.h"
+
+#include <algorithm>
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Analysis/CaptureTracking.h"
+#include "llvm/Analysis/LazyValueInfo.h"
+#include "llvm/Analysis/ValueTracking.h"
+#include "llvm/Config/llvm-config.h"
+#include "llvm/IR/Analysis.h"
+#include "llvm/IR/BasicBlock.h"
+#include "llvm/IR/CFG.h"
+#include "llvm/IR/ConstantRange.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/Dominators.h"
+#include "llvm/IR/GlobalVariable.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/IntrinsicInst.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/Passes/PassBuilder.h"
+#if __has_include("llvm/Plugins/PassPlugin.h")
+#include "llvm/Plugins/PassPlugin.h"
+#else
+#include "llvm/Passes/PassPlugin.h"
+#endif
+#include "llvm/Transforms/Utils/ModuleUtils.h"
+
+using namespace llvm;
+
+namespace {
+
+static constexpr int MaxErrno = 4095;
+
+// is Range in [-4095, 0) ?
+static bool isErrnoRange(const ConstantRange &Range)
+{
+ unsigned Width = Range.getBitWidth();
+
+ if (Width != 32 && Width != 64)
+ return false;
+
+ auto L = APInt(Width, 0) - APInt(Width, MaxErrno);
+ auto R = APInt(Width, 0);
+ return !Range.isEmptySet() && ConstantRange(L, R).contains(Range);
+}
+
+/* The errors which clean paths of a function return */
+using ReturnValues = SmallVector<int64_t, 8>;
+
+/* Return true and set N if RANGE holds a single errno */
+static bool singleErrno(const ConstantRange &Range, int64_t &N)
+{
+ const APInt *C = Range.getSingleElement();
+
+ if (!C || !isErrnoRange(Range))
+ return false;
+ N = C->getSExtValue();
+ return true;
+}
+
+/* Return true and set N if the value at use U is a single errno */
+static bool singleErrnoAtUse(const Use &U, LazyValueInfo &LVI, int64_t &N)
+{
+ return U.get()->getType()->isIntegerTy() &&
+ singleErrno(LVI.getConstantRangeAtUse(U, false), N);
+}
+
+/* Return true and set N if V is an errno constant */
+static bool errnoConstant(const Value *V, int64_t &N)
+{
+ auto *C = dyn_cast<ConstantInt>(V);
+
+ if (!C || (C->getBitWidth() != 32 && C->getBitWidth() != 64))
+ return false;
+
+ N = C->getSExtValue();
+ return N >= -MaxErrno && N < 0;
+}
+
+/* A sign extension or a truncation between integers of at least 32 bits */
+static bool preservesErrnoCast(const CastInst *Cast)
+{
+ Type *Src = Cast->getSrcTy(), *Dst = Cast->getDestTy();
+
+ if (Src->isIntegerTy() && Dst->isIntegerTy())
+ return (Cast->getOpcode() == Instruction::SExt ||
+ Cast->getOpcode() == Instruction::Trunc) &&
+ Src->getIntegerBitWidth() >= 32 && Dst->getIntegerBitWidth() >= 32;
+ return false;
+}
+
+/*
+ * Add the errors which V, the value returned in RET, is known to be on the
+ * edges to RET from CLEANBBS to VALUES. Follow the casts in RET which keep the
+ * error to a PHI in RET. A PHI in an earlier block is not followed: a later
+ * branch may split its values between clean and dirty paths.
+ */
+static void collectErrnos(Value *V, BasicBlock *Ret, LazyValueInfo &LVI,
+ const SmallPtrSetImpl<BasicBlock *> &CleanBBs, ReturnValues &Values)
+{
+ int64_t N;
+
+ while (auto *Cast = dyn_cast<CastInst>(V)) {
+ if (Cast->getParent() != Ret || !preservesErrnoCast(Cast))
+ return;
+ if (singleErrnoAtUse(Cast->getOperandUse(0), LVI, N)) {
+ Values.push_back(N);
+ return;
+ }
+ V = Cast->getOperand(0);
+ }
+
+ auto *Phi = dyn_cast<PHINode>(V);
+
+ if (!Phi || Phi->getParent() != Ret)
+ return;
+ for (unsigned i = 0; i < Phi->getNumIncomingValues(); i++) {
+ Value *Incoming = Phi->getIncomingValue(i);
+ BasicBlock *Pred = Phi->getIncomingBlock(i);
+
+ if (!CleanBBs.contains(Pred))
+ continue;
+ if (errnoConstant(Incoming, N)) {
+ Values.push_back(N);
+ continue;
+ }
+ /*
+ * LazyValueInfo knows the ranges of integers only. On an edge it
+ * also uses the branch condition, as in "if (err == -EINVAL) goto
+ * out", where err is the error.
+ */
+ if (Incoming->getType()->isIntegerTy() &&
+ singleErrno(LVI.getConstantRangeOnEdge(Incoming, Pred, Ret), N))
+ Values.push_back(N);
+ }
+}
+
+class CleanInstructions {
+ DominatorTree &DT;
+ DenseMap<const Instruction *, bool> PrivateWrites;
+
+ bool privateDestination(const Value *Pointer, const Instruction &I)
+ {
+ auto It = PrivateWrites.find(&I);
+ if (It != PrivateWrites.end())
+ return It->second;
+ auto *Allocation = dyn_cast<AllocaInst>(getUnderlyingObject(Pointer));
+ bool Private = false;
+
+ if (Allocation && Allocation->getFunction() == I.getFunction()) {
+ // A later call may receive the initialized object. Only captures
+ // that can precede this write make the write externally visible.
+#if LLVM_VERSION_MAJOR >= 21
+ Private = !PointerMayBeCapturedBefore(Allocation, true, &I, &DT, true, 256);
+#else
+ Private = !PointerMayBeCapturedBefore(Allocation, true, true, &I, &DT, true, 256);
+#endif
+ }
+ PrivateWrites[&I] = Private;
+ return Private;
+ }
+
+public:
+ explicit CleanInstructions(DominatorTree &DT) : DT(DT) {}
+
+ bool isClean(const Instruction &I)
+ {
+ if (auto *Store = dyn_cast<StoreInst>(&I))
+ return !Store->isVolatile() && !Store->isAtomic() &&
+ privateDestination(Store->getPointerOperand(), I);
+ if (auto *Memory = dyn_cast<MemIntrinsic>(&I))
+ return !Memory->isVolatile() &&
+ privateDestination(Memory->getRawDest(), I);
+ if (auto *Intrinsic = dyn_cast<IntrinsicInst>(&I)) {
+ switch (Intrinsic->getIntrinsicID()) {
+ case Intrinsic::assume:
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end:
+ case Intrinsic::invariant_start:
+ case Intrinsic::invariant_end:
+ return true;
+ default:
+ break;
+ }
+ }
+ return !I.mayHaveSideEffects();
+ }
+};
+
+static bool cleanBB(BasicBlock *BB, CleanInstructions &Clean)
+{
+ for (Instruction &I : *BB) {
+ if (!Clean.isClean(I))
+ return false;
+ }
+ return true;
+}
+
+/* Add the errors which clean paths of F are known to return to VALUES */
+static void cleanReturnValues(Function &F, LazyValueInfo &LVI, CleanInstructions &Clean,
+ ReturnValues &Values)
+{
+ SmallPtrSet<BasicBlock *, 32> CleanBBs;
+ SmallVector<BasicBlock *, 32> Worklist;
+ bool Changed;
+
+ /* The clean blocks which the entry reaches through clean blocks */
+ if (cleanBB(&F.getEntryBlock(), Clean)) {
+ CleanBBs.insert(&F.getEntryBlock());
+ Worklist.push_back(&F.getEntryBlock());
+ }
+ while (!Worklist.empty()) {
+ BasicBlock *BB = Worklist.pop_back_val();
+
+ for (BasicBlock *Successor : successors(BB)) {
+ if (!CleanBBs.contains(Successor) && cleanBB(Successor, Clean)) {
+ CleanBBs.insert(Successor);
+ Worklist.push_back(Successor);
+ }
+ }
+ }
+
+ /*
+ * Keep the blocks which only clean paths reach: after a dirty path joins
+ * a clean one, a branch may depend on what the dirty path did
+ */
+ do {
+ Changed = false;
+ for (BasicBlock &BB : F) {
+ if (!CleanBBs.contains(&BB))
+ continue;
+ for (BasicBlock *Pred : predecessors(&BB)) {
+ if (!CleanBBs.contains(Pred)) {
+ CleanBBs.erase(&BB);
+ Changed = true;
+ break;
+ }
+ }
+ }
+ } while (Changed);
+
+ /*
+ * The returns of the clean blocks which only clean paths reach or which
+ * one of them branches to
+ */
+ for (BasicBlock &BB : F) {
+ auto *Return = dyn_cast<ReturnInst>(BB.getTerminator());
+ bool Entered = CleanBBs.contains(&BB);
+ int64_t N;
+
+ if (!Return || !Return->getReturnValue() || !cleanBB(&BB, Clean))
+ continue;
+ for (BasicBlock *Pred : predecessors(&BB))
+ Entered |= CleanBBs.contains(Pred);
+ if (!Entered)
+ continue;
+ if (singleErrnoAtUse(Return->getOperandUse(0), LVI, N)) {
+ Values.push_back(N);
+ continue;
+ }
+ collectErrnos(Return->getReturnValue(), &BB, LVI, CleanBBs, Values);
+ }
+ llvm::sort(Values);
+ Values.erase(std::unique(Values.begin(), Values.end()), Values.end());
+}
+
+struct FunctionDescription {
+ Function *Fn;
+ unsigned Flags;
+ ReturnValues Values;
+};
+
+static void emitDescriptions(Module &M, ArrayRef<FunctionDescription> Functions)
+{
+ auto *I32 = Type::getInt32Ty(M.getContext());
+ auto *I64 = Type::getInt64Ty(M.getContext());
+ unsigned PtrSize = M.getDataLayout().getPointerSize();
+ auto N = [&](unsigned V) -> Constant * { return ConstantInt::get(I32, V); };
+
+ for (const auto &F : Functions) {
+ unsigned Size = FMODRET_META_HEADER_SIZE(PtrSize) +
+ F.Values.size() * FMODRET_META_VALUE_SIZE;
+ SmallVector<Constant *, 16> Fields = {
+ N(FMODRET_META_MAGIC), N(FMODRET_META_VERSION), N(Size), N(F.Flags),
+ F.Fn, N(F.Values.size()), N(0),
+ };
+
+ for (int64_t V : F.Values)
+ Fields.push_back(ConstantInt::getSigned(I64, V));
+ Constant *Record = ConstantStruct::getAnon(M.getContext(), Fields, true);
+ auto *GV = new GlobalVariable(M, Record->getType(), true, GlobalValue::PrivateLinkage,
+ Record, "fmodret.function." + F.Fn->getName());
+ GV->setSection(".BTF_fmodret_candidates");
+ GV->setAlignment(M.getDataLayout().getPointerABIAlignment(0));
+ appendToUsed(M, {GV});
+ }
+}
+
+#if LLVM_VERSION_MAJOR >= 24
+class BpfScanIRPass : public RequiredPassInfoMixin<BpfScanIRPass>
+#else
+class BpfScanIRPass : public PassInfoMixin<BpfScanIRPass>
+#endif
+{
+public:
+ PreservedAnalyses run(Module &M, ModuleAnalysisManager &MAM)
+ {
+ // Avoid emitting duplicate descriptions if bpf_scan_ir runs again
+ // on the same module.
+ if (M.getNamedMetadata("bpf_scan_ir.done"))
+ return PreservedAnalyses::all();
+
+ SmallVector<FunctionDescription, 32> Descriptions;
+ auto &FAM = MAM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
+
+ for (Function &F : M) {
+ if (F.isIntrinsic() || F.isDeclaration() || F.hasAvailableExternallyLinkage())
+ continue;
+ Type *Ret = F.getReturnType();
+ unsigned Flags = 0;
+ ReturnValues Values;
+
+ if (!Ret->isIntegerTy(32) && !Ret->isIntegerTy(64))
+ continue;
+ if (F.isWeakForLinker())
+ Flags |= FMODRET_FN_WEAK;
+ if (Ret->isIntegerTy(64))
+ Flags |= FMODRET_FN_RET64;
+ auto &LVI = FAM.getResult<LazyValueAnalysis>(F);
+ auto &DT = FAM.getResult<DominatorTreeAnalysis>(F);
+ CleanInstructions Clean(DT);
+
+ cleanReturnValues(F, LVI, Clean, Values);
+ /* A weak definition always has a record, see fmodret_metadata.h */
+ if (!Values.empty() || (Flags & FMODRET_FN_WEAK))
+ Descriptions.push_back({&F, Flags, std::move(Values)});
+ }
+ if (Descriptions.empty())
+ return PreservedAnalyses::all();
+ emitDescriptions(M, Descriptions);
+ M.getOrInsertNamedMetadata("bpf_scan_ir.done");
+
+ PreservedAnalyses PA;
+ PA.preserve<FunctionAnalysisManagerModuleProxy>();
+ PA.preserveSet<CFGAnalyses>();
+ PA.preserve<LazyValueAnalysis>();
+ return PA;
+ }
+
+ static bool isRequired() { return true; }
+};
+
+} // namespace
+
+extern "C" LLVM_ATTRIBUTE_WEAK PassPluginLibraryInfo llvmGetPassPluginInfo()
+{
+ return {
+ LLVM_PLUGIN_API_VERSION,
+ "bpf_scan_ir",
+ "1.0",
+ [](PassBuilder &PB) {
+ PB.registerOptimizerLastEPCallback(
+ [](ModulePassManager &MPM, OptimizationLevel, auto...) {
+ MPM.addPass(BpfScanIRPass());
+ }
+ );
+ }
+ };
+}
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 75374c64b8cc78..3bae985492288f 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -772,6 +772,7 @@ static const char *const section_white_list[] =
".llvm.call-graph-profile", /* call graph */
"__llvm_covfun",
"__llvm_covmap",
+ ".BTF_fmodret_candidates",
".klp.symid", /* objtool --klp-symids */
NULL
};
diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index d0f20042895763..e6ba499dc87661 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -27,6 +27,11 @@ SECTIONS {
__ex_table 0 : ALIGN(4) { KEEP(*(__ex_table)) }
__patchable_function_entries 0 : { *(__patchable_function_entries) }
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+ .BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }
+#else
+ /DISCARD/ : { *(.BTF_fmodret_candidates) }
+#endif
.init.klp_funcs 0 : ALIGN(8) { KEEP(*(.init.klp_funcs)) }
.init.klp_objects 0 : ALIGN(8) { KEEP(*(.init.klp_objects)) }
diff --git a/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
new file mode 100644
index 00000000000000..1580fd614010ba
--- /dev/null
+++ b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
@@ -0,0 +1,111 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+===============
+bpftool-fmodret
+===============
+-------------------------------------------------------------------------------
+tool to inspect BPF fmod_ret targets
+-------------------------------------------------------------------------------
+
+:Manual section: 8
+
+.. include:: substitutions.rst
+
+SYNOPSIS
+========
+
+**bpftool** [*OPTIONS*] **fmodret** *COMMAND*
+
+*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } *FILE* }
+
+*COMMANDS* := { **list** | **show** | **help** }
+
+FMODRET COMMANDS
+================
+
+| **bpftool** **fmodret list** [*OBJECT*]
+| **bpftool** **fmodret show** *FUNC* [*OBJECT*]
+| **bpftool** **fmodret help**
+|
+| *FUNC* := { **name** *NAME* | **id** *BTF_ID* }
+| *OBJECT* := { **file** *FILE* | **vmlinux** | **module** *NAME* }
+
+DESCRIPTION
+===========
+
+bpftool fmodret list [*OBJECT*]
+ List the fmod_ret targets, the functions which the kernel build tags with
+ the **bpf_fmodret** BTF decl tag, in vmlinux and in loaded modules. Each
+ entry contains the BTF ID of the function, its name and the name of the
+ object it belongs to. Targets which sleepable programs may attach to are
+ marked **sleepable**.
+
+ With **file**, read the BTF from *FILE* (for example a vmlinux image or a
+ **BTF** file) instead of the running kernel. With **vmlinux** or
+ **module**, list only the targets of vmlinux or of the module *NAME*.
+
+ With **--json**, each entry contains **btf_id**, **name**, **owner** and
+ **sleepable** fields.
+
+ The verifier also accepts functions by its static rules, such as the error
+ injection list or **security_** hooks. Such functions are listed only if
+ the kernel build tagged them. Even if they are listed, the restrictions of
+ their **bpf_fmodret_retval:** tags do not apply to them: programs may
+ return any value, and sleepable programs may attach to some of them without
+ the **sleepable** mark. This may change in the future by either not listing
+ them, or listing them without restrictions. Normal verifier and tracing
+ checks apply to all of them.
+
+bpftool fmodret show *FUNC* [*OBJECT*]
+ Show the function named *NAME* or with BTF ID *BTF_ID* in detail. It does
+ not have to be a target. A name may belong to several functions, for
+ example to functions of different modules. A BTF ID is looked up in
+ vmlinux, unless **module** or **file** is given. *OBJECT* is as for
+ **list**.
+
+ Besides the BTF ID, the name and the object, show the prototype of the
+ function and if it is a target: **found by the kernel build**, **declared**
+ or **not tagged**. For a target show if sleepable programs may attach to it
+ and the values which programs may return: 0 and the values of its
+ **bpf_fmodret_retval:** tag with the names of the errors on the host, or
+ **any** for a declared target. A tag which the verifier would reject,
+ malformed, not the only one or with values which the return type cannot
+ hold, is shown as invalid. The values do not apply to a function which the
+ static rules allow, see **list**. For the running kernel, also show the
+ type of the entry of the function in the error injection list, if the list
+ is available in debugfs.
+
+ With **--json**, each function contains **btf_id**, **name**, **owner**,
+ **prototype**, **target** (**found**, **declared** or **none**) and
+ **sleepable** fields, a **retval** array for a target found by the build
+ with the values programs may return, 0 included, or a **retval_error**
+ string if its tag is invalid, and an **error_injection** field if the
+ function is in the error injection list. A BTF ID gives one function, a
+ name an array of them.
+
+bpftool fmodret help
+ Print short help message.
+
+OPTIONS
+========
+
+.. include:: common_options.rst
+
+-B, --base-btf *FILE*
+ Pass the base BTF of a split BTF *FILE*, usually the BTF of vmlinux. Raw
+ BTF of a module needs it. A module image may carry the base it was built
+ against in its **.BTF.base** section instead, but its BTF IDs then differ
+ from those of the running kernel.
+
+EXAMPLES
+========
+
+**# bpftool fmodret show name bpf_testmod_retval_join module bpf_test_retval**
+
+::
+
+ 123456 bpf_testmod_retval_join (bpf_test_retval)
+ int bpf_testmod_retval_join(int *result, int value)
+ target: found by the kernel build
+ sleepable: no
+ retval: 0, -34 (ERANGE), -22 (EINVAL)
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index f38ae5c4043917..2c64c22a50269a 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -21,7 +21,7 @@ SYNOPSIS
**bpftool** **version**
*OBJECT* := { **map** | **prog** | **link** | **cgroup** | **perf** | **net** | **feature** |
-**btf** | **gen** | **struct_ops** | **iter** }
+**btf** | **gen** | **struct_ops** | **iter** | **fmodret** }
*OPTIONS* := { { **-V** | **--version** } | |COMMON_OPTIONS| }
diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index a28af31937ad4b..83288c22b3d34e 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -675,6 +675,37 @@ _bpftool()
;;
esac
;;
+ fmodret)
+ case $command in
+ list|show)
+ case $prev in
+ file)
+ _filedir
+ ;;
+ module)
+ COMPREPLY=( $( compgen -W "$( command ls \
+ /sys/kernel/btf 2>/dev/null | \
+ command grep -vx -e vmlinux -e '.*\.inline' )" \
+ -- "$cur" ) )
+ ;;
+ name|id)
+ ;;
+ *)
+ if [[ $prev == show ]]; then
+ COMPREPLY=( $( compgen -W 'name id' -- "$cur" ) )
+ else
+ _bpftool_one_of_list 'file vmlinux module'
+ fi
+ ;;
+ esac
+ return 0
+ ;;
+ *)
+ [[ $prev == $object ]] && \
+ COMPREPLY=( $( compgen -W 'help list show' -- "$cur" ) )
+ ;;
+ esac
+ ;;
map)
case $command in
show|list|dump|peek|pop|dequeue|freeze)
diff --git a/tools/bpf/bpftool/fmodret.c b/tools/bpf/bpftool/fmodret.c
new file mode 100644
index 00000000000000..941166da53cc23
--- /dev/null
+++ b/tools/bpf/bpftool/fmodret.c
@@ -0,0 +1,896 @@
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include <dirent.h>
+#include <errno.h>
+#include <limits.h>
+#include <stdarg.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+
+#include "main.h"
+
+/*
+ * Decl tags added by resolve_btfids to functions in BTF_FMODRET sets. A target
+ * which sleepable programs may attach to has both tags. A target which the
+ * build found by analysing its code has a "bpf_fmodret_retval:" tag with the
+ * values programs may return, such as "bpf_fmodret_retval:-22,-12".
+ */
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+#define TAG_FMODRET_RETVAL "bpf_fmodret_retval:"
+
+#define SYS_BTF_DIR "/sys/kernel/btf"
+#define ERROR_INJECTION_LIST "/sys/kernel/debug/error_injection/list"
+
+struct fmodret_entry {
+ __u32 btf_id;
+ bool target;
+ bool sleepable;
+ const char *retval;
+ /* More than one "bpf_fmodret_retval:" tag, which the verifier rejects */
+ bool retval_dup;
+};
+
+struct fmodret_filter {
+ /* Only vmlinux, or only the module with this name */
+ bool vmlinux;
+ const char *module;
+ /* The function to show, by name or by BTF ID */
+ const char *name;
+ __u32 btf_id;
+ unsigned int found;
+ /* The BTF is the one of the running kernel */
+ bool live;
+};
+
+typedef int (*btf_cb)(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter);
+
+#define ERRNO_NAME(e) { e, #e }
+
+/* The errors of the uapi and the kernel internal ones, which libc lacks */
+static const struct {
+ int value;
+ const char *name;
+} errno_names[] = {
+ ERRNO_NAME(EPERM),
+ ERRNO_NAME(ENOENT),
+ ERRNO_NAME(ESRCH),
+ ERRNO_NAME(EINTR),
+ ERRNO_NAME(EIO),
+ ERRNO_NAME(ENXIO),
+ ERRNO_NAME(E2BIG),
+ ERRNO_NAME(ENOEXEC),
+ ERRNO_NAME(EBADF),
+ ERRNO_NAME(ECHILD),
+ ERRNO_NAME(EAGAIN),
+ ERRNO_NAME(ENOMEM),
+ ERRNO_NAME(EACCES),
+ ERRNO_NAME(EFAULT),
+ ERRNO_NAME(ENOTBLK),
+ ERRNO_NAME(EBUSY),
+ ERRNO_NAME(EEXIST),
+ ERRNO_NAME(EXDEV),
+ ERRNO_NAME(ENODEV),
+ ERRNO_NAME(ENOTDIR),
+ ERRNO_NAME(EISDIR),
+ ERRNO_NAME(EINVAL),
+ ERRNO_NAME(ENFILE),
+ ERRNO_NAME(EMFILE),
+ ERRNO_NAME(ENOTTY),
+ ERRNO_NAME(ETXTBSY),
+ ERRNO_NAME(EFBIG),
+ ERRNO_NAME(ENOSPC),
+ ERRNO_NAME(ESPIPE),
+ ERRNO_NAME(EROFS),
+ ERRNO_NAME(EMLINK),
+ ERRNO_NAME(EPIPE),
+ ERRNO_NAME(EDOM),
+ ERRNO_NAME(ERANGE),
+ ERRNO_NAME(EDEADLK),
+#if defined(EDEADLOCK) && EDEADLOCK != EDEADLK
+ ERRNO_NAME(EDEADLOCK),
+#endif
+ ERRNO_NAME(ENAMETOOLONG),
+ ERRNO_NAME(ENOLCK),
+ ERRNO_NAME(ENOSYS),
+ ERRNO_NAME(ENOTEMPTY),
+ ERRNO_NAME(ELOOP),
+ ERRNO_NAME(ENOMSG),
+ ERRNO_NAME(EIDRM),
+ ERRNO_NAME(ECHRNG),
+ ERRNO_NAME(EL2NSYNC),
+ ERRNO_NAME(EL3HLT),
+ ERRNO_NAME(EL3RST),
+ ERRNO_NAME(ELNRNG),
+ ERRNO_NAME(EUNATCH),
+ ERRNO_NAME(ENOCSI),
+ ERRNO_NAME(EL2HLT),
+ ERRNO_NAME(EBADE),
+ ERRNO_NAME(EBADR),
+ ERRNO_NAME(EXFULL),
+ ERRNO_NAME(ENOANO),
+ ERRNO_NAME(EBADRQC),
+ ERRNO_NAME(EBADSLT),
+ ERRNO_NAME(EBFONT),
+ ERRNO_NAME(ENOSTR),
+ ERRNO_NAME(ENODATA),
+ ERRNO_NAME(ETIME),
+ ERRNO_NAME(ENOSR),
+ ERRNO_NAME(ENONET),
+ ERRNO_NAME(ENOPKG),
+ ERRNO_NAME(EREMOTE),
+ ERRNO_NAME(ENOLINK),
+ ERRNO_NAME(EADV),
+ ERRNO_NAME(ESRMNT),
+ ERRNO_NAME(ECOMM),
+ ERRNO_NAME(EPROTO),
+ ERRNO_NAME(EMULTIHOP),
+ ERRNO_NAME(EDOTDOT),
+ ERRNO_NAME(EBADMSG),
+ ERRNO_NAME(EOVERFLOW),
+ ERRNO_NAME(ENOTUNIQ),
+ ERRNO_NAME(EBADFD),
+ ERRNO_NAME(EREMCHG),
+ ERRNO_NAME(ELIBACC),
+ ERRNO_NAME(ELIBBAD),
+ ERRNO_NAME(ELIBSCN),
+ ERRNO_NAME(ELIBMAX),
+ ERRNO_NAME(ELIBEXEC),
+ ERRNO_NAME(EILSEQ),
+ ERRNO_NAME(ERESTART),
+ ERRNO_NAME(ESTRPIPE),
+ ERRNO_NAME(EUSERS),
+ ERRNO_NAME(ENOTSOCK),
+ ERRNO_NAME(EDESTADDRREQ),
+ ERRNO_NAME(EMSGSIZE),
+ ERRNO_NAME(EPROTOTYPE),
+ ERRNO_NAME(ENOPROTOOPT),
+ ERRNO_NAME(EPROTONOSUPPORT),
+ ERRNO_NAME(ESOCKTNOSUPPORT),
+ ERRNO_NAME(EOPNOTSUPP),
+ ERRNO_NAME(EPFNOSUPPORT),
+ ERRNO_NAME(EAFNOSUPPORT),
+ ERRNO_NAME(EADDRINUSE),
+ ERRNO_NAME(EADDRNOTAVAIL),
+ ERRNO_NAME(ENETDOWN),
+ ERRNO_NAME(ENETUNREACH),
+ ERRNO_NAME(ENETRESET),
+ ERRNO_NAME(ECONNABORTED),
+ ERRNO_NAME(ECONNRESET),
+ ERRNO_NAME(ENOBUFS),
+ ERRNO_NAME(EISCONN),
+ ERRNO_NAME(ENOTCONN),
+ ERRNO_NAME(ESHUTDOWN),
+ ERRNO_NAME(ETOOMANYREFS),
+ ERRNO_NAME(ETIMEDOUT),
+ ERRNO_NAME(ECONNREFUSED),
+ ERRNO_NAME(EHOSTDOWN),
+ ERRNO_NAME(EHOSTUNREACH),
+ ERRNO_NAME(EALREADY),
+ ERRNO_NAME(EINPROGRESS),
+ ERRNO_NAME(ESTALE),
+ ERRNO_NAME(EUCLEAN),
+ ERRNO_NAME(ENOTNAM),
+ ERRNO_NAME(ENAVAIL),
+ ERRNO_NAME(EISNAM),
+ ERRNO_NAME(EREMOTEIO),
+ ERRNO_NAME(EDQUOT),
+ ERRNO_NAME(ENOMEDIUM),
+ ERRNO_NAME(EMEDIUMTYPE),
+ ERRNO_NAME(ECANCELED),
+ ERRNO_NAME(ENOKEY),
+ ERRNO_NAME(EKEYEXPIRED),
+ ERRNO_NAME(EKEYREVOKED),
+ ERRNO_NAME(EKEYREJECTED),
+ ERRNO_NAME(EOWNERDEAD),
+ ERRNO_NAME(ENOTRECOVERABLE),
+ ERRNO_NAME(ERFKILL),
+ ERRNO_NAME(EHWPOISON),
+#ifdef EFTYPE
+ ERRNO_NAME(EFTYPE),
+#endif
+ { 512, "ERESTARTSYS" },
+ { 513, "ERESTARTNOINTR" },
+ { 514, "ERESTARTNOHAND" },
+ { 515, "ENOIOCTLCMD" },
+ { 516, "ERESTART_RESTARTBLOCK" },
+ { 517, "EPROBE_DEFER" },
+ { 518, "EOPENSTALE" },
+ { 519, "ENOPARAM" },
+ { 521, "EBADHANDLE" },
+ { 522, "ENOTSYNC" },
+ { 523, "EBADCOOKIE" },
+ { 524, "ENOTSUPP" },
+ { 525, "ETOOSMALL" },
+ { 526, "ESERVERFAULT" },
+ { 527, "EBADTYPE" },
+ { 528, "EJUKEBOX" },
+ { 529, "EIOCBQUEUED" },
+ { 530, "ERECALLCONFLICT" },
+ { 531, "ENOGRACE" },
+};
+
+static const char *errno_name(long long value)
+{
+ size_t i;
+
+ for (i = 0; i < ARRAY_SIZE(errno_names); i++)
+ if (-(long long)errno_names[i].value == value)
+ return errno_names[i].name;
+ return NULL;
+}
+
+static int cmp_entry(const void *a, const void *b)
+{
+ const struct fmodret_entry *x = a, *y = b;
+
+ return (x->btf_id > y->btf_id) - (x->btf_id < y->btf_id);
+}
+
+/* The first type ID which belongs to BTF and not to its base */
+static __u32 first_own_type(const struct btf *btf)
+{
+ const struct btf *base = btf__base_btf(btf);
+
+ return base ? btf__type_cnt(base) : 1;
+}
+
+/*
+ * Return the function which T tags with decl tag NAME, or 0. A NAME which ends
+ * with ':' matches the tags which start with it.
+ */
+static __u32 tag_func(const struct btf *btf, const struct btf_type *t, const char *name)
+{
+ size_t len = strlen(name);
+ const char *tag;
+
+ if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1)
+ return 0;
+ tag = btf__name_by_offset(btf, t->name_off);
+ if (strncmp(tag, name, len) || (name[len - 1] != ':' && tag[len]))
+ return 0;
+ if (!btf_is_func(btf__type_by_id(btf, t->type)))
+ return 0;
+ return t->type;
+}
+
+/* Return the fmod_ret targets of BTF, sorted by BTF ID */
+static int collect_targets(const struct btf *btf, struct fmodret_entry **targets,
+ size_t *target_cnt)
+{
+ __u32 id, func_id, start_id = first_own_type(btf);
+ __u32 nr_types = btf__type_cnt(btf);
+ struct fmodret_entry *entries = NULL, *entry, *tmp;
+ size_t cnt = 0, capacity = 0;
+
+ for (id = start_id; id < nr_types; id++) {
+ func_id = tag_func(btf, btf__type_by_id(btf, id), TAG_FMODRET);
+ if (!func_id)
+ continue;
+
+ if (cnt == capacity) {
+ capacity = capacity ? 2 * capacity : 64;
+ tmp = realloc(entries, capacity * sizeof(*entries));
+ if (!tmp) {
+ free(entries);
+ return -ENOMEM;
+ }
+ entries = tmp;
+ }
+ entries[cnt].btf_id = func_id;
+ entries[cnt].sleepable = false;
+ cnt++;
+ }
+ if (cnt)
+ qsort(entries, cnt, sizeof(*entries), cmp_entry);
+
+ /* The other tags mark functions which are already targets */
+ for (id = start_id; cnt && id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ struct fmodret_entry key = {};
+
+ key.btf_id = tag_func(btf, t, TAG_FMODRET_SLEEPABLE);
+ if (key.btf_id) {
+ entry = bsearch(&key, entries, cnt, sizeof(*entries), cmp_entry);
+ if (entry)
+ entry->sleepable = true;
+ }
+ }
+
+ *targets = entries;
+ *target_cnt = cnt;
+ return 0;
+}
+
+/* Fill ENTRY from the decl tags of function FUNC_ID */
+static void func_entry(const struct btf *btf, __u32 func_id, struct fmodret_entry *entry)
+{
+ __u32 id, nr_types = btf__type_cnt(btf);
+
+ memset(entry, 0, sizeof(*entry));
+ entry->btf_id = func_id;
+ for (id = first_own_type(btf); id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (tag_func(btf, t, TAG_FMODRET) == func_id)
+ entry->target = true;
+ else if (tag_func(btf, t, TAG_FMODRET_SLEEPABLE) == func_id)
+ entry->sleepable = true;
+ else if (tag_func(btf, t, TAG_FMODRET_RETVAL) == func_id) {
+ entry->retval_dup = entry->retval;
+ entry->retval = btf__name_by_offset(btf, t->name_off) +
+ strlen(TAG_FMODRET_RETVAL);
+ }
+ }
+}
+
+static void list_target(const char *owner, const struct btf *btf,
+ const struct fmodret_entry *entry)
+{
+ const struct btf_type *t = btf__type_by_id(btf, entry->btf_id);
+ const char *name = btf__name_by_offset(btf, t->name_off);
+
+ if (json_output) {
+ jsonw_start_object(json_wtr);
+ jsonw_uint_field(json_wtr, "btf_id", entry->btf_id);
+ jsonw_string_field(json_wtr, "name", name);
+ jsonw_string_field(json_wtr, "owner", owner);
+ jsonw_bool_field(json_wtr, "sleepable", entry->sleepable);
+ jsonw_end_object(json_wtr);
+ } else {
+ printf("%u %s (%s)%s\n", entry->btf_id, name, owner,
+ entry->sleepable ? " sleepable" : "");
+ }
+}
+
+static int list_btf(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter)
+{
+ struct fmodret_entry *targets = NULL;
+ size_t cnt = 0, i;
+ int err;
+
+ err = collect_targets(btf, &targets, &cnt);
+ if (err) {
+ p_err("failed to collect fmod_ret targets of %s: %s", owner,
+ strerror(-err));
+ return err;
+ }
+ for (i = 0; i < cnt; i++)
+ list_target(owner, btf, &targets[i]);
+ free(targets);
+ return 0;
+}
+
+struct error_injection {
+ char *name;
+ char *module;
+ char *type;
+};
+
+static struct error_injection *error_injections;
+static size_t error_injection_cnt;
+
+/*
+ * Read the error injection list of the running kernel. Its lines are
+ * "FUNC\tTYPE" and "FUNC [MODULE]\tTYPE". Without debugfs the list is empty.
+ */
+static void read_error_injection_list(void)
+{
+ struct error_injection *tmp;
+ char *line = NULL;
+ size_t len = 0;
+ FILE *f;
+
+ f = fopen(ERROR_INJECTION_LIST, "r");
+ if (!f)
+ return;
+ while (getline(&line, &len, f) > 0) {
+ char *name, *module = NULL, *type, *save;
+
+ /* "NAME [MODULE]\tTYPE" or "NAME\tTYPE" */
+ name = strtok_r(line, " \t\n", &save);
+ type = strtok_r(NULL, " \t\n", &save);
+ if (type && type[0] == '[') {
+ module = type + 1;
+ module[strcspn(module, "]")] = '\0';
+ type = strtok_r(NULL, " \t\n", &save);
+ }
+ if (!name || !type)
+ continue;
+ tmp = realloc(error_injections,
+ (error_injection_cnt + 1) * sizeof(*error_injections));
+ if (!tmp)
+ break;
+ error_injections = tmp;
+ tmp = &error_injections[error_injection_cnt];
+ tmp->name = strdup(name);
+ tmp->module = strdup(module ?: "");
+ tmp->type = strdup(type);
+ if (!tmp->name || !tmp->module || !tmp->type) {
+ free(tmp->name);
+ free(tmp->module);
+ free(tmp->type);
+ break;
+ }
+ error_injection_cnt++;
+ }
+ free(line);
+ fclose(f);
+}
+
+static void free_error_injection_list(void)
+{
+ size_t i;
+
+ for (i = 0; i < error_injection_cnt; i++) {
+ free(error_injections[i].name);
+ free(error_injections[i].module);
+ free(error_injections[i].type);
+ }
+ free(error_injections);
+ error_injections = NULL;
+ error_injection_cnt = 0;
+}
+
+/* Return the error injection type of function NAME of OWNER, or NULL */
+static const char *error_injection_type(const char *name, const char *owner)
+{
+ const char *module = strcmp(owner, "vmlinux") ? owner : "";
+ size_t i;
+
+ for (i = 0; i < error_injection_cnt; i++)
+ if (!strcmp(error_injections[i].name, name) &&
+ !strcmp(error_injections[i].module, module))
+ return error_injections[i].type;
+ return NULL;
+}
+
+static void __printf(2, 0) prototype_printf(void *ctx, const char *fmt, va_list args)
+{
+ vfprintf(ctx, fmt, args);
+}
+
+/* Return the declaration of function FUNC_ID, or NULL */
+static char *func_prototype(const struct btf *btf, __u32 func_id)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func_id);
+ LIBBPF_OPTS(btf_dump_emit_type_decl_opts, opts,
+ .field_name = btf__name_by_offset(btf, t->name_off),
+ );
+ struct btf_dump *d;
+ char *buf = NULL;
+ size_t size = 0;
+ FILE *f;
+ int err;
+
+ f = open_memstream(&buf, &size);
+ if (!f)
+ return NULL;
+ d = btf_dump__new(btf, prototype_printf, f, NULL);
+ err = libbpf_get_error(d);
+ if (!err) {
+ err = btf_dump__emit_type_decl(d, t->type, &opts);
+ btf_dump__free(d);
+ }
+ fclose(f);
+ if (err) {
+ free(buf);
+ return NULL;
+ }
+ return buf;
+}
+
+/* Show 0 and the values of a "bpf_fmodret_retval:" tag as a JSON array */
+static void show_retval_json(const char *retval)
+{
+ const char *p = retval;
+ char *end;
+
+ jsonw_name(json_wtr, "retval");
+ jsonw_start_array(json_wtr);
+ jsonw_int(json_wtr, 0);
+ while (*p) {
+ long long value;
+
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno)
+ break;
+ jsonw_int(json_wtr, value);
+ p = *end == ',' ? end + 1 : end;
+ }
+ jsonw_end_array(json_wtr);
+}
+
+/*
+ * Check the "bpf_fmodret_retval:" tag of FUNC_ID as the verifier does: negative
+ * decimal numbers separated by commas, which the integer or enum of 4 or 8 bytes
+ * that the function returns can hold. Return why it is invalid, or NULL.
+ */
+static const char *retval_invalid(const struct btf *btf, __u32 func_id,
+ const char *retval)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func_id);
+ const char *p = retval;
+ int ret_id;
+ char *end;
+
+ t = btf__type_by_id(btf, t->type);
+ ret_id = t ? btf__resolve_type(btf, t->type) : -EINVAL;
+ t = ret_id > 0 ? btf__type_by_id(btf, ret_id) : NULL;
+ if (!t || !(btf_is_int(t) || btf_is_any_enum(t)) || (t->size != 4 && t->size != 8))
+ return "the function does not return an integer of 4 or 8 bytes";
+ for (;;) {
+ long long value;
+
+ if (*p != '-')
+ return "malformed tag";
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno || value >= 0)
+ return "malformed tag";
+ if (t->size == 4 && value < INT_MIN)
+ return "a value does not fit the return type";
+ if (!*end)
+ return NULL;
+ if (*end != ',')
+ return "malformed tag";
+ p = end + 1;
+ }
+}
+
+/* Show the values of a "bpf_fmodret_retval:" tag with the names of errors */
+static void show_retval_plain(const char *retval)
+{
+ const char *p = retval, *name;
+ char *end;
+
+ printf("\tretval: 0");
+ while (*p) {
+ long long value;
+
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno)
+ break;
+ printf(", %lld", value);
+ name = errno_name(value);
+ if (name)
+ printf(" (%s)", name);
+ p = *end == ',' ? end + 1 : end;
+ }
+ printf("\n");
+}
+
+static int show_function(const char *owner, const struct btf *btf,
+ const struct fmodret_entry *entry,
+ struct fmodret_filter *filter)
+{
+ const struct btf_type *t = btf__type_by_id(btf, entry->btf_id);
+ const char *name = btf__name_by_offset(btf, t->name_off);
+ const char *injection = NULL, *invalid = NULL;
+ char *prototype;
+
+ /* The verifier rejects programs on a function with an invalid tag */
+ if (entry->target && entry->retval_dup)
+ invalid = "more than one tag";
+ else if (entry->target && entry->retval)
+ invalid = retval_invalid(btf, entry->btf_id, entry->retval);
+ prototype = func_prototype(btf, entry->btf_id);
+ if (!prototype) {
+ p_err("failed to print the prototype of %s", name);
+ return -EINVAL;
+ }
+ if (filter->live)
+ injection = error_injection_type(name, owner);
+
+ if (json_output) {
+ jsonw_start_object(json_wtr);
+ jsonw_uint_field(json_wtr, "btf_id", entry->btf_id);
+ jsonw_string_field(json_wtr, "name", name);
+ jsonw_string_field(json_wtr, "owner", owner);
+ jsonw_string_field(json_wtr, "prototype", prototype);
+ jsonw_string_field(json_wtr, "target", !entry->target ? "none" :
+ entry->retval ? "found" : "declared");
+ jsonw_bool_field(json_wtr, "sleepable", entry->target && entry->sleepable);
+ if (invalid)
+ jsonw_string_field(json_wtr, "retval_error", invalid);
+ else if (entry->target && entry->retval)
+ show_retval_json(entry->retval);
+ if (injection)
+ jsonw_string_field(json_wtr, "error_injection", injection);
+ jsonw_end_object(json_wtr);
+ } else {
+ printf("%u %s (%s)\n", entry->btf_id, name, owner);
+ printf("\t%s\n", prototype);
+ if (entry->target) {
+ printf("\ttarget: %s\n", entry->retval ? "found by the kernel build" :
+ "declared");
+ printf("\tsleepable: %s\n", entry->sleepable ? "yes" : "no");
+ if (invalid)
+ printf("\tretval: invalid, %s\n", invalid);
+ else if (entry->retval)
+ show_retval_plain(entry->retval);
+ else
+ printf("\tretval: any\n");
+ } else {
+ printf("\ttarget: not tagged, the verifier may accept it by other rules\n");
+ }
+ if (injection)
+ printf("\terror injection: %s\n", injection);
+ }
+ free(prototype);
+ filter->found++;
+ return 0;
+}
+
+static int show_btf(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter)
+{
+ __u32 id, nr_types = btf__type_cnt(btf);
+ struct fmodret_entry entry;
+ int err;
+
+ if (filter->btf_id) {
+ if (filter->btf_id < first_own_type(btf) || filter->btf_id >= nr_types ||
+ !btf_is_func(btf__type_by_id(btf, filter->btf_id)))
+ return 0;
+ func_entry(btf, filter->btf_id, &entry);
+ return show_function(owner, btf, &entry, filter);
+ }
+
+ for (id = first_own_type(btf); id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_func(t) ||
+ strcmp(btf__name_by_offset(btf, t->name_off), filter->name))
+ continue;
+ func_entry(btf, id, &entry);
+ err = show_function(owner, btf, &entry, filter);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+static int filter_module(const struct dirent *entry)
+{
+ size_t len = strlen(entry->d_name);
+
+ /* Skip ".", "..", vmlinux and the ".inline" companions of modules */
+ return entry->d_name[0] != '.' && strcmp(entry->d_name, "vmlinux") &&
+ !(len > 7 && !strcmp(entry->d_name + len - 7, ".inline"));
+}
+
+static int walk_modules(struct btf *vmlinux, struct fmodret_filter *filter, btf_cb cb)
+{
+ struct dirent **modules = NULL;
+ int i, nr_modules, err = 0;
+
+ if (filter->module) {
+ struct btf *btf;
+
+ btf = btf__load_module_btf(filter->module, vmlinux);
+ err = libbpf_get_error(btf);
+ if (err) {
+ p_err("failed to load BTF of module %s: %s", filter->module,
+ strerror(-err));
+ return err;
+ }
+ err = cb(filter->module, btf, filter);
+ btf__free(btf);
+ return err;
+ }
+
+ nr_modules = scandir(SYS_BTF_DIR, &modules, filter_module, alphasort);
+ if (nr_modules < 0) {
+ /* There is no module directory without module BTF, which is fine */
+ if (errno == ENOENT)
+ return 0;
+ err = -errno;
+ p_err("failed to read %s: %s", SYS_BTF_DIR, strerror(errno));
+ return err;
+ }
+ for (i = 0; i < nr_modules; i++) {
+ const char *name = modules[i]->d_name;
+ struct btf *btf;
+
+ btf = btf__load_module_btf(name, vmlinux);
+ err = libbpf_get_error(btf);
+ if (err == -ENOENT) {
+ /* The module was unloaded meanwhile */
+ err = 0;
+ continue;
+ }
+ if (err) {
+ p_err("failed to load BTF of module %s: %s", name, strerror(-err));
+ break;
+ }
+ err = cb(name, btf, filter);
+ btf__free(btf);
+ if (err)
+ break;
+ }
+ for (i = 0; i < nr_modules; i++)
+ free(modules[i]);
+ free(modules);
+ return err;
+}
+
+static int walk_live_btf(struct fmodret_filter *filter, btf_cb cb)
+{
+ struct btf *vmlinux;
+ int err = 0;
+
+ vmlinux = btf__load_vmlinux_btf();
+ err = libbpf_get_error(vmlinux);
+ if (err) {
+ p_err("failed to load vmlinux BTF: %s", strerror(-err));
+ return err;
+ }
+ if (!filter->module)
+ err = cb("vmlinux", vmlinux, filter);
+ if (!err && !filter->vmlinux)
+ err = walk_modules(vmlinux, filter, cb);
+ btf__free(vmlinux);
+ return err;
+}
+
+static int walk_file_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)
+{
+ struct btf *btf;
+ int err;
+
+ btf = base_btf ? btf__parse_split(path, base_btf) : btf__parse(path, NULL);
+ err = libbpf_get_error(btf);
+ if (err) {
+ p_err("failed to parse BTF from %s: %s", path, strerror(-err));
+ return err;
+ }
+ err = cb(path, btf, filter);
+ btf__free(btf);
+ return err;
+}
+
+/* Parse the optional OBJECT := { file FILE | vmlinux | module NAME } */
+static int parse_object(int argc, char **argv, const char **path,
+ struct fmodret_filter *filter)
+{
+ if (!argc)
+ return 0;
+ if (is_prefix(*argv, "vmlinux")) {
+ filter->vmlinux = true;
+ NEXT_ARG();
+ } else if (is_prefix(*argv, "file")) {
+ NEXT_ARG();
+ if (!REQ_ARGS(1))
+ return -1;
+ *path = GET_ARG();
+ } else if (is_prefix(*argv, "module")) {
+ NEXT_ARG();
+ if (!REQ_ARGS(1))
+ return -1;
+ filter->module = GET_ARG();
+ }
+ if (argc)
+ return BAD_ARG();
+ return 0;
+}
+
+static int walk_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)
+{
+ return path ? walk_file_btf(path, filter, cb) : walk_live_btf(filter, cb);
+}
+
+static int do_list(int argc, char **argv)
+{
+ struct fmodret_filter filter = {};
+ const char *path = NULL;
+ int err;
+
+ if (parse_object(argc, argv, &path, &filter))
+ return -1;
+
+ if (json_output)
+ jsonw_start_array(json_wtr);
+ err = walk_btf(path, &filter, list_btf);
+ if (json_output)
+ jsonw_end_array(json_wtr);
+ return err;
+}
+
+static int do_show(int argc, char **argv)
+{
+ struct fmodret_filter filter = {};
+ const char *path = NULL;
+ unsigned long id;
+ char *endptr;
+ int err;
+
+ if (!REQ_ARGS(2))
+ return -1;
+ if (is_prefix(*argv, "name")) {
+ NEXT_ARG();
+ filter.name = GET_ARG();
+ } else if (is_prefix(*argv, "id")) {
+ NEXT_ARG();
+ errno = 0;
+ id = strtoul(*argv, &endptr, 0);
+ if (errno || *endptr || !id || id > UINT_MAX || **argv == '-') {
+ p_err("can't parse %s as BTF ID", *argv);
+ return -1;
+ }
+ filter.btf_id = id;
+ NEXT_ARG();
+ } else {
+ p_err("expected 'name' or 'id', got: '%s'", *argv);
+ return -1;
+ }
+ if (parse_object(argc, argv, &path, &filter))
+ return -1;
+ /* The BTF IDs of different modules overlap */
+ if (filter.btf_id && !path && !filter.module)
+ filter.vmlinux = true;
+
+ filter.live = !path;
+ if (filter.live)
+ read_error_injection_list();
+ /* A BTF ID gives one function, a name an array of them */
+ if (json_output && !filter.btf_id)
+ jsonw_start_array(json_wtr);
+ err = walk_btf(path, &filter, show_btf);
+ if (!err && !filter.found) {
+ if (filter.name)
+ p_err("no function named %s", filter.name);
+ else
+ p_err("no function with BTF ID %u", filter.btf_id);
+ err = -ENOENT;
+ }
+ if (json_output && !filter.btf_id)
+ jsonw_end_array(json_wtr);
+ free_error_injection_list();
+ return err;
+}
+
+static int do_help(int argc, char **argv)
+{
+ if (json_output) {
+ jsonw_null(json_wtr);
+ return 0;
+ }
+
+ fprintf(stderr,
+ "Usage: %1$s %2$s list [OBJECT]\n"
+ " %1$s %2$s show FUNC [OBJECT]\n"
+ " %1$s %2$s help\n"
+ "\n"
+ " FUNC := { name NAME | id BTF_ID }\n"
+ " OBJECT := { file FILE | vmlinux | module NAME }\n"
+ " " HELP_SPEC_OPTIONS " |\n"
+ " {-B|--base-btf} FILE }\n"
+ "\n",
+ bin_name, "fmodret");
+ return 0;
+}
+
+static const struct cmd cmds[] = {
+ { "list", do_list },
+ { "show", do_show },
+ { "help", do_help },
+ { 0 }
+};
+
+int do_fmodret(int argc, char **argv)
+{
+ return cmd_select(cmds, argc, argv, do_help);
+}
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index d15fec71b64eec..e2a7571b097c7e 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -65,7 +65,7 @@ static int do_help(int argc, char **argv)
" %s batch file FILE\n"
" %s version\n"
"\n"
- " OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | token }\n"
+ " OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | fmodret | token }\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-V|--version} }\n"
"",
@@ -91,6 +91,7 @@ static const struct cmd commands[] = {
{ "gen", do_gen },
{ "struct_ops", do_struct_ops },
{ "iter", do_iter },
+ { "fmodret", do_fmodret },
{ "token", do_token },
{ "version", do_version },
{ 0 }
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index b1eb4b5d991b8a..4d877b803b4765 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -178,6 +178,7 @@ int do_tracelog(int argc, char **arg) __weak;
int do_feature(int argc, char **argv) __weak;
int do_struct_ops(int argc, char **argv) __weak;
int do_iter(int argc, char **argv) __weak;
+int do_fmodret(int argc, char **argv) __weak;
int do_token(int argc, char **argv) __weak;
int parse_u32_arg(int *argc, char ***argv, __u32 *val, const char *what);
diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
index 602e1508bf0922..af777afa451b27 100644
--- a/tools/bpf/resolve_btfids/Build
+++ b/tools/bpf/resolve_btfids/Build
@@ -1,6 +1,7 @@
hostprogs := resolve_btfids
resolve_btfids-y += main.o
+resolve_btfids-y += fmodret.o
resolve_btfids-y += btf_colors.o
resolve_btfids-y += rbtree.o
resolve_btfids-y += zalloc.o
diff --git a/tools/bpf/resolve_btfids/fmodret.c b/tools/bpf/resolve_btfids/fmodret.c
new file mode 100644
index 00000000000000..df44b93eea734c
--- /dev/null
+++ b/tools/bpf/resolve_btfids/fmodret.c
@@ -0,0 +1,1205 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Read the records which the bpf_scan_ir compiler plugin writes to
+ * .BTF_fmodret_candidates, see scripts/llvm-plugins/bpf_scan_ir.cpp, and find
+ * the functions which can become fmod_ret targets. A record lists the errors
+ * which clean paths of a function return, that is paths without side effects.
+ *
+ * A function is chosen if
+ * - exactly one record describes its body, or one record of a strong
+ * definition next to records of the weak ones it overrides,
+ * - the record is of a strong definition or the symbol is still weak,
+ * - the record lists at least one error,
+ * - it has an ftrace call site and is not init, exit or noinstr code,
+ * - its name belongs to exactly one ELF function and one BTF FUNC,
+ * - it returns an integer of the size the record describes.
+ *
+ * The plugin records no errors for functions which return pointers yet.
+ */
+
+#include <errno.h>
+#include <gelf.h>
+#include <libelf.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <bpf/btf.h>
+#include <linux/kernel.h>
+
+#include "fmodret.h"
+#include "../../../scripts/bpf/fmodret_metadata.h"
+
+#define FMODRET_CANDIDATES_SECTION ".BTF_fmodret_candidates"
+#define NO_FUNCTION SIZE_MAX
+
+struct elf_function {
+ const char *name;
+ GElf_Addr value;
+ GElf_Xword size;
+ Elf64_Section shndx;
+ bool traceable;
+ /* The symbol is weak: no strong definition replaced it at the link */
+ bool weak_symbol;
+ /* The records which describe the body at this address */
+ unsigned int nr_strong, nr_weak;
+ size_t strong, weak;
+};
+
+struct elf_function_list {
+ struct elf_function *functions;
+ size_t nr;
+ size_t capacity;
+};
+
+struct fmodret_stats {
+ size_t records, unmapped, with_values;
+ size_t ambiguous_body, not_traceable, ambiguous_name;
+ size_t overridden, no_btf, return_mismatch, added;
+};
+
+static int find_section(Elf *elf, size_t shstrndx, const char *wanted,
+ Elf_Scn **result)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ const char *name;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ return -1;
+ if (!strcmp(name, wanted)) {
+ *result = scn;
+ return 0;
+ }
+ }
+
+ *result = NULL;
+ return 0;
+}
+
+static int find_symbol(Elf *elf, const char *wanted, GElf_Sym *result,
+ bool *found)
+{
+ Elf_Scn *scn = NULL;
+
+ *found = false;
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_symbols;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+ nr_symbols = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_symbols; i++) {
+ const char *name;
+ GElf_Sym sym;
+
+ if (!gelf_getsym(data, i, &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF || !sym.st_name)
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (!name)
+ return -1;
+ if (strcmp(name, wanted))
+ continue;
+ *result = sym;
+ *found = true;
+ return 0;
+ }
+ }
+ return 0;
+}
+
+static int section_for_address(Elf *elf, GElf_Addr address,
+ Elf64_Section *result)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (!(shdr.sh_flags & SHF_EXECINSTR) || address < shdr.sh_addr ||
+ address - shdr.sh_addr >= shdr.sh_size)
+ continue;
+ *result = elf_ndxscn(scn);
+ return 0;
+ }
+ *result = SHN_UNDEF;
+ return 0;
+}
+
+static bool relocation_offset(const GElf_Shdr *target, GElf_Addr offset,
+ size_t width, size_t *result)
+{
+ if (target->sh_size < width)
+ return false;
+ if (offset >= target->sh_addr &&
+ offset - target->sh_addr <= target->sh_size - width) {
+ *result = offset - target->sh_addr;
+ return true;
+ }
+ /* ET_REL relocation offsets are relative to their target section. */
+ if (offset <= target->sh_size - width) {
+ *result = offset;
+ return true;
+ }
+ return false;
+}
+
+static uint64_t read_elf_value(const unsigned char *p, size_t size,
+ unsigned int encoding)
+{
+ uint64_t value = 0;
+ size_t i;
+
+ if (encoding == ELFDATA2LSB) {
+ for (i = 0; i < size; i++)
+ value |= (uint64_t)p[i] << (8 * i);
+ } else {
+ for (i = 0; i < size; i++)
+ value = (value << 8) | p[i];
+ }
+ return value;
+}
+
+static int copy_section_data(Elf_Scn *scn, const GElf_Shdr *shdr,
+ unsigned char **result)
+{
+ Elf_Data *data = NULL;
+ unsigned char *buf;
+
+ buf = calloc(1, shdr->sh_size ?: 1);
+ if (!buf)
+ return -ENOMEM;
+ while ((data = elf_getdata(scn, data))) {
+ if (data->d_off > shdr->sh_size ||
+ data->d_size > shdr->sh_size - data->d_off) {
+ free(buf);
+ return -EINVAL;
+ }
+ memcpy(buf + data->d_off, data->d_buf, data->d_size);
+ }
+ *result = buf;
+ return 0;
+}
+
+static int add_elf_function(struct elf_function_list *list, const char *name,
+ const GElf_Sym *sym)
+{
+ struct elf_function *functions;
+ size_t capacity;
+
+ if (list->nr == list->capacity) {
+ capacity = list->capacity ? 2 * list->capacity : 1024;
+ if (capacity > SIZE_MAX / sizeof(*functions))
+ return -E2BIG;
+ functions = realloc(list->functions, capacity * sizeof(*functions));
+ if (!functions)
+ return -ENOMEM;
+ list->functions = functions;
+ list->capacity = capacity;
+ }
+
+ list->functions[list->nr++] = (struct elf_function) {
+ .name = name,
+ .value = sym->st_value,
+ .size = sym->st_size,
+ .shndx = sym->st_shndx,
+ .weak_symbol = GELF_ST_BIND(sym->st_info) == STB_WEAK,
+ .strong = NO_FUNCTION,
+ .weak = NO_FUNCTION,
+ };
+ return 0;
+}
+
+static int collect_elf_functions(Elf *elf, struct elf_function_list *list)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_symbols;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+
+ nr_symbols = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_symbols; i++) {
+ const char *name;
+ GElf_Sym sym;
+ int err;
+
+ if (!gelf_getsym(data, i, &sym))
+ return -1;
+ if (GELF_ST_TYPE(sym.st_info) != STT_FUNC ||
+ sym.st_shndx == SHN_UNDEF || !sym.st_name)
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (!name)
+ return -1;
+ err = add_elf_function(list, name, &sym);
+ if (err)
+ return err;
+ }
+ }
+ return 0;
+}
+
+static int cmp_elf_function_location(const void *va, const void *vb)
+{
+ const struct elf_function *a = va;
+ const struct elf_function *b = vb;
+
+ if (a->shndx != b->shndx)
+ return a->shndx > b->shndx ? 1 : -1;
+ if (a->value != b->value)
+ return a->value > b->value ? 1 : -1;
+ return 0;
+}
+
+static int cmp_elf_function_name(const void *va, const void *vb)
+{
+ const struct elf_function *a = va;
+ const struct elf_function *b = vb;
+
+ return strcmp(a->name, b->name) ?: cmp_elf_function_location(va, vb);
+}
+
+/* Return the first function at VALUE in SHNDX. Functions are sorted by location. */
+static size_t function_at(const struct elf_function_list *list,
+ Elf64_Section shndx, GElf_Addr value)
+{
+ size_t lo = 0, hi = list->nr;
+
+ while (lo < hi) {
+ size_t mid = lo + (hi - lo) / 2;
+ const struct elf_function *f = &list->functions[mid];
+
+ if (f->shndx < shndx || (f->shndx == shndx && f->value < value))
+ lo = mid + 1;
+ else
+ hi = mid;
+ }
+ if (lo == list->nr || list->functions[lo].shndx != shndx ||
+ list->functions[lo].value != value)
+ return NO_FUNCTION;
+ return lo;
+}
+
+static bool function_contains(const struct elf_function *function,
+ GElf_Addr address)
+{
+ if (address < function->value)
+ return false;
+ if (!function->size)
+ return address == function->value;
+ return address - function->value < function->size;
+}
+
+enum ftrace_site_kind {
+ FTRACE_SITE_MCOUNT,
+ FTRACE_SITE_PATCHABLE,
+ FTRACE_SITE_UNKNOWN,
+};
+
+/*
+ * The NOPs which arm64 puts before a function with CALL_OPS. From the entry of
+ * an overridden weak function, whose body has no symbol left, a longer distance
+ * could reach the next function. Other arches may need more.
+ */
+#define MAX_PATCHABLE_FUNCTION_ENTRY_SIZE 8
+
+/* Return true if function I or another function at its address contains ADDRESS */
+static bool alias_contains(const struct elf_function_list *list, size_t i,
+ GElf_Addr address)
+{
+ const struct elf_function *f = &list->functions[i];
+
+ for (;;) {
+ if (function_contains(&list->functions[i], address))
+ return true;
+ if (!i || list->functions[i - 1].shndx != f->shndx ||
+ list->functions[i - 1].value != f->value)
+ return false;
+ i--;
+ }
+}
+
+/*
+ * Mark the functions at (SHNDX, VALUE) traceable, or, with CHECK, only those
+ * which contain ADDRESS.
+ */
+static void mark_aliases(struct elf_function_list *list, Elf64_Section shndx,
+ GElf_Addr value, GElf_Addr address, bool check)
+{
+ size_t first = function_at(list, shndx, value), i;
+
+ if (first == NO_FUNCTION)
+ return;
+ for (i = first; i < list->nr && list->functions[i].shndx == shndx &&
+ list->functions[i].value == value; i++) {
+ if (check && !function_contains(&list->functions[i], address))
+ continue;
+ list->functions[i].traceable = true;
+ }
+}
+
+/*
+ * Mark the function containing an ftrace call site. Functions are sorted by
+ * section and address. Mark all aliases at the same address: at run time BTF
+ * attachment is name based and may resolve to any of them.
+ */
+static void mark_traceable_function(struct elf_function_list *list,
+ Elf64_Section shndx, GElf_Addr address,
+ enum ftrace_site_kind kind)
+{
+ size_t left = 0;
+ size_t right = list->nr;
+ size_t first;
+
+ /* Find the first function after ADDRESS in SHNDX. */
+ while (left < right) {
+ size_t middle = left + (right - left) / 2;
+ struct elf_function *function = &list->functions[middle];
+
+ if (function->shndx < shndx ||
+ (function->shndx == shndx && function->value <= address))
+ left = middle + 1;
+ else
+ right = middle;
+ }
+ /*
+ * A patchable entry is at the start of its function, a few bytes into it
+ * (after an arm64 BTI) or in the NOPs before it
+ */
+ first = SIZE_MAX;
+ if (kind != FTRACE_SITE_MCOUNT) {
+ if (left && list->functions[left - 1].shndx == shndx &&
+ address - list->functions[left - 1].value <
+ MAX_PATCHABLE_FUNCTION_ENTRY_SIZE &&
+ alias_contains(list, left - 1, address))
+ first = left - 1;
+ else if (left < list->nr && list->functions[left].shndx == shndx &&
+ list->functions[left].value - address <=
+ MAX_PATCHABLE_FUNCTION_ENTRY_SIZE)
+ first = left;
+ }
+ if (first != SIZE_MAX) {
+ mark_aliases(list, shndx, list->functions[first].value, address, false);
+ return;
+ }
+ if (kind == FTRACE_SITE_PATCHABLE || !left)
+ return;
+
+ /* The nearest preceding function owns the call site. */
+ first = left - 1;
+ if (list->functions[first].shndx != shndx)
+ return;
+ mark_aliases(list, shndx, list->functions[first].value, address, true);
+}
+
+static int mark_relocated_ftrace_sites(Elf *elf, size_t target_ndx,
+ const unsigned char *target_data,
+ const GElf_Shdr *target_shdr,
+ size_t begin, size_t end,
+ size_t ptr_size, unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind,
+ bool *relocated)
+{
+ Elf_Scn *scn = NULL;
+
+ *relocated = false;
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_relocations;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if ((shdr.sh_type != SHT_RELA && shdr.sh_type != SHT_REL) ||
+ (shdr.sh_info && shdr.sh_info != target_ndx))
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+
+ nr_relocations = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_relocations; i++) {
+ Elf_Scn *symtab_scn;
+ Elf_Data *symtab_data;
+ GElf_Shdr symtab_shdr;
+ GElf_Sxword addend;
+ GElf_Addr raw_offset;
+ GElf_Addr address;
+ GElf_Sym sym;
+ size_t sym_ndx;
+ size_t offset;
+
+ if (shdr.sh_type == SHT_RELA) {
+ GElf_Rela rela;
+
+ if (!gelf_getrela(data, i, &rela))
+ return -1;
+ sym_ndx = GELF_R_SYM(rela.r_info);
+ raw_offset = rela.r_offset;
+ addend = rela.r_addend;
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ return -1;
+ sym_ndx = GELF_R_SYM(rel.r_info);
+ raw_offset = rel.r_offset;
+ if (!relocation_offset(target_shdr, raw_offset,
+ ptr_size, &offset))
+ continue;
+ addend = read_elf_value(target_data + offset,
+ ptr_size, encoding);
+ }
+ if (!relocation_offset(target_shdr, raw_offset, ptr_size,
+ &offset))
+ continue;
+ if (offset < begin || offset >= end)
+ continue;
+ *relocated = true;
+
+ /*
+ * PIE vmlinux uses symbol-less RELATIVE relocations. Its
+ * addend is the link-time call-site address, represented as
+ * a signed ELF addend.
+ */
+ if (!sym_ndx) {
+ Elf64_Section target_shndx;
+
+ address = (GElf_Addr)addend;
+ if (section_for_address(elf, address,
+ &target_shndx))
+ return -1;
+ if (target_shndx != SHN_UNDEF)
+ mark_traceable_function(functions,
+ target_shndx,
+ address, kind);
+ continue;
+ }
+
+ symtab_scn = elf_getscn(elf, shdr.sh_link);
+ if (!symtab_scn ||
+ !gelf_getshdr(symtab_scn, &symtab_shdr))
+ return -1;
+ symtab_data = elf_getdata(symtab_scn, NULL);
+ if (!symtab_data || elf_getdata(symtab_scn, symtab_data) ||
+ !gelf_getsym(symtab_data, sym_ndx, &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF)
+ continue;
+ if (sym.st_shndx >= SHN_LORESERVE)
+ continue;
+ if (addend < 0) {
+ GElf_Addr delta = -(addend + 1);
+
+ delta++;
+ if (delta > sym.st_value)
+ continue;
+ address = sym.st_value - delta;
+ } else {
+ if ((GElf_Addr)addend > UINT64_MAX - sym.st_value)
+ continue;
+ address = sym.st_value + addend;
+ }
+ mark_traceable_function(functions, sym.st_shndx, address,
+ kind);
+ }
+ }
+ return 0;
+}
+
+static int read_ftrace_site_range(Elf *elf, Elf_Scn *scn, size_t begin,
+ size_t end, size_t ptr_size,
+ unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind)
+{
+ unsigned char *buf = NULL;
+ GElf_Shdr shdr;
+ bool relocated;
+ size_t offset;
+ int err;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (begin > end || end > shdr.sh_size ||
+ (end - begin) % ptr_size)
+ return -EINVAL;
+ if (begin == end)
+ return 0;
+ err = copy_section_data(scn, &shdr, &buf);
+ if (err)
+ return err;
+ err = mark_relocated_ftrace_sites(elf, elf_ndxscn(scn), buf, &shdr,
+ begin, end, ptr_size, encoding,
+ functions, kind, &relocated);
+ if (err || relocated)
+ goto out;
+
+ for (offset = begin; offset < end; offset += ptr_size) {
+ GElf_Addr address;
+ Elf64_Section shndx;
+
+ address = read_elf_value(buf + offset, ptr_size, encoding);
+ err = section_for_address(elf, address, &shndx);
+ if (err)
+ goto out;
+ if (shndx != SHN_UNDEF)
+ mark_traceable_function(functions, shndx, address, kind);
+ }
+out:
+ free(buf);
+ return err;
+}
+
+static bool section_prefix(const char *name, const char *prefix)
+{
+ size_t len = strlen(prefix);
+
+ return !strncmp(name, prefix, len) &&
+ (!name[len] || name[len] == '.');
+}
+
+static bool section_cannot_be_attached(const char *name)
+{
+ return section_prefix(name, ".init.text") ||
+ section_prefix(name, ".exit.text") ||
+ section_prefix(name, ".noinstr.text");
+}
+
+static int remove_unattachable_functions(Elf *elf,
+ struct elf_function_list *functions,
+ size_t shstrndx)
+{
+ GElf_Sym noinstr_start;
+ GElf_Sym noinstr_end;
+ bool found_start;
+ bool found_end;
+ size_t i;
+ int err;
+
+ err = find_symbol(elf, "__noinstr_text_start", &noinstr_start,
+ &found_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__noinstr_text_end", &noinstr_end, &found_end);
+ if (err)
+ return err;
+
+ for (i = 0; i < functions->nr; i++) {
+ struct elf_function *function = &functions->functions[i];
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ const char *name;
+
+ if (!function->traceable)
+ continue;
+ scn = elf_getscn(elf, function->shndx);
+ if (!scn || !gelf_getshdr(scn, &shdr))
+ return -1;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ return -1;
+ if (section_cannot_be_attached(name))
+ function->traceable = false;
+ else if (found_start && found_end &&
+ function->shndx == noinstr_start.st_shndx &&
+ noinstr_start.st_shndx == noinstr_end.st_shndx &&
+ function->value >= noinstr_start.st_value &&
+ function->value < noinstr_end.st_value)
+ function->traceable = false;
+ }
+ return 0;
+}
+
+static int read_ftrace_symbol_range(Elf *elf, const GElf_Sym *start,
+ const GElf_Sym *stop, size_t ptr_size,
+ unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind)
+{
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ size_t begin;
+ size_t end;
+
+ if (start->st_shndx != stop->st_shndx)
+ return -EINVAL;
+ scn = elf_getscn(elf, start->st_shndx);
+ if (!scn || !gelf_getshdr(scn, &shdr))
+ return -1;
+ if (start->st_value < shdr.sh_addr || stop->st_value < start->st_value ||
+ stop->st_value - shdr.sh_addr > shdr.sh_size)
+ return -EINVAL;
+ begin = start->st_value - shdr.sh_addr;
+ end = stop->st_value - shdr.sh_addr;
+ return read_ftrace_site_range(elf, scn, begin, end, ptr_size, encoding,
+ functions, kind);
+}
+
+/*
+ * Mark the functions with an ftrace call site. A module keeps the sites in the
+ * section FTRACE_CALLSITE_SECTION: __patchable_function_entries if PATCHABLE,
+ * else __mcount_loc. x86 also has call padding entries, which are not call
+ * sites, in __patchable_function_entries. vmlinux merges the sites into init
+ * data between the __start_mcount_loc and __stop_mcount_loc symbols, see
+ * MCOUNT_REC().
+ */
+static int read_ftrace_sites(Elf *elf, size_t shstrndx,
+ size_t ptr_size, unsigned int encoding,
+ struct elf_function_list *functions, bool patchable)
+{
+ const char *section = patchable ? "__patchable_function_entries" : "__mcount_loc";
+ Elf_Scn *scn = NULL;
+ GElf_Sym start;
+ GElf_Sym stop;
+ GElf_Sym patchable_start;
+ GElf_Sym patchable_stop;
+ bool found_start;
+ bool found_stop;
+ bool found_patchable_start;
+ bool found_patchable_stop;
+ GElf_Shdr shdr;
+ int err;
+
+ err = find_section(elf, shstrndx, section, &scn);
+ if (err)
+ return err;
+ if (scn) {
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ err = read_ftrace_site_range(elf, scn, 0, shdr.sh_size,
+ ptr_size, encoding, functions,
+ patchable ? FTRACE_SITE_PATCHABLE :
+ FTRACE_SITE_MCOUNT);
+ if (err)
+ return err;
+ }
+
+ err = find_symbol(elf, "__start_mcount_loc", &start, &found_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__stop_mcount_loc", &stop, &found_stop);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__start_patchable_function_entries",
+ &patchable_start, &found_patchable_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__stop_patchable_function_entries",
+ &patchable_stop, &found_patchable_stop);
+ if (err)
+ return err;
+ if (found_start && found_stop) {
+ if (found_patchable_start && found_patchable_stop) {
+ err = read_ftrace_symbol_range(elf, &start,
+ &patchable_start,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_MCOUNT);
+ if (err)
+ return err;
+ err = read_ftrace_symbol_range(elf, &patchable_start,
+ &patchable_stop,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_PATCHABLE);
+ } else {
+ err = read_ftrace_symbol_range(elf, &start, &stop,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_UNKNOWN);
+ }
+ if (err)
+ return err;
+ }
+
+ return remove_unattachable_functions(elf, functions, shstrndx);
+}
+
+/*
+ * Find the function each record points to. In a module the pointer is a
+ * relocation, in a linked image it is the address. REFS maps each pointer
+ * sized slot of the section to a function.
+ */
+static int resolve_record_relocations(Elf *elf, const struct elf_function_list *functions,
+ size_t metadata_ndx, const GElf_Shdr *metadata_shdr,
+ const unsigned char *metadata, size_t *refs,
+ size_t ptr_size, unsigned int encoding)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ Elf_Data *data, *symbols;
+ Elf_Scn *symtab;
+ GElf_Shdr shdr;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if ((shdr.sh_type != SHT_RELA && shdr.sh_type != SHT_REL) ||
+ shdr.sh_info != metadata_ndx)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ symtab = elf_getscn(elf, shdr.sh_link);
+ symbols = symtab ? elf_getdata(symtab, NULL) : NULL;
+ if (!data || !symbols || elf_getdata(scn, data))
+ return -1;
+ for (i = 0; i < shdr.sh_size / shdr.sh_entsize; i++) {
+ GElf_Rela rela;
+ GElf_Sym sym;
+ size_t offset;
+
+ if (shdr.sh_type == SHT_RELA) {
+ if (!gelf_getrela(data, i, &rela))
+ return -1;
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ return -1;
+ rela = (GElf_Rela){ .r_offset = rel.r_offset,
+ .r_info = rel.r_info };
+ /* REL addends are stored in the pointer itself. */
+ if (!relocation_offset(metadata_shdr, rel.r_offset, ptr_size,
+ &offset))
+ return -EINVAL;
+ rela.r_addend = read_elf_value(metadata + offset, ptr_size,
+ encoding);
+ }
+ if (!relocation_offset(metadata_shdr, rela.r_offset, ptr_size, &offset) ||
+ offset % ptr_size)
+ return -EINVAL;
+ if (!gelf_getsym(symbols, GELF_R_SYM(rela.r_info), &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF || sym.st_shndx >= SHN_LORESERVE)
+ continue;
+ refs[offset / ptr_size] = function_at(functions, sym.st_shndx,
+ sym.st_value + rela.r_addend);
+ }
+ }
+ return 0;
+}
+
+static size_t record_function(Elf *elf, const struct elf_function_list *functions,
+ const unsigned char *buf, const size_t *refs,
+ size_t offset, size_t ptr_size, unsigned int encoding)
+{
+ Elf64_Section section;
+ GElf_Ehdr ehdr;
+ GElf_Addr value;
+
+ if (refs[offset / ptr_size] != NO_FUNCTION)
+ return refs[offset / ptr_size];
+ if (!gelf_getehdr(elf, &ehdr) || ehdr.e_type == ET_REL)
+ return NO_FUNCTION;
+ value = read_elf_value(buf + offset, ptr_size, encoding);
+ if (!value || section_for_address(elf, value, §ion))
+ return NO_FUNCTION;
+ return function_at(functions, section, value);
+}
+
+/* Remember the records of a body for all the functions at its address */
+static void add_record(struct elf_function_list *functions, size_t first,
+ size_t offset, bool weak)
+{
+ struct elf_function *f = &functions->functions[first];
+ size_t i;
+
+ for (i = first; i < functions->nr &&
+ !cmp_elf_function_location(f, &functions->functions[i]); i++) {
+ if (weak) {
+ functions->functions[i].nr_weak++;
+ functions->functions[i].weak = offset;
+ } else {
+ functions->functions[i].nr_strong++;
+ functions->functions[i].strong = offset;
+ }
+ }
+}
+
+static int parse_records(Elf *elf, struct elf_function_list *functions,
+ const unsigned char *buf, size_t size, const size_t *refs,
+ size_t ptr_size, unsigned int encoding,
+ struct fmodret_stats *stats)
+{
+ size_t offset = 0;
+
+ while (offset < size) {
+ unsigned int flags, length, nr_values;
+ size_t function;
+
+ if (size - offset < FMODRET_META_HEADER_SIZE(ptr_size) ||
+ read_elf_value(buf + offset, 4, encoding) != FMODRET_META_MAGIC ||
+ read_elf_value(buf + offset + 4, 4, encoding) != FMODRET_META_VERSION)
+ goto invalid;
+ length = read_elf_value(buf + offset + 8, 4, encoding);
+ flags = read_elf_value(buf + offset + 12, 4, encoding);
+ nr_values = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);
+ if (length < FMODRET_META_HEADER_SIZE(ptr_size) || length > size - offset ||
+ length % ptr_size ||
+ nr_values != (length - FMODRET_META_HEADER_SIZE(ptr_size)) /
+ FMODRET_META_VALUE_SIZE ||
+ (length - FMODRET_META_HEADER_SIZE(ptr_size)) % FMODRET_META_VALUE_SIZE)
+ goto invalid;
+ stats->records++;
+ function = record_function(elf, functions, buf, refs, offset + 16,
+ ptr_size, encoding);
+ /* A function in a discarded section has no ELF function */
+ if (function == NO_FUNCTION)
+ stats->unmapped++;
+ else
+ add_record(functions, function, offset, flags & FMODRET_FN_WEAK);
+ offset += length;
+ }
+ return 0;
+invalid:
+ fprintf(stderr,
+ "invalid or old fmodret metadata at offset %#zx; rebuild all plugin objects\n",
+ offset);
+ return -EINVAL;
+}
+
+/* Return the record which describes the body of F, or NO_FUNCTION */
+static size_t chosen_record(const struct elf_function *f)
+{
+ if (f->nr_strong == 1)
+ return f->strong;
+ if (!f->nr_strong && f->nr_weak == 1)
+ return f->weak;
+ return NO_FUNCTION;
+}
+
+struct btf_func {
+ const char *name;
+ __u32 id;
+};
+
+static int cmp_btf_func(const void *a, const void *b)
+{
+ const struct btf_func *x = a, *y = b;
+
+ return strcmp(x->name, y->name);
+}
+
+/*
+ * Return the FUNCs of BTF, without those of its base, sorted by name. The BTF
+ * is sorted only after the tags are added, and searching all of it for each
+ * candidate is slow for vmlinux.
+ */
+static struct btf_func *collect_btf_funcs(const struct btf *btf, size_t *cnt)
+{
+ const struct btf *base = btf__base_btf(btf);
+ __u32 id, nr_types = btf__type_cnt(btf);
+ struct btf_func *funcs;
+ size_t n = 0;
+
+ funcs = malloc(nr_types * sizeof(*funcs));
+ if (!funcs)
+ return NULL;
+ for (id = base ? btf__type_cnt(base) : 1; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_func(t))
+ continue;
+ funcs[n].name = btf__name_by_offset(btf, t->name_off);
+ funcs[n++].id = id;
+ }
+ qsort(funcs, n, sizeof(*funcs), cmp_btf_func);
+ *cnt = n;
+ return funcs;
+}
+
+/* Return the BTF ID of the only FUNC named NAME, or 0 */
+static int find_btf_func(const struct btf_func *funcs, size_t cnt, const char *name)
+{
+ const struct btf_func key = { .name = name }, *f;
+
+ f = bsearch(&key, funcs, cnt, sizeof(*funcs), cmp_btf_func);
+ if (!f)
+ return 0;
+ if ((f > funcs && !strcmp(f[-1].name, name)) ||
+ (f + 1 < funcs + cnt && !strcmp(f[1].name, name)))
+ return 0;
+ return f->id;
+}
+
+/* Return the size of the integer which FUNC returns, or 0 */
+static __u32 btf_func_int_return_size(const struct btf *btf, int func)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func);
+ int ret;
+
+ t = btf__type_by_id(btf, t->type);
+ if (!t || !btf_is_func_proto(t) || !t->type)
+ return 0;
+ ret = btf__resolve_type(btf, t->type);
+ if (ret < 0)
+ return 0;
+ t = btf__type_by_id(btf, ret);
+ if (!btf_is_int(t) && !btf_is_enum(t) && !btf_is_enum64(t))
+ return 0;
+ return t->size;
+}
+
+/* Format the values of the record at OFFSET as "-22,-12" */
+static char *format_values(const unsigned char *buf, size_t offset, size_t ptr_size,
+ unsigned int encoding)
+{
+ unsigned int nr = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);
+ const unsigned char *p = buf + offset + FMODRET_META_HEADER_SIZE(ptr_size);
+ size_t size = (size_t)nr * 21 + 1, len = 0;
+ char *s = malloc(size);
+ unsigned int i;
+
+ if (!s)
+ return NULL;
+ for (i = 0; i < nr; i++) {
+ int64_t value = read_elf_value(p + i * FMODRET_META_VALUE_SIZE,
+ FMODRET_META_VALUE_SIZE, encoding);
+
+ len += snprintf(s + len, size - len, "%s%lld", i ? "," : "",
+ (long long)value);
+ }
+ s[len] = 0;
+ return s;
+}
+
+static int cmp_retval(const void *va, const void *vb)
+{
+ const struct fmodret_retval *a = va;
+ const struct fmodret_retval *b = vb;
+
+ return a->btf_id > b->btf_id ? 1 : a->btf_id < b->btf_id ? -1 : 0;
+}
+
+/* Choose the targets among the functions and describe them in RESULT */
+static int choose_functions(struct elf_function_list *functions, const struct btf *btf,
+ const struct btf_func *btf_funcs, size_t nr_btf_funcs,
+ const unsigned char *buf, size_t ptr_size,
+ unsigned int encoding, struct fmodret_retvals *result,
+ struct fmodret_stats *stats, int verbose)
+{
+ size_t i = 0, j;
+
+ qsort(functions->functions, functions->nr, sizeof(*functions->functions),
+ cmp_elf_function_name);
+ for (i = 0; i < functions->nr; i = j) {
+ const struct elf_function *f = &functions->functions[i];
+ size_t record = chosen_record(f);
+ const char *reason = NULL;
+ unsigned int flags = 0;
+ struct fmodret_retval *r;
+ __u32 size;
+ int id = 0;
+
+ for (j = i + 1; j < functions->nr; j++)
+ if (strcmp(f->name, functions->functions[j].name))
+ break;
+
+ if (!f->nr_strong && !f->nr_weak)
+ continue;
+ if (record == NO_FUNCTION) {
+ stats->ambiguous_body++;
+ reason = "ambiguous-body";
+ goto rejected;
+ }
+ /*
+ * A strong definition without values has no record. If it
+ * replaced a weak one, only the record of the weak one points
+ * to its body.
+ */
+ if (!f->nr_strong && !f->weak_symbol) {
+ stats->overridden++;
+ reason = "overridden-weak";
+ goto rejected;
+ }
+ if (!read_elf_value(buf + record + 16 + ptr_size, 4, encoding))
+ continue;
+ stats->with_values++;
+ flags = read_elf_value(buf + record + 12, 4, encoding);
+ if (j - i != 1) {
+ stats->ambiguous_name++;
+ reason = "ambiguous-name";
+ goto rejected;
+ }
+ if (!f->traceable) {
+ stats->not_traceable++;
+ reason = "not-traceable";
+ goto rejected;
+ }
+ id = find_btf_func(btf_funcs, nr_btf_funcs, f->name);
+ if (id <= 0) {
+ stats->no_btf++;
+ reason = "no-unique-BTF-FUNC";
+ goto rejected;
+ }
+ size = btf_func_int_return_size(btf, id);
+ if (size != (flags & FMODRET_FN_RET64 ? 8 : 4)) {
+ stats->return_mismatch++;
+ reason = "BTF-return-type";
+ }
+rejected:
+ if (reason) {
+ if (verbose > 1)
+ fprintf(stderr, "fmodret: rejected %s: %s\n", f->name, reason);
+ continue;
+ }
+
+ if (result->cnt == result->cap) {
+ size_t cap = result->cap ? 2 * result->cap : 256;
+
+ r = realloc(result->retvals, cap * sizeof(*r));
+ if (!r)
+ return -ENOMEM;
+ result->retvals = r;
+ result->cap = cap;
+ }
+ r = &result->retvals[result->cnt];
+ r->btf_id = id;
+ r->values = format_values(buf, record, ptr_size, encoding);
+ if (!r->values)
+ return -ENOMEM;
+ result->cnt++;
+ stats->added++;
+ if (verbose > 1)
+ fprintf(stderr, "fmodret: %s: BTF ID %d, retval %s\n",
+ f->name, id, r->values);
+ }
+ if (result->cnt)
+ qsort(result->retvals, result->cnt, sizeof(*result->retvals), cmp_retval);
+ return 0;
+}
+
+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,
+ bool patchable_ftrace, int verbose)
+{
+ struct elf_function_list functions = {};
+ struct btf_func *btf_funcs = NULL;
+ struct fmodret_stats stats = {};
+ unsigned char *buf = NULL;
+ size_t nr_btf_funcs;
+ size_t *refs = NULL;
+ size_t shstrndx, ptr_size, i;
+ unsigned int encoding;
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ int err;
+
+ if (elf_getshdrstrndx(elf, &shstrndx))
+ return -EINVAL;
+ err = find_section(elf, shstrndx, FMODRET_CANDIDATES_SECTION, &scn);
+ if (err || !scn)
+ return err;
+ if (!gelf_getshdr(scn, &shdr))
+ return -EINVAL;
+ ptr_size = gelf_getclass(elf) == ELFCLASS64 ? 8 : 4;
+ encoding = elf_getident(elf, NULL)[EI_DATA];
+ if (shdr.sh_size % ptr_size)
+ return -EINVAL;
+
+ err = collect_elf_functions(elf, &functions);
+ if (err)
+ goto out;
+ qsort(functions.functions, functions.nr, sizeof(*functions.functions),
+ cmp_elf_function_location);
+
+ err = copy_section_data(scn, &shdr, &buf);
+ if (err)
+ goto out;
+ refs = malloc((shdr.sh_size / ptr_size ?: 1) * sizeof(*refs));
+ if (!refs) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < shdr.sh_size / ptr_size; i++)
+ refs[i] = NO_FUNCTION;
+ err = resolve_record_relocations(elf, &functions, elf_ndxscn(scn), &shdr, buf,
+ refs, ptr_size, encoding);
+ if (err)
+ goto out;
+ err = parse_records(elf, &functions, buf, shdr.sh_size, refs, ptr_size,
+ encoding, &stats);
+ if (err)
+ goto out;
+ err = read_ftrace_sites(elf, shstrndx, ptr_size, encoding, &functions,
+ patchable_ftrace);
+ if (err)
+ goto out;
+ btf_funcs = collect_btf_funcs(btf, &nr_btf_funcs);
+ if (!btf_funcs) {
+ err = -ENOMEM;
+ goto out;
+ }
+ err = choose_functions(&functions, btf, btf_funcs, nr_btf_funcs, buf, ptr_size,
+ encoding, result, &stats, verbose);
+ if (err)
+ goto out;
+
+ if (verbose)
+ fprintf(stderr,
+ "fmodret: %zu records (%zu without a function), %zu functions with values, %zu added; rejected: %zu ambiguous body, %zu ambiguous name, %zu not traceable, %zu overridden weak, %zu without unique BTF FUNC, %zu BTF return type\n",
+ stats.records, stats.unmapped, stats.with_values, stats.added,
+ stats.ambiguous_body,
+ stats.ambiguous_name, stats.not_traceable, stats.overridden,
+ stats.no_btf, stats.return_mismatch);
+out:
+ if (err == -1)
+ fprintf(stderr, "failed to read %s: %s\n", FMODRET_CANDIDATES_SECTION,
+ elf_errmsg(-1));
+ free(btf_funcs);
+ free(refs);
+ free(buf);
+ free(functions.functions);
+ return err;
+}
+
+void fmodret_free(struct fmodret_retvals *retvals)
+{
+ size_t i;
+
+ for (i = 0; i < retvals->cnt; i++)
+ free(retvals->retvals[i].values);
+ free(retvals->retvals);
+ retvals->retvals = NULL;
+ retvals->cnt = retvals->cap = 0;
+}
diff --git a/tools/bpf/resolve_btfids/fmodret.h b/tools/bpf/resolve_btfids/fmodret.h
new file mode 100644
index 00000000000000..c1d5db64970b62
--- /dev/null
+++ b/tools/bpf/resolve_btfids/fmodret.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __RESOLVE_BTFIDS_FMODRET_H
+#define __RESOLVE_BTFIDS_FMODRET_H
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <libelf.h>
+#include <linux/types.h>
+
+struct btf;
+
+/* A function found by the analysis, and the values it may return */
+struct fmodret_retval {
+ __u32 btf_id;
+ char *values;
+};
+
+/* Sorted by BTF ID */
+struct fmodret_retvals {
+ struct fmodret_retval *retvals;
+ size_t cnt, cap;
+};
+
+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,
+ bool patchable_ftrace, int verbose);
+void fmodret_free(struct fmodret_retvals *retvals);
+
+#endif
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index c16dd93fa6d4c2..124c4f01c8878f 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -69,6 +69,18 @@
* - rewrites the prototype of KF_IMPLICIT_ARGS kfuncs.
*
* These kfunc annotations were historically produced by pahole.
+ *
+ * Functions found in BTF_SET8_FMODRET sets are the targets of BPF fmod_ret
+ * programs. For each of them resolve_btfids emits a "bpf_fmodret" decl tag,
+ * and also "bpf_fmodret_sleepable" when KF_SLEEPABLE is set, so the sleepable
+ * tag never comes alone. The kernel builds the set of fmod_ret targets from
+ * these tags when it loads BTF.
+ *
+ * Functions which the analysis of the compiler plugin records, see fmodret.c,
+ * get "bpf_fmodret" too, and "bpf_fmodret_retval:" with the values they
+ * return on paths without side effects, such as "bpf_fmodret_retval:-22,-12".
+ * The verifier lets programs attached to them return only these values or 0.
+ * A function in a BTF_FMODRET set keeps only its own tags.
*/
#define _GNU_SOURCE
@@ -90,6 +102,7 @@
#include <bpf/btf.h>
#include <bpf/libbpf.h>
#include "btf_colors.h"
+#include "fmodret.h"
#include <subcmd/parse-options.h>
#define BTF_IDS_SECTION ".BTF_ids"
@@ -148,6 +161,11 @@ struct object {
struct btf *orig_btf;
bool distill_base;
bool extract_inline;
+ /* The ftrace call sites are in __patchable_function_entries */
+ bool patchable_ftrace;
+
+ /* Functions found by the analysis of the compiler plugin, see fmodret.c */
+ struct fmodret_retvals fmodret_retvals;
struct {
int fd;
@@ -197,7 +215,11 @@ struct btf_links {
#define DECL_TAG_FASTCALL "bpf_fastcall"
#define DECL_TAG_KFUNC "bpf_kfunc"
+#define DECL_TAG_FMODRET "bpf_fmodret"
+#define DECL_TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+#define DECL_TAG_FMODRET_RETVAL "bpf_fmodret_retval:"
+#define KF_SLEEPABLE (1 << 5)
#define KF_FASTCALL (1 << 12)
#define KF_ARENA_RET (1 << 13)
#define KF_ARENA_ARG1 (1 << 14)
@@ -208,6 +230,7 @@ struct btf_links {
#define PARAM_SUFFIX_ARENA "__arena"
#define PARAM_SUFFIX_ARENA_NULLABLE "__arena__nullable"
+/* A function found in a BTF ID set: a kfunc or an fmod_ret target */
struct kfunc {
struct rb_node rb_node;
const char *name;
@@ -221,6 +244,7 @@ struct btf2btf_context {
u32 nr_decl_tags;
u32 max_decl_tags;
struct rb_root kfuncs;
+ struct rb_root fmodrets;
};
static int verbose;
@@ -1256,16 +1280,16 @@ static int push_decl_tag_id(struct btf2btf_context *ctx, u32 decl_tag_id)
return 0;
}
-static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)
+static int push_set_func(struct rb_root *root, const char *kind, struct kfunc *kfunc)
{
- struct rb_node **p = &ctx->kfuncs.rb_node;
+ struct rb_node **p = &root->rb_node;
struct rb_node *parent = NULL;
struct kfunc *k;
/*
- * Dedup by BTF ID: collecting the same kfunc twice is a no-op,
- * UNLESS the kfunc flags are inconsistent, in which case we
- * fail hard because it indicates a bug in a kfunc set declaration.
+ * Dedup by BTF ID: collecting the same function twice is a no-op,
+ * UNLESS its flags are inconsistent, in which case we fail hard
+ * because it indicates a bug in a BTF ID set declaration.
*/
while (*p) {
parent = *p;
@@ -1278,8 +1302,8 @@ static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)
} else if (k->flags == kfunc->flags) {
return 0;
} else {
- pr_err("ERROR: resolve_btfids: kfunc %s has inconsistent flags across BTF ID sets: 0x%x != 0x%x\n",
- kfunc->name, k->flags, kfunc->flags);
+ pr_err("ERROR: resolve_btfids: %s %s has inconsistent flags across BTF ID sets: 0x%x != 0x%x\n",
+ kind, kfunc->name, k->flags, kfunc->flags);
return -EINVAL;
}
}
@@ -1290,7 +1314,7 @@ static int push_kfunc(struct btf2btf_context *ctx, struct kfunc *kfunc)
*k = *kfunc;
rb_link_node(&k->rb_node, parent, p);
- rb_insert_color(&k->rb_node, &ctx->kfuncs);
+ rb_insert_color(&k->rb_node, root);
return 0;
}
@@ -1344,7 +1368,16 @@ static bool is_arena_param(const struct btf *btf, const struct btf_param *param)
param_name_has_suffix(name, PARAM_SUFFIX_ARENA_NULLABLE);
}
-static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
+typedef int (*set_func_fn)(struct btf2btf_context *ctx, const char *name,
+ u32 btf_id, u32 flags);
+
+/*
+ * Call FN for each function in the BTF ID sets which have SET_FLAG set. Reject
+ * an entry with flags other than ALLOWED_FLAGS, also if its function is missing.
+ */
+static int collect_set_funcs(struct object *obj, struct btf2btf_context *ctx,
+ u32 set_flag, u32 allowed_flags, const char *kind,
+ set_func_fn fn)
{
Elf_Data *idlist = obj->efile.idlist;
struct btf *btf = ctx->btf;
@@ -1356,24 +1389,39 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
for (next = rb_first(&obj->sets); next; next = rb_next(next)) {
struct btf_id_set8 *set8;
struct btf_id *set_id;
- u64 set_addr;
+ u64 set_addr, off;
set_id = rb_entry(next, struct btf_id, rb_node);
if (set_id->kind != BTF_ID_KIND_SET8 || set_id->addr_cnt != 1)
continue;
set_addr = set_id->addr[0];
- set8 = idlist->d_buf + (set_addr - obj->efile.idlist_addr);
- if (!(set8->flags & BTF_SET8_KFUNCS))
+ off = set_addr - obj->efile.idlist_addr;
+ /* A malformed set must not reach out of the section */
+ if (set_addr < obj->efile.idlist_addr || off > idlist->d_size ||
+ idlist->d_size - off < sizeof(*set8) || set_id->cnt < 0 ||
+ (u64)set_id->cnt > (idlist->d_size - off - sizeof(*set8)) /
+ sizeof(set8->pairs[0])) {
+ pr_err("ERROR: resolve_btfids: set %s is out of %s\n",
+ set_id->name, BTF_IDS_SECTION);
+ return -EINVAL;
+ }
+ set8 = idlist->d_buf + off;
+ if (!(set8->flags & set_flag))
continue;
for (u32 i = 0; i < set_id->cnt; i++) {
size_t off = (char *)&set8->pairs[i] - (char *)set8;
const char *name = find_name_by_addr(obj, set_addr + off);
- struct kfunc kfunc;
+ u32 flags = set8->pairs[i].flags;
s32 func_id;
int err;
+ if (flags & ~allowed_flags) {
+ pr_err("ERROR: resolve_btfids: %s %s in %s has unsupported flags 0x%x\n",
+ kind, name ?: "?", set_id->name, flags & ~allowed_flags);
+ return -EINVAL;
+ }
if (!name) {
pr_err("WARN: resolve_btfids: no BTF ID symbol for %s entry %u\n",
set_id->name, i);
@@ -1383,18 +1431,15 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
func_id = btf__find_by_name_kind_own(btf, name, BTF_KIND_FUNC);
if (func_id < 0) {
- pr_err("WARN: resolve_btfids: no BTF func for kfunc %s in %s\n",
- name, set_id->name);
+ pr_err("WARN: resolve_btfids: no BTF func for %s %s in %s\n",
+ kind, name, set_id->name);
warnings++;
continue;
}
- pr_debug("found kfunc %s in %s\n", name, set_id->name);
+ pr_debug("found %s %s in %s\n", kind, name, set_id->name);
- kfunc.name = name;
- kfunc.btf_id = func_id;
- kfunc.flags = set8->pairs[i].flags;
- err = push_kfunc(ctx, &kfunc);
+ err = fn(ctx, name, func_id, flags);
if (err)
return err;
}
@@ -1403,6 +1448,33 @@ static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
return 0;
}
+static int push_kfunc(struct btf2btf_context *ctx, const char *name,
+ u32 btf_id, u32 flags)
+{
+ struct kfunc kfunc = { .name = name, .btf_id = btf_id, .flags = flags };
+
+ return push_set_func(&ctx->kfuncs, "kfunc", &kfunc);
+}
+
+static int push_fmodret(struct btf2btf_context *ctx, const char *name,
+ u32 btf_id, u32 flags)
+{
+ struct kfunc fmodret = { .name = name, .btf_id = btf_id, .flags = flags };
+
+ return push_set_func(&ctx->fmodrets, "fmod_ret target", &fmodret);
+}
+
+static int collect_kfuncs(struct object *obj, struct btf2btf_context *ctx)
+{
+ return collect_set_funcs(obj, ctx, BTF_SET8_KFUNCS, ~0U, "kfunc", push_kfunc);
+}
+
+static int collect_fmodrets(struct object *obj, struct btf2btf_context *ctx)
+{
+ return collect_set_funcs(obj, ctx, BTF_SET8_FMODRET, KF_SLEEPABLE,
+ "fmod_ret target", push_fmodret);
+}
+
static int build_btf2btf_context(struct object *obj, struct btf2btf_context *ctx)
{
int err;
@@ -1421,6 +1493,12 @@ static int build_btf2btf_context(struct object *obj, struct btf2btf_context *ctx
return err;
}
+ err = collect_fmodrets(obj, ctx);
+ if (err) {
+ pr_err("ERROR: resolve_btfids: failed to collect fmod_ret targets from BTF\n");
+ return err;
+ }
+
return 0;
}
@@ -1727,6 +1805,40 @@ static int add_decl_tag(struct btf2btf_context *ctx, const char *tag_name,
return push_decl_tag_id(ctx, new_id);
}
+static bool has_set_func(const struct rb_root *root, u32 btf_id)
+{
+ const struct rb_node *node = root->rb_node;
+
+ while (node) {
+ const struct kfunc *k = rb_entry(node, struct kfunc, rb_node);
+
+ if (btf_id < k->btf_id)
+ node = node->rb_left;
+ else if (btf_id > k->btf_id)
+ node = node->rb_right;
+ else
+ return true;
+ }
+ return false;
+}
+
+static int add_fmodret_retval_tags(struct btf2btf_context *ctx,
+ const struct fmodret_retval *retval)
+{
+ char *tag;
+ int err;
+
+ err = add_decl_tag(ctx, DECL_TAG_FMODRET, retval->btf_id, -1);
+ if (err)
+ return err;
+
+ if (asprintf(&tag, "%s%s", DECL_TAG_FMODRET_RETVAL, retval->values) < 0)
+ return -ENOMEM;
+ err = add_decl_tag(ctx, tag, retval->btf_id, -1);
+ free(tag);
+ return err;
+}
+
static int btf2btf(struct object *obj)
{
struct btf2btf_context ctx = {};
@@ -1761,10 +1873,36 @@ static int btf2btf(struct object *obj)
goto out;
}
+ for (next = rb_first(&ctx.fmodrets); next; next = rb_next(next)) {
+ struct kfunc *fmodret = rb_entry(next, struct kfunc, rb_node);
+
+ err = add_decl_tag(&ctx, DECL_TAG_FMODRET, fmodret->btf_id, -1);
+ if (err)
+ goto out;
+
+ if (fmodret->flags & KF_SLEEPABLE) {
+ err = add_decl_tag(&ctx, DECL_TAG_FMODRET_SLEEPABLE,
+ fmodret->btf_id, -1);
+ if (err)
+ goto out;
+ }
+ }
+
+ for (size_t i = 0; i < obj->fmodret_retvals.cnt; i++) {
+ const struct fmodret_retval *retval = &obj->fmodret_retvals.retvals[i];
+
+ if (has_set_func(&ctx.fmodrets, retval->btf_id))
+ continue;
+ err = add_fmodret_retval_tags(&ctx, retval);
+ if (err)
+ goto out;
+ }
+
err = 0;
out:
free(ctx.decl_tags);
free_kfuncs(&ctx.kfuncs);
+ free_kfuncs(&ctx.fmodrets);
return err;
}
@@ -2113,6 +2251,8 @@ int main(int argc, const char **argv)
"distill --btf_base and emit .BTF.base section data"),
OPT_BOOLEAN(0, "inline", &obj.extract_inline,
"extract location BTF into a .BTF.inline file"),
+ OPT_BOOLEAN(0, "patchable_ftrace", &obj.patchable_ftrace,
+ "ftrace call sites are in __patchable_function_entries"),
OPT_CALLBACK(0, "btf_link", &btf_links, "section:module:btf-file",
"patch a BTF link with --patch_btfids", parse_btf_link),
OPT_STRING(0, "patch_btfids", &btfids_path, "file",
@@ -2171,6 +2311,10 @@ int main(int argc, const char **argv)
if (load_btf(&obj))
goto out;
+ if (fmodret_collect(obj.efile.elf, obj.btf, &obj.fmodret_retvals,
+ obj.patchable_ftrace, verbose))
+ goto out;
+
if (btf2btf(&obj))
goto out;
@@ -2228,6 +2372,7 @@ int main(int argc, const char **argv)
btf_id__free_all(&obj.funcs);
btf_id__free_all(&obj.sets);
free(obj.addr_syms);
+ fmodret_free(&obj.fmodret_retvals);
free(obj.efile.func_symbols);
if (obj.efile.elf) {
elf_end(obj.efile.elf);
diff --git a/tools/include/linux/btf_ids.h b/tools/include/linux/btf_ids.h
index 4fe5c5f1558ca9..3b5b2856c6285f 100644
--- a/tools/include/linux/btf_ids.h
+++ b/tools/include/linux/btf_ids.h
@@ -13,13 +13,21 @@ struct btf_id_set {
/* This flag implies BTF_SET8 holds kfunc(s) */
#define BTF_SET8_KFUNCS (1 << 0)
+/*
+ * This flag implies BTF_SET8 holds functions which BPF fmod_ret, fentry and
+ * fexit programs may attach to. Per-function flags: KF_SLEEPABLE only.
+ */
+#define BTF_SET8_FMODRET (1 << 1)
+
+struct btf_id_set8_pair {
+ u32 id;
+ u32 flags;
+};
+
struct btf_id_set8 {
u32 cnt;
u32 flags;
- struct {
- u32 id;
- u32 flags;
- } pairs[];
+ struct btf_id_set8_pair pairs[];
};
#ifdef CONFIG_DEBUG_INFO_BTF
@@ -214,6 +222,12 @@ __BTF_SET8_START(name, local, BTF_SET8_KFUNCS)
#define BTF_KFUNCS_END(name) \
BTF_SET8_END(name)
+#define BTF_FMODRET_START(name) \
+__BTF_SET8_START(name, local, BTF_SET8_FMODRET)
+
+#define BTF_FMODRET_END(name) \
+BTF_SET8_END(name)
+
#else
#define BTF_ID_LIST(name) static u32 __maybe_unused name[128];
@@ -230,6 +244,8 @@ BTF_SET8_END(name)
#define BTF_SET8_END(name)
#define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };
#define BTF_KFUNCS_END(name)
+#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };
+#define BTF_FMODRET_END(name)
#endif /* CONFIG_DEBUG_INFO_BTF */
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 931c5c412f15d8..e6120bee45052a 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -4657,6 +4657,7 @@ static int validate_ibt(struct objtool_file *file)
!strcmp(sec->name, "__jump_table") ||
!strcmp(sec->name, ".init.klp_funcs") ||
!strcmp(sec->name, "__mcount_loc") ||
+ !strcmp(sec->name, ".BTF_fmodret_candidates") ||
!strcmp(sec->name, ".llvm.call-graph-profile") ||
!strcmp(sec->name, ".llvm_bb_addr_map") ||
!strcmp(sec->name, "__tracepoints") ||
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 77682bf28c6ece..84554a2adfc1bc 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -47,7 +47,7 @@ TEST_PROGS_EXTENDED := \
ima_setup.sh verify_sig_setup.sh
TEST_KMODS := bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
- bpf_test_modorder_y.ko bpf_test_rqspinlock.ko
+ bpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_retval.ko
TEST_KMOD_TARGETS = $(addprefix $(OUTPUT)/,$(TEST_KMODS))
# Compile but not part of 'make run_tests'
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index 49dfe939467cc8..5461c32cd3cd2c 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -11,6 +11,7 @@ CONFIG_BPF_STREAM_PARSER=y
CONFIG_BPF_SYSCALL=y
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
CONFIG_CGROUP_BPF=y
+CONFIG_CLANG_PLUGIN_BPF_SCAN_IR=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_MLDSA=y
CONFIG_CRYPTO_SHA256=y
@@ -56,6 +57,7 @@ CONFIG_IPV6_TUNNEL=y
CONFIG_KEYS=y
CONFIG_LIRC=y
CONFIG_LIVEPATCH=y
+CONFIG_LLVM_PLUGINS=y
CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_MEMCG=y
diff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
new file mode 100644
index 00000000000000..7b374fe6813027
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include "fmodret_retval.skel.h"
+#include "fmodret_retval_ext.skel.h"
+
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_RETVAL "bpf_fmodret_retval:"
+#define TAG_TEST_RETVAL "test_retval:"
+
+#define RETVAL_MODULE "bpf_test_retval"
+
+/* The decl tags of a function */
+struct func_tags {
+ const char *test_retval;
+ const char *retval;
+ bool target;
+};
+
+static bool tag_value(const char *name, const char *key, const char **value)
+{
+ size_t len = strlen(key);
+
+ if (strncmp(name, key, len))
+ return false;
+ *value = name + len;
+ return true;
+}
+
+/*
+ * Return the decl tags of the functions of module BTF, indexed by BTF ID. Only
+ * the types of the module itself are read, not those of its vmlinux base.
+ */
+static struct func_tags *collect_tags(const struct btf *btf, int start_id)
+{
+ int id, nr_types = btf__type_cnt(btf);
+ struct func_tags *tags;
+
+ tags = calloc(nr_types, sizeof(*tags));
+ if (!tags)
+ return NULL;
+ for (id = start_id; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ struct func_tags *f;
+ const char *name;
+
+ if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1 ||
+ t->type >= nr_types)
+ continue;
+ f = &tags[t->type];
+ name = btf__name_by_offset(btf, t->name_off);
+ if (!tag_value(name, TAG_TEST_RETVAL, &f->test_retval) &&
+ !tag_value(name, TAG_RETVAL, &f->retval) && !strcmp(name, TAG_FMODRET))
+ f->target = true;
+ }
+
+ return tags;
+}
+
+/* Compare the tags of every fixture with the values it expects */
+static void test_tags(const struct btf *btf, const struct func_tags *tags, int start_id)
+{
+ int id, nr_types = btf__type_cnt(btf), nr_fixtures = 0;
+
+ for (id = start_id; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ const char *name;
+
+ if (!btf_is_func(t) || !tags[id].test_retval)
+ continue;
+ name = btf__name_by_offset(btf, t->name_off);
+ ASSERT_STREQ(tags[id].retval ?: "", tags[id].test_retval, name);
+ if (*tags[id].test_retval)
+ ASSERT_TRUE(tags[id].target, name);
+ else if (strcmp(name, "bpf_testmod_retval_declared"))
+ ASSERT_FALSE(tags[id].target, name);
+ nr_fixtures++;
+ }
+ ASSERT_GT(nr_fixtures, 0, "fixtures");
+
+ id = btf__find_by_name_kind(btf, "bpf_testmod_retval_declared", BTF_KIND_FUNC);
+ if (ASSERT_GE(id, start_id, "declared"))
+ ASSERT_TRUE(tags[id].target, "declared_target");
+}
+
+/* Open the programs again with only the extension, to replace FUNC of TARGET */
+static struct fmodret_retval_ext *open_extension(struct bpf_program *target,
+ const char *func, char *log, size_t log_size)
+{
+ struct fmodret_retval_ext *skel;
+
+ skel = fmodret_retval_ext__open();
+ if (!ASSERT_OK_PTR(skel, "fmodret_retval_ext__open"))
+ return NULL;
+ bpf_program__set_autoload(skel->progs.restricted, false);
+ bpf_program__set_autoload(skel->progs.unrestricted, false);
+ if (!ASSERT_OK(bpf_program__set_attach_target(skel->progs.extension,
+ bpf_program__fd(target), func),
+ "set_attach_target")) {
+ fmodret_retval_ext__destroy(skel);
+ return NULL;
+ }
+ bpf_program__set_log_buf(skel->progs.extension, log, log_size);
+ return skel;
+}
+
+/*
+ * An extension could return any value for an fmod_ret program, so it must not
+ * replace one whose return values are restricted, neither at load nor when it
+ * is attached later to another program.
+ */
+static void test_extension(void)
+{
+ struct fmodret_retval_ext *targets, *skel = NULL;
+ struct bpf_link *link;
+ char log[4096] = {};
+
+ if (!is_jit_enabled()) {
+ printf("%s:SKIP:extensions need the JIT\n", __func__);
+ test__skip();
+ return;
+ }
+
+ targets = fmodret_retval_ext__open();
+ if (!ASSERT_OK_PTR(targets, "fmodret_retval_ext__open"))
+ return;
+ bpf_program__set_autoload(targets->progs.extension, false);
+ if (!ASSERT_OK(fmodret_retval_ext__load(targets), "load_targets"))
+ goto out;
+
+ skel = open_extension(targets->progs.restricted, "restricted_value", log, sizeof(log));
+ if (!skel)
+ goto out;
+ ASSERT_ERR(fmodret_retval_ext__load(skel), "load_restricted");
+ ASSERT_HAS_SUBSTR(log, "Cannot extend an fmod_ret program with restricted return values",
+ "log_restricted");
+ fmodret_retval_ext__destroy(skel);
+
+ skel = open_extension(targets->progs.unrestricted, "unrestricted_value", log, sizeof(log));
+ if (!skel)
+ goto out;
+ if (!ASSERT_OK(fmodret_retval_ext__load(skel), "load_unrestricted"))
+ goto out;
+ link = bpf_program__attach_freplace(skel->progs.extension,
+ bpf_program__fd(targets->progs.restricted),
+ "restricted_value");
+ if (!ASSERT_ERR_PTR(link, "attach_restricted"))
+ bpf_link__destroy(link);
+ link = bpf_program__attach_freplace(skel->progs.extension,
+ bpf_program__fd(targets->progs.unrestricted),
+ "unrestricted_value");
+ if (ASSERT_OK_PTR(link, "attach_unrestricted"))
+ bpf_link__destroy(link);
+out:
+ fmodret_retval_ext__destroy(skel);
+ fmodret_retval_ext__destroy(targets);
+}
+
+void test_fmodret_retval(void)
+{
+ struct btf *vmlinux_btf, *btf = NULL;
+ struct func_tags *tags = NULL;
+ int id, start_id, nr_types;
+ bool has_fixtures = false;
+
+ if (!env.has_testmod) {
+ test__skip();
+ return;
+ }
+
+ if (!ASSERT_OK(load_module(RETVAL_MODULE ".ko", env_verbosity > VERBOSE_NONE),
+ "load_module"))
+ return;
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "btf__load_vmlinux_btf"))
+ goto out;
+ btf = btf__load_module_btf(RETVAL_MODULE, vmlinux_btf);
+ if (!btf && errno == ENOENT) {
+ printf("%s:SKIP:%s has no BTF\n", __func__, RETVAL_MODULE);
+ test__skip();
+ goto out;
+ }
+ if (!ASSERT_OK_PTR(btf, "btf__load_module_btf"))
+ goto out;
+ start_id = btf__type_cnt(vmlinux_btf);
+ nr_types = btf__type_cnt(btf);
+ tags = collect_tags(btf, start_id);
+ if (!ASSERT_OK_PTR(tags, "collect_tags"))
+ goto out;
+
+ /* The fixtures are built only with the compiler plugin */
+ for (id = start_id; id < nr_types && !has_fixtures; id++)
+ has_fixtures = tags[id].test_retval;
+ if (!has_fixtures) {
+ printf("%s:SKIP:%s has no %s tags\n", __func__, RETVAL_MODULE, TAG_TEST_RETVAL);
+ test__skip();
+ goto out;
+ }
+
+ if (test__start_subtest("tags"))
+ test_tags(btf, tags, start_id);
+ if (test__start_subtest("extension"))
+ test_extension();
+ RUN_TESTS(fmodret_retval);
+out:
+ free(tags);
+ btf__free(btf);
+ btf__free(vmlinux_btf);
+ unload_module(RETVAL_MODULE, env_verbosity > VERBOSE_NONE);
+}
diff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
new file mode 100644
index 00000000000000..62efe11af826fb
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <linux/filter.h>
+#include <bpf/btf.h>
+
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+
+static bool func_has_tag(const struct btf *btf, const char *func, const char *tag)
+{
+ int func_id, id, nr_types;
+
+ func_id = btf__find_by_name_kind(btf, func, BTF_KIND_FUNC);
+ if (func_id < 0)
+ return false;
+
+ nr_types = btf__type_cnt(btf);
+ for (id = 1; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_decl_tag(t) || t->type != func_id ||
+ btf_decl_tag(t)->component_idx != -1)
+ continue;
+ if (!strcmp(btf__name_by_offset(btf, t->name_off), tag))
+ return true;
+ }
+
+ return false;
+}
+
+static void test_vmlinux_tags(void)
+{
+ struct btf *btf;
+
+ btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(btf, "btf__load_vmlinux_btf"))
+ return;
+
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test", TAG_FMODRET), "modify_return_test");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test2", TAG_FMODRET),
+ "modify_return_test2");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test_tp", TAG_FMODRET),
+ "modify_return_test_tp");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_fentry_test1", TAG_FMODRET), "fentry_test1");
+ ASSERT_TRUE(func_has_tag(btf, "update_socket_protocol", TAG_FMODRET),
+ "update_socket_protocol");
+
+ /* Only bpf_fentry_test1 is a sleepable target */
+ ASSERT_TRUE(func_has_tag(btf, "bpf_fentry_test1", TAG_FMODRET_SLEEPABLE),
+ "fentry_test1_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test2", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test2_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test_tp", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test_tp_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "update_socket_protocol", TAG_FMODRET_SLEEPABLE),
+ "update_socket_protocol_sleepable");
+
+ /* A function which is not a target has no tag */
+ ASSERT_GT(btf__find_by_name_kind(btf, "bpf_fentry_test2", BTF_KIND_FUNC), 0,
+ "fentry_test2_exists");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_fentry_test2", TAG_FMODRET), "fentry_test2");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_fentry_test2", TAG_FMODRET_SLEEPABLE),
+ "fentry_test2_sleepable");
+
+ btf__free(btf);
+}
+
+static void test_module_tags(void)
+{
+ struct btf *vmlinux_btf, *module_btf;
+
+ if (!env.has_testmod) {
+ test__skip();
+ return;
+ }
+
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "btf__load_vmlinux_btf"))
+ return;
+ module_btf = btf__load_module_btf("bpf_testmod", vmlinux_btf);
+ if (!module_btf && errno == ENOENT) {
+ printf("%s:SKIP:bpf_testmod has no BTF\n", __func__);
+ test__skip();
+ goto out;
+ }
+ if (!ASSERT_OK_PTR(module_btf, "btf__load_module_btf"))
+ goto out;
+
+ ASSERT_TRUE(func_has_tag(module_btf, "bpf_testmod_trampoline_count_test", TAG_FMODRET),
+ "trampoline_count_test");
+ ASSERT_FALSE(func_has_tag(module_btf, "bpf_testmod_trampoline_count_test",
+ TAG_FMODRET_SLEEPABLE), "trampoline_count_test_sleepable");
+ ASSERT_FALSE(func_has_tag(module_btf, "bpf_testmod_fentry_test1", TAG_FMODRET),
+ "testmod_fentry_test1");
+
+ btf__free(module_btf);
+out:
+ btf__free(vmlinux_btf);
+}
+
+/* Load a minimal fmod_ret program which attaches to function BTF_ID */
+static int load_fmod_ret(int btf_id, __u32 prog_flags)
+{
+ struct bpf_insn insns[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .expected_attach_type = BPF_MODIFY_RETURN,
+ .prog_flags = prog_flags,
+ .attach_btf_id = btf_id,
+ );
+
+ return bpf_prog_load(BPF_PROG_TYPE_TRACING, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+}
+
+static void check_load(const char *func, __u32 prog_flags, bool allowed, const char *name)
+{
+ int btf_id, fd;
+
+ btf_id = libbpf_find_vmlinux_btf_id(func, BPF_MODIFY_RETURN);
+ if (!ASSERT_GT(btf_id, 0, func))
+ return;
+ fd = load_fmod_ret(btf_id, prog_flags);
+ if (allowed)
+ ASSERT_GE(fd, 0, name);
+ else
+ ASSERT_LT(fd, 0, name);
+ if (fd >= 0)
+ close(fd);
+}
+
+static void test_attach_permission(void)
+{
+ check_load("bpf_modify_return_test", 0, true, "target");
+ check_load("bpf_fentry_test1", 0, true, "sleepable_target");
+ check_load("bpf_fentry_test2", 0, false, "not_a_target");
+
+ check_load("bpf_fentry_test1", BPF_F_SLEEPABLE, true, "sleepable_prog_sleepable_target");
+ check_load("bpf_modify_return_test", BPF_F_SLEEPABLE, false,
+ "sleepable_prog_nonsleepable_target");
+}
+
+void test_fmodret_tags(void)
+{
+ if (test__start_subtest("vmlinux"))
+ test_vmlinux_tags();
+ if (test__start_subtest("module"))
+ test_module_tags();
+ if (test__start_subtest("attach_permission"))
+ test_attach_permission();
+}
diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval.c b/tools/testing/selftests/bpf/progs/fmodret_retval.c
new file mode 100644
index 00000000000000..bf5c6f53a99936
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/fmodret_retval.c
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * fmod_ret programs on the targets which the LLVM IR scanner finds in
+ * bpf_test_retval, see test_kmods/bpf_testmod_retval.c. They may return only
+ * 0 or the values of the "bpf_fmodret_retval:" tag of their target.
+ */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+/* bpf_testmod_retval_constant() returns -EINVAL */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+int BPF_PROG(constant_einval, int ret)
+{
+ return -22;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+int BPF_PROG(constant_zero, int ret)
+{
+ return 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+int BPF_PROG(constant_other, int ret)
+{
+ return -1;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+__naked void constant_unknown(void)
+{
+ asm volatile (
+ "call %[bpf_get_prandom_u32];"
+ "exit;"
+ :
+ : __imm(bpf_get_prandom_u32)
+ : __clobber_all);
+}
+
+/* The function returns int, so only the low 32 bits count */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+__naked void constant_32bit(void)
+{
+ asm volatile (
+ "w0 = -22;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* Not 0, so the function is not called, but the caller would see 0 */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+__naked void constant_high_bits(void)
+{
+ asm volatile (
+ "r0 = 1;"
+ "r0 <<= 32;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* bpf_testmod_retval_join() returns -EINVAL or -ERANGE */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+int BPF_PROG(join_einval, int *result, int value, int ret)
+{
+ return -22;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+int BPF_PROG(join_erange, int *result, int value, int ret)
+{
+ return -34;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_join")
+__failure __msg("it should be 0 or one of -34,-22")
+int BPF_PROG(join_other, int *result, int value, int ret)
+{
+ return -1;
+}
+
+/* Each path returns an allowed value */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+__naked void join_both_paths(void)
+{
+ asm volatile (
+ "r2 = *(u64 *)(r1 + 8);"
+ "if r2 s< 0 goto 1f;"
+ "r0 = -34;"
+ "exit;"
+"1:"
+ "r0 = -22;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/*
+ * One of two paths to the same exit returns a forbidden value. The verifier
+ * checks the allowed one first and must not prune the other.
+ */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__failure __msg("it should be 0 or one of -34,-22")
+__naked void join_one_forbidden(void)
+{
+ asm volatile (
+ "r2 = *(u64 *)(r1 + 8);"
+ "if r2 s< 0 goto 1f;"
+ "r0 = -34;"
+ "goto 2f;"
+"1:"
+ "r0 = -1;"
+"2:"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* bpf_testmod_retval_wide() returns s64, so all 64 bits count */
+SEC("fmod_ret/bpf_testmod_retval_wide")
+__success
+__load_if_JITed()
+__naked void wide_64bit(void)
+{
+ asm volatile (
+ "r0 = -14;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+SEC("fmod_ret/bpf_testmod_retval_wide")
+__failure __msg("it should be 0 or one of -14")
+__naked void wide_32bit(void)
+{
+ asm volatile (
+ "w0 = -14;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* Declared targets are not restricted, in module and vmlinux BTF */
+SEC("fmod_ret/bpf_testmod_retval_declared")
+__success
+int BPF_PROG(declared_module_any, int ret)
+{
+ return -1;
+}
+
+SEC("fmod_ret/bpf_modify_return_test")
+__success
+int BPF_PROG(declared_vmlinux_any, int a, int *b, int ret)
+{
+ return -1;
+}
+
+/* The error injection list allows any value although the target has a tag */
+SEC("fmod_ret/bpf_testmod_retval_injectable")
+__success
+int BPF_PROG(static_rule_tagged, int value, int ret)
+{
+ return -1;
+}
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __uint(max_entries, 1);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(__u32));
+} jmp_table SEC(".maps");
+
+/* A program in a program array may return any value */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("tail calls are not allowed in fmod_ret programs with restricted return values")
+int BPF_PROG(constant_tail_call, int ret)
+{
+ bpf_tail_call(ctx, &jmp_table, 0);
+ return 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_declared")
+__success
+int BPF_PROG(declared_tail_call, int ret)
+{
+ bpf_tail_call(ctx, &jmp_table, 0);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
new file mode 100644
index 00000000000000..dd105eea05ee46
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * An extension of fmod_ret programs, see prog_tests/fmodret_retval.c. The
+ * return values of the program on bpf_testmod_retval_constant() are restricted
+ * by its "bpf_fmodret_retval:" tag, those of the program on the declared target
+ * are not. An extension cannot replace the main function of a tracing program,
+ * so each of them calls a global function which the extension replaces.
+ */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+/* Keeps clang from folding the calls */
+int value_bias;
+
+__noinline int restricted_value(int value)
+{
+ return value + value_bias;
+}
+
+__noinline int unrestricted_value(int value)
+{
+ return value + value_bias;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+int BPF_PROG(restricted, int ret)
+{
+ int zero = 0;
+
+ if (restricted_value(ret))
+ return -22;
+ /*
+ * fmod_ret BPF progs must return constants (0 or -22 in this case).
+ * The barrier here forces GCC to return a constant
+ */
+ barrier_var(zero);
+ return zero;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_declared")
+int BPF_PROG(unrestricted, int ret)
+{
+ return unrestricted_value(ret);
+}
+
+SEC("freplace")
+int extension(int value)
+{
+ return -1;
+}
diff --git a/tools/testing/selftests/bpf/progs/fsession_test.c b/tools/testing/selftests/bpf/progs/fsession_test.c
index 86e8a2fe467e20..4295f6a78dd128 100644
--- a/tools/testing/selftests/bpf/progs/fsession_test.c
+++ b/tools/testing/selftests/bpf/progs/fsession_test.c
@@ -4,6 +4,8 @@
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
+extern int bpf_fentry_test1(int a) __ksym;
+
char _license[] SEC("license") = "GPL";
__u64 test1_entry_result = 0;
diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
index bbeea0d512e338..51a7f605f9c5d7 100644
--- a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
+++ b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
@@ -3,6 +3,8 @@
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
+extern int bpf_fentry_test1(int a) __ksym;
+
char _license[] SEC("license") = "GPL";
__u64 test1_entry_result = 0;
diff --git a/tools/testing/selftests/bpf/progs/tracing_multi_check.c b/tools/testing/selftests/bpf/progs/tracing_multi_check.c
index b2959ba711790e..9b008ecd65651e 100644
--- a/tools/testing/selftests/bpf/progs/tracing_multi_check.c
+++ b/tools/testing/selftests/bpf/progs/tracing_multi_check.c
@@ -8,7 +8,7 @@ char _license[] SEC("license") = "GPL";
int pid = 0;
bool test_cookies = false;
-/* bpf_fentry_test1 is exported as kfunc via vmlinux.h */
+extern int bpf_fentry_test1(int a) __ksym;
extern const void bpf_fentry_test2 __ksym;
extern const void bpf_fentry_test3 __ksym;
extern const void bpf_fentry_test4 __ksym;
diff --git a/tools/testing/selftests/bpf/test_kmods/Makefile b/tools/testing/selftests/bpf/test_kmods/Makefile
index 031c7454ce65f6..f4af3c3ab54c2d 100644
--- a/tools/testing/selftests/bpf/test_kmods/Makefile
+++ b/tools/testing/selftests/bpf/test_kmods/Makefile
@@ -19,10 +19,23 @@ Q = @
endif
MODULES = bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
- bpf_test_modorder_y.ko bpf_test_rqspinlock.ko
+ bpf_test_modorder_y.ko bpf_test_rqspinlock.ko bpf_test_retval.ko
$(foreach m,$(MODULES),$(eval obj-m += $(m:.ko=.o)))
+bpf_test_retval-y := bpf_test_retval_main.o
+# The functions for bpf_scan_ir use clang extensions and need the plugin
+bpf_test_retval-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_testmod_retval.o \
+ bpf_testmod_retval_strong.o
+# Instrumentation and branch profiling make every path of the fixtures dirty
+GCOV_PROFILE_bpf_testmod_retval.o := n
+KCOV_INSTRUMENT_bpf_testmod_retval.o := n
+KMSAN_SANITIZE_bpf_testmod_retval.o := n
+KASAN_SANITIZE_bpf_testmod_retval.o := n
+KCSAN_SANITIZE_bpf_testmod_retval.o := n
+UBSAN_SANITIZE_bpf_testmod_retval.o := n
+CFLAGS_bpf_testmod_retval.o += -DDISABLE_BRANCH_PROFILING
+
CFLAGS_bpf_testmod.o = -I$(src)
# When BPF_STRICT_BUILD != 0, a missing KDIR is fatal (the default).
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c b/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c
new file mode 100644
index 00000000000000..c3a9093fa3310c
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_test_retval_main.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A module for the functions of bpf_testmod_retval.c and
+ * bpf_testmod_retval_strong.c, see prog_tests/fmodret_retval.c. They are not
+ * in bpf_testmod: pahole drops the per-CPU variables of a module which GNU ld
+ * links from several objects.
+ */
+#include <linux/module.h>
+
+MODULE_AUTHOR("Anton Protopopov");
+MODULE_DESCRIPTION("BPF selftest module for fmod_ret targets found by bpf_scan_ir");
+MODULE_LICENSE("GPL");
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index 41ff36f0ca77e9..91cb311a2dc8b1 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -2607,15 +2607,9 @@ struct bpf_struct_ops testmod_multi_st_ops = {
extern int bpf_fentry_test1(int a);
-BTF_KFUNCS_START(bpf_testmod_trampoline_count_ids)
+BTF_FMODRET_START(bpf_testmod_trampoline_count_ids)
BTF_ID_FLAGS(func, bpf_testmod_trampoline_count_test)
-BTF_KFUNCS_END(bpf_testmod_trampoline_count_ids)
-
-static const struct
-btf_kfunc_id_set bpf_testmod_trampoline_count_fmodret_set = {
- .owner = THIS_MODULE,
- .set = &bpf_testmod_trampoline_count_ids,
-};
+BTF_FMODRET_END(bpf_testmod_trampoline_count_ids)
static int bpf_testmod_init(void)
{
@@ -2633,7 +2627,6 @@ static int bpf_testmod_init(void)
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_testmod_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_testmod_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_testmod_kfunc_set);
- ret = ret ?: register_btf_fmodret_id_set(&bpf_testmod_trampoline_count_fmodret_set);
ret = ret ?: register_bpf_struct_ops(&bpf_bpf_testmod_ops, bpf_testmod_ops);
ret = ret ?: register_bpf_struct_ops(&bpf_testmod_ops2, bpf_testmod_ops2);
ret = ret ?: register_bpf_struct_ops(&bpf_testmod_ops3, bpf_testmod_ops3);
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
new file mode 100644
index 00000000000000..fd63c6317cb944
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
@@ -0,0 +1,311 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Functions for the fmod_ret targets which the LLVM IR scanner finds, see
+ * prog_tests/fmodret_retval.c. The "test_retval:" decl tag of each lists the
+ * values the scanner must find on paths without side effects. An empty list
+ * means that the function must not become such a target.
+ */
+#include <linux/atomic.h>
+#include <linux/btf_ids.h>
+#include <linux/compiler.h>
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/error-injection.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#define RETVAL(values) __attribute__((btf_decl_tag("test_retval:" values)))
+
+static __used int bpf_testmod_retval_state;
+static __used int *bpf_testmod_retval_captured;
+
+static noinline void bpf_testmod_retval_effect(void)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, 1);
+}
+
+static noinline void bpf_testmod_retval_capture(int *ptr)
+{
+ WRITE_ONCE(bpf_testmod_retval_captured, ptr);
+}
+
+static noinline int bpf_testmod_retval_store(int value)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, value);
+ return value;
+}
+
+static noinline int bpf_testmod_retval_clear(int value)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, 0);
+ return value - 1;
+}
+
+/* Constant errors and both return widths */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_constant(void)
+{
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("-14") s64 bpf_testmod_retval_wide(void)
+{
+ return -EFAULT;
+}
+
+static __used noinline RETVAL("-1") int bpf_testmod_retval_first_errno(void)
+{
+ return -1;
+}
+
+static __used noinline RETVAL("-4095") int bpf_testmod_retval_last_errno(void)
+{
+ return -MAX_ERRNO;
+}
+
+/*
+ * PTR_ERR() of the ERR_PTR() of an inlined helper. Not recognized yet: the
+ * error reaches the return through a PHI in an earlier block.
+ */
+static __always_inline int *bpf_testmod_retval_slot(int *buf, int idx)
+{
+ if (idx < 0)
+ return ERR_PTR(-EINVAL);
+ if (idx > 8)
+ return ERR_PTR(-EFAULT);
+ return &buf[idx];
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_err_ptr(int *buf, int idx)
+{
+ int *slot = bpf_testmod_retval_slot(buf, idx);
+
+ if (IS_ERR(slot))
+ return PTR_ERR(slot);
+ *slot = 1;
+ return 0;
+}
+
+/* A clean error path may join a dirty success path */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_before_store(int *result, int value)
+{
+ if (value < 0)
+ return -EINVAL;
+ *result = value;
+ return 0;
+}
+
+static __used noinline RETVAL("-34,-22") int bpf_testmod_retval_join(int *result, int value)
+{
+ int ret;
+
+ if (value < 0)
+ ret = -EINVAL;
+ else if (value > 32)
+ ret = -ERANGE;
+ else {
+ *result = value;
+ ret = 0;
+ }
+ return ret;
+}
+
+/*
+ * An int error which a long function returns: the PHI of the two calls keeps
+ * the sign extension in the block of the return
+ */
+static __used noinline RETVAL("-22") long bpf_testmod_retval_sign_extend(int value)
+{
+ int ret;
+
+ if (value < 0)
+ ret = -EINVAL;
+ else if (value > 32)
+ ret = bpf_testmod_retval_store(value);
+ else
+ ret = bpf_testmod_retval_clear(value);
+ return ret;
+}
+
+/* An assumption does not make a path dirty */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_assume(int value)
+{
+ __builtin_assume(value > 0);
+ return -EINVAL;
+}
+
+/* Writes to private stack objects are clean, optnone keeps them in the IR */
+static __used noinline RETVAL("-22") __attribute__((optnone))
+int bpf_testmod_retval_private_store(int value)
+{
+ int scratch[2];
+
+ scratch[0] = value;
+ scratch[1] = 0;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("-22") __attribute__((optnone))
+int bpf_testmod_retval_private_memset(void)
+{
+ char scratch[32];
+
+ __builtin_memset(scratch, 0, sizeof(scratch));
+ return -EINVAL;
+}
+
+/* An error which is not known exactly is not recorded */
+static __used noinline RETVAL("") int bpf_testmod_retval_masked(unsigned int value)
+{
+ return -1 - (value & 31);
+}
+
+static __used noinline RETVAL("") long bpf_testmod_retval_ptr_err(void *ptr, int *state)
+{
+ if (IS_ERR(ptr))
+ return PTR_ERR(ptr);
+ *state = 1;
+ return 0;
+}
+
+/* Pointer functions are not targets yet */
+static __used noinline RETVAL("") void *bpf_testmod_retval_error_pointer(void)
+{
+ return ERR_PTR(-ENOMEM);
+}
+
+/* Every error path has an observable effect */
+static __used noinline RETVAL("") int bpf_testmod_retval_argument_store(int *state)
+{
+ *state = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_global_store(void)
+{
+ bpf_testmod_retval_state = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_volatile_read(int *state)
+{
+ return READ_ONCE(*state) ? -EINVAL : -ERANGE;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_atomic(atomic_t *state)
+{
+ atomic_inc(state);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_call(void)
+{
+ bpf_testmod_retval_effect();
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_barrier(void)
+{
+ barrier();
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("")
+int bpf_testmod_retval_only_error_is_dirty(int *state, bool reject)
+{
+ if (reject) {
+ *state = 1;
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static __used noinline RETVAL("")
+int bpf_testmod_retval_lock_held(spinlock_t *lock) __acquires(lock)
+{
+ spin_lock(lock);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_stack_escape(void)
+{
+ int scratch;
+
+ bpf_testmod_retval_capture(&scratch);
+ scratch = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_external_memset(void *dest)
+{
+ __builtin_memset(dest, 0, 32);
+ return -EINVAL;
+}
+
+/* Not recognized yet, although they have no side effects */
+static __used noinline RETVAL("") int bpf_testmod_retval_balanced_spinlock(spinlock_t *lock)
+{
+ spin_lock(lock);
+ spin_unlock(lock);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_input_errno(int error)
+{
+ if (error >= -MAX_ERRNO && error < 0)
+ return error;
+ return 0;
+}
+
+/* No ftrace call site, but call padding in __patchable_function_entries on x86 */
+static __used noinline RETVAL("") notrace int bpf_testmod_retval_notrace(void)
+{
+ return -EINVAL;
+}
+
+/*
+ * A weak definition with an error, replaced by a strong one without errors in
+ * bpf_testmod_retval_strong.c
+ */
+int bpf_testmod_retval_overridden(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_overridden(void)
+{
+ return -EINVAL;
+}
+
+/*
+ * Two weak definitions: this one with a side effect is linked first and wins
+ * over the one with an error in bpf_testmod_retval_strong.c
+ */
+int bpf_testmod_retval_two_weak(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_two_weak(void)
+{
+ bpf_testmod_retval_effect();
+ return 0;
+}
+
+/*
+ * A function on the error injection list gets its tag, but the static rules let
+ * programs return any value
+ */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_injectable(int value)
+{
+ if (value < 0)
+ return -EINVAL;
+ bpf_testmod_retval_effect();
+ return 0;
+}
+ALLOW_ERROR_INJECTION(bpf_testmod_retval_injectable, ERRNO);
+
+/* A declared target keeps only its own tags */
+static __used noinline RETVAL("") int bpf_testmod_retval_declared(void)
+{
+ return -EINVAL;
+}
+
+BTF_FMODRET_START(bpf_testmod_retval_declared_ids)
+BTF_ID_FLAGS(func, bpf_testmod_retval_declared)
+BTF_FMODRET_END(bpf_testmod_retval_declared_ids)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
new file mode 100644
index 00000000000000..a246d7f619bc51
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Replaces the weak bpf_testmod_retval_overridden() of bpf_testmod_retval.c.
+ * This definition returns no errors, so the function must not become an
+ * fmod_ret target, see prog_tests/fmodret_retval.c.
+ *
+ * The weak bpf_testmod_retval_two_weak() here loses the link to the one of
+ * bpf_testmod_retval.c, so its error must not make the function a target
+ * either.
+ */
+#include <linux/compiler.h>
+#include <linux/errno.h>
+
+int bpf_testmod_retval_overridden(void);
+
+noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_overridden(void)
+{
+ return 0;
+}
+
+int bpf_testmod_retval_two_weak(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_two_weak(void)
+{
+ return -EINVAL;
+}
|
| 33/3 | 2026/11/12 15:34 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 68243 output: 32 thoughts: 1487 Thoughts: **KMSAN Evaluation: BPF FMODRET Patch Series** Okay, let's break this down. My goal is to determine if this patch series warrants a dedicated KMSAN fuzzing session, and to do that, I need to understand the changes and, critically, whether they introduce new uninitialized memory risks that KASAN or other standard tools won't catch. This is a crucial distinction. First, I see the target is amd64, so that simplifies things a bit. Standard KASAN builds are in place, so I need to carefully look for the gaps. I'm starting with the diff. I see changes in Makefiles and Kconfig related to LLVM plugins, which suggests some new code generation or analysis. Then, the inclusion of `s64 *fmodret_retvals;` and `u32 fmodret_retval_cnt;` in `struct bpf_prog_aux` is interesting. This, along with associated BTF changes (`btf_id_set8_entry`, `BTF_SET8_FMODRET`, etc.), points toward new mechanisms for tracking return values related to function modifications. I'm zeroing in on `kernel/bpf/btf.c` and specifically `btf_register_fmodret_tags`. This function looks like it's building a set of "fmod_ret" targets from BTF (BPF Type Format) declarations. I need to understand this memory allocation carefully, which uses `kmalloc`. I see `kmalloc(struct_size(set, pairs, cnt), ...)` which implies dynamic memory allocation. The initial assignments `set->flags = 0;` and `set->cnt = 0;` are good, but I need to make sure the loop that follows initializes everything it allocates. I'm tracing the data flow. The loop `set->pairs[set->cnt].id = func_id;` and `set->pairs[set->cnt].flags = 0;` looks correct. I can see that `cnt` elements are allocated and initialized precisely. The `sort()` call is typical. The following loop sets `KF_SLEEPABLE` flag. The important part is that the set is assigned to `tab->sets[BTF_KFUNC_HOOK_FMODRET]`. I need to now find out if this `set` ever gets copied to user space, because that's a classic KMSAN entry point. I'm now going to grep for `BTF_KFUNC_HOOK_FMODRET` and `kfunc_set_tab` to trace the uses of that data. I'm looking for where this potentially uninitialized data could become visible to userspace. If it *does* leak to userspace, then it's a candidate for KMSAN. If KASAN handles the allocations appropriately, I may not need KMSAN. I'll need to look at how this data gets used and what assumptions are made about it. It seems unlikely at first glance, given the initialization, but I have to be absolutely sure. The verifier checks are key here! I need to ensure the verifier logic makes no assumptions about what's in that memory that's not explicitly initialized. So far, it looks like a close call. I need to confirm there's no way this information is indirectly exposed or that the verifier makes assumptions about initialized state. The existence of verifier checks for these return values is a good sign. The user space tools might offer a clue, so I need to review those as well. |
| 34/3 | 2026/11/12 15:34 | tool | grepper |
0mArgs:{
"Expression": "BTF_KFUNC_HOOK_FMODRET|kfunc_set_tab"
}Results: {
"Output": "kernel/bpf/btf.c=211=enum btf_kfunc_hook {\n--\nkernel/bpf/btf.c-217-\tBTF_KFUNC_HOOK_SYSCALL,\nkernel/bpf/btf.c:218:\tBTF_KFUNC_HOOK_FMODRET,\nkernel/bpf/btf.c-219-\tBTF_KFUNC_HOOK_CGROUP,\n--\nkernel/bpf/btf.c=235=struct btf_kfunc_hook_filter {\n--\nkernel/bpf/btf.c-239-\nkernel/bpf/btf.c:240:struct btf_kfunc_set_tab {\nkernel/bpf/btf.c-241-\tstruct btf_id_set8 *sets[BTF_KFUNC_HOOK_MAX];\n--\nkernel/bpf/btf.c=256=struct btf {\n--\nkernel/bpf/btf.c-270-\tstruct rcu_head rcu;\nkernel/bpf/btf.c:271:\tstruct btf_kfunc_set_tab *kfunc_set_tab;\nkernel/bpf/btf.c-272-\tstruct btf_id_dtor_kfunc_tab *dtor_kfunc_tab;\n--\nkernel/bpf/btf.c=1811=static void btf_free_id(struct btf *btf)\n--\nkernel/bpf/btf.c-1837-\nkernel/bpf/btf.c:1838:static void btf_free_kfunc_set_tab(struct btf *btf)\nkernel/bpf/btf.c-1839-{\nkernel/bpf/btf.c:1840:\tstruct btf_kfunc_set_tab *tab = btf-\u003ekfunc_set_tab;\nkernel/bpf/btf.c-1841-\tint hook;\n--\nkernel/bpf/btf.c-1847-\tkfree(tab);\nkernel/bpf/btf.c:1848:\tbtf-\u003ekfunc_set_tab = NULL;\nkernel/bpf/btf.c-1849-}\n--\nkernel/bpf/btf.c=1895=static void btf_free(struct btf *btf)\n--\nkernel/bpf/btf.c-1898-\tbtf_free_dtor_kfunc_tab(btf);\nkernel/bpf/btf.c:1899:\tbtf_free_kfunc_set_tab(btf);\nkernel/bpf/btf.c-1900-\tbtf_free_struct_ops_tab(btf);\n--\nkernel/bpf/btf.c=9872=static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook,\n--\nkernel/bpf/btf.c-9878-\tbool add_filter = !!kset-\u003efilter;\nkernel/bpf/btf.c:9879:\tstruct btf_kfunc_set_tab *tab;\nkernel/bpf/btf.c-9880-\tstruct btf_id_set8 *set;\n--\nkernel/bpf/btf.c-9891-\nkernel/bpf/btf.c:9892:\ttab = btf-\u003ekfunc_set_tab;\nkernel/bpf/btf.c-9893-\n--\nkernel/bpf/btf.c-9914-\t\t\treturn -ENOMEM;\nkernel/bpf/btf.c:9915:\t\tbtf-\u003ekfunc_set_tab = tab;\nkernel/bpf/btf.c-9916-\t}\n--\nkernel/bpf/btf.c-9978-end:\nkernel/bpf/btf.c:9979:\tbtf_free_kfunc_set_tab(btf);\nkernel/bpf/btf.c-9980-\treturn ret;\n--\nkernel/bpf/btf.c=9983=static u32 *btf_kfunc_id_set_contains(const struct btf *btf,\n--\nkernel/bpf/btf.c-9991-\t\treturn NULL;\nkernel/bpf/btf.c:9992:\tif (!btf-\u003ekfunc_set_tab)\nkernel/bpf/btf.c-9993-\t\treturn NULL;\nkernel/bpf/btf.c:9994:\tset = btf-\u003ekfunc_set_tab-\u003esets[hook];\nkernel/bpf/btf.c-9995-\tif (!set)\n--\nkernel/bpf/btf.c=10003=static bool __btf_kfunc_is_allowed(const struct btf *btf,\n--\nkernel/bpf/btf.c-10012-\t\treturn false;\nkernel/bpf/btf.c:10013:\tif (!btf-\u003ekfunc_set_tab)\nkernel/bpf/btf.c-10014-\t\treturn false;\nkernel/bpf/btf.c-10015-\nkernel/bpf/btf.c:10016:\thook_filter = \u0026btf-\u003ekfunc_set_tab-\u003ehook_filters[hook];\nkernel/bpf/btf.c-10017-\tfor (i = 0; i \u003c hook_filter-\u003enr_filters; i++) {\n--\nkernel/bpf/btf.c=10072=bool btf_kfunc_is_allowed(const struct btf *btf,\n--\nkernel/bpf/btf.c-10096- * keeping the reference for the duration of the call provides the necessary\nkernel/bpf/btf.c:10097: * protection for looking up a well-formed btf-\u003ekfunc_set_tab.\nkernel/bpf/btf.c-10098- */\n--\nkernel/bpf/btf.c=10123=int btf_kfunc_check_flag(const struct btf *btf, u32 kfunc_btf_id, u32 flag)\n--\nkernel/bpf/btf.c-10130-\tfor (hook = 0; hook \u003c BTF_KFUNC_HOOK_MAX; hook++) {\nkernel/bpf/btf.c:10131:\t\tif (hook == BTF_KFUNC_HOOK_FMODRET)\nkernel/bpf/btf.c-10132-\t\t\tcontinue;\n--\nkernel/bpf/btf.c=10146=u32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,\n--\nkernel/bpf/btf.c-10148-{\nkernel/bpf/btf.c:10149:\tif (!__btf_kfunc_is_allowed(btf, BTF_KFUNC_HOOK_FMODRET, kfunc_btf_id, prog))\nkernel/bpf/btf.c-10150-\t\treturn NULL;\nkernel/bpf/btf.c-10151-\nkernel/bpf/btf.c:10152:\treturn btf_kfunc_id_set_contains(btf, BTF_KFUNC_HOOK_FMODRET, kfunc_btf_id);\nkernel/bpf/btf.c-10153-}\n--\nkernel/bpf/btf.c=10226=static int btf_register_fmodret_tags(struct btf *btf)\n--\nkernel/bpf/btf.c-10228-\tu32 id, func_id, nr_types = btf_nr_types(btf), cnt = 0;\nkernel/bpf/btf.c:10229:\tstruct btf_kfunc_set_tab *tab;\nkernel/bpf/btf.c-10230-\tstruct btf_id_set8 *set;\n--\nkernel/bpf/btf.c-10264-\nkernel/bpf/btf.c:10265:\ttab = btf-\u003ekfunc_set_tab;\nkernel/bpf/btf.c-10266-\tif (!tab) {\n--\nkernel/bpf/btf.c-10271-\t\t}\nkernel/bpf/btf.c:10272:\t\tbtf-\u003ekfunc_set_tab = tab;\nkernel/bpf/btf.c-10273-\t}\nkernel/bpf/btf.c:10274:\tif (tab-\u003esets[BTF_KFUNC_HOOK_FMODRET]) {\nkernel/bpf/btf.c-10275-\t\tkfree(set);\n--\nkernel/bpf/btf.c-10277-\t}\nkernel/bpf/btf.c:10278:\ttab-\u003esets[BTF_KFUNC_HOOK_FMODRET] = set;\nkernel/bpf/btf.c-10279-\treturn 0;\n--\nkernel/bpf/verifier.c=23952=btf_attach_func_proto(struct bpf_verifier_log *log, struct btf *btf, u32 func_id)\n--\nkernel/bpf/verifier.c-23964-\t/*\nkernel/bpf/verifier.c:23965:\t * btf_kfunc_check_flag() reads kfunc_set_tab, which for a module is\nkernel/bpf/verifier.c-23966-\t * stable only once it is live; hold a module ref across the read to\n"
} |
| 35/3 | 2026/11/12 15:34 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 73038 output: 24 thoughts: 2079 Thoughts: **Deep Dive into `fmodret_retval_next` and `fmodret_retval_init`** Alright, let's dissect these functions, starting with `fmodret_retval_next`. The `buf` size is 24, and there's a crucial check here: `if (!len || len >= sizeof(buf) || (*end && !end[1])) return -EINVAL;`. `memcpy` and null termination seem straightforward enough. The core is extracting a numerical value from a comma-separated string. The logic is defensively structured. I'm satisfied. Now, on to `fmodret_retval_init`. The `kmalloc_array` call immediately stands out. It's allocating memory, but it doesn't initialize it. Okay, let's see how `retvals` is populated. I need to be certain there isn't an uninitialized read. First `cnt` is calculated. My concern is malformed inputs and potential off-by-one errors. Then there's the second loop: `for (list = values, cnt = 0; *list;)`. The structure is: parse a token, and if it fails, go to `invalid`. The concern is trailing characters, or more / fewer commas than expected in the input string. If a parse fails, the function correctly frees `retvals`. However, what if there are *fewer* tokens read than the initial count? The loops are tightly bound, and the logic is sound. Let me verify: In `fmodret_retval_next` if `*end` is a comma it advances past the comma `*list = end + 1`. The initial `cnt` is calculated by the number of commas present, plus one. Thus, if there are K commas, initial `cnt` is K + 1. The second loop parses and advances. For each token, if the parse succeeds, `*list` advances. If all tokens succeed, exactly K + 1 elements are read. At the end, `aux->fmodret_retval_cnt = cnt` equals the number of parsed elements, and every parsed element will be written! The loop is correct. Even if `cnt` were smaller (which it isn't), `aux->fmodret_retval_cnt = cnt` is set to the number of elements written (`retvals[cnt++] = v;`). And since the `aux->fmodret_retvals` are explicitly read in loops, there is no uninitialized read. Finally, Let's check `btf_find_decl_tag_value` in `kernel/bpf/btf.c`. |
| 36/3 | 2026/11/12 15:35 | tool | grepper |
0mArgs:{
"Expression": "btf_find_decl_tag_value"
}Results: {
"Output": "include/linux/bpf.h=3435=int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prog,\ninclude/linux/bpf.h-3436-\t\t\t struct btf *btf, const struct btf_type *t);\ninclude/linux/bpf.h:3437:const char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type *pt,\ninclude/linux/bpf.h-3438-\t\t\t\t int comp_idx, const char *tag_key);\n--\nkernel/bpf/btf.c=3639=int btf_find_next_decl_tag(const struct btf *btf, const struct btf_type *pt,\n--\nkernel/bpf/btf.c-3660-\nkernel/bpf/btf.c:3661:const char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type *pt,\nkernel/bpf/btf.c-3662-\t\t\t\t int comp_idx, const char *tag_key)\n--\nkernel/bpf/btf.c=3686=btf_find_graph_root(const struct btf *btf, const struct btf_type *pt,\n--\nkernel/bpf/btf.c-3698-\t\treturn BTF_FIELD_IGNORE;\nkernel/bpf/btf.c:3699:\tvalue_type = btf_find_decl_tag_value(btf, pt, comp_idx, \"contains:\");\nkernel/bpf/btf.c-3700-\tif (IS_ERR(value_type))\n--\nkernel/bpf/verifier.c=2571=static int bpf_find_exception_callback_insn_off(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-2590-\nkernel/bpf/verifier.c:2591:\tname = btf_find_decl_tag_value(btf, t, -1, \"exception_callback:\");\nkernel/bpf/verifier.c-2592-\tif (IS_ERR(name)) {\n--\nkernel/bpf/verifier.c=23839=static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,\n--\nkernel/bpf/verifier.c-23851-\t\treturn 0;\nkernel/bpf/verifier.c:23852:\tvalues = btf_find_decl_tag_value(btf, t, -1, BTF_FMODRET_RETVAL_TAG);\nkernel/bpf/verifier.c-23853-\tif (IS_ERR(values))\n"
} |
| 37/3 | 2026/11/12 15:35 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 76436 output: 38 thoughts: 28 Thoughts: Okay, let's dive into this. Here's my take on those thoughts, as if they're my own: **Analyzing `btf_find_decl_tag_value`** Right, time to examine `btf_find_decl_tag_value` within `kernel/bpf/btf.c`. I need to understand how we're extracting tag values from declarations within the BTF (BPF Type Format) metadata. Given my experience, I'm expecting this function to be critical for tasks like verifying type compatibility, enforcing security constraints, or even optimizing BPF programs. I'm immediately thinking about how it handles nested structures, potential for recursion, and error conditions. I want to see if it efficiently uses BTF's internal structures to avoid unnecessary iterations, and how it deals with the potential for tag names to be missing or ambiguous. It should be relatively straightforward to read, as the functions relating to BTF are well-trodden ground. I'll pay careful attention to any bitwise operations or pointer arithmetic, since this is kernel code and precision is critical. |
| 38/3 | 2026/11/12 15:35 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/btf.c",
"FirstLine": 3655,
"LineCount": 35
}Results: {
"Contents": "3655:\t\t\t\tcontinue;\n3656:\t\t\treturn i;\n3657:\t\t}\n3658:\t\treturn -ENOENT;\n3659:\t}\n3660:\t\n3661:\tconst char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type *pt,\n3662:\t\t\t\t\t int comp_idx, const char *tag_key)\n3663:\t{\n3664:\t\tconst char *value = NULL;\n3665:\t\tconst struct btf_type *t;\n3666:\t\tint len, id;\n3667:\t\n3668:\t\tid = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key,\n3669:\t\t\t\t\t btf_named_start_id(btf, false) - 1);\n3670:\t\tif (id \u003c 0)\n3671:\t\t\treturn ERR_PTR(id);\n3672:\t\n3673:\t\tt = btf_type_by_id(btf, id);\n3674:\t\tlen = strlen(tag_key);\n3675:\t\tvalue = __btf_name_by_offset(btf, t-\u003ename_off) + len;\n3676:\t\n3677:\t\t/* Prevent duplicate entries for same type */\n3678:\t\tid = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key, id);\n3679:\t\tif (id \u003e= 0)\n3680:\t\t\treturn ERR_PTR(-EEXIST);\n3681:\t\n3682:\t\treturn value;\n3683:\t}\n3684:\t\n3685:\tstatic int\n3686:\tbtf_find_graph_root(const struct btf *btf, const struct btf_type *pt,\n3687:\t\t\t const struct btf_type *t, int comp_idx, u32 off,\n3688:\t\t\t int sz, struct btf_field_info *info,\n3689:\t\t\t enum btf_field_type head_type)\n"
} |
| 39/3 | 2026/11/12 15:35 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 77305 output: 21 thoughts: 1035 Thoughts: **Analyzing `btf_register_fmodret_tags`**
Okay, let's break down this `btf_register_fmodret_tags` function. The goal is to build a set of fmod_ret targets from the BTF (Berkeley Type Format) declaration tags. I need to iterate through the BTF types, find those with the `BTF_FMODRET_TAG` and `BTF_FMODRET_SLEEPABLE_TAG`, and construct a `btf_id_set8` structure to store the function IDs.
First, I initialize a counter, `cnt`, to determine the size of the set. Then I allocate memory for `btf_id_set8`, handling the `kmalloc` failure case by returning an error. I fill the `set->pairs` array with function IDs that have the `BTF_FMODRET_TAG`. I sort the set using `sort` with a comparison function, `btf_id_cmp_func`. The sleepable tags (`BTF_FMODRET_SLEEPABLE_TAG`) are then handled, marking existing function pairs within the set.
Now, a couple of issues that strike me... I need to ensure that the `kfunc_set_tab` table exists. If it doesn't, I allocate it; otherwise, I use it. The code checks if a specific slot in `kfunc_set_tab` (`BTF_KFUNC_HOOK_FMODRET`) is already populated. My initial concern was a potential memory leak if this slot was already occupied. However, let's step through this. The code `if (!tab) { tab = kzalloc_obj(*tab, ...); btf->kfunc_set_tab = tab; }` initializes the structure if `tab` does not exist. The subsequent check, `if (tab->sets[BTF_KFUNC_HOOK_FMODRET])`, handles the existing case. If `!tab`, `tab` was just allocated and zeroed, so `tab->sets[BTF_KFUNC_HOOK_FMODRET]` is NULL. If `tab` already existed, `tab->sets[...]` could be set. So no memory leak there, as I thought. That's memory management, not uninitialized memory.
Aha, a related point I have to clarify is the structure, `struct btf_id_set8`. I need to determine whether `btf_id_set8` is ever copied to user space because if so, there are other considerations. I'll need to check the kernel code for usages of this structure; a quick `grep` should provide the answers.
|
| 40/3 | 2026/11/12 15:35 | tool | grepper |
0mArgs:{
"Expression": "btf_id_set8"
}Results: {
"Output": "\nFull output is too long, showing 200 out of 220 lines.\nUse more precise expression or PathPrefixes if possible.\n\nNumber of matching lines per file (19 files in total):\nfs/binfmt_misc_bpf.c:1\nfs/bpf_fs_kfuncs.c:1\nfs/verity/measure.c:1\ninclude/linux/btf.h:5\ninclude/linux/btf_ids.h:7\nkernel/bpf/btf.c:10\nkernel/bpf/helpers.c:1\nkernel/sched/ext/cid.h:2\nkernel/sched/ext/ext.c:11\nkernel/sched/ext/idle.h:2\nkernel/trace/bpf_trace.c:1\nnet/core/bpf_ksock.c:1\nnet/core/filter.c:1\nnet/core/xdp.c:1\nnet/ipv4/bpf_tcp_ops.c:2\nnet/sched/bpf_qdisc.c:1\ntools/bpf/resolve_btfids/main.c:2\ntools/include/linux/btf_ids.h:7\ntools/testing/selftests/bpf/prog_tests/resolve_btfids.c:1\n\nfs/binfmt_misc_bpf.c=313=static int bm_bpf_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)\nfs/binfmt_misc_bpf.c-314-{\nfs/binfmt_misc_bpf.c:315:\tif (!btf_id_set8_contains(\u0026bm_bpf_kfunc_ids, kfunc_id))\nfs/binfmt_misc_bpf.c-316-\t\treturn 0;\n--\nfs/bpf_fs_kfuncs.c=530=static int bpf_fs_kfuncs_filter(const struct bpf_prog *prog, u32 kfunc_id)\nfs/bpf_fs_kfuncs.c-531-{\nfs/bpf_fs_kfuncs.c:532:\tif (!btf_id_set8_contains(\u0026bpf_fs_kfunc_set_ids, kfunc_id))\nfs/bpf_fs_kfuncs.c-533-\t\treturn 0;\n--\nfs/verity/measure.c=181=static int bpf_get_fsverity_digest_filter(const struct bpf_prog *prog, u32 kfunc_id)\nfs/verity/measure.c-182-{\nfs/verity/measure.c:183:\tif (!btf_id_set8_contains(\u0026fsverity_set_ids, kfunc_id))\nfs/verity/measure.c-184-\t\treturn 0;\n--\ninclude/linux/btf.h=138=struct btf_kfunc_id_set {\ninclude/linux/btf.h-139-\tstruct module *owner;\ninclude/linux/btf.h:140:\tstruct btf_id_set8 *set;\ninclude/linux/btf.h-141-\tbtf_kfunc_filter_t filter;\n--\ninclude/linux/btf.h=556=static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id)\n--\ninclude/linux/btf.h-560-\ninclude/linux/btf.h:561:static inline struct btf_id_set8_pair *\ninclude/linux/btf.h:562:btf_id_set8_entry(const struct btf_id_set8 *set, u32 id)\ninclude/linux/btf.h-563-{\n--\ninclude/linux/btf.h-566-\ninclude/linux/btf.h:567:static inline bool btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)\ninclude/linux/btf.h-568-{\ninclude/linux/btf.h:569:\treturn btf_id_set8_entry(set, id) != NULL;\ninclude/linux/btf.h-570-}\n--\ninclude/linux/btf_ids.h=8=struct btf_id_set {\n--\ninclude/linux/btf_ids.h-21-\ninclude/linux/btf_ids.h:22:struct btf_id_set8_pair {\ninclude/linux/btf_ids.h-23-\tu32 id;\n--\ninclude/linux/btf_ids.h-26-\ninclude/linux/btf_ids.h:27:struct btf_id_set8 {\ninclude/linux/btf_ids.h-28-\tu32 cnt;\ninclude/linux/btf_ids.h-29-\tu32 flags;\ninclude/linux/btf_ids.h:30:\tstruct btf_id_set8_pair pairs[];\ninclude/linux/btf_ids.h-31-};\n--\ninclude/linux/btf_ids.h=214=asm(\t\t\t\t\t\t\t\\\n--\ninclude/linux/btf_ids.h-217-\".popsection; \\n\");\t\\\ninclude/linux/btf_ids.h:218:extern struct btf_id_set8 name;\ninclude/linux/btf_ids.h-219-\n--\ninclude/linux/btf_ids.h=230=BTF_SET8_END(name)\n--\ninclude/linux/btf_ids.h-243-#define BTF_SET_END(name)\ninclude/linux/btf_ids.h:244:#define BTF_SET8_START(name) static struct btf_id_set8 __maybe_unused name = { 0 };\ninclude/linux/btf_ids.h-245-#define BTF_SET8_END(name)\ninclude/linux/btf_ids.h:246:#define BTF_KFUNCS_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_KFUNCS };\ninclude/linux/btf_ids.h-247-#define BTF_KFUNCS_END(name)\ninclude/linux/btf_ids.h:248:#define BTF_FMODRET_START(name) static struct btf_id_set8 __maybe_unused name = { .flags = BTF_SET8_FMODRET };\ninclude/linux/btf_ids.h-249-#define BTF_FMODRET_END(name)\n--\nkernel/bpf/btf.c=240=struct btf_kfunc_set_tab {\nkernel/bpf/btf.c:241:\tstruct btf_id_set8 *sets[BTF_KFUNC_HOOK_MAX];\nkernel/bpf/btf.c-242-\tstruct btf_kfunc_hook_filter hook_filters[BTF_KFUNC_HOOK_MAX];\n--\nkernel/bpf/btf.c=9872=static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook,\n--\nkernel/bpf/btf.c-9875-\tstruct btf_kfunc_hook_filter *hook_filter;\nkernel/bpf/btf.c:9876:\tstruct btf_id_set8 *add_set = kset-\u003eset;\nkernel/bpf/btf.c-9877-\tbool vmlinux_set = !btf_is_module(btf);\n--\nkernel/bpf/btf.c-9879-\tstruct btf_kfunc_set_tab *tab;\nkernel/bpf/btf.c:9880:\tstruct btf_id_set8 *set;\nkernel/bpf/btf.c-9881-\tu32 set_cnt, i;\n--\nkernel/bpf/btf.c-9931-\t * hence re-sorting the final set again is required to make binary\nkernel/bpf/btf.c:9932:\t * searching the set using btf_id_set8_contains function work.\nkernel/bpf/btf.c-9933-\t *\n--\nkernel/bpf/btf.c=9983=static u32 *btf_kfunc_id_set_contains(const struct btf *btf,\n--\nkernel/bpf/btf.c-9986-{\nkernel/bpf/btf.c:9987:\tstruct btf_id_set8_pair *pair;\nkernel/bpf/btf.c:9988:\tstruct btf_id_set8 *set;\nkernel/bpf/btf.c-9989-\n--\nkernel/bpf/btf.c-9996-\t\treturn NULL;\nkernel/bpf/btf.c:9997:\tpair = btf_id_set8_entry(set, kfunc_btf_id);\nkernel/bpf/btf.c-9998-\tif (!pair)\n--\nkernel/bpf/btf.c=10226=static int btf_register_fmodret_tags(struct btf *btf)\n--\nkernel/bpf/btf.c-10229-\tstruct btf_kfunc_set_tab *tab;\nkernel/bpf/btf.c:10230:\tstruct btf_id_set8 *set;\nkernel/bpf/btf.c-10231-\n--\nkernel/bpf/btf.c-10253-\tfor (id = btf_start_id(btf); id \u003c nr_types; id++) {\nkernel/bpf/btf.c:10254:\t\tstruct btf_id_set8_pair *pair;\nkernel/bpf/btf.c-10255-\n--\nkernel/bpf/btf.c-10259-\t\t\tcontinue;\nkernel/bpf/btf.c:10260:\t\tpair = btf_id_set8_entry(set, func_id);\nkernel/bpf/btf.c-10261-\t\tif (pair)\n--\nkernel/bpf/helpers.c=5269=static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)\nkernel/bpf/helpers.c-5270-{\nkernel/bpf/helpers.c:5271:\tif (!btf_id_set8_contains(\u0026user_mem_btf_ids, kfunc_id))\nkernel/bpf/helpers.c-5272-\t\treturn 0;\n--\nkernel/sched/ext/cid.h=69=extern struct scx_cid_topo __rcu *scx_cid_topo;\nkernel/sched/ext/cid.h:70:extern struct btf_id_set8 scx_kfunc_ids_init_cids;\nkernel/sched/ext/cid.h:71:extern struct btf_id_set8 scx_kfunc_ids_cid;\nkernel/sched/ext/cid.h-72-\n--\nkernel/sched/ext/ext.c=12249=int scx_kfunc_context_filter(const struct bpf_prog *prog, u32 kfunc_id)\nkernel/sched/ext/ext.c-12250-{\nkernel/sched/ext/ext.c:12251:\tbool in_unlocked = btf_id_set8_contains(\u0026scx_kfunc_ids_unlocked, kfunc_id);\nkernel/sched/ext/ext.c:12252:\tbool in_init_cids = btf_id_set8_contains(\u0026scx_kfunc_ids_init_cids, kfunc_id);\nkernel/sched/ext/ext.c:12253:\tbool in_select_cpu = btf_id_set8_contains(\u0026scx_kfunc_ids_select_cpu, kfunc_id);\nkernel/sched/ext/ext.c:12254:\tbool in_enqueue = btf_id_set8_contains(\u0026scx_kfunc_ids_enqueue_dispatch, kfunc_id);\nkernel/sched/ext/ext.c:12255:\tbool in_dispatch = btf_id_set8_contains(\u0026scx_kfunc_ids_dispatch, kfunc_id);\nkernel/sched/ext/ext.c:12256:\tbool in_cpu_release = btf_id_set8_contains(\u0026scx_kfunc_ids_cpu_release, kfunc_id);\nkernel/sched/ext/ext.c:12257:\tbool in_idle = btf_id_set8_contains(\u0026scx_kfunc_ids_idle, kfunc_id);\nkernel/sched/ext/ext.c:12258:\tbool in_any = btf_id_set8_contains(\u0026scx_kfunc_ids_any, kfunc_id);\nkernel/sched/ext/ext.c:12259:\tbool in_cpu_only = btf_id_set8_contains(\u0026scx_kfunc_ids_cpu_only, kfunc_id);\nkernel/sched/ext/ext.c:12260:\tbool in_cid = btf_id_set8_contains(\u0026scx_kfunc_ids_cid, kfunc_id);\nkernel/sched/ext/ext.c:12261:\tbool in_no_tracing = btf_id_set8_contains(\u0026scx_kfunc_ids_no_tracing, kfunc_id);\nkernel/sched/ext/ext.c-12262-\tu32 moff, flags;\n--\nkernel/sched/ext/idle.h=17=struct task_struct;\nkernel/sched/ext/idle.h-18-\nkernel/sched/ext/idle.h:19:extern struct btf_id_set8 scx_kfunc_ids_idle;\nkernel/sched/ext/idle.h:20:extern struct btf_id_set8 scx_kfunc_ids_select_cpu;\nkernel/sched/ext/idle.h-21-\n--\nkernel/trace/bpf_trace.c=3466=static int bpf_session_filter(const struct bpf_prog *prog, u32 kfunc_id)\nkernel/trace/bpf_trace.c-3467-{\nkernel/trace/bpf_trace.c:3468:\tif (!btf_id_set8_contains(\u0026session_kfunc_set_ids, kfunc_id))\nkernel/trace/bpf_trace.c-3469-\t\treturn 0;\n--\nnet/core/bpf_ksock.c=343=static int bpf_ksock_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)\nnet/core/bpf_ksock.c-344-{\nnet/core/bpf_ksock.c:345:\tif (!btf_id_set8_contains(\u0026ksock_kfunc_btf_ids, kfunc_id))\nnet/core/bpf_ksock.c-346-\t\treturn 0;\n--\nnet/core/filter.c=13075=static int tracing_iter_filter(const struct bpf_prog *prog, u32 kfunc_id)\nnet/core/filter.c-13076-{\nnet/core/filter.c:13077:\tif (btf_id_set8_contains(\u0026bpf_sk_iter_kfunc_ids, kfunc_id) \u0026\u0026\nnet/core/filter.c-13078-\t prog-\u003eexpected_attach_type != BPF_TRACE_ITER)\n--\nnet/core/xdp.c=988=bool bpf_dev_bound_kfunc_id(u32 btf_id)\nnet/core/xdp.c-989-{\nnet/core/xdp.c:990:\treturn btf_id_set8_contains(\u0026xdp_metadata_kfunc_ids, btf_id);\nnet/core/xdp.c-991-}\n--\nnet/ipv4/bpf_tcp_ops.c=389=static int bpf_tcp_ops_set_flags_kfunc_filter(const struct bpf_prog *prog,\n--\nnet/ipv4/bpf_tcp_ops.c-391-{\nnet/ipv4/bpf_tcp_ops.c:392:\tif (!btf_id_set8_contains(\u0026bpf_tcp_ops_set_flags_kfunc_set, kfunc_id))\nnet/ipv4/bpf_tcp_ops.c-393-\t\treturn 0;\n--\nnet/ipv4/bpf_tcp_ops.c=412=static int bpf_tcp_ops_set_rcvlowat_kfunc_filter(const struct bpf_prog *prog,\n--\nnet/ipv4/bpf_tcp_ops.c-416-\nnet/ipv4/bpf_tcp_ops.c:417:\tif (!btf_id_set8_contains(\u0026bpf_tcp_ops_set_rcvlowat_kfunc_set, kfunc_id))\nnet/ipv4/bpf_tcp_ops.c-418-\t\treturn 0;\n--\nnet/sched/bpf_qdisc.c=318=static int bpf_qdisc_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)\n--\nnet/sched/bpf_qdisc.c-321-\nnet/sched/bpf_qdisc.c:322:\tif (!btf_id_set8_contains(\u0026qdisc_kfunc_ids, kfunc_id))\nnet/sched/bpf_qdisc.c-323-\t\treturn 0;\n--\ntools/bpf/resolve_btfids/main.c=978=static int sets_patch(struct object *obj)\n--\ntools/bpf/resolve_btfids/main.c-985-\twhile (next) {\ntools/bpf/resolve_btfids/main.c:986:\t\tstruct btf_id_set8 *set8 = NULL;\ntools/bpf/resolve_btfids/main.c-987-\t\tstruct btf_id_set *set = NULL;\n--\ntools/bpf/resolve_btfids/main.c=1378=static int collect_set_funcs(struct object *obj, struct btf2btf_context *ctx,\n--\ntools/bpf/resolve_btfids/main.c-1389-\tfor (next = rb_first(\u0026obj-\u003esets); next; next = rb_next(next)) {\ntools/bpf/resolve_btfids/main.c:1390:\t\tstruct btf_id_set8 *set8;\ntools/bpf/resolve_btfids/main.c-1391-\t\tstruct btf_id *set_id;\n--\ntools/include/linux/btf_ids.h=8=struct btf_id_set {\n--\ntools/include/linux/btf_ids.h-21-\ntools/include/linux/btf_ids.h:22:struct btf_id_set8_pair {\ntools/include/linux/btf_ids.h-23-\tu32 id;\n--\ntools/include/linux/btf_ids.h-26-\ntools/include/linux/btf_ids.h:27:struct btf_id_set8 {\ntools/include/linux/btf_ids.h-28-\tu32 cnt;\ntools/include/linux/btf_ids.h-29-\tu32 flags;\ntools/include/linux/btf_ids.h:30:\tstruct btf_id_set8_pair pairs[];\ntools/include/linux/btf_ids.h-31-};\n--\ntools/include/linux/btf_ids.h=213=asm(\t\t\t\t\t\t\t\\\n\n"
} |
| 41/3 | 2026/11/12 15:35 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 84030 output: 31 thoughts: 212 Thoughts: **Internal Kernel Structures and Userspace Exposure** Okay, so `btf_id_set8` is exclusively an internal kernel concern, used for kfunc and fmodret verification. No issues there, it's not exposed to userspace. Now, let's turn our attention to `struct bpf_prog_aux`. Specifically, I need to evaluate the potential for information leakage through `fmodret_retvals` and `fmodret_retval_cnt`. Are either of these made available to the outside world? The obvious place to start is the `bpf_prog_get_info_by_fd` function within `kernel/bpf/syscall.c`. I need to review how this function populates the `bpf_prog_info` structure and determine if any fields from `prog_aux`, specifically `fmodret_retvals` or `fmodret_retval_cnt`, are copied into the accessible `bpf_prog_info` struct. That's where the investigation needs to focus. |
| 42/3 | 2026/11/12 15:35 | tool | grepper |
0mArgs:{
"Expression": "fmodret_retval",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "kernel/bpf/syscall.c=2488=static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)\n--\nkernel/bpf/syscall.c-2492-\tkfree(prog-\u003eaux-\u003ectx_arg_info);\nkernel/bpf/syscall.c:2493:\tkfree(prog-\u003eaux-\u003efmodret_retvals);\nkernel/bpf/syscall.c-2494-\tif (prog-\u003eaux-\u003eattach_btf)\n--\nkernel/bpf/verifier.c=13088=static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn,\n--\nkernel/bpf/verifier.c-13567-\t\t */\nkernel/bpf/verifier.c:13568:\t\tif (env-\u003eprog-\u003eaux-\u003efmodret_retvals) {\nkernel/bpf/verifier.c-13569-\t\t\tverbose(env, \"tail calls are not allowed in fmod_ret programs with restricted return values\\n\");\n--\nkernel/bpf/verifier.c=20239=static bool program_returns_void(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-20272- * A function which the build found by analysing its code has a\nkernel/bpf/verifier.c:20273: * \"bpf_fmodret_retval:\" decl tag with the values it returns on paths without\nkernel/bpf/verifier.c:20274: * side effects, for example \"bpf_fmodret_retval:-22,-12\". See the comment at\nkernel/bpf/verifier.c-20275- * the top of tools/bpf/resolve_btfids/main.c. An fmod_ret program attached to\n--\nkernel/bpf/verifier.c-20277- */\nkernel/bpf/verifier.c:20278:#define BTF_FMODRET_RETVAL_TAG \"bpf_fmodret_retval:\"\nkernel/bpf/verifier.c-20279-\nkernel/bpf/verifier.c-20280-/* Parse the next value of the list at *LIST and move *LIST past it */\nkernel/bpf/verifier.c:20281:static int fmodret_retval_next(const char **list, s64 *value)\nkernel/bpf/verifier.c-20282-{\n--\nkernel/bpf/verifier.c-20294-\nkernel/bpf/verifier.c:20295:static bool fmodret_retval_allowed(const struct bpf_prog_aux *aux, u64 value)\nkernel/bpf/verifier.c-20296-{\n--\nkernel/bpf/verifier.c-20305-\t\treturn true;\nkernel/bpf/verifier.c:20306:\tfor (i = 0; i \u003c aux-\u003efmodret_retval_cnt; i++) {\nkernel/bpf/verifier.c:20307:\t\tif (aux-\u003efmodret_retvals[i] == (s64)value)\nkernel/bpf/verifier.c-20308-\t\t\treturn true;\n--\nkernel/bpf/verifier.c-20313-/* The program must return a constant: 0 or one of the values of the tag */\nkernel/bpf/verifier.c:20314:static int check_fmodret_retval(struct bpf_verifier_env *env, int regno,\nkernel/bpf/verifier.c-20315-\t\t\t\tconst char *exit_ctx, const char *reg_name)\n--\nkernel/bpf/verifier.c-20321-\tif (reg-\u003etype == SCALAR_VALUE \u0026\u0026 tnum_is_const(reg-\u003evar_off) \u0026\u0026\nkernel/bpf/verifier.c:20322:\t fmodret_retval_allowed(aux, reg-\u003evar_off.value))\nkernel/bpf/verifier.c-20323-\t\treturn 0;\n--\nkernel/bpf/verifier.c-20325-\tverbose(env, \"%s the register %s must be a constant, it should be 0 or one of\", exit_ctx, reg_name);\nkernel/bpf/verifier.c:20326:\tfor (i = 0; i \u003c aux-\u003efmodret_retval_cnt; i++)\nkernel/bpf/verifier.c:20327:\t\tverbose(env, \"%s%lld\", i ? \",\" : \" \", aux-\u003efmodret_retvals[i]);\nkernel/bpf/verifier.c-20328-\tverbose(env, \"\\n\");\n--\nkernel/bpf/verifier.c=20332=static int check_return_code(struct bpf_verifier_env *env, int regno, const char *reg_name)\n--\nkernel/bpf/verifier.c-20387-\nkernel/bpf/verifier.c:20388:\tif (env-\u003eprog-\u003eaux-\u003efmodret_retvals)\nkernel/bpf/verifier.c:20389:\t\treturn check_fmodret_retval(env, regno, exit_ctx, reg_name);\nkernel/bpf/verifier.c-20390-\n--\nkernel/bpf/verifier.c=23823=static int check_attach_modify_return(unsigned long addr, const char *func_name)\n--\nkernel/bpf/verifier.c-23835- * Programs attached to a function found by the analysis may return only the\nkernel/bpf/verifier.c:23836: * values of its \"bpf_fmodret_retval:\" tag. A function which the static rules\nkernel/bpf/verifier.c-23837- * allow stays unrestricted.\nkernel/bpf/verifier.c-23838- */\nkernel/bpf/verifier.c:23839:static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,\nkernel/bpf/verifier.c-23840-\t\t\t unsigned long addr, const char *tname)\n--\nkernel/bpf/verifier.c-23868-\tfor (list = values, cnt = 0; *list;) {\nkernel/bpf/verifier.c:23869:\t\tif (fmodret_retval_next(\u0026list, \u0026v) || v \u003e= 0 || (size == 4 \u0026\u0026 v \u003c S32_MIN))\nkernel/bpf/verifier.c-23870-\t\t\tgoto invalid;\n--\nkernel/bpf/verifier.c-23875-\nkernel/bpf/verifier.c:23876:\taux-\u003efmodret_retvals = retvals;\nkernel/bpf/verifier.c:23877:\taux-\u003efmodret_retval_cnt = cnt;\nkernel/bpf/verifier.c-23878-\t/* The interpreter returns only 32 bits */\n--\nkernel/bpf/verifier.c=24108=int bpf_check_attach_target(struct bpf_verifier_log *log,\n--\nkernel/bpf/verifier.c-24307-\t\t */\nkernel/bpf/verifier.c:24308:\t\tif (prog_extension \u0026\u0026 tgt_prog-\u003eaux-\u003efmodret_retvals) {\nkernel/bpf/verifier.c-24309-\t\t\tbpf_log(log, \"Cannot extend an fmod_ret program with restricted return values\\n\");\n--\nkernel/bpf/verifier.c=24602=static int check_attach_btf_id(struct bpf_verifier_env *env)\n--\nkernel/bpf/verifier.c-24690-\nkernel/bpf/verifier.c:24691:\t/* Only a kernel function has a \"bpf_fmodret_retval:\" tag */\nkernel/bpf/verifier.c-24692-\tif (prog-\u003eexpected_attach_type == BPF_MODIFY_RETURN \u0026\u0026 !tgt_prog) {\nkernel/bpf/verifier.c:24693:\t\tret = fmodret_retval_init(env, btf_id, tgt_info.tgt_addr, tgt_info.tgt_name);\nkernel/bpf/verifier.c-24694-\t\tif (ret)\n"
} |
| 43/3 | 2026/11/12 15:35 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 86597 output: 31 thoughts: 119 Thoughts: **Investigating Potential Uninitialized Memory Read in BPF Program Information Retrieval** Okay, so I'm diving into this BPF code, and it seems like we're *not* copying information to userspace directly through `bpf_prog_get_info_by_fd`. That's good to know, but... hold on. My gut is telling me there's something else at play here, and I need to check my assumptions. Specifically, I need to look at the interaction with those attach target checks – `bpf_check_attach_target`, `check_attach_btf_id`, that whole family of functions. Are we potentially creating a way to read uninitialized memory through `tgt_prog->aux->fmodret_retvals` in those functions? That's the key question. Now, I *do* remember that `prog->aux` is allocated using `kzalloc` in either `bpf_prog_alloc` or `bpf_prog_alloc_no_stats`. But that *kzalloc* call isn't an ironclad guarantee of zero initialization for all members of that structure, so I need to verify *exactly* how `prog->aux` is allocated, to make sure it's behaving like I think it should. I'm going to follow the allocation path to confirm this. Let's dig deeper and see what's what. |
| 44/3 | 2026/11/12 15:36 | tool | grepper |
0mArgs:{
"Expression": "prog-\u003eaux = ",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "no matches"
} |
| 45/3 | 2026/11/12 15:36 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 87058 output: 32 thoughts: 24 Thoughts: Here's my thought process, as requested: **Investigating BPF Program Allocation** Okay, so I need to understand how BPF programs are being allocated within the kernel. My immediate thought is to zero in on the core allocation function. "bpf_prog_alloc" is the likely candidate for the primary allocation mechanism. It's a pretty standard naming convention. I'll need to see how and where this function is called, as that'll quickly paint a picture of the BPF program lifecycle. I'll start by grepping the kernel sources. It makes sense to start in `kernel/bpf/core.c` as the core BPF functionality is likely centered there. That's the best place to start looking for the allocation details. |
| 46/3 | 2026/11/12 15:36 | tool | grepper |
0mArgs:{
"Expression": "bpf_prog_alloc",
"PathPrefixes": [
"kernel/bpf/"
]
}Results: {
"Output": "kernel/bpf/core.c=98=enum page_size_enum {\n--\nkernel/bpf/core.c-101-\nkernel/bpf/core.c:102:struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flags)\nkernel/bpf/core.c-103-{\n--\nkernel/bpf/core.c-148-\nkernel/bpf/core.c:149:struct bpf_prog *bpf_prog_alloc(unsigned int size, gfp_t gfp_extra_flags)\nkernel/bpf/core.c-150-{\n--\nkernel/bpf/core.c-154-\nkernel/bpf/core.c:155:\tprog = bpf_prog_alloc_no_stats(size, gfp_extra_flags);\nkernel/bpf/core.c-156-\tif (!prog)\n--\nkernel/bpf/core.c-175-\nkernel/bpf/core.c:176:int bpf_prog_alloc_jited_linfo(struct bpf_prog *prog)\nkernel/bpf/core.c-177-{\n--\nkernel/bpf/core.c=3042=struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,\n--\nkernel/bpf/core.c-3064-\tif (!bpf_prog_is_offloaded(fp-\u003eaux)) {\nkernel/bpf/core.c:3065:\t\t*err = bpf_prog_alloc_jited_linfo(fp);\nkernel/bpf/core.c-3066-\t\tif (*err)\n--\nkernel/bpf/fixups.c=1854=static int jit_subprogs(struct bpf_verifier_env *env)\n--\nkernel/bpf/fixups.c-1899-\nkernel/bpf/fixups.c:1900:\terr = bpf_prog_alloc_jited_linfo(prog);\nkernel/bpf/fixups.c-1901-\tif (err)\n--\nkernel/bpf/fixups.c-1918-\t\t */\nkernel/bpf/fixups.c:1919:\t\tfunc[i] = bpf_prog_alloc_no_stats(bpf_prog_size(len), GFP_USER);\nkernel/bpf/fixups.c-1920-\t\tif (!func[i])\n--\nkernel/bpf/syscall.c=2403=static void bpf_audit_prog(const struct bpf_prog *prog, unsigned int op)\n--\nkernel/bpf/syscall.c-2421-\nkernel/bpf/syscall.c:2422:static int bpf_prog_alloc_id(struct bpf_prog *prog)\nkernel/bpf/syscall.c-2423-{\n--\nkernel/bpf/syscall.c=2953=static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_attr *attr_log)\n--\nkernel/bpf/syscall.c-3077-\t/* plain bpf_prog allocation */\nkernel/bpf/syscall.c:3078:\tprog = bpf_prog_alloc(bpf_prog_size(attr-\u003einsn_cnt), GFP_USER);\nkernel/bpf/syscall.c-3079-\tif (!prog) {\n--\nkernel/bpf/syscall.c-3180-\nkernel/bpf/syscall.c:3181:\terr = bpf_prog_alloc_id(prog);\nkernel/bpf/syscall.c-3182-\tif (err)\n--\nkernel/bpf/syscall.c-3184-\nkernel/bpf/syscall.c:3185:\t/* Upon success of bpf_prog_alloc_id(), the BPF prog is\nkernel/bpf/syscall.c-3186-\t * effectively publicly exposed. However, retrieving via\n--\nkernel/bpf/trampoline.c=1122=static struct bpf_shim_tramp_link *cgroup_shim_alloc(const struct bpf_prog *prog,\n--\nkernel/bpf/trampoline.c-1133-\nkernel/bpf/trampoline.c:1134:\tp = bpf_prog_alloc(1, 0);\nkernel/bpf/trampoline.c-1135-\tif (!p) {\n"
} |
| 47/3 | 2026/11/12 15:36 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 88438 output: 36 thoughts: 27 Thoughts: Okay, let's dive into this. Here's my thought process laid out: **Examining `bpf_prog_alloc_no_stats`** Alright, I need to understand this function, `bpf_prog_alloc_no_stats`, and its place within the broader BPF framework. That means I need to start by navigating to it within the kernel source code. Specifically, the file indicated is `kernel/bpf/core.c`. This strongly suggests that it's a fundamental part of BPF program management and allocation, since `core.c` often houses core functionalities. I'm immediately interested in how this function operates at a low level, considering I probably already know its purpose. Given the name, I'm expecting it probably allocates a BPF program without tracking the stats. I bet it has other counterparts that track stats. I'll need to look at its implementation for more context. I'm also ready to trace its call sites to understand where it's used within the broader context of BPF program loading and execution. Knowing this can help me understand the implications of it. |
| 48/3 | 2026/11/12 15:36 | tool | read-file |
0mArgs:{
"File": "kernel/bpf/core.c",
"FirstLine": 102,
"LineCount": 40
}Results: {
"Contents": " 102:\tstruct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flags)\n 103:\t{\n 104:\t\tgfp_t gfp_flags = bpf_memcg_flags(GFP_KERNEL | __GFP_ZERO | gfp_extra_flags);\n 105:\t\tstruct bpf_prog_aux *aux;\n 106:\t\tstruct bpf_prog *fp;\n 107:\t\n 108:\t\tsize = round_up(size, __PAGE_SIZE);\n 109:\t\tfp = __vmalloc(size, gfp_flags);\n 110:\t\tif (fp == NULL)\n 111:\t\t\treturn NULL;\n 112:\t\n 113:\t\taux = kzalloc_obj(*aux, bpf_memcg_flags(GFP_KERNEL | gfp_extra_flags));\n 114:\t\tif (aux == NULL) {\n 115:\t\t\tvfree(fp);\n 116:\t\t\treturn NULL;\n 117:\t\t}\n 118:\t\tfp-\u003eactive = __alloc_percpu_gfp(sizeof(u8[BPF_NR_CONTEXTS]), 4,\n 119:\t\t\t\t\t\tbpf_memcg_flags(GFP_KERNEL | gfp_extra_flags));\n 120:\t\tif (!fp-\u003eactive) {\n 121:\t\t\tvfree(fp);\n 122:\t\t\tkfree(aux);\n 123:\t\t\treturn NULL;\n 124:\t\t}\n 125:\t\n 126:\t\tfp-\u003epages = size / PAGE_SIZE;\n 127:\t\tfp-\u003eaux = aux;\n 128:\t\tfp-\u003eaux-\u003emain_prog_aux = aux;\n 129:\t\tfp-\u003eaux-\u003eprog = fp;\n 130:\t\tfp-\u003ejit_requested = ebpf_jit_enabled();\n 131:\t\tfp-\u003ejit_required = IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON);\n 132:\t\tfp-\u003eblinding_requested = bpf_jit_blinding_enabled(fp);\n 133:\t#ifdef CONFIG_CGROUP_BPF\n 134:\t\taux-\u003ecgroup_atype = CGROUP_BPF_ATTACH_TYPE_INVALID;\n 135:\t#endif\n 136:\t\n 137:\t\tINIT_LIST_HEAD_RCU(\u0026fp-\u003eaux-\u003eksym.lnode);\n 138:\t#ifdef CONFIG_FINEIBT\n 139:\t\tINIT_LIST_HEAD_RCU(\u0026fp-\u003eaux-\u003eksym_prefix.lnode);\n 140:\t#endif\n 141:\t\tmutex_init(\u0026fp-\u003eaux-\u003eused_maps_mutex);\n"
} |
| 49/3 | 2026/11/12 15:36 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 89446 output: 519 thoughts: 5349 Thoughts: **KMSAN Assessment: NeedsKMSAN = false** Okay, so I've gone through this patch series with a fine-toothed comb, considering every angle related to uninitialized memory. Initially, the `aux = kzalloc_obj(*aux, ...)` call made me double-check - because that zeros out the `aux` struct, so `fmodret_retvals` starts out NULL, and `fmodret_retval_cnt` at zero by default. But that's just the starting state, and I need to look closer at what happens *after*. Let's see... I made sure to eliminate any lingering thoughts about `KMSAN_SANITIZE` flags in Makefiles, because those don't seem to apply in the relevant locations. I went through the four criteria for `NeedsKMSAN=true` and made sure there were no kernel structures copied to userspace, no conditional logic on uninitialized memory, and no changed bounds checks that open the door to reading uninitialized bytes. So, this patch series introduces an LLVM plugin to scan kernel IR for functions that are candidates for `fmodret` annotations, which the build tools then parse to create BTF metadata. The kernel-side code uses this metadata to restrict the return values of BPF programs attached to these annotated functions. Now, let's break down the relevant kernel modifications: `btf_register_fmodret_tags` is critical. It allocates `struct btf_id_set8` with `kmalloc` and, *crucially*, explicitly initializes `cnt`, `flags`, and every entry in the `pairs[]` array. `fmodret_retval_init` does something similar by parsing a string and populating the `retvals` array using `kmalloc_array`, ensuring all elements are properly initialized. In addition, the aux memory is zeroed out by `kzalloc_obj`, and that means `fmodret_retvals` starts as NULL, and is initialized or set to specific values after that. No new kernel data structures or memory are copied to user space, eliminating information-leak concerns. Any issues related to the parsing of string tags, or possible out of bounds issues, are not related to uninitialized memory, but KASAN issues. The same applies for possible memory safety bugs in array indexing. Given all of this, there aren't any potential uninitialized memory reads or leaks here that would *only* be detected by KMSAN. Anything that could go wrong in this patch would be out-of-bounds access, a memory lifetime issue, or possibly an integer overflow, which KASAN, KMEMLEAK, or standard bug detectors would catch. Therefore, the risk is not that of *uninitialized* memory, but something else entirely that's readily catchable by existing detectors. KMSAN is definitely not required. The conclusion is, `NeedsKMSAN = false`. |
| 50/3 | 2026/11/12 15:37 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch series introduces build-time and verification mechanisms for BPF fmod_ret targets. Specifically, it adds an LLVM pass plugin to scan IR for clean error-returning functions, records metadata in ELF sections, annotates BTF types with declaration tags in resolve_btfids, and updates the kernel BPF verifier and BTF subsystem to restrict fmod_ret return values based on these tags.\n\nFrom a memory sanitization perspective:\n1. No uninitialized memory usage:\n - In `kernel/bpf/btf.c`, `btf_register_fmodret_tags()` allocates `struct btf_id_set8` using `kmalloc()` and explicitly initializes `cnt`, `flags`, and all populated `pairs[]` elements (`id` and `flags`).\n - In `kernel/bpf/verifier.c`, `fmodret_retval_init()` parses tag values and allocates `retvals` via `kmalloc_array()`, writing every element sequentially up to `cnt`.\n - `prog-\u003eaux` is allocated using `kzalloc_obj()`, ensuring that `fmodret_retvals` and `fmodret_retval_cnt` are zero-initialized by default.\n - Stack buffers (e.g. `char buf[24]` in `fmodret_retval_next()`) are fully initialized (`memcpy` followed by explicit null termination) before being passed to `kstrtos64()`.\n2. No kernel-to-user info-leaks:\n - The newly introduced structures (`struct btf_id_set8`, `aux-\u003efmodret_retvals`) are kernel-internal tracking structures and are never copied to userspace via `copy_to_user`, `put_user`, socket options, or BPF syscall inspection interfaces.\n3. KASAN vs KMSAN applicability:\n - Potential risks in this series pertain to bounds checking, string parsing/tokenization, or dynamic array management. Any out-of-bounds access or use-after-free bugs in these paths are already covered by KASAN.\n - There are no uninitialized memory reads, uninitialized padding leaks, or uninitialized branch conditions that would uniquely require KMSAN.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch series introduces build-time and verification mechanisms for BPF fmod_ret targets. Specifically, it adds an LLVM pass plugin to scan IR for clean error-returning functions, records metadata in ELF sections, annotates BTF types with declaration tags in resolve_btfids, and updates the kernel BPF verifier and BTF subsystem to restrict fmod_ret return values based on these tags.\n\nFrom a memory sanitization perspective:\n1. No uninitialized memory usage:\n - In `kernel/bpf/btf.c`, `btf_register_fmodret_tags()` allocates `struct btf_id_set8` using `kmalloc()` and explicitly initializes `cnt`, `flags`, and all populated `pairs[]` elements (`id` and `flags`).\n - In `kernel/bpf/verifier.c`, `fmodret_retval_init()` parses tag values and allocates `retvals` via `kmalloc_array()`, writing every element sequentially up to `cnt`.\n - `prog-\u003eaux` is allocated using `kzalloc_obj()`, ensuring that `fmodret_retvals` and `fmodret_retval_cnt` are zero-initialized by default.\n - Stack buffers (e.g. `char buf[24]` in `fmodret_retval_next()`) are fully initialized (`memcpy` followed by explicit null termination) before being passed to `kstrtos64()`.\n2. No kernel-to-user info-leaks:\n - The newly introduced structures (`struct btf_id_set8`, `aux-\u003efmodret_retvals`) are kernel-internal tracking structures and are never copied to userspace via `copy_to_user`, `put_user`, socket options, or BPF syscall inspection interfaces.\n3. KASAN vs KMSAN applicability:\n - Potential risks in this series pertain to bounds checking, string parsing/tokenization, or dynamic array management. Any out-of-bounds access or use-after-free bugs in these paths are already covered by KASAN.\n - There are no uninitialized memory reads, uninitialized padding leaks, or uninitialized branch conditions that would uniquely require KMSAN.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|