KCOV records which code a task reached. kcov_dataflow records what values flowed through it: the arguments an instrumented function was called with and the value it returned, per task, in execution order. This gives a coverage-guided fuzzer a signal that plain edge coverage cannot provide two calls to the same function with different arguments look identical to edge coverage but differ here. The values come from two SanitizerCoverage callbacks the compiler inserts at function boundaries: __sanitizer_cov_trace_args at entry, once per argument __sanitizer_cov_trace_ret at return For a pointer to a struct the callback passes the field offset table the compiler derived from DWARF, and the kernel expands the struct into its individual field values rather than recording an opaque address. Reads of traced memory use copy_from_kernel_nofault() (a typed get_kernel_nofault() for the 1/2/4/8-byte scalar cases), so a NULL or ERR_PTR the callee received is recorded as KCOV_DF_MAGIC_BAD instead of faulting. The passes are not upstream; they need a clang/rustc built with the trace-args/trace-ret RFC [1]. When a task has no session enabled the whole path is one boolean check. Core (kernel/kcov_dataflow.c, split out of kcov.c on request): - Own debugfs device /sys/kernel/debug/kcov_dataflow with its own ioctl namespace ('d') and per-task mmap'd buffer, independent of KCOV, so both can run at once. - The session is a refcounted object (kcov_df_get/put) with a back-pointer from the task, mirroring mainline kcov's t->kcov: fd close from a sibling thread, a forked child, and task exit can no longer race into a use-after-free. task_struct gains kcov_df and the small kcov_df_* working set (include/linux/sched.h); fork clears it (kernel/fork.c) and exit tears any session down (kernel/exit.c). - Local (KCOV_DF_ENABLE) writes reserve buffer space with a plain area[0] update; remote (KCOV_DF_REMOTE_ENABLE) writes merge with a bounded atomic64_try_cmpxchg() loop. The two are mutually exclusive on one buffer, so they never both update area[0]. - kcov_df_inert_context() rejects !in_task() and, crucially, pagefault_disabled() context. The trace-cmp callback is reachable from the ORC unwinder that KASAN runs on every slab free (stack_trace_save()); copy_from_kernel_nofault() brackets its loads with pagefault_disable(), so gating on that flag contains a whole class of self-instrumentation storms that would otherwise trip the soft-lockup watchdog, with no coverage exclusion needed in mm/ or arch/. - A per-task sequence guard (bit 31) suppresses re-entry nested inside our own callback under INSTRUMENT_ALL. uapi (include/uapi/linux/kcov_dataflow.h): area[0] counts the record words that follow; each record is a header word (sequence, type, value count, size, argument index), the instrumented PC with the KASLR offset removed like the PCs mainline kcov records, the traced pointer (ENTRY/RET) or the comparison type (CMP), then the value words. kcov.c: the __sanitizer_cov_trace_cmp*() / trace_switch() callbacks now route their operand pairs through kcov_trace_cmp() in , which feeds both the existing KCOV_MODE_TRACE_CMP buffer and, when a dataflow session is live, the dataflow buffer; kcov.c itself no longer references dataflow. kcov_check_handle() moves to as a static inline so kcov_dataflow.c can validate KCOV_DF_REMOTE_ENABLE handles with it. objtool (tools/objtool/check.c): list kcov_df_trace_cmp() and the two trace-args/ret entry points in uaccess_safe_builtin[], next to their mainline peer write_comp_data(). The compiler emits the cmp callbacks inside user_access_begin()/end() regions, and calling out of such a region is only allowed to listed functions. They qualify on the same grounds: notrace, __no_sanitize_coverage, no locks, no allocation, storing only into the task's own buffer; kcov_df_reserve() is __always_inline so the property does not depend on an inlining decision. Build system (scripts/Makefile.kcov, scripts/Makefile.lib): - CFLAGS_KCOV_DATAFLOW = -fsanitize-coverage=trace-args,trace-ret (plus -fno-inline under CONFIG_KCOV_DATAFLOW_NO_INLINE), and the matching RUSTFLAGS for Rust objects. - Per-file opt-in KCOV_DATAFLOW_.o := y, or whole-kernel with CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL; KCOV_INSTRUMENT := n still excludes noinstr objects. - kcov_dataflow.o is built without KCOV, KASAN, KCSAN, UBSAN or KMSAN instrumentation to keep the collector from tracing itself. Kconfig (lib/Kconfig.debug): CONFIG_KCOV_DATAFLOW_ARGS / _RET (depend on KCOV, CC_IS_CLANG, DEBUG_INFO and the cc-option/rustc-option probe for the pass), CONFIG_KCOV_DATAFLOW_NO_INLINE and CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL. Because the options gate on cc-option, a stock clang simply leaves them unset rather than failing. [1] https://discourse.llvm.org/t/rfc-sanitizercoverage-add-fsanitize-coverage-trace-args-trace-ret/91026 [2] https://github.com/llvm/llvm-project/pull/201410 [3] https://github.com/llvm/llvm-project/pull/218254 [4] https://github.com/llvm/llvm-project/pull/218265 Link: https://github.com/yskzalloc/kcov-dataflow/ Signed-off-by: Yunseong Kim --- include/linux/kcov.h | 116 ++++ include/linux/sched.h | 34 + include/uapi/linux/kcov_dataflow.h | 92 +++ kernel/Makefile | 9 + kernel/exit.c | 1 + kernel/fork.c | 1 + kernel/kcov.c | 56 +- kernel/kcov_dataflow.c | 1193 ++++++++++++++++++++++++++++++++++++ lib/Kconfig.debug | 52 ++ scripts/Makefile.kcov | 17 + scripts/Makefile.lib | 14 + tools/objtool/check.c | 4 + 12 files changed, 1558 insertions(+), 31 deletions(-) diff --git a/include/linux/kcov.h b/include/linux/kcov.h index 895b761b2db15..55e1405bc4bc4 100644 --- a/include/linux/kcov.h +++ b/include/linux/kcov.h @@ -3,6 +3,7 @@ #define _LINUX_KCOV_H #include +#include #include struct task_struct; @@ -28,6 +29,14 @@ enum kcov_mode { void kcov_task_init(struct task_struct *t); void kcov_task_exit(struct task_struct *t); +#if defined(CONFIG_KCOV_DATAFLOW_ARGS) || defined(CONFIG_KCOV_DATAFLOW_RET) +void kcov_dataflow_task_init(struct task_struct *t); +void kcov_dataflow_task_exit(struct task_struct *t); +#else +static inline void kcov_dataflow_task_init(struct task_struct *t) {} +static inline void kcov_dataflow_task_exit(struct task_struct *t) {} +#endif + #define kcov_prepare_switch(t) \ do { \ (t)->kcov_mode |= KCOV_IN_CTXSW; \ @@ -43,6 +52,29 @@ void kcov_remote_start(u64 handle); void kcov_remote_stop(void); struct kcov_common_handle_id kcov_common_handle(void); +/* + * Validate a remote handle: it must be a well-formed kcov_remote_handle() + * encoding, and each caller states which subsystem/instance combinations it + * accepts. Shared by KCOV_REMOTE_ENABLE and KCOV_DF_REMOTE_ENABLE so both + * collectors take handles from the same partitioned namespace. + */ +static inline bool kcov_check_handle(u64 handle, bool common_valid, + bool uncommon_valid, bool zero_valid) +{ + if (handle & ~(KCOV_SUBSYSTEM_MASK | KCOV_INSTANCE_MASK)) + return false; + switch (handle & KCOV_SUBSYSTEM_MASK) { + case KCOV_SUBSYSTEM_COMMON: + return (handle & KCOV_INSTANCE_MASK) ? + common_valid : zero_valid; + case KCOV_SUBSYSTEM_USB: + return uncommon_valid; + default: + return false; + } + return false; +} + static inline void kcov_remote_start_common(struct kcov_common_handle_id id) { kcov_remote_start(kcov_remote_handle(KCOV_SUBSYSTEM_COMMON, id.val)); @@ -107,4 +139,88 @@ static inline void kcov_remote_start_usb_softirq(u64 id) {} static inline void kcov_remote_stop_softirq(void) {} #endif /* CONFIG_KCOV */ + +/* + * kcov_dataflow remote API. The collector is a separate object from mainline + * kcov and is only linked in when at least one of the two capture modes is + * configured (see kernel/Makefile), so gate the declarations the same way + * kcov_dataflow_task_init() above is gated; a caller that brackets a region for + * both collectors then still builds on a KCOV-only config. + */ +#if defined(CONFIG_KCOV_DATAFLOW_ARGS) || defined(CONFIG_KCOV_DATAFLOW_RET) +void kcov_df_remote_start(u64 handle); +void kcov_df_remote_stop(void); +#else +static inline void kcov_df_remote_start(u64 handle) {} +static inline void kcov_df_remote_stop(void) {} +#endif + +/* + * Handle-typed wrapper mirroring kcov_remote_start_common(), so a subsystem that + * already routes its mainline kcov remote sections by struct + * kcov_common_handle_id can open a dataflow section on the very same handle + * without knowing how it is encoded. The two collectors keep separate per-task + * state and separate handle tables, so a section of each may be nested around + * the same region; user space registers the identical handle value with + * KCOV_REMOTE_ENABLE and KCOV_DF_REMOTE_ENABLE to collect both. + * + * Unlike kcov_remote_start(), the dataflow section may only be opened from + * sleepable task context: kcov_df_remote_start()/kcov_df_remote_stop() take a + * mutex and may allocate or free the worker's scratch area. Both are no-ops in + * softirq/hardirq context, so a softirq-bracketing call site collects no + * dataflow records rather than misbehaving. A call site that is only + * sometimes atomic (spinlock held, preemption or irqs disabled) must not use + * this wrapper; CONFIG_DEBUG_ATOMIC_SLEEP reports such a caller. + * + * Without CONFIG_KCOV the handle carries no value (see struct + * kcov_common_handle_id), and dataflow depends on KCOV, so this is a no-op. + */ +#ifdef CONFIG_KCOV +static inline void kcov_df_remote_start_common(struct kcov_common_handle_id id) +{ + kcov_df_remote_start(kcov_remote_handle(KCOV_SUBSYSTEM_COMMON, id.val)); +} +#else +static inline void kcov_df_remote_start_common(struct kcov_common_handle_id id) +{ +} +#endif +#if defined(CONFIG_KCOV_ENABLE_COMPARISONS) && \ + (defined(CONFIG_KCOV_DATAFLOW_ARGS) || defined(CONFIG_KCOV_DATAFLOW_RET)) +/* + * CONFIG_KCOV_ENABLE_COMPARISONS provides ONE trace-cmp instrumentation shared by + * mainline kcov and kcov-dataflow. kcov.c's __sanitizer_cov_trace_cmp*() callbacks + * route each operand pair through kcov_trace_cmp() below, which fans it out: + * mainline kcov always sees it (write_comp_data() records only when the task is + * in KCOV_MODE_TRACE_CMP), and a task with a live dataflow session gets a copy in + * its dataflow buffer as well. The two collectors are independent fds with no + * cross-exclusion, so a task may collect for both at once, and a dataflow-side + * drop (inert context, full buffer) never costs mainline kcov a record. kcov.c + * never references the dataflow side, one cmp symbol feeds both collectors, and + * there is no separate df_cmp symbol or compiler change. + * + * The dataflow branch is gated by a static key so that, while no dataflow session + * is live, this whole-kernel hot path is a patched-out NOP that costs nothing on + * top of mainline write_comp_data() (kcov_df_cmp_key is inc'd on dataflow enable + * in kcov_dataflow.c). + */ +DECLARE_STATIC_KEY_FALSE(kcov_df_cmp_key); +void write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip); +void kcov_df_trace_cmp(u64 type, u64 arg1, u64 arg2, u64 ip); +static inline notrace void +kcov_trace_cmp(u64 type, u64 arg1, u64 arg2, u64 ip) +{ + write_comp_data(type, arg1, arg2, ip); /* mainline kcov */ + if (static_branch_unlikely(&kcov_df_cmp_key) && current->kcov_df_enabled) + kcov_df_trace_cmp(type, arg1, arg2, ip); /* kcov-dataflow */ +} +#elif defined(CONFIG_KCOV_ENABLE_COMPARISONS) +/* Comparisons without a dataflow build: route straight to mainline kcov. */ +void write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip); +static inline notrace void +kcov_trace_cmp(u64 type, u64 arg1, u64 arg2, u64 ip) +{ + write_comp_data(type, arg1, arg2, ip); +} +#endif #endif /* _LINUX_KCOV_H */ diff --git a/include/linux/sched.h b/include/linux/sched.h index 83416924701e8..49e506fd616c0 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1553,6 +1553,40 @@ struct task_struct { /* KCOV sequence number: */ int kcov_sequence; +#if defined(CONFIG_KCOV_DATAFLOW_ARGS) || defined(CONFIG_KCOV_DATAFLOW_RET) + /* + * KCOV dataflow per-task record sequence counter (24 bits used) plus, + * in bit 31, the recursion guard held while a callback is running: + */ + u32 kcov_df_seq; + + /* KCOV dataflow: separate buffer for trace-args/trace-ret */ + unsigned int kcov_df_size; + void *kcov_df_area; + bool kcov_df_enabled; + + /* + * The kcov_dataflow object this task's session belongs to, NULL when + * no session is active. The task holds a reference on it for the whole + * session, whether local (KCOV_DF_ENABLE, mirrors t->kcov) or remote + * (kcov_df_remote_start()), so the buffer can never be freed under an + * instrumented callback and both task exit and kcov_df_remote_stop() + * reach the exact object without a hash lookup. + */ + struct kcov_dataflow *kcov_df; + + /* + * Nesting depth of kcov_df_remote_start() on this task: 0 while no + * remote session is active (including during a local session), 1 for + * a normal bracketed work item. If a buggy caller nests, the inner + * start()s only bump this and the inner stop()s only decrement it, so + * the OUTER session (buffer + ref) is torn down exactly once, at the + * outermost stop -- never early, which would otherwise drop the ref + * and free the buffer out from under the still-running outer worker. + */ + int kcov_df_remote_depth; +#endif + /* Collect coverage from softirq context: */ unsigned int kcov_softirq; diff --git a/include/uapi/linux/kcov_dataflow.h b/include/uapi/linux/kcov_dataflow.h new file mode 100644 index 0000000000000..db3112a45832c --- /dev/null +++ b/include/uapi/linux/kcov_dataflow.h @@ -0,0 +1,92 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _LINUX_KCOV_DATAFLOW_H +#define _LINUX_KCOV_DATAFLOW_H + +#include +#include + +/* + * User space ABI of /sys/kernel/debug/kcov_dataflow, see + * Documentation/dev-tools/kcov-dataflow.rst. + * + * KCOV_DF_INIT_TRACK takes the buffer size in u64 words by value (same + * convention as KCOV_INIT_TRACE). KCOV_DF_REMOTE_ENABLE takes a pointer to a + * __u64 remote handle encoded with kcov_remote_handle() (linux/kcov.h), so the + * full 64-bit value survives 32-bit and compat callers. + */ +#define KCOV_DF_INIT_TRACK _IOR('d', 1, unsigned long) +#define KCOV_DF_ENABLE _IO('d', 100) +#define KCOV_DF_DISABLE _IO('d', 101) +#define KCOV_DF_REMOTE_ENABLE _IOW('d', 102, __u64) +#define KCOV_DF_REMOTE_DISABLE _IO('d', 103) + +/* + * Buffer layout (all u64 words): + * + * area[0] number of record words written after area[0] + * area[1 + n ..] records, back to back, each: + * + * [0] header see KCOV_DF_HDR_* below + * [1] pc instrumented location; KASLR offset removed, like + * the PCs mainline kcov records + * [2] ENTRY/RET: the traced value's address (full pointer); may be a + * NULL/ERR_PTR value the callee received, in which case the + * value words hold KCOV_DF_MAGIC_BAD + * CMP: comparison type, KCOV_CMP_SIZE()/KCOV_CMP_CONST bits + * (linux/kcov.h) + * [3 .. 3 + nvals) value words: the scalar (nvals == 1), the expanded + * struct fields, or the two CMP operands (nvals == 2) + * + * The header packs: + * + * bits 0..23 per-task record sequence number + * bits 28..31 record type, KCOV_DF_TYPE_* + * bits 32..47 nvals, the number of value words that follow word [2] + * bits 48..55 ENTRY/RET: size in bytes of the traced argument/return value + * (clamped to 255) + * bits 56..63 ENTRY: argument index (clamped to 255); RET: 0 + * + * A consumer walks the buffer as + * + * pos = 1; + * while (pos < 1 + area[0]) { + * hdr = area[pos]; + * nvals = KCOV_DF_HDR_NVALS(hdr); + * ... + * pos += KCOV_DF_RECORD_WORDS(nvals); + * } + * + * area[0] never exceeds the buffer size minus one, and every counted word has + * been written, so the walk above stays inside the mapping. + */ +#define KCOV_DF_TYPE_CMP 0xC +#define KCOV_DF_TYPE_ENTRY 0xE +#define KCOV_DF_TYPE_RET 0xF + +#define KCOV_DF_HDR_SEQ_MASK 0x00FFFFFFULL +#define KCOV_DF_HDR_TYPE_SHIFT 28 +#define KCOV_DF_HDR_TYPE_MASK 0xFULL +#define KCOV_DF_HDR_NVALS_SHIFT 32 +#define KCOV_DF_HDR_NVALS_MASK 0xFFFFULL +#define KCOV_DF_HDR_SIZE_SHIFT 48 +#define KCOV_DF_HDR_SIZE_MASK 0xFFULL +#define KCOV_DF_HDR_ARGIDX_SHIFT 56 +#define KCOV_DF_HDR_ARGIDX_MASK 0xFFULL + +#define KCOV_DF_HDR_SEQ(h) ((h) & KCOV_DF_HDR_SEQ_MASK) +#define KCOV_DF_HDR_TYPE(h) (((h) >> KCOV_DF_HDR_TYPE_SHIFT) & KCOV_DF_HDR_TYPE_MASK) +#define KCOV_DF_HDR_NVALS(h) (((h) >> KCOV_DF_HDR_NVALS_SHIFT) & KCOV_DF_HDR_NVALS_MASK) +#define KCOV_DF_HDR_SIZE(h) (((h) >> KCOV_DF_HDR_SIZE_SHIFT) & KCOV_DF_HDR_SIZE_MASK) +#define KCOV_DF_HDR_ARGIDX(h) (((h) >> KCOV_DF_HDR_ARGIDX_SHIFT) & KCOV_DF_HDR_ARGIDX_MASK) + +/* Words per record: header, pc, pointer/cmp-type, then the value words. */ +#define KCOV_DF_RECORD_HDR_WORDS 3 +#define KCOV_DF_RECORD_WORDS(nvals) (KCOV_DF_RECORD_HDR_WORDS + (nvals)) + +/* Largest nvals a record can carry; longer field lists are truncated. */ +#define KCOV_DF_MAX_VALS KCOV_DF_HDR_NVALS_MASK + +/* Value word written when the traced pointer or a field could not be read. */ +#define KCOV_DF_MAGIC_BAD 0xBADADD85ULL + +#endif /* _LINUX_KCOV_DATAFLOW_H */ diff --git a/kernel/Makefile b/kernel/Makefile index 1e1a31673577d..307b7fd1e1f96 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -44,6 +44,12 @@ KCSAN_SANITIZE_kcov.o := n UBSAN_SANITIZE_kcov.o := n KMSAN_SANITIZE_kcov.o := n +KCOV_INSTRUMENT_kcov_dataflow.o := n +KASAN_SANITIZE_kcov_dataflow.o := n +KCSAN_SANITIZE_kcov_dataflow.o := n +UBSAN_SANITIZE_kcov_dataflow.o := n +KMSAN_SANITIZE_kcov_dataflow.o := n + CONTEXT_ANALYSIS_kcov.o := y CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) -fno-stack-protector @@ -98,6 +104,9 @@ obj-$(CONFIG_AUDIT) += audit.o auditfilter.o obj-$(CONFIG_AUDITSYSCALL) += auditsc.o audit_watch.o audit_fsnotify.o audit_tree.o obj-$(CONFIG_GCOV_KERNEL) += gcov/ obj-$(CONFIG_KCOV) += kcov.o +ifneq ($(CONFIG_KCOV_DATAFLOW_ARGS)$(CONFIG_KCOV_DATAFLOW_RET),) +obj-y += kcov_dataflow.o +endif obj-$(CONFIG_KPROBES) += kprobes.o obj-$(CONFIG_FAIL_FUNCTION) += fail_function.o obj-$(CONFIG_KGDB) += debug/ diff --git a/kernel/exit.c b/kernel/exit.c index 97686af895013..8881661d635ba 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -939,6 +939,7 @@ void __noreturn do_exit(long code) kthread_do_exit(kthread, code); kcov_task_exit(tsk); + kcov_dataflow_task_exit(tsk); kmsan_task_exit(tsk); synchronize_group_exit(tsk, code); diff --git a/kernel/fork.c b/kernel/fork.c index 416758c8a3d43..d26b9dd39872e 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -985,6 +985,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) tsk->worker_private = NULL; kcov_task_init(tsk); + kcov_dataflow_task_init(tsk); kmsan_task_create(tsk); kmap_local_fork(tsk); diff --git a/kernel/kcov.c b/kernel/kcov.c index 35420f0ac524d..cac9b69e197ed 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -232,7 +232,14 @@ void notrace __sanitizer_cov_trace_pc(void) EXPORT_SYMBOL(__sanitizer_cov_trace_pc); #ifdef CONFIG_KCOV_ENABLE_COMPARISONS -static void notrace write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip) +/* + * Mainline kcov comparison writer: appends to the task's own kcov buffer, and + * only in KCOV_MODE_TRACE_CMP. The fan-out that also feeds the kcov-dataflow + * buffer lives in kcov_trace_cmp() in , so kcov.c never references + * the dataflow side itself. This writer is only non-static so that header helper + * (which the cmp callbacks below call) can reach it. + */ +void notrace write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip) { struct task_struct *t; u64 *area; @@ -267,55 +274,59 @@ static void notrace write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip) } } +/* + * The __sanitizer_cov_trace_cmp*() callbacks stay here in kcov.c (one shared, + * compiler-emitted symbol per comparison -- no separate df_cmp symbol, no + * compiler change). Each routes its operand pair through kcov_trace_cmp() + * (defined in ), which records into mainline kcov and, when this + * task has a dataflow session, into kcov-dataflow too. kcov.c never names the + * dataflow side; that fan-out lives entirely in the header. + */ void notrace __sanitizer_cov_trace_cmp1(u8 arg1, u8 arg2) { - write_comp_data(KCOV_CMP_SIZE(0), arg1, arg2, _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(0), arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_cmp1); void notrace __sanitizer_cov_trace_cmp2(u16 arg1, u16 arg2) { - write_comp_data(KCOV_CMP_SIZE(1), arg1, arg2, _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(1), arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_cmp2); void notrace __sanitizer_cov_trace_cmp4(u32 arg1, u32 arg2) { - write_comp_data(KCOV_CMP_SIZE(2), arg1, arg2, _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(2), arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_cmp4); void notrace __sanitizer_cov_trace_cmp8(kcov_u64 arg1, kcov_u64 arg2) { - write_comp_data(KCOV_CMP_SIZE(3), arg1, arg2, _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(3), arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_cmp8); void notrace __sanitizer_cov_trace_const_cmp1(u8 arg1, u8 arg2) { - write_comp_data(KCOV_CMP_SIZE(0) | KCOV_CMP_CONST, arg1, arg2, - _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(0) | KCOV_CMP_CONST, arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_const_cmp1); void notrace __sanitizer_cov_trace_const_cmp2(u16 arg1, u16 arg2) { - write_comp_data(KCOV_CMP_SIZE(1) | KCOV_CMP_CONST, arg1, arg2, - _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(1) | KCOV_CMP_CONST, arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_const_cmp2); void notrace __sanitizer_cov_trace_const_cmp4(u32 arg1, u32 arg2) { - write_comp_data(KCOV_CMP_SIZE(2) | KCOV_CMP_CONST, arg1, arg2, - _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(2) | KCOV_CMP_CONST, arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_const_cmp4); void notrace __sanitizer_cov_trace_const_cmp8(kcov_u64 arg1, kcov_u64 arg2) { - write_comp_data(KCOV_CMP_SIZE(3) | KCOV_CMP_CONST, arg1, arg2, - _RET_IP_); + kcov_trace_cmp(KCOV_CMP_SIZE(3) | KCOV_CMP_CONST, arg1, arg2, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_const_cmp8); @@ -344,7 +355,7 @@ void notrace __sanitizer_cov_trace_switch(kcov_u64 val, void *arg) return; } for (i = 0; i < count; i++) - write_comp_data(type, cases[i + 2], val, _RET_IP_); + kcov_trace_cmp(type, cases[i + 2], val, _RET_IP_); } EXPORT_SYMBOL(__sanitizer_cov_trace_switch); #endif /* ifdef CONFIG_KCOV_ENABLE_COMPARISONS */ @@ -587,23 +598,6 @@ static void kcov_fault_in_area(struct kcov *kcov) READ_ONCE(area[offset]); } -static inline bool kcov_check_handle(u64 handle, bool common_valid, - bool uncommon_valid, bool zero_valid) -{ - if (handle & ~(KCOV_SUBSYSTEM_MASK | KCOV_INSTANCE_MASK)) - return false; - switch (handle & KCOV_SUBSYSTEM_MASK) { - case KCOV_SUBSYSTEM_COMMON: - return (handle & KCOV_INSTANCE_MASK) ? - common_valid : zero_valid; - case KCOV_SUBSYSTEM_USB: - return uncommon_valid; - default: - return false; - } - return false; -} - static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd, unsigned long arg) __must_hold(&kcov->lock) diff --git a/kernel/kcov_dataflow.c b/kernel/kcov_dataflow.c new file mode 100644 index 0000000000000..641d6bc763864 --- /dev/null +++ b/kernel/kcov_dataflow.c @@ -0,0 +1,1193 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KCOV Dataflow: per-task function argument/return value capture. + * + * Exposes /sys/kernel/debug/kcov_dataflow, completely independent from + * /sys/kernel/debug/kcov. Own buffer, own ioctl, own mmap. + * + * The user-visible ABI: + * + * ioctls, the record layout and the header bit fields, is defined in + * . In short, every record is + * + * [hdr][pc][ptr or cmp type][nvals value words] + * + * appended after area[0], which counts the record words written so far. + */ +#define pr_fmt(fmt) "kcov_dataflow: " fmt + +#define DISABLE_BRANCH_PROFILING +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * Comparison capture is shared with mainline kcov; it only exists when both the + * trace-cmp instrumentation and the dataflow task state are configured in. + */ +#if defined(CONFIG_KCOV_ENABLE_COMPARISONS) && \ + (defined(CONFIG_KCOV_DATAFLOW_ARGS) || defined(CONFIG_KCOV_DATAFLOW_RET)) +#define KCOV_DF_HAVE_CMP 1 +#endif + +#define KCOV_DF_IS_ERR(p) ((unsigned long)(p) >= (unsigned long)-4095UL) + +/* + * Bit 31 of task_struct::kcov_df_seq is the per-task recursion guard, held + * while one of the callbacks below runs. The record sequence number lives in + * the low 24 bits (KCOV_DF_HDR_SEQ_MASK) and is advanced with kcov_df_next_seq() + * so that it wraps inside its own field and can never carry into the guard. + */ +#define KCOV_DF_SEQ_GUARD BIT(31) + +/* + * Per-worker private scratch size (u64 words), KCOV's remote-area model: a + * remote kworker collects into its OWN scratch and merges it into the shared + * ->area at kcov_df_remote_stop(). Fixed and small (8 MiB) -- one work item's + * coverage, not a whole buffer -- so the pool of recycled scratch areas stays + * bounded regardless of how many kworkers churn. Overflowing a scratch just + * drops that worker's excess records (same as a full buffer), never corrupts. + */ +#define KCOV_DF_REMOTE_WORDS (1UL << 20) + +struct kcov_dataflow { + struct mutex lock; + unsigned int size; /* in u64 words */ + void *area; + /* + * Task with a local (KCOV_DF_ENABLE) session on this object, NULL if + * none. Mirrors struct kcov::t: that task holds its own reference (see + * ->refcount) and points back at us through task_struct::kcov_df, so + * KCOV_DF_DISABLE, close() and task exit all unwire the same session. + */ + struct task_struct *t; + /* + * Lifetime refcount (KCOV's struct kcov pattern). The open fd holds one + * ref; the task enabled with KCOV_DF_ENABLE holds one for as long as its + * session lasts (dropped by KCOV_DF_DISABLE, by close() from that task, + * or by task exit -- it cannot be unwired from another task); each + * kcov_df_remote_start() takes one and the matching kcov_df_remote_stop() + * drops it. Whoever drops the LAST ref frees ->area and the object + * (kcov_df_put), so an instrumented callback can never write through a + * freed buffer, whichever task does the final close(). + */ + refcount_t refcount; + u64 remote_handle; /* handle for remote lookup, 0 if not published */ +#ifdef KCOV_DF_HAVE_CMP + /* + * Whether this fd holds a ref on kcov_df_cmp_key, tracked SEPARATELY for + * the local (KCOV_DF_ENABLE) and remote (KCOV_DF_REMOTE_ENABLE) sources. + * A single shared flag let a KCOV_DF_DISABLE drop the key while a remote + * handle was still published -- silently losing the live remote workers' + * comparison records. Two flags mean releasing one source never pulls the + * key out from under the other. Both are only touched under ->lock. + */ + bool cmp_key_local; + bool cmp_key_remote; +#endif +}; + +/* Which activation source holds the cmp static key (see kcov_df_cmp_key_hold). */ +enum { KCOV_DF_CMP_LOCAL, KCOV_DF_CMP_REMOTE }; + +#ifdef KCOV_DF_HAVE_CMP +/* + * Static key gating the per-comparison dataflow check in kcov_trace_cmp() + * (linux/kcov.h). It is a patched-out NOP until at least one dataflow session is + * live, so trace-cmp across the WHOLE kernel costs nothing extra while no + * dataflow fuzzing runs; only an active session flips it on. Refcounted: inc on + * each source's first enable, dec on its disable/close/exit (idempotent, + * tracked per source via cmp_key_local / cmp_key_remote so releasing one never + * drops the key from under the other). + * + * The key is only ever inc'd/dec'd from ioctl, close() and do_exit() context, + * under df->lock -- never from kcov_df_remote_stop() or the last kcov_df_put(), + * so a subsystem's worker path never ends up under cpus_read_lock() and + * jump_label_mutex. The static_branch_{inc,dec}() text-patch is amortised -- it + * fires only on the 0->1 and 1->0 transitions, not per fd while sessions overlap. + */ +DEFINE_STATIC_KEY_FALSE(kcov_df_cmp_key); +EXPORT_SYMBOL(kcov_df_cmp_key); + +static void kcov_df_cmp_key_hold(struct kcov_dataflow *df, int which) +{ + bool *held = which == KCOV_DF_CMP_LOCAL ? &df->cmp_key_local + : &df->cmp_key_remote; + + lockdep_assert_held(&df->lock); + if (!*held) { + *held = true; + static_branch_inc(&kcov_df_cmp_key); + } +} + +static void kcov_df_cmp_key_release(struct kcov_dataflow *df, int which) +{ + bool *held = which == KCOV_DF_CMP_LOCAL ? &df->cmp_key_local + : &df->cmp_key_remote; + + lockdep_assert_held(&df->lock); + if (*held) { + *held = false; + static_branch_dec(&kcov_df_cmp_key); + } +} + +static bool kcov_df_cmp_key_held(struct kcov_dataflow *df) +{ + return df->cmp_key_local || df->cmp_key_remote; +} +#else +static void kcov_df_cmp_key_hold(struct kcov_dataflow *df, int which) {} +static void kcov_df_cmp_key_release(struct kcov_dataflow *df, int which) {} +static bool kcov_df_cmp_key_held(struct kcov_dataflow *df) { return false; } +#endif + +/* Remote dataflow: handle-based lookup (follows KCOV's kcov_remote_map pattern) */ +static DEFINE_MUTEX(kcov_df_remote_lock); +static DEFINE_HASHTABLE(kcov_df_remote_map, 4); + +struct kcov_df_remote { + u64 handle; + struct kcov_dataflow *df; + struct hlist_node hnode; +}; + +static struct kcov_df_remote *kcov_df_remote_find(u64 handle) +{ + struct kcov_df_remote *remote; + + hash_for_each_possible(kcov_df_remote_map, remote, hnode, handle) { + if (remote->handle == handle) + return remote; + } + return NULL; +} + +/* Unpublish @df's remote handle, if any; no new remote session can start. */ +static void kcov_df_remote_unpublish(struct kcov_dataflow *df) +{ + struct kcov_df_remote *remote; + + mutex_lock(&kcov_df_remote_lock); + if (df->remote_handle) { + remote = kcov_df_remote_find(df->remote_handle); + if (remote) { + hash_del(&remote->hnode); + kfree(remote); + } + df->remote_handle = 0; + } + mutex_unlock(&kcov_df_remote_lock); +} + +static void kcov_df_get(struct kcov_dataflow *df) +{ + refcount_inc(&df->refcount); +} + +/* + * Drop a reference; the last one frees the buffer and the object. Only called + * from sleepable task context (ioctl, close(), do_exit(), and remote_stop() + * which requires it), so vfree() here is fine. No caller may touch @df after + * its own kcov_df_put(). Every path that unwires a session releases its cmp + * key ref under df->lock first, so nothing is left to balance here. + */ +static void kcov_df_put(struct kcov_dataflow *df) +{ + if (refcount_dec_and_test(&df->refcount)) { + WARN_ON_ONCE(kcov_df_cmp_key_held(df)); + vfree(df->area); + kfree(df); + } +} + +/* + * Touch every page of a buffer before a task starts collecting into it, the + * same way kcov_fault_in_area() does for KCOV_ENABLE: on configurations with + * lazily populated vmalloc mappings the first access would otherwise fault + * from inside an instrumented callback, and code on the vmalloc fault path may + * itself be instrumented. + */ +static void kcov_df_fault_in_area(u64 *area, unsigned long size) +{ + unsigned long stride = PAGE_SIZE / sizeof(u64); + unsigned long off; + + for (off = 0; off < size; off += stride) + READ_ONCE(area[off]); +} + +/* + * Pool of recycled per-worker scratch areas (KCOV's kcov_remote_areas). All are + * KCOV_DF_REMOTE_WORDS u64s. While parked on the freelist the area's first bytes + * hold this list_head; while in use word[0] is the scratch write cursor. Guarded + * by kcov_df_remote_lock. + */ +struct kcov_df_scratch { + struct list_head list; +}; +static LIST_HEAD(kcov_df_scratch_pool); +static unsigned long kcov_df_scratch_pool_nr; /* idle areas parked in the pool */ + +/* Take a scratch area from the pool, or NULL if empty (caller vmalloc()s one). */ +static void *kcov_df_scratch_get(void) +{ + struct kcov_df_scratch *s; + + if (list_empty(&kcov_df_scratch_pool)) + return NULL; + s = list_first_entry(&kcov_df_scratch_pool, struct kcov_df_scratch, list); + list_del(&s->list); + kcov_df_scratch_pool_nr--; + return s; +} + +/* Return a scratch area to the pool for reuse. */ +static void kcov_df_scratch_put(void *area) +{ + struct kcov_df_scratch *s = area; + + INIT_LIST_HEAD(&s->list); + list_add(&s->list, &kcov_df_scratch_pool); + kcov_df_scratch_pool_nr++; +} + +/* + * Merge a remote worker's private scratch into the shared ->area, appending its + * records at the shared write cursor. This is the ONE many-writers path (several + * kworkers merge concurrently), so it claims its region with a cmpxchg loop on + * area[0]: the bounds are checked against the value about to be committed, and + * the commit only happens when the record fits. area[0] therefore never exceeds + * the buffer capacity and every counted word has been written, so a consumer + * walking area[0] words stays inside its mapping. A concurrent reset by user + * space (writing area[0] = 0 to restart collection) simply makes the cmpxchg + * fail and the loop re-read the new cursor; there is no subtract, so the + * counter can never go negative or wrap past the bounds check. Each merge claims + * a disjoint [start, start+n), so concurrent merges don't overlap and need no + * lock. @df is kept alive by the caller's reference, so ->area is stable here. + * + * ->area is never written through kcov_df_reserve() while a remote handle is + * published (KCOV_DF_ENABLE refuses that), so this atomic cursor update never + * races a plain read-modify-write of the same word. + */ +static void kcov_df_merge(struct kcov_dataflow *df, const u64 *scratch) +{ + u64 *area = df->area; + atomic64_t *cursor; + u64 n, count, capacity; + s64 old; + + if (!area) + return; + /* + * scratch[0] is an EXACT high-water of written words: kcov_df_reserve() + * commits the count only after a record fits, so every counted word was + * really written -- the merge never publishes the unwritten + * (recycled/uninitialized) tail of a pooled scratch. The clamp below is thus + * belt-and-suspenders against a stray count. + */ + n = scratch[0]; + if (n > KCOV_DF_REMOTE_WORDS - 1) + n = KCOV_DF_REMOTE_WORDS - 1; + if (!n) + return; + + capacity = df->size - 1; /* words after area[0] */ + cursor = (atomic64_t *)&area[0]; + old = atomic64_read(cursor); + do { + count = old; + /* Full (or a garbage cursor from user space): drop the records. */ + if (count > capacity || n > capacity - count) + return; + } while (!atomic64_try_cmpxchg(cursor, &old, count + n)); + memcpy(&area[1 + count], &scratch[1], n * sizeof(u64)); +} + +/* + * Reserve @record_len u64 words in the current task's buffer. On success return + * true and store the 1-based start index of the record's data region. + * + * Single-writer discipline, identical to mainline kcov.c: the current task is the + * ONLY instrumented writer of @area. In remote mode @area is this kworker's OWN + * private scratch; in local (KCOV_DF_ENABLE) mode it is the enabling task's own + * mmapped buffer -- and only one task can hold that (the KCOV_DF_ENABLE EBUSY + * guard, which also refuses a buffer with a published remote handle, so + * kcov_df_merge() never touches this word concurrently). Two tasks never write + * the same @area here, so no atomic is needed: validate FIRST and commit the + * count (area[0]) only on success, so area[0] is always an EXACT high-water of + * written words and no consumer (userspace or kcov_df_merge()) ever sees an + * unwritten/recycled slot. + * + * (Publishing a worker's scratch into the shared ->area is the SEPARATE + * kcov_df_merge() path, which DOES reserve atomically because many kworkers merge + * concurrently.) + * + * READ_ONCE/WRITE_ONCE because in local mode userspace may reset area[0] to 0 + * between operations. That reset can only drive the count to 0, never negative + * (there is no subtract), so a racing reset may drop records but can never produce + * an out-of-bounds store. This is exactly mainline kcov's contract. + * + * __always_inline because kcov_df_trace_cmp() below is on objtool's + * uaccess_safe_builtin[] list, and objtool rejects any out-of-line call made + * from such a function; do not leave that to the optimizer. + */ +static __always_inline notrace __no_sanitize_coverage bool +kcov_df_reserve(struct task_struct *t, u64 *area, u32 record_len, + unsigned long *start_index) +{ + unsigned long count = READ_ONCE(area[0]); + + *start_index = 1 + count; + if (count >= t->kcov_df_size || + record_len > t->kcov_df_size - *start_index) + return false; + WRITE_ONCE(area[0], count + record_len); + return true; +} + +/* + * Contexts where dataflow collection must stay completely inert. + * + * Beyond the obvious !in_task() case, this bails whenever page faults are + * disabled. copy_from_kernel_nofault() -- used by kcov_df_write() below to read + * traced pointers, and, crucially, by the ORC stack unwinder that KASAN runs on + * every slab free (set_track_prepare() -> stack_trace_save()) -- brackets its + * raw loads with pagefault_disable(), and those loads carry trace-cmp/trace-args + * instrumentation. Without this bail a single stack walk under a fuzzing + KASAN + * workload floods the collector with a callback per load and soft-locks the CPU. + * + * pagefault_disabled() is true throughout any such nofault region no matter + * which instrumented leaf issued the callback, so testing it here contains the + * whole class of self-instrumentation storms -- the bit-31 recursion guard below + * only covers re-entry nested inside our own callback, not a fresh entry from + * the unwinder/KASAN path. Contained entirely to this file: no coverage + * exclusion in mm/ or arch/ is needed. + * + * The trade-off is that records are also dropped inside unrelated + * pagefault_disable() regions (kmap_atomic() on HIGHMEM, futex and perf + * callchain probes, ...). Those are short and rare on the fuzzing workloads this + * targets; a per-task "in nofault region" flag would remove the coupling at the + * cost of touching mm/maccess.c. + */ +static __always_inline notrace __no_sanitize_coverage bool +kcov_df_inert_context(void) +{ + return !in_task() || pagefault_disabled(); +} + +/* Same as kcov.c: record PCs with the KASLR offset removed. */ +static __always_inline notrace __no_sanitize_coverage u64 +kcov_df_canonicalize_ip(u64 ip) +{ +#ifdef CONFIG_RANDOMIZE_BASE + ip -= kaslr_offset(); +#endif + return ip; +} + +/* + * Advance the task's 24-bit record sequence number, keeping the guard bit set. + * Masking the increment keeps the counter from ever carrying into + * KCOV_DF_SEQ_GUARD, which would reopen re-entry in the middle of a record. + */ +static __always_inline notrace __no_sanitize_coverage u32 +kcov_df_next_seq(struct task_struct *t) +{ + u32 seq = (t->kcov_df_seq + 1) & KCOV_DF_HDR_SEQ_MASK; + + t->kcov_df_seq = KCOV_DF_SEQ_GUARD | seq; + return seq; +} + +static __always_inline notrace __no_sanitize_coverage u64 +kcov_df_hdr(u64 type, u32 nvals, u32 size, u32 arg_idx, u32 seq) +{ + return (type << KCOV_DF_HDR_TYPE_SHIFT) | + ((u64)nvals << KCOV_DF_HDR_NVALS_SHIFT) | + ((u64)min_t(u32, size, KCOV_DF_HDR_SIZE_MASK) << + KCOV_DF_HDR_SIZE_SHIFT) | + ((u64)min_t(u32, arg_idx, KCOV_DF_HDR_ARGIDX_MASK) << + KCOV_DF_HDR_ARGIDX_SHIFT) | + (seq & KCOV_DF_HDR_SEQ_MASK); +} + +/* + * Core write function for ENTRY/RET records. + * Uses the same READ_ONCE/WRITE_ONCE pattern as write_comp_data() in kcov.c. + * + * @num_fields is the length of the compiler-supplied @offsets table (pairs of + * offset,size) for an expanded struct, 0 for a scalar read directly from @ptr + * with width @size. It is clamped to KCOV_DF_MAX_VALS so the record length can + * never wrap and the field loop is bounded by the words actually reserved. + */ +static noinline notrace __no_sanitize_coverage void +kcov_df_write(u64 type, u64 pc, u32 arg_idx, u32 size, void *ptr, + u64 *offsets, u32 num_fields) +{ + struct task_struct *t = current; + u64 *area; + unsigned long start_index; + u32 nvals, seq, i; + + if (kcov_df_inert_context()) + return; + + if (!t->kcov_df_enabled) + return; + + /* + * Prevent recursion: functions called by this callback + * (copy_from_kernel_nofault) may be instrumented. Use the + * sequence counter's high bit as a per-task guard. + */ + if (t->kcov_df_seq & KCOV_DF_SEQ_GUARD) + return; + t->kcov_df_seq |= KCOV_DF_SEQ_GUARD; + /* Paired with the barrier() before the guard is cleared at out:. */ + barrier(); + + area = (u64 *)t->kcov_df_area; + if (!area) + goto out; + + if (num_fields > KCOV_DF_MAX_VALS) + num_fields = KCOV_DF_MAX_VALS; + /* Record: header + pc + ptr, then the fields or one scalar word. */ + nvals = num_fields > 0 ? num_fields : 1; + + if (!kcov_df_reserve(t, area, KCOV_DF_RECORD_WORDS(nvals), &start_index)) + goto out; + + seq = kcov_df_next_seq(t); + area[start_index] = kcov_df_hdr(type, nvals, size, arg_idx, seq); + area[start_index + 1] = kcov_df_canonicalize_ip(pc); + area[start_index + 2] = (u64)(unsigned long)ptr; + + if (num_fields == 0) { + u64 val = 0; + u32 sz = size; + + /* + * Read the scalar with a compile-time-constant width for the + * common sizes so the compiler folds away copy_from_kernel_ + * nofault()'s runtime size loop and alignment branching; fall + * back to the variable-size byte copy for anything else. A + * faulting read leaves val == 0, matching the prior best-effort + * behaviour. + */ + if (ptr && !KCOV_DF_IS_ERR(ptr)) { + switch (sz) { + case 8: { + u64 v = 0; + + if (!get_kernel_nofault(v, (u64 *)ptr)) + val = v; + break; + } + case 4: { + u32 v = 0; + + if (!get_kernel_nofault(v, (u32 *)ptr)) + val = v; + break; + } + case 2: { + u16 v = 0; + + if (!get_kernel_nofault(v, (u16 *)ptr)) + val = v; + break; + } + case 1: { + u8 v = 0; + + if (!get_kernel_nofault(v, (u8 *)ptr)) + val = v; + break; + } + default: + if (sz > sizeof(val)) + sz = sizeof(val); + copy_from_kernel_nofault(&val, ptr, sz); + } + } + area[start_index + 3] = val; + } else { + if (!ptr || KCOV_DF_IS_ERR(ptr)) { + for (i = 0; i < num_fields; i++) + area[start_index + 3 + i] = KCOV_DF_MAGIC_BAD; + goto out; + } + for (i = 0; i < num_fields; i++) { + u64 off, sz, val = KCOV_DF_MAGIC_BAD; + void *fa; + + if (copy_from_kernel_nofault(&off, &offsets[i * 2], sizeof(off)) || + copy_from_kernel_nofault(&sz, &offsets[i * 2 + 1], sizeof(sz))) { + area[start_index + 3 + i] = KCOV_DF_MAGIC_BAD; + continue; + } + fa = (void *)((unsigned long)ptr + off); + val = 0; + + if (sz <= sizeof(val)) { + if (copy_from_kernel_nofault(&val, fa, sz)) + val = KCOV_DF_MAGIC_BAD; + } else { + if (copy_from_kernel_nofault(&val, fa, sizeof(val))) + val = KCOV_DF_MAGIC_BAD; + } + area[start_index + 3 + i] = val; + } + } +out: + /* + * Paired with the barrier() after setting the guard at the top. + * Ensures all record writes are complete before we clear the + * recursion guard. + */ + barrier(); + t->kcov_df_seq &= ~KCOV_DF_SEQ_GUARD; +} + +/* + * The two compiler-emitted entry points are on objtool's uaccess_safe_builtin[] + * list, like the __sanitizer_cov_trace_cmp*() callbacks. The trace-args call is + * planted before the terminator of the function's entry block (so that every + * spilled value dominates it), not at its first instruction: a function that + * opens a user access region and then does an unsafe_get_user() -- an asm goto, + * hence a block terminator -- gets the callback AFTER the stac, and objtool + * reports "call to __sanitizer_cov_trace_args() with UACCESS enabled". + * + * objtool validates a listed function with AC set and rejects any out-of-line + * call from it, and kcov_df_write() calls copy_from_kernel_nofault(), so bracket + * the call with user_access_save()/restore(): that clears AC for the whole + * record write (the kasan_report() pattern) and keeps SMAP/PAN protection in + * force while the collector runs. It compiles to nothing on architectures + * without the feature. + */ +#ifdef CONFIG_KCOV_DATAFLOW_ARGS +noinline void notrace __no_sanitize_coverage +__sanitizer_cov_trace_args(u64 pc, u32 arg_idx, u32 arg_size, void *arg_ptr, + u64 *offsets, u32 num_fields); + +noinline void notrace __no_sanitize_coverage +__sanitizer_cov_trace_args(u64 pc, u32 arg_idx, u32 arg_size, void *arg_ptr, + u64 *offsets, u32 num_fields) +{ + unsigned long ua_flags = user_access_save(); + + kcov_df_write(KCOV_DF_TYPE_ENTRY, pc, arg_idx, arg_size, arg_ptr, + offsets, num_fields); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(__sanitizer_cov_trace_args); +#endif + +#ifdef CONFIG_KCOV_DATAFLOW_RET +noinline void notrace __no_sanitize_coverage +__sanitizer_cov_trace_ret(u64 pc, u32 ret_size, void *ret_val, + u64 *offsets, u32 num_fields); + +noinline void notrace __no_sanitize_coverage +__sanitizer_cov_trace_ret(u64 pc, u32 ret_size, void *ret_val, + u64 *offsets, u32 num_fields) +{ + unsigned long ua_flags = user_access_save(); + + kcov_df_write(KCOV_DF_TYPE_RET, pc, 0, ret_size, ret_val, + offsets, num_fields); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(__sanitizer_cov_trace_ret); +#endif + +#ifdef KCOV_DF_HAVE_CMP +/* + * Comparison capture (input-to-state). Reached from the shared + * __sanitizer_cov_trace_cmp*() callbacks (kcov.c) via kcov_trace_cmp() + * (linux/kcov.h), which fans out to mainline kcov and, when this task has a + * dataflow session, here as well, so trace-cmp operand pairs land in the SAME + * unified TLV buffer as the arg/ret records. Both operands are recorded, so a + * userspace consumer can use them for input-to-state matching, complementing + * the arg/ret records. + * + * Record: [header(CMP|nvals=2|seq)][pc][cmp_type][arg1][arg2]. + * cmp_type carries KCOV_CMP_SIZE()/KCOV_CMP_CONST bits (see linux/kcov.h) so the + * consumer knows operand width and whether one side was a compile-time constant. + * + * On objtool's uaccess_safe_builtin[] list, so this function makes no + * out-of-line call (kcov_df_reserve() and the helpers are __always_inline). + */ +noinline notrace __no_sanitize_coverage void +kcov_df_trace_cmp(u64 cmp_type, u64 arg1, u64 arg2, u64 ip) +{ + struct task_struct *t = current; + u64 *area; + unsigned long start_index; + u32 seq; + + if (kcov_df_inert_context()) + return; + if (!t->kcov_df_enabled) + return; + /* Same recursion guard as kcov_df_write(): bit 31 of the seq counter. */ + if (t->kcov_df_seq & KCOV_DF_SEQ_GUARD) + return; + t->kcov_df_seq |= KCOV_DF_SEQ_GUARD; + barrier(); + + area = (u64 *)t->kcov_df_area; + if (!area) + goto out; + + /* Single-writer exact-count reservation: see kcov_df_reserve(). */ + if (!kcov_df_reserve(t, area, KCOV_DF_RECORD_WORDS(2), &start_index)) + goto out; + + seq = kcov_df_next_seq(t); + area[start_index] = kcov_df_hdr(KCOV_DF_TYPE_CMP, 2, 0, 0, seq); + area[start_index + 1] = kcov_df_canonicalize_ip(ip); + area[start_index + 2] = cmp_type; + area[start_index + 3] = arg1; + area[start_index + 4] = arg2; +out: + barrier(); + t->kcov_df_seq &= ~KCOV_DF_SEQ_GUARD; +} +EXPORT_SYMBOL(kcov_df_trace_cmp); +#endif /* KCOV_DF_HAVE_CMP */ + +/* Called from kernel/fork.c to clear inherited state. */ +void kcov_dataflow_task_init(struct task_struct *t) +{ + t->kcov_df_area = NULL; + t->kcov_df_size = 0; + t->kcov_df_seq = 0; + t->kcov_df_enabled = false; + t->kcov_df = NULL; + t->kcov_df_remote_depth = 0; +} + +/* Called from kernel/exit.c to tear down the exiting task's session, if any. */ +void kcov_dataflow_task_exit(struct task_struct *t) +{ + struct kcov_dataflow *df = t->kcov_df; + + if (!df) + return; + + if (t->kcov_df_remote_depth > 0) { + /* + * A remote kworker exited between kcov_df_remote_start() and + * _stop() (should not happen -- they bracket a single work item). + * Defensive: drop its partial scratch and release the ref so + * neither the buffer nor the object leaks. + */ + void *scratch = t->kcov_df_area; + + t->kcov_df_enabled = false; + t->kcov_df_area = NULL; + t->kcov_df_size = 0; + t->kcov_df = NULL; + t->kcov_df_remote_depth = 0; + vfree(scratch); + kcov_df_put(df); + return; + } + + /* + * Local (KCOV_DF_ENABLE) session on the exiting task. Mirror + * kcov_task_exit(): unwire the task, clear df->t so the object never + * keeps a pointer to a freed task_struct (which a later ioctl or + * close() would compare against current), release the cmp key this + * session held and drop the session's reference. + */ + t->kcov_df_enabled = false; + t->kcov_df_area = NULL; + t->kcov_df_size = 0; + t->kcov_df = NULL; + + mutex_lock(&df->lock); + WARN_ON_ONCE(df->t != t); + df->t = NULL; + kcov_df_cmp_key_release(df, KCOV_DF_CMP_LOCAL); + mutex_unlock(&df->lock); + kcov_df_put(df); +} + +/* File operations for /sys/kernel/debug/kcov_dataflow */ + +static int kcov_df_open(struct inode *inode, struct file *filep) +{ + struct kcov_dataflow *df; + + df = kzalloc_obj(struct kcov_dataflow, GFP_KERNEL); + if (!df) + return -ENOMEM; + mutex_init(&df->lock); + refcount_set(&df->refcount, 1); /* the open fd's reference */ + filep->private_data = df; + return nonseekable_open(inode, filep); +} + +/* + * Unwire the local session that @current holds on @df. Caller holds df->lock + * and must drop the session's reference with kcov_df_put() after unlocking. + */ +static void kcov_df_disable_local(struct kcov_dataflow *df) +{ + lockdep_assert_held(&df->lock); + WARN_ON_ONCE(df->t != current || current->kcov_df != df); + + current->kcov_df_enabled = false; + current->kcov_df_area = NULL; + current->kcov_df_size = 0; + current->kcov_df = NULL; + df->t = NULL; + kcov_df_cmp_key_release(df, KCOV_DF_CMP_LOCAL); +} + +static int kcov_df_close(struct inode *inode, struct file *filep) +{ + struct kcov_dataflow *df = filep->private_data; + bool put_session = false; + + /* Unpublish from remote hash: no new users can start */ + kcov_df_remote_unpublish(df); + + mutex_lock(&df->lock); + kcov_df_cmp_key_release(df, KCOV_DF_CMP_REMOTE); + /* + * Only the enabled task can unwire its own session. If another task + * (a sibling thread, a fork()ed child, an SCM_RIGHTS recipient) does + * the final close(), the enabled task keeps its reference and keeps + * collecting until it exits, exactly like mainline kcov. + */ + if (df->t == current) { + kcov_df_disable_local(df); + put_session = true; + } + mutex_unlock(&df->lock); + + if (put_session) + kcov_df_put(df); + /* + * Drop the fd's reference. If remote workers or the enabled task still + * hold refs, the LAST of them frees ->area via kcov_df_put() -- no drain + * loop, no lost-decrement wedge. The hash entry was already unpublished + * above, so no new remote user can start on this object. + */ + kcov_df_put(df); + return 0; +} + +static int kcov_df_mmap(struct file *filep, struct vm_area_struct *vma) +{ + struct kcov_dataflow *df = filep->private_data; + unsigned long size, off; + struct page *page; + void *area; + int res = 0; + + mutex_lock(&df->lock); + size = df->size * sizeof(u64); + if (!df->area || vma->vm_pgoff != 0 || + vma->vm_end - vma->vm_start != size) { + res = -EINVAL; + goto out; + } + area = df->area; + mutex_unlock(&df->lock); + + vm_flags_set(vma, VM_DONTEXPAND); + for (off = 0; off < size; off += PAGE_SIZE) { + page = vmalloc_to_page(area + off); + res = vm_insert_page(vma, vma->vm_start + off, page); + if (res) + return res; + } + return 0; +out: + mutex_unlock(&df->lock); + return res; +} + +static long kcov_df_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) +{ + struct kcov_dataflow *df = filep->private_data; + bool put_session = false; + unsigned long size; + u64 handle = 0; + int res = 0; + + /* + * Fetch the remote handle from user space before taking df->lock. + * get_user() may fault and take mmap_lock, but kcov_df_mmap() takes + * df->lock while holding mmap_lock -- doing the copy under df->lock + * would invert that order and deadlock (reported by lockdep). + */ + if (cmd == KCOV_DF_REMOTE_ENABLE && get_user(handle, (u64 __user *)arg)) + return -EFAULT; + + mutex_lock(&df->lock); + switch (cmd) { + case KCOV_DF_INIT_TRACK: + if (df->area) { + res = -EBUSY; + break; + } + size = arg; + if (size < 2 || size > (128 << 20) / sizeof(u64)) { + res = -EINVAL; + break; + } + mutex_unlock(&df->lock); + { + void *area = vmalloc_user(size * sizeof(u64)); + + if (!area) + return -ENOMEM; + mutex_lock(&df->lock); + if (df->area) { + mutex_unlock(&df->lock); + vfree(area); + return -EBUSY; + } + df->area = area; + df->size = size; + } + break; + + case KCOV_DF_ENABLE: + /* + * One writer per buffer: refuse if this object already has a + * local session, if this task already has one (on any fd), or + * if the buffer is (or may still be) a remote merge target -- a + * published handle, or workers still in flight after + * KCOV_DF_REMOTE_DISABLE (any ref beyond the fd's own). The + * local reservation is a plain read-modify-write of area[0] + * that must never race kcov_df_merge()'s atomic one. + */ + if (!df->area || df->t || df->remote_handle || + refcount_read(&df->refcount) != 1 || current->kcov_df) { + res = -EBUSY; + break; + } + kcov_df_fault_in_area(df->area, df->size); + kcov_df_get(df); /* put in KCOV_DF_DISABLE, close() or task exit */ + df->t = current; + current->kcov_df = df; + current->kcov_df_area = df->area; + current->kcov_df_size = df->size; + current->kcov_df_seq = 0; + current->kcov_df_remote_depth = 0; + /* Publish the session state before the enable flag. */ + barrier(); + current->kcov_df_enabled = true; + kcov_df_cmp_key_hold(df, KCOV_DF_CMP_LOCAL); + break; + + case KCOV_DF_DISABLE: + if (df->t != current) { + res = -EINVAL; + break; + } + kcov_df_disable_local(df); + put_session = true; + break; + + case KCOV_DF_REMOTE_ENABLE: { + struct kcov_df_remote *remote; + + if (!df->area || + !kcov_check_handle(handle, true, true, false)) { + res = -EINVAL; + break; + } + /* + * One handle per fd (a second one would leak the first entry + * and leave it pointing at a freed object after close()), and + * never while a local session writes the buffer directly. + */ + if (df->t || df->remote_handle) { + res = -EBUSY; + break; + } + remote = kzalloc_obj(struct kcov_df_remote, GFP_KERNEL); + if (!remote) { + res = -ENOMEM; + break; + } + remote->handle = handle; + remote->df = df; + mutex_lock(&kcov_df_remote_lock); + if (kcov_df_remote_find(handle)) { + mutex_unlock(&kcov_df_remote_lock); + kfree(remote); + res = -EEXIST; + break; + } + hash_add(kcov_df_remote_map, &remote->hnode, handle); + df->remote_handle = handle; + mutex_unlock(&kcov_df_remote_lock); + kcov_df_cmp_key_hold(df, KCOV_DF_CMP_REMOTE); + break; + } + + case KCOV_DF_REMOTE_DISABLE: + kcov_df_remote_unpublish(df); + kcov_df_cmp_key_release(df, KCOV_DF_CMP_REMOTE); + break; + + default: + res = -ENOTTY; + } + mutex_unlock(&df->lock); + + if (put_session) + kcov_df_put(df); + return res; +} + +/* Remote dataflow implementation */ + +/* + * Open a remote dataflow section on this task for @handle. Must be called from + * sleepable task context (it takes a mutex and may vmalloc() the scratch); in + * softirq/hardirq context it is a no-op, as is the matching stop, so the pair + * stays balanced for a call site that brackets a softirq-reachable region. + */ +void kcov_df_remote_start(u64 handle) +{ + struct kcov_df_remote *remote; + struct kcov_dataflow *df; + void *scratch; + + /* Dataflow remote coverage is collected in task (kworker) context only. */ + if (!in_task()) + return; + /* + * A task should only run one session at a time (KCOV's rule). If a + * buggy caller nests inside a remote section, don't re-init and don't + * take a second ref -- just count the depth so the matching inner + * stop() leaves the outer session intact (see kcov_df_remote_stop()). + * Coverage from the nested region is attributed to the outer handle, + * which is safe (no corruption, no early free) even though it is + * imprecise. Inside a local (KCOV_DF_ENABLE) session the depth stays + * 0, so the inner stop() is a no-op and the local session's wiring is + * left untouched; its records simply go to its own buffer. + * + * This check comes first so that every early return below only ever + * happens with no session live -- then the matching stop() has nothing + * to tear down and can never truncate an outer section. + */ + if (current->kcov_df) { + WARN_ON_ONCE(1); + if (current->kcov_df_remote_depth > 0 && + current->kcov_df_remote_depth < INT_MAX) + current->kcov_df_remote_depth++; + return; + } + if (!handle) + return; + + /* mutex_lock()'s might_sleep() reports an atomic (non-sleepable) caller. */ + mutex_lock(&kcov_df_remote_lock); + remote = kcov_df_remote_find(handle); + if (!remote || !remote->df || !remote->df->area) { + mutex_unlock(&kcov_df_remote_lock); + return; + } + df = remote->df; + kcov_df_get(df); /* keep @df (and ->area) alive until _stop() */ + scratch = kcov_df_scratch_get(); /* reuse a pooled scratch if any */ + mutex_unlock(&kcov_df_remote_lock); + + if (!scratch) { + scratch = vmalloc(KCOV_DF_REMOTE_WORDS * sizeof(u64)); + if (!scratch) { + kcov_df_put(df); + return; + } + } + ((u64 *)scratch)[0] = 0; /* reset the scratch write cursor */ + kcov_df_fault_in_area(scratch, KCOV_DF_REMOTE_WORDS); + + /* + * Point this task at its OWN private scratch, NOT df->area. It collects + * here while it runs; kcov_df_remote_stop() merges it into the shared + * buffer. So multiple kworkers on one handle never write the same buffer. + */ + current->kcov_df_area = scratch; + current->kcov_df_size = KCOV_DF_REMOTE_WORDS; + current->kcov_df_seq = 0; + current->kcov_df = df; /* pocket it for _stop(); no hash relookup */ + current->kcov_df_remote_depth = 1; + /* + * Publish all session state BEFORE the enable flag (mirrors kcov_start()). + * kcov_df_write() gates on kcov_df_enabled and then reads kcov_df_area, so + * the buffer/handle must be visible first; the barrier keeps the compiler + * from hoisting the enable above them. + */ + barrier(); + current->kcov_df_enabled = true; +} +EXPORT_SYMBOL_GPL(kcov_df_remote_start); + +void kcov_df_remote_stop(void) +{ + struct kcov_dataflow *df = current->kcov_df; + void *scratch; + + /* + * Same context rule as kcov_df_remote_start(): a stop() in softirq + * context pairs with a start() that was a no-op, and must not touch + * the interrupted task's live session. + */ + if (!in_task()) + return; + /* No remote session (a local session ignores a stray stop). */ + if (!df || current->kcov_df_remote_depth == 0) + return; + + /* + * Unwind a nested start() (buggy caller): only the OUTERMOST stop tears + * the session down. Inner stops just decrement the depth and return, so + * the buffer/ref survive until the worker is really done with them. + */ + if (--current->kcov_df_remote_depth > 0) + return; + + scratch = current->kcov_df_area; + + /* + * Stop writing FIRST: clear the per-task pointers so this task can no + * longer enter kcov_df_write() / touch the scratch. Then it is safe to + * merge and recycle the scratch and drop the ref. + */ + current->kcov_df_enabled = false; + current->kcov_df_area = NULL; + current->kcov_df_size = 0; + current->kcov_df = NULL; + + if (scratch) { + /* + * Publish this worker's records into the shared buffer, + * then return the scratch to the pool for the next worker. + */ + kcov_df_merge(df, scratch); + mutex_lock(&kcov_df_remote_lock); + kcov_df_scratch_put(scratch); + mutex_unlock(&kcov_df_remote_lock); + } + + /* + * Drop the ref taken in kcov_df_remote_start(). If this is the last one, + * kcov_df_put() frees ->area right here -- safe, because no task writes + * ->area directly anymore (workers write scratch; the merge above is + * done). Dropping via the pocketed @df (not a hash lookup) means an + * already-unpublished entry can never strand the count. + */ + kcov_df_put(df); +} +EXPORT_SYMBOL_GPL(kcov_df_remote_stop); + +static const struct file_operations kcov_df_fops = { + .open = kcov_df_open, + .unlocked_ioctl = kcov_df_ioctl, + .compat_ioctl = kcov_df_ioctl, + .mmap = kcov_df_mmap, + .release = kcov_df_close, +}; + +/* + * Reclaim idle per-worker scratch under memory pressure. The pool otherwise only + * ever grows to the peak number of concurrent remote kworkers (each area is 8 MiB) + * and is never returned to the allocator; a shrinker lets the VM take the idle + * (parked) areas back when it needs the memory. Only pooled areas are freeable; + * in-use scratch is not on the list. mutex_trylock keeps the shrinker best-effort + * and free of any lock-ordering risk. + */ +static unsigned long +kcov_df_scratch_shrink_count(struct shrinker *sh, struct shrink_control *sc) +{ + unsigned long nr; + + if (!mutex_trylock(&kcov_df_remote_lock)) + return 0; + nr = kcov_df_scratch_pool_nr; + mutex_unlock(&kcov_df_remote_lock); + return nr ? nr : SHRINK_EMPTY; +} + +static unsigned long +kcov_df_scratch_shrink_scan(struct shrinker *sh, struct shrink_control *sc) +{ + struct kcov_df_scratch *s, *tmp; + LIST_HEAD(victims); + unsigned long freed = 0; + + if (!mutex_trylock(&kcov_df_remote_lock)) + return SHRINK_STOP; + /* + * Detach victims under the lock; free them (each 8 MiB) after unlocking + * so the vfree() latency stays off concurrent remote_start()/stop(). + */ + while (freed < sc->nr_to_scan && !list_empty(&kcov_df_scratch_pool)) { + s = list_first_entry(&kcov_df_scratch_pool, + struct kcov_df_scratch, list); + list_move(&s->list, &victims); + kcov_df_scratch_pool_nr--; + freed++; + } + mutex_unlock(&kcov_df_remote_lock); + + list_for_each_entry_safe(s, tmp, &victims, list) + vfree(s); + return freed; +} + +static int __init kcov_dataflow_init(void) +{ + struct shrinker *shrinker; + + debugfs_create_file_unsafe("kcov_dataflow", 0600, NULL, NULL, + &kcov_df_fops); + + shrinker = shrinker_alloc(0, "kcov-df-scratch"); + if (shrinker) { + shrinker->count_objects = kcov_df_scratch_shrink_count; + shrinker->scan_objects = kcov_df_scratch_shrink_scan; + shrinker->seeks = DEFAULT_SEEKS; + shrinker_register(shrinker); + } else { + pr_warn("scratch shrinker unavailable, idle remote scratch areas will not be reclaimed\n"); + } + return 0; +} +device_initcall(kcov_dataflow_init); diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 6871352681480..1818d3d0147c7 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2231,6 +2231,58 @@ config KCOV_SELFTEST On test failure, causes the kernel to panic. Recommended to be enabled, ensuring critical functionality works as intended. +config KCOV_DATAFLOW_ARGS + bool "Enable KCOV dataflow: function argument capture" + depends on KCOV + depends on CC_IS_CLANG + depends on DEBUG_INFO + depends on $(cc-option,-fsanitize-coverage=trace-args) + depends on !RUST || $(rustc-option,-Cllvm-args=-sanitizer-coverage-trace-args) + help + Captures function arguments at entry via /sys/kernel/debug/kcov_dataflow. + Struct pointer arguments are auto-expanded using compiler DebugInfo + metadata, recording individual field values at runtime. + Enable per-module with: KCOV_DATAFLOW_file.o := y in the Makefile. + Requires clang with -fsanitize-coverage=trace-args support (and, + with CONFIG_RUST, a rustc whose LLVM has the matching pass), plus + debug info: select any CONFIG_DEBUG_INFO_DWARF* option under + "Compile-time checks and compiler options" to satisfy DEBUG_INFO. + +config KCOV_DATAFLOW_RET + bool "Enable KCOV dataflow: return value capture" + depends on KCOV + depends on CC_IS_CLANG + depends on DEBUG_INFO + depends on $(cc-option,-fsanitize-coverage=trace-ret) + depends on !RUST || $(rustc-option,-Cllvm-args=-sanitizer-coverage-trace-ret) + help + Captures function return values via /sys/kernel/debug/kcov_dataflow. + Struct pointer returns are auto-expanded using compiler DebugInfo + metadata, recording individual field values at runtime. + Enable per-module with: KCOV_DATAFLOW_file.o := y in the Makefile. + Requires clang with -fsanitize-coverage=trace-ret support (and, + with CONFIG_RUST, a rustc whose LLVM has the matching pass), plus + debug info: select any CONFIG_DEBUG_INFO_DWARF* option under + "Compile-time checks and compiler options" to satisfy DEBUG_INFO. + +config KCOV_DATAFLOW_NO_INLINE + bool "Disable inlining for dataflow-instrumented files" + depends on KCOV_DATAFLOW_ARGS || KCOV_DATAFLOW_RET + help + Adds -fno-inline to files instrumented with KCOV_DATAFLOW. + This ensures every function boundary is preserved, giving + complete argument visibility. Disable for lower overhead at the + cost of losing argument records for inlined functions. + +config KCOV_DATAFLOW_INSTRUMENT_ALL + bool "Instrument all kernel code with dataflow coverage" + depends on KCOV_DATAFLOW_ARGS || KCOV_DATAFLOW_RET + help + Instrument all kernel objects with trace-args/trace-ret + automatically. Individual files or directories can opt out + with KCOV_DATAFLOW_file.o := n or KCOV_DATAFLOW := n. + Warning: significantly increases code size and boot time. + menuconfig RUNTIME_TESTING_MENU bool "Runtime Testing" default y diff --git a/scripts/Makefile.kcov b/scripts/Makefile.kcov index 78305a84ba9d2..5fd2aa69d8fd5 100644 --- a/scripts/Makefile.kcov +++ b/scripts/Makefile.kcov @@ -9,3 +9,20 @@ kcov-rflags-$(CONFIG_KCOV_ENABLE_COMPARISONS) += -Cllvm-args=-sanitizer-coverage export CFLAGS_KCOV := $(kcov-flags-y) export RUSTFLAGS_KCOV := $(kcov-rflags-y) + +# KCOV dataflow: trace function args and return values. Each kind is gated by +# its own Kconfig symbol, matching the #ifdef around the callback it emits calls +# to in kernel/kcov_dataflow.c (an instrumented object must never reference a +# callback that is not compiled in). Both variables are empty on a KCOV-only +# kernel, so a stray per-file KCOV_DATAFLOW_file.o := y is harmless there. +kcov-dataflow-flags-$(CONFIG_KCOV_DATAFLOW_ARGS) += -fsanitize-coverage=trace-args +kcov-dataflow-flags-$(CONFIG_KCOV_DATAFLOW_RET) += -fsanitize-coverage=trace-ret +kcov-dataflow-flags-$(CONFIG_KCOV_DATAFLOW_NO_INLINE) += -fno-inline + +# Rust: only add the trace-args/ret llvm-args (sancov-module pass and level=3 +# are already provided by RUSTFLAGS_KCOV since KCOV_DATAFLOW depends on KCOV). +kcov-dataflow-rflags-$(CONFIG_KCOV_DATAFLOW_ARGS) += -Cllvm-args=-sanitizer-coverage-trace-args +kcov-dataflow-rflags-$(CONFIG_KCOV_DATAFLOW_RET) += -Cllvm-args=-sanitizer-coverage-trace-ret + +export CFLAGS_KCOV_DATAFLOW := $(kcov-dataflow-flags-y) +export RUSTFLAGS_KCOV_DATAFLOW := $(kcov-dataflow-rflags-y) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 0a4fdd8bd975d..b32fa67ce99af 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -88,6 +88,20 @@ _c_flags += $(if $(patsubst n%,, \ _rust_flags += $(if $(patsubst n%,, \ $(KCOV_INSTRUMENT_$(target-stem).o)$(KCOV_INSTRUMENT)$(if $(is-kernel-object),$(CONFIG_KCOV_INSTRUMENT_ALL))), \ $(RUSTFLAGS_KCOV)) +# KCOV dataflow. The outer test only honours an explicit KCOV opt-out +# (KCOV_INSTRUMENT_file.o := n / KCOV_INSTRUMENT := n, the noinstr exclusions): +# it does not require a KCOV opt-in, so per-file KCOV_DATAFLOW_file.o := y works +# for modules and out-of-tree objects too. The inner test is the dataflow opt-in: +# per-file/per-directory, or every kernel object under +# CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL. +_c_flags += $(if $(patsubst n%,, \ + $(KCOV_INSTRUMENT_$(target-stem).o)$(KCOV_INSTRUMENT)y),$(if $(patsubst n%,, \ + $(KCOV_DATAFLOW_$(target-stem).o)$(KCOV_DATAFLOW)$(if $(is-kernel-object),$(CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL))), \ + $(CFLAGS_KCOV_DATAFLOW))) +_rust_flags += $(if $(patsubst n%,, \ + $(KCOV_INSTRUMENT_$(target-stem).o)$(KCOV_INSTRUMENT)y),$(if $(patsubst n%,, \ + $(KCOV_DATAFLOW_$(target-stem).o)$(KCOV_DATAFLOW)$(if $(is-kernel-object),$(CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL))), \ + $(RUSTFLAGS_KCOV_DATAFLOW))) endif # diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 464f6c9d9ff0b..ae6fe47886395 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -1219,6 +1219,10 @@ static const char *uaccess_safe_builtin[] = { "__tsan_unaligned_write16", /* KCOV */ "write_comp_data", + /* KCOV dataflow */ + "kcov_df_trace_cmp", + "__sanitizer_cov_trace_args", + "__sanitizer_cov_trace_ret", "check_kcov_mode", "__sanitizer_cov_trace_pc", "__sanitizer_cov_trace_const_cmp1", -- 2.47.3 Document the kcov_dataflow subsystem under Documentation/dev-tools/: - Prerequisites and the Kconfig options (KCOV_DATAFLOW_ARGS / _RET, NO_INLINE, INSTRUMENT_ALL) and the compiler they require. - Per-file (KCOV_DATAFLOW_.o := y) and whole-kernel instrumentation. - A worked collection example: open /sys/kernel/debug/kcov_dataflow, KCOV_DF_INIT_TRACK, mmap, KCOV_DF_ENABLE, run the workload, and walk the buffer. - The record layout: area[0] as the record-word count, the header word fields (sequence, type, value count, size, argument index), the PC with the KASLR offset removed, the traced pointer or comparison type, and the value words. - Safety properties and the ioctl interface reference. - Coexistence with KCOV, Rust module support, and the fork/child tracing pattern. Add the ioctl 'd' numbers (KCOV_DF_INIT_TRACK and 100-103) to Documentation/userspace-api/ioctl/ioctl-number.rst, link the new file from the dev-tools index, and add the MAINTAINERS entries for kernel/kcov_dataflow.c and include/uapi/linux/kcov_dataflow.h. Signed-off-by: Yunseong Kim --- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kcov-dataflow.rst | 449 +++++++++++++++++++++ Documentation/userspace-api/ioctl/ioctl-number.rst | 2 + MAINTAINERS | 2 + 4 files changed, 454 insertions(+) diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index 59cbb77b33ff4..541c58cc65ea5 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -24,6 +24,7 @@ Documentation/process/debugging/index.rst context-analysis sparse kcov + kcov-dataflow gcov kasan kmsan diff --git a/Documentation/dev-tools/kcov-dataflow.rst b/Documentation/dev-tools/kcov-dataflow.rst new file mode 100644 index 0000000000000..4c023032fea00 --- /dev/null +++ b/Documentation/dev-tools/kcov-dataflow.rst @@ -0,0 +1,449 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow: function argument and return value extraction +============================================================= + +KCOV-Dataflow captures function arguments and return values, including +automatic struct field decomposition, at instrumented kernel function +boundaries. It provides per-task, lock-free ring buffers accessible via +``mmap()``, enabling data-flow-aware fuzzing and post-mortem contract +verification. + +Unlike KCOV's ``trace-pc`` which reports *which* code executed, +KCOV-Dataflow reports *what values* were passed and returned. This is +a completely separate device from ``/sys/kernel/debug/kcov``. + +Prerequisites +------------- + +KCOV-Dataflow requires Clang/LLVM with the ``trace-args`` and +``trace-ret`` SanitizerCoverage extensions. Standard (unpatched) +compilers will not expose these Kconfig options. + +To enable KCOV-Dataflow, configure the kernel with:: + + CONFIG_KCOV=y + CONFIG_KCOV_DATAFLOW_ARGS=y + CONFIG_KCOV_DATAFLOW_RET=y + +Optional: instrument the entire kernel (significant overhead):: + + CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL=y + +Coverage data becomes accessible once debugfs is mounted:: + + mount -t debugfs none /sys/kernel/debug + +Per-module instrumentation +-------------------------- + +To instrument a specific module, add to its Makefile:: + + KCOV_DATAFLOW_my_module.o := y + +For example, to instrument the Android binder driver:: + + # drivers/android/Makefile + KCOV_DATAFLOW_binder.o := y + KCOV_DATAFLOW_binder_alloc.o := y + +To instrument an entire directory, set the variable without a filename:: + + # fs/Makefile + KCOV_DATAFLOW := y + +The build system automatically adds the required compiler flags +(``-fsanitize-coverage=trace-args,trace-ret``). Debug info is provided +by ``CONFIG_DEBUG_INFO`` which is a Kconfig dependency. + +Data collection +--------------- + +The following program demonstrates how to collect function argument and +return value data for a single syscall: + +.. code-block:: c + + #include + #include + #include + #include + #include + #include + #include + #include + + #include /* ioctls, record layout, helpers */ + #define BUF_SIZE (1 << 20) /* 1M words = 8MB */ + + int main(void) + { + int fd; + uint64_t *buf, n, i; + + fd = open("/sys/kernel/debug/kcov_dataflow", O_RDWR); + if (fd == -1) + perror("open"), exit(1); + + /* Allocate buffer (size in u64 words). */ + if (ioctl(fd, KCOV_DF_INIT_TRACK, BUF_SIZE)) + perror("ioctl(INIT)"), exit(1); + + /* Map the buffer into user space. */ + buf = (uint64_t *)mmap(NULL, BUF_SIZE * sizeof(uint64_t), + PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + if (buf == MAP_FAILED) + perror("mmap"), exit(1); + + /* Enable data-flow collection for this task. */ + if (ioctl(fd, KCOV_DF_ENABLE, 0)) + perror("ioctl(ENABLE)"), exit(1); + + /* Reset counter. */ + __atomic_store_n(&buf[0], 0, __ATOMIC_RELAXED); + + /* === Trigger syscall(s) here === */ + read(-1, NULL, 0); + + /* Read how many words were written. */ + n = __atomic_load_n(&buf[0], __ATOMIC_RELAXED); + + /* Parse TLV records. */ + i = 1; + while (i + KCOV_DF_RECORD_HDR_WORDS <= 1 + n) { + uint64_t hdr = buf[i]; + uint64_t pc = buf[i + 1]; /* KASLR offset removed */ + uint64_t ptr = buf[i + 2]; /* traced pointer (ENTRY/RET) */ + uint32_t type = KCOV_DF_HDR_TYPE(hdr); + uint32_t num_vals = KCOV_DF_HDR_NVALS(hdr); + uint32_t seq = KCOV_DF_HDR_SEQ(hdr); + uint32_t arg_idx = KCOV_DF_HDR_ARGIDX(hdr); + uint32_t size = KCOV_DF_HDR_SIZE(hdr); + + if (!num_vals || (type != KCOV_DF_TYPE_ENTRY && + type != KCOV_DF_TYPE_RET && + type != KCOV_DF_TYPE_CMP)) { + i++; /* garbage (e.g. reset mid-run): resync */ + continue; + } + if (type != KCOV_DF_TYPE_CMP) + printf("[%s] seq=%u pc=0x%lx ptr=0x%lx arg_idx=%u size=%u val=0x%lx\n", + type == KCOV_DF_TYPE_ENTRY ? "ENTRY" : "RET", + seq, pc, ptr, arg_idx, size, buf[i + 3]); + i += KCOV_DF_RECORD_WORDS(num_vals); + } + + if (ioctl(fd, KCOV_DF_DISABLE, 0)) + perror("ioctl(DISABLE)"), exit(1); + + munmap(buf, BUF_SIZE * sizeof(uint64_t)); + close(fd); + return 0; + } + +Ring buffer format +------------------ + +The buffer is an array of ``u64`` words:: + + buf[0]: atomic counter -- total words written + +Each record occupies 3 + N words: + +.. list-table:: + :header-rows: 1 + + * - Offset + - Field + - Description + * - 0 + - header + - bits[63:56] = arg_idx (0 for return), bits[55:48] = size in bytes + (clamped to 255), bits[47:32] = num_vals (>= 1), + bits[31:28] = type: ``KCOV_DF_TYPE_ENTRY`` (0xE), + ``KCOV_DF_TYPE_RET`` (0xF) or ``KCOV_DF_TYPE_CMP`` (0xC), + bits[23:0] = sequence number + * - 1 + - pc + - Instrumented function address with the KASLR offset removed (same + as the PCs mainline kcov records), so it can be symbolized against + vmlinux; add the runtime offset back for ``/proc/kallsyms`` + * - 2 + - ptr / cmp_type + - ENTRY/RET: the full 64-bit traced pointer (may be NULL/ERR_PTR, in + which case the values are ``0xBADADD85``). CMP: the comparison + type, ``KCOV_CMP_SIZE()``/``KCOV_CMP_CONST`` bits from linux/kcov.h + * - 3..3+num_vals + - values + - Struct field values, a single scalar, or the two CMP operands + +``area[0]`` never exceeds the buffer size minus one and every counted word +has been written, so a consumer that walks ``area[0]`` words never leaves +its mapping. All of the above is defined in ``include/uapi/linux/kcov_dataflow.h`` +(``KCOV_DF_HDR_*()``, ``KCOV_DF_RECORD_WORDS()``). + +Magic values: + +- ``0xBADADD85``: field read failed (pointer was invalid/freed/poisoned) + +Safety +------ + +- Callbacks are ``notrace``, ``__no_sanitize_coverage``, ``noinline`` + to prevent recursion. +- All pointer reads use ``copy_from_kernel_nofault()`` -- survives + freed, poisoned, or unmapped memory. +- An ``in_task()`` guard rejects calls from hardirq/softirq/NMI context, + preventing reentrant buffer corruption. +- No ``printk`` or allocation in the data path. +- When not enabled for a task, overhead is a single boolean check. + +Ioctl interface +--------------- + +.. list-table:: + :header-rows: 1 + + * - Command + - Value + - Description + * - KCOV_DF_INIT_TRACK + - ``_IOR('d', 1, unsigned long)`` + - Allocate buffer (size in u64 words) + * - KCOV_DF_ENABLE + - ``_IO('d', 100)`` + - Start collection for current task + * - KCOV_DF_DISABLE + - ``_IO('d', 101)`` + - Stop collection + * - KCOV_DF_REMOTE_ENABLE + - ``_IOW('d', 102, __u64)`` -- argument is a pointer to the handle + - Publish buffer for kworker/kthread remote capture + * - KCOV_DF_REMOTE_DISABLE + - ``_IO('d', 103)`` + - Unpublish buffer from remote capture + +Compatibility +------------- + +KCOV-Dataflow is completely independent from legacy KCOV: + +- Separate device: ``/sys/kernel/debug/kcov_dataflow`` +- Separate ioctl namespace (``'d'`` vs ``'c'``) +- Separate per-task buffer +- Both can be used simultaneously without interference +- syzkaller and other KCOV users are unaffected + +Rust module support +------------------- + +Rust kernel modules are instrumented natively through the build system. +The ``KCOV_DATAFLOW_.o := y`` mechanism works identically for +Rust and C modules. The build system passes +``-Cllvm-args=-sanitizer-coverage-trace-args`` and +``-Cllvm-args=-sanitizer-coverage-trace-ret`` to rustc via +``RUSTFLAGS_KCOV_DATAFLOW``. + +Example Makefile for a Rust module:: + + obj-m := my_rust_module.o + KCOV_DATAFLOW_my_rust_module.o := y + +Requires a rustc built against LLVM with trace-args/trace-ret support +and ``CONFIG_RUST=y`` in the kernel config. + +Selftests +--------- + +Automated tests and visualization tools are in +``tools/testing/selftests/kcov_dataflow/``:: + + # Automated ioctl interface test (TAP output): + make -C tools/testing/selftests/kcov_dataflow + vng --user root --exec \ + tools/testing/selftests/kcov_dataflow/user_ioctl/user_ioctl + + # Load a test module and view captured records: + make LLVM=1 CC=clang M=tools/testing/selftests/kcov_dataflow/eight_struct_args_c modules + vng --user root --exec \ + "python3 tools/testing/selftests/kcov_dataflow/trigger-view.py \ + eight_struct_args_c --ko \ + tools/testing/selftests/kcov_dataflow/eight_struct_args_c/eight_struct_args_c.ko" + + # Binderfs ioctl capture test (requires CONFIG_ANDROID_BINDER_IPC): + make -C tools/testing/selftests/kcov_dataflow/binderfs + vng --user root --exec \ + tools/testing/selftests/kcov_dataflow/binderfs/binderfs_test + +See ``tools/testing/selftests/kcov_dataflow/README.rst`` for details. + +Tracing child processes +----------------------- + +KCOV-Dataflow is per-task: after ``fork()``, the child does not inherit +the enabled state. To trace child processes, re-enable on the inherited +file descriptor in the child before ``exec()``. The ``mmap``'d buffer is +shared (``MAP_SHARED``), so both parent and child write to the same ring +buffer atomically. + +.. code-block:: c + + #include + #include + #include + #include + #include + #include + #include + #include + + #include /* ioctls, record layout, helpers */ + #define BUF_SIZE (1 << 20) + + int main(int argc, char **argv) + { + int fd = open("/sys/kernel/debug/kcov_dataflow", O_RDWR); + ioctl(fd, KCOV_DF_INIT_TRACK, BUF_SIZE); + uint64_t *buf = mmap(NULL, BUF_SIZE * 8, + PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + + /* Enable for parent task. */ + ioctl(fd, KCOV_DF_ENABLE, 0); + __atomic_store_n(&buf[0], 0, __ATOMIC_RELAXED); + + pid_t pid = fork(); + if (pid == 0) { + /* + * Child: re-enable on inherited fd. + * The shared mmap buffer receives records from both tasks. + */ + ioctl(fd, KCOV_DF_ENABLE, 0); + execvp(argv[1], &argv[1]); + _exit(1); + } + + waitpid(pid, NULL, 0); + ioctl(fd, KCOV_DF_DISABLE, 0); + + uint64_t n = __atomic_load_n(&buf[0], __ATOMIC_RELAXED); + printf("Captured %lu words from parent + child\n", n); + + munmap(buf, BUF_SIZE * 8); + close(fd); + return 0; + } + +Note: the child's ``ioctl(fd, KCOV_DF_ENABLE)`` will fail if the parent +has not yet called ``KCOV_DF_DISABLE``, because only one task can be +associated with a descriptor at a time. For true multi-process tracing, +open a separate ``kcov_dataflow`` fd per child, or disable in the parent +before the child enables (as shown above -- the parent is blocked in +``waitpid`` so it generates no records during that time anyway). + +Remote tracing (kworker/kthread) +-------------------------------- + +To capture data from kernel threads (kworkers, kthreads) that are not +direct descendants of user space, use the remote API: + +1. User space allocates and publishes a buffer with ``KCOV_DF_REMOTE_ENABLE`` +2. The kernel module calls ``kcov_df_remote_start()`` at work entry +3. The kernel module calls ``kcov_df_remote_stop()`` at work exit +4. User space reads the buffer and unpublishes with ``KCOV_DF_REMOTE_DISABLE`` + +User space setup: + +.. code-block:: c + + #include + #include + #include + #include + #include + #include + + #include /* kcov_remote_handle() */ + #include + #define BUF_SIZE (1 << 20) + + int main(void) + { + int fd = open("/sys/kernel/debug/kcov_dataflow", O_RDWR); + ioctl(fd, KCOV_DF_INIT_TRACK, BUF_SIZE); + uint64_t *buf = mmap(NULL, BUF_SIZE * 8, + PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + __atomic_store_n(&buf[0], 0, __ATOMIC_RELAXED); + + /* + * Publish the buffer under a remote handle. The handle must be a + * valid kcov_remote_handle() encoding (KCOV_SUBSYSTEM_COMMON with a + * nonzero instance, or KCOV_SUBSYSTEM_USB) and is the value the + * kernel side passes to kcov_df_remote_start(); one handle per fd, + * and not while KCOV_DF_ENABLE is active on the same fd. + */ + __u64 handle = kcov_remote_handle(KCOV_SUBSYSTEM_COMMON, 1); + if (ioctl(fd, KCOV_DF_REMOTE_ENABLE, &handle)) + perror("ioctl(REMOTE_ENABLE)"), exit(1); + + /* Trigger kworker activity (e.g., write to a file, ioctl). */ + /* ... */ + sleep(1); + + /* Unpublish and read results. */ + ioctl(fd, KCOV_DF_REMOTE_DISABLE, 0); + + uint64_t n = __atomic_load_n(&buf[0], __ATOMIC_RELAXED); + printf("Captured %lu words from kworker\n", n); + + munmap(buf, BUF_SIZE * 8); + close(fd); + return 0; + } + +Kernel module side (called from kworker context): + +.. code-block:: c + + #include + + void my_work_fn(struct work_struct *work) + { + kcov_df_remote_start(); + /* ... instrumented code runs here ... */ + kcov_df_remote_stop(); + } + +Only one buffer can be published at a time. ``kcov_df_remote_start()`` +is a no-op if no buffer is published or if the current task already has +dataflow enabled. + +Limitations +----------- + +ABI argument mapping + The LLVM pass maps IR-level arguments to source-level parameters using + ``DILocalVariable`` debug records (``-g`` required). This correctly + handles hidden ``sret`` pointers, struct decomposition into multiple + registers, and C++ ``this`` pointers. + + When debug info is absent or stripped, the pass falls back to positional + indexing which may misattribute arguments in functions with ABI-inserted + hidden parameters. The kernel is always built with ``-g``, so this + limitation does not apply to kernel use. + +Struct-by-value reassembly + When a small struct is passed by value and the ABI decomposes it into + multiple scalar registers (e.g., ``struct { int x; int y; }`` as two + ``i32`` values on x86_64), the pass reassembles the fragments into a + stack slot. The struct field offsets are preserved, but if a field was + entirely optimized away (no debug record), that slot contains zero. + + In kernel code, structs are always passed by pointer, so this case + does not arise. + +Optimized builds + At ``-O2`` and above, LLVM may eliminate ``#dbg_value`` records for + arguments that are dead or fully inlined. Such arguments will emit a + trace with a null pointer (producing ``0xBADADD85`` in all field + positions), indicating the argument existed but its value was + unavailable at runtime. diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index 2fc53093752d1..7864b2e7fb476 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -240,6 +240,8 @@ Code Seq# Include File Comments 'd' 00-FF linux/char/drm/drm.h conflict! 'd' 02-40 pcmcia/ds.h conflict! 'd' F0-FF linux/digi1.h +'d' 01 uapi/linux/kcov_dataflow.h conflict! +'d' 64-67 uapi/linux/kcov_dataflow.h conflict! 'e' all linux/digi1.h conflict! 'f' 00-1F linux/ext2_fs.h conflict! 'f' 00-1F linux/ext3_fs.h conflict! diff --git a/MAINTAINERS b/MAINTAINERS index b91655b34f0ef..d79e04b108c1c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14092,7 +14092,9 @@ B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%2 F: Documentation/dev-tools/kcov.rst F: include/linux/kcov.h F: include/uapi/linux/kcov.h +F: include/uapi/linux/kcov_dataflow.h F: kernel/kcov.c +F: kernel/kcov_dataflow.c F: scripts/Makefile.kcov KCSAN -- 2.47.3 Add a kselftest tests for the kcov_dataflow ioctl interface in user_ioctl/. Nine cases cover the fd lifecycle and error paths without a custom compiler, so it runs on any kernel that has the device: - KCOV_DF_INIT_TRACK tests 3 cases: 1. accepted 2. too-small size -> -EINVAL 3. second init -> -EBUSY - mmap before init -> fails; enable works with or without a prior mmap. - KCOV_DF_DISABLE without an enabled session -> -EINVAL. - A second fd trying to enable while this task already has a session -> -EBUSY. - After enabling and running a syscall, any records present parse: the walk starts with a known type (ENTRY/RET, or CMP when CONFIG_KCOV_ENABLE_COMPARISONS interleaves them), each record has at least one value word, and the walk ends exactly at area[0] inside the buffer. The test SKIPs cleanly when /sys/kernel/debug/kcov_dataflow is absent (CONFIG_KCOV_DATAFLOW_ARGS not built). Assisted-by: Claude:claude-opus-4-6 [kiro-chat] Signed-off-by: Yunseong Kim --- .../selftests/kcov_dataflow/user_ioctl/Makefile | 5 + .../selftests/kcov_dataflow/user_ioctl/README.rst | 11 ++ .../kcov_dataflow/user_ioctl/user_ioctl.c | 168 +++++++++++++++++++++ 3 files changed, 184 insertions(+) diff --git a/tools/testing/selftests/kcov_dataflow/user_ioctl/Makefile b/tools/testing/selftests/kcov_dataflow/user_ioctl/Makefile new file mode 100644 index 000000000000..1cb3d9b41c07 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/user_ioctl/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 +# Standalone build of the ioctl test: make -C tools/testing/selftests/kcov_dataflow/user_ioctl +TEST_GEN_PROGS := user_ioctl +CFLAGS += -Wall -O2 $(KHDR_INCLUDES) +include ../../lib.mk diff --git a/tools/testing/selftests/kcov_dataflow/user_ioctl/README.rst b/tools/testing/selftests/kcov_dataflow/user_ioctl/README.rst new file mode 100644 index 000000000000..55072de189d3 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/user_ioctl/README.rst @@ -0,0 +1,11 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests: user_ioctl +=================================== + +Automated ioctl interface test (kselftest harness, 9 TAP cases): INIT_TRACK +argument checking, double init, mmap before init, ENABLE/DISABLE pairing, +a second fd failing with -EBUSY, and record validity after a syscall:: + + make -C tools/testing/selftests TARGETS=kcov_dataflow + tools/testing/selftests/kcov_dataflow/user_ioctl/user_ioctl diff --git a/tools/testing/selftests/kcov_dataflow/user_ioctl/user_ioctl.c b/tools/testing/selftests/kcov_dataflow/user_ioctl/user_ioctl.c new file mode 100644 index 000000000000..d7b04c368ced --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/user_ioctl/user_ioctl.c @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * kcov_dataflow_test.c - Selftest for /sys/kernel/debug/kcov_dataflow + * + * Verifies the ioctl interface: open, INIT_TRACK, mmap, ENABLE, DISABLE. + * With INSTRUMENT_ALL, also verifies that records are produced for + * syscalls executed while recording is active. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../kselftest_harness.h" + + +#define BUF_SIZE 65536 + +#define DF_TYPE_ENTRY KCOV_DF_TYPE_ENTRY +#define DF_TYPE_RET KCOV_DF_TYPE_RET + +FIXTURE(kcov_dataflow) { + int fd; + uint64_t *buf; +}; + +FIXTURE_SETUP(kcov_dataflow) +{ + self->fd = open("/sys/kernel/debug/kcov_dataflow", O_RDWR); + if (self->fd < 0) + SKIP(return, "kcov_dataflow not available (need CONFIG_KCOV_DATAFLOW_ARGS)"); + self->buf = MAP_FAILED; +} + +FIXTURE_TEARDOWN(kcov_dataflow) +{ + if (self->buf != MAP_FAILED) + munmap(self->buf, BUF_SIZE * sizeof(uint64_t)); + if (self->fd >= 0) + close(self->fd); +} + +TEST_F(kcov_dataflow, init_track) +{ + int ret = ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE); + + ASSERT_EQ(0, ret); +} + +TEST_F(kcov_dataflow, init_track_too_small) +{ + int ret = ioctl(self->fd, KCOV_DF_INIT_TRACK, 1UL); + + ASSERT_EQ(-1, ret); + ASSERT_EQ(EINVAL, errno); +} + +TEST_F(kcov_dataflow, init_track_double) +{ + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + ASSERT_EQ(-1, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + ASSERT_EQ(EBUSY, errno); +} + +TEST_F(kcov_dataflow, mmap_before_init) +{ + self->buf = mmap(NULL, BUF_SIZE * sizeof(uint64_t), + PROT_READ | PROT_WRITE, MAP_SHARED, self->fd, 0); + ASSERT_EQ(MAP_FAILED, self->buf); +} + +TEST_F(kcov_dataflow, enable_disable) +{ + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + self->buf = mmap(NULL, BUF_SIZE * sizeof(uint64_t), + PROT_READ | PROT_WRITE, MAP_SHARED, self->fd, 0); + ASSERT_NE(MAP_FAILED, self->buf); + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_ENABLE, 0)); + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_DISABLE, 0)); +} + +TEST_F(kcov_dataflow, enable_without_mmap) +{ + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + /* enable works even without mmap (mmap is optional for setup) */ + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_ENABLE, 0)); + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_DISABLE, 0)); +} + +TEST_F(kcov_dataflow, disable_without_enable) +{ + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + ASSERT_EQ(-1, ioctl(self->fd, KCOV_DF_DISABLE, 0)); + ASSERT_EQ(EINVAL, errno); +} + +TEST_F(kcov_dataflow, double_enable) +{ + int fd2; + + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + self->buf = mmap(NULL, BUF_SIZE * sizeof(uint64_t), + PROT_READ | PROT_WRITE, MAP_SHARED, self->fd, 0); + ASSERT_NE(MAP_FAILED, self->buf); + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_ENABLE, 0)); + + /* Second fd should fail to enable (task already active) */ + fd2 = open("/sys/kernel/debug/kcov_dataflow", O_RDWR); + ASSERT_GE(fd2, 0); + ASSERT_EQ(0, ioctl(fd2, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + ASSERT_EQ(-1, ioctl(fd2, KCOV_DF_ENABLE, 0)); + ASSERT_EQ(EBUSY, errno); + close(fd2); + + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_DISABLE, 0)); +} + +TEST_F(kcov_dataflow, records_captured) +{ + uint64_t count; + + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_INIT_TRACK, (unsigned long)BUF_SIZE)); + self->buf = mmap(NULL, BUF_SIZE * sizeof(uint64_t), + PROT_READ | PROT_WRITE, MAP_SHARED, self->fd, 0); + ASSERT_NE(MAP_FAILED, self->buf); + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_ENABLE, 0)); + + /* Trigger some kernel code in this task */ + getpid(); + + ASSERT_EQ(0, ioctl(self->fd, KCOV_DF_DISABLE, 0)); + + count = self->buf[0]; + /* + * With INSTRUMENT_ALL, getpid() produces records; without it count may + * be 0. Whatever was written must parse: known types (CMP records are + * interleaved with CONFIG_KCOV_ENABLE_COMPARISONS=y), at least one value + * word each, and a walk that ends exactly at area[0] inside the buffer. + */ + ASSERT_LE(count, (uint64_t)BUF_SIZE - 1); + if (count > 0) { + uint64_t pos = 1, end = 1 + count; + unsigned int nargs = 0; + + while (pos + KCOV_DF_RECORD_HDR_WORDS <= end) { + uint64_t hdr = self->buf[pos]; + unsigned int type = KCOV_DF_HDR_TYPE(hdr); + unsigned int nvals = KCOV_DF_HDR_NVALS(hdr); + + ASSERT_TRUE(type == DF_TYPE_ENTRY || type == DF_TYPE_RET || + type == KCOV_DF_TYPE_CMP); + ASSERT_GE(nvals, 1); + if (type != KCOV_DF_TYPE_CMP) + nargs++; + pos += KCOV_DF_RECORD_WORDS(nvals); + } + ASSERT_EQ(end, pos); + ASSERT_GT(nargs, 0); + } +} + +TEST_HARNESS_MAIN -- 2.47.3 Add a test module that shows kcov_dataflow detecting a function-boundary contract violation that leaves no crash and no KASAN report. ffi_alloc_buf() has the postcondition "returns 0 implies out->buffer is valid", but its async path with an empty pool returns 0 while leaving out->buffer == NULL. The caller, ffi_check_result(), trusts the contract and dereferences the buffer. Because kcov_dataflow captures the struct fields at both boundaries, the violation is visible in the record stream: 0x0 = ffi_alloc_buf({0x0, 0x0, 0x0, 0x0}, 0x100, 0x10, 0x1) 0xfffffff2 = ffi_check_result({0x0, 0x110, 0x0, 0x0}) ^ buffer still NULL after a 0 return The module is opted into instrumentation with KCOV_DATAFLOW_rust_ffi_contract.o := y and driven through a debugfs trigger file; kselftest script will check the expanded struct at each boundary, the scalar arguments (256, 16, 1) and the two return values. Assisted-by: Claude:claude-opus-4-6 [kiro-chat] Signed-off-by: Yunseong Kim --- .../kcov_dataflow/rust_ffi_contract/Makefile | 3 + .../kcov_dataflow/rust_ffi_contract/README.rst | 13 +++ .../rust_ffi_contract/rust_ffi_contract.c | 125 +++++++++++++++++++++ 3 files changed, 141 insertions(+) diff --git a/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/Makefile b/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/Makefile new file mode 100644 index 0000000000000..d2a0261070b1c --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-m := rust_ffi_contract.o +KCOV_DATAFLOW_rust_ffi_contract.o := y diff --git a/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/README.rst b/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/README.rst new file mode 100644 index 0000000000000..291621fa799cd --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/README.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests: rust_ffi_contract +========================================== + +FFI contract violation detection: ffi_alloc_buf() returns 0 but leaves +alloc->buffer NULL, and ffi_check_result() receives that NULL. The test +checks the expanded ``struct ffi_alloc`` at both boundaries, the scalar +arguments (256, 16, 1), the 0 return and the -EFAULT from the checker. +Opted in with ``KCOV_DATAFLOW_rust_ffi_contract.o := y``:: + + ./test_modules.py -t rust_ffi_contract + ./trigger-view.py rust_ffi_contract -C 8 diff --git a/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/rust_ffi_contract.c b/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/rust_ffi_contract.c new file mode 100644 index 0000000000000..071bd25dfec11 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/rust_ffi_contract/rust_ffi_contract.c @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * rust_ffi_contract.c - Demonstrates kcov_dataflow detecting an FFI + * contract violation at a function boundary. + * + * The pattern: caller passes a struct pointer to callee. Callee's + * contract says "returns 0 implies out->buffer is valid". A bug in + * the async path returns 0 but leaves buffer=NULL. + * + * kcov_dataflow captures: + * [ENTRY] ffi_alloc_buf(alloc={.buffer=NULL, .data_size=0}, 256, 16, 1) + * [RET] ffi_alloc_buf() = 0 + * [ENTRY] ffi_check_result(alloc={.buffer=NULL, .data_size=0x110, ...}) + * ^ proves contract violated + * [RET] ffi_check_result() = -EFAULT + * + * Write to /sys/kernel/debug/kcov_dataflow_test/rust_ffi_trigger to run. + */ +#include +#include +#include + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("FFI contract violation detection via kcov_dataflow"); + +struct ffi_alloc { + void *buffer; + u64 data_size; + u32 free_async; + u32 flags; +}; + +/* Prototypes */ +int ffi_alloc_buf(struct ffi_alloc *alloc, u64 data_size, + u64 offsets_size, int is_async); +int ffi_check_result(struct ffi_alloc *alloc); + +/* + * Callee with contract: returns 0 implies alloc->buffer is valid. + * BUG: async path with free_async==0 returns 0 but buffer stays NULL. + */ +noinline int ffi_alloc_buf(struct ffi_alloc *alloc, u64 data_size, + u64 offsets_size, int is_async) +{ + /* + * data_size + offsets_size is used on every path so that the compiler + * keeps offsets_size alive (an unused parameter is dropped at -O2 and + * callers then pass poison, leaving nothing to trace). + */ + if (!is_async) { + alloc->buffer = kmalloc(data_size + offsets_size, GFP_KERNEL); + if (!alloc->buffer) + return -ENOMEM; + return 0; + } + /* BUG: returns success but buffer is NULL when pool empty */ + if (alloc->free_async == 0) { + alloc->buffer = NULL; + alloc->data_size = data_size + offsets_size; + return 0; /* contract violation */ + } + alloc->buffer = kmalloc(data_size + offsets_size, GFP_KERNEL); + alloc->free_async--; + return 0; +} +EXPORT_SYMBOL(ffi_alloc_buf); + +/* Caller that trusts the contract */ +noinline int ffi_check_result(struct ffi_alloc *alloc) +{ + if (!alloc->buffer) { + pr_err("ffi_contract: VIOLATION detected - buffer is NULL after success\n"); + return -EFAULT; + } + kfree(alloc->buffer); + return 0; +} +EXPORT_SYMBOL(ffi_check_result); + +static struct dentry *test_dir; + +static ssize_t rust_ffi_trigger_write(struct file *f, const char __user *buf, + size_t count, loff_t *ppos) +{ + struct ffi_alloc alloc = { .buffer = NULL, .data_size = 0, + .free_async = 0, .flags = 0 }; + int ret; + + /* + * Keep the initializer: the callee provably writes alloc->buffer before + * reading it, so without the barrier the compiler drops the NULL store + * and the ENTRY record would show stack garbage instead of NULL. + */ + barrier_data(&alloc); + + /* Trigger the bug: is_async=1, free_async=0 */ + ret = ffi_alloc_buf(&alloc, 256, 16, 1); + pr_info("ffi_contract: ffi_alloc_buf returned %d, buffer=%p\n", + ret, alloc.buffer); + + if (ret == 0) + ffi_check_result(&alloc); + + return count; +} + +static const struct file_operations rust_ffi_trigger_fops = { + .write = rust_ffi_trigger_write, +}; + +static int __init ffi_contract_init(void) +{ + test_dir = debugfs_create_dir("kcov_dataflow_test", NULL); + debugfs_create_file("rust_ffi_trigger", 0200, test_dir, NULL, + &rust_ffi_trigger_fops); + return 0; +} + +static void __exit ffi_contract_exit(void) +{ + debugfs_remove_recursive(test_dir); +} + +module_init(ffi_contract_init); +module_exit(ffi_contract_exit); -- 2.47.3 Add a kselftest that exercises the binder driver through binderfs with kcov_dataflow recording active, checking that argument records are captured at real driver ioctl boundaries rather than in a purpose-built module. The test mounts binderfs, creates a device with BINDER_CTL_ADD, enables recording, and issues BINDER_VERSION and BINDER_SET_MAX_THREADS. It then walks the buffer: every record must carry a known type (ENTRY/RET, or CMP when comparisons are interleaved) and at least one value word, the walk must end exactly at area[0], and at least one ENTRY/RET record must have been produced. It needs binder instrumented, i.e. KCOV_DATAFLOW := y in drivers/android/Makefile or CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL=y, and SKIPs cleanly when binderfs is unavailable. Assisted-by: Claude:claude-opus-4-6 [kiro-chat] Signed-off-by: Yunseong Kim --- .../selftests/kcov_dataflow/binderfs/Makefile | 5 + .../selftests/kcov_dataflow/binderfs/README.rst | 13 ++ .../kcov_dataflow/binderfs/binderfs_test.c | 195 +++++++++++++++++++++ 3 files changed, 213 insertions(+) diff --git a/tools/testing/selftests/kcov_dataflow/binderfs/Makefile b/tools/testing/selftests/kcov_dataflow/binderfs/Makefile new file mode 100644 index 000000000000..b35de6264992 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/binderfs/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 +# Standalone build of the binderfs test: make -C tools/testing/selftests/kcov_dataflow/binderfs +TEST_GEN_PROGS := binderfs_test +CFLAGS += -Wall -O2 $(KHDR_INCLUDES) +include ../../lib.mk diff --git a/tools/testing/selftests/kcov_dataflow/binderfs/README.rst b/tools/testing/selftests/kcov_dataflow/binderfs/README.rst new file mode 100644 index 000000000000..7fcdce1955c1 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/binderfs/README.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests: binderfs +================================= + +Exercises the binder driver via binderfs with kcov_dataflow recording +active and verifies that argument records are captured at the binder +ioctl boundaries. Needs CONFIG_ANDROID_BINDERFS=y and binder instrumented +(``KCOV_DATAFLOW := y`` in drivers/android/Makefile or +CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL=y); SKIPs without binderfs:: + + make -C tools/testing/selftests TARGETS=kcov_dataflow + tools/testing/selftests/kcov_dataflow/binderfs/binderfs_test diff --git a/tools/testing/selftests/kcov_dataflow/binderfs/binderfs_test.c b/tools/testing/selftests/kcov_dataflow/binderfs/binderfs_test.c new file mode 100644 index 000000000000..650798e09b20 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/binderfs/binderfs_test.c @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * binderfs selftest for kcov_dataflow + * + * Exercises the binder driver via binderfs with kcov_dataflow recording + * active, then verifies that function argument records were captured at + * binder ioctl boundaries. + * + * Requires: CONFIG_ANDROID_BINDER_IPC=y (or _RUST), CONFIG_ANDROID_BINDERFS=y + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +#define BUF_SIZE (1 << 20) +#define BINDERFS_PATH "/tmp/binderfs_test" +#define BINDER_DEV BINDERFS_PATH "/my_binder" + +static int setup_binderfs(void) +{ + struct binderfs_device dev = {}; + + mkdir(BINDERFS_PATH, 0755); + + if (mount("binder", BINDERFS_PATH, "binder", 0, NULL)) { + if (errno == ENODEV || errno == ENOENT) { + printf("SKIP: binderfs not available\n"); + return -1; + } + perror("mount binderfs"); + return -1; + } + + /* Create a binder device via BINDER_CTL_ADD ioctl */ + int ctl_fd; + + ctl_fd = open(BINDERFS_PATH "/binder-control", O_RDONLY); + if (ctl_fd < 0) { + perror("open binder-control"); + umount(BINDERFS_PATH); + return -1; + } + + strcpy(dev.name, "my_binder"); + if (ioctl(ctl_fd, BINDER_CTL_ADD, &dev) && errno != EEXIST) { + perror("BINDER_CTL_ADD"); + close(ctl_fd); + umount(BINDERFS_PATH); + return -1; + } + close(ctl_fd); + return 0; +} + +static void cleanup_binderfs(void) +{ + umount(BINDERFS_PATH); + rmdir(BINDERFS_PATH); +} + +int main(void) +{ + uint64_t *buf; + int df_fd, binder_fd; + uint64_t total; + int valid = 0; + + printf("TAP version 13\n"); + printf("1..3\n"); + + /* Setup binderfs */ + if (setup_binderfs()) { + printf("ok 1 # SKIP binderfs not available\n"); + printf("ok 2 # SKIP\n"); + printf("ok 3 # SKIP\n"); + return 0; + } + + /* Open kcov_dataflow */ + df_fd = open("/sys/kernel/debug/kcov_dataflow", O_RDWR); + if (df_fd < 0) { + printf("not ok 1 cannot open kcov_dataflow\n"); + cleanup_binderfs(); + return 1; + } + + if (ioctl(df_fd, KCOV_DF_INIT_TRACK, BUF_SIZE)) { + printf("not ok 1 INIT_TRACK failed\n"); + close(df_fd); + cleanup_binderfs(); + return 1; + } + + buf = mmap(NULL, BUF_SIZE * sizeof(uint64_t), + PROT_READ | PROT_WRITE, MAP_SHARED, df_fd, 0); + if (buf == MAP_FAILED) { + printf("not ok 1 mmap failed\n"); + close(df_fd); + cleanup_binderfs(); + return 1; + } + + printf("ok 1 kcov_dataflow.binderfs_setup\n"); + + /* Open binder device */ + binder_fd = open(BINDER_DEV, O_RDWR | O_CLOEXEC); + if (binder_fd < 0) { + printf("not ok 2 cannot open %s: %s\n", BINDER_DEV, + strerror(errno)); + munmap(buf, BUF_SIZE * sizeof(uint64_t)); + close(df_fd); + cleanup_binderfs(); + return 1; + } + + /* Enable recording and exercise binder ioctls */ + ioctl(df_fd, KCOV_DF_ENABLE, 0); + __atomic_store_n(&buf[0], 0, __ATOMIC_RELAXED); + + /* BINDER_VERSION - simple ioctl that exercises the binder path */ + struct binder_version ver = {}; + + ioctl(binder_fd, BINDER_VERSION, &ver); + + /* BINDER_SET_MAX_THREADS */ + uint32_t max_threads = 4; + + ioctl(binder_fd, BINDER_SET_MAX_THREADS, &max_threads); + + ioctl(df_fd, KCOV_DF_DISABLE, 0); + + total = __atomic_load_n(&buf[0], __ATOMIC_RELAXED); + close(binder_fd); + + if (total > 0) + printf("ok 2 kcov_dataflow.binderfs_captured # %lu words\n", + (unsigned long)total); + else + printf("not ok 2 kcov_dataflow.binderfs_captured # 0 words\n"); + + /* + * Walk the records: every header must carry a known type and at least + * one value word, the walk must end exactly at area[0], and at least one + * ENTRY/RET record must come from the binder ioctls (CMP records are + * interleaved with CONFIG_KCOV_ENABLE_COMPARISONS=y). + */ + if (total <= BUF_SIZE - 1) { + uint64_t pos = 1, end = 1 + total; + unsigned long nargs = 0; + + while (pos + KCOV_DF_RECORD_HDR_WORDS <= end) { + uint64_t hdr = buf[pos]; + uint32_t type = KCOV_DF_HDR_TYPE(hdr); + uint32_t nvals = KCOV_DF_HDR_NVALS(hdr); + + if (nvals < 1 || (type != KCOV_DF_TYPE_ENTRY && + type != KCOV_DF_TYPE_RET && + type != KCOV_DF_TYPE_CMP)) + break; + if (type != KCOV_DF_TYPE_CMP) + nargs++; + pos += KCOV_DF_RECORD_WORDS(nvals); + } + if (pos == end && nargs > 0) + valid = 1; + else + printf("# walk stopped at word %lu of %lu, %lu ENTRY/RET records\n", + (unsigned long)pos, (unsigned long)end, nargs); + } + + if (valid) + printf("ok 3 kcov_dataflow.binderfs_valid_records\n"); + else + printf("not ok 3 kcov_dataflow.binderfs_valid_records\n"); + + printf("# Totals: pass:%d fail:%d skip:0\n", + valid ? 3 : 2, valid ? 0 : 1); + + munmap(buf, BUF_SIZE * sizeof(uint64_t)); + close(df_fd); + cleanup_binderfs(); + return valid ? 0 : 1; +} -- 2.47.3 Add C and Rust modules that verify struct-pointer argument expansion, the part of the collector that turns a pointer into its individual field values from the compiler's DWARF offsets. Three families exercise the expansion: - Flat sf_1()..sf_8(): sf_N takes N struct pointers (s1*..sN*) whose fields are 0x11, 0x22, ...; this covers plain field expansion and multiple struct-pointer arguments in one call. - Value-nested stf_1()..stf_8(): stN embeds every smaller struct by value, so the nesting deepens with N and each level is a distinct struct-pointer argument (on-stack; st8 is built on the heap to stay under the frame limit). - Pointer-linked stpf_1()..stpf_8(): every member is a pointer to a separately allocated node, so expansion is followed through the heap; run over both kmalloc and vmalloc allocations. Plus pointer forwarding (sf_fwd -> sf_fwd_inner) and a struct return value (sf_ret_struct). eight_struct_args_rust mirrors this with no_mangle rsf_*/rstf_*/rstpf_* exported functions and needs CONFIG_RUST. Each object is opted in with KCOV_DATAFLOW_.o := y; kselftest script will trigger them and checks the expanded field values (0x11, 0x22, ...) per argument and every return value, so the modules validate the captured data, not merely that records appeared. Assisted-by: Claude:claude-opus-4-6 [kiro-chat] Signed-off-by: Yunseong Kim --- .../kcov_dataflow/eight_struct_args_c/Makefile | 3 + .../kcov_dataflow/eight_struct_args_c/README.rst | 13 + .../eight_struct_args_c/eight_struct_args_c.c | 533 +++++++++++++++++ .../kcov_dataflow/eight_struct_args_rust/Makefile | 3 + .../eight_struct_args_rust/README.rst | 11 + .../eight_struct_args_rust.rs | 646 +++++++++++++++++++++ 6 files changed, 1209 insertions(+) diff --git a/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/Makefile b/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/Makefile new file mode 100644 index 0000000000000..04ff83f0a9625 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-m := eight_struct_args_c.o +KCOV_DATAFLOW_eight_struct_args_c.o := y diff --git a/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/README.rst b/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/README.rst new file mode 100644 index 0000000000000..62cddee78cd36 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/README.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests: eight_struct_args_c +============================================ + +C module with 1-8 struct pointer arguments (flat s1..s8), value-nested +st1..st8 and pointer-linked stp1..stp8 towers (on stack, kmalloc and +vmalloc), pointer forwarding and a struct return value. Opted in with +``KCOV_DATAFLOW_eight_struct_args_c.o := y``; test_modules.py checks the +expanded fields (0x11, 0x22, ...) and every return value:: + + ./test_modules.py -t eight_struct_args_c + ./trigger-view.py eight_struct_args_c --raw diff --git a/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/eight_struct_args_c.c b/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/eight_struct_args_c.c new file mode 100644 index 0000000000000..c7d06a8e94c38 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/eight_struct_args_c/eight_struct_args_c.c @@ -0,0 +1,533 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * eight_struct_args_c.c - Verify kcov_dataflow captures struct pointer + * arguments with automatic field expansion. + * + * Three families of structs are exercised: + * + * - Flat structs s1..s8: sN has N u64 members side by side; sf_N takes N + * struct pointer args (s1*..sN*). Tests plain field expansion and multiple + * struct-pointer arguments. + * + * - Recursively (value) nested structs st1..st8: stN embeds every smaller + * struct by value, so the nesting deepens with N: + * st1 = { u64 field0 } + * st2 = { u64 field0, st1 field1 } // { v, {v} } + * stN = { u64 field0, st1 field1, ... st(N-1) field(N-1) } + * The deepest chain in st8 is eight levels deep. Used by the stack tests. + * + * - Pointer-linked nested structs stp1..stp8: every member is a POINTER to a + * separately allocated object, so the nesting is followed through the heap: + * stp1 = { u64 *field0 } + * stp2 = { u64 *field0, stp1 *field1 } // { *v, *{v} } + * stpN = { u64 *field0, stp1 *field1, ... stp(N-1) *field(N-1) } + * Used by the dynamic-allocation (kmalloc/vmalloc) tests. + * + * Write to /sys/kernel/debug/kcov_dataflow_test/trigger_struct to invoke. + */ +#include +#include +#include +#include + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("KCOV dataflow struct field expansion test (flat + nested)"); + +/* Flat structs: sN has N u64 members. */ +struct s1 { u64 a; }; +struct s2 { u64 a; u64 b; }; +struct s3 { u64 a; u64 b; u64 c; }; +struct s4 { u64 a; u64 b; u64 c; u64 d; }; +struct s5 { u64 a; u64 b; u64 c; u64 d; u64 e; }; +struct s6 { u64 a; u64 b; u64 c; u64 d; u64 e; u64 f; }; +struct s7 { u64 a; u64 b; u64 c; u64 d; u64 e; u64 f; u64 g; }; +struct s8 { u64 a; u64 b; u64 c; u64 d; u64 e; u64 f; u64 g; u64 h; }; + +/* + * Recursively (value) nested structs: stN = { u64 field0; st1 field1; ...; + * st(N-1) field(N-1); }. Each stN contains every smaller struct by value, so + * the nesting depth grows with N (st8 is eight levels deep along its st7 chain). + */ +struct st1 { u64 field0; }; +struct st2 { u64 field0; struct st1 field1; }; +struct st3 { u64 field0; struct st1 field1; struct st2 field2; }; +struct st4 { + u64 field0; + struct st1 field1; + struct st2 field2; + struct st3 field3; +}; +struct st5 { + u64 field0; + struct st1 field1; + struct st2 field2; + struct st3 field3; + struct st4 field4; +}; +struct st6 { + u64 field0; + struct st1 field1; + struct st2 field2; + struct st3 field3; + struct st4 field4; + struct st5 field5; +}; +struct st7 { + u64 field0; + struct st1 field1; + struct st2 field2; + struct st3 field3; + struct st4 field4; + struct st5 field5; + struct st6 field6; +}; +struct st8 { + u64 field0; + struct st1 field1; + struct st2 field2; + struct st3 field3; + struct st4 field4; + struct st5 field5; + struct st6 field6; + struct st7 field7; +}; + +/* + * Pointer-linked nested structs: every member is a POINTER to a separately + * allocated object. stpN = { u64 *field0; stp1 *field1; ...; stp(N-1) + * *field(N-1); }. The dynamic-allocation tests build one of these per allocator. + */ +struct stp1 { u64 *field0; }; +struct stp2 { u64 *field0; struct stp1 *field1; }; +struct stp3 { u64 *field0; struct stp1 *field1; struct stp2 *field2; }; +struct stp4 { + u64 *field0; + struct stp1 *field1; + struct stp2 *field2; + struct stp3 *field3; +}; +struct stp5 { + u64 *field0; + struct stp1 *field1; + struct stp2 *field2; + struct stp3 *field3; + struct stp4 *field4; +}; +struct stp6 { + u64 *field0; + struct stp1 *field1; + struct stp2 *field2; + struct stp3 *field3; + struct stp4 *field4; + struct stp5 *field5; +}; +struct stp7 { + u64 *field0; + struct stp1 *field1; + struct stp2 *field2; + struct stp3 *field3; + struct stp4 *field4; + struct stp5 *field5; + struct stp6 *field6; +}; +struct stp8 { + u64 *field0; + struct stp1 *field1; + struct stp2 *field2; + struct stp3 *field3; + struct stp4 *field4; + struct stp5 *field5; + struct stp6 *field6; + struct stp7 *field7; +}; + +/* Prototypes: sf_N takes N struct pointer arguments (s1*, s2*, ..., sN*) */ +u64 sf_1(struct s1 *a); +u64 sf_2(struct s1 *a, struct s2 *b); +u64 sf_3(struct s1 *a, struct s2 *b, struct s3 *c); +u64 sf_4(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d); +u64 sf_5(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, struct s5 *e); +u64 sf_6(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, struct s5 *e, + struct s6 *f); +u64 sf_7(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, struct s5 *e, + struct s6 *f, struct s7 *g); +u64 sf_8(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, struct s5 *e, + struct s6 *f, struct s7 *g, struct s8 *h); + +/* stf_N takes a pointer to the value-nested stN and sums every reachable field0. */ +u64 stf_1(struct st1 *p); +u64 stf_2(struct st2 *p); +u64 stf_3(struct st3 *p); +u64 stf_4(struct st4 *p); +u64 stf_5(struct st5 *p); +u64 stf_6(struct st6 *p); +u64 stf_7(struct st7 *p); +u64 stf_8(struct st8 *p); + +/* stpf_N follows the pointer-linked stpN and sums every reachable *field0. */ +u64 stpf_1(struct stp1 *p); +u64 stpf_2(struct stp2 *p); +u64 stpf_3(struct stp3 *p); +u64 stpf_4(struct stp4 *p); +u64 stpf_5(struct stp5 *p); +u64 stpf_6(struct stp6 *p); +u64 stpf_7(struct stp7 *p); +u64 stpf_8(struct stp8 *p); + +noinline u64 sf_1(struct s1 *a) { return a->a; } +EXPORT_SYMBOL(sf_1); + +noinline u64 sf_2(struct s1 *a, struct s2 *b) { return a->a + b->b; } +EXPORT_SYMBOL(sf_2); + +noinline u64 sf_3(struct s1 *a, struct s2 *b, struct s3 *c) +{ + return a->a + b->b + c->c; +} +EXPORT_SYMBOL(sf_3); + +noinline u64 sf_4(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d) +{ + return a->a + b->b + c->c + d->d; +} +EXPORT_SYMBOL(sf_4); + +noinline u64 sf_5(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, + struct s5 *e) +{ + return a->a + b->b + c->c + d->d + e->e; +} +EXPORT_SYMBOL(sf_5); + +noinline u64 sf_6(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, + struct s5 *e, struct s6 *f) +{ + return a->a + b->b + c->c + d->d + e->e + f->f; +} +EXPORT_SYMBOL(sf_6); + +noinline u64 sf_7(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, + struct s5 *e, struct s6 *f, struct s7 *g) +{ + return a->a + b->b + c->c + d->d + e->e + f->f + g->g; +} +EXPORT_SYMBOL(sf_7); + +noinline u64 sf_8(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d, + struct s5 *e, struct s6 *f, struct s7 *g, struct s8 *h) +{ + return a->a + b->b + c->c + d->d + e->e + f->f + g->g + h->h; +} +EXPORT_SYMBOL(sf_8); + +/* + * Value-nested functions. Each reads its own field0 and forwards the address of + * every nested member into the matching stf_k, so the whole recursive tower is + * walked and each nesting level is a distinct instrumented struct-pointer arg. + */ +noinline u64 stf_1(struct st1 *p) { return p->field0; } +EXPORT_SYMBOL(stf_1); + +noinline u64 stf_2(struct st2 *p) +{ + return p->field0 + stf_1(&p->field1); +} +EXPORT_SYMBOL(stf_2); + +noinline u64 stf_3(struct st3 *p) +{ + return p->field0 + stf_1(&p->field1) + stf_2(&p->field2); +} +EXPORT_SYMBOL(stf_3); + +noinline u64 stf_4(struct st4 *p) +{ + return p->field0 + stf_1(&p->field1) + stf_2(&p->field2) + + stf_3(&p->field3); +} +EXPORT_SYMBOL(stf_4); + +noinline u64 stf_5(struct st5 *p) +{ + return p->field0 + stf_1(&p->field1) + stf_2(&p->field2) + + stf_3(&p->field3) + stf_4(&p->field4); +} +EXPORT_SYMBOL(stf_5); + +noinline u64 stf_6(struct st6 *p) +{ + return p->field0 + stf_1(&p->field1) + stf_2(&p->field2) + + stf_3(&p->field3) + stf_4(&p->field4) + stf_5(&p->field5); +} +EXPORT_SYMBOL(stf_6); + +noinline u64 stf_7(struct st7 *p) +{ + return p->field0 + stf_1(&p->field1) + stf_2(&p->field2) + + stf_3(&p->field3) + stf_4(&p->field4) + stf_5(&p->field5) + + stf_6(&p->field6); +} +EXPORT_SYMBOL(stf_7); + +noinline u64 stf_8(struct st8 *p) +{ + return p->field0 + stf_1(&p->field1) + stf_2(&p->field2) + + stf_3(&p->field3) + stf_4(&p->field4) + stf_5(&p->field5) + + stf_6(&p->field6) + stf_7(&p->field7); +} +EXPORT_SYMBOL(stf_8); + +/* + * Pointer-linked functions. Each dereferences its own *field0 and forwards each + * (already pointer-typed) nested member into the matching stpf_k, following the + * heap-linked tower. + */ +noinline u64 stpf_1(struct stp1 *p) { return *p->field0; } +EXPORT_SYMBOL(stpf_1); + +noinline u64 stpf_2(struct stp2 *p) +{ + return *p->field0 + stpf_1(p->field1); +} +EXPORT_SYMBOL(stpf_2); + +noinline u64 stpf_3(struct stp3 *p) +{ + return *p->field0 + stpf_1(p->field1) + stpf_2(p->field2); +} +EXPORT_SYMBOL(stpf_3); + +noinline u64 stpf_4(struct stp4 *p) +{ + return *p->field0 + stpf_1(p->field1) + stpf_2(p->field2) + + stpf_3(p->field3); +} +EXPORT_SYMBOL(stpf_4); + +noinline u64 stpf_5(struct stp5 *p) +{ + return *p->field0 + stpf_1(p->field1) + stpf_2(p->field2) + + stpf_3(p->field3) + stpf_4(p->field4); +} +EXPORT_SYMBOL(stpf_5); + +noinline u64 stpf_6(struct stp6 *p) +{ + return *p->field0 + stpf_1(p->field1) + stpf_2(p->field2) + + stpf_3(p->field3) + stpf_4(p->field4) + stpf_5(p->field5); +} +EXPORT_SYMBOL(stpf_6); + +noinline u64 stpf_7(struct stp7 *p) +{ + return *p->field0 + stpf_1(p->field1) + stpf_2(p->field2) + + stpf_3(p->field3) + stpf_4(p->field4) + stpf_5(p->field5) + + stpf_6(p->field6); +} +EXPORT_SYMBOL(stpf_7); + +noinline u64 stpf_8(struct stp8 *p) +{ + return *p->field0 + stpf_1(p->field1) + stpf_2(p->field2) + + stpf_3(p->field3) + stpf_4(p->field4) + stpf_5(p->field5) + + stpf_6(p->field6) + stpf_7(p->field7); +} +EXPORT_SYMBOL(stpf_8); + +u64 sf_fwd_inner(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d); +u64 sf_fwd(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d); +struct s4 sf_ret_struct(struct s1 *a, struct s2 *b); + +/* Pointer forwarding: callee receives pointer and passes it to another func */ +noinline u64 sf_fwd_inner(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d) +{ + return a->a + b->b + c->c + d->d; +} +EXPORT_SYMBOL(sf_fwd_inner); + +noinline u64 sf_fwd(struct s1 *a, struct s2 *b, struct s3 *c, struct s4 *d) +{ + return sf_fwd_inner(a, b, c, d); +} +EXPORT_SYMBOL(sf_fwd); + +/* Struct return value */ +noinline struct s4 sf_ret_struct(struct s1 *a, struct s2 *b) +{ + struct s4 ret = { .a = a->a, .b = b->a, .c = b->b, .d = a->a + b->b }; + + return ret; +} +EXPORT_SYMBOL(sf_ret_struct); + +/* Allocator shims so run_stp8() can build the pointer tree with either API. */ +static void *t_kmalloc(size_t n) { return kmalloc(n, GFP_KERNEL); } +static void *t_vmalloc(size_t n) { return vmalloc(n); } +static void t_kfree(void *p) { kfree(p); } +static void t_vfree(void *p) { vfree(p); } + +/* + * Build the pointer-linked stp8 tower with @alloc (each node separately + * allocated), run stpf_8() over it, then free every node with @fr. Sub-nodes + * are shared (a DAG); each unique allocation is freed exactly once. + */ +static u64 run_stp8(void *(*alloc)(size_t), void (*fr)(void *)) +{ + u64 ret = 0; + u64 *l1 = alloc(sizeof(u64)); + u64 *l2 = alloc(sizeof(u64)); + u64 *l3 = alloc(sizeof(u64)); + u64 *l4 = alloc(sizeof(u64)); + u64 *l5 = alloc(sizeof(u64)); + u64 *l6 = alloc(sizeof(u64)); + u64 *l7 = alloc(sizeof(u64)); + u64 *l8 = alloc(sizeof(u64)); + struct stp1 *p1 = alloc(sizeof(*p1)); + struct stp2 *p2 = alloc(sizeof(*p2)); + struct stp3 *p3 = alloc(sizeof(*p3)); + struct stp4 *p4 = alloc(sizeof(*p4)); + struct stp5 *p5 = alloc(sizeof(*p5)); + struct stp6 *p6 = alloc(sizeof(*p6)); + struct stp7 *p7 = alloc(sizeof(*p7)); + struct stp8 *p8 = alloc(sizeof(*p8)); + + if (l1 && l2 && l3 && l4 && l5 && l6 && l7 && l8 && + p1 && p2 && p3 && p4 && p5 && p6 && p7 && p8) { + *l1 = 0x11; *l2 = 0x22; *l3 = 0x33; *l4 = 0x44; + *l5 = 0x55; *l6 = 0x66; *l7 = 0x77; *l8 = 0x88; + + p1->field0 = l1; + p2->field0 = l2; p2->field1 = p1; + p3->field0 = l3; p3->field1 = p1; p3->field2 = p2; + p4->field0 = l4; p4->field1 = p1; p4->field2 = p2; + p4->field3 = p3; + p5->field0 = l5; p5->field1 = p1; p5->field2 = p2; + p5->field3 = p3; p5->field4 = p4; + p6->field0 = l6; p6->field1 = p1; p6->field2 = p2; + p6->field3 = p3; p6->field4 = p4; p6->field5 = p5; + p7->field0 = l7; p7->field1 = p1; p7->field2 = p2; + p7->field3 = p3; p7->field4 = p4; p7->field5 = p5; + p7->field6 = p6; + p8->field0 = l8; p8->field1 = p1; p8->field2 = p2; + p8->field3 = p3; p8->field4 = p4; p8->field5 = p5; + p8->field6 = p6; p8->field7 = p7; + + ret = stpf_8(p8); + } + + fr(p8); fr(p7); fr(p6); fr(p5); fr(p4); fr(p3); fr(p2); fr(p1); + fr(l8); fr(l7); fr(l6); fr(l5); fr(l4); fr(l3); fr(l2); fr(l1); + return ret; +} + +static struct dentry *test_dir; + +static ssize_t trigger_write(struct file *f, const char __user *buf, + size_t count, loff_t *ppos) +{ + struct s1 v1 = { .a = 0x11 }; + struct s2 v2 = { .a = 0x11, .b = 0x22 }; + struct s3 v3 = { .a = 0x11, .b = 0x22, .c = 0x33 }; + struct s4 v4 = { .a = 0x11, .b = 0x22, .c = 0x33, .d = 0x44 }; + struct s5 v5 = { .a = 0x11, .b = 0x22, .c = 0x33, .d = 0x44, + .e = 0x55 }; + struct s6 v6 = { .a = 0x11, .b = 0x22, .c = 0x33, .d = 0x44, + .e = 0x55, .f = 0x66 }; + struct s7 v7 = { .a = 0x11, .b = 0x22, .c = 0x33, .d = 0x44, + .e = 0x55, .f = 0x66, .g = 0x77 }; + struct s8 v8 = { .a = 0x11, .b = 0x22, .c = 0x33, .d = 0x44, + .e = 0x55, .f = 0x66, .g = 0x77, .h = 0x88 }; + + /* Recursively (value) nested values: each embeds all the smaller ones. */ + struct st1 t1 = { .field0 = 0x11 }; + struct st2 t2 = { .field0 = 0x22, .field1 = t1 }; + struct st3 t3 = { .field0 = 0x33, .field1 = t1, .field2 = t2 }; + struct st4 t4 = { .field0 = 0x44, .field1 = t1, .field2 = t2, + .field3 = t3 }; + struct st5 t5 = { .field0 = 0x55, .field1 = t1, .field2 = t2, + .field3 = t3, .field4 = t4 }; + struct st6 t6 = { .field0 = 0x66, .field1 = t1, .field2 = t2, + .field3 = t3, .field4 = t4, .field5 = t5 }; + struct st7 t7 = { .field0 = 0x77, .field1 = t1, .field2 = t2, + .field3 = t3, .field4 = t4, .field5 = t5, + .field6 = t6 }; + u64 sum = 0; + + /* Flat struct tests: sf_N takes N struct pointer args */ + sum += sf_1(&v1); + sum += sf_2(&v1, &v2); + sum += sf_3(&v1, &v2, &v3); + sum += sf_4(&v1, &v2, &v3, &v4); + sum += sf_5(&v1, &v2, &v3, &v4, &v5); + sum += sf_6(&v1, &v2, &v3, &v4, &v5, &v6); + sum += sf_7(&v1, &v2, &v3, &v4, &v5, &v6, &v7); + sum += sf_8(&v1, &v2, &v3, &v4, &v5, &v6, &v7, &v8); + + /* Value-nested struct tests (on-stack) */ + sum += stf_1(&t1); + sum += stf_2(&t2); + sum += stf_3(&t3); + sum += stf_4(&t4); + sum += stf_5(&t5); + sum += stf_6(&t6); + sum += stf_7(&t7); + /* + * st8 is 1 KiB; keeping it on the stack alongside t1..t7 blows the 2048-byte + * frame limit (-Wframe-larger-than). Build it on the heap (member-wise, so no + * 1 KiB compound-literal temporary lands on the stack either). + */ + { + struct st8 *t8 = kmalloc(sizeof(*t8), GFP_KERNEL); + + if (t8) { + t8->field0 = 0x88; + t8->field1 = t1; + t8->field2 = t2; + t8->field3 = t3; + t8->field4 = t4; + t8->field5 = t5; + t8->field6 = t6; + t8->field7 = t7; + sum += stf_8(t8); + kfree(t8); + } + } + + /* Dynamic allocation: pointer-linked stp8, each node separately alloc'd */ + sum += run_stp8(t_kmalloc, t_kfree); /* heap/slab */ + sum += run_stp8(t_vmalloc, t_vfree); /* vmalloc address space */ + + /* Pointer forwarding: sf_fwd receives pointers and forwards to inner */ + sum += sf_fwd(&v1, &v2, &v3, &v4); + + /* Struct return value */ + { + struct s4 ret = sf_ret_struct(&v1, &v2); + + sum += ret.a + ret.b + ret.c + ret.d; + } + + /* Keep every call above from being optimised away (sum is otherwise dead). */ + OPTIMIZER_HIDE_VAR(sum); + return count; +} + +static const struct file_operations trigger_fops = { + .write = trigger_write, +}; + +static int __init eight_struct_args_init(void) +{ + test_dir = debugfs_create_dir("kcov_dataflow_test", NULL); + debugfs_create_file("trigger_struct", 0200, test_dir, NULL, + &trigger_fops); + return 0; +} + +static void __exit eight_struct_args_exit(void) +{ + debugfs_remove_recursive(test_dir); +} + +module_init(eight_struct_args_init); +module_exit(eight_struct_args_exit); diff --git a/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/Makefile b/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/Makefile new file mode 100644 index 0000000000000..3017a24774051 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-m := eight_struct_args_rust.o +KCOV_DATAFLOW_eight_struct_args_rust.o := y diff --git a/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/README.rst b/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/README.rst new file mode 100644 index 0000000000000..06e8f8070f6c2 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/README.rst @@ -0,0 +1,11 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests: eight_struct_args_rust +=============================================== + +Rust equivalent of eight_struct_args_c (rsf_*, rstf_*, rstpf_* with +``#[no_mangle]``), built only with CONFIG_RUST=y. Opted in with +``KCOV_DATAFLOW_eight_struct_args_rust.o := y``:: + + ./test_modules.py -t eight_struct_args_rust + ./trigger-view.py eight_struct_args_rust --raw diff --git a/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/eight_struct_args_rust.rs b/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/eight_struct_args_rust.rs new file mode 100644 index 0000000000000..e5cc3cb87591e --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/eight_struct_args_rust/eight_struct_args_rust.rs @@ -0,0 +1,646 @@ +// SPDX-License-Identifier: GPL-2.0 +//! Verify kcov_dataflow captures struct pointer arguments with automatic +//! field expansion for Rust #[repr(C)] structs. +//! +//! Rust equivalent of eight_struct_args_c. Two families are exercised: +//! - Flat structs S1..S8 (1-8 u64 members) via rsf_N. +//! - Recursively (value) nested structs St1..St8, where StN embeds every +//! smaller struct by value: +//! St1 = { field0 } +//! St2 = { field0, field1: St1 } // { v, {v} } +//! StN = { field0, field1: St1, ..., field(N-1): St(N-1) } +//! so St8 is eight levels deep along its St7 chain. Each rstf_N reads its +//! own field0 and forwards each nested member's address into rstf_k. +//! - Pointer-linked nested structs Stp1..Stp8, where every member is a raw +//! pointer to a separately allocated object: +//! Stp1 = { field0: *const u64 } +//! StpN = { field0: *const u64, field1: *const Stp1, ... } +//! The heap (KBox) test builds this tower and follows it via rstpf_N. +//! +//! Write to /sys/kernel/debug/kcov_dataflow_test/trigger_struct_rust to invoke. + +#![allow(missing_docs)] + +use kernel::prelude::*; +use kernel::alloc::KBox; +use kernel::c_str; + +module !{ + type:EightStructArgsRust, + name: "eight_struct_args_rust", + authors: ["kcov-dataflow"], + description: "Struct field expansion test for kcov_dataflow (Rust)", + license: "GPL", +} +#[repr(C)] +pub struct S1 { + pub a : u64 +} +#[repr(C)] +pub struct S2 { + pub a : u64, pub b : u64 +} +#[repr(C)] +pub struct S3 { + pub a : u64, pub b : u64, pub c : u64 +} +#[repr(C)] +pub struct S4 { + pub a : u64, pub b : u64, pub c : u64, pub d : u64 +} +#[repr(C)] +pub struct S5 { + pub a : u64, pub b : u64, pub c : u64, pub d : u64, pub e : u64 +} +#[repr(C)] +pub struct S6 { + pub a : u64, pub b : u64, pub c : u64, pub d : u64, pub e : u64, + pub f : u64 +} +#[repr(C)] +pub struct S7 { + pub a : u64, pub b : u64, pub c : u64, pub d : u64, pub e : u64, + pub f : u64, pub g : u64 +} +#[repr(C)] +pub struct S8 { + pub a : u64, pub b : u64, pub c : u64, pub d : u64, pub e : u64, + pub f : u64, pub g : u64, pub h : u64 +} +// Recursively nested: StN = { field0, field1: St1, ..., field(N-1): St(N-1) }. +// Copy so a smaller value can be embedded into every larger one. +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St1 { + pub field0 : u64 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St2 { + pub field0 : u64, pub field1 : St1 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St3 { + pub field0 : u64, pub field1 : St1, pub field2 : St2 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St4 { + pub field0 : u64, pub field1 : St1, pub field2 : St2, pub field3 : St3 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St5 { + pub field0 : u64, pub field1 : St1, pub field2 : St2, pub field3 : St3, + pub field4 : St4 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St6 { + pub field0 : u64, pub field1 : St1, pub field2 : St2, pub field3 : St3, + pub field4 : St4, pub field5 : St5 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St7 { + pub field0 : u64, pub field1 : St1, pub field2 : St2, pub field3 : St3, + pub field4 : St4, pub field5 : St5, pub field6 : St6 +} +#[repr(C)] +#[derive(Clone, Copy)] +pub struct St8 { + pub field0 : u64, pub field1 : St1, pub field2 : St2, pub field3 : St3, + pub field4 : St4, pub field5 : St5, pub field6 : St6, + pub field7 : St7 +} +// Pointer-linked nested: every member is a raw pointer to a separately +// allocated object. StpN = { field0: *const u64, field1: *const Stp1, ... }. +#[repr(C)] +pub struct Stp1 { + pub field0 : *const u64 +} +#[repr(C)] +pub struct Stp2 { + pub field0 : *const u64, pub field1 : *const Stp1 +} +#[repr(C)] +pub struct Stp3 { + pub field0 : *const u64, pub field1 : *const Stp1, + pub field2 : *const Stp2 +} +#[repr(C)] +pub struct Stp4 { + pub field0 : *const u64, pub field1 : *const Stp1, + pub field2 : *const Stp2, pub field3 : *const Stp3 +} +#[repr(C)] +pub struct Stp5 { + pub field0 : *const u64, pub field1 : *const Stp1, + pub field2 : *const Stp2, pub field3 : *const Stp3, + pub field4 : *const Stp4 +} +#[repr(C)] +pub struct Stp6 { + pub field0 : *const u64, pub field1 : *const Stp1, + pub field2 : *const Stp2, pub field3 : *const Stp3, + pub field4 : *const Stp4, pub field5 : *const Stp5 +} +#[repr(C)] +pub struct Stp7 { + pub field0 : *const u64, pub field1 : *const Stp1, + pub field2 : *const Stp2, pub field3 : *const Stp3, + pub field4 : *const Stp4, pub field5 : *const Stp5, + pub field6 : *const Stp6 +} +#[repr(C)] +pub struct Stp8 { + pub field0 : *const u64, pub field1 : *const Stp1, + pub field2 : *const Stp2, pub field3 : *const Stp3, + pub field4 : *const Stp4, pub field5 : *const Stp5, + pub field6 : *const Stp6, pub field7 : *const Stp7 +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_1(a : *const S1) -> u64 +{ + unsafe + { + (*a).a + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_2(a : *const S1, b : *const S2) -> u64 +{ + unsafe + { + (*a).a + (*b).b + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_4(a : *const S1, b : *const S2, c : *const S3, + d : *const S4) -> u64 +{ + unsafe + { + (*a).a + (*b).b + (*c).c + (*d).d + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_8(a : *const S1, b : *const S2, c : *const S3, + d : *const S4, e : *const S5, f : *const S6, + g : *const S7, h : *const S8) -> u64 +{ + unsafe + { + (*a).a + (*b).b + (*c).c + (*d).d + (*e).e + (*f).f + (*g).g + + (*h).h + } +} + +// Recursively nested: each reads its own field0 and forwards every nested +// member's address into the matching rstf_k, walking the whole tower. +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_1(p : *const St1) -> u64 +{ + unsafe + { + (*p).field0 + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_2(p : *const St2) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_3(p : *const St3) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + rstf_2(&(*p).field2) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_4(p : *const St4) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + rstf_2(&(*p).field2) + + rstf_3(&(*p).field3) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_5(p : *const St5) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + rstf_2(&(*p).field2) + + rstf_3(&(*p).field3) + rstf_4(&(*p).field4) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_6(p : *const St6) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + rstf_2(&(*p).field2) + + rstf_3(&(*p).field3) + rstf_4(&(*p).field4) + + rstf_5(&(*p).field5) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_7(p : *const St7) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + rstf_2(&(*p).field2) + + rstf_3(&(*p).field3) + rstf_4(&(*p).field4) + + rstf_5(&(*p).field5) + rstf_6(&(*p).field6) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstf_8(p : *const St8) -> u64 +{ + unsafe + { + (*p).field0 + rstf_1(&(*p).field1) + rstf_2(&(*p).field2) + + rstf_3(&(*p).field3) + rstf_4(&(*p).field4) + + rstf_5(&(*p).field5) + rstf_6(&(*p).field6) + + rstf_7(&(*p).field7) + } +} + +// Pointer-linked: each dereferences its own *field0 and forwards each +// (already pointer-typed) nested member into the matching rstpf_k. +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_1(p : *const Stp1) -> u64 +{ + unsafe + { + *(*p).field0 + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_2(p : *const Stp2) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_3(p : *const Stp3) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + rstpf_2((*p).field2) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_4(p : *const Stp4) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + rstpf_2((*p).field2) + + rstpf_3((*p).field3) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_5(p : *const Stp5) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + rstpf_2((*p).field2) + + rstpf_3((*p).field3) + rstpf_4((*p).field4) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_6(p : *const Stp6) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + rstpf_2((*p).field2) + + rstpf_3((*p).field3) + rstpf_4((*p).field4) + + rstpf_5((*p).field5) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_7(p : *const Stp7) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + rstpf_2((*p).field2) + + rstpf_3((*p).field3) + rstpf_4((*p).field4) + + rstpf_5((*p).field5) + rstpf_6((*p).field6) + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rstpf_8(p : *const Stp8) -> u64 +{ + unsafe + { + *(*p).field0 + rstpf_1((*p).field1) + rstpf_2((*p).field2) + + rstpf_3((*p).field3) + rstpf_4((*p).field4) + + rstpf_5((*p).field5) + rstpf_6((*p).field6) + + rstpf_7((*p).field7) + } +} + +// Build the pointer-linked Stp8 tower with KBox (each node its own allocation), +// run rstpf_8 over it, and return the sum. The KBoxes own the storage and hold +// raw pointers into their siblings; everything is freed when they drop at the +// end of this function. `?` frees any already-allocated KBoxes on OOM. +fn build_and_run_stp8() -> Result +{ + let l1 = KBox::new (0x11u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l2 = KBox::new (0x22u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l3 = KBox::new (0x33u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l4 = KBox::new (0x44u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l5 = KBox::new (0x55u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l6 = KBox::new (0x66u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l7 = KBox::new (0x77u64, kernel::alloc::flags::GFP_KERNEL) ? ; + let l8 = KBox::new (0x88u64, kernel::alloc::flags::GFP_KERNEL) ? ; + + let p1 = KBox::new (Stp1{ field0: &*l1 }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p2 = KBox::new (Stp2{ field0: &*l2, field1: &*p1 }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p3 = KBox::new (Stp3{ field0: &*l3, field1: &*p1, field2: &*p2 }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p4 = KBox::new ( + Stp4{ field0: &*l4, field1: &*p1, field2: &*p2, field3: &*p3 }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p5 = KBox::new (Stp5{ + field0: &*l5, + field1: &*p1, + field2: &*p2, + field3: &*p3, + field4: &*p4 + }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p6 = KBox::new (Stp6{ + field0: &*l6, + field1: &*p1, + field2: &*p2, + field3: &*p3, + field4: &*p4, + field5: &*p5 + }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p7 = KBox::new (Stp7{ + field0: &*l7, + field1: &*p1, + field2: &*p2, + field3: &*p3, + field4: &*p4, + field5: &*p5, + field6: &*p6 + }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + let p8 = KBox::new (Stp8{ + field0: &*l8, + field1: &*p1, + field2: &*p2, + field3: &*p3, + field4: &*p4, + field5: &*p5, + field6: &*p6, + field7: &*p7 + }, + kernel::alloc::flags::GFP_KERNEL) ? + ; + + Ok(rstpf_8(&*p8)) +} + +/* Pointer forwarding: receives pointers and passes to inner */ +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_fwd_inner(a : *const S1, b : *const S2, c : *const S3, + d : *const S4) -> u64 +{ + unsafe + { + (*a).a + (*b).b + (*c).c + (*d).d + } +} + +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_fwd(a : *const S1, b : *const S2, c : *const S3, + d : *const S4) -> u64{ rsf_fwd_inner(a, b, c, d) } + +/* Struct return value */ +#[no_mangle] +#[inline(never)] +pub extern "C" fn rsf_ret_struct(a : *const S1, b : *const S2) + ->S4 +{ + unsafe + { + S4 + { +a: + (*a).a, b : (*b).a, c : (*b).b, d : (*a).a + (*b).b + } + } +} + +unsafe extern "C" fn write_handler(_file : *mut kernel::bindings::file, + _buf : *const core::ffi::c_char, + count : usize, + _ppos : *mut kernel::bindings::loff_t, ) + -> kernel::ffi::c_long +{ + let v1 = S1{ a: 0x11 }; + let v2 = S2{ a: 0x11, b: 0x22 }; + let v3 = S3{ a: 0x11, b: 0x22, c: 0x33 }; + let v4 = S4{ a: 0x11, b: 0x22, c: 0x33, d: 0x44 }; + let v5 = S5{ a: 0x11, b: 0x22, c: 0x33, d: 0x44, e: 0x55 }; + let v6 = S6{ a: 0x11, b: 0x22, c: 0x33, d: 0x44, e: 0x55, f: 0x66 }; + let v7 = + S7{ a: 0x11, b: 0x22, c: 0x33, d: 0x44, e: 0x55, f: 0x66, g: 0x77 }; + let v8 = S8{ + a: 0x11, + b: 0x22, + c: 0x33, + d: 0x44, + e: 0x55, + f: 0x66, + g: 0x77, + h: 0x88 + }; + + // Recursively nested values: each embeds all the smaller ones (Copy). + let t1 = St1{ field0: 0x11 }; + let t2 = St2{ field0: 0x22, field1: t1 }; + let t3 = St3{ field0: 0x33, field1: t1, field2: t2 }; + let t4 = St4{ field0: 0x44, field1: t1, field2: t2, field3: t3 }; + let t5 = + St5{ field0: 0x55, field1: t1, field2: t2, field3: t3, field4: t4 }; + let t6 = St6{ + field0: 0x66, + field1: t1, + field2: t2, + field3: t3, + field4: t4, + field5: t5 + }; + let t7 = St7{ + field0: 0x77, + field1: t1, + field2: t2, + field3: t3, + field4: t4, + field5: t5, + field6: t6 + }; + let t8 = St8{ + field0: 0x88, + field1: t1, + field2: t2, + field3: t3, + field4: t4, + field5: t5, + field6: t6, + field7: t7 + }; + + let mut sum : u64 = 0; + sum = sum.wrapping_add(rsf_1(&v1 as *const S1)); + sum = sum.wrapping_add(rsf_2(&v1 as *const S1, &v2 as *const S2)); + sum = sum.wrapping_add(rsf_4(&v1 as *const S1, &v2 as *const S2, + &v3 as *const S3, &v4 as *const S4)); + sum = sum.wrapping_add(rsf_8(&v1 as *const S1, &v2 as *const S2, + &v3 as *const S3, &v4 as *const S4, + &v5 as *const S5, &v6 as *const S6, + &v7 as *const S7, &v8 as *const S8)); + + // Recursively nested struct tests + sum = sum.wrapping_add(rstf_1(&t1 as *const St1)); + sum = sum.wrapping_add(rstf_2(&t2 as *const St2)); + sum = sum.wrapping_add(rstf_3(&t3 as *const St3)); + sum = sum.wrapping_add(rstf_4(&t4 as *const St4)); + sum = sum.wrapping_add(rstf_5(&t5 as *const St5)); + sum = sum.wrapping_add(rstf_6(&t6 as *const St6)); + sum = sum.wrapping_add(rstf_7(&t7 as *const St7)); + sum = sum.wrapping_add(rstf_8(&t8 as *const St8)); + + // Pointer forwarding: rsf_fwd receives and passes to rsf_fwd_inner + sum = sum.wrapping_add(rsf_fwd(&v1 as *const S1, &v2 as *const S2, + &v3 as *const S3, &v4 as *const S4)); + + // Struct return value + let ret = rsf_ret_struct(&v1 as *const S1, &v2 as *const S2); + sum = sum.wrapping_add(ret.a + ret.b + ret.c + ret.d); + + // Dynamic allocation: pointer-linked Stp8 tower (each node its own KBox) + if let + Ok(s) = build_and_run_stp8() + { + sum = sum.wrapping_add(s); + } + + core::hint::black_box(sum); + count as kernel::ffi::c_long +} + +#[repr(transparent)] +struct SyncFops(kernel::bindings::file_operations); +unsafe impl Sync for SyncFops +{ +} + +static FOPS : SyncFops = SyncFops(kernel::bindings::file_operations{ + write: Some(unsafe{ core::mem::transmute(write_handler as *const()) }), + ..unsafe{ core::mem::zeroed() } +}); + +struct EightStructArgsRust { + dir : *mut kernel::bindings::dentry, +} + +impl kernel::Module for EightStructArgsRust +{ + fn init(_module: &'static ThisModule) -> Result { + let dir = unsafe { + kernel::bindings::debugfs_create_dir( + c_str!("kcov_dataflow_test").as_char_ptr(), + core::ptr::null_mut(), + ) + }; + unsafe { + kernel::bindings::debugfs_create_file_unsafe( + c_str!("trigger_struct_rust").as_char_ptr(), + 0o222, + dir, + core::ptr::null_mut(), + &FOPS.0, + ) + }; + Ok(Self { dir }) +} +} + +impl Drop for EightStructArgsRust +{ + fn drop(&mut self) + { + unsafe{ kernel::bindings::debugfs_remove(self.dir) }; + } +} + +unsafe impl Send for EightStructArgsRust +{ +} +unsafe impl Sync for EightStructArgsRust +{ +} -- 2.47.3 Add a Rust module that exercises remote dataflow collection: work a task queues to a kworker, bracketed with kcov_df_remote_start(handle) / kcov_df_remote_stop(), is attributed back to the buffer user space published for that handle with KCOV_DF_REMOTE_ENABLE. A CompositeStore of three RBTrees (standing in for separate lookup tables) is driven through three phases on system_wq: populate: fill all three trees update: insert new entries, read existing ones, overwrite some drain: remove every entry via a cursor The work item calls kcov_df_remote_start(1) on entry and kcov_df_remote_stop() before completing; kselftest script publishe handle 1 (KCOV_SUBSYSTEM_COMMON, instance 1) with KCOV_DF_REMOTE_ENABLE, triggers the kworker through a debugfs file, waits for completion, and checks that records from all three CompositeStore phases were captured from the kworker rather than the triggering task. Needs CONFIG_RUST. Assisted-by: Claude:claude-opus-4-6 [kiro-chat] Signed-off-by: Yunseong Kim --- .../kcov_dataflow/rust_kworker_remote/Makefile | 3 + .../kcov_dataflow/rust_kworker_remote/README.rst | 13 ++ .../rust_kworker_remote/rust_kworker_remote.rs | 207 +++++++++++++++++++++ 3 files changed, 223 insertions(+) diff --git a/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/Makefile b/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/Makefile new file mode 100644 index 0000000000000..cb7392a50b1a9 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-m := rust_kworker_remote.o +KCOV_DATAFLOW_rust_kworker_remote.o := y diff --git a/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/README.rst b/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/README.rst new file mode 100644 index 0000000000000..aff597ab67aea --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/README.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests: rust_kworker_remote +============================================ + +Rust module testing kcov_df_remote_start()/kcov_df_remote_stop() from +kworker context: the trigger queues a work item on system_wq whose three +phases (populate/update/drain of a CompositeStore of RBTrees) run with +remote capture on handle 1, which the runner publishes with +KCOV_DF_REMOTE_ENABLE. Built only with CONFIG_RUST=y:: + + ./test_modules.py -t rust_kworker_remote + ./trigger-view.py rust_kworker_remote --remote diff --git a/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/rust_kworker_remote.rs b/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/rust_kworker_remote.rs new file mode 100644 index 0000000000000..65c5722c383cc --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/rust_kworker_remote/rust_kworker_remote.rs @@ -0,0 +1,207 @@ +// SPDX-License-Identifier: GPL-2.0 +//! Test kcov_df_remote_start/stop from kworker context. +//! +//! A composite struct holds three RBTrees (simulating RBTree/XArray/maple_tree +//! workloads). Three work phases run on system_wq: +//! Phase 1 (populate): fill all three trees +//! Phase 2 (update): insert new values, read existing, overwrite +//! Phase 3 (drain): remove all entries +//! +//! User space publishes a buffer with KCOV_DF_REMOTE_ENABLE, writes to +//! /sys/kernel/debug/kcov_dataflow_test/trigger_kworker_remote, then reads +//! the captured records. + +#![allow(missing_docs)] + +use kernel::prelude::*; +use kernel::sync::{Arc, Completion}; +use kernel::workqueue::{self, impl_has_work, new_work, Work, WorkItem}; +use kernel::rbtree::RBTree; +use kernel::c_str; + +module! { + type: RustKworkerRemote, + name: "rust_kworker_remote", + authors: ["kcov-dataflow"], + description: "Test kcov_df_remote capturing from kworker (RBTree composite)", + license: "GPL", +} + +// Extern bindings for kcov_dataflow remote API (kernel/kcov_dataflow.c) +unsafe extern "C" { + fn kcov_df_remote_start(handle: u64); + fn kcov_df_remote_stop(); +} + +/// Composite data structure: three trees with different key ranges. +/// Simulates a real driver managing multiple lookup tables. +struct CompositeStore { + /// Primary index (keys 0..N) + primary: RBTree, + /// Secondary/auxiliary index (keys 100..N) + aux: RBTree, + /// Scratch/temp space (keys 200..N) + scratch: RBTree, +} + +impl CompositeStore { + fn new() -> Self { + Self { + primary: RBTree::new(), + aux: RBTree::new(), + scratch: RBTree::new(), + } + } + + /// Phase 1: populate all three trees with initial data. + #[inline(never)] + fn populate(&mut self) -> Result { + for i in 0u64..8 { + self.primary.try_create_and_insert(i, i * 0x1111, GFP_KERNEL)?; + } + for i in 100u64..108 { + self.aux.try_create_and_insert(i, i * 0x2222, GFP_KERNEL)?; + } + for i in 200u64..208 { + self.scratch.try_create_and_insert(i, i * 0x3333, GFP_KERNEL)?; + } + Ok(()) + } + + /// Phase 2: insert more, read existing, overwrite some. + #[inline(never)] + fn update(&mut self) -> Result { + // Insert new entries into primary + for i in 8u64..12 { + self.primary.try_create_and_insert(i, i * 0x4444, GFP_KERNEL)?; + } + // Read from aux (get passes &K which is a struct arg) + for i in 100u64..108 { + let _ = self.aux.get(&i); + } + // Overwrite scratch entries + for i in 200u64..204 { + self.scratch.remove(&i); + self.scratch.try_create_and_insert(i, i * 0x5555, GFP_KERNEL)?; + } + Ok(()) + } + + /// Phase 3: drain all trees. + #[inline(never)] + fn drain(&mut self) { + while let Some(c) = self.primary.cursor_front_mut() { + c.remove_current(); + } + while let Some(c) = self.aux.cursor_front_mut() { + c.remove_current(); + } + while let Some(c) = self.scratch.cursor_front_mut() { + c.remove_current(); + } + } +} + +/// Work item that runs three phases in kworker context with remote capture. +#[pin_data] +struct RemoteWork { + #[pin] + work: Work, + #[pin] + done: Completion, +} + +impl_has_work! { + impl HasWork for RemoteWork { self.work } +} + +impl RemoteWork { + fn new() -> Result> { + Arc::pin_init(pin_init!(RemoteWork { + work <- new_work!("RemoteWork::work"), + done <- Completion::new(), + }), GFP_KERNEL) + } +} + +impl WorkItem for RemoteWork { + type Pointer = Arc; + + fn run(this: Arc) { + // Enable remote kcov_dataflow capture for this kworker task. + // SAFETY: FFI call to exported kernel symbol; no-op if no buffer published. + // Handle 1 matches what trigger-view.py passes via KCOV_DF_REMOTE_ENABLE. + unsafe { kcov_df_remote_start(1) }; + + let mut store = CompositeStore::new(); + let _ = store.populate(); + let _ = store.update(); + store.drain(); + + // SAFETY: FFI call to exported kernel symbol; disables capture. + unsafe { kcov_df_remote_stop() }; + + this.done.complete_all(); + } +} + +// --- Debugfs trigger (same raw pattern as eight_struct_args_rust) --- + +unsafe extern "C" fn write_handler( + _file: *mut kernel::bindings::file, + _buf: *const core::ffi::c_char, + count: usize, + _ppos: *mut kernel::bindings::loff_t, +) -> kernel::ffi::c_long { + let work = match RemoteWork::new() { + Ok(w) => w, + Err(_) => return -(kernel::bindings::ENOMEM as kernel::ffi::c_long), + }; + let waiter = work.clone(); + let _ = workqueue::system().enqueue(work); + waiter.done.wait_for_completion(); + count as kernel::ffi::c_long +} + +#[repr(transparent)] +struct SyncFops(kernel::bindings::file_operations); +unsafe impl Sync for SyncFops {} + +static FOPS: SyncFops = SyncFops(kernel::bindings::file_operations { + write: Some(unsafe { core::mem::transmute(write_handler as *const ()) }), + ..unsafe { core::mem::zeroed() } +}); + +struct RustKworkerRemote { + dir: *mut kernel::bindings::dentry, +} + +impl kernel::Module for RustKworkerRemote { + fn init(_module: &'static ThisModule) -> Result { + let dir = unsafe { + kernel::bindings::debugfs_create_dir( + c_str!("kcov_dataflow_test").as_char_ptr(), + core::ptr::null_mut(), + ) + }; + unsafe { + kernel::bindings::debugfs_create_file_unsafe( + c_str!("trigger_kworker_remote").as_char_ptr(), + 0o222, + dir, + core::ptr::null_mut(), + &FOPS.0, + ) + }; + Ok(Self { dir }) + } +} + +impl Drop for RustKworkerRemote { + fn drop(&mut self) { + unsafe { kernel::bindings::debugfs_remove(self.dir) }; + } +} + +unsafe impl Send for RustKworkerRemote {} +unsafe impl Sync for RustKworkerRemote {} -- 2.47.3 Wire the kcov_dataflow test modules into a standard kselftest target and add the viewer they are built on. trigger-view.py loads a module with finit_module(), enables recording (KCOV_DF_ENABLE, or KCOV_DF_REMOTE_ENABLE with "--remote" argument), writes the module's debugfs trigger file, unloads it, and decodes the buffer into a call tree or, with "--raw", a flat record list. It symbolizes PCs against a vmlinux with addr2line or against /proc/kallsyms, adding the running kernel's KASLR offset back to the recorded PCs (runtime _text minus the link-time _text from System.map / nm / a per-architecture default), and demangles Rust v0 symbols. test_modules.py is the KTAP runner: it drives run_capture() from trigger-view.py for each module and compares the captured arguments, struct fields and return values against the values the module uses emitting the module's call tree as diagnostics. A test passes only when the data came back intact, not merely when records appeared. Modules that were not built are reported as SKIP. The Makefile builds user_ioctl and binderfs as ordinary kselftest programs and the .ko modules via Kbuild against KDIR (the Rust ones only when the configured kernel has CONFIG_RUST=y); config lists the kernel options the tests need, and settings raises the per-test timeout for the in-VM runs. The whole target builds with make -C tools/testing/selftests TARGETS=kcov_dataflow \ LLVM=1 CC=clang [RUSTC=... RUST_LIB_SRC=...] and runs under run_kselftest.sh. Assisted-by: Claude:claude-opus-4-6 [kiro-chat] Signed-off-by: Yunseong Kim --- tools/testing/selftests/kcov_dataflow/.gitignore | 4 + tools/testing/selftests/kcov_dataflow/Kbuild | 10 + tools/testing/selftests/kcov_dataflow/Makefile | 46 ++ tools/testing/selftests/kcov_dataflow/README.rst | 69 ++ tools/testing/selftests/kcov_dataflow/config | 11 + tools/testing/selftests/kcov_dataflow/settings | 1 + .../selftests/kcov_dataflow/test_modules.py | 249 +++++++ .../selftests/kcov_dataflow/trigger-view.py | 755 +++++++++++++++++++++ 8 files changed, 1145 insertions(+) diff --git a/tools/testing/selftests/kcov_dataflow/.gitignore b/tools/testing/selftests/kcov_dataflow/.gitignore new file mode 100644 index 0000000000000..4f2957a017957 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/.gitignore @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +user_ioctl/user_ioctl +binderfs/binderfs_test +__pycache__/ diff --git a/tools/testing/selftests/kcov_dataflow/Kbuild b/tools/testing/selftests/kcov_dataflow/Kbuild new file mode 100644 index 0000000000000..2e19e9008fdca --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/Kbuild @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Test modules, built as external modules against the configured kernel tree +# by the selftest Makefile ("make -C $(KDIR) M=$(CURDIR) modules"). Every +# directory opts its object into dataflow instrumentation with +# KCOV_DATAFLOW_.o := y, the same per-file switch in-tree code uses. +obj-m += rust_ffi_contract/ +obj-m += eight_struct_args_c/ +obj-$(CONFIG_RUST) += eight_struct_args_rust/ +obj-$(CONFIG_RUST) += rust_kworker_remote/ diff --git a/tools/testing/selftests/kcov_dataflow/Makefile b/tools/testing/selftests/kcov_dataflow/Makefile new file mode 100644 index 0000000000000..fc979e2d4ecc3 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/Makefile @@ -0,0 +1,46 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# kcov_dataflow selftests +# +# user_ioctl and binderfs are ordinary kselftest programs. The test modules +# (one per directory, listed in Kbuild) are built by kbuild against KDIR and +# are loaded, triggered and checked by test_modules.py; trigger-view.py is +# the interactive viewer the runner is built on. +# +# KDIR is the configured kernel build tree. It defaults to the source tree +# this directory lives in; point it at the O= directory for out-of-tree +# builds. Pass the same LLVM=1 CC=clang [RUSTC= RUST_LIB_SRC=] the kernel was +# built with so that kbuild picks the toolchain that has the trace-args and +# trace-ret passes. +KDIR ?= $(abspath ../../../..) + +TEST_GEN_PROGS := user_ioctl/user_ioctl binderfs/binderfs_test +TEST_PROGS := test_modules.py +TEST_FILES := trigger-view.py + +CFLAGS += -Wall -O2 $(KHDR_INCLUDES) + +# The .ko files kbuild produces for KDIR's configuration, so that they are +# built by "all" and copied by "install"; the Rust modules need CONFIG_RUST. +KMODS := rust_ffi_contract eight_struct_args_c +ifneq ($(shell grep -s ^CONFIG_RUST=y $(KDIR)/.config),) +KMODS += eight_struct_args_rust rust_kworker_remote +endif +TEST_GEN_FILES := $(foreach m,$(KMODS),$(m)/$(m).ko) + +include ../lib.mk + +ifneq ($(wildcard $(KDIR)/.config),) +$(TEST_GEN_FILES): modules +modules: + $(Q)$(MAKE) -C $(KDIR) M=$(CURDIR) modules +clean_modules: + $(Q)$(MAKE) -C $(KDIR) M=$(CURDIR) clean +else +$(TEST_GEN_FILES): + @echo "SKIP $(notdir $@): no configured kernel tree at $(KDIR), set KDIR=" +clean_modules: +endif + +clean: clean_modules +.PHONY: modules clean_modules diff --git a/tools/testing/selftests/kcov_dataflow/README.rst b/tools/testing/selftests/kcov_dataflow/README.rst new file mode 100644 index 0000000000000..1929a357aca47 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/README.rst @@ -0,0 +1,69 @@ +.. SPDX-License-Identifier: GPL-2.0 + +KCOV-Dataflow Selftests +======================= + +Selftests for ``/sys/kernel/debug/kcov_dataflow`` (see +Documentation/dev-tools/kcov-dataflow.rst). + +Layout +------ + +Makefile, Kbuild + kselftest build: the C programs are built by lib.mk, the test modules + (one directory each, listed in Kbuild) by kbuild against ``KDIR``. +user_ioctl/ + ioctl interface test (kselftest harness, TAP). +binderfs/ + binder ioctls under recording (TAP). +rust_ffi_contract/, eight_struct_args_c/, eight_struct_args_rust/, +rust_kworker_remote/ + test modules; each README.rst says what the module exercises. +test_modules.py + KTAP runner: loads every module, triggers it with recording active and + checks the captured arguments, struct fields and return values against + the values the module uses. Modules that are not built are SKIPped. +trigger-view.py + Interactive viewer the runner is built on (call tree or ``--raw`` + records, kallsyms/addr2line symbolization, ``--remote`` capture). + +Kernel +------ + +The kernel and the modules must be built with a clang that has the +trace-args/trace-ret passes (and, for the Rust modules, a rustc built +against that LLVM). The config fragment ``config`` lists what the tests +need; with virtme-ng:: + + vng --build --config tools/testing/selftests/kcov_dataflow/config \ + LLVM=1 CC=clang RUSTC=$RUSTC RUST_LIB_SRC=$RUST_LIB_SRC + +Build +----- + +From the kernel tree, with the same toolchain variables:: + + make LLVM=1 headers + make -C tools/testing/selftests TARGETS=kcov_dataflow \ + LLVM=1 CC=clang RUSTC=$RUSTC RUST_LIB_SRC=$RUST_LIB_SRC + +``KDIR`` defaults to the source tree; pass ``KDIR=`` for out-of-tree +builds. The Rust modules are built only when ``KDIR/.config`` has +``CONFIG_RUST=y``. ``make ... install INSTALL_PATH=`` produces a +self-contained tree with ``run_kselftest.sh``. + +Run +--- + +On the target (root, debugfs mounted):: + + vng --user root --exec \ + "tools/testing/selftests/kcov_dataflow/test_modules.py" + tools/testing/selftests/kcov_dataflow/user_ioctl/user_ioctl + tools/testing/selftests/kcov_dataflow/binderfs/binderfs_test + +or, from an installed tree, ``run_kselftest.sh -c kcov_dataflow``. +``test_modules.py -t -C 8`` runs one module and echoes eight +records of context around each module record; ``trigger-view.py +[--raw] [-C N] [--remote] [--vmlinux vmlinux]`` shows the capture +without checking it. diff --git a/tools/testing/selftests/kcov_dataflow/config b/tools/testing/selftests/kcov_dataflow/config new file mode 100644 index 0000000000000..7f3a2fda0641d --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/config @@ -0,0 +1,11 @@ +CONFIG_KCOV=y +CONFIG_KCOV_DATAFLOW_ARGS=y +CONFIG_KCOV_DATAFLOW_RET=y +CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL=y +CONFIG_KCOV_DATAFLOW_NO_INLINE=y +CONFIG_DEBUG_INFO_DWARF5=y +CONFIG_DEBUG_FS=y +CONFIG_MODULES=y +CONFIG_ANDROID_BINDER_IPC=y +CONFIG_ANDROID_BINDERFS=y +CONFIG_RUST=y diff --git a/tools/testing/selftests/kcov_dataflow/settings b/tools/testing/selftests/kcov_dataflow/settings new file mode 100644 index 0000000000000..694d70710ff08 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/settings @@ -0,0 +1 @@ +timeout=300 diff --git a/tools/testing/selftests/kcov_dataflow/test_modules.py b/tools/testing/selftests/kcov_dataflow/test_modules.py new file mode 100755 index 0000000000000..13cb706a06ff6 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/test_modules.py @@ -0,0 +1,249 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +""" +test_modules.py - run the kcov_dataflow test modules, one KTAP test each. + +Every module is loaded, triggered with recording active and unloaded by +trigger-view.py's run_capture(). The records that belong to the module are +then compared with the values its trigger function passes and returns, so a +test passes only when the instrumented arguments, struct field expansions +and return values came back intact through the kcov_dataflow buffer. The +module's call tree is echoed as KTAP diagnostics. + + ./test_modules.py # all modules + ./test_modules.py -t rust_ffi_contract -C 8 --vmlinux vmlinux + +Modules that were not built (no CONFIG_RUST, no toolchain) are reported as +SKIP; a kernel without /sys/kernel/debug/kcov_dataflow skips everything. +""" +import argparse +import contextlib +import importlib.util +import io +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, os.path.join(HERE, "..", "kselftest")) +import ksft # noqa: E402 + + +def _load_trigger_view(): + spec = importlib.util.spec_from_file_location( + "trigger_view", os.path.join(HERE, "trigger-view.py")) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +tv = _load_trigger_view() + + +class Check: + """Collects expectation failures for one module.""" + + def __init__(self): + self.failures = [] + + def eq(self, what, got, want): + if got != want: + self.failures.append(f"{what}: got {fmt(got)}, want {fmt(want)}") + + def true(self, what, cond): + if not cond: + self.failures.append(what) + + +def fmt(v): + if isinstance(v, list): + return "[" + ", ".join(fmt(x) for x in v) + "]" + if isinstance(v, int): + return f"0x{v:x}" + return str(v) + + +def entries(cap, recs, func): + return [r for r in recs if r["type"] == tv.DF_TYPE_ENTRY and func in cap.funcs(r)] + + +def rets(cap, recs, func): + return [r["val"] for r in recs if r["type"] == tv.DF_TYPE_RET and func in cap.funcs(r)] + + +def flat_sum(n): + """sf_n() returns a->a + b->b + ... over s1..sn: 0x11 + 0x22 + ...""" + return sum(0x11 * k for k in range(1, n + 1)) + + +def nested_sum(n, _memo={}): + """ + stf_n()/stpf_n() return field0 (0x11 * n) plus the recursive sums of the + embedded st1..st(n-1); the same values are used for the value-nested and + the pointer-linked towers. + """ + if n not in _memo: + _memo[n] = 0x11 * n + sum(nested_sum(k) for k in range(1, n)) + return _memo[n] + + +def check_struct_family(cap, recs, c, p, flat_ns, stpf8_runs): + """ + Shared expectations for eight_struct_args_c (p="") and + eight_struct_args_rust (p="r"): @flat_ns are the sf_N called by the + trigger, @stpf8_runs how often the pointer-linked tower is walked. + """ + for n in flat_ns: + ents = entries(cap, recs, f"{p}sf_{n}") + c.true(f"{p}sf_{n}: ENTRY records", bool(ents)) + for k in range(n): + # arg k is a struct s(k+1) * whose fields are 0x11, 0x22, ... + got = [r["vals"] for r in ents if r["arg_idx"] == k] + c.true(f"{p}sf_{n} arg[{k}]: ENTRY record", bool(got)) + for vals in got: + c.eq(f"{p}sf_{n} arg[{k}] expanded fields", vals, + [0x11 * (j + 1) for j in range(k + 1)]) + # rustc may alias identical bodies (rsf_1 == rstf_1), so the RET + # list can carry the alias's calls too: check every value. + got = rets(cap, recs, f"{p}sf_{n}") + c.true(f"{p}sf_{n} RET values all {fmt(flat_sum(n))}: {fmt(got)}", + bool(got) and all(v == flat_sum(n) for v in got)) + + for fam, calls in ((f"{p}stf", 1), (f"{p}stpf", stpf8_runs)): + for n in range(1, 9): + got = rets(cap, recs, f"{fam}_{n}") + c.true(f"{fam}_{n}: RET records", bool(got)) + c.true(f"{fam}_{n} RET values all {fmt(nested_sum(n))}: {fmt(got)}", + all(v == nested_sum(n) for v in got)) + c.eq(f"{fam}_8 RET count", len(rets(cap, recs, f"{fam}_8")), calls) + + for f in (f"{p}sf_fwd", f"{p}sf_fwd_inner"): + c.eq(f"{f} RET", rets(cap, recs, f), [flat_sum(4)]) + + c.true(f"{p}sf_ret_struct: ENTRY records", + bool(entries(cap, recs, f"{p}sf_ret_struct"))) + c.true(f"{p}sf_ret_struct: RET record", + bool(rets(cap, recs, f"{p}sf_ret_struct"))) + + +def check_eight_struct_args_c(cap, recs, c): + check_struct_family(cap, recs, c, "", range(1, 9), stpf8_runs=2) + + +def check_eight_struct_args_rust(cap, recs, c): + check_struct_family(cap, recs, c, "r", (1, 2, 4, 8), stpf8_runs=1) + + +def check_rust_ffi_contract(cap, recs, c): + """ + ffi_alloc_buf(&alloc = {NULL, 0, 0, 0}, 256, 16, is_async=1) records + data_size + offsets_size and returns 0 without filling alloc->buffer; + ffi_check_result() then sees {NULL, 0x110, 0, 0}. The records must show + the violated contract at both boundaries. + """ + ents = entries(cap, recs, "ffi_alloc_buf") + by_arg = {r["arg_idx"]: r for r in ents} + c.eq("ffi_alloc_buf ENTRY arg indexes", sorted(by_arg), [0, 1, 2, 3]) + if 0 in by_arg: + c.eq("ffi_alloc_buf arg[0] struct ffi_alloc fields", + by_arg[0]["vals"], [0, 0, 0, 0]) + if 1 in by_arg: + c.eq("ffi_alloc_buf arg[1] data_size", by_arg[1]["val"], 256) + if 2 in by_arg: + c.eq("ffi_alloc_buf arg[2] offsets_size", by_arg[2]["val"], 16) + if 3 in by_arg: + c.eq("ffi_alloc_buf arg[3] is_async", by_arg[3]["val"], 1) + c.eq("ffi_alloc_buf RET (claims success)", rets(cap, recs, "ffi_alloc_buf"), [0]) + + ents = entries(cap, recs, "ffi_check_result") + c.true("ffi_check_result: ENTRY record", bool(ents)) + for r in ents: + c.eq("ffi_check_result arg[0] {buffer NULL: contract violated, " + "data_size, free_async, flags}", r["vals"], [0, 0x110, 0, 0]) + got = rets(cap, recs, "ffi_check_result") + c.true(f"ffi_check_result RET -EFAULT: {fmt(got)}", + len(got) == 1 and got[0] & 0xffffffff == 0xfffffff2) + + +def check_rust_kworker_remote(cap, recs, c): + """ + The trigger only queues a work item and waits; the records come from the + kworker that called kcov_df_remote_start(REMOTE_HANDLE). All three phases + of CompositeStore must show up (v0-mangled names keep the method names). + """ + c.true("records captured from the kworker", bool(recs)) + names = set().union(*(cap.funcs(r) for r in recs)) if recs else set() + for phase in ("populate", "update", "drain"): + c.true(f"CompositeStore::{phase} recorded", + any("CompositeStore" in n and phase in n for n in names)) + + +TESTS = ( + ("rust_ffi_contract", False, check_rust_ffi_contract), + ("eight_struct_args_c", False, check_eight_struct_args_c), + ("eight_struct_args_rust", False, check_eight_struct_args_rust), + ("rust_kworker_remote", True, check_rust_kworker_remote), +) + + +def diag_tree(cap, recs, vmlinux): + out = io.StringIO() + with contextlib.redirect_stdout(out): + tv.print_tree(recs, cap.syms, vmlinux, {}, cap.ko_path, + cap.mod_text_start) + for line in out.getvalue().splitlines(): + ksft.print_msg(line) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("-t", "--test", action="append", + help="run only this module (repeatable)") + parser.add_argument("-C", "--context", type=int, default=0, + help="echo N records before/after each module record") + parser.add_argument("--vmlinux", help="vmlinux for addr2line and KASLR") + args = parser.parse_args() + + tests = [t for t in TESTS if not args.test or t[0] in args.test] + ksft.print_header() + ksft.set_plan(len(tests)) + + skip_all = None + if not os.path.exists(tv.KCOV_DF_PATH): + skip_all = f"{tv.KCOV_DF_PATH} not available (CONFIG_KCOV_DATAFLOW_ARGS/RET)" + elif os.geteuid() != 0: + skip_all = "must run as root" + + vmlinux = tv.find_vmlinux(args.vmlinux) + for name, remote, check in tests: + if skip_all: + ksft.test_result_skip(f"{name}: {skip_all}") + continue + ko = tv.find_module(name) + if not ko: + ksft.test_result_skip(f"{name}: {name}.ko not built") + continue + try: + cap = tv.run_capture(ko, remote=remote, vmlinux=vmlinux, + log=ksft.print_msg) + except OSError as e: + ksft.test_result_fail(f"{name}: {e}") + continue + + recs = cap.module_records() + ksft.print_msg(f"{name}: {cap.total_words} words, {len(cap.records)} " + f"records, {len(recs)} from {name} " + f"(kaslr_offset=0x{cap.kaslr_offset:x})") + diag_tree(cap, cap.context_records(args.context) if args.context + else recs, vmlinux) + + c = Check() + check(cap, recs, c) + for f in c.failures: + ksft.print_msg(f"FAIL {name}: {f}") + ksft.test_result(not c.failures, name) + + ksft.finished() + + +if __name__ == "__main__": + main() diff --git a/tools/testing/selftests/kcov_dataflow/trigger-view.py b/tools/testing/selftests/kcov_dataflow/trigger-view.py new file mode 100755 index 0000000000000..b17e49da402d7 --- /dev/null +++ b/tools/testing/selftests/kcov_dataflow/trigger-view.py @@ -0,0 +1,755 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +""" +trigger-view.py - Load a test module, trigger it with kcov_dataflow +recording active, then pretty-print the captured records. + +Usage: + python3 trigger-view.py eight_struct_args_c + python3 trigger-view.py rust_ffi_contract --raw -C 8 + python3 trigger-view.py rust_kworker_remote --remote + python3 trigger-view.py --vmlinux vmlinux --kaslr-offset 0x... + +run_capture() does the work and is also what test_modules.py drives: + 1. Opens /sys/kernel/debug/kcov_dataflow, inits and mmaps the buffer + 2. Loads the module via finit_module() (its init noise is not recorded) + 3. Enables recording: KCOV_DF_ENABLE for this task, or with --remote + KCOV_DF_REMOTE_ENABLE with handle REMOTE_HANDLE, which the module's + kworker opens with kcov_df_remote_start(REMOTE_HANDLE) + 4. Writes the trigger file(s) the module created under TRIGGER_DIR + 5. Disables recording and unloads the module + 6. Parses the records (layout: include/uapi/linux/kcov_dataflow.h) + +The CLI then prints them as a call tree, or flat with --raw, with kallsyms +symbol resolution and addr2line source lines (vmlinux / module .ko). + +Recorded PCs have the KASLR offset removed (same as mainline kcov), so +the runtime offset is derived from /proc/kallsyms and System.map / vmlinux +(or a per-architecture default) and added back for symbolization; use +--kaslr-offset to override. Records must contain at least one value word +and one of the three record types, otherwise the parser resyncs word by +word (e.g. after a userspace reset of area[0] mid-run). +""" +import os +import sys +import struct +import ctypes +import ctypes.util +import argparse +import fcntl +import platform +import subprocess +import shutil + +# Constants -- must match include/uapi/linux/kcov_dataflow.h +DF_TYPE_CMP = 0xC +DF_TYPE_ENTRY = 0xE +DF_TYPE_RET = 0xF +MAGIC_BAD = 0xBADADD85 +BUF_SIZE = 1048576 # 1M words = 8MB + +# Record header word: bits 0-23 seq | 28-31 type | 32-47 nvals | +# 48-55 arg/ret size | 56-63 arg index. Word 1 is the pc (KASLR offset +# removed, like mainline kcov), word 2 the traced pointer (ENTRY/RET) or the +# comparison type (CMP), then nvals value words. +def hdr_seq(h): + return h & 0x00FFFFFF + +def hdr_type(h): + return (h >> 28) & 0xF + +def hdr_nvals(h): + return (h >> 32) & 0xFFFF + +def hdr_size(h): + return (h >> 48) & 0xFF + +def hdr_arg_idx(h): + return (h >> 56) & 0xFF + +RECORD_HDR_WORDS = 3 + +# Runtime KASLR offset (see kaslr_offset()); added back to every recorded pc +# so /proc/kallsyms lookups work, subtracted again for addr2line on vmlinux. +KASLR_OFFSET = 0 + +# Ioctl numbers +def _IOR(t, nr, size): + return (2 << 30) | (ord(t) << 8) | nr | (size << 16) + +def _IOW(t, nr, size): + return (1 << 30) | (ord(t) << 8) | nr | (size << 16) + +def _IO(t, nr): + return (ord(t) << 8) | nr + +KCOV_DF_INIT_TRACK = _IOR('d', 1, 8) +KCOV_DF_ENABLE = _IO('d', 100) +KCOV_DF_DISABLE = _IO('d', 101) +KCOV_DF_REMOTE_ENABLE = _IOW('d', 102, 8) # arg: pointer to a __u64 handle +KCOV_DF_REMOTE_DISABLE = _IO('d', 103) + +KCOV_DF_PATH = "/sys/kernel/debug/kcov_dataflow" + +# Every test module creates its trigger file(s) in this debugfs directory; +# writing to them runs the instrumented test functions. +TRIGGER_DIR = "/sys/kernel/debug/kcov_dataflow_test" + +# Remote handle registered with KCOV_DF_REMOTE_ENABLE; must match the +# kcov_df_remote_start(1) call in the rust_kworker_remote test module +# (KCOV_SUBSYSTEM_COMMON, instance 1). +REMOTE_HANDLE = 1 + +# syscall numbers +_machine = platform.machine() +if _machine == "aarch64": + SYS_FINIT_MODULE = 273 + SYS_DELETE_MODULE = 106 +else: # x86_64 + SYS_FINIT_MODULE = 313 + SYS_DELETE_MODULE = 176 + +SELFTEST_DIR = os.path.dirname(os.path.abspath(__file__)) + + +def load_kallsyms(): + """Load kernel symbols for PC resolution.""" + syms = [] + try: + with open("/proc/kallsyms") as f: + for line in f: + parts = line.split() + if len(parts) >= 3: + addr = int(parts[0], 16) + name = parts[2] + mod = parts[3].strip("[]") if len(parts) > 3 else "" + syms.append((addr, name, mod)) + except (PermissionError, FileNotFoundError): + pass + syms.sort() + return syms + + +def runtime_text(syms): + """Runtime address of _text from kallsyms, 0 if hidden.""" + return next((a for a, n, m in syms if n == "_text" and not m), 0) + + +# Link-time address of _text per architecture, used only when neither +# System.map nor vmlinux is available: x86_64 __START_KERNEL +# (__START_KERNEL_map + CONFIG_PHYSICAL_START), arm64 KIMAGE_VADDR. +LINKTIME_TEXT_DEFAULT = { + "x86_64": 0xffffffff81000000, + "aarch64": 0xffff800080000000, +} + + +def linktime_text(vmlinux=None): + """Return (link-time address of _text, source description) or (0, "").""" + rel = os.uname().release + candidates = [] + if vmlinux: + candidates.append(os.path.join(os.path.dirname(vmlinux) or ".", "System.map")) + candidates += ["System.map", f"/boot/System.map-{rel}", + f"/usr/lib/debug/boot/System.map-{rel}"] + for sm in candidates: + try: + with open(sm) as f: + for line in f: + parts = line.split() + if len(parts) == 3 and parts[2] == "_text": + return int(parts[0], 16), sm + except (OSError, ValueError): + continue + if vmlinux and shutil.which("nm"): + try: + r = subprocess.run(["nm", "--defined-only", vmlinux], + capture_output=True, text=True, timeout=300) + for line in r.stdout.splitlines(): + parts = line.split() + if len(parts) == 3 and parts[2] == "_text": + return int(parts[0], 16), f"nm {vmlinux}" + except (OSError, subprocess.TimeoutExpired): + pass + link = LINKTIME_TEXT_DEFAULT.get(platform.machine(), 0) + return link, f"{platform.machine()} default" if link else "" + + +def kaslr_offset(syms, vmlinux=None): + """ + Runtime KASLR offset: recorded PCs have it removed (kcov's + canonicalize_ip()), /proc/kallsyms has it applied. Computed as the + runtime _text (kallsyms) minus the link-time _text (System.map, nm + vmlinux, or the architecture default). KASLR offsets are 2 MiB aligned + on x86_64 and arm64, which is used as a sanity check on the result. + """ + runtime = runtime_text(syms) + if not runtime: + print("# warning: _text not in /proc/kallsyms (kptr_restrict?); " + "PCs will not symbolize", file=sys.stderr) + return 0 + link, source = linktime_text(vmlinux) + if not link: + print(f"# warning: no System.map/vmlinux and no default _text for " + f"{platform.machine()}; pass --kaslr-offset", file=sys.stderr) + return 0 + off = runtime - link + if off % (2 << 20): + print(f"# warning: kaslr offset 0x{off:x} from {source} is not 2 MiB " + f"aligned; check CONFIG_PHYSICAL_START/KIMAGE_VADDR or pass " + f"--kaslr-offset", file=sys.stderr) + return off + + +# Rust symbol demangling via llvm-cxxfilt or rustfilt +_demangler = None + +def _init_demangler(): + global _demangler + for tool in ["llvm-cxxfilt", "rustfilt", "c++filt"]: + path = shutil.which(tool) + if path: + _demangler = path + return + _demangler = "" + +_demangled = {} + +def demangle(name): + """Demangle a Rust/C++ symbol name (memoized: one process per name).""" + global _demangler + if _demangler is None: + _init_demangler() + if not _demangler or not name.startswith("_R"): + return name + if name not in _demangled: + try: + r = subprocess.run([_demangler, name], capture_output=True, + text=True, timeout=2) + _demangled[name] = r.stdout.strip() if r.returncode == 0 else name + except (OSError, subprocess.TimeoutExpired): + _demangled[name] = name + return _demangled[name] + + +def find_vmlinux(vmlinux=None): + """Locate vmlinux for addr2line: explicit path, else the usual places.""" + if vmlinux: + return vmlinux + for p in ["vmlinux", "/boot/vmlinux", "/usr/lib/debug/boot/vmlinux"]: + if os.path.exists(p): + return p + return None + + +def _a2l_target(pc, vmlinux, ko_path, mod_text_base): + """(binary, address in it) to symbolize pc with, or None.""" + if ko_path and mod_text_base and pc >= mod_text_base: + return ko_path, pc - mod_text_base + if vmlinux: + return vmlinux, pc - KASLR_OFFSET # vmlinux holds link-time addresses + return None + + +def resolve_lines(pcs, vmlinux, cache, ko_path=None, mod_text_base=0): + """ + Resolve every pc in @pcs to file:line into @cache, one addr2line run + per binary: a DWARF5 vmlinux takes hundreds of ms to open, so one + process per record does not scale to thousands of records. + """ + todo = {} + for pc in pcs: + if pc in cache: + continue + cache[pc] = "" + tgt = _a2l_target(pc, vmlinux, ko_path, mod_text_base) + if tgt: + todo.setdefault(tgt[0], []).append((pc, tgt[1])) + for binary, pairs in todo.items(): + try: + r = subprocess.run( + ["addr2line", "-e", binary] + [f"0x{a:x}" for _, a in pairs], + capture_output=True, text=True, timeout=300) + except (subprocess.TimeoutExpired, FileNotFoundError): + continue + for (pc, _), loc in zip(pairs, r.stdout.splitlines()): + loc = loc.strip() + if loc and loc != "??:0" and loc != "??:?": + # Shorten path: keep only filename:line + cache[pc] = loc.rsplit("/", 1)[-1] + + +def resolve_line(pc, vmlinux, cache, ko_path=None, mod_text_base=0): + """Resolve one PC to source file:line using addr2line (cached).""" + if pc not in cache: + resolve_lines([pc], vmlinux, cache, ko_path, mod_text_base) + return cache[pc] + + +def get_kernel_meta(): + """Collect kernel build metadata.""" + meta = {"release": os.uname().release} + try: + with open("/proc/version") as f: + v = f.read().strip() + meta["version"] = v + # Extract compiler version + if "gcc" in v.lower(): + meta["compiler"] = v.split("(")[1].split(")")[0] if "(" in v else "" + elif "clang" in v.lower(): + idx = v.lower().find("clang") + meta["compiler"] = v[idx:idx+30].split(")")[0] + except OSError: + pass + return meta + + +def print_kernel_meta(meta, ko_path=None): + """Print kernel metadata header/footer.""" + print(f"# {'=' * 60}") + print(f"# Kernel: {meta.get('release', 'unknown')}") + print(f"# Build: {meta.get('version', 'unknown')[:80]}") + if meta.get('compiler'): + print(f"# Compiler: {meta['compiler']}") + # Read rustc version from .ko .comment section + if ko_path: + try: + r = subprocess.run( + ["readelf", "-p", ".comment", ko_path], + capture_output=True, text=True, timeout=5) + for line in r.stdout.splitlines(): + if "rustc" in line: + ver = line.split("]", 1)[-1].strip() + print(f"# Rustc: {ver}") + break + except (OSError, subprocess.TimeoutExpired): + pass + print(f"# {'=' * 60}") + + +def lookup(pc, syms): + """Nearest kallsyms entry <= pc as (name, offset, module) or None.""" + if not syms: + return None + lo, hi = 0, len(syms) - 1 + while lo < hi: + mid = (lo + hi + 1) // 2 + if syms[mid][0] <= pc: + lo = mid + else: + hi = mid - 1 + addr, name, mod = syms[lo] + if addr > pc: + return None + return name, pc - addr, mod + + +def symbolize(pc, syms): + """Find nearest symbol <= pc. Returns (display_name, module_tag).""" + hit = lookup(pc, syms) + if not hit: + return f"0x{pc:x}", "" + name, offset, mod = hit + dname = demangle(name) + display = f"{dname}+0x{offset:x}" if offset else dname + return display, f" [{mod}]" if mod else "" + + +def format_val(v): + """Format a captured value.""" + if v == MAGIC_BAD: + return "FAULT" + if v == 0: + return "0x0" + return f"0x{v:x}" + + +def find_module(name): + """ + Find the .ko for test @name: /.ko in the source tree, or + .ko next to this script in an installed (make install) tree. + """ + for ko_path in (os.path.join(SELFTEST_DIR, name, f"{name}.ko"), + os.path.join(SELFTEST_DIR, f"{name}.ko")): + if os.path.exists(ko_path): + return ko_path + return None + + +def finit_module(ko_path): + """Load a kernel module via finit_module syscall.""" + libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True) + fd = os.open(ko_path, os.O_RDONLY) + ret = libc.syscall(SYS_FINIT_MODULE, fd, b"", 0) + os.close(fd) + if ret != 0: + errno = ctypes.get_errno() + raise OSError(errno, f"finit_module({ko_path}): {os.strerror(errno)}") + + +def delete_module(name): + """Unload a kernel module.""" + libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True) + ret = libc.syscall(SYS_DELETE_MODULE, name.encode(), 0) + if ret != 0: + errno = ctypes.get_errno() + raise OSError(errno, f"delete_module({name}): {os.strerror(errno)}") + + +def trigger_module(): + """ + Write to every trigger file the loaded module created under TRIGGER_DIR. + Opened without O_CREAT: debugfs directories have no ->create, so a + "w"-mode open of a missing name fails with EOPNOTSUPP, not ENOENT. + """ + try: + names = sorted(os.listdir(TRIGGER_DIR)) + except OSError: + names = [] + hits = [] + for n in names: + path = os.path.join(TRIGGER_DIR, n) + try: + fd = os.open(path, os.O_WRONLY) + except OSError: + continue + try: + os.write(fd, b"1") + finally: + os.close(fd) + hits.append(path) + if not hits: + raise FileNotFoundError(f"no trigger file under {TRIGGER_DIR}") + return hits + + +def parse_records(buf, total_words): + """Parse the ring buffer into a list of records.""" + records = [] + pos = 1 + end = min(1 + total_words, BUF_SIZE) + while pos + RECORD_HDR_WORDS <= end: + hdr = buf[pos] + rtype = hdr_type(hdr) + num_vals = hdr_nvals(hdr) + + # Every record the kernel writes has nvals >= 1 and a known type; + # anything else is garbage (e.g. a userspace reset mid-run): resync. + if rtype not in (DF_TYPE_ENTRY, DF_TYPE_RET, DF_TYPE_CMP) \ + or num_vals == 0 or pos + RECORD_HDR_WORDS + num_vals > end: + pos += 1 + continue + + pc = int(buf[pos + 1]) + KASLR_OFFSET + ptr = int(buf[pos + 2]) # ENTRY/RET: traced pointer; CMP: cmp type + if rtype == DF_TYPE_CMP: + pos += RECORD_HDR_WORDS + num_vals + continue + + # Valid records always have a non-zero PC (kernel text address) + if pc == 0: + pos += 1 + continue + + vals = [int(buf[pos + RECORD_HDR_WORDS + vi]) for vi in range(num_vals)] + records.append({ + "type": rtype, + "seq": hdr_seq(hdr), + "pc": pc, + "ptr": ptr, + "arg_idx": hdr_arg_idx(hdr), + "size": hdr_size(hdr), + "val": vals[0], + "vals": vals, + }) + pos += RECORD_HDR_WORDS + num_vals + return records + + +class Capture: + """Everything run_capture() collected for one module run.""" + + def __init__(self, ko_path, mod_name, records, syms, total_words, + mod_text_start, kaslr_off): + self.ko_path = ko_path + self.mod_name = mod_name + self.records = records + self.syms = syms + self.total_words = total_words + self.mod_text_start = mod_text_start + self.kaslr_offset = kaslr_off + self.runtime_text = runtime_text(syms) + self._mod_syms = any(m == mod_name for _, _, m in syms) + # Aliases: rustc's merge-functions makes identical bodies (e.g. the + # one-field rsf_1 and rstf_1) share one address, so a PC can carry + # several names. + self._names = {} + for addr, name, mod in syms: + self._names.setdefault((addr, mod), set()).add(name) + + def is_module_pc(self, pc): + """True if pc lies in the test module (kallsyms, else .text start).""" + if self._mod_syms: + hit = lookup(pc, self.syms) + return bool(hit) and hit[2] == self.mod_name + # Fallback: if no module symbols (kptr_restrict), use .text start + return bool(self.mod_text_start) and pc >= self.mod_text_start + + def funcs(self, rec): + """All raw kallsyms names of the function a record belongs to.""" + hit = lookup(rec["pc"], self.syms) + if not hit: + return set() + name, offset, mod = hit + return self._names.get((rec["pc"] - offset, mod), {name}) + + def module_records(self): + return [r for r in self.records if self.is_module_pc(r["pc"])] + + def context_records(self, n): + """Module records plus n records before/after each of them.""" + keep = set() + for i, r in enumerate(self.records): + if self.is_module_pc(r["pc"]): + keep.update(range(max(0, i - n), + min(len(self.records), i + n + 1))) + return [self.records[i] for i in sorted(keep)] + + +def run_capture(ko_path, remote=False, vmlinux=None, kaslr_override=None, + log=None): + """ + Load @ko_path, record while its trigger file(s) are written, unload it + and return a Capture. @remote publishes the buffer for REMOTE_HANDLE + instead of enabling recording for this task. Raises OSError. + """ + global KASLR_OFFSET + log = log or (lambda msg: print(f"# {msg}")) + + # Ensure kallsyms shows real addresses + try: + with open("/proc/sys/kernel/kptr_restrict", "w") as f: + f.write("0") + except OSError: + pass + + df_fd = os.open(KCOV_DF_PATH, os.O_RDWR) + try: + # Init + mmap + fcntl.ioctl(df_fd, KCOV_DF_INIT_TRACK, BUF_SIZE) + libc = ctypes.CDLL(ctypes.util.find_library("c"), use_errno=True) + libc.mmap.restype = ctypes.c_void_p + libc.mmap.argtypes = [ + ctypes.c_void_p, ctypes.c_size_t, ctypes.c_int, + ctypes.c_int, ctypes.c_int, ctypes.c_long + ] + buf_ptr = libc.mmap(None, BUF_SIZE * 8, 0x3, 0x01, df_fd, 0) + if buf_ptr == ctypes.c_void_p(-1).value: + errno = ctypes.get_errno() + raise OSError(errno, f"mmap: {os.strerror(errno)}") + buf = (ctypes.c_uint64 * BUF_SIZE).from_address(buf_ptr) + + # Load module first (its init generates noise with INSTRUMENT_ALL) + mod_name = os.path.basename(ko_path).replace(".ko", "") + finit_module(ko_path) + log(f"Loaded {mod_name}") + try: + # Module .text address, the PC filter fallback without kallsyms + mod_text_start = 0 + try: + with open(f"/sys/module/{mod_name}/sections/.text") as f: + mod_text_start = int(f.read().strip(), 16) + except (OSError, ValueError): + pass + + # Enable recording AFTER load, BEFORE trigger (no loader noise). + # Remote: the handle is passed by pointer (a __u64 in a buffer), + # so the full 64-bit value survives 32-bit/compat callers. + if remote: + fcntl.ioctl(df_fd, KCOV_DF_REMOTE_ENABLE, + struct.pack("Q", REMOTE_HANDLE)) + else: + fcntl.ioctl(df_fd, KCOV_DF_ENABLE, 0) + buf[0] = 0 + try: + for path in trigger_module(): + log(f"Triggered {path}") + finally: + fcntl.ioctl(df_fd, KCOV_DF_REMOTE_DISABLE if remote + else KCOV_DF_DISABLE, 0) + + # Read kallsyms while the module is still loaded + syms = load_kallsyms() + finally: + try: + delete_module(mod_name) + except OSError as e: + log(f"warning: {e}") + + if kaslr_override is not None: + KASLR_OFFSET = kaslr_override + else: + KASLR_OFFSET = kaslr_offset(syms, find_vmlinux(vmlinux)) + + total = int(buf[0]) + records = parse_records(buf, total) + return Capture(ko_path, mod_name, records, syms, total, + mod_text_start, KASLR_OFFSET) + finally: + os.close(df_fd) + + +def print_raw(records, syms, vmlinux=None, cache=None, ko_path=None, mod_text_base=0): + """Print records in raw format with source line on left.""" + if cache is None: + cache = {} + # Pre-resolve all locations (one addr2line run) to find max width + resolve_lines([r["pc"] for r in records], vmlinux, cache, ko_path, + mod_text_base) + locs = [cache[r["pc"]] for r in records] + max_w = max((len(l) for l in locs if l), default=0) + max_w = max(max_w, 10) # minimum width + + for i, r in enumerate(records): + name, mod = symbolize(r["pc"], syms) + sym = f"{name}{mod}" + t = "ENTRY" if r["type"] == DF_TYPE_ENTRY else "RET " + arg_idx = r["arg_idx"] + size = r["size"] + left = f"{locs[i]:>{max_w}s}" if locs[i] else f"{'':>{max_w}s}" + vals = format_val(r["val"]) if len(r["vals"]) == 1 else \ + "{" + ", ".join(format_val(v) for v in r["vals"]) + "}" + print(f"{left} [{t}] seq={r['seq']:3d} {sym} " + f"arg[{arg_idx}]({size}) @0x{r['ptr']:x} = {vals}") + + +def print_tree(records, syms, vmlinux=None, cache=None, ko_path=None, mod_text_base=0): + """Print records as indented call tree with source line on left.""" + if cache is None: + cache = {} + # Pre-resolve all PCs (one addr2line run) for alignment + resolve_lines([r["pc"] for r in records], vmlinux, cache, ko_path, + mod_text_base) + max_w = max((len(v) for v in cache.values() if v), default=10) + max_w = max(max_w, 10) + + depth = 0 + call_stack = [] # Stack of (name, mod, args_str, pc) for matching returns + i = 0 + while i < len(records): + r = records[i] + name, mod = symbolize(r["pc"], syms) + + if r["type"] == DF_TYPE_ENTRY: + # Collect all args for this call (same PC, consecutive entries); + # order by index, as the pass emits dead-arg traces last. + args = [] + pc = r["pc"] + while i < len(records) and records[i]["type"] == DF_TYPE_ENTRY \ + and records[i]["pc"] == pc: + vals = records[i]["vals"] + if len(vals) > 1: + fields = ", ".join(format_val(v) for v in vals) + args.append((records[i]["arg_idx"], "{" + fields + "}")) + else: + args.append((records[i]["arg_idx"], + format_val(records[i]["val"]))) + i += 1 + args_str = ", ".join(a for _, a in sorted(args, key=lambda x: x[0])) + call_stack.append((name, mod, args_str, pc)) + depth += 1 + else: + # Pop void calls (no return record) until we find matching PC + while call_stack and call_stack[-1][3] != r["pc"]: + depth = max(0, depth - 1) + indent = " " * depth + vname, vmod, vargs, vpc = call_stack.pop() + loc = resolve_line(vpc, vmlinux, cache, ko_path, mod_text_base) + left = f"{loc:>{max_w}s}" if loc else f"{'':>{max_w}s}" + print(f"{left} {indent}{vname}({vargs}){vmod}") + depth = max(0, depth - 1) + indent = " " * depth + ret_size = r["size"] + loc = resolve_line(r["pc"], vmlinux, cache, ko_path, mod_text_base) + left = f"{loc:>{max_w}s}" if loc else f"{'':>{max_w}s}" + if call_stack: + cname, cmod, cargs, _ = call_stack.pop() + if ret_size == 0: + print(f"{left} {indent}{cname}({cargs}){cmod}") + else: + print(f"{left} {indent}{format_val(r['val'])} = {cname}({cargs}){cmod}") + else: + if ret_size == 0: + print(f"{left} {indent}{name}(){mod}") + else: + print(f"{left} {indent}{format_val(r['val'])} = {name}(){mod}") + i += 1 + + # Flush remaining void calls on the stack + while call_stack: + depth = max(0, depth - 1) + indent = " " * depth + vname, vmod, vargs, vpc = call_stack.pop() + loc = resolve_line(vpc, vmlinux, cache, ko_path, mod_text_base) + left = f"{loc:>{max_w}s}" if loc else f"{'':>{max_w}s}" + print(f"{left} {indent}{vname}({vargs}){vmod}") + + +def main(): + parser = argparse.ArgumentParser( + description="Load a test module with kcov_dataflow and view records") + parser.add_argument("module", help="Test module name (e.g. eight_struct_args_c)") + parser.add_argument("--raw", action="store_true", + help="Print raw records instead of tree") + parser.add_argument("--ko", help="Explicit path to .ko file") + parser.add_argument("--context", "-C", type=int, default=0, + help="Show N records before/after each module record") + parser.add_argument("--vmlinux", help="Path to vmlinux for addr2line") + parser.add_argument("--remote", action="store_true", + help="Use KCOV_DF_REMOTE_ENABLE for kworker capture") + parser.add_argument("--kaslr-offset", type=lambda x: int(x, 0), + help="Override the runtime KASLR offset added to PCs") + args = parser.parse_args() + + ko_path = args.ko or find_module(args.module) + if not ko_path or not os.path.exists(ko_path): + print(f"Cannot find module for '{args.module}'", file=sys.stderr) + print("Build it first: make -C tools/testing/selftests " + "TARGETS=kcov_dataflow LLVM=1 CC=clang", file=sys.stderr) + sys.exit(1) + + try: + cap = run_capture(ko_path, remote=args.remote, vmlinux=args.vmlinux, + kaslr_override=args.kaslr_offset) + except OSError as e: + print(f"{args.module}: {e}", file=sys.stderr) + sys.exit(1) + + print(f"# Captured {cap.total_words} words (kaslr_offset=0x{cap.kaslr_offset:x}, " + f"_text=0x{cap.runtime_text:x})") + print(f"# {len(cap.records)} records") + + if cap.syms or cap.mod_text_start: + if args.context > 0: + records = cap.context_records(args.context) + print(f"# showing {len(records)} records with context={args.context} " + f"around {cap.mod_name}\n") + else: + records = cap.module_records() + print(f"# {len(records)} from {cap.mod_name}\n") + else: + records = cap.records + print("") + + meta = get_kernel_meta() + print_kernel_meta(meta, ko_path=ko_path) + + vmlinux = find_vmlinux(args.vmlinux) + show = print_raw if args.raw else print_tree + show(records, cap.syms, vmlinux, {}, ko_path, cap.mod_text_start) + + print_kernel_meta(meta, ko_path=ko_path) + + +if __name__ == "__main__": + main() -- 2.47.3